cvpr cvpr2013 cvpr2013-465 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Manmohan Chandraker, Dikpal Reddy, Yizhou Wang, Ravi Ramamoorthi
Abstract: We present a theory that addresses the problem of determining shape from the (small or differential) motion of an object with unknown isotropic reflectance, under arbitrary unknown distant illumination, , for both orthographic and perpsective projection. Our theory imposes fundamental limits on the hardness of surface reconstruction, independent of the method involved. Under orthographic projection, we prove that three differential motions suffice to yield an invariant that relates shape to image derivatives, regardless of BRDF and illumination. Under perspective projection, we show that four differential motions suffice to yield depth and a linear constraint on the surface gradient, with unknown BRDF and lighting. Further, we delineate the topological classes up to which reconstruction may be achieved using the invariants. Finally, we derive a general stratification that relates hardness of shape recovery to scene complexity. Qualitatively, our invariants are homogeneous partial differential equations for simple lighting and inhomogeneous for complex illumination. Quantitatively, our framework shows that the minimal number of motions required to resolve shape is greater for more complex scenes. Prior works that assume brightness constancy, Lambertian BRDF or a known directional light source follow as special cases of our stratification. We illustrate with synthetic and real data how potential reconstruction methods may exploit our framework.
Reference: text
sentIndex sentText sentNum sentScore
1 edu Abstract We present a theory that addresses the problem of determining shape from the (small or differential) motion of an object with unknown isotropic reflectance, under arbitrary unknown distant illumination, , for both orthographic and perpsective projection. [sent-4, score-0.937]
2 Our theory imposes fundamental limits on the hardness of surface reconstruction, independent of the method involved. [sent-5, score-0.572]
3 Under orthographic projection, we prove that three differential motions suffice to yield an invariant that relates shape to image derivatives, regardless of BRDF and illumination. [sent-6, score-0.904]
4 Under perspective projection, we show that four differential motions suffice to yield depth and a linear constraint on the surface gradient, with unknown BRDF and lighting. [sent-7, score-1.261]
5 Finally, we derive a general stratification that relates hardness of shape recovery to scene complexity. [sent-9, score-0.478]
6 Qualitatively, our invariants are homogeneous partial differential equations for simple lighting and inhomogeneous for complex illumination. [sent-10, score-0.535]
7 Introduction An open problem in computer vision since early works on optical flow has been to determine the shape of an object with unknown reflectance undergoing differential motion, when observed by a static camera under unknown illumination. [sent-15, score-0.993]
8 This paper presents a theory to solve the problem for both orthographic and perspective camera projections, with arbitrary unknown distant lighting (directional or area). [sent-16, score-0.79]
9 Unlike traditional approaches to shape recovery from motion like optical flow [6, 10] or multiview stereo [15], our theory does not make physically incorrect assumptions like brightness constancy, or simplifying ones like Lambertian reflectance. [sent-17, score-0.625]
10 In Section 3, we correctly model the dependence of image formation on the bidirectional reflectance distribution function (BRDF) and illumination, to derive a physically valid differential flow relation. [sent-18, score-0.513]
11 Remarkably, it can be shown that even when the BRDF and illumination are unknown, the differential flow constrains the shape of an object through an invariant relating surface depth to image derivatives. [sent-19, score-0.925]
12 For orthographic projections, considered in Section 4, three differential motions suffice and the invariant is a quasilinear partial differential equation (PDE). [sent-21, score-1.142]
13 For perspective projections, we show in Section 5 that surface depth may be directly recovered from four differential motions, with an additional linear PDE constraining the surface normal. [sent-22, score-1.119]
14 Besides characterizing the invariants, in each case, we also study the precise extent to which surface reconstruction may be performed. [sent-24, score-0.408]
15 Notably, the brightness constancy relation popularly used in optical flow [6, 10] is a special case of our differential flow relation, as are more physically-based studies that relate the motion field to radiometric entities assuming diffuse reflectance [11, 12]. [sent-26, score-0.974]
16 Note that the limits imposed by our theory on the hardness of motion field estimation are fundamental ones they hold true regardless of the actual reconstruction method employed. [sent-30, score-0.505]
17 In summary, this paper makes the following contributions: • A theory that relates shape to object motion, for unknown isotropic BRDF and illumination (directional or area), under orthographic and perspective projections. [sent-31, score-0.839]
18 A stratification of the hardness of shape recovery from motion, under various imaging conditions (Table 1). [sent-33, score-0.392]
19 The qualitative hardness of shape from motion is indicated by the nature of reconstruction invariant and quantified by minimal number of required motions. [sent-36, score-0.452]
20 While attempts have been made to relax the assumption for certain reflectance models [5, 12], our work provides the first unified theoretical and computational paradigm that relates shape recovery to image derivatives from object motion, with unknown BRDF and illumination. [sent-41, score-0.507]
21 Similar to our setup, albeit limited to Lambertian reflectance and directional lighting, passive photometric stereo methods [9, 18] use object motion to reconstruct a dense depth map. [sent-44, score-0.412]
22 recover shape under nonLambertian reflectance using an isometric relationship between change in intensity profiles under light source motion and surface normal differences [14]. [sent-53, score-0.554]
23 The object BRDF is assumed isotropic and homogeneous (or having slow spatial variation), with an unknown functional form. [sent-62, score-0.299]
24 μ where − − − , (4) Differential flow relation Assuming isotropic BRDF ρ, the image intensity of a 3D point x, imaged at pixel u, is I(u, t) = σ(x)ρ(n, x) , (5) where σ is the albedo and n is the surface normal at the point. [sent-90, score-0.604]
25 The BRDF ρ is usually written as a function of incident and outgoing directions, but for fixed lighting and view, can be seen as a function of surface position and orientation. [sent-92, score-0.375]
26 Denoting E = log I, we n(ot∇e that t(hωe ×alnb)ed =o can b∇e easily eliminated by dividing out I(u, t), to yield the differential flow relation: (∇uE)? [sent-110, score-0.426]
27 (8) The differential flow relation in (7) and (8) is a strict generalization of the brightness constancy relation used by the vast majority of prior works on optical flow [6, 10]. [sent-113, score-0.808]
28 (9) In the following, we explore the extent to which the motion field μ and object shape may be recovered using (8), under both orthographic and perspective image formation. [sent-120, score-0.554]
29 Precisely, we show that it is possible to eliminate all BRDF and lighting effects in an image sequence, leaving a simple relationship between image derivatives, surface depths and normals. [sent-121, score-0.455]
30 Orthographic Projection In this section, we consider recovery of the shape of an object with unknown BRDF, using a sequence of differential motions. [sent-123, score-0.619]
31 (13) Note thatpi, qi and ωi are known from the images and calibration, while surface depth z and the entity π related to normals and BRDF are unknown. [sent-133, score-0.365]
32 Under orthographic projection, surface depth under unknown BRDF may not be unambiguously recovered using solely motion as the cue. [sent-138, score-0.88]
33 BRDF-Invariant Constraints on Surface While one may not use (10) directly to obtain depth, we may still exploit the rank deficiency to infer information about the surface depth, as stated by the following: Proposition 2. [sent-141, score-0.363]
34 For an object with unknown BRDF, observed under unknown lighting and orthographic camera, three differential motions suffice to yield a BRDF and lighting invariant relation between image derivatives and surface geometry. [sent-142, score-1.796]
35 Thus, we may directly relate surface depth and gradient to image intensity, even for unknown BRDF and illumination. [sent-174, score-0.587]
36 Surface Depth Estimation Next, we consider the precise extent to which surface depth may be recovered using Proposition 2. [sent-178, score-0.468]
37 Two or more differential motions of a surface with unknown BRDF, with a colocated source and sensor, yield level curves of surface depth, corresponding to known depths of some (possibly isolated) points on the surface. [sent-187, score-1.555]
38 Thus, (19) allows reconstruction of level curves of the surface, with unknown BRDF, under colocated illumination. [sent-205, score-0.479]
39 Note that (19) is a first-order, homogeneous, quasilinear partial differential equation (PDE). [sent-206, score-0.399]
40 Three or more differential motions of a surface with unknown BRDF, under unknown illumination, yield characteristic surface curves C(x(s) , y(s) , z(s)), defined by λ1+1 λ2zddsx = λ3+1 λ4zddsy = −γ41ddzs (25) corresponding to depths at some (possibly isolated) points. [sent-209, score-1.642]
41 Given orthographic images (left) under five differential motions of a surface with non-Lambertian BRDF under colocated illumination, level curves of the surface are reconstructed using (24) (center). [sent-213, score-1.408]
42 The surface may be reconstructed by interpolating between the level curves (right). [sent-215, score-0.401]
43 Given orthographic images (left) under five differential motions of a surface with non-Lambertian BRDF under unknown lighting, characteristic curves of (25) are reconstructed (center). [sent-218, score-1.253]
44 The surface is reconstructed by interpolating between the characteristic curves (right). [sent-220, score-0.424]
45 Note that dz is zero for the colocated case since characteristic curves correspond to level curves of depth, while it is in general non-zero for the non-colocated case. [sent-230, score-0.338]
46 Figure 1illustrates the characteristic curves recovered for a synthetic sphere and vase, rendered under colocated illumination. [sent-232, score-0.308]
47 Orthographic images are recorded for five differential motions, with arbitrary rotations of approximately 0. [sent-233, score-0.337]
48 After computing depths along several characteristic curves using the BRDF-invariant relation (23), we interpolate depths between the curves, to recover the entire surface geometry. [sent-236, score-0.664]
49 Surprisingly, we obtain even stronger results in the perspective case, showing that with four or more differential motions with unknown BRDF, we can directly recover surface depth, as well as a linear constraint on the derivatives of the depth. [sent-240, score-1.135]
50 Differential Flow Relation In the perspective case, one may rewrite (8) as (compare to the linear relation in (10) for the orthographic case), p? [sent-245, score-0.439]
51 Now, one may derive a theory similar to the orthographic case by treating z/(1 + βz), 1/(1 + βz) and π as independent variables and using the rank deficiency (note the form ofp? [sent-257, score-0.358]
52 ) arising from a sequence of m ≥ 4 differential motions. [sent-258, score-0.297]
53 × Instead, in the following, we take a closer look at the perspective equations for differential flow, to show that they yield a more comprehensive solution for surface geometry. [sent-260, score-0.723]
54 BRDF-Invariant Depth Estimation We demonstrate that under perspective projection, object motion can completely specify the surface depth, without any initial information: Proposition 5. [sent-263, score-0.469]
55 Four or more differential motions of a surface with unknown BRDF, under unknown illumination, suffice to yield under perspective projection: (i) the surface depth (ii) a linear constraint on the derivatives of surface depth. [sent-264, score-2.041]
56 (32) + + Thus, in the perspective case, one may directly use (3 1) to recover the surface depth. [sent-321, score-0.439]
57 4 = 0, (33) which is a linear constraint on surface depth derivatives. [sent-325, score-0.365]
58 Three or more differential motions of a surface with unknown BRDF, under unknown illumination, suffice to yield under perspective projection the surface depth and the slope of the gradient. [sent-331, score-1.77]
59 Thus, we have shown that in the perspective case, even when BRDF and illumination are unknown, one may derive an invariant that relates shape to object motion, through a linear relation and a linear PDE on the surface depth. [sent-332, score-0.764]
60 1 Direct Depth Recovery As established by Proposition 5, under perspective projection, one may directly recover the surface depth using (3 1). [sent-338, score-0.539]
61 An object with unknown BRDF is imaged with perspective projection under unknown illumination after undergoing four arbitrary differential motions (approximately 0. [sent-340, score-1.218]
62 Note that no prior knowledge of the surface is required in the perspective case, even at isolated points. [sent-342, score-0.406]
63 (a) One of five images (four motions) under perspective projection, with arbitrary non-Lambertian BRDF and unknown lighting. [sent-353, score-0.343]
64 (a) One of five images (four motions), with arbitrary nonLambertian BRDF and unknown lighting, under perspective projection. [sent-356, score-0.343]
65 The object is imaged under perspective projection after undergoing five random differential motions (approximately 0. [sent-361, score-0.717]
66 Stratification of Shape from Motion Our theory not only shows the possibility of shape recovery under unknown BRDFs and lighting, but also derives the minimum computational and imaging budget required and the precise extent to which surface shape may be recovered. [sent-364, score-0.809]
67 In this section, we discuss how this work establishes a theoretical notion that relates the hardness of surface reconstruction to the scene complexity, which supports our intuitive understanding of an “easy” reconstruction problem versus a “hard” one. [sent-365, score-0.639]
68 Generalization of [1] For the Lambertian BRDF, under known directional lighting, Basri and Frolova [1] show that shape and image derivatives may be related by a quasilinear PDE. [sent-366, score-0.331]
69 In particular, the framework of this paper can also handle general BRDFs, unknown directional or area lighting and various camera projections. [sent-392, score-0.409]
70 Shape may be recovered under Lambertian BRDF and known lighting with one motion, while an unknown BRDF with colocated lighting requires two motions under orthography and three under perspective projection. [sent-394, score-0.954]
71 An unknown BRDF, unknown illumination and perspective projection may be considered an even “harder” reconstruction problem, for which the minimal imaging requirement is four motions. [sent-395, score-0.749]
72 Similarly, simpler illuminations like colocated lighting result in a homogeneous PDE, while more complex illuminations make the PDE inhomogeneous, whose solution is arguably harder. [sent-396, score-0.343]
73 Thus, the nature of the reconstruction invariant is a qualitative indicator of the hardness of shape from motion. [sent-397, score-0.331]
74 For a given scene complexity, the hardness of reconstruction is quantified by the minimum number of motions specified by our theory for reconstruction to be possible. [sent-398, score-0.636]
75 The camera is calibrated and a hand-eye calibration is performed to transform the gyroscope motions into the camera coordinate system. [sent-408, score-0.304]
76 222555222866 motions, of a real object with unknown BRDF, acquired under unknown illumination. [sent-414, score-0.376]
77 (Right) Views of surface reconstructed using the theory of Section 5. [sent-416, score-0.396]
78 In Figure 5, we show the reconstruction for a porcelain bas-relief sculpture with fine depth variations, using six differential motions. [sent-424, score-0.48]
79 The reconstructed surface using the solution in (3 1) is shown on the right. [sent-426, score-0.294]
80 This demonstrates the practical utility of our theory which does not assume brightness constancy or simplified forms of the BRDF and illumination, rather it correctly accounts for shading changes through an invariant that eliminates the BRDF and lighting. [sent-428, score-0.311]
81 Again, note that the unknown BRDF is clearly nonLambertian and the lighting is unknown, yet our theory allows surface reconstruction with fine details. [sent-430, score-0.748]
82 Discussion and Future Work This paper answers the question of what motion reveals about shape, with unknown isotropic BRDF and arbitrary, unknown distant illumination, for orthographic and perspective projections. [sent-432, score-0.85]
83 We derive differential flow invariants that relate image derivatives to shape and exactly characterize the object geometry that can be recovered. [sent-433, score-0.575]
84 This work generalizes traditional notions of brightness constancy or Lambertian BRDFs in the optical flow and multiview stereo literatures. [sent-434, score-0.335]
85 (Left)Oneofel veni putimages,relatedbytendifer n- tial motions, of a real object with unknown BRDF, acquired under unknown illumination. [sent-436, score-0.376]
86 (Center) Top view of the surface reconstructed using the theory of Section 5. [sent-438, score-0.396]
87 Correctly accounting for intensity variations, in spite of unknown BRDF and illumination, allows us to recover surface details such as the fine striations between the lobes of the shell. [sent-440, score-0.478]
88 Our results are not just valid for a particular approach to reconstruction, rather they impose fundamental limits on the hardness of surface reconstruction. [sent-441, score-0.47]
89 In the process, we also present a stratification of shape from motion that relates hardness of reconstruction to scene complexity qualitatively in terms of the nature of the involved PDE and quantitatively in terms of the minimum number of required motions. [sent-442, score-0.545]
90 This work and previous results have shown that differential motions carry rich information, independent of the lighting and BRDF. [sent-447, score-0.59]
91 This paper developed a general theory to understand the information carried in the motion cue, while prior works like [3] have taken a first step towards considering differential photometric stereo for the lighting cue. [sent-448, score-0.68]
92 An interesting future direction would be to consider small viewpoint changes, as well as a unified framework that combines all the differential cues. [sent-449, score-0.297]
93 Following optical flow studies [11, 17], we make a distinction between entities directly expressed in terms of intrinsic surface coordinates (such as albedo) and those expressed in 3D coordinates (such as camera direction). [sent-459, score-0.474]
94 The 3D position vector of a point on the surface at time t is x(a, b, t), while the corresponding surface normal is n(a, b, t). [sent-460, score-0.554]
95 (37) The isotropic BRDF on the surface is a function of normal and position, denoted by ρ(n, x). [sent-466, score-0.362]
96 δt (39) where the surface entities correspond to (a? [sent-496, score-0.309]
97 2 establishesi t=hat 1 t,·he· integral surface of (42), S : z = u(x, y), is indeed the surface under consideration and the Scoe :f zfi =cien ut( xfu,yn)ctions can be obtained from three or more differential motions of the surface. [sent-522, score-1.01]
98 This completes the proof that the characteristic curves C, given by (25), reside on the surface S. [sent-544, score-0.395]
99 A theory of differential photometric stereo for unknown isotropic BRDFs. [sent-565, score-0.742]
100 Dense shape reconstruction of a moving object under arbitrary, unknown lighting. [sent-644, score-0.331]
wordName wordTfidf (topN-words)
[('brdf', 0.589), ('differential', 0.297), ('surface', 0.265), ('unknown', 0.188), ('motions', 0.183), ('pde', 0.162), ('orthographic', 0.161), ('hardness', 0.153), ('colocated', 0.135), ('perspective', 0.115), ('lighting', 0.11), ('stratification', 0.105), ('theory', 0.102), ('quasilinear', 0.102), ('depth', 0.1), ('lambertian', 0.094), ('proposition', 0.09), ('motion', 0.089), ('illumination', 0.085), ('relation', 0.084), ('flow', 0.083), ('reconstruction', 0.083), ('brdfs', 0.081), ('depths', 0.08), ('recovery', 0.074), ('curves', 0.073), ('isotropic', 0.073), ('directional', 0.073), ('constancy', 0.069), ('reflectance', 0.068), ('suffice', 0.067), ('brightness', 0.064), ('derivatives', 0.062), ('shape', 0.06), ('characteristic', 0.057), ('projection', 0.056), ('relates', 0.055), ('ev', 0.052), ('frolova', 0.051), ('inhomogeneous', 0.048), ('stereo', 0.047), ('ramamoorthi', 0.046), ('yield', 0.046), ('rewrite', 0.045), ('gyroscope', 0.045), ('optical', 0.044), ('entities', 0.044), ('eu', 0.044), ('chandraker', 0.044), ('recovered', 0.043), ('zx', 0.042), ('invariants', 0.042), ('nonlambertian', 0.042), ('shading', 0.041), ('arbitrary', 0.04), ('zy', 0.039), ('imaged', 0.039), ('camera', 0.038), ('homogeneous', 0.038), ('distant', 0.036), ('orthography', 0.036), ('albedo', 0.036), ('invariant', 0.035), ('photometric', 0.035), ('may', 0.034), ('setup', 0.034), ('reconstructedsurface', 0.034), ('simakov', 0.034), ('physically', 0.034), ('relations', 0.033), ('quantified', 0.032), ('derive', 0.031), ('dikpal', 0.03), ('deficiency', 0.03), ('canas', 0.03), ('illuminations', 0.03), ('reconstructed', 0.029), ('zickler', 0.029), ('helmholtz', 0.028), ('inputimage', 0.028), ('ravi', 0.028), ('multiview', 0.028), ('basri', 0.027), ('undergoing', 0.027), ('fundamental', 0.027), ('span', 0.026), ('pseudoinverse', 0.026), ('extent', 0.026), ('isolated', 0.026), ('field', 0.026), ('limits', 0.025), ('recover', 0.025), ('vanishes', 0.024), ('reciprocity', 0.024), ('nec', 0.024), ('pz', 0.024), ('normal', 0.024), ('specular', 0.023), ('source', 0.023), ('radiometric', 0.023)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000001 465 cvpr-2013-What Object Motion Reveals about Shape with Unknown BRDF and Lighting
Author: Manmohan Chandraker, Dikpal Reddy, Yizhou Wang, Ravi Ramamoorthi
Abstract: We present a theory that addresses the problem of determining shape from the (small or differential) motion of an object with unknown isotropic reflectance, under arbitrary unknown distant illumination, , for both orthographic and perpsective projection. Our theory imposes fundamental limits on the hardness of surface reconstruction, independent of the method involved. Under orthographic projection, we prove that three differential motions suffice to yield an invariant that relates shape to image derivatives, regardless of BRDF and illumination. Under perspective projection, we show that four differential motions suffice to yield depth and a linear constraint on the surface gradient, with unknown BRDF and lighting. Further, we delineate the topological classes up to which reconstruction may be achieved using the invariants. Finally, we derive a general stratification that relates hardness of shape recovery to scene complexity. Qualitatively, our invariants are homogeneous partial differential equations for simple lighting and inhomogeneous for complex illumination. Quantitatively, our framework shows that the minimal number of motions required to resolve shape is greater for more complex scenes. Prior works that assume brightness constancy, Lambertian BRDF or a known directional light source follow as special cases of our stratification. We illustrate with synthetic and real data how potential reconstruction methods may exploit our framework.
2 0.47343358 75 cvpr-2013-Calibrating Photometric Stereo by Holistic Reflectance Symmetry Analysis
Author: Zhe Wu, Ping Tan
Abstract: Under unknown directional lighting, the uncalibrated Lambertian photometric stereo algorithm recovers the shape of a smooth surface up to the generalized bas-relief (GBR) ambiguity. We resolve this ambiguity from the halfvector symmetry, which is observed in many isotropic materials. Under this symmetry, a 2D BRDF slice with low-rank structure can be obtained from an image, if the surface normals and light directions are correctly recovered. In general, this structure is destroyed by the GBR ambiguity. As a result, we can resolve the ambiguity by restoring this structure. We develop a simple algorithm of auto-calibration from separable homogeneous specular reflection of real images. Compared with previous methods, this method takes a holistic approach to exploiting reflectance symmetry and produces superior results.
3 0.36370289 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
Author: Zhenglong Zhou, Zhe Wu, Ping Tan
Abstract: We present a method to capture both 3D shape and spatially varying reflectance with a multi-view photometric stereo technique that works for general isotropic materials. Our data capture setup is simple, which consists of only a digital camera and a handheld light source. From a single viewpoint, we use a set of photometric stereo images to identify surface points with the same distance to the camera. We collect this information from multiple viewpoints and combine it with structure-from-motion to obtain a precise reconstruction of the complete 3D shape. The spatially varying isotropic bidirectional reflectance distributionfunction (BRDF) is captured by simultaneously inferring a set of basis BRDFs and their mixing weights at each surface point. According to our experiments, the captured shapes are accurate to 0.3 millimeters. The captured reflectance has relative root-mean-square error (RMSE) of 9%.
4 0.34332025 54 cvpr-2013-BRDF Slices: Accurate Adaptive Anisotropic Appearance Acquisition
Author: Jirí Filip, Radomír Vávra, Michal Haindl, Pavel Žid, Mikuláš Krupika, Vlastimil Havran
Abstract: In this paper we introduce unique publicly available dense anisotropic BRDF data measurements. We use this dense data as a reference for performance evaluation of the proposed BRDF sparse angular sampling and interpolation approach. The method is based on sampling of BRDF subspaces at fixed elevations by means of several adaptively-represented, uniformly distributed, perpendicular slices. Although this proposed method requires only a sparse sampling of material, the interpolation provides a very accurate reconstruction, visually and computationally comparable to densely measured reference. Due to the simple slices measurement and method’s robustness it allows for a highly accurate acquisition of BRDFs. This in comparison with standard uniform angular sampling, is considerably faster yet uses far less samples.
5 0.2265723 443 cvpr-2013-Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
Author: Feng Lu, Yasuyuki Matsushita, Imari Sato, Takahiro Okabe, Yoichi Sato
Abstract: We propose an uncalibrated photometric stereo method that works with general and unknown isotropic reflectances. Our method uses a pixel intensity profile, which is a sequence of radiance intensities recorded at a pixel across multi-illuminance images. We show that for general isotropic materials, the geodesic distance between intensity profiles is linearly related to the angular difference of their surface normals, and that the intensity distribution of an intensity profile conveys information about the reflectance properties, when the intensity profile is obtained under uniformly distributed directional lightings. Based on these observations, we show that surface normals can be estimated up to a convex/concave ambiguity. A solution method based on matrix decomposition with missing data is developed for a reliable estimation. Quantitative and qualitative evaluations of our method are performed using both synthetic and real-world scenes.
6 0.22531359 423 cvpr-2013-Template-Based Isometric Deformable 3D Reconstruction with Sampling-Based Focal Length Self-Calibration
8 0.19618632 21 cvpr-2013-A New Perspective on Uncalibrated Photometric Stereo
10 0.15996383 286 cvpr-2013-Mirror Surface Reconstruction from a Single Image
11 0.14779282 244 cvpr-2013-Large Displacement Optical Flow from Nearest Neighbor Fields
12 0.1397081 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
13 0.13321914 226 cvpr-2013-Intrinsic Characterization of Dynamic Surfaces
14 0.12350707 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
15 0.12263364 111 cvpr-2013-Dense Reconstruction Using 3D Object Shape Priors
16 0.12057285 298 cvpr-2013-Multi-scale Curve Detection on Surfaces
17 0.11233475 71 cvpr-2013-Boundary Cues for 3D Object Shape Recovery
18 0.10715658 27 cvpr-2013-A Theory of Refractive Photo-Light-Path Triangulation
19 0.10381584 124 cvpr-2013-Determining Motion Directly from Normal Flows Upon the Use of a Spherical Eye Platform
20 0.099062361 289 cvpr-2013-Monocular Template-Based 3D Reconstruction of Extensible Surfaces with Local Linear Elasticity
topicId topicWeight
[(0, 0.178), (1, 0.301), (2, 0.0), (3, 0.087), (4, -0.06), (5, -0.177), (6, -0.154), (7, 0.034), (8, 0.063), (9, 0.009), (10, -0.083), (11, -0.114), (12, -0.107), (13, -0.116), (14, 0.148), (15, 0.121), (16, 0.253), (17, -0.163), (18, -0.03), (19, -0.095), (20, 0.003), (21, -0.099), (22, 0.004), (23, 0.077), (24, -0.01), (25, -0.08), (26, -0.073), (27, 0.047), (28, -0.003), (29, 0.024), (30, -0.193), (31, 0.036), (32, -0.005), (33, -0.05), (34, -0.019), (35, -0.029), (36, 0.073), (37, -0.038), (38, 0.044), (39, -0.024), (40, 0.109), (41, 0.006), (42, 0.03), (43, 0.038), (44, -0.058), (45, -0.011), (46, -0.03), (47, -0.029), (48, 0.077), (49, 0.083)]
simIndex simValue paperId paperTitle
1 0.94162416 75 cvpr-2013-Calibrating Photometric Stereo by Holistic Reflectance Symmetry Analysis
Author: Zhe Wu, Ping Tan
Abstract: Under unknown directional lighting, the uncalibrated Lambertian photometric stereo algorithm recovers the shape of a smooth surface up to the generalized bas-relief (GBR) ambiguity. We resolve this ambiguity from the halfvector symmetry, which is observed in many isotropic materials. Under this symmetry, a 2D BRDF slice with low-rank structure can be obtained from an image, if the surface normals and light directions are correctly recovered. In general, this structure is destroyed by the GBR ambiguity. As a result, we can resolve the ambiguity by restoring this structure. We develop a simple algorithm of auto-calibration from separable homogeneous specular reflection of real images. Compared with previous methods, this method takes a holistic approach to exploiting reflectance symmetry and produces superior results.
same-paper 2 0.93022025 465 cvpr-2013-What Object Motion Reveals about Shape with Unknown BRDF and Lighting
Author: Manmohan Chandraker, Dikpal Reddy, Yizhou Wang, Ravi Ramamoorthi
Abstract: We present a theory that addresses the problem of determining shape from the (small or differential) motion of an object with unknown isotropic reflectance, under arbitrary unknown distant illumination, , for both orthographic and perpsective projection. Our theory imposes fundamental limits on the hardness of surface reconstruction, independent of the method involved. Under orthographic projection, we prove that three differential motions suffice to yield an invariant that relates shape to image derivatives, regardless of BRDF and illumination. Under perspective projection, we show that four differential motions suffice to yield depth and a linear constraint on the surface gradient, with unknown BRDF and lighting. Further, we delineate the topological classes up to which reconstruction may be achieved using the invariants. Finally, we derive a general stratification that relates hardness of shape recovery to scene complexity. Qualitatively, our invariants are homogeneous partial differential equations for simple lighting and inhomogeneous for complex illumination. Quantitatively, our framework shows that the minimal number of motions required to resolve shape is greater for more complex scenes. Prior works that assume brightness constancy, Lambertian BRDF or a known directional light source follow as special cases of our stratification. We illustrate with synthetic and real data how potential reconstruction methods may exploit our framework.
3 0.8321861 443 cvpr-2013-Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
Author: Feng Lu, Yasuyuki Matsushita, Imari Sato, Takahiro Okabe, Yoichi Sato
Abstract: We propose an uncalibrated photometric stereo method that works with general and unknown isotropic reflectances. Our method uses a pixel intensity profile, which is a sequence of radiance intensities recorded at a pixel across multi-illuminance images. We show that for general isotropic materials, the geodesic distance between intensity profiles is linearly related to the angular difference of their surface normals, and that the intensity distribution of an intensity profile conveys information about the reflectance properties, when the intensity profile is obtained under uniformly distributed directional lightings. Based on these observations, we show that surface normals can be estimated up to a convex/concave ambiguity. A solution method based on matrix decomposition with missing data is developed for a reliable estimation. Quantitative and qualitative evaluations of our method are performed using both synthetic and real-world scenes.
4 0.82725459 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
Author: Zhenglong Zhou, Zhe Wu, Ping Tan
Abstract: We present a method to capture both 3D shape and spatially varying reflectance with a multi-view photometric stereo technique that works for general isotropic materials. Our data capture setup is simple, which consists of only a digital camera and a handheld light source. From a single viewpoint, we use a set of photometric stereo images to identify surface points with the same distance to the camera. We collect this information from multiple viewpoints and combine it with structure-from-motion to obtain a precise reconstruction of the complete 3D shape. The spatially varying isotropic bidirectional reflectance distributionfunction (BRDF) is captured by simultaneously inferring a set of basis BRDFs and their mixing weights at each surface point. According to our experiments, the captured shapes are accurate to 0.3 millimeters. The captured reflectance has relative root-mean-square error (RMSE) of 9%.
5 0.77725446 54 cvpr-2013-BRDF Slices: Accurate Adaptive Anisotropic Appearance Acquisition
Author: Jirí Filip, Radomír Vávra, Michal Haindl, Pavel Žid, Mikuláš Krupika, Vlastimil Havran
Abstract: In this paper we introduce unique publicly available dense anisotropic BRDF data measurements. We use this dense data as a reference for performance evaluation of the proposed BRDF sparse angular sampling and interpolation approach. The method is based on sampling of BRDF subspaces at fixed elevations by means of several adaptively-represented, uniformly distributed, perpendicular slices. Although this proposed method requires only a sparse sampling of material, the interpolation provides a very accurate reconstruction, visually and computationally comparable to densely measured reference. Due to the simple slices measurement and method’s robustness it allows for a highly accurate acquisition of BRDFs. This in comparison with standard uniform angular sampling, is considerably faster yet uses far less samples.
7 0.70263374 21 cvpr-2013-A New Perspective on Uncalibrated Photometric Stereo
8 0.59488517 409 cvpr-2013-Spectral Modeling and Relighting of Reflective-Fluorescent Scenes
9 0.56691557 435 cvpr-2013-Towards Contactless, Low-Cost and Accurate 3D Fingerprint Identification
11 0.52231151 423 cvpr-2013-Template-Based Isometric Deformable 3D Reconstruction with Sampling-Based Focal Length Self-Calibration
12 0.50534159 226 cvpr-2013-Intrinsic Characterization of Dynamic Surfaces
13 0.50488067 289 cvpr-2013-Monocular Template-Based 3D Reconstruction of Extensible Surfaces with Local Linear Elasticity
14 0.44961673 286 cvpr-2013-Mirror Surface Reconstruction from a Single Image
15 0.43890724 298 cvpr-2013-Multi-scale Curve Detection on Surfaces
16 0.43454981 391 cvpr-2013-Sensing and Recognizing Surface Textures Using a GelSight Sensor
17 0.40554333 330 cvpr-2013-Photometric Ambient Occlusion
18 0.40244645 354 cvpr-2013-Relative Volume Constraints for Single View 3D Reconstruction
19 0.39705047 27 cvpr-2013-A Theory of Refractive Photo-Light-Path Triangulation
20 0.39240783 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
topicId topicWeight
[(10, 0.092), (16, 0.036), (26, 0.098), (28, 0.015), (33, 0.272), (66, 0.078), (67, 0.034), (69, 0.038), (83, 0.131), (87, 0.11)]
simIndex simValue paperId paperTitle
1 0.92222029 400 cvpr-2013-Single Image Calibration of Multi-axial Imaging Systems
Author: Amit Agrawal, Srikumar Ramalingam
Abstract: Imaging systems consisting of a camera looking at multiple spherical mirrors (reflection) or multiple refractive spheres (refraction) have been used for wide-angle imaging applications. We describe such setups as multi-axial imaging systems, since a single sphere results in an axial system. Assuming an internally calibrated camera, calibration of such multi-axial systems involves estimating the sphere radii and locations in the camera coordinate system. However, previous calibration approaches require manual intervention or constrained setups. We present a fully automatic approach using a single photo of a 2D calibration grid. The pose of the calibration grid is assumed to be unknown and is also recovered. Our approach can handle unconstrained setups, where the mirrors/refractive balls can be arranged in any fashion, not necessarily on a grid. The axial nature of rays allows us to compute the axis of each sphere separately. We then show that by choosing rays from two or more spheres, the unknown pose of the calibration grid can be obtained linearly and independently of sphere radii and locations. Knowing the pose, we derive analytical solutions for obtaining the sphere radius and location. This leads to an interesting result that 6-DOF pose estimation of a multi-axial camera can be done without the knowledge of full calibration. Simulations and real experiments demonstrate the applicability of our algorithm.
same-paper 2 0.91868693 465 cvpr-2013-What Object Motion Reveals about Shape with Unknown BRDF and Lighting
Author: Manmohan Chandraker, Dikpal Reddy, Yizhou Wang, Ravi Ramamoorthi
Abstract: We present a theory that addresses the problem of determining shape from the (small or differential) motion of an object with unknown isotropic reflectance, under arbitrary unknown distant illumination, , for both orthographic and perpsective projection. Our theory imposes fundamental limits on the hardness of surface reconstruction, independent of the method involved. Under orthographic projection, we prove that three differential motions suffice to yield an invariant that relates shape to image derivatives, regardless of BRDF and illumination. Under perspective projection, we show that four differential motions suffice to yield depth and a linear constraint on the surface gradient, with unknown BRDF and lighting. Further, we delineate the topological classes up to which reconstruction may be achieved using the invariants. Finally, we derive a general stratification that relates hardness of shape recovery to scene complexity. Qualitatively, our invariants are homogeneous partial differential equations for simple lighting and inhomogeneous for complex illumination. Quantitatively, our framework shows that the minimal number of motions required to resolve shape is greater for more complex scenes. Prior works that assume brightness constancy, Lambertian BRDF or a known directional light source follow as special cases of our stratification. We illustrate with synthetic and real data how potential reconstruction methods may exploit our framework.
3 0.90287673 432 cvpr-2013-Three-Dimensional Bilateral Symmetry Plane Estimation in the Phase Domain
Author: Ramakrishna Kakarala, Prabhu Kaliamoorthi, Vittal Premachandran
Abstract: We show that bilateral symmetry plane estimation for three-dimensional (3-D) shapes may be carried out accurately, and efficiently, in the spherical harmonic domain. Our methods are valuable for applications where spherical harmonic expansion is already employed, such as 3-D shape registration, morphometry, and retrieval. We show that the presence of bilateral symmetry in the 3-D shape is equivalent to a linear phase structure in the corresponding spherical harmonic coefficients, and provide algorithms for estimating the orientation of the symmetry plane. The benefit of using spherical harmonic phase is that symmetry estimation reduces to matching a compact set of descriptors, without the need to solve a correspondence problem. Our methods work on point clouds as well as large-scale mesh models of 3-D shapes.
4 0.901205 75 cvpr-2013-Calibrating Photometric Stereo by Holistic Reflectance Symmetry Analysis
Author: Zhe Wu, Ping Tan
Abstract: Under unknown directional lighting, the uncalibrated Lambertian photometric stereo algorithm recovers the shape of a smooth surface up to the generalized bas-relief (GBR) ambiguity. We resolve this ambiguity from the halfvector symmetry, which is observed in many isotropic materials. Under this symmetry, a 2D BRDF slice with low-rank structure can be obtained from an image, if the surface normals and light directions are correctly recovered. In general, this structure is destroyed by the GBR ambiguity. As a result, we can resolve the ambiguity by restoring this structure. We develop a simple algorithm of auto-calibration from separable homogeneous specular reflection of real images. Compared with previous methods, this method takes a holistic approach to exploiting reflectance symmetry and produces superior results.
5 0.89871842 127 cvpr-2013-Discovering the Structure of a Planar Mirror System from Multiple Observations of a Single Point
Author: Ilya Reshetouski, Alkhazur Manakov, Ayush Bandhari, Ramesh Raskar, Hans-Peter Seidel, Ivo Ihrke
Abstract: We investigate the problem of identifying the position of a viewer inside a room of planar mirrors with unknown geometry in conjunction with the room’s shape parameters. We consider the observations to consist of angularly resolved depth measurements of a single scene point that is being observed via many multi-bounce interactions with the specular room geometry. Applications of this problem statement include areas such as calibration, acoustic echo cancelation and time-of-flight imaging. We theoretically analyze the problem and derive sufficient conditions for a combination of convex room geometry, observer, and scene point to be reconstructable. The resulting constructive algorithm is exponential in nature and, therefore, not directly applicable to practical scenarios. To counter the situation, we propose theoretically devised geometric constraints that enable an efficient pruning of the solution space and develop a heuristic randomized search algorithm that uses these constraints to obtain an effective solution. We demonstrate the effectiveness of our algorithm on extensive simulations as well as in a challenging real-world calibration scenario.
6 0.89763302 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
7 0.8968668 21 cvpr-2013-A New Perspective on Uncalibrated Photometric Stereo
8 0.89466792 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
9 0.89446449 443 cvpr-2013-Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
10 0.89211583 109 cvpr-2013-Dense Non-rigid Point-Matching Using Random Projections
11 0.88430357 344 cvpr-2013-Radial Distortion Self-Calibration
12 0.88172489 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
13 0.88072872 152 cvpr-2013-Exemplar-Based Face Parsing
14 0.87995958 71 cvpr-2013-Boundary Cues for 3D Object Shape Recovery
15 0.87992114 365 cvpr-2013-Robust Real-Time Tracking of Multiple Objects by Volumetric Mass Densities
16 0.87783021 280 cvpr-2013-Maximum Cohesive Grid of Superpixels for Fast Object Localization
17 0.87760401 311 cvpr-2013-Occlusion Patterns for Object Class Detection
18 0.87712461 111 cvpr-2013-Dense Reconstruction Using 3D Object Shape Priors
19 0.87678379 147 cvpr-2013-Ensemble Learning for Confidence Measures in Stereo Vision
20 0.87631953 19 cvpr-2013-A Minimum Error Vanishing Point Detection Approach for Uncalibrated Monocular Images of Man-Made Environments