cvpr cvpr2013 cvpr2013-303 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Zhenglong Zhou, Zhe Wu, Ping Tan
Abstract: We present a method to capture both 3D shape and spatially varying reflectance with a multi-view photometric stereo technique that works for general isotropic materials. Our data capture setup is simple, which consists of only a digital camera and a handheld light source. From a single viewpoint, we use a set of photometric stereo images to identify surface points with the same distance to the camera. We collect this information from multiple viewpoints and combine it with structure-from-motion to obtain a precise reconstruction of the complete 3D shape. The spatially varying isotropic bidirectional reflectance distributionfunction (BRDF) is captured by simultaneously inferring a set of basis BRDFs and their mixing weights at each surface point. According to our experiments, the captured shapes are accurate to 0.3 millimeters. The captured reflectance has relative root-mean-square error (RMSE) of 9%.
Reference: text
sentIndex sentText sentNum sentScore
1 Our data capture setup is simple, which consists of only a digital camera and a handheld light source. [sent-2, score-0.455]
2 From a single viewpoint, we use a set of photometric stereo images to identify surface points with the same distance to the camera. [sent-3, score-0.461]
3 The spatially varying isotropic bidirectional reflectance distributionfunction (BRDF) is captured by simultaneously inferring a set of basis BRDFs and their mixing weights at each surface point. [sent-5, score-0.927]
4 The captured reflectance has relative root-mean-square error (RMSE) of 9%. [sent-8, score-0.434]
5 Introduction Appearance capture methods recover both 3D shape and surface reflectance of objects, allowing photorealistic ren- dering of the captured objects from arbitrary viewpoints and lighting conditions. [sent-10, score-0.93]
6 Typically, appearance capture is performed with sophisticated hardware setups such as the light stage of Ghosh et al. [sent-12, score-0.281]
7 Our simplest setup only contains a digital camera and a handheld moving light source. [sent-17, score-0.367]
8 The appearance of opaque objects is well represented by a bi-directional reflectance distribution function (BRDF). [sent-22, score-0.349]
9 Their performance degrades when the real objects have different reflectance from the assumed model. [sent-26, score-0.349]
10 We exploit reflectance symmetries to work on objects with general spatially varying isotropic BRDF. [sent-27, score-0.582]
11 We collect iso-depth contours from multiple viewpoints to reconstruct the complete 3D shape. [sent-31, score-0.223]
12 We assume the BRDF at each surface point is a linear combination of a few basis isotropic BRDFs which are represented by 3D discrete tables to handle general material. [sent-37, score-0.393]
13 Texture color at each surface point is decided according to its image projections. [sent-43, score-0.213]
14 Based on a precise 3D reconstruction, parametric reflectance functions can be fitted at each surface point according to the image observations, as in [28, 17]. [sent-47, score-0.641]
15 Since different sensors are used for shape and reflectance capture, this registration is difficult and often causes artifacts in misaligned regions. [sent-49, score-0.436]
16 Some methods [23, 1] combine reflectance recovered from photometric stereo and shape recovered from structuredlight, where registration is relatively simple. [sent-50, score-0.74]
17 However, they 111444888200 need to capture images under both structured-light and varying directional light at each viewpoint, which is tedious and requires a more complicated setup than ours. [sent-51, score-0.271]
18 Our method belongs to photometric approaches that capture both shape and reflectance from the same set of images. [sent-53, score-0.668]
19 The performance of these methods degrades when the real objects have different reflectance from the assumed model. [sent-57, score-0.349]
20 Recently, a few algorithms [3, 13] were proposed for appearance capture by exploiting various reflectance symmetries that are valid for a broader class of objects. [sent-65, score-0.437]
21 [5] both re- covered iso-contours of depth and gradient magnitude for isotropic surfaces. [sent-69, score-0.28]
22 Both methods are built upon reflectance symmetry embedded in ‘isotropic pairs’ introduced in [3 1]. [sent-72, score-0.418]
23 First, we reconstruct a complete 3D shape rather than a single-view normal map. [sent-74, score-0.273]
24 Second, we combine multi-view geometry and photometric cues to avoid fragile iterative optimization of shape and reflectance. [sent-75, score-0.231]
25 These methods are only applicable to near-flat surfaces where the surface normals are known beforehand. [sent-79, score-0.256]
26 At each viewpoint, we capture photometric stereo images with a moving light source. [sent-83, score-0.415]
27 Once the shape is fixed, we estimate a set of basis isotropic BRDFs and their mixing weights at each surface point to model the surface reflectance. [sent-87, score-0.741]
28 Iso-depth contour estimation Alldrin and Kriegman [2] observed that isotropy allows almost trivial estimation of iso-depth contours in the absence of global illumination effects such as shadows and inter-reflections. [sent-91, score-0.236]
29 Under orthographic projection and directional lighting that moves on a view-centered circle, the plane spanned by the viewing direction and the surface normal direction of an isotropic 1 surface point can be recovered precisely according to the symmetry of the observed pixel intensity profile. [sent-94, score-1.209]
30 In the camera local coordinate system, where the zaxis is aligned with the viewing direction, this plane gives the azimuth angle of the surface normal, which is the angle between the x-axis and the normal’s projection in the xyplane. [sent-95, score-0.76]
31 Figure 1 (a) shows the observed pixel intensities under 36 different lighting directions on a view-centered circle. [sent-98, score-0.302]
32 The red symmetry axis of these observations provides a good estimation of the azimuth angle. [sent-100, score-0.391]
33 Once azimuth angles are computed, at each pixel, we can recover an iso-depth contour by tracing along the directions perpendicular to the xy-plane projection of the surface normal there. [sent-101, score-0.899]
34 For easier reference, we refer this direction of a projected surface normal as the azimuth direction in this paper. [sent-102, score-0.729]
35 Handheld Point Light Source In practice, it is more convenient to capture images with a handheld bulb, i. [sent-103, score-0.223]
36 So we compute spatially variant lighting directions at each pixel, and interpolate the desired observations from recorded pixel intensities. [sent-106, score-0.337]
37 The lighting directions at each pixel are then computed according to the 3D positions of that pixel and the light sources. [sent-110, score-0.477]
38 To allow flexible data capture, we interpolate observations under lighting directions lying on a view-centered circle, and compute the azimuth angle from these interpolated observations. [sent-111, score-0.659]
39 Here, we follow [2] to refer it as isotropy because bilateral symmetry is often observed for isotropic surfaces. [sent-113, score-0.339]
40 2e4lsthazimuAth1a8n0ApgFonlrabogiveuplr oniveasdop tsfrueinaotphieancsphrofitaycxpnhgerolf’i3s60norm(ca)ldi53e0%c 0tion;1(b)2cOarsdet3ofhFaudri4oewsri5canb6rek7this symmetry; (c) the intensity profile of most of isotropic BRDFs in [21] can be well represented by a 2-order Fourier series. [sent-116, score-0.3]
41 WecomputeaDelaunytriangulation ftheorignal lighting directions (red dots) in the projective plane. [sent-118, score-0.258]
42 Left: the circle radius d is the mean distance between the red dots and the viewpoint v. [sent-120, score-0.265]
43 As shown in the left of Figure 2, the original lighting directions at a pixel are represented by the red points. [sent-124, score-0.259]
44 The radius d of the blue circle is computed as the mean distance between the red dots and the viewpoint v. [sent-127, score-0.265]
45 Points in the penumbra also cause problems in the reflectance estimation in Section 5. [sent-136, score-0.425]
46 Consider a Lambertian point with surface normal n = (nx , ny , nz) and albedo ρ. [sent-138, score-0.289]
47 Its intensity should be ρrnx cos θ + ρrny sin θ − ρznz when the lighting direction is (r cos θ, r sins iθn, −θ z−). [sent-139, score-0.34]
48 For each BRDF in the database, we uniformly sample ninety normals along a longitude on the visible upper hemisphere, and render them under a light moving on a view-centered circle. [sent-146, score-0.211]
49 ≤So 2 we always apply RitAhN noSrAmCa tizoe dfit a second order Fourier series to each observed intensity profile, and estimate the azimuth angle according to the symmetry of the fitted curve. [sent-153, score-0.594]
50 As shown by the green vertical line in Figure 1 (b), our estimated azimuth angle is closer to the ground truth. [sent-154, score-0.413]
51 Tracing Contours Once an azimuth angle is computed at each pixel, we proceed to generate iso-depth contours. [sent-156, score-0.413]
52 Starting from every pixel, we iteratively trace along the two directions perpendicular to the azimuth direction with a step of 0. [sent-157, score-0.466]
53 Specifically, suppose the estimated azimuth angle is θ at a pixel x. [sent-159, score-0.463]
54 2W)e, sthinen(θ replace d+ xand d− according to the azimuth angles of x+ and x− respectively and continue to trace. [sent-163, score-0.426]
55 To avoid tracing across discontinuous surface points, we use the method described in the ‘NPR camera’ [24] to identify discontinuities. [sent-166, score-0.28]
56 Multi-view depth propagation A standard structure-from-motion algorithm such as [19, 30] can reconstruct a set of sparse 3D points on the object. [sent-171, score-0.279]
57 could be affected by moving highlights, we compute a median image at each viewpoint by taking the median intensity of each pixel and use these images for feature matching. [sent-178, score-0.337]
58 Reconstructed 3D points are combined with the traced isodepth contours to recover the complete 3D shape. [sent-179, score-0.29]
59 We perform a depth propagation to assign the depth of x to all pixels on Ci. [sent-182, score-0.252]
60 We then use the surface normal of x to select L (L = 7 in our implementation) most front parallel views where x is visible. [sent-208, score-0.337]
61 We assume p is visible in all these L images and check the consistency of the azimuth angles at its projections. [sent-209, score-0.429]
62 The azimuth angles at corresponding pixels in two different views uniquely decide a 3D normal direction 2. [sent-210, score-0.609]
63 If different combinations of these L views all lead to consistent 3D normals (the angle between any two normals is within T degrees), we consider p as consistent. [sent-211, score-0.323]
64 Shape Optimization After depth propagation, we have a set of 3D points, each with a normal direction estimated. [sent-224, score-0.272]
65 We apply the Poisson surface reconstruction [15] to these points to obtain a triangulated surface. [sent-225, score-0.263]
66 This surface is further optimized according to [23] by fusing the 3D point positions and their normal directions. [sent-226, score-0.338]
67 Reflectance Capture We assume the surface reflectance can be represented by a linear combination of several (K=2) basis isotropic BRDFs. [sent-228, score-0.742]
68 Once the 3D shape is reconstructed, we follow × [16] to estimate the basis BRDFs and their mixing weights at each point on the surface. [sent-229, score-0.221]
69 We consider the general tri2An azimuth angle in one view (with the camera center) decides a plane where the normal must lie in. [sent-230, score-0.587]
70 In constructing the matrix V, we avoid pixels observed from slanted viewing directions (the angle between viewing direction and surface normal is larger than 40 degrees in our implementation), where a small shape reconstruction error can cause a big change in their projected image positions. [sent-240, score-0.816]
71 To further improve reflectance capture accuracy, we first compute H from a subset of precisely reconstructed 3D points, whose reconstructed normals from different combinations of azimuth angles are consistent within 1. [sent-243, score-0.992]
72 e aT,h we hfiicrsht setup euss leidn a rh aimndagheelsd a bt 1ul2b0 as light source to ensure data capture flexibility. [sent-249, score-0.271]
73 A Handheld System Consisting of just a video camera and a handheld light source, this system is compact and portable. [sent-264, score-0.303]
74 At each viewpoint, we moved a handheld bulb to capture a short video clip (about two minutes), and then uniformly sampled about 100 images with different lighting directions. [sent-265, score-0.392]
75 (c) is a rendering according to the captured reflectance from the same viewpoint and lighting condition as the input image in (a). [sent-272, score-0.727]
76 To provide a quantitative evaluation on shape capture, we visualize the shape reconstruction error (measured in millimeters) in (d). [sent-273, score-0.257]
77 72 LEDs were uniformly distributed on two concentric circles of diameter 400 and 600 millimeters respectively. [sent-282, score-0.312]
78 The camera was synchronized with the LED lights such that at each video frame, there was only one light turned on. [sent-284, score-0.222]
79 At each viewpoint, we captured 30 images with different lighting directions in 12 seconds (at 4fps). [sent-285, score-0.256]
80 Hence, at a general surface point, the local lighting directions will form two conics in the projective plane as illustrated on the right of Figure 2. [sent-292, score-0.422]
81 When computing azimuth angles, we performed a Delaunay triangulation based interpolation as introduced in Section 4. [sent-293, score-0.322]
82 (c) rendering with the recovered reflectance model from the same viewpoint and lighting condition as the image in (a). [sent-303, score-0.679]
83 v and the original lighting directions the red dots in the inner (or outer) conic. [sent-307, score-0.278]
84 The rusted metal ‘Cup’ has quickly change reflectance over its surface. [sent-321, score-0.349]
85 Their median (or mean) shape reconstruction error was 0. [sent-325, score-0.231]
86 The iterative shape and reflectance optimization in [3] is compli- × (a)(b)(c)(d) Figure9. [sent-336, score-0.436]
87 (b) the shape computed from the estimated normal according to [34]. [sent-341, score-0.261]
88 o m(cp) uitse a rendering from novel lighting direction according to the estimated normal and reflectance. [sent-355, score-0.413]
89 The median (and mean) angular error of normal directions is 13. [sent-362, score-0.309]
90 computed azimuth angles in 1minute, and traced iso-depth contours in 1. [sent-374, score-0.537]
91 Depth propagation took 16 minutes (for 40 viewpoints), and the final shape optimization took 1 minute. [sent-376, score-0.309]
92 Much of the involved process including azimuth angle computation, isodepth contour tracing, and BRDF mixing weight computation can be easily parallelized. [sent-381, score-0.653]
93 Discussion We propose a method to capture both shape and reflectance of real objects with spatially variant isotropic reflectance. [sent-383, score-0.757]
94 Our method requires a simple hardware setup and is able to capture 3D shapes accurate to 0. [sent-384, score-0.226]
95 (Note that θd is the angle between viewing and lighting directions as shown in Figure 4. [sent-395, score-0.343]
96 ) Hence, during reflectance capturing, we can only discretize θd to two levels, and cannot capture Fresnel effects faithfully. [sent-396, score-0.437]
97 Toward reconstructing surfaceswith arbitrary isotropic reflectance : A stratified pho- [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] tometric stereo approach. [sent-411, score-0.655]
98 A coaxial optical scanner for synchronous acquisition of 3d geometry and surface reflectance. [sent-518, score-0.231]
99 Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. [sent-586, score-0.223]
100 A data- [22] [23] [24] [25] [26] [27] [28] [29] [30] [3 1] driven reflectance model. [sent-594, score-0.349]
wordName wordTfidf (topN-words)
[('reflectance', 0.349), ('azimuth', 0.322), ('brdf', 0.277), ('millimeters', 0.201), ('isotropic', 0.192), ('brdfs', 0.19), ('surface', 0.164), ('photometric', 0.144), ('handheld', 0.135), ('normal', 0.125), ('lighting', 0.124), ('light', 0.119), ('viewpoint', 0.102), ('mixing', 0.097), ('circle', 0.094), ('normals', 0.092), ('contours', 0.091), ('angle', 0.091), ('depth', 0.088), ('capture', 0.088), ('shape', 0.087), ('directions', 0.085), ('holroyd', 0.083), ('tracing', 0.081), ('isotropy', 0.078), ('propagation', 0.076), ('isodepth', 0.076), ('penumbra', 0.076), ('hardware', 0.074), ('viewpoints', 0.071), ('dots', 0.069), ('symmetry', 0.069), ('traced', 0.069), ('coaxial', 0.067), ('contour', 0.067), ('stereo', 0.064), ('setup', 0.064), ('intensity', 0.063), ('rmse', 0.062), ('diameter', 0.062), ('median', 0.061), ('reconstruct', 0.061), ('depths', 0.059), ('direction', 0.059), ('alldrin', 0.059), ('rendering', 0.056), ('frog', 0.056), ('angles', 0.055), ('took', 0.055), ('points', 0.054), ('lights', 0.054), ('tan', 0.054), ('fourier', 0.053), ('led', 0.053), ('check', 0.052), ('diffuse', 0.052), ('leds', 0.052), ('acls', 0.05), ('snyder', 0.05), ('tometric', 0.05), ('cup', 0.05), ('pixel', 0.05), ('camera', 0.049), ('circles', 0.049), ('according', 0.049), ('projective', 0.049), ('ghosh', 0.049), ('recovered', 0.048), ('views', 0.048), ('captured', 0.047), ('cos', 0.047), ('curless', 0.047), ('specular', 0.046), ('rusinkiewicz', 0.046), ('acm', 0.046), ('reconstruction', 0.045), ('bulb', 0.045), ('shade', 0.045), ('profile', 0.045), ('lawrence', 0.044), ('reconstructed', 0.043), ('intensities', 0.043), ('viewing', 0.043), ('buddha', 0.041), ('spatially', 0.041), ('precise', 0.04), ('nehab', 0.039), ('parametric', 0.039), ('error', 0.038), ('justification', 0.037), ('fresnel', 0.037), ('basis', 0.037), ('calibrate', 0.037), ('interpolate', 0.037), ('minutes', 0.036), ('coded', 0.036), ('spacetime', 0.036), ('peers', 0.036), ('slanted', 0.036), ('identify', 0.035)]
simIndex simValue paperId paperTitle
same-paper 1 1.0 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
Author: Zhenglong Zhou, Zhe Wu, Ping Tan
Abstract: We present a method to capture both 3D shape and spatially varying reflectance with a multi-view photometric stereo technique that works for general isotropic materials. Our data capture setup is simple, which consists of only a digital camera and a handheld light source. From a single viewpoint, we use a set of photometric stereo images to identify surface points with the same distance to the camera. We collect this information from multiple viewpoints and combine it with structure-from-motion to obtain a precise reconstruction of the complete 3D shape. The spatially varying isotropic bidirectional reflectance distributionfunction (BRDF) is captured by simultaneously inferring a set of basis BRDFs and their mixing weights at each surface point. According to our experiments, the captured shapes are accurate to 0.3 millimeters. The captured reflectance has relative root-mean-square error (RMSE) of 9%.
2 0.38306117 443 cvpr-2013-Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
Author: Feng Lu, Yasuyuki Matsushita, Imari Sato, Takahiro Okabe, Yoichi Sato
Abstract: We propose an uncalibrated photometric stereo method that works with general and unknown isotropic reflectances. Our method uses a pixel intensity profile, which is a sequence of radiance intensities recorded at a pixel across multi-illuminance images. We show that for general isotropic materials, the geodesic distance between intensity profiles is linearly related to the angular difference of their surface normals, and that the intensity distribution of an intensity profile conveys information about the reflectance properties, when the intensity profile is obtained under uniformly distributed directional lightings. Based on these observations, we show that surface normals can be estimated up to a convex/concave ambiguity. A solution method based on matrix decomposition with missing data is developed for a reliable estimation. Quantitative and qualitative evaluations of our method are performed using both synthetic and real-world scenes.
3 0.36370289 465 cvpr-2013-What Object Motion Reveals about Shape with Unknown BRDF and Lighting
Author: Manmohan Chandraker, Dikpal Reddy, Yizhou Wang, Ravi Ramamoorthi
Abstract: We present a theory that addresses the problem of determining shape from the (small or differential) motion of an object with unknown isotropic reflectance, under arbitrary unknown distant illumination, , for both orthographic and perpsective projection. Our theory imposes fundamental limits on the hardness of surface reconstruction, independent of the method involved. Under orthographic projection, we prove that three differential motions suffice to yield an invariant that relates shape to image derivatives, regardless of BRDF and illumination. Under perspective projection, we show that four differential motions suffice to yield depth and a linear constraint on the surface gradient, with unknown BRDF and lighting. Further, we delineate the topological classes up to which reconstruction may be achieved using the invariants. Finally, we derive a general stratification that relates hardness of shape recovery to scene complexity. Qualitatively, our invariants are homogeneous partial differential equations for simple lighting and inhomogeneous for complex illumination. Quantitatively, our framework shows that the minimal number of motions required to resolve shape is greater for more complex scenes. Prior works that assume brightness constancy, Lambertian BRDF or a known directional light source follow as special cases of our stratification. We illustrate with synthetic and real data how potential reconstruction methods may exploit our framework.
4 0.35861301 75 cvpr-2013-Calibrating Photometric Stereo by Holistic Reflectance Symmetry Analysis
Author: Zhe Wu, Ping Tan
Abstract: Under unknown directional lighting, the uncalibrated Lambertian photometric stereo algorithm recovers the shape of a smooth surface up to the generalized bas-relief (GBR) ambiguity. We resolve this ambiguity from the halfvector symmetry, which is observed in many isotropic materials. Under this symmetry, a 2D BRDF slice with low-rank structure can be obtained from an image, if the surface normals and light directions are correctly recovered. In general, this structure is destroyed by the GBR ambiguity. As a result, we can resolve the ambiguity by restoring this structure. We develop a simple algorithm of auto-calibration from separable homogeneous specular reflection of real images. Compared with previous methods, this method takes a holistic approach to exploiting reflectance symmetry and produces superior results.
5 0.2907083 54 cvpr-2013-BRDF Slices: Accurate Adaptive Anisotropic Appearance Acquisition
Author: Jirí Filip, Radomír Vávra, Michal Haindl, Pavel Žid, Mikuláš Krupika, Vlastimil Havran
Abstract: In this paper we introduce unique publicly available dense anisotropic BRDF data measurements. We use this dense data as a reference for performance evaluation of the proposed BRDF sparse angular sampling and interpolation approach. The method is based on sampling of BRDF subspaces at fixed elevations by means of several adaptively-represented, uniformly distributed, perpendicular slices. Although this proposed method requires only a sparse sampling of material, the interpolation provides a very accurate reconstruction, visually and computationally comparable to densely measured reference. Due to the simple slices measurement and method’s robustness it allows for a highly accurate acquisition of BRDFs. This in comparison with standard uniform angular sampling, is considerably faster yet uses far less samples.
6 0.22546177 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
7 0.21617821 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
9 0.19706 21 cvpr-2013-A New Perspective on Uncalibrated Photometric Stereo
11 0.15356928 409 cvpr-2013-Spectral Modeling and Relighting of Reflective-Fluorescent Scenes
12 0.15020686 330 cvpr-2013-Photometric Ambient Occlusion
13 0.13954248 354 cvpr-2013-Relative Volume Constraints for Single View 3D Reconstruction
14 0.13740513 71 cvpr-2013-Boundary Cues for 3D Object Shape Recovery
15 0.13712007 423 cvpr-2013-Template-Based Isometric Deformable 3D Reconstruction with Sampling-Based Focal Length Self-Calibration
16 0.13252972 27 cvpr-2013-A Theory of Refractive Photo-Light-Path Triangulation
17 0.13059102 196 cvpr-2013-HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences
18 0.1166074 230 cvpr-2013-Joint 3D Scene Reconstruction and Class Segmentation
19 0.11589505 286 cvpr-2013-Mirror Surface Reconstruction from a Single Image
20 0.11338705 245 cvpr-2013-Layer Depth Denoising and Completion for Structured-Light RGB-D Cameras
topicId topicWeight
[(0, 0.207), (1, 0.315), (2, 0.009), (3, 0.105), (4, -0.023), (5, -0.186), (6, -0.184), (7, 0.08), (8, 0.083), (9, -0.031), (10, -0.118), (11, -0.183), (12, -0.098), (13, -0.09), (14, 0.074), (15, 0.107), (16, 0.226), (17, -0.146), (18, 0.003), (19, -0.097), (20, 0.022), (21, -0.019), (22, -0.017), (23, 0.055), (24, 0.008), (25, -0.051), (26, -0.027), (27, 0.008), (28, 0.023), (29, 0.048), (30, -0.102), (31, 0.009), (32, -0.058), (33, -0.041), (34, 0.033), (35, -0.021), (36, 0.035), (37, 0.01), (38, 0.081), (39, -0.017), (40, 0.08), (41, 0.008), (42, 0.021), (43, 0.013), (44, -0.018), (45, -0.021), (46, -0.054), (47, -0.024), (48, 0.015), (49, 0.078)]
simIndex simValue paperId paperTitle
1 0.9639616 75 cvpr-2013-Calibrating Photometric Stereo by Holistic Reflectance Symmetry Analysis
Author: Zhe Wu, Ping Tan
Abstract: Under unknown directional lighting, the uncalibrated Lambertian photometric stereo algorithm recovers the shape of a smooth surface up to the generalized bas-relief (GBR) ambiguity. We resolve this ambiguity from the halfvector symmetry, which is observed in many isotropic materials. Under this symmetry, a 2D BRDF slice with low-rank structure can be obtained from an image, if the surface normals and light directions are correctly recovered. In general, this structure is destroyed by the GBR ambiguity. As a result, we can resolve the ambiguity by restoring this structure. We develop a simple algorithm of auto-calibration from separable homogeneous specular reflection of real images. Compared with previous methods, this method takes a holistic approach to exploiting reflectance symmetry and produces superior results.
same-paper 2 0.92987537 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
Author: Zhenglong Zhou, Zhe Wu, Ping Tan
Abstract: We present a method to capture both 3D shape and spatially varying reflectance with a multi-view photometric stereo technique that works for general isotropic materials. Our data capture setup is simple, which consists of only a digital camera and a handheld light source. From a single viewpoint, we use a set of photometric stereo images to identify surface points with the same distance to the camera. We collect this information from multiple viewpoints and combine it with structure-from-motion to obtain a precise reconstruction of the complete 3D shape. The spatially varying isotropic bidirectional reflectance distributionfunction (BRDF) is captured by simultaneously inferring a set of basis BRDFs and their mixing weights at each surface point. According to our experiments, the captured shapes are accurate to 0.3 millimeters. The captured reflectance has relative root-mean-square error (RMSE) of 9%.
3 0.92562431 443 cvpr-2013-Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
Author: Feng Lu, Yasuyuki Matsushita, Imari Sato, Takahiro Okabe, Yoichi Sato
Abstract: We propose an uncalibrated photometric stereo method that works with general and unknown isotropic reflectances. Our method uses a pixel intensity profile, which is a sequence of radiance intensities recorded at a pixel across multi-illuminance images. We show that for general isotropic materials, the geodesic distance between intensity profiles is linearly related to the angular difference of their surface normals, and that the intensity distribution of an intensity profile conveys information about the reflectance properties, when the intensity profile is obtained under uniformly distributed directional lightings. Based on these observations, we show that surface normals can be estimated up to a convex/concave ambiguity. A solution method based on matrix decomposition with missing data is developed for a reliable estimation. Quantitative and qualitative evaluations of our method are performed using both synthetic and real-world scenes.
4 0.87694818 465 cvpr-2013-What Object Motion Reveals about Shape with Unknown BRDF and Lighting
Author: Manmohan Chandraker, Dikpal Reddy, Yizhou Wang, Ravi Ramamoorthi
Abstract: We present a theory that addresses the problem of determining shape from the (small or differential) motion of an object with unknown isotropic reflectance, under arbitrary unknown distant illumination, , for both orthographic and perpsective projection. Our theory imposes fundamental limits on the hardness of surface reconstruction, independent of the method involved. Under orthographic projection, we prove that three differential motions suffice to yield an invariant that relates shape to image derivatives, regardless of BRDF and illumination. Under perspective projection, we show that four differential motions suffice to yield depth and a linear constraint on the surface gradient, with unknown BRDF and lighting. Further, we delineate the topological classes up to which reconstruction may be achieved using the invariants. Finally, we derive a general stratification that relates hardness of shape recovery to scene complexity. Qualitatively, our invariants are homogeneous partial differential equations for simple lighting and inhomogeneous for complex illumination. Quantitatively, our framework shows that the minimal number of motions required to resolve shape is greater for more complex scenes. Prior works that assume brightness constancy, Lambertian BRDF or a known directional light source follow as special cases of our stratification. We illustrate with synthetic and real data how potential reconstruction methods may exploit our framework.
5 0.81700754 21 cvpr-2013-A New Perspective on Uncalibrated Photometric Stereo
Author: Thoma Papadhimitri, Paolo Favaro
Abstract: We investigate the problem of reconstructing normals, albedo and lights of Lambertian surfaces in uncalibrated photometric stereo under the perspective projection model. Our analysis is based on establishing the integrability constraint. In the orthographicprojection case, it is well-known that when such constraint is imposed, a solution can be identified only up to 3 parameters, the so-called generalized bas-relief (GBR) ambiguity. We show that in the perspective projection case the solution is unique. We also propose a closed-form solution which is simple, efficient and robust. We test our algorithm on synthetic data and publicly available real data. Our quantitative tests show that our method outperforms all prior work of uncalibrated photometric stereo under orthographic projection.
6 0.77728546 54 cvpr-2013-BRDF Slices: Accurate Adaptive Anisotropic Appearance Acquisition
8 0.74457443 409 cvpr-2013-Spectral Modeling and Relighting of Reflective-Fluorescent Scenes
9 0.62126505 435 cvpr-2013-Towards Contactless, Low-Cost and Accurate 3D Fingerprint Identification
10 0.58762342 330 cvpr-2013-Photometric Ambient Occlusion
12 0.53629911 423 cvpr-2013-Template-Based Isometric Deformable 3D Reconstruction with Sampling-Based Focal Length Self-Calibration
13 0.5294382 354 cvpr-2013-Relative Volume Constraints for Single View 3D Reconstruction
14 0.52072531 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
15 0.50390077 27 cvpr-2013-A Theory of Refractive Photo-Light-Path Triangulation
16 0.50148779 391 cvpr-2013-Sensing and Recognizing Surface Textures Using a GelSight Sensor
17 0.50064325 289 cvpr-2013-Monocular Template-Based 3D Reconstruction of Extensible Surfaces with Local Linear Elasticity
18 0.49588615 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
19 0.4906171 226 cvpr-2013-Intrinsic Characterization of Dynamic Surfaces
20 0.48704147 71 cvpr-2013-Boundary Cues for 3D Object Shape Recovery
topicId topicWeight
[(10, 0.125), (16, 0.066), (26, 0.034), (28, 0.028), (33, 0.223), (66, 0.186), (67, 0.035), (69, 0.06), (79, 0.012), (87, 0.124)]
simIndex simValue paperId paperTitle
1 0.92496222 75 cvpr-2013-Calibrating Photometric Stereo by Holistic Reflectance Symmetry Analysis
Author: Zhe Wu, Ping Tan
Abstract: Under unknown directional lighting, the uncalibrated Lambertian photometric stereo algorithm recovers the shape of a smooth surface up to the generalized bas-relief (GBR) ambiguity. We resolve this ambiguity from the halfvector symmetry, which is observed in many isotropic materials. Under this symmetry, a 2D BRDF slice with low-rank structure can be obtained from an image, if the surface normals and light directions are correctly recovered. In general, this structure is destroyed by the GBR ambiguity. As a result, we can resolve the ambiguity by restoring this structure. We develop a simple algorithm of auto-calibration from separable homogeneous specular reflection of real images. Compared with previous methods, this method takes a holistic approach to exploiting reflectance symmetry and produces superior results.
2 0.90423816 127 cvpr-2013-Discovering the Structure of a Planar Mirror System from Multiple Observations of a Single Point
Author: Ilya Reshetouski, Alkhazur Manakov, Ayush Bandhari, Ramesh Raskar, Hans-Peter Seidel, Ivo Ihrke
Abstract: We investigate the problem of identifying the position of a viewer inside a room of planar mirrors with unknown geometry in conjunction with the room’s shape parameters. We consider the observations to consist of angularly resolved depth measurements of a single scene point that is being observed via many multi-bounce interactions with the specular room geometry. Applications of this problem statement include areas such as calibration, acoustic echo cancelation and time-of-flight imaging. We theoretically analyze the problem and derive sufficient conditions for a combination of convex room geometry, observer, and scene point to be reconstructable. The resulting constructive algorithm is exponential in nature and, therefore, not directly applicable to practical scenarios. To counter the situation, we propose theoretically devised geometric constraints that enable an efficient pruning of the solution space and develop a heuristic randomized search algorithm that uses these constraints to obtain an effective solution. We demonstrate the effectiveness of our algorithm on extensive simulations as well as in a challenging real-world calibration scenario.
same-paper 3 0.87587678 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
Author: Zhenglong Zhou, Zhe Wu, Ping Tan
Abstract: We present a method to capture both 3D shape and spatially varying reflectance with a multi-view photometric stereo technique that works for general isotropic materials. Our data capture setup is simple, which consists of only a digital camera and a handheld light source. From a single viewpoint, we use a set of photometric stereo images to identify surface points with the same distance to the camera. We collect this information from multiple viewpoints and combine it with structure-from-motion to obtain a precise reconstruction of the complete 3D shape. The spatially varying isotropic bidirectional reflectance distributionfunction (BRDF) is captured by simultaneously inferring a set of basis BRDFs and their mixing weights at each surface point. According to our experiments, the captured shapes are accurate to 0.3 millimeters. The captured reflectance has relative root-mean-square error (RMSE) of 9%.
4 0.85734195 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
Author: Jonathan T. Barron, Jitendra Malik
Abstract: In this paper we extend the “shape, illumination and reflectance from shading ” (SIRFS) model [3, 4], which recovers intrinsic scene properties from a single image. Though SIRFS performs well on images of segmented objects, it performs poorly on images of natural scenes, which contain occlusion and spatially-varying illumination. We therefore present Scene-SIRFS, a generalization of SIRFS in which we have a mixture of shapes and a mixture of illuminations, and those mixture components are embedded in a “soft” segmentation of the input image. We additionally use the noisy depth maps provided by RGB-D sensors (in this case, the Kinect) to improve shape estimation. Our model takes as input a single RGB-D image and produces as output an improved depth map, a set of surface normals, a reflectance image, a shading image, and a spatially varying model of illumination. The output of our model can be used for graphics applications, or for any application involving RGB-D images.
5 0.85472345 344 cvpr-2013-Radial Distortion Self-Calibration
Author: José Henrique Brito, Roland Angst, Kevin Köser, Marc Pollefeys
Abstract: In cameras with radial distortion, straight lines in space are in general mapped to curves in the image. Although epipolar geometry also gets distorted, there is a set of special epipolar lines that remain straight, namely those that go through the distortion center. By finding these straight epipolar lines in camera pairs we can obtain constraints on the distortion center(s) without any calibration object or plumbline assumptions in the scene. Although this holds for all radial distortion models we conceptually prove this idea using the division distortion model and the radial fundamental matrix which allow for a very simple closed form solution of the distortion center from two views (same distortion) or three views (different distortions). The non-iterative nature of our approach makes it immune to local minima and allows finding the distortion center also for cropped images or those where no good prior exists. Besides this, we give comprehensive relations between different undistortion models and discuss advantages and drawbacks.
6 0.84764993 443 cvpr-2013-Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
7 0.8202669 54 cvpr-2013-BRDF Slices: Accurate Adaptive Anisotropic Appearance Acquisition
8 0.81711113 21 cvpr-2013-A New Perspective on Uncalibrated Photometric Stereo
9 0.81568694 360 cvpr-2013-Robust Estimation of Nonrigid Transformation for Point Set Registration
10 0.81508303 365 cvpr-2013-Robust Real-Time Tracking of Multiple Objects by Volumetric Mass Densities
11 0.81385815 465 cvpr-2013-What Object Motion Reveals about Shape with Unknown BRDF and Lighting
12 0.8130759 400 cvpr-2013-Single Image Calibration of Multi-axial Imaging Systems
13 0.81283873 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
14 0.81148702 330 cvpr-2013-Photometric Ambient Occlusion
15 0.81027031 222 cvpr-2013-Incorporating User Interaction and Topological Constraints within Contour Completion via Discrete Calculus
16 0.80999374 155 cvpr-2013-Exploiting the Power of Stereo Confidences
18 0.80939126 61 cvpr-2013-Beyond Point Clouds: Scene Understanding by Reasoning Geometry and Physics
19 0.80911899 337 cvpr-2013-Principal Observation Ray Calibration for Tiled-Lens-Array Integral Imaging Display
20 0.80836642 19 cvpr-2013-A Minimum Error Vanishing Point Detection Approach for Uncalibrated Monocular Images of Man-Made Environments