iccv iccv2013 iccv2013-284 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Jaesik Park, Sudipta N. Sinha, Yasuyuki Matsushita, Yu-Wing Tai, In So Kweon
Abstract: We propose a method for accurate 3D shape reconstruction using uncalibrated multiview photometric stereo. A coarse mesh reconstructed using multiview stereo is first parameterized using a planar mesh parameterization technique. Subsequently, multiview photometric stereo is performed in the 2D parameter domain of the mesh, where all geometric and photometric cues from multiple images can be treated uniformly. Unlike traditional methods, there is no need for merging view-dependent surface normal maps. Our key contribution is a new photometric stereo based mesh refinement technique that can efficiently reconstruct meshes with extremely fine geometric details by directly estimating a displacement texture map in the 2D parameter domain. We demonstrate that intricate surface geometry can be reconstructed using several challenging datasets containing surfaces with specular reflections, multiple albedos and complex topologies.
Reference: text
sentIndex sentText sentNum sentScore
1 A coarse mesh reconstructed using multiview stereo is first parameterized using a planar mesh parameterization technique. [sent-3, score-1.982]
2 Subsequently, multiview photometric stereo is performed in the 2D parameter domain of the mesh, where all geometric and photometric cues from multiple images can be treated uniformly. [sent-4, score-1.039]
3 Unlike traditional methods, there is no need for merging view-dependent surface normal maps. [sent-5, score-0.252]
4 Our key contribution is a new photometric stereo based mesh refinement technique that can efficiently reconstruct meshes with extremely fine geometric details by directly estimating a displacement texture map in the 2D parameter domain. [sent-6, score-1.784]
5 We demonstrate that intricate surface geometry can be reconstructed using several challenging datasets containing surfaces with specular reflections, multiple albedos and complex topologies. [sent-7, score-0.238]
6 With recent progress in structure from motion (SfM) [12] and multiview stereo (MVS) [18], it is nowadays possible to reconstruct 3D models for many challenging scenes. [sent-10, score-0.289]
7 On the other hand, photometric methods, such as shape-from-shading [9] and photometric stereo [25], use shading cues to estimate per-pixel surface normal maps but do not directly provide depth estimates. [sent-13, score-1.242]
8 In this paper, we present a new multiview photometric stereo method that efficiently combines geometric and photometric cues. [sent-19, score-1.045]
9 It starts by recovering a coarse 3D mesh using existing state of the art SfM and MVS techniques. [sent-21, score-0.79]
10 The idea of transforming this mesh into a parameterized 2D space using a distortion minimizing piecewise continuous 3D-to2D mapping lies at the core of our method. [sent-22, score-0.789]
11 Unlike prior methods that use explicit 3D representations [6, 15], we use a planar parameterization of the mesh [19] and cast the mesh refinement problem into one of estimating a displacement map texture in the 2D parameter domain. [sent-23, score-1.961]
12 We show that both photometric stereo based surface normal estimation and mesh refinement can be efficiently and accurately performed in the parameterized 2D space. [sent-24, score-1.725]
13 First, images from multiple viewpoints can be naturally handled when performing multiview photometric stereo in the 2D parameter domain, because all the images can be registered without introducing large pixel distortions. [sent-26, score-0.711]
14 As surface normals can be directly estimated in this space using multiple registered images captured under varying lighting, it avoids the needs to first estimate per-view normal maps and then merge normal maps obtained from multiple viewpoints. [sent-27, score-0.536]
15 Second, we can efficiently recover an extremely detailed 3D mesh exploiting the full resolution available in the input images. [sent-29, score-0.827]
16 The level of geometric detail in our representation can be easily controlled by specifying the appropriate resolution of the estimated displacement map and the optimization is more efficient than direct 3D methods that must resort to subdividing the mesh and refining the vertex positions. [sent-30, score-1.029]
17 We also perform a quantitative evaluation which demonstrates the advantage of our mesh refinement technique over existing methods [6, 15]. [sent-33, score-0.912]
18 Structure-from-motion is used to calibrate the cameras and multiview stereo is used to recover a coarse mesh. [sent-36, score-0.4]
19 After parameterizing the mesh, multiview photometric stereo and mesh refinement are performed in the 2D parameter domain. [sent-37, score-1.52]
20 We use a different 3D shape representation, which is key to the high accuracy and efficiency of the normal map estimation and mesh refinement steps in our method. [sent-42, score-1.083]
21 These methods can be broadly classified into 2D depth map refinement approaches and the ones that perform 3D mesh refinement. [sent-45, score-0.995]
22 [15] propose an efficient method for 2D depth map refinement by adjusting depth values using orthogonality between depth gradients and surface orientations. [sent-47, score-0.537]
23 Okatani and Deguchi [16] propose a probabilistic framework for shape refinement using the first-order derivative of surface normals. [sent-50, score-0.32]
24 For 3D mesh refinement, Rushmier and Bernardini [17] adjust local normal directions obtained using photometric stereo. [sent-52, score-1.191]
25 [15] state that their 2D depth refinement method can be extended to handle a 3D mesh. [sent-54, score-0.242]
26 [11] introduce a generalized method for modeling nonLambertian surfaces by wavelet-based BRDFs and use it for mesh refinement. [sent-56, score-0.752]
27 [6] iteratively refine mesh polygons by minimizing a quadratic energy function. [sent-58, score-0.744]
28 [26] use the spherical harmonics representation to estimate global illumination, and refine a preliminary mesh using photometric stereo by minimizing ? [sent-60, score-1.241]
29 [23] first integrate per-view normal maps into partial meshes, then deforms them using thin-plate offsets to improve the alignment while preserving geometric details. [sent-64, score-0.209]
30 These 3D mesh refinement methods generally use a high-resolution mesh in order to enclose high frequency details obtained by photometric methods; however, determining the appropriate mesh resolution is non-trivial due to the view-dependent variation of effective resolutions. [sent-65, score-2.729]
31 In contrast, our method allows the mesh resolution to be derived directly from the normal map resolution and avoids the problem of undersampling mesh vertices. [sent-66, score-1.741]
32 In addition, our 2D parameterization approach performs mesh refinement efficiently, where only 1D vertex displacements are optimized rather than directly working in the 3D coordinates. [sent-67, score-1.066]
33 For now, let us assume that all the cameras are calibrated and the initial base mesh is available. [sent-73, score-0.846]
34 The methods for calibration and obtaining the initial base mesh are later explained in Sec. [sent-74, score-0.848]
35 After describing the mesh parameterization scheme, we explain how surface normal estimation and mesh refinement is performed in the 2D parameter domain. [sent-76, score-2.01]
36 Mesh Parameterization In our method, first the triangulated base mesh denoted by M, is mapped to a planar parameterized space using a piecewise mcoanptpineduo tuos afu pnlcantioarn fa : mRe3t →riz eRd2 ,s pwachiech us i sn referred to as mesh parameterization [19]→ →(se Re Fig. [sent-80, score-1.802]
37 While this process is not limited to a particular mesh parameterization method, in this paper, we use the Iso-charts method proposed by Zhou et al. [sent-82, score-0.866]
38 [30], which minimizes non-uniform distortions of the original mesh by finding optimal cuts that partition the mesh into segments. [sent-83, score-1.442]
39 Each connected segment 11 116622 Algorithm 1: Image Warping Input: Image I,camera projection matrix P, mesh M and i Itsm faagcee visibility Output: Warped image I? [sent-84, score-0.745]
40 a 2D point u = [u,v]T in U to a 3D point x on the mesh M. [sent-91, score-0.721]
41 Specifically, our method finds the projected mesh faMce that encloses pixel u in the U coordinates, determines 3D position cxl? [sent-105, score-0.745]
42 During image warping, only visible mesh faces are considered and z-buffering is used to find which faces are visible to the camera. [sent-113, score-0.721]
43 Unlike single-view photometric stereo, in our case, we have more observations from different nearby viewpoints that are reasonably well aligned using the base mesh geometry. [sent-117, score-1.221]
44 An example of surface normal map and displacement map estimation. [sent-119, score-0.436]
45 (b) Initial normal map obtained from the base mesh, in U. [sent-121, score-0.265]
46 Therefore, the parameterization allows images from multiple viewpoints to be used effectively for multiview photometric stereo. [sent-127, score-0.651]
47 In this section, we introduce our method for estimating surface normals given warped images I? [sent-128, score-0.248]
48 Here, = N ∈ Rp×3 is an albedo-scaled surface normal matrix, and × LN ∈ ∈ R R3×q represents a lighting matrix. [sent-132, score-0.252]
49 Unlike the singlevLie ∈w photometric stereo case, I many missing elements has as most 3D points are not visible from all the viewpoints. [sent-133, score-0.52]
50 Therefore, we compute surface normals N using subsets of the observations which form dense block matrices in I. [sent-134, score-0.196]
51 Next, given an observation matrix IS, we apply the uncalibrated photometric stereo method of Hayakawa [4] to each IS. [sent-139, score-0.553]
52 To automatically resolve the linear 11 116633 ambiguity A, we use the mesh normals Nf ∈ Rp×3 obtained from the base mesh, which is coarse yet reasonably close to the correct surface normal. [sent-142, score-1.063]
53 NSA−1 ≈ Nf = Using the pseudo-inverse of Nf, we s oNlve for A and obtain the surface normal estimate as NˆS ? [sent-144, score-0.252]
54 NˆAS =← (UN3ΣfTN321Af)−−11,NfTU3Σ321, (2) NˆS where is a disambiguated surface normal matrix for subset S. [sent-145, score-0.277]
55 nfTnS (= wS) is a weighting factor, which is the cosine of the angle between the face normal and estimated normal vectors, and M = ΣwS normalizes the weighted sum of nS. [sent-148, score-0.262]
56 Figure 2 shows an example of the computed surface normal maps. [sent-150, score-0.252]
57 Geometry Refinement The major advantage of working in the 2D parameter domain is that 3D mesh refinement can be performed by estimating a 2D displacement map of the base mesh M. [sent-153, score-1.911]
58 as finding the optimal displacement d ∈ R per pixel u as x∗ (u) = x(u) + d(u)nf (u) , (4) where nf is a unit face normal of the triangle in M to which x is mapped, nanitd f axce∗ niso rtmhea l r oeffi tnheed t 3iaDn position. [sent-155, score-0.363]
59 Now, given photometric normals np obtained via photometric stereo and the initial position x ∈ M, we evsitaim phaoteto mthee displacement d ˆth by minimizing nth xe following energy function: dˆ = argdmin? [sent-157, score-1.077]
60 (5) is a data term that encourages the surface gradient at x∗ to be orthogonal to the orientation of photometric normal np. [sent-163, score-0.591]
61 However, we estimate only a single displacement for each 3D point, optimizing a single scalar instead of three coordinates thereby reducing mesh refinement to estimating the optimal displacement map. [sent-166, score-1.155]
62 This operation is important as it prevents seams from occurring on the chart boundaries by encouraging points across seams to have similar displacement values. [sent-176, score-0.217]
63 e sFoolru example, × a base mesh with as few as 2K vertices with a 512 512 displacement map can generate K26 v2eKrti ceeffsewctiitvhea av5er1t2ic×es51. [sent-186, score-1.031]
64 S2idnicseour approach directly estimates a displacement map on a coarse mesh, our 3D models can be efficiently stored and rendered efficiently on modern graphics hardware that supports displacement mapping [21]. [sent-187, score-0.404]
65 Each cell of the table shows accuracy ( × 10−3) and completeness (%) for two experiments, mesh perturbation wanitdh m theesh g rroeusonldut tiorunt (see tacexht fcoelrl more details). [sent-201, score-0.86]
66 Using the visibility of the SfM point cloud, we estimate a depth range for each viewpoint and then perform plane-sweep stereo matching for each viewpoint using two other images captured from adjacent viewpoints under identical lighting. [sent-205, score-0.397]
67 Sub-pixel refinement is then performed on these depth maps using a standard local parabolic refinement of the aggregated matching costs. [sent-207, score-0.44]
68 The unary potentials are computed using free space occupancy of the 3D points in the depth map [5], where the contributions from depth maps are weighted by their confidences. [sent-215, score-0.221]
69 Finally, from the labeled grid, we recover a triangulated mesh M using marching dcu gbreids, [ w13e]. [sent-219, score-0.791]
70 r c Wovee prefer nMguVlaSt eind computing our base mesh over a visual-hull based approach [6], since MVS yields more accurate mesh in our experience, especially for objects with large concavities or complex topologies. [sent-220, score-1.544]
71 Results We first quantitatively evaluate our method using synthetic datasets and compare our method with existing stateof-the-art approaches [15, 6] focusing on the performance of our mesh refinement algorithm. [sent-222, score-0.913]
72 In this evaluation, we used synthetically rendered images and the original mesh as the preliminary mesh. [sent-223, score-0.721]
73 To simulate errors and irregularities of real data, these meshes are corrupted by adding noise, and vertices are sub-sampled to produce meshes with smaller triangle counts. [sent-231, score-0.232]
74 In this test, we perturb the original mesh by adding random vertex displacements as noise and then use the Taubin operator [22] to apply mesh smoothing. [sent-233, score-1.471]
75 In this test, we vary the number of faces of the base mesh to analyze the effect of mesh resolutions. [sent-236, score-1.544]
76 Using a mesh simplification technique [8], we generate meshes with 25K, 50K, and 70K vertices. [sent-237, score-0.816]
77 Given the ground truth mesh G, we measure othne accuracy Goifv ethne rheefi gnerodu mnde tshru tRh by computing tshuer accuracy arnacdy yco omfp thleete rnefeisnse mde mtreicssh th Rat b are uosmepdu itnthe Middlebury multiview stereo benchmark [18]. [sent-239, score-1.01]
78 Accuracy refers to the distance d ∈ distR→G such that x% of the points are within distance 11 116655 Bunny - perturbation level 3 Gargoyle - perturbation level 3 %ta)(iRo146208 0 24HONeurhnsab6ndezR)%(oita1086420 0 246ONHuerhsn8abndez10 Accuracy (? [sent-241, score-0.22]
79 The graph corresponds to mesh perturbation experiment in Table 2 when perturbation level is 3. [sent-246, score-0.941]
80 The result corresponds to mesh perturbation experi- ment in Table 2 where perturbation level is 3. [sent-251, score-0.941]
81 For the mesh perturbation experiment, our method consistently performs better than Nehab et al. [sent-258, score-0.831]
82 [15] because our method naturally avoids mesh flipping and overlapping triangles. [sent-259, score-0.745]
83 As our approach estimates a displacement map whose resolution is derived from the original image resolution, our method recovers fine geometric details regardless of the resolution of the base mesh (see mesh resolution in Table 2). [sent-262, score-1.939]
84 We compare the computational cost of our mesh refinement method with that of Nehab et al. [sent-270, score-0.892]
85 Base mesh resolutions, reconstruction accuracies, and computation times are shown under sub-figures. [sent-285, score-0.758]
86 Our method does not require tuning the mesh resolution because it is automatically determined by the input image resolution. [sent-286, score-0.777]
87 Even though our normal estimation method assumes Lambertian reflectances, the normal aggregation process of Eq. [sent-298, score-0.262]
88 Since no valid normal could be estimated from any of the viewpoints, our method is unable to refine the coarse mesh in this region. [sent-307, score-0.896]
89 Discussions Our 3D reconstruction approach enables the acquisition of high-fidelity 3D models where a mesh parameterization scheme is used to fuse photometric and geometric cues. [sent-309, score-1.327]
90 First, we have used a linear photometric stereo approach for efficiency reasons, but the accuracy of our system can be potentially boosted using recent advances in robust photometric stereo [10]. [sent-311, score-1.04]
91 Dark and textureless surfaces are currently difficult to handle in our method due to the lack of reliable photometric or geometric cues. [sent-312, score-0.446]
92 In the future, we plan to explore a joint optimization approach that simultaneously estimates surface normals and scene depth for greater accuracy and robustness. [sent-313, score-0.267]
93 Photometric stereo under a light source with arbitrary motion. [sent-344, score-0.206]
94 Optimal integration of photometric and geometric surface measurements using inaccurate reflectance/illumination knowledge. [sent-436, score-0.511]
95 A comparison and evaluation of multi-view stereo reconstruction algorithms. [sent-451, score-0.218]
96 Fusing multiview and photometric stereo for 3d reconstruction under uncalibrated illumination. [sent-520, score-0.698]
97 High-quality shape from multi-view stereo and shading under general illumination. [sent-528, score-0.242]
98 Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multi-view stereo. [sent-537, score-0.339]
99 11 116677 one of input images, the base mesh from MVS, and the refined mesh. [sent-559, score-0.823]
100 The corresponding surface normal and displacement maps are shown in the supplementary material. [sent-560, score-0.399]
wordName wordTfidf (topN-words)
[('mesh', 0.721), ('photometric', 0.339), ('stereo', 0.181), ('refinement', 0.171), ('parameterization', 0.145), ('normal', 0.131), ('surface', 0.121), ('displacement', 0.12), ('perturbation', 0.11), ('multiview', 0.108), ('nehab', 0.106), ('base', 0.102), ('mvs', 0.087), ('normals', 0.075), ('meshes', 0.075), ('depth', 0.071), ('nf', 0.062), ('viewpoints', 0.059), ('gargoyle', 0.059), ('vertices', 0.056), ('resolution', 0.056), ('geometric', 0.051), ('sfm', 0.049), ('ndez', 0.048), ('korea', 0.046), ('hern', 0.045), ('hernandez', 0.045), ('coarse', 0.044), ('bunny', 0.043), ('vogiatzis', 0.041), ('albedos', 0.039), ('buddha', 0.039), ('distr', 0.039), ('nsa', 0.039), ('teapot', 0.039), ('reconstruction', 0.037), ('illuminated', 0.036), ('matsushita', 0.035), ('seams', 0.035), ('acquisition', 0.034), ('mapping', 0.034), ('parameterized', 0.034), ('shading', 0.033), ('uncalibrated', 0.033), ('lambertian', 0.033), ('barycentric', 0.032), ('vlasic', 0.032), ('distg', 0.032), ('okatani', 0.032), ('map', 0.032), ('surfaces', 0.031), ('viewpoint', 0.031), ('warping', 0.03), ('wilburn', 0.03), ('rusinkiewicz', 0.03), ('lensch', 0.03), ('completeness', 0.029), ('vertex', 0.029), ('mapped', 0.029), ('warped', 0.029), ('planar', 0.028), ('rp', 0.028), ('shape', 0.028), ('uo', 0.028), ('efficiently', 0.027), ('maps', 0.027), ('chart', 0.027), ('intricate', 0.027), ('triangle', 0.026), ('calibration', 0.025), ('light', 0.025), ('marching', 0.025), ('disambiguated', 0.025), ('textureless', 0.025), ('recovering', 0.025), ('fine', 0.024), ('pixel', 0.024), ('visibility', 0.024), ('avoids', 0.024), ('pipeline', 0.024), ('cameras', 0.023), ('energy', 0.023), ('recover', 0.023), ('faithfully', 0.023), ('silhouette', 0.023), ('estimating', 0.023), ('pattern', 0.022), ('voxels', 0.022), ('triangulated', 0.022), ('calibrate', 0.021), ('domain', 0.021), ('synthetic', 0.021), ('technique', 0.02), ('pages', 0.02), ('specular', 0.02), ('seitz', 0.02), ('rectangles', 0.02), ('unary', 0.02), ('controlled', 0.02), ('asia', 0.02)]
simIndex simValue paperId paperTitle
same-paper 1 1.0 284 iccv-2013-Multiview Photometric Stereo Using Planar Mesh Parameterization
Author: Jaesik Park, Sudipta N. Sinha, Yasuyuki Matsushita, Yu-Wing Tai, In So Kweon
Abstract: We propose a method for accurate 3D shape reconstruction using uncalibrated multiview photometric stereo. A coarse mesh reconstructed using multiview stereo is first parameterized using a planar mesh parameterization technique. Subsequently, multiview photometric stereo is performed in the 2D parameter domain of the mesh, where all geometric and photometric cues from multiple images can be treated uniformly. Unlike traditional methods, there is no need for merging view-dependent surface normal maps. Our key contribution is a new photometric stereo based mesh refinement technique that can efficiently reconstruct meshes with extremely fine geometric details by directly estimating a displacement texture map in the 2D parameter domain. We demonstrate that intricate surface geometry can be reconstructed using several challenging datasets containing surfaces with specular reflections, multiple albedos and complex topologies.
2 0.25331774 199 iccv-2013-High Quality Shape from a Single RGB-D Image under Uncalibrated Natural Illumination
Author: Yudeog Han, Joon-Young Lee, In So Kweon
Abstract: We present a novel framework to estimate detailed shape of diffuse objects with uniform albedo from a single RGB-D image. To estimate accurate lighting in natural illumination environment, we introduce a general lighting model consisting oftwo components: global and local models. The global lighting model is estimated from the RGB-D input using the low-dimensional characteristic of a diffuse reflectance model. The local lighting model represents spatially varying illumination and it is estimated by using the smoothlyvarying characteristic of illumination. With both the global and local lighting model, we can estimate complex lighting variations in uncontrolled natural illumination conditions accurately. For high quality shape capture, a shapefrom-shading approach is applied with the estimated lighting model. Since the entire process is done with a single RGB-D input, our method is capable of capturing the high quality shape details of a dynamic object under natural illumination. Experimental results demonstrate the feasibility and effectiveness of our method that dramatically improves shape details of the rough depth input.
3 0.24868801 281 iccv-2013-Multi-view Normal Field Integration for 3D Reconstruction of Mirroring Objects
Author: Michael Weinmann, Aljosa Osep, Roland Ruiters, Reinhard Klein
Abstract: In this paper, we present a novel, robust multi-view normal field integration technique for reconstructing the full 3D shape of mirroring objects. We employ a turntablebased setup with several cameras and displays. These are used to display illumination patterns which are reflected by the object surface. The pattern information observed in the cameras enables the calculation of individual volumetric normal fields for each combination of camera, display and turntable angle. As the pattern information might be blurred depending on the surface curvature or due to nonperfect mirroring surface characteristics, we locally adapt the decoding to the finest still resolvable pattern resolution. In complex real-world scenarios, the normal fields contain regions without observations due to occlusions and outliers due to interreflections and noise. Therefore, a robust reconstruction using only normal information is challenging. Via a non-parametric clustering of normal hypotheses derived for each point in the scene, we obtain both the most likely local surface normal and a local surface consistency estimate. This information is utilized in an iterative mincut based variational approach to reconstruct the surface geometry.
4 0.18201302 11 iccv-2013-A Fully Hierarchical Approach for Finding Correspondences in Non-rigid Shapes
Author: Ivan Sipiran, Benjamin Bustos
Abstract: This paper presents a hierarchical method for finding correspondences in non-rigid shapes. We propose a new representation for 3D meshes: the decomposition tree. This structure characterizes the recursive decomposition process of a mesh into regions of interest and keypoints. The internal nodes contain regions of interest (which may be recursively decomposed) and the leaf nodes contain the keypoints to be matched. We also propose a hierarchical matching algorithm that performs in a level-wise manner. The matching process is guided by the similarity between regions in high levels of the tree, until reaching the keypoints stored in the leaves. This allows us to reduce the search space of correspondences, making also the matching process efficient. We evaluate the effectiveness of our approach using the SHREC’2010 robust correspondence benchmark. In addition, we show that our results outperform the state of the art.
5 0.17054483 36 iccv-2013-Accurate and Robust 3D Facial Capture Using a Single RGBD Camera
Author: Yen-Lin Chen, Hsiang-Tao Wu, Fuhao Shi, Xin Tong, Jinxiang Chai
Abstract: This paper presents an automatic and robust approach that accurately captures high-quality 3D facial performances using a single RGBD camera. The key of our approach is to combine the power of automatic facial feature detection and image-based 3D nonrigid registration techniques for 3D facial reconstruction. In particular, we develop a robust and accurate image-based nonrigid registration algorithm that incrementally deforms a 3D template mesh model to best match observed depth image data and important facial features detected from single RGBD images. The whole process is fully automatic and robust because it is based on single frame facial registration framework. The system is flexible because it does not require any strong 3D facial priors such as blendshape models. We demonstrate the power of our approach by capturing a wide range of 3D facial expressions using a single RGBD camera and achieve state-of-the-art accuracy by comparing against alternative methods.
6 0.15342808 319 iccv-2013-Point-Based 3D Reconstruction of Thin Objects
7 0.13689658 9 iccv-2013-A Flexible Scene Representation for 3D Reconstruction Using an RGB-D Camera
8 0.13410459 382 iccv-2013-Semi-dense Visual Odometry for a Monocular Camera
9 0.13002263 90 iccv-2013-Content-Aware Rotation
10 0.1278912 343 iccv-2013-Real-World Normal Map Capture for Nearly Flat Reflective Surfaces
11 0.12474429 1 iccv-2013-3DNN: Viewpoint Invariant 3D Geometry Matching for Scene Understanding
12 0.11486274 310 iccv-2013-Partial Sum Minimization of Singular Values in RPCA for Low-Level Vision
13 0.11391178 389 iccv-2013-Shortest Paths with Curvature and Torsion
14 0.11234897 317 iccv-2013-Piecewise Rigid Scene Flow
15 0.10908811 304 iccv-2013-PM-Huber: PatchMatch with Huber Regularization for Stereo Matching
16 0.10637377 219 iccv-2013-Internet Based Morphable Model
17 0.10423439 128 iccv-2013-Dynamic Probabilistic Volumetric Models
18 0.098683335 12 iccv-2013-A General Dense Image Matching Framework Combining Direct and Feature-Based Costs
19 0.094222866 111 iccv-2013-Detecting Dynamic Objects with Multi-view Background Subtraction
20 0.091457061 387 iccv-2013-Shape Anchors for Data-Driven Multi-view Reconstruction
topicId topicWeight
[(0, 0.168), (1, -0.217), (2, -0.065), (3, 0.021), (4, -0.016), (5, -0.012), (6, 0.052), (7, -0.169), (8, -0.012), (9, -0.037), (10, -0.032), (11, 0.06), (12, -0.022), (13, 0.079), (14, 0.063), (15, -0.096), (16, -0.024), (17, -0.0), (18, -0.002), (19, -0.022), (20, 0.021), (21, 0.113), (22, 0.012), (23, -0.028), (24, -0.054), (25, 0.04), (26, -0.067), (27, 0.045), (28, 0.066), (29, -0.01), (30, 0.051), (31, -0.03), (32, -0.002), (33, -0.082), (34, 0.077), (35, -0.07), (36, 0.031), (37, 0.089), (38, -0.033), (39, -0.043), (40, 0.002), (41, 0.061), (42, -0.009), (43, -0.008), (44, -0.016), (45, 0.041), (46, -0.046), (47, -0.106), (48, 0.052), (49, -0.014)]
simIndex simValue paperId paperTitle
same-paper 1 0.96417433 284 iccv-2013-Multiview Photometric Stereo Using Planar Mesh Parameterization
Author: Jaesik Park, Sudipta N. Sinha, Yasuyuki Matsushita, Yu-Wing Tai, In So Kweon
Abstract: We propose a method for accurate 3D shape reconstruction using uncalibrated multiview photometric stereo. A coarse mesh reconstructed using multiview stereo is first parameterized using a planar mesh parameterization technique. Subsequently, multiview photometric stereo is performed in the 2D parameter domain of the mesh, where all geometric and photometric cues from multiple images can be treated uniformly. Unlike traditional methods, there is no need for merging view-dependent surface normal maps. Our key contribution is a new photometric stereo based mesh refinement technique that can efficiently reconstruct meshes with extremely fine geometric details by directly estimating a displacement texture map in the 2D parameter domain. We demonstrate that intricate surface geometry can be reconstructed using several challenging datasets containing surfaces with specular reflections, multiple albedos and complex topologies.
2 0.83031368 281 iccv-2013-Multi-view Normal Field Integration for 3D Reconstruction of Mirroring Objects
Author: Michael Weinmann, Aljosa Osep, Roland Ruiters, Reinhard Klein
Abstract: In this paper, we present a novel, robust multi-view normal field integration technique for reconstructing the full 3D shape of mirroring objects. We employ a turntablebased setup with several cameras and displays. These are used to display illumination patterns which are reflected by the object surface. The pattern information observed in the cameras enables the calculation of individual volumetric normal fields for each combination of camera, display and turntable angle. As the pattern information might be blurred depending on the surface curvature or due to nonperfect mirroring surface characteristics, we locally adapt the decoding to the finest still resolvable pattern resolution. In complex real-world scenarios, the normal fields contain regions without observations due to occlusions and outliers due to interreflections and noise. Therefore, a robust reconstruction using only normal information is challenging. Via a non-parametric clustering of normal hypotheses derived for each point in the scene, we obtain both the most likely local surface normal and a local surface consistency estimate. This information is utilized in an iterative mincut based variational approach to reconstruct the surface geometry.
3 0.78192645 199 iccv-2013-High Quality Shape from a Single RGB-D Image under Uncalibrated Natural Illumination
Author: Yudeog Han, Joon-Young Lee, In So Kweon
Abstract: We present a novel framework to estimate detailed shape of diffuse objects with uniform albedo from a single RGB-D image. To estimate accurate lighting in natural illumination environment, we introduce a general lighting model consisting oftwo components: global and local models. The global lighting model is estimated from the RGB-D input using the low-dimensional characteristic of a diffuse reflectance model. The local lighting model represents spatially varying illumination and it is estimated by using the smoothlyvarying characteristic of illumination. With both the global and local lighting model, we can estimate complex lighting variations in uncontrolled natural illumination conditions accurately. For high quality shape capture, a shapefrom-shading approach is applied with the estimated lighting model. Since the entire process is done with a single RGB-D input, our method is capable of capturing the high quality shape details of a dynamic object under natural illumination. Experimental results demonstrate the feasibility and effectiveness of our method that dramatically improves shape details of the rough depth input.
4 0.754646 319 iccv-2013-Point-Based 3D Reconstruction of Thin Objects
Author: Benjamin Ummenhofer, Thomas Brox
Abstract: 3D reconstruction deals with the problem of finding the shape of an object from a set of images. Thin objects that have virtually no volumepose a special challengefor reconstruction with respect to shape representation and fusion of depth information. In this paper we present a dense pointbased reconstruction method that can deal with this special class of objects. We seek to jointly optimize a set of depth maps by treating each pixel as a point in space. Points are pulled towards a common surface by pairwise forces in an iterative scheme. The method also handles the problem of opposed surfaces by means of penalty forces. Efficient optimization is achieved by grouping points to superpixels and a spatial hashing approach for fast neighborhood queries. We show that the approach is on a par with state-of-the-art methods for standard multi view stereo settings and gives superior results for thin objects.
5 0.66451246 16 iccv-2013-A Generic Deformation Model for Dense Non-rigid Surface Registration: A Higher-Order MRF-Based Approach
Author: Yun Zeng, Chaohui Wang, Xianfeng Gu, Dimitris Samaras, Nikos Paragios
Abstract: We propose a novel approach for dense non-rigid 3D surface registration, which brings together Riemannian geometry and graphical models. To this end, we first introduce a generic deformation model, called Canonical Distortion Coefficients (CDCs), by characterizing the deformation of every point on a surface using the distortions along its two principle directions. This model subsumes the deformation groups commonly used in surface registration such as isometry and conformality, and is able to handle more complex deformations. We also derive its discrete counterpart which can be computed very efficiently in a closed form. Based on these, we introduce a higher-order Markov Random Field (MRF) model which seamlessly integrates our deformation model and a geometry/texture similarity metric. Then we jointly establish the optimal correspondences for all the points via maximum a posteriori (MAP) inference. Moreover, we develop a parallel optimization algorithm to efficiently perform the inference for the proposed higher-order MRF model. The resulting registration algorithm outperforms state-of-the-art methods in both dense non-rigid 3D surface registration and tracking.
6 0.65580386 183 iccv-2013-Geometric Registration Based on Distortion Estimation
7 0.64933103 56 iccv-2013-Automatic Registration of RGB-D Scans via Salient Directions
8 0.64221698 139 iccv-2013-Elastic Fragments for Dense Scene Reconstruction
9 0.64021772 343 iccv-2013-Real-World Normal Map Capture for Nearly Flat Reflective Surfaces
10 0.62292123 228 iccv-2013-Large-Scale Multi-resolution Surface Reconstruction from RGB-D Sequences
11 0.60732961 262 iccv-2013-Matching Dry to Wet Materials
12 0.58000541 30 iccv-2013-A Simple Model for Intrinsic Image Decomposition with Depth Cues
13 0.57889342 9 iccv-2013-A Flexible Scene Representation for 3D Reconstruction Using an RGB-D Camera
14 0.56898469 304 iccv-2013-PM-Huber: PatchMatch with Huber Regularization for Stereo Matching
15 0.5439198 102 iccv-2013-Data-Driven 3D Primitives for Single Image Understanding
16 0.53763843 100 iccv-2013-Curvature-Aware Regularization on Riemannian Submanifolds
17 0.5276069 255 iccv-2013-Local Signal Equalization for Correspondence Matching
18 0.52742475 407 iccv-2013-Subpixel Scanning Invariant to Indirect Lighting Using Quadratic Code Length
19 0.52025503 28 iccv-2013-A Rotational Stereo Model Based on XSlit Imaging
20 0.51700693 382 iccv-2013-Semi-dense Visual Odometry for a Monocular Camera
topicId topicWeight
[(2, 0.053), (7, 0.013), (13, 0.011), (26, 0.086), (27, 0.011), (31, 0.039), (35, 0.015), (40, 0.013), (42, 0.098), (44, 0.012), (48, 0.021), (64, 0.05), (73, 0.036), (80, 0.174), (89, 0.236), (95, 0.011), (98, 0.026)]
simIndex simValue paperId paperTitle
1 0.90216386 171 iccv-2013-Fix Structured Learning of 2013 ICCV paper k2opt.pdf
Author: empty-author
Abstract: Submodular functions can be exactly minimized in polynomial time, and the special case that graph cuts solve with max flow [19] has had significant impact in computer vision [5, 21, 28]. In this paper we address the important class of sum-of-submodular (SoS) functions [2, 18], which can be efficiently minimized via a variant of max flow called submodular flow [6]. SoS functions can naturally express higher order priors involving, e.g., local image patches; however, it is difficult to fully exploit their expressive power because they have so many parameters. Rather than trying to formulate existing higher order priors as an SoS function, we take a discriminative learning approach, effectively searching the space of SoS functions for a higher order prior that performs well on our training set. We adopt a structural SVM approach [15, 34] and formulate the training problem in terms of quadratic programming; as a result we can efficiently search the space of SoS priors via an extended cutting-plane algorithm. We also show how the state-of-the-art max flow method for vision problems [11] can be modified to efficiently solve the submodular flow problem. Experimental comparisons are made against the OpenCVimplementation ofthe GrabCut interactive seg- mentation technique [28], which uses hand-tuned parameters instead of machine learning. On a standard dataset [12] our method learns higher order priors with hundreds of parameter values, and produces significantly better segmentations. While our focus is on binary labeling problems, we show that our techniques can be naturally generalized to handle more than two labels.
same-paper 2 0.89584684 284 iccv-2013-Multiview Photometric Stereo Using Planar Mesh Parameterization
Author: Jaesik Park, Sudipta N. Sinha, Yasuyuki Matsushita, Yu-Wing Tai, In So Kweon
Abstract: We propose a method for accurate 3D shape reconstruction using uncalibrated multiview photometric stereo. A coarse mesh reconstructed using multiview stereo is first parameterized using a planar mesh parameterization technique. Subsequently, multiview photometric stereo is performed in the 2D parameter domain of the mesh, where all geometric and photometric cues from multiple images can be treated uniformly. Unlike traditional methods, there is no need for merging view-dependent surface normal maps. Our key contribution is a new photometric stereo based mesh refinement technique that can efficiently reconstruct meshes with extremely fine geometric details by directly estimating a displacement texture map in the 2D parameter domain. We demonstrate that intricate surface geometry can be reconstructed using several challenging datasets containing surfaces with specular reflections, multiple albedos and complex topologies.
3 0.88671207 127 iccv-2013-Dynamic Pooling for Complex Event Recognition
Author: Weixin Li, Qian Yu, Ajay Divakaran, Nuno Vasconcelos
Abstract: The problem of adaptively selecting pooling regions for the classification of complex video events is considered. Complex events are defined as events composed of several characteristic behaviors, whose temporal configuration can change from sequence to sequence. A dynamic pooling operator is defined so as to enable a unified solution to the problems of event specific video segmentation, temporal structure modeling, and event detection. Video is decomposed into segments, and the segments most informative for detecting a given event are identified, so as to dynamically determine the pooling operator most suited for each sequence. This dynamic pooling is implemented by treating the locations of characteristic segments as hidden information, which is inferred, on a sequence-by-sequence basis, via a large-margin classification rule with latent variables. Although the feasible set of segment selections is combinatorial, it is shown that a globally optimal solution to the inference problem can be obtained efficiently, through the solution of a series of linear programs. Besides the coarselevel location of segments, a finer model of video struc- ture is implemented by jointly pooling features of segmenttuples. Experimental evaluation demonstrates that the re- sulting event detector has state-of-the-art performance on challenging video datasets.
4 0.87944508 154 iccv-2013-Face Recognition via Archetype Hull Ranking
Author: Yuanjun Xiong, Wei Liu, Deli Zhao, Xiaoou Tang
Abstract: The archetype hull model is playing an important role in large-scale data analytics and mining, but rarely applied to vision problems. In this paper, we migrate such a geometric model to address face recognition and verification together through proposing a unified archetype hull ranking framework. Upon a scalable graph characterized by a compact set of archetype exemplars whose convex hull encompasses most of the training images, the proposed framework explicitly captures the relevance between any query and the stored archetypes, yielding a rank vector over the archetype hull. The archetype hull ranking is then executed on every block of face images to generate a blockwise similarity measure that is achieved by comparing two different rank vectors with respect to the same archetype hull. After integrating blockwise similarity measurements with learned importance weights, we accomplish a sensible face similarity measure which can support robust and effective face recognition and verification. We evaluate the face similarity measure in terms of experiments performed on three benchmark face databases Multi-PIE, Pubfig83, and LFW, demonstrat- ing its performance superior to the state-of-the-arts.
5 0.84465665 196 iccv-2013-Hierarchical Data-Driven Descent for Efficient Optimal Deformation Estimation
Author: Yuandong Tian, Srinivasa G. Narasimhan
Abstract: Real-world surfaces such as clothing, water and human body deform in complex ways. The image distortions observed are high-dimensional and non-linear, making it hard to estimate these deformations accurately. The recent datadriven descent approach [17] applies Nearest Neighbor estimators iteratively on a particular distribution of training samples to obtain a globally optimal and dense deformation field between a template and a distorted image. In this work, we develop a hierarchical structure for the Nearest Neighbor estimators, each of which can have only a local image support. We demonstrate in both theory and practice that this algorithm has several advantages over the nonhierarchical version: it guarantees global optimality with significantly fewer training samples, is several orders faster, provides a metric to decide whether a given image is “hard” (or “easy ”) requiring more (or less) samples, and can handle more complex scenes that include both global motion and local deformation. The proposed algorithm successfully tracks a broad range of non-rigid scenes including water, clothing, and medical images, and compares favorably against several other deformation estimation and tracking approaches that do not provide optimality guarantees.
6 0.8410306 78 iccv-2013-Coherent Motion Segmentation in Moving Camera Videos Using Optical Flow Orientations
7 0.84089762 447 iccv-2013-Volumetric Semantic Segmentation Using Pyramid Context Features
8 0.84019238 66 iccv-2013-Building Part-Based Object Detectors via 3D Geometry
9 0.84017384 128 iccv-2013-Dynamic Probabilistic Volumetric Models
10 0.84011376 160 iccv-2013-Fast Object Segmentation in Unconstrained Video
11 0.83990264 190 iccv-2013-Handling Occlusions with Franken-Classifiers
12 0.83973044 204 iccv-2013-Human Attribute Recognition by Rich Appearance Dictionary
13 0.83930814 65 iccv-2013-Breaking the Chain: Liberation from the Temporal Markov Assumption for Tracking Human Poses
14 0.83913523 386 iccv-2013-Sequential Bayesian Model Update under Structured Scene Prior for Semantic Road Scenes Labeling
15 0.83880723 387 iccv-2013-Shape Anchors for Data-Driven Multi-view Reconstruction
16 0.83844346 379 iccv-2013-Semantic Segmentation without Annotating Segments
17 0.83821803 444 iccv-2013-Viewing Real-World Faces in 3D
18 0.83791494 281 iccv-2013-Multi-view Normal Field Integration for 3D Reconstruction of Mirroring Objects
19 0.83694279 107 iccv-2013-Deformable Part Descriptors for Fine-Grained Recognition and Attribute Prediction
20 0.83670968 433 iccv-2013-Understanding High-Level Semantics by Modeling Traffic Patterns