iccv iccv2013 iccv2013-423 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Salil Tambe, Ashok Veeraraghavan, Amit Agrawal
Abstract: Current Light Field (LF) cameras offer fixed resolution in space, time and angle which is decided a-priori and is independent of the scene. These cameras either trade-off spatial resolution to capture single-shot LF [20, 27, 12] or tradeoff temporal resolution by assuming a static scene to capture high spatial resolution LF [18, 3]. Thus, capturing high spatial resolution LF video for dynamic scenes remains an open and challenging problem. We present the concept, design and implementation of a LF video camera that allows capturing high resolution LF video. The spatial, angular and temporal resolution are not fixed a-priori and we exploit the scene-specific redundancy in space, time and angle. Our reconstruction is motion-aware and offers a continuum of resolution tradeoff with increasing motion in the scene. The key idea is (a) to design efficient multiplexing matrices that allow resolution tradeoffs, (b) use dictionary learning and sparse repre- sentations for robust reconstruction, and (c) perform local motion-aware adaptive reconstruction. We perform extensive analysis and characterize the performance of our motion-aware reconstruction algorithm. We show realistic simulations using a graphics simulator as well as real results using a LCoS based programmable camera. We demonstrate novel results such as high resolution digital refocusing for dynamic moving objects.
Reference: text
sentIndex sentText sentNum sentScore
1 edu Abstract Current Light Field (LF) cameras offer fixed resolution in space, time and angle which is decided a-priori and is independent of the scene. [sent-2, score-0.269]
2 These cameras either trade-off spatial resolution to capture single-shot LF [20, 27, 12] or tradeoff temporal resolution by assuming a static scene to capture high spatial resolution LF [18, 3]. [sent-3, score-0.967]
3 Thus, capturing high spatial resolution LF video for dynamic scenes remains an open and challenging problem. [sent-4, score-0.382]
4 We present the concept, design and implementation of a LF video camera that allows capturing high resolution LF video. [sent-5, score-0.355]
5 The spatial, angular and temporal resolution are not fixed a-priori and we exploit the scene-specific redundancy in space, time and angle. [sent-6, score-0.343]
6 Our reconstruction is motion-aware and offers a continuum of resolution tradeoff with increasing motion in the scene. [sent-7, score-0.364]
7 The key idea is (a) to design efficient multiplexing matrices that allow resolution tradeoffs, (b) use dictionary learning and sparse repre- sentations for robust reconstruction, and (c) perform local motion-aware adaptive reconstruction. [sent-8, score-0.451]
8 We demonstrate novel results such as high resolution digital refocusing for dynamic moving objects. [sent-11, score-0.476]
9 Introduction Traditionally cameras have required photographers to make trade-offs in terms of depth of field (DOF), dynamic range, shutter speed and ISO during the capture itself. [sent-13, score-0.193]
10 For example, a single-shot LF camera offers tradeoff between spatial and angular resolution and captures a low spaAmit Agrawal Mitsubishi Electric Research Labs (MERL) 201 Broadway, Cambridge, MA 02139 agrawal@merl . [sent-19, score-0.396]
11 Current light field capture designs offer fixed, a-priori, and scene independent space-time-angle resolution. [sent-21, score-0.255]
12 They are unable to cross the resolution barrier required for capturing high spatial resolution LF video. [sent-22, score-0.564]
13 LF super-resolution techniques have recently begun breaking this resolution barrier. [sent-23, score-0.265]
14 Our approach overcomes this barrier via motion-aware adaptive reconstruction using a programmable aperture camera. [sent-24, score-0.732]
15 [18] captured several multiplexed coded aperture images and demultiplexed them. [sent-34, score-0.618]
16 However, this requires the scene to be static, thereby trading off temporal resolution for high spatial resolution. [sent-35, score-0.326]
17 Thus, it is clear that there exists a resolution barrier (Figure 1) for capturing high resolution LF video. [sent-36, score-0.522]
18 0 camera [12] and LF super-resolution techniques [4] have begun breaking this resolution barrier. [sent-38, score-0.311]
19 1For the rest of the paper, high resolution LF refers spatial sensor resolution in LF reconstruction. [sent-39, score-0.52]
20 Our approach allows capturing high spatial resolution LF for dynamic scenes. [sent-45, score-0.35]
21 Our key concept is to overcome the fixed, a-priori and scene independent resolution trade-offs offered by previous cameras. [sent-47, score-0.249]
22 Conceptually, we use several coded aperture patterns (one per time frame), which would allow reconstructing a high resolution LF if the scene was static. [sent-48, score-0.813]
23 While previous approaches have used Hadamard multiplexing for designing the codes [18], we learn them using dictionary learning (DL) and sparse representations. [sent-50, score-0.25]
24 Thus, our design is a synergy between near-optimal patterns used for multiplexing and reconstruction algorithm. [sent-51, score-0.249]
25 Figures 2 and 3 show a motivating example of a dynamic scene with static grass and moving butterfly and beetle. [sent-52, score-0.3]
26 [18] can recover high spatial resolution LF but only on the static parts (grass) and show artifacts on moving objects (butterfly/beetle). [sent-55, score-0.464]
27 Our approach provides high resolution LF for both moving and static scene parts. [sent-56, score-0.394]
28 Contributions • • We present the concept, design and implementation of a LeF p veisdeenot camera aenptd, dreescoignnst aruncdti iomnp algorithm tnh oatf allows capturing high resolution LF video by analysing the spatial, temporal and angular resolution trade-offs. [sent-59, score-0.698]
29 We propose a dictionary learning and sparse representWateio pnr obpaosseed algorithm fo lera rfnuilnl gre asnodlu stpioanrs eL rFe reconstruction and show how to adapt the algorithm to object/scene motion. [sent-60, score-0.243]
30 We also show how to optimize the programmable aperture patterns using the learned dictionary. [sent-61, score-0.556]
31 Notice the low spatial resolution of Lytro, as well as artifacts on moving objects for Liang et al. [sent-85, score-0.401]
32 Note that our approach results in high resolution LF information without any artifacts. [sent-89, score-0.211]
33 Most single shot light field cameras multiplex the 4-D LF onto the 2D sensor, losing spatial resolution to capture the angular information in the LF. [sent-95, score-0.648]
34 Such cameras employ either a lenslet array close to the sensor [24, 12], a mask close to the sensor [30] or an array of lens/prism outside the main lens [13]. [sent-96, score-0.294]
35 Recently, [22] extended the mask based method of [30] to exploit sparse representations in order to recover full resolution LF. [sent-97, score-0.282]
36 Our method is similar in spirit but works to recover the loss of temporal resolution in [18]. [sent-105, score-0.246]
37 Recently, several LF super-resolution algorithms have been proposed to recover the lost resolution [12, 4]. [sent-108, score-0.211]
38 0 camera [12] recovers the lost resolution by placing the microlens array at a different location compared to the original design [24]. [sent-110, score-0.359]
39 Similarly, the Raytrix camera [27] uses a microlens array with lenses of different focal length to improve spatial resolution. [sent-111, score-0.196]
40 Thus, improving the spatial resolution of LF cam- × eras is an active area of research. [sent-112, score-0.253]
41 Programmable Aperture Imaging: Programmable aperture imaging [18] allows capturing light fields at the spatial resolution of the sensor. [sent-113, score-0.854]
42 In principle, each coded aperture can be a pin-hole placed at a different location in the aperture. [sent-114, score-0.505]
43 A set of M2 images are required to achieve an angular resolution of M M. [sent-115, score-0.308]
44 lHigohwte evffeir-, temporal resolution is sacrificed to achieve higher spatial resolution in LF. [sent-118, score-0.499]
45 Firstly, we learn a sparse basis dictionary from real LF data and use it along with the sparse reconstruction framework. [sent-122, score-0.276]
46 Secondly, unlike [3], we adapt our reconstruction algorithm to the local motion of the scene, thereby preserving both motion and disparity information. [sent-123, score-0.24]
47 Finally, we also search for near-optimal aperture codes so as to improve the reconstruction performance. [sent-124, score-0.545]
48 [1] has shown resolution tradeoffs in a single image capture. [sent-126, score-0.247]
49 [1] require moving a slit/pinhole in the aperture and a static mask close to the sensor. [sent-130, score-0.545]
50 Our design is simpler using only a dynamic coded aperture. [sent-131, score-0.23]
51 Coded Aperture: Coded aperture imaging has been widely used in astronomy [28] to overcome the limitations imposed by a pinhole camera. [sent-133, score-0.392]
52 The concept of placing a coded mask close to the sensor for LF capture was proposed by [30]. [sent-134, score-0.265]
53 Coded masks have also been used for estimating scene depth from single image [15],and for compressive LF [22] and video acquisition [21]. [sent-135, score-0.213]
54 CS has been shown useful for light transport capture [25] and even LF capture [3]. [sent-137, score-0.22]
55 However, these techniques still assume scene to be static for the duration of captured images and cannot handle moving objects. [sent-138, score-0.231]
56 Programmable Light Field Acquisition Consider the two-plane parameterizations of the lightfield LF(u, v, s, t), where (u, v) represents co-ordinates on the aperture plane and (s, t) represents co-ordinates on the sensor plane. [sent-141, score-0.418]
57 Let us assume that the aperture can be divided into M M sub-apertures. [sent-142, score-0.362]
58 The spatial resolution of the captured LF is determined by the sensor resolution, while the angular resolution is determined by the number of sub-apertures (and is equal to the number of images acquired). [sent-145, score-0.665]
59 Any motion of scene elements during the acquisition time results in significant reconstruction artifacts (see Figures 2 and 3). [sent-147, score-0.28]
60 Conceptually, we also use several coded aperture patterns (one per frame), which allows reconstructing a high resolution LF if the scene was static. [sent-151, score-0.813]
61 Firstly, we learn optimized dictionaries and coded aperture patterns that along with sparsity regularized reconstruction algorithms allow for the recovery of light fields from as few as three captured frames. [sent-153, score-0.964]
62 Our motion-aware reconstruction automatically chooses the best window length for each patch. [sent-158, score-0.193]
63 Compressive LF Sensing Consider a programmable LF camera with spatial resolution N N pixels and angular resolution M M. [sent-165, score-0.77]
64 Let ct t(iuo,n nv )N de ×n Note pthixee eclsod aendd aperture ruessedol uatt ifornam Me t ×. [sent-166, score-0.362]
65 Each captured im ×a Pge p riexsuelltss nint a alin veeacrt osert y of equations given by yt where Ct is a P2 = Ctxt, (2) P2M2 matrix that encodes the aperture × code used at time× fPrame t. [sent-176, score-0.41]
66 It is important to learn a good dictionary that can faithfully represent the light fields we intend to capture. [sent-188, score-0.259]
67 The ×× quality of a dictionary is decided by its ability to reliably reconstruct light fields with varying amounts of (a) disparity, (b) texture, and (c) occlusion relationships. [sent-189, score-0.259]
68 For learning the dictionary, we render light fields in a graphics rendering engine (Povray) with varying texture, disparity and occlusions (e. [sent-190, score-0.25]
69 As the patch size increases, the learned dictionary can better capture the disparity dependent redundancies in the LF, thereby improving the reconstruction performance. [sent-197, score-0.341]
70 (Middle) Plot showing the optimal number of frames used in motion-aware reconstruction as a function of the average patch velocity. [sent-213, score-0.238]
71 Figure 5 (left) shows the reconstruction PSNR as a function of the number of frames F (window length) used for reconstruction when the scene remains static (red plot). [sent-218, score-0.414]
72 Notice that the reconstruction performance varies as a function of both object velocity and the window length(WL) used in reconstruction. [sent-223, score-0.196]
73 We use this relationship between velocity and optimal window length in order to decide the number of frames used in reconstruction on a patch-wise basis. [sent-226, score-0.299]
74 Firstly, while selecting aperture codes, we ensure that the average disparity between adjacent aperture codes are minimized. [sent-232, score-0.841]
75 Optimizing Aperture Codes Now we discuss how to optimize aperture codes to improve the SNR. [sent-241, score-0.423]
76 Figure 6 shows the 25 coded aperture masks that were found using this approach. [sent-252, score-0.527]
77 To demonstrate that our optimized patterns indeed improve performance, we compare it with the performance of 10, 000 randomly generated aperture codes. [sent-253, score-0.422]
78 Prototype Our prototype system for motion-aware LF video capture uses a Liquid Crystal on Silica (LCoS) modulator as the spatial light modulator (SLM). [sent-258, score-0.34]
79 (c) All 25 coded aperture masks used in our approach. [sent-263, score-0.527]
80 ato Wr eis f oaltl othwe eheffe ocpttivieca aperture plane of the imaging system. [sent-268, score-0.392]
81 m7a mximm,u wme aperture s thizee cthenatt our system can support is 10 10 mm) and group it into 25 2 2 mm sized pinphoorletsi st o1 0o×bt1ai0nm m5m m×) a5n adn ggruolaurp iret sinotlout 2io5n 2. [sent-273, score-0.392]
82 fT zeros aonSd e ones, transmitting light where the LCoS pattern is one and blocking light where it is zero. [sent-275, score-0.282]
83 This enables us to capture multiplexed angular views of the light field at a very high rate by simply changing the multiplexing pattern at the LCoS. [sent-276, score-0.47]
84 Note that we capture a 25 fps video, where each captured frame is a coded aperture multiplexed image of the scene and our angular LF resolution is 5 5. [sent-280, score-0.992]
85 Figure 7 shows three captured multiplexed images at different time frames (frames 13, 88 and 330), along with the refocused images (front and back) for the entire scene. [sent-284, score-0.225]
86 Ntoo otivceer athllat m tohteiroen are no × artifacts in digital refocusing on the moving object. [sent-288, score-0.272]
87 Reconstruction of Dynamic LF Views: However, the true merit of a LF video camera is in obtaining artifact free angular information for dynamic scenes. [sent-290, score-0.234]
88 We compare our motion-aware reconstruction with another reconstruction using fixed WL of F = 25 frames for each pixel. [sent-301, score-0.313]
89 The input image and computed optical flow for frame 153 are shown along with digital refocusing (front and back) using reconstructed LF. [sent-306, score-0.218]
90 The fixed window length reconstruction utilizes the same learned dictionary and sparse reconstruction, but results in low-resolution refocusing. [sent-309, score-0.314]
91 Discussions and Conclusions We presented a novel programmable aperture light field camera that exploits highly optimized coded aperture patterns and a dictionary learning/sparse representations based framework for high resolution LF reconstruction. [sent-312, score-1.624]
92 Most LF cameras suffer from a resolution trade-off resulting in significant loss of spatial resolution. [sent-313, score-0.311]
93 Our method allows reconstruction of light-fields at the spatial resolution of the image sensor. [sent-314, score-0.375]
94 Compared to previous programmable aperture based LF methods, we achieve a much higher temporal resolution on account of the motion-aware sparse regularized reconstruction algorithm. [sent-315, score-0.926]
95 However, since our codes are 50% and because transparent polarization based LCOS modulators suffer from an additional 50% loss that DMD implementations do not suffer from, we end up with × more than 75% light loss. [sent-316, score-0.202]
96 (Top) Three frames of the captured video using our setup shows the orange EXPO marker moving from right to left. [sent-376, score-0.266]
97 A zoomed in view of moving object clearly demonstrates that our motion-aware reconstruction successfully removes artifacts on the moving object. [sent-406, score-0.352]
98 Reinterpretable imager: Towards variable post-capture space, angle and time resolution in photography. [sent-414, score-0.211]
99 [18] and fixed window length reconstruction with our motion-aware reconstruction. [sent-530, score-0.193]
100 Dappled photography: Mask enhanced cameras for heterodyned light fields and coded aperture refocusing. [sent-698, score-0.734]
wordName wordTfidf (topN-words)
[('lf', 0.692), ('aperture', 0.362), ('resolution', 0.211), ('programmable', 0.163), ('coded', 0.143), ('light', 0.141), ('lcos', 0.123), ('reconstruction', 0.122), ('compressive', 0.098), ('angular', 0.097), ('dictionary', 0.088), ('moving', 0.082), ('refocusing', 0.082), ('psnr', 0.078), ('frames', 0.069), ('lytro', 0.068), ('multiplexing', 0.068), ('artifacts', 0.066), ('multiplexed', 0.065), ('static', 0.063), ('barrier', 0.062), ('codes', 0.061), ('dynamic', 0.059), ('cameras', 0.058), ('butterfly', 0.058), ('sensor', 0.056), ('disparity', 0.056), ('agrawal', 0.056), ('sensing', 0.055), ('photography', 0.055), ('liang', 0.055), ('plenoptic', 0.054), ('field', 0.048), ('captured', 0.048), ('patch', 0.047), ('camera', 0.046), ('wl', 0.043), ('refocused', 0.043), ('array', 0.043), ('spatial', 0.042), ('digital', 0.042), ('horowitz', 0.041), ('capturing', 0.038), ('scene', 0.038), ('mask', 0.038), ('optical', 0.038), ('window', 0.037), ('levoy', 0.037), ('velocity', 0.037), ('front', 0.037), ('tradeoffs', 0.036), ('imager', 0.035), ('modulator', 0.035), ('profusion', 0.035), ('reinterpretable', 0.035), ('slm', 0.035), ('marker', 0.035), ('temporal', 0.035), ('length', 0.034), ('sparse', 0.033), ('reconstructed', 0.033), ('veeraraghavan', 0.032), ('video', 0.032), ('motion', 0.031), ('dictionaries', 0.031), ('begun', 0.031), ('microlens', 0.031), ('georgiev', 0.031), ('patterns', 0.031), ('fields', 0.03), ('imaging', 0.03), ('mm', 0.03), ('optimized', 0.029), ('adaption', 0.029), ('vaish', 0.029), ('babacan', 0.029), ('raytrix', 0.029), ('merl', 0.029), ('rice', 0.029), ('notice', 0.029), ('xt', 0.029), ('design', 0.028), ('reconstructing', 0.028), ('capture', 0.028), ('recovery', 0.027), ('expo', 0.027), ('prototype', 0.027), ('parallax', 0.026), ('compressed', 0.024), ('db', 0.024), ('flow', 0.023), ('enables', 0.023), ('breaking', 0.023), ('transport', 0.023), ('adaptive', 0.023), ('acquisition', 0.023), ('graphics', 0.023), ('shot', 0.023), ('isometry', 0.023), ('masks', 0.022)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000006 423 iccv-2013-Towards Motion Aware Light Field Video for Dynamic Scenes
Author: Salil Tambe, Ashok Veeraraghavan, Amit Agrawal
Abstract: Current Light Field (LF) cameras offer fixed resolution in space, time and angle which is decided a-priori and is independent of the scene. These cameras either trade-off spatial resolution to capture single-shot LF [20, 27, 12] or tradeoff temporal resolution by assuming a static scene to capture high spatial resolution LF [18, 3]. Thus, capturing high spatial resolution LF video for dynamic scenes remains an open and challenging problem. We present the concept, design and implementation of a LF video camera that allows capturing high resolution LF video. The spatial, angular and temporal resolution are not fixed a-priori and we exploit the scene-specific redundancy in space, time and angle. Our reconstruction is motion-aware and offers a continuum of resolution tradeoff with increasing motion in the scene. The key idea is (a) to design efficient multiplexing matrices that allow resolution tradeoffs, (b) use dictionary learning and sparse repre- sentations for robust reconstruction, and (c) perform local motion-aware adaptive reconstruction. We perform extensive analysis and characterize the performance of our motion-aware reconstruction algorithm. We show realistic simulations using a graphics simulator as well as real results using a LCoS based programmable camera. We demonstrate novel results such as high resolution digital refocusing for dynamic moving objects.
2 0.45869982 252 iccv-2013-Line Assisted Light Field Triangulation and Stereo Matching
Author: Zhan Yu, Xinqing Guo, Haibing Lin, Andrew Lumsdaine, Jingyi Yu
Abstract: Light fields are image-based representations that use densely sampled rays as a scene description. In this paper, we explore geometric structures of 3D lines in ray space for improving light field triangulation and stereo matching. The triangulation problem aims to fill in the ray space with continuous and non-overlapping simplices anchored at sampled points (rays). Such a triangulation provides a piecewise-linear interpolant useful for light field superresolution. We show that the light field space is largely bilinear due to 3D line segments in the scene, and direct triangulation of these bilinear subspaces leads to large errors. We instead present a simple but effective algorithm to first map bilinear subspaces to line constraints and then apply Constrained Delaunay Triangulation (CDT). Based on our analysis, we further develop a novel line-assisted graphcut (LAGC) algorithm that effectively encodes 3D line constraints into light field stereo matching. Experiments on synthetic and real data show that both our triangulation and LAGC algorithms outperform state-of-the-art solutions in accuracy and visual quality.
3 0.16735449 271 iccv-2013-Modeling the Calibration Pipeline of the Lytro Camera for High Quality Light-Field Image Reconstruction
Author: Donghyeon Cho, Minhaeng Lee, Sunyeong Kim, Yu-Wing Tai
Abstract: Light-field imaging systems have got much attention recently as the next generation camera model. A light-field imaging system consists of three parts: data acquisition, manipulation, and application. Given an acquisition system, it is important to understand how a light-field camera converts from its raw image to its resulting refocused image. In this paper, using the Lytro camera as an example, we describe step-by-step procedures to calibrate a raw light-field image. In particular, we are interested in knowing the spatial and angular coordinates of the micro lens array and the resampling process for image reconstruction. Since Lytro uses a hexagonal arrangement of a micro lens image, additional treatments in calibration are required. After calibration, we analyze and compare the performances of several resampling methods for image reconstruction with and without calibration. Finally, a learning based interpolation method is proposed which demonstrates a higher quality image reconstruction than previous interpolation methods including a method used in Lytro software.
4 0.12094256 82 iccv-2013-Compensating for Motion during Direct-Global Separation
Author: Supreeth Achar, Stephen T. Nuske, Srinivasa G. Narasimhan
Abstract: Separating the direct and global components of radiance can aid shape recovery algorithms and can provide useful information about materials in a scene. Practical methods for finding the direct and global components use multiple images captured under varying illumination patterns and require the scene, light source and camera to remain stationary during the image acquisition process. In this paper, we develop a motion compensation method that relaxes this condition and allows direct-global separation to beperformed on video sequences of dynamic scenes captured by moving projector-camera systems. Key to our method is being able to register frames in a video sequence to each other in the presence of time varying, high frequency active illumination patterns. We compare our motion compensated method to alternatives such as single shot separation and frame interleaving as well as ground truth. We present results on challenging video sequences that include various types of motions and deformations in scenes that contain complex materials like fabric, skin, leaves and wax.
5 0.11168343 173 iccv-2013-Fluttering Pattern Generation Using Modified Legendre Sequence for Coded Exposure Imaging
Author: Hae-Gon Jeon, Joon-Young Lee, Yudeog Han, Seon Joo Kim, In So Kweon
Abstract: Finding a good binary sequence is critical in determining theperformance ofthe coded exposure imaging, butprevious methods mostly rely on a random search for finding the binary codes, which could easily fail to find good long sequences due to the exponentially growing search space. In this paper, we present a new computationally efficient algorithm for generating the binary sequence, which is especially well suited for longer sequences. We show that the concept of the low autocorrelation binary sequence that has been well exploited in the information theory community can be applied for generating the fluttering patterns of the shutter, propose a new measure of a good binary sequence, and present a new algorithm by modifying the Legendre sequence for the coded exposure imaging. Experiments using both synthetic and real data show that our new algorithm consistently generates better binary sequencesfor the coded exposure problem, yielding better deblurring and resolution enhancement results compared to the previous methods for generating the binary codes.
6 0.10357096 108 iccv-2013-Depth from Combining Defocus and Correspondence Using Light-Field Cameras
7 0.10194284 384 iccv-2013-Semi-supervised Robust Dictionary Learning via Efficient l-Norms Minimization
8 0.10090867 161 iccv-2013-Fast Sparsity-Based Orthogonal Dictionary Learning for Image Restoration
9 0.091888025 51 iccv-2013-Anchored Neighborhood Regression for Fast Example-Based Super-Resolution
10 0.090461552 281 iccv-2013-Multi-view Normal Field Integration for 3D Reconstruction of Mirroring Objects
11 0.089294754 405 iccv-2013-Structured Light in Sunlight
12 0.086847953 209 iccv-2013-Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation
13 0.082251109 397 iccv-2013-Space-Time Tradeoffs in Photo Sequencing
14 0.077101924 276 iccv-2013-Multi-attributed Dictionary Learning for Sparse Coding
15 0.076084048 244 iccv-2013-Learning View-Invariant Sparse Representations for Cross-View Action Recognition
16 0.075211205 317 iccv-2013-Piecewise Rigid Scene Flow
17 0.070932403 280 iccv-2013-Multi-view 3D Reconstruction from Uncalibrated Radially-Symmetric Cameras
18 0.069873571 322 iccv-2013-Pose Estimation and Segmentation of People in 3D Movies
19 0.067112498 354 iccv-2013-Robust Dictionary Learning by Error Source Decomposition
20 0.067038082 128 iccv-2013-Dynamic Probabilistic Volumetric Models
topicId topicWeight
[(0, 0.152), (1, -0.089), (2, -0.039), (3, 0.062), (4, -0.114), (5, -0.025), (6, -0.038), (7, -0.136), (8, -0.016), (9, 0.001), (10, -0.011), (11, -0.019), (12, 0.072), (13, 0.008), (14, -0.073), (15, -0.05), (16, -0.081), (17, 0.021), (18, -0.014), (19, 0.073), (20, 0.036), (21, -0.026), (22, -0.105), (23, 0.019), (24, -0.062), (25, 0.005), (26, -0.095), (27, -0.113), (28, 0.044), (29, -0.001), (30, 0.04), (31, 0.086), (32, 0.059), (33, -0.081), (34, 0.055), (35, 0.136), (36, -0.114), (37, -0.058), (38, 0.0), (39, 0.094), (40, 0.115), (41, 0.073), (42, 0.01), (43, -0.026), (44, 0.073), (45, -0.098), (46, 0.06), (47, 0.182), (48, 0.032), (49, -0.102)]
simIndex simValue paperId paperTitle
same-paper 1 0.94338244 423 iccv-2013-Towards Motion Aware Light Field Video for Dynamic Scenes
Author: Salil Tambe, Ashok Veeraraghavan, Amit Agrawal
Abstract: Current Light Field (LF) cameras offer fixed resolution in space, time and angle which is decided a-priori and is independent of the scene. These cameras either trade-off spatial resolution to capture single-shot LF [20, 27, 12] or tradeoff temporal resolution by assuming a static scene to capture high spatial resolution LF [18, 3]. Thus, capturing high spatial resolution LF video for dynamic scenes remains an open and challenging problem. We present the concept, design and implementation of a LF video camera that allows capturing high resolution LF video. The spatial, angular and temporal resolution are not fixed a-priori and we exploit the scene-specific redundancy in space, time and angle. Our reconstruction is motion-aware and offers a continuum of resolution tradeoff with increasing motion in the scene. The key idea is (a) to design efficient multiplexing matrices that allow resolution tradeoffs, (b) use dictionary learning and sparse repre- sentations for robust reconstruction, and (c) perform local motion-aware adaptive reconstruction. We perform extensive analysis and characterize the performance of our motion-aware reconstruction algorithm. We show realistic simulations using a graphics simulator as well as real results using a LCoS based programmable camera. We demonstrate novel results such as high resolution digital refocusing for dynamic moving objects.
2 0.82041264 271 iccv-2013-Modeling the Calibration Pipeline of the Lytro Camera for High Quality Light-Field Image Reconstruction
Author: Donghyeon Cho, Minhaeng Lee, Sunyeong Kim, Yu-Wing Tai
Abstract: Light-field imaging systems have got much attention recently as the next generation camera model. A light-field imaging system consists of three parts: data acquisition, manipulation, and application. Given an acquisition system, it is important to understand how a light-field camera converts from its raw image to its resulting refocused image. In this paper, using the Lytro camera as an example, we describe step-by-step procedures to calibrate a raw light-field image. In particular, we are interested in knowing the spatial and angular coordinates of the micro lens array and the resampling process for image reconstruction. Since Lytro uses a hexagonal arrangement of a micro lens image, additional treatments in calibration are required. After calibration, we analyze and compare the performances of several resampling methods for image reconstruction with and without calibration. Finally, a learning based interpolation method is proposed which demonstrates a higher quality image reconstruction than previous interpolation methods including a method used in Lytro software.
3 0.77488542 252 iccv-2013-Line Assisted Light Field Triangulation and Stereo Matching
Author: Zhan Yu, Xinqing Guo, Haibing Lin, Andrew Lumsdaine, Jingyi Yu
Abstract: Light fields are image-based representations that use densely sampled rays as a scene description. In this paper, we explore geometric structures of 3D lines in ray space for improving light field triangulation and stereo matching. The triangulation problem aims to fill in the ray space with continuous and non-overlapping simplices anchored at sampled points (rays). Such a triangulation provides a piecewise-linear interpolant useful for light field superresolution. We show that the light field space is largely bilinear due to 3D line segments in the scene, and direct triangulation of these bilinear subspaces leads to large errors. We instead present a simple but effective algorithm to first map bilinear subspaces to line constraints and then apply Constrained Delaunay Triangulation (CDT). Based on our analysis, we further develop a novel line-assisted graphcut (LAGC) algorithm that effectively encodes 3D line constraints into light field stereo matching. Experiments on synthetic and real data show that both our triangulation and LAGC algorithms outperform state-of-the-art solutions in accuracy and visual quality.
4 0.66629243 405 iccv-2013-Structured Light in Sunlight
Author: Mohit Gupta, Qi Yin, Shree K. Nayar
Abstract: Strong ambient illumination severely degrades the performance of structured light based techniques. This is especially true in outdoor scenarios, where the structured light sources have to compete with sunlight, whose power is often 2-5 orders of magnitude larger than the projected light. In this paper, we propose the concept of light-concentration to overcome strong ambient illumination. Our key observation is that given a fixed light (power) budget, it is always better to allocate it sequentially in several portions of the scene, as compared to spreading it over the entire scene at once. For a desired level of accuracy, we show that by distributing light appropriately, the proposed approach requires 1-2 orders lower acquisition time than existing approaches. Our approach is illumination-adaptive as the optimal light distribution is determined based on a measurement of the ambient illumination level. Since current light sources have a fixed light distribution, we have built a prototype light source that supports flexible light distribution by controlling the scanning speed of a laser scanner. We show several high quality 3D scanning results in a wide range of outdoor scenarios. The proposed approach will benefit 3D vision systems that need to operate outdoors under extreme ambient illumination levels on a limited time and power budget.
5 0.57422936 407 iccv-2013-Subpixel Scanning Invariant to Indirect Lighting Using Quadratic Code Length
Author: Nicolas Martin, Vincent Couture, Sébastien Roy
Abstract: We present a scanning method that recovers dense subpixel camera-projector correspondence without requiring any photometric calibration nor preliminary knowledge of their relative geometry. Subpixel accuracy is achieved by considering several zero-crossings defined by the difference between pairs of unstructured patterns. We use gray-level band-pass white noise patterns that increase robustness to indirect lighting and scene discontinuities. Simulated and experimental results show that our method recovers scene geometry with high subpixel precision, and that it can handle many challenges of active reconstruction systems. We compare our results to state of the art methods such as micro phase shifting and modulated phase shifting.
6 0.55731791 82 iccv-2013-Compensating for Motion during Direct-Global Separation
7 0.53981704 28 iccv-2013-A Rotational Stereo Model Based on XSlit Imaging
9 0.47454652 207 iccv-2013-Illuminant Chromaticity from Image Sequences
10 0.45878065 173 iccv-2013-Fluttering Pattern Generation Using Modified Legendre Sequence for Coded Exposure Imaging
11 0.44670814 164 iccv-2013-Fibonacci Exposure Bracketing for High Dynamic Range Imaging
12 0.41345167 304 iccv-2013-PM-Huber: PatchMatch with Huber Regularization for Stereo Matching
13 0.40980998 135 iccv-2013-Efficient Image Dehazing with Boundary Constraint and Contextual Regularization
14 0.40706757 255 iccv-2013-Local Signal Equalization for Correspondence Matching
15 0.40237951 108 iccv-2013-Depth from Combining Defocus and Correspondence Using Light-Field Cameras
16 0.39474091 348 iccv-2013-Refractive Structure-from-Motion on Underwater Images
17 0.39139941 397 iccv-2013-Space-Time Tradeoffs in Photo Sequencing
18 0.39021012 98 iccv-2013-Cross-Field Joint Image Restoration via Scale Map
19 0.37748876 151 iccv-2013-Exploiting Reflection Change for Automatic Reflection Removal
20 0.37202206 382 iccv-2013-Semi-dense Visual Odometry for a Monocular Camera
topicId topicWeight
[(2, 0.06), (7, 0.016), (13, 0.016), (26, 0.109), (27, 0.015), (31, 0.058), (38, 0.023), (42, 0.062), (50, 0.18), (64, 0.063), (73, 0.031), (78, 0.016), (89, 0.171), (98, 0.043)]
simIndex simValue paperId paperTitle
same-paper 1 0.86891538 423 iccv-2013-Towards Motion Aware Light Field Video for Dynamic Scenes
Author: Salil Tambe, Ashok Veeraraghavan, Amit Agrawal
Abstract: Current Light Field (LF) cameras offer fixed resolution in space, time and angle which is decided a-priori and is independent of the scene. These cameras either trade-off spatial resolution to capture single-shot LF [20, 27, 12] or tradeoff temporal resolution by assuming a static scene to capture high spatial resolution LF [18, 3]. Thus, capturing high spatial resolution LF video for dynamic scenes remains an open and challenging problem. We present the concept, design and implementation of a LF video camera that allows capturing high resolution LF video. The spatial, angular and temporal resolution are not fixed a-priori and we exploit the scene-specific redundancy in space, time and angle. Our reconstruction is motion-aware and offers a continuum of resolution tradeoff with increasing motion in the scene. The key idea is (a) to design efficient multiplexing matrices that allow resolution tradeoffs, (b) use dictionary learning and sparse repre- sentations for robust reconstruction, and (c) perform local motion-aware adaptive reconstruction. We perform extensive analysis and characterize the performance of our motion-aware reconstruction algorithm. We show realistic simulations using a graphics simulator as well as real results using a LCoS based programmable camera. We demonstrate novel results such as high resolution digital refocusing for dynamic moving objects.
2 0.8586123 148 iccv-2013-Example-Based Facade Texture Synthesis
Author: Dengxin Dai, Hayko Riemenschneider, Gerhard Schmitt, Luc Van_Gool
Abstract: There is an increased interest in the efficient creation of city models, be it virtual or as-built. We present a method for synthesizing complex, photo-realistic facade images, from a single example. After parsing the example image into its semantic components, a tiling for it is generated. Novel tilings can then be created, yielding facade textures with different dimensions or with occluded parts inpainted. A genetic algorithm guides the novel facades as well as inpainted parts to be consistent with the example, both in terms of their overall structure and their detailed textures. Promising results for multiple standard datasets in particular for the different building styles they contain demonstrate the potential of the method. – –
3 0.83089519 155 iccv-2013-Facial Action Unit Event Detection by Cascade of Tasks
Author: Xiaoyu Ding, Wen-Sheng Chu, Fernando De_La_Torre, Jeffery F. Cohn, Qiao Wang
Abstract: Automatic facial Action Unit (AU) detection from video is a long-standing problem in facial expression analysis. AU detection is typically posed as a classification problem between frames or segments of positive examples and negative ones, where existing work emphasizes the use of different features or classifiers. In this paper, we propose a method called Cascade of Tasks (CoT) that combines the use ofdifferent tasks (i.e., , frame, segment and transition)for AU event detection. We train CoT in a sequential manner embracing diversity, which ensures robustness and generalization to unseen data. In addition to conventional framebased metrics that evaluate frames independently, we propose a new event-based metric to evaluate detection performance at event-level. We show how the CoT method consistently outperforms state-of-the-art approaches in both frame-based and event-based metrics, across three public datasets that differ in complexity: CK+, FERA and RUFACS.
4 0.82109296 361 iccv-2013-Robust Trajectory Clustering for Motion Segmentation
Author: Feng Shi, Zhong Zhou, Jiangjian Xiao, Wei Wu
Abstract: Due to occlusions and objects ’ non-rigid deformation in the scene, the obtained motion trajectories from common trackers may contain a number of missing or mis-associated entries. To cluster such corrupted point based trajectories into multiple motions is still a hard problem. In this paper, we present an approach that exploits temporal and spatial characteristics from tracked points to facilitate segmentation of incomplete and corrupted trajectories, thereby obtain highly robust results against severe data missing and noises. Our method first uses the Discrete Cosine Transform (DCT) bases as a temporal smoothness constraint on trajectory projection to ensure the validity of resulting components to repair pathological trajectories. Then, based on an observation that the trajectories of foreground and background in a scene may have different spatial distributions, we propose a two-stage clustering strategy that first performs foreground-background separation then segments remaining foreground trajectories. We show that, with this new clustering strategy, sequences with complex motions can be accurately segmented by even using a simple trans- lational model. Finally, a series of experiments on Hopkins 155 dataset andBerkeley motion segmentation dataset show the advantage of our method over other state-of-the-art motion segmentation algorithms in terms of both effectiveness and robustness.
5 0.77174819 414 iccv-2013-Temporally Consistent Superpixels
Author: Matthias Reso, Jörn Jachalsky, Bodo Rosenhahn, Jörn Ostermann
Abstract: Superpixel algorithms represent a very useful and increasingly popular preprocessing step for a wide range of computer vision applications, as they offer the potential to boost efficiency and effectiveness. In this regards, this paper presents a highly competitive approach for temporally consistent superpixelsfor video content. The approach is based on energy-minimizing clustering utilizing a novel hybrid clustering strategy for a multi-dimensional feature space working in a global color subspace and local spatial subspaces. Moreover, a new contour evolution based strategy is introduced to ensure spatial coherency of the generated superpixels. For a thorough evaluation the proposed approach is compared to state of the art supervoxel algorithms using established benchmarks and shows a superior performance.
6 0.76706922 19 iccv-2013-A Learning-Based Approach to Reduce JPEG Artifacts in Image Matting
7 0.76627684 33 iccv-2013-A Unified Video Segmentation Benchmark: Annotation, Metrics and Analysis
8 0.76554811 95 iccv-2013-Cosegmentation and Cosketch by Unsupervised Learning
9 0.76331091 396 iccv-2013-Space-Time Robust Representation for Action Recognition
10 0.76249444 196 iccv-2013-Hierarchical Data-Driven Descent for Efficient Optimal Deformation Estimation
11 0.76205581 411 iccv-2013-Symbiotic Segmentation and Part Localization for Fine-Grained Categorization
12 0.76182568 165 iccv-2013-Find the Best Path: An Efficient and Accurate Classifier for Image Hierarchies
13 0.76033753 150 iccv-2013-Exemplar Cut
14 0.75978845 102 iccv-2013-Data-Driven 3D Primitives for Single Image Understanding
15 0.75969768 420 iccv-2013-Topology-Constrained Layered Tracking with Latent Flow
16 0.75456995 156 iccv-2013-Fast Direct Super-Resolution by Simple Functions
17 0.75453484 127 iccv-2013-Dynamic Pooling for Complex Event Recognition
18 0.7539885 8 iccv-2013-A Deformable Mixture Parsing Model with Parselets
19 0.75384903 107 iccv-2013-Deformable Part Descriptors for Fine-Grained Recognition and Attribute Prediction
20 0.75360274 78 iccv-2013-Coherent Motion Segmentation in Moving Camera Videos Using Optical Flow Orientations