nips nips2002 nips2002-202 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Kinh Tieu, Erik G. Miller
Abstract: In [1] we introduced a linear statistical model of joint color changes in images due to variation in lighting and certain non-geometric camera parameters. We did this by measuring the mappings of colors in one image of a scene to colors in another image of the same scene under different lighting conditions. Here we increase the flexibility of this color flow model by allowing flow coefficients to vary according to a low order polynomial over the image. This allows us to better fit smoothly varying lighting conditions as well as curved surfaces without endowing our model with too much capacity. We show results on image matching and shadow removal and detection.
Reference: text
sentIndex sentText sentNum sentScore
1 edu Abstract In [1] we introduced a linear statistical model of joint color changes in images due to variation in lighting and certain non-geometric camera parameters. [sent-6, score-1.209]
2 We did this by measuring the mappings of colors in one image of a scene to colors in another image of the same scene under different lighting conditions. [sent-7, score-1.104]
3 Here we increase the flexibility of this color flow model by allowing flow coefficients to vary according to a low order polynomial over the image. [sent-8, score-0.767]
4 This allows us to better fit smoothly varying lighting conditions as well as curved surfaces without endowing our model with too much capacity. [sent-9, score-0.287]
5 We show results on image matching and shadow removal and detection. [sent-10, score-0.444]
6 Light sources, shadows, camera aperture, exposure time, transducer non-linearities, and camera processing (such as auto-gain-control and color balancing) can all affect the final image of a scene. [sent-12, score-1.141]
7 These effects have a significant impact on the images obtained with cameras and hence on image processing algorithms, often hampering or eliminating our ability to produce reliable recognition algorithms. [sent-13, score-0.436]
8 Addressing the variability of images due to these photic parameters has been an important problem in machine vision. [sent-14, score-0.252]
9 We distinguish photic parameters from geometric parameters, such as camera orientation or blurring, that affect which parts of the scene a particular pixel represents. [sent-15, score-0.326]
10 We also note that photic parameters are more general than “lighting parameters” and include anything which affects the final RGB values in an image given that the geometric parameters and the objects in the scene have been fixed. [sent-16, score-0.475]
11 We present a statistical linear model of color change space that is learned by observing how the colors in static images change jointly under common, naturally occurring lighting changes. [sent-17, score-1.271]
12 Such a model can be used for a number of tasks, including synthesis of images of new objects under different lighting conditions, image matching, and shadow detection. [sent-18, score-0.745]
13 The model uses no prior knowledge of lighting conditions, surface reflectances, or other parameters during data collection and modeling. [sent-22, score-0.249]
14 It also has no built-in knowledge of the physics of image acquisition or “typical” image color changes, such as brightness changes. [sent-23, score-1.282]
15 While it may not apply to all scenes equally well, it is a model of frequently occurring joint color changes, which is meant to apply to all scenes. [sent-25, score-0.744]
16 Third, while our model is linear in color change space, each joint color change that we model (a 3-D vector field) is completely arbitrary, and is not itself restricted to being linear. [sent-26, score-1.571]
17 After discussing previous work in Section 2, we introduce the color flow model and how it is obtained from observations in Section 3. [sent-28, score-0.727]
18 In Section 4, we show how the model and a single observed image can be used to generate a large family of related images. [sent-29, score-0.274]
19 In Section 5 we give preliminary results for image matching (object recognition) and shadow detection. [sent-31, score-0.444]
20 2 Previous work The color constancy literature contains a large body of work on estimating surface reflectances and various photic parameters from images. [sent-32, score-0.951]
21 Another technique is to estimate the relative illuminant or mapping of colors under an unknown illuminant to a canonical one. [sent-38, score-0.296]
22 The intersection of the mappings for each pixel in an image is used to choose a “best” mapping. [sent-40, score-0.332]
23 [7] trained a back-propagation multi-layer neural network to estimate the parameters of a linear color mapping. [sent-41, score-0.745]
24 The approach in [8] works in the log color spectra space where the effect of a relative illuminant is a set of constant shifts in the scalar coefficients of linear models for the image colors and illuminant. [sent-42, score-1.25]
25 [9] bypasses the need to predict specific scene properties by proving that the set of images of a gray Lambertian convex object under all lighting conditions form a convex cone. [sent-44, score-0.499]
26 1 We wanted a model which, based upon a single image (instead of three required by [9]), could make useful predictions about other images of the same scene. [sent-45, score-0.413]
27 3 Color flows In the following, let C = {(r, g, b)T ∈ R3 : 0 ≤ r ≤ 255, 0 ≤ g ≤ 255, 0 ≤ b ≤ 255} be the set of all possible observable image color 3-vectors. [sent-47, score-0.987]
28 Let the vector-valued color of an image pixel p be denoted by c(p) ∈ C. [sent-48, score-1.038]
29 Suppose we are given two P -pixel RGB color images I1 and I2 of the same scene taken under two different photic parameters θ1 and θ2 (the images are registered). [sent-49, score-1.182]
30 The authors’ results on color images also do not address the issue of metamers, and assume that light is composed of only the wavelengths red, green, and blue. [sent-51, score-0.886]
31 a b c d e f Figure 1: Matching non-linear color changes. [sent-52, score-0.727]
32 corresponding image pixels pk and pk , 1 ≤ k ≤ P , in the two images represents a single1 2 color mapping c(pk ) → c(pk ) that is conveniently represented by the vector difference: 1 2 d(pk , pk ) = c(pk ) − c(pk ). [sent-56, score-1.496]
33 1 2 2 1 (1) By computing P vector differences (one for each pair of pixels) and placing each at the point c(pk ) in color space C, we have a partially observed color flow: 1 Φ (c(pk )) = d(pk , pk ), 1 1 2 1≤k≤P (2) defined at points in C for which there are colors in image I1 . [sent-57, score-1.984]
34 a vector field Φ defined at all points in C) from a partially observed color flow Φ , we must address two issues. [sent-60, score-0.741]
35 Second, there may be multiple pixels of a particular color in image I1 that are mapped to different colors in image I2 . [sent-62, score-1.398]
36 We use a radial basis function estimator which defines the flow at a color point (r, g, b) T as the weighted proximity-based average of nearby observed “flow vectors”. [sent-63, score-0.759]
37 Note that color flows are defined so that a color point with only a single nearby neighbor will inherit a flow vector that is nearly parallel to its neighbor. [sent-65, score-1.472]
38 The idea is that if a particular color, under a photic parameter change θ1 → θ2 , is observed to get a little bit darker and a little bit bluer, for example, then its neighbors in color space are also defined to exhibit this behavior. [sent-66, score-0.963]
39 1 Structure in the space of color flows Consider a flat Lambertian surface that may have different reflectances as a function of the wavelength. [sent-68, score-0.793]
40 While in principle it is possible for a change in lighting to map any color from such a surface to any other color independently of all other colors 2 , we know from experience that many such joint maps are unlikely. [sent-69, score-1.904]
41 This suggests that while the marginal distribution of mappings for a particular color is broadly distributed, the space of possible joint color maps (i. [sent-70, score-1.511]
42 In learning a statistical model of color flows, many common color flows can be anticipated such as ones that make colors a little darker, lighter, or more red. [sent-73, score-1.578]
43 These types of flows can be well modeled with a simple global 3x3 matrix A that maps a color c 1 in image I1 to a color c2 in image I2 via c2 = Ac1 . [sent-74, score-1.993]
44 Such photic changes will tend 2 By carefully choosing properties such as the surface reflectance of a point as a function of wave˜ length and lighting any mapping Φ can, in principle, be observed even on a flat Lambertian surface. [sent-77, score-0.408]
45 The rightmost image is an ideal quotient image, corresponding to a Figure 3: Effects of the first three eigenflows. [sent-82, score-0.313]
46 to leave the bright and dim parts of the image alone, while spreading the central colors of color space toward the margins. [sent-85, score-1.111]
47 For a linear imaging process, the ratio of the brightnesses of two images, or quotient image [12], should vary smoothly except at surface normal boundaries. [sent-86, score-0.471]
48 However as shown in Figure 2, the quotient image is a function not only of surface normal, but also of albedo– direct evidence of a non-linear imaging process. [sent-87, score-0.396]
49 Another pair of images exhibiting a nonlinear color flow is shown in Figures 1a and b. [sent-88, score-0.864]
50 Notice that the brighter areas of the original image get brighter and the darker portions get darker. [sent-89, score-0.356]
51 2 Color eigenflows We wanted to capture the structure in color flow space by observing real-world data in an unsupervised fashion. [sent-91, score-0.743]
52 A one square meter color palette was printed on standard non-glossy plotter paper using every color that could be produced by a Hewlett Packard DesignJet 650C. [sent-92, score-1.483]
53 Images of the poster were captured using the video camera under a wide variety of lighting conditions, including various intervals during sunrise, sunset, at midday, and with various combinations of office lights and outdoor lighting (controlled by adjusting blinds). [sent-96, score-0.541]
54 j j We chose image pairs I j = (I1 , I2 ), 1 ≤ j ≤ 800, by randomly and independently selecting individual images from the set of raw images. [sent-99, score-0.397]
55 Each image pair was then used to estimate a full color flow Φ(I j ). [sent-100, score-0.987]
56 We call the principal components of the color flow data “color eigenflows”, or just eigenflows, 4 for short. [sent-103, score-0.751]
57 We emphasize that these principal components of color flows have nothing to do with the distribution of colors in images, but only model the distribution of changes in color. [sent-104, score-0.905]
58 Our work is very different from approaches that compute principal components in the intensity or color space itself [14, 15]. [sent-106, score-0.751]
59 25 color flow linear diagonal gray world rms error 20 15 10 5 a 0 1 2 3 image 4 b Figure 4: Image matching. [sent-108, score-1.139]
60 Bottom row: best approximation to original images using eigenflows and the source image a. [sent-110, score-0.426]
61 4 Using color flows to synthesize novel images How do we generate a new image from a source image and a color flow Φ? [sent-113, score-2.14]
62 For each pixel p in the new image, its color c (p) can be computed as c c (p) = c(p) + αΦ(ˆ(p)), (4) where c(p) is color in the source image and α is a scalar multiplier that represents the “quantity of flow”. [sent-114, score-1.794]
63 ˆ(p) is interpreted to be the color vector closest to c(p) (in color space) c at which Φ has been computed. [sent-115, score-1.454]
64 Figure 3 shows the effect of the first three eigenflows on an image of a face. [sent-117, score-0.26]
65 The original image is in the middle of each row while the other images show the application of each eigenflow with α values between ±4 standard deviations. [sent-118, score-0.413]
66 We stress that the eigenflows were only computed once (on the color palette data), and that they were applied to the face image without any knowledge of the parameters under which the face image was taken. [sent-125, score-1.344]
67 1 Flowing one image to another Suppose we have two images and we pose the question of whether they are images of the same object or scene. [sent-127, score-0.576]
68 We suggest that if we can “flow” one image to another then the images are likely to be of the same scene. [sent-128, score-0.397]
69 Let us treat an image I as a function that takes a color flow and returns a difference image D by placing at each (x,y) pixel in D the color change vector Φ(c(p x,y )). [sent-129, score-2.111]
70 The difference image basis for I and set of eigenflows Ψi , 1 ≤ i ≤ E, is Di = I(Ψi ). [sent-130, score-0.288]
71 The set of images S that can be formed using a source image and a set of eigenflows is S = {S : S = E I + i=1 γi Di }, where the γi ’s are scalars, and here I is just an image, and not a function. [sent-131, score-0.426]
72 We can only flow image I1 to another image I2 if it is possible to represent the difference image as a linear combination of the Di ’s, i. [sent-133, score-0.826]
73 We find the optimal (in the least-squares sense) γi ’s by solving the system E D= γ i Di , i=1 (5) a b e c d f Figure 5: Modeling lighting changes with color flows. [sent-136, score-0.94]
74 While clipping can only reduce the error between a synthetic image and a target image, it may change which solution is optimal in some cases. [sent-153, score-0.384]
75 1 Image matching One use of the color change model is for image matching. [sent-155, score-1.086]
76 We first examined our ability to flow a source image to a matching target image under different photic parameters. [sent-157, score-0.74]
77 The linear method finds the matrix A in Equation 3 that minimizes the L2 error between the synthetic and target images; diagonal does the same with a diagonal A; gray world linearly matches the mean R, G, B values of the synthetic and target images. [sent-159, score-0.204]
78 In a second experiment (Figure 4), we matched images of a face taken under various camera parameters but with constant lighting. [sent-161, score-0.248]
79 2 Local flows In another test, the source and target images were taken under very different lighting conditions. [sent-164, score-0.367]
80 Furthermore, shadowing effects and lighting direction changed between the two images. [sent-165, score-0.208]
81 a b c d Figure 6: Backgrounding with color flows. [sent-174, score-0.727]
82 c For each of the two regions (from background subtraction), a “flow” was done between the original image and the new image based on the pixels in each region. [sent-177, score-0.561]
83 d The color flow of the original image using the eigenflow coefficients recovered from the shadow region. [sent-178, score-1.113]
84 The color flow using the coefficients from the non-shadow region are unable to give a reasonable reconstruction of the new image. [sent-179, score-0.754]
85 We performed one experiment to measure the over-fitting of our method versus the others by trying to flow an original image to its reflection (Figure 5). [sent-181, score-0.26]
86 Note that while our method had lower error (which is undesirable), there was still a significant spread between matching images and non-matching images. [sent-187, score-0.21]
87 We believe we can improve differentiation between matching and non-matching image pairs by assigning a cost to the change in γ i across each image patch. [sent-188, score-0.619]
88 For matching images, sharp changes would only be necessary at shadow boundaries or changes in the surface orientation relative to directional light sources. [sent-190, score-0.332]
89 Shadows can also move across an image and appear as moving objects. [sent-194, score-0.26]
90 Many of these problems could be eliminated if we could recognize that a particular region of an image is equivalent to a previously seen version of the scene, but under a different lighting. [sent-195, score-0.287]
91 Figure 6a shows how color flows may be used to distinguish between a new object and a shadow by flowing both regions. [sent-196, score-0.895]
92 A constant color flow across an entire region may not model the image change well. [sent-197, score-1.055]
93 That is, we can find the best least squares fit of the difference image allowing our γ estimates to vary linearly or quadratically over the image. [sent-199, score-0.379]
94 We implemented this technique by computing flows γx,y between corresponding image patches (indexed by x and y), and then minimizing the following form: (γx,y − M cx,y )T Σ−1 (γx,y − M cx,y ). [sent-200, score-0.279]
95 Allowing the γ’s to vary over the image greatly increases the capacity of a matcher, but by limiting this variation to linear or quadratic variation, the capacity is still not able to qualitatively match “non-matching” images. [sent-204, score-0.406]
96 Note that this smooth variation in eigenflow coefficients can model either a nearby light source or a smoothly curving surface, since either of these conditions will result in a smoothly varying lighting change. [sent-205, score-0.38]
97 8 Table 1: Error residuals for shadow and non-shadow regions after color flows. [sent-212, score-0.853]
98 Much larger experiments need to be performed to establish the utility of the color change model for particular applications. [sent-217, score-0.768]
99 However, since the color change model represents a compact description of lighting changes, including nonlinearities, we are optimistic about these applications. [sent-218, score-0.951]
100 Color correction of face images under different illuminants by rgb eigenfaces. [sent-336, score-0.28]
wordName wordTfidf (topN-words)
[('color', 0.727), ('ows', 0.298), ('image', 0.26), ('ow', 0.228), ('lighting', 0.183), ('eigen', 0.171), ('images', 0.137), ('shadow', 0.126), ('colors', 0.124), ('photic', 0.115), ('pk', 0.115), ('illuminant', 0.086), ('rgb', 0.08), ('camera', 0.077), ('shadows', 0.068), ('surface', 0.066), ('scene', 0.066), ('matching', 0.058), ('ectance', 0.053), ('quotient', 0.053), ('pixel', 0.051), ('poster', 0.05), ('constancy', 0.043), ('ectances', 0.043), ('object', 0.042), ('change', 0.041), ('darker', 0.038), ('lambertian', 0.038), ('brightness', 0.035), ('quadratically', 0.034), ('coef', 0.034), ('face', 0.034), ('capacity', 0.032), ('smoothly', 0.031), ('cients', 0.03), ('changes', 0.03), ('flow', 0.03), ('varying', 0.029), ('aperture', 0.029), ('backgrounding', 0.029), ('brainard', 0.029), ('brighter', 0.029), ('clipping', 0.029), ('illuminants', 0.029), ('palette', 0.029), ('source', 0.029), ('rms', 0.029), ('gray', 0.028), ('difference', 0.028), ('pixels', 0.027), ('surfaces', 0.027), ('region', 0.027), ('vary', 0.026), ('ce', 0.026), ('re', 0.025), ('tieu', 0.025), ('wall', 0.025), ('lights', 0.025), ('effects', 0.025), ('principal', 0.024), ('video', 0.023), ('ijcv', 0.023), ('synthesis', 0.022), ('light', 0.022), ('mappings', 0.021), ('synthetic', 0.021), ('variation', 0.02), ('patches', 0.019), ('maps', 0.019), ('spectra', 0.019), ('nearby', 0.018), ('di', 0.018), ('quadratic', 0.018), ('linear', 0.018), ('target', 0.018), ('illumination', 0.018), ('objects', 0.017), ('geometric', 0.017), ('joint', 0.017), ('linearly', 0.017), ('imaging', 0.017), ('placing', 0.017), ('conditions', 0.017), ('elds', 0.016), ('miller', 0.016), ('wanted', 0.016), ('diagonal', 0.016), ('spectral', 0.016), ('shifts', 0.016), ('row', 0.016), ('world', 0.016), ('error', 0.015), ('allowing', 0.014), ('background', 0.014), ('bit', 0.014), ('berkeley', 0.014), ('top', 0.014), ('recognition', 0.014), ('observed', 0.014), ('convex', 0.013)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999976 202 nips-2002-Unsupervised Color Constancy
Author: Kinh Tieu, Erik G. Miller
Abstract: In [1] we introduced a linear statistical model of joint color changes in images due to variation in lighting and certain non-geometric camera parameters. We did this by measuring the mappings of colors in one image of a scene to colors in another image of the same scene under different lighting conditions. Here we increase the flexibility of this color flow model by allowing flow coefficients to vary according to a low order polynomial over the image. This allows us to better fit smoothly varying lighting conditions as well as curved surfaces without endowing our model with too much capacity. We show results on image matching and shadow removal and detection.
2 0.42350176 105 nips-2002-How to Combine Color and Shape Information for 3D Object Recognition: Kernels do the Trick
Author: B. Caputo, Gy. Dorkó
Abstract: This paper presents a kernel method that allows to combine color and shape information for appearance-based object recognition. It doesn't require to define a new common representation, but use the power of kernels to combine different representations together in an effective manner. These results are achieved using results of statistical mechanics of spin glasses combined with Markov random fields via kernel functions. Experiments show an increase in recognition rate up to 5.92% with respect to conventional strategies. 1
3 0.31979755 173 nips-2002-Recovering Intrinsic Images from a Single Image
Author: Marshall F. Tappen, William T. Freeman, Edward H. Adelson
Abstract: We present an algorithm that uses multiple cues to recover shading and reflectance intrinsic images from a single image. Using both color information and a classifier trained to recognize gray-scale patterns, each image derivative is classified as being caused by shading or a change in the surface’s reflectance. Generalized Belief Propagation is then used to propagate information from areas where the correct classification is clear to areas where it is ambiguous. We also show results on real images.
4 0.28377172 148 nips-2002-Morton-Style Factorial Coding of Color in Primary Visual Cortex
Author: Javier R. Movellan, Thomas Wachtler, Thomas D. Albright, Terrence Sejnowski
Abstract: We introduce the notion of Morton-style factorial coding and illustrate how it may help understand information integration and perceptual coding in the brain. We show that by focusing on average responses one may miss the existence of factorial coding mechanisms that become only apparent when analyzing spike count histograms. We show evidence suggesting that the classical/non-classical receptive field organization in the cortex effectively enforces the development of Morton-style factorial codes. This may provide some cues to help understand perceptual coding in the brain and to develop new unsupervised learning algorithms. While methods like ICA (Bell & Sejnowski, 1997) develop independent codes, in Morton-style coding the goal is to make two or more external aspects of the world become independent when conditioning on internal representations. In this paper we introduce the notion of Morton-style factorial coding and illustrate how it may help analyze information integration and perceptual organization in the brain. In the neurosciences factorial codes are often studied in the context of mean tuning curves. A tuning curve is called separable if it can be expressed as the product of terms selectively influenced by different stimulus dimensions. Separable tuning curves are taken as evidence of factorial coding mechanisms. In this paper we show that by focusing on average responses one may miss the existence of factorial coding mechanisms that become only apparent when analyzing spike count histograms. Morton (1969) analyzed a wide variety of psychophysical experiments on word perception and showed that they could be explained using a model in which stimulus and context have separable effects on perception. More precisely, in Mortons’ model the joint effect of stimulus and context on a perceptual representation can be obtained by multiplying terms selectively controlled by stimulus and by context, i.e., £ © # #
5 0.16524257 39 nips-2002-Bayesian Image Super-Resolution
Author: Michael E. Tipping, Christopher M. Bishop
Abstract: The extraction of a single high-quality image from a set of lowresolution images is an important problem which arises in fields such as remote sensing, surveillance, medical imaging and the extraction of still images from video. Typical approaches are based on the use of cross-correlation to register the images followed by the inversion of the transformation from the unknown high resolution image to the observed low resolution images, using regularization to resolve the ill-posed nature of the inversion process. In this paper we develop a Bayesian treatment of the super-resolution problem in which the likelihood function for the image registration parameters is based on a marginalization over the unknown high-resolution image. This approach allows us to estimate the unknown point spread function, and is rendered tractable through the introduction of a Gaussian process prior over images. Results indicate a significant improvement over techniques based on MAP (maximum a-posteriori) point optimization of the high resolution image and associated registration parameters. 1
6 0.16504554 100 nips-2002-Half-Lives of EigenFlows for Spectral Clustering
7 0.16368429 10 nips-2002-A Model for Learning Variance Components of Natural Images
9 0.10426193 74 nips-2002-Dynamic Structure Super-Resolution
10 0.098810531 182 nips-2002-Shape Recipes: Scene Representations that Refer to the Image
11 0.093328863 57 nips-2002-Concurrent Object Recognition and Segmentation by Graph Partitioning
12 0.086542889 132 nips-2002-Learning to Detect Natural Image Boundaries Using Brightness and Texture
13 0.08637739 122 nips-2002-Learning About Multiple Objects in Images: Factorial Learning without Factorial Search
14 0.085278019 126 nips-2002-Learning Sparse Multiscale Image Representations
15 0.081977762 87 nips-2002-Fast Transformation-Invariant Factor Analysis
16 0.079324916 2 nips-2002-A Bilinear Model for Sparse Coding
17 0.071821705 133 nips-2002-Learning to Perceive Transparency from the Statistics of Natural Scenes
18 0.060186405 127 nips-2002-Learning Sparse Topographic Representations with Products of Student-t Distributions
19 0.057449397 16 nips-2002-A Prototype for Automatic Recognition of Spontaneous Facial Actions
20 0.055335723 51 nips-2002-Classifying Patterns of Visual Motion - a Neuromorphic Approach
topicId topicWeight
[(0, -0.157), (1, 0.042), (2, 0.011), (3, 0.367), (4, 0.0), (5, -0.146), (6, 0.314), (7, -0.044), (8, -0.088), (9, -0.042), (10, -0.074), (11, -0.044), (12, -0.114), (13, -0.184), (14, -0.079), (15, -0.226), (16, -0.337), (17, -0.058), (18, -0.009), (19, -0.164), (20, 0.205), (21, -0.015), (22, 0.009), (23, -0.0), (24, -0.039), (25, -0.074), (26, -0.06), (27, 0.1), (28, 0.047), (29, -0.073), (30, -0.024), (31, 0.029), (32, 0.161), (33, 0.005), (34, 0.014), (35, 0.011), (36, 0.025), (37, -0.001), (38, -0.036), (39, -0.005), (40, -0.017), (41, -0.0), (42, 0.035), (43, -0.092), (44, -0.002), (45, 0.014), (46, -0.005), (47, -0.001), (48, -0.015), (49, 0.041)]
simIndex simValue paperId paperTitle
same-paper 1 0.9563328 202 nips-2002-Unsupervised Color Constancy
Author: Kinh Tieu, Erik G. Miller
Abstract: In [1] we introduced a linear statistical model of joint color changes in images due to variation in lighting and certain non-geometric camera parameters. We did this by measuring the mappings of colors in one image of a scene to colors in another image of the same scene under different lighting conditions. Here we increase the flexibility of this color flow model by allowing flow coefficients to vary according to a low order polynomial over the image. This allows us to better fit smoothly varying lighting conditions as well as curved surfaces without endowing our model with too much capacity. We show results on image matching and shadow removal and detection.
2 0.79710931 105 nips-2002-How to Combine Color and Shape Information for 3D Object Recognition: Kernels do the Trick
Author: B. Caputo, Gy. Dorkó
Abstract: This paper presents a kernel method that allows to combine color and shape information for appearance-based object recognition. It doesn't require to define a new common representation, but use the power of kernels to combine different representations together in an effective manner. These results are achieved using results of statistical mechanics of spin glasses combined with Markov random fields via kernel functions. Experiments show an increase in recognition rate up to 5.92% with respect to conventional strategies. 1
3 0.69821721 173 nips-2002-Recovering Intrinsic Images from a Single Image
Author: Marshall F. Tappen, William T. Freeman, Edward H. Adelson
Abstract: We present an algorithm that uses multiple cues to recover shading and reflectance intrinsic images from a single image. Using both color information and a classifier trained to recognize gray-scale patterns, each image derivative is classified as being caused by shading or a change in the surface’s reflectance. Generalized Belief Propagation is then used to propagate information from areas where the correct classification is clear to areas where it is ambiguous. We also show results on real images.
4 0.56704861 182 nips-2002-Shape Recipes: Scene Representations that Refer to the Image
Author: William T. Freeman, Antonio Torralba
Abstract: The goal of low-level vision is to estimate an underlying scene, given an observed image. Real-world scenes (eg, albedos or shapes) can be very complex, conventionally requiring high dimensional representations which are hard to estimate and store. We propose a low-dimensional representation, called a scene recipe, that relies on the image itself to describe the complex scene configurations. Shape recipes are an example: these are the regression coefficients that predict the bandpassed shape from image data. We describe the benefits of this representation, and show two uses illustrating their properties: (1) we improve stereo shape estimates by learning shape recipes at low resolution and applying them at full resolution; (2) Shape recipes implicitly contain information about lighting and materials and we use them for material segmentation.
5 0.50519156 148 nips-2002-Morton-Style Factorial Coding of Color in Primary Visual Cortex
Author: Javier R. Movellan, Thomas Wachtler, Thomas D. Albright, Terrence Sejnowski
Abstract: We introduce the notion of Morton-style factorial coding and illustrate how it may help understand information integration and perceptual coding in the brain. We show that by focusing on average responses one may miss the existence of factorial coding mechanisms that become only apparent when analyzing spike count histograms. We show evidence suggesting that the classical/non-classical receptive field organization in the cortex effectively enforces the development of Morton-style factorial codes. This may provide some cues to help understand perceptual coding in the brain and to develop new unsupervised learning algorithms. While methods like ICA (Bell & Sejnowski, 1997) develop independent codes, in Morton-style coding the goal is to make two or more external aspects of the world become independent when conditioning on internal representations. In this paper we introduce the notion of Morton-style factorial coding and illustrate how it may help analyze information integration and perceptual organization in the brain. In the neurosciences factorial codes are often studied in the context of mean tuning curves. A tuning curve is called separable if it can be expressed as the product of terms selectively influenced by different stimulus dimensions. Separable tuning curves are taken as evidence of factorial coding mechanisms. In this paper we show that by focusing on average responses one may miss the existence of factorial coding mechanisms that become only apparent when analyzing spike count histograms. Morton (1969) analyzed a wide variety of psychophysical experiments on word perception and showed that they could be explained using a model in which stimulus and context have separable effects on perception. More precisely, in Mortons’ model the joint effect of stimulus and context on a perceptual representation can be obtained by multiplying terms selectively controlled by stimulus and by context, i.e., £ © # #
6 0.36374182 100 nips-2002-Half-Lives of EigenFlows for Spectral Clustering
8 0.30686963 10 nips-2002-A Model for Learning Variance Components of Natural Images
9 0.29685974 39 nips-2002-Bayesian Image Super-Resolution
10 0.28333467 57 nips-2002-Concurrent Object Recognition and Segmentation by Graph Partitioning
11 0.28261542 126 nips-2002-Learning Sparse Multiscale Image Representations
12 0.27698693 122 nips-2002-Learning About Multiple Objects in Images: Factorial Learning without Factorial Search
13 0.27402931 133 nips-2002-Learning to Perceive Transparency from the Statistics of Natural Scenes
14 0.24661089 132 nips-2002-Learning to Detect Natural Image Boundaries Using Brightness and Texture
15 0.23514089 150 nips-2002-Multiple Cause Vector Quantization
16 0.23093624 87 nips-2002-Fast Transformation-Invariant Factor Analysis
17 0.22826606 2 nips-2002-A Bilinear Model for Sparse Coding
18 0.21010014 16 nips-2002-A Prototype for Automatic Recognition of Spontaneous Facial Actions
19 0.19406715 127 nips-2002-Learning Sparse Topographic Representations with Products of Student-t Distributions
20 0.18845728 74 nips-2002-Dynamic Structure Super-Resolution
topicId topicWeight
[(1, 0.011), (5, 0.326), (11, 0.018), (23, 0.02), (42, 0.067), (54, 0.117), (55, 0.028), (67, 0.027), (68, 0.042), (74, 0.115), (92, 0.024), (98, 0.074)]
simIndex simValue paperId paperTitle
same-paper 1 0.80839723 202 nips-2002-Unsupervised Color Constancy
Author: Kinh Tieu, Erik G. Miller
Abstract: In [1] we introduced a linear statistical model of joint color changes in images due to variation in lighting and certain non-geometric camera parameters. We did this by measuring the mappings of colors in one image of a scene to colors in another image of the same scene under different lighting conditions. Here we increase the flexibility of this color flow model by allowing flow coefficients to vary according to a low order polynomial over the image. This allows us to better fit smoothly varying lighting conditions as well as curved surfaces without endowing our model with too much capacity. We show results on image matching and shadow removal and detection.
2 0.75883055 16 nips-2002-A Prototype for Automatic Recognition of Spontaneous Facial Actions
Author: M.S. Bartlett, G.C. Littlewort, T.J. Sejnowski, J.R. Movellan
Abstract: We present ongoing work on a project for automatic recognition of spontaneous facial actions. Spontaneous facial expressions differ substantially from posed expressions, similar to how continuous, spontaneous speech differs from isolated words produced on command. Previous methods for automatic facial expression recognition assumed images were collected in controlled environments in which the subjects deliberately faced the camera. Since people often nod or turn their heads, automatic recognition of spontaneous facial behavior requires methods for handling out-of-image-plane head rotations. Here we explore an approach based on 3-D warping of images into canonical views. We evaluated the performance of the approach as a front-end for a spontaneous expression recognition system using support vector machines and hidden Markov models. This system employed general purpose learning mechanisms that can be applied to recognition of any facial movement. The system was tested for recognition of a set of facial actions defined by the Facial Action Coding System (FACS). We showed that 3D tracking and warping followed by machine learning techniques directly applied to the warped images, is a viable and promising technology for automatic facial expression recognition. One exciting aspect of the approach presented here is that information about movement dynamics emerged out of filters which were derived from the statistics of images.
3 0.52522194 173 nips-2002-Recovering Intrinsic Images from a Single Image
Author: Marshall F. Tappen, William T. Freeman, Edward H. Adelson
Abstract: We present an algorithm that uses multiple cues to recover shading and reflectance intrinsic images from a single image. Using both color information and a classifier trained to recognize gray-scale patterns, each image derivative is classified as being caused by shading or a change in the surface’s reflectance. Generalized Belief Propagation is then used to propagate information from areas where the correct classification is clear to areas where it is ambiguous. We also show results on real images.
4 0.50935698 52 nips-2002-Cluster Kernels for Semi-Supervised Learning
Author: Olivier Chapelle, Jason Weston, Bernhard SchĂślkopf
Abstract: We propose a framework to incorporate unlabeled data in kernel classifier, based on the idea that two points in the same cluster are more likely to have the same label. This is achieved by modifying the eigenspectrum of the kernel matrix. Experimental results assess the validity of this approach. 1
5 0.50623953 124 nips-2002-Learning Graphical Models with Mercer Kernels
Author: Francis R. Bach, Michael I. Jordan
Abstract: We present a class of algorithms for learning the structure of graphical models from data. The algorithms are based on a measure known as the kernel generalized variance (KGV), which essentially allows us to treat all variables on an equal footing as Gaussians in a feature space obtained from Mercer kernels. Thus we are able to learn hybrid graphs involving discrete and continuous variables of arbitrary type. We explore the computational properties of our approach, showing how to use the kernel trick to compute the relevant statistics in linear time. We illustrate our framework with experiments involving discrete and continuous data.
6 0.50502539 132 nips-2002-Learning to Detect Natural Image Boundaries Using Brightness and Texture
7 0.50453413 175 nips-2002-Reinforcement Learning to Play an Optimal Nash Equilibrium in Team Markov Games
8 0.50406414 74 nips-2002-Dynamic Structure Super-Resolution
9 0.50350738 127 nips-2002-Learning Sparse Topographic Representations with Products of Student-t Distributions
10 0.50252998 39 nips-2002-Bayesian Image Super-Resolution
11 0.50215757 3 nips-2002-A Convergent Form of Approximate Policy Iteration
12 0.50132668 152 nips-2002-Nash Propagation for Loopy Graphical Games
13 0.50098979 53 nips-2002-Clustering with the Fisher Score
14 0.49925879 2 nips-2002-A Bilinear Model for Sparse Coding
15 0.49699926 89 nips-2002-Feature Selection by Maximum Marginal Diversity
16 0.49683943 122 nips-2002-Learning About Multiple Objects in Images: Factorial Learning without Factorial Search
17 0.4963842 68 nips-2002-Discriminative Densities from Maximum Contrast Estimation
18 0.49598414 48 nips-2002-Categorization Under Complexity: A Unified MDL Account of Human Learning of Regular and Irregular Categories
19 0.49567634 169 nips-2002-Real-Time Particle Filters
20 0.49440548 27 nips-2002-An Impossibility Theorem for Clustering