Refereed Journal and Conference Papers

Measuring Perceived Differences in Surface Texture due to Changes in Higher Order Statistics
Journal of Optical Society of America A, Vol. 27, Issue 5, pp. 1232-1244

Also selected for inclusion in: 09:31 13/07/2010Virtual Journal for Biomedical Optics (VJBO) Vol. 5, Iss. 9 — Jul. 6, 2010

Abstract
We investigate the ability of humans to perceive changes in appearance of images of surface texture caused by the variation of their higher order statistics. We incrementally randomise their phase spectra while holding their first and second order statistics constant in order to ensure that the change in appearance is due solely to changes in third and other higher order statistics. Stimuli comprise both of natural and synthetically generated naturalistic images, the latter being used to prevent observers from making pixelwise comparisons. A difference scaling method is used to derive the perceptual scales for each observer, which show a sigmoidal relationship with the degree of randomisation. Observers were maximally sensitive to changes within the 20-60% randomisation range. In order to account for this behaviour we propose a biologically plausible model that computes the variance of local measurements of phase congruency.

On uniform resampling and gaze analysis of bidirectional texture functions
ACM Transactions on Applied Perception (TAP), 6(3), 2009.

Abstract
The use of illumination and view-dependent texture information is recently the best way to capture the appearance of real-world materials accurately. One example is the Bidirectional Texture Function. The main disadvantage of these data is their massive size. In this article, we employ perceptually-based methods to allow more efficient handling of these data. In the first step, we analyse different uniform resampling by means of a psychophysical study with 11 subjects, comparing original data with rendering of a uniformly resampled version over the hemisphere of illumination and view-dependent textural measurements. We have found that down-sampling in view and illumination azimuthal angles is less apparent than in elevation angles and that illumination directions can be down-sampled more than view directions without loss of visual accuracy. In the second step, we analyzed subjects gaze fixation during the experiment. The gaze analysis confirmed results from the experiment and revealed that subjects were fixating at locations aligned with direction of main gradient in rendered stimuli. As this gradient was mostly aligned with illumination gradient, we conclude that subjects were observing materials mainly in direction of illumination gradient. Our results provide interesting insights in human perception of real materials and show promising consequences for development of more efficient compression and rendering algorithms using these kind of massive data.

Modeling visual search on a rough surface
Journal of Vision, 9(4), 1-12 2009.

Abstract
The LNL (linear, non-linear, linear) model has previously been successfully applied to the problem of texture segmentation. In this study we investigate the extent to which a simple LNL model can simulate human performance in a search task involving a target on a textured surface. Two different classes of surface are considered: 1/f-noise and near-regular textures. We find that in both cases the search performance of the model does not differ significantly from that of people, over a wide range of task difficulties.

Bidirectional Texture Function Modeling: A State of the Art Survey
To appear in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.

Abstract
An ever-growing number of real world computer vision applications require classification, segmentation, retrieval, or realistic rendering of genuine materials. However, the appearance of real materials dramatically changes with illumination and viewing variations. Thus, the only reliable representation of material visual properties requires capturing of its reflectance in as wide range of light and camera position combinations as possible. This is a principle of the recent most advanced texture representation, the Bidirectional Texture Function (BTF). Multispectral BTF is a seven-dimensional function that depends on view and illumination directions as well as on planar texture coordinates. BTF is typically obtained by measurement of thousands of images covering many combinations of illumination and viewing angles. However, the large size of such measurements has prohibited their practical exploitation in any sensible application until recently. During the last few years the first BTF measurement, compression, modeling and rendering methods have emerged. In this paper we categorize, critically survey, and psychophysically compare such approaches, which were published in this newly arising and important computer vision & graphics area.

A Psychophysically Validated Metric for Bidirectional Texture Data Reduction
To appear in ACM Transactions on Graphics 27(5) (Proceedings of ACM SIGGRAPH Asia 2008)

Abstract
Bidirectional Texture Functions (BTF) are commonly thought to provide the most realistic perceptual experience of materials from rendered images. The key to providing efficient compression of BTF is the decision as to how much of the data should be preserved. We use psychophysical experiments to show that this decision depends critically upon the material concerned. Furthermore, we develop a BTF derived metric that enables us to automatically set a material's compression parameters in such a way as to provide users with a predefined perceptual quality. We investigate the correlation of three different BTF metrics with psychophysically derived data. Eight materials were presented to eleven naive observers who were asked to judge the perceived quality of BTF renderings as the amount of preserved data was varied. The metric showing the highest correlation with the thresholds set by the observers was the mean variance of individual BTF images. This metric was then used to automatically determine the material-specific compression parameters used in a vector quantisation scheme. The results were successfully validated in an experiment with six additional materials and eighteen observers. We show that using the psychophysically reduced BTF data significantly improves performance of a PCA-based compression method. On average, we were able to increase the compression ratios, and decrease processing times, by a factor of four without any differences being perceived.

Visual Search for a Target Against a 1/f^&beta Continuous Textured Background
Vision Research, 48(21), 2008, 2193-203

Abstract
We present synthetic surface textures as a novel class of stimuli for use in visual search ex-periments. Surface textures have certain advantages over both the arrays of abstract dis-crete items commonly used in search studies and photographs of natural scenes. In this study we investigate how changing the properties of the surface and target influence the difficulty of a search task. We present a comparison with Itti and Koch’s saliency model and find that it fails to model human behaviour on these surfaces. In particular it does not re-spond to changes in orientation in the same manner as human observers.

Perceived roughness of 1/f^&beta noise surfaces
Vision Research, 48, 2008, 1791-1797.

Abstract
We report results from a new methodology for investigating the visually perceived properties of surface textures. Densely sampled two-dimensional 1/f^&beta noise processes are used to model natural looking surfaces, which are rendered using combined point-source and ambient lighting. Surfaces are shown in motion to provide rich cues to their relief. They are generated in real time to enable observers to dynamically manipulate surface parameters. A method of adjustment is employed to investigate the effects that the two surface parameters, magnitude roll-off factor and RMS height, have on perceived roughness. The results are used to develop an estimation method for perceived roughness.

Optimal illumination for three-image photometric stereo using sensitivity analysis
IEE Proceedings - Vision, Image, and Signal Processing -- April 2006 -- Volume 153, Issue 2, p. 149-159.

Abstract
The optimal placement of the illumination for three-image photometric stereo acquisition of smooth and rough surface textures with respect to camera noise is derived and verified experimentally. The sensitivities of the scaled surface normal elements are derived and used to provide expressions for the noise variances. An overall figure of merit is developed by considering image-based rendering (i.e. relighting) of Lambertian surfaces. This metric is optimised numerically with respect to the illumination angles. An orthogonal configuration was found to be optimal. With regard to constant slant, the optimal separation between the tilt angles of successive illumination vectors was found to be 120°. The optimal slant angle was found to be 90° for smooth surface textures and 55° for rough surface textures.

Capture and Synthesis of 3D Surface Texture
International Journal of Computer Vision (VISI), 62(1-2), 2005, pp177-194.

Abstract
We present and compare five approaches for capturing, synthesising and relighting real 3D surface textures. Unlike 2D texture synthesis techniques they allow the captured textures to be relit using illumination conditions that differ from those of the original. We adapted a texture quilting method due to Efros and combined this with five different relighting representations, comprising: a set of three photometric images; surface gradient and albedo maps; polynomial texture maps; and two eigen based representations using 3 and 6 base images. We used twelve real textures to perform quantitative tests on the relighting methods in isolation. We developed a qualitative test for the assessment of the complete synthesis systems. Ten observers were asked to rank the images obtained from the five methods using five real textures. Statistical tests were applied to the rankings. The six-base-image eigen method produced the best quantitative relighting results and in particular was better able to cope with specular surfaces. However, in the qualitative tests there were no significant performance differences detected between it and the other two top performers. Our conclusion is therefore that the cheaper gradient and three-base-image eigen methods should be used in preference, especially where the surfaces are Lambertian or near Lambertian.

Classifying Surface Texture While Simultaneously Estimating Illumination
International Journal of Computer Vision (VISI), 62(1-2), 2005, pp83-96.

Abstract
We propose a novel classifier that both classifies surface texture and simultaneously estimates the unknown illumination conditions. A new formal model of the dependency of texture features on lighting direction is developed which shows that their mean vectors are trigonometric functions of the illuminations' tilt and slant angles. This is used to develop a probabilistic description of feature behaviour which forms the basis of the new classifier. Given a feature set from an image of an unknown texture captured under unknown illumination conditions the algorithm first estimates the most likely illumination direction for each possible texture class. These estimates are used to calculate the class likelihoods and the classification is made accordingly. The ability of the classifier to estimate illuminant direction, and to assign the correct class, was tested on 55 real texture samples in two stages. The classifier was able to accurately estimate both the tilt and the slant angles of the light source for the majority of textures and gave a 98% classification rate.

Real-time per-pixel rendering of textiles for virtual textile catalogues
International Journal of Clothing Science and Technology, Vol. 16 No.1/2, 2004

Abstract
We present recent results from an EPSRC funded project VirTex (Virtual Textile Catalogues). The goal of this project is to develop graphics and image-processing software for the capture, storage, search, retrieval and visualisation of 3D textile samples. The ultimate objective is to develop a web-based application that allows the user to search a database for suitable textiles and to visualize selected samples using real-time photorealistic 3D animation. The main novelty of this work is in the combined use of photometric stereo and real-time per-pixel-rendering for the capture and visualisation of textile samples. Photometric stereo is a simple method that allows both bump map and colour map of a surface texture to be captured digitally. It uses a single fixed camera to obtain three images under three different illumination conditions. The colour map is the image that would be obtained under diffuse lighting. The bump map describes the small undulations of the surface relief. When imported into a standard graphics program these images can be used to texture 3D models. The appearance is particularly photorealistic, especially under changing illumination and viewpoints. The viewer can manipulate both viewpoint and lighting to gain a deeper perception of the properties of the textile sample. In addition, these images can be used with 3D models of products to provide extremely accurate visualisations for the customer. Until recently, these images could only be rendered using ray-tracing software. However, recent consumer-level graphics cards from companies such as Nvidia, ATI and 3Dlabs provide real-time per-pixel shading. We have developed software that takes advantage of the advanced rendering features of these cards to render images in real-time. It uses photometrically acquired bump and colour maps of textiles to provide real-time visualisation of a textile sample, under user-controlled illumination, pose and flex.

Resolving handwriting from background printing using photometric stereo
Pattern Recognition 36, pp1869 – 1879, 2003

Abstract
We propose a scheme to resolve handwriting from background printing. The scheme detects the indentations made by the pen in the paper. Photometric stereo is used to recover the surface; a matched filter and classifierer are used to detect the stroke indentation. We assess the e0ect of uniform and textured background s on the recovery of the stroke and test the scheme on practical examples. The technique was found to work well with script written with a ballpoint pen andcoulde0ectively suppress even dark and strongly textured backgrounds. We conclude that this is a useful complement to existing techniques for background removal and is especially useful when there is no template available.

A Comparison of Three Rough Surface Classifiers
IEE Proceedings Vision, Image and Signal Processing, Vol. 149, No.5, October 2002, pp263-271.

Abstract
In this paper texture analysis techniques are used to segment rough surfaces into regions of homogeneous texture. The performance of three rough surface classifiers was assessed and compared. The classifiers differ in their discrimination as well as their input and computational requirements. Simulation and experiment were used to identify the limitations of the classifiers and to identify which classifier is best suited to a particular task. A series of guidelines for the choice of classifier are presented and justified. 

Modelling deposition of surface texture
Electronic Letters, Vol. 37, No. 12, 7th June 2001, pp749-750.

Abstract
A class of surfaces formed by deposition are analysed and modelled. The surface derivative fields are estimated using photometric stereo and their spectra modelled. A published technique that uses the repeated addition of a primitive to the surface is tested. The quality of the model is assessed by comparing simulated images with the originals. When used with photometric stereo and a surface spectrum model the technique is useful for modelling the appearance of the surface throughout the deposition process.

Evaluating Kube and Pentland's fractal imaging model
IEEE Trans. on Image Processing Volume: 10 Issue: 4 , April 2001, pp534 -542.

Abstract
The aim of this paper is to assess the validity of a model, proposed by Kube and Pentland [Kube88], that relates a rough surface to its image texture. Simulation was used to assess whether a linear approximation is appropriate, and whether the optimal linear filter agrees with the predictions of Kube's model. The predictions of Kube's model about image directionality were also assessed on real images. It was found that a linear model is capable of modelling the imaging process for a large class of surfaces, and that, subject to a small modification, Kube's model accurately predicts the relationship between surface and image spectra.

Rough surface classification using first order statistics from photometric stereo
Pattern Recognition Letters, 21, 2000, pp593-604.

Abstract
Rough surfaces can be classified by the first order statistics of their derivative fields, estimated using photometric stereo. Such a scheme is proposed, and found to be more accurate and robust than  image-intensity based classification. It is particularly effective when applied to directional surfaces, even under rotation. The scheme is therefore robust and conomic---suitable for many applications and worthy of further investigation.

Rotation invariant classification of rough surfaces
IEE Proceedings Vis. Image and Sig. Vol. 146, No. 6, December 1999

Abstract
Rotation of a rough, textured surface will not produce a simple rotation of the image texture. It follows that where image texture is a function of surface topography existing rotation invariant texture classification algorithms are not robust to surface rotation. The effect of surface rotation on the observed image is analyzed using an existing theory, a novel scheme to stabilize classification accuracy is proposed and evaluated. The scheme uses photometric stereo to estimate the surface derivatives, which  are then used as the input to a classifier. Simulations indicate that, where the level of image noise is moderate or low, the approach is successful in maintaining classification accuracy. Furthermore, in some circumstances, the extra information used by the algorithm allows classification accuracy superior to that based on image alone-even without rotation.

Sidescan sonar: a directional filter of seabed texture?
IEE Proceedings Radar, Sonar and Navigation, Vol. 146, No. 1, February 1999, pp65-72.

Abstract
As the exploration of the seafloor is extended ever further, automated classification and interpretation of sonar images is becoming increasingly important. However, many of the image processing techniques employed for these purposes, consider the images simply as a collection of textures and neglect the process by which these images are formed. The most important aspect which is thus neglected is the directionality of the imaging process and its subsequent effect on the image textures produced. This paper reports the results of a systematic investigation of the effect of the sonar process on image texture directionality, illustrating the importance of the relative orientation of the sonar and the seabed features during the sonar image capture process. A frequency domain model for examining and quantifying these orientation effects is then developed and the subsequent effect on a classification system is demonstrated.

Illuminant tilt estimation from images of isotropic texture
IEE Proceedings Vision, Image and Signal Processing, August 1996, Vol.144, No.4, pp.213-219

Abstract
This paper presents a new illuminant tilt angle estimator that works with isotropic textures. It has been developed from a frequency domain model of images of three-dimensional texture due to Kube and Pentland [1]. It is compared with a spatial domain estimator due to Knill [2]. The frequency and spatial domain theory behind each of the estimators is related via an alternative proof of the basic phenomena that both estimators exploit: that is that the variance of the partial derivative of the image is at a maximum when the partial derivative is taken in the direction of the illuminant?s tilt. Results obtained using both simulated and real textures suggest that the frequency domain estimator is more accurate.

Why illuminant direction is fundamental to texture analysis
IEE Proceedings Vision, Image and Signal Processing, 1995, August Vol.142, No.4, pp.199-206

Abstract
This paper shows that directed illumination used in the image acquisition process can act as a directional filter of three-dimensional texture. Although these effects are fundamental to texture analysis, as many classification and segmentation schemes use directional feature measures [14, 17], it is believed that they have not been investigated before from such a perspective. An image model of texture is presented which, given the illuminant vector, may used to predict the directional characteristics of image texture. Simulations and the results of laboratory experiments are presented that confirm the predicted directional filtering effects. The image model is used to predict the output of a directional texture measure : Laws? L5E5 operator [1]. Empirical results using four samples of isotropic texture confirm that the operator's output is significantly affected by changes in the angle of tilt of the illuminant. They also show that the model provides a good basis for predicting the behaviour of such operators. Finally the effect of changes in illuminant tilt on the distributions of the operator for two isotropic textures are presented. These results show that considerable mis-classification would result in using an L5E5 based classifier if the illuminant tilt angle were changed between training and classification sessions.

Detecting changes to the Amplitude and Phase Spectra of Textured Stimuli: Effects of Display Time and Retinal Eccentricity
AVA Christmas Meeting, Paris, France, Dec 17-18, 2010

Abstract
The Discrete Fast Fourier Transform allows us to easily extract an image’s amplitude and phase spectra. The extent to which the early visual system codes image properties in terms of parameters of these spectra has been an active research topic for a number of years. Both parameters have been shown to contain important information regarding the appearance of an image [Tadmor and Tolhurst, Vis. Res., 33:141-145, 1993] and a large body of work investigated the effects of manipulating these spectra on the recognition or classification of image content. Here, we use a novel means of investigating sensitivity to amplitude and phase spectra properties, by using synthetic images of textured surfaces that are broadband in the frequency domain, and by testing the ability of observers to detect degradations of their spectral content (smoothing of the amplitude spectrum, or randomisation of the phase spectrum). We directly compare the effects of display time and retinal eccentricity on detection of these two manipulations, by using stimuli matched for difficulty of detection. We find no difference between the time-courses for the detection of degradation in the two spectra; in both cases, accuracy rises above chance with display times greater than 80 ms. Increasing retinal eccentricity to 9 deg, however, has a significantly stronger effect on the accuracy of detecting degradations of the amplitude spectrum than of the phase spectrum.

Human subjectivity to apparent randomness of surface textures
2nd CIE Expert Symposium on Appearance, Gent, Belgium, Sept 8-10, 2010

Abstract

Human perception of surface directionality
2nd CIE Expert Symposium on Appearance, Gent, Belgium, Sept 8-10, 2010

Abstract

The effect of mesoscale surface roughness on perceived gloss
2nd CIE Expert Symposium on Appearance, Gent, Belgium, Sept 8-10, 2010

Abstract

Gaze-Motivated Compression of Illumination and View Dependent Textures
Twentieth conference of the International Association for Pattern Recognition, Istanbul, August 23-26, 2010

Abstract
Abstract—Illumination and view dependent texture provide ample information on the appearance of real materials at the cost of enormous data storage requirements. Hence, past research focused mainly on compression and modelling of these data, however, few papers have explicitly addressed the way in which humans perceive these compressed data. We analyzed human gaze information to determine appropriate texture statistics. These statistics were then exploited in a pilot illumination and view direction dependent data compression algorithm. Our results showed that taking into account local texture variance can increase compression of current methods more than twofold, while preserving original realistic appearance and allowing fast data reconstruction.

Perceptually Relevant Pattern Recognition Applied to Cork Quality Detection
ICIAR 2009 (The International Conference on Image Analysis and Recognition, Canada, July 2009

Abstract

Perceived Randomness of Surface Textures,
MINET Conference, Measurement, Sensation and Cognition, National Physics Laboratory
Abstract

Stochastic Search on a Homogeneous Surface Texture
AVA Easter Meeting and AGM 2009, 31st MARCH 2009

Abstract
Visual search is often thought of as being in some sense guided (Wolfe, 2007, Integrated Models of Cognitive Systems, Ed W. Gray, 99-119). Recent work by Najemnik and Geisler (2008, Journal of Vision, 8(3), 1-14) found that human performance was close to ideal when searching for a target hiding in noise. Using similar stimuli, we present a comparison between human search performance and that of a stochastic model. We use a search task involving naturalistic continuous surface textures (Clarke et al., 2008, Vision Research, 48(21), 2193-2203) and use the results from a Signal-Detection Experiment to model target detectability. Our stochastic model makes random saccades, weighted by empirically derived saccade amplitude and direction distributions coupled with the target detection model. We compare the model against human performance in terms of the number of saccades required by seven observers to find the target, and find a close match between the two. We also use the Voronoi method (Over et al., 2006, Behaviour Research Methods 38(2), 251-261) to analyse the uniformity of the spatial distribution of fixations and find that observers perform similarly to the stochastic model. This suggests that a stochastic process is sufficient to model human search strategies, and that inhibition of return, and other memory-dependent processes, do not have a large role to play in our search task.

Perceived characteristics of 1/f^&beta noise surfaces
European Conference on Visual Perception, Utrecht, the Netherlands. August 24th-28th, 2008

Abstract
We report results from experiments investigating the relationships between physical properties of surface textures and their visually perceived roughness. The textures are of a novel kind; densely sampled two-dimensional 1/f^&beta noise processes are used to create surface height maps, which are rendered using combined point-source and ambient lighting to give images that strongly resemble surfaces of natural materials such as stone. At the same time as having a natural appearance, these images are fully parameterised and controllable. Surfaces are shown in motion to provide rich cues to their relief, and are generated in real time to enable observers to manipulate surface parameters dynamically. A method of adjustment is used to investigate the effects of the two surface parameters, magnitude roll-off factor and RMS height, on perceived roughness. The results are used to develop a model for perceived roughness of these surfaces, based on Gaussian bandpass filtering of the height spectrum. After using isotropic surfaces in these first experiments, we then extend the model to surfaces with directional height spectra.

Modelling Visual search for a Target Against a 1/f^&beta Continuous Textured Background
European Conference on Visual Perception, Utrecht, the Netherlands. August 24th-28th, 2008

Abstract
We present synthetic surface textures as a novel class of stimuli for use in visual search experi- ments. These are created from surface height maps with 1/fb properties, which are rendered to give images that strongly resemble surfaces of natural materials such as stone. These textured images have certain advantages over both photographs of natural scenes and the arrays of abstract discrete items commonly used in search studies; they appear naturalistic, yet are fully parameterised and controllable. As there is no semantic information present, these images are ideal for testing models of low-level processes in the control of visual attention. The difficulty of the search task can be modified by changing the parameters of the surface and the target. Experimental results show that the Itti ^Koch saliency model fails to replicate the sensitivity of human observers to elongated indentations in a surface when they are oriented close to the direction of illumination. We present results from an alternative model of visual search on such surfaces based on Gabor filters.

On Optimal Resampling of View and Illumination Dependent Textures
In proceedings of 5th Symposium on Applied Perception in Graphics and Visualization (APGV), ACM Press, August 2008, pp.131-134

Abstract
The use of illumination and view dependent textural information is one way to capture the realistic appearance of genuine materials. One example of such data is the bidirectional texture function. The main disadvantage of these data, that makes their further application very difficult, is their massive size. Perceptually-based methods can determine optimal uniform resampling of these data that allows considerable reduction of a number of view and illumination dependent samples. In this paper we propose to achieve this goal by means of a psychophysical study, comparing original data rendering with rendering of their uniformly resampled version over the hemisphere of illumination and view dependent textural measurements. The resampling was done separately for elevation and azimuthal angles as well as in illumination and view space. Our results shown promising consequences for compression and modeling algorithms using this kind of massive data.

Perceived Directionality of 1/ƒ^&beta Noise Surfaces.
In proceedings of 5th Symposium on Applied Perception in Graphics and Visualization (APGV), ACM Press, August 2008, pp.203

Abstract
We presents the results of a novel psychophysical investigation of the directionality of 1/ƒ^&beta noise surfaces. These surfaces, which are normally isotropic, are modified by adding a directionality term which controls the texture's directional variance. They are rendered and animated using a simple Lambertian model. The Direct Ratio Estimation method is used to derive the psychophysical scale. We show that a simple exponential is sufficient to model the relationship between the perceived directionality and a surface's directional variance.

Measurement of Perceptual Roughness in Fractal Surfaces
Proc. CIE Expert Symposium on Visual Appearance, Paris, France, October 2006.

Abstract
In this paper we present an investigation into visually perceived surface roughness. First we present psychophysical evidence that suggests that there is a simple relationship between perceived roughness and two well known surface parameters: fractal dimension and rms roughness. And that neither are good estimators, on there own, of perceived roughness. Second we present a measurement model for deriving the perceived roughness of a surface from its height function which is motivated by the spatial frequency channel model of the human visual system.

Can two specular pixels calibrate photometric stereo?
Tenth IEEE International Conference on Computer Vision (ICCV'05) Beijing, October 2005, V2, pp. 1850-1857.

Abstract
Lambertian photometric stereo with unknown light source parameters is ambiguous. Provided that the object imaged constitutes a surface, the ambiguity is represented by the group of Generalised Bas-Relief (GBR) transformations. We show that this ambiguity is resolved when specular reflection is present in two images taken under two different light source directions. We identify all configurations of the two directional lights which are singular and show that they can easily be tested for. While previous work used optimisation algorithms to apply the constraints implied by the specular reflectance component, we have developed a linear algorithm to achieve this goal. Our theory can be utilised to construct fast algorithms for automatic reconstruction of smooth glossy surfaces.

Illumination-invariant texture classification using single training images.
Texture 2005: Proceedings of the 4th international workshop on texture analysis and synthesis, pp. 31-36, Beijing, China, 2005.

Abstract
The appearance of a surface texture is highly dependent on illumination. This is why current surface texture classification methods require multiple training images captured under a variety of illumination conditions for each class. We show that a single training image per class can be sufficient if the surfaces are of uniform albedo and smooth and shallow relief, and the illumination is sufficiently far from the texture macro-normal. The performance of our approach is demonstrated on classification of 20 textures in the Pho-Tex database. For test images which are most different from the training images (different instances of the same texture observed, non-equal illumination slants), the success rate achieved is in the range of 60–80%. When the test images differ from the training ones only in illumination tilt, the success rate achieved is well above 95%.

On optimal light configurations in photometric stereo
Proc. IEEE International Conference on Computer Vision, Beijing, October 2005, pp. 1707-1712.

Abstract
This paper develops new theory for the optimal placement of photometric stereo lighting in the presence of camera noise. We show that for three lights, any triplet of orthogonal light directions minimises the uncertainty in scaled normal computation. The assumptions are that the camera noise is additive and normally distributed, and uncertainty is defined as the expectation of squared distance of scaled normal to the ground truth. If the camera noise is of zero mean and variance σ^2, the optimal (minimum) uncertainty in the scaled normal is 3σ^2. For case of n > 3 lights, we show that the minimum uncertainty is 9σ^2/n, and identify sets of light configurations which reach this theoretical minimum.

Self-similarity based editing of 3D surface textures
Texture 2005: Proceedings of the 4th international workshop on texture analysis and synthesis, Beijing, China, 2005.

Abstract
This paper presents inexpensive methods for selfsimilarity based editing of real-world 3D surface textures. Unlike self-similarity based 2D texture editing approaches which only make changes to pixel color or intensity values, these techniques also allow surface geometry, reflectance and other representations of the captured 3D surface textures to be edited and relit using illumination directions that differ from those of the original. A single editing operation at a given location affects all similar areas and produces changes on all images of the sample rendered under different conditions. We perform painting, cloning and warping operations over two sets of 3D surface texture representations: (1) surface height and albedo maps, and (2) eigen base images. The surface height and albedo maps are commonly used for bump mapping, and eigen base images can be used for representing 3D surface textures with complex reflectance. The result representations can be efficiently used in modern computer graphics packages for real-time applications as rendering process only requires simple graphics calculations such as weighted sums of base images.

Two-image comparison under different illumination conditions
British Machine Vision Conference, 2005.

Abstract
We present a new theory that shows that two images of the same object taken under two different light directions can be made virtually identical by filtering each image by a directional derivative filter. The direction and magnitude of the derivative is generally different for each image, and depends on illumination directions used. The method requires the object surface to be of uniform Lambertian reflectance and a shallow relief, and the light directions used to be sufficiently inclined from the surface macro-normal. For a specific case when the surface consists of spherical patches, the two images can be made identical, for any two light directions used provided that none of them is perpendicular to the viewing axis. We provide some simple experiments which illustrate the validity of this theory.

Estimating Parameters of an Illumination Model for the Synthesis of Specular Surface Textures
Proc. of The Fourth International Conference on Computer and Information Technology (CIT'04), September 14-16, 2004, Wuhan, China, pp716-721.

Abstract
This paper proposes a method to estimate the parameters of an illumination model and then uses these parameters for the synthesis of specular surface textures. We used the relationship between surface gradient maps in the frequency domain as a constraint for the separation of diffuse and specular components.
During the estimation, we always keep errors between the real images and reconstructed images as small as possible. The estimated parameters form sample surface representation maps, which are then used as inputs for the synthesis of large representation maps. The synthesized representation maps are finally relit using the illumination model to produce new images under arbitrary illumination directions.

On the Relations between Three Methods for Representing 3D Surface Textures Under Arbitrary Illumination Directions
Proc. of The Fourth International Conference on Computer and Information Technology (CIT'04), September 14-16, 2004, Wuhan, China, pp807-812.

Abstract
Representing the appearances of surfaces illuminated from different directions has long been an active research topic. While many representation methods have been proposed, the relationships between the different representations have been less well researched. These relationships are important, as they provide (a) an insight into the different capabilities of the surface representations, and (b) a means by which they may be converted to common computer graphics application formats. In this paper, we introduce a single mathematical framework and use it to express three commonly used surface texture relighting representations: Surface Gradients, Polynomial Texture Maps (PTM) and Eigen base images. The framework explicitly reveals the relations between the three methods, and from this we propose a set of conversion methods.

Optimal illumination for three-image photometric stereo acquisition of surface texture
Texture2003, The 3rd International Workshop on Texture Analysis and Synthesis, Nice, 17 Oct. 2003

Abstract
The optimal placement of the illumination for three-image photometric stereo when used for capturing 3D surface texture is derived and verified experimentally. The sensitivities of the scaled surface normal elements with respect to the input images are derived and used to provide expressions for the noise variances. An overall figure of merit is developed by considering image-based rendering (i.e. relighting) of Lambertian surfaces. This metric is optimised with respect to the illumination angles. The optimal difference between tilt angles of successive illumination vectors was found to be 120°. The optimal slant angle was found to be 90° for smooth surfaces and 55° for rough surfaces.

On Capturing 3D Isotropic Surface Texture using Uncalibrated Photometric Stereo
Texture2003, The 3rd International Workshop on Texture Analysis and Synthesis, Nice, 17 Oct. 2003

Abstract
We propose an uncalibrated method of acquiring the normal and albedo fields of an isotropic 3D surface texture. The method is 'uncalibrated' in that it also provides estimates of the unknown illumination vectors. Thus, given a set of images taken from a single viewpoint but under varied and unknown illumination, our approach returns estimates of: the surface normal field, the albedo field, and the illumination vectors. We assume the use of single point lighting and Lambertian isotropic surfaces. We also assume that the major variations in the data are in the x-y plane - due to realistically rough surfaces. We use Hayakawa's uncalibrated photometric stereo algorithm to simultaneously estimate the scaled surface normals and the illumination vectors in an arbitrary co-ordinate system. As the variance in the data is mostly concentrated in the x-y plane, the z-axis is assumed to be given by the third eigen vector. Orientation in the x-y plane is determined by applying a frequency domain method for estimating illumination tilt angles.

Comparison of Five 3D Surface Texture Synthesis Methods
Texture2003, The 3rd International Workshop on Texture Analysis and Synthesis, Nice, 17 Oct. 2003

Abstract
We present and compare five approaches for synthesising and relighting real 3D surface textures. We adapted Efros's texture quilting method and combined it with five different relighting representations, comprising: a set of three photometric images; surface gradient and albedo maps; polynomial texture maps; and two eigen based representations using 3 and 6 base images. We used twelve real textures to perform quantitative tests on the relighting methods in isolation. We developed a qualitative test for the assessment of the complete synthesis systems. Ten observers were asked to rank the images obtained from the five methods using five real textures. Statistical tests were applied to the rankings. Our conclusion is that the cheaper gradient and three-base-image eigen methods should be used in preference to the other methods; especially where the surfaces are Lambertian or near Lambertian.

Combining Gradient and Albedo Data for Rotation Invariant Classification of 3D Surface Texture
ICCV 2003, Nice, 13-16 October 2003

Abstract
We present a new texture classification scheme which is invariant to surface-rotation. Many texture classification approaches have been presented in the past that are image-rotation invariant, However, image rotation is not necessarily the same as surface rotation. We have therefore developed a classifier that uses invariants that are derived from surface properties rather than image properties. Previously we developed a scheme that used surface gradient (normal) fields estimated using photometric stereo. In this paper we augment these data with albedo information and an also employ an additional feature set: the radial spectrum.  We used 30 real textures to test the new classifier. A classification accuracy of 91% was achieved when albedo and gradient 1D polar and radial features were combined. The best performance was also achieved by using 2D albedo and gradient spectra. The classification accuracy is 99%.

Real-Time Per-Pixel Rendering of Bump-mapped Textures Captured using Photometric Stereo
Vision, Video, and Graphics 2003, Bath, UK. 10-11 July 2003

Estimating Lighting Direction and Classifying Textures
BMVC 2002, Cardiff, 2-5 Sept 2002.

Abstract
The appearance of a rough surface is affected by the direction from which it is lit and texture classifiers should account for this. We propose a classifier that is robust to lighting direction---even when the direction is unknown. An existing model of the dependency of texture features on lighting direction is used to develop a probabilistic model. Given a feature set, the algorithm estimates the most likely illumination direction for each texture class. The likelihoods of each candidate (with their estimated lighting) are compared to classify the sample. The ability of the classifier to identify illuminant direction, and to assign the correct class, was tested on 25 real texture samples. The classifier was able to accurately estimate both the azimuth and the zenith of the light source for most textures and gave a 98\% classification rate.

Capture and Synthesis of 3D Surface Texture
Texture2002, The 2nd International Workshop on Texture Analysis and Synthesis, Copenhagen, 1 June 2002, pp41-45.

Abstract
This paper presents and compares six novel approaches for capturing, synthesising and relighting real 3D surface textures. Unlike 2D texture synthesis these techniques allow the captured textures to be relit or rendered using a wide variety of illumination directions. The methods we use for synthesis of 3D surface representations are derived from a popular 2D method. Synthesis can be applied either before or after relighting. Results obtained using a limited set of real textures indicate that the best images are obtained when image-based or gradient-based relighting is used after synthesis.

Illuminant Rotation Invariant Classification of 3D Surface Textures using Lissajous`s Ellipses
Texture2002, The 2nd International Workshop on Texture Analysis and Synthesis, Copenhagen, 1 June 2002, pp103-107.

Abstract
Changes in the angle of illumination incident upon a 3D surface texture can significantly alter its appearance. Such variations affect texture feature images and can dramatically increase the failure rates of texture classifiers. In a  previous paper we presented theory and experimental results that showed that changes in illuminant tilt angle cause  texture clusters to describe Lissajous`s ellipses in feature space. In this paper we use this model to develop a classifier that can classify surface textures imaged under unknown illumination tilt angles.

The Effect of Illuminant Rotation on Texture Filters: Lissajous's Ellipses
ECCV2002, European Conference on Computer Vision 2002, Copenhagen, 27 May - 1 June, 2002, Part III, pp289-303.

Abstract
Changes in the angle of illumination incident upon a 3D surface texture can significantly change its appearance. These changes can affect the output of texture features to such an extent that they cause complete misclassification. We present new theory and experimental results that show that changes in illumination tilt angle cause texture clusters to describe Lissajous's ellipses in feature space. 
We focus on texture features that may be modelled as a linear filter followed by an energy estimation process e.g. Laws filters, Gabor filters, ring and wedge filters. This general texture filter model is combined with a linear approximation of Lambert's cosine law to predict that the outputs of these filters are sinusoidal functions of illuminant tilt. 
Experimentation with 30 real textures verifies this proposal. Furthermore we use these results to show that the clusters of distinct textures describe different elliptical paths in feature space as illuminant tilt varies. These results have significant implications for illuminant tilt invariant texture classification.

Segmentation of Rough Surfaces using Reflectance
BMVC2001, British Machine Vision Conference, 10-13 September 2001,Vol. 1, pp323-332.

Abstract
The segmentation of rough surfaces using their reflectance properties is considered.We present a technique to estimate the orientation of surface facets whose reflectance functions are unknown. The reflectance characteristics of each facet are estimated individually allowing this technique to be applied to non-homogeneous surfaces. Non-Lambertian components are attenuated allowing shape estimation with classical photometric stereo.Simulations with rough surfaces rendered with Phong's model indicate that this approach extends the range of reflectance functions to which classicalphotometric stereo can be applied.The recovered surface derivatives, together with the original intensity images are used to construct reflectance maps.These are used as features for segmentation.A reflectance based classifier is found to be more accurate than an intensity classifier.

A Comparison of Three Rough Surface Classifiers
BMVC2001, British Machine Vision Conference, 10-13 September 2001,Vol. 2, pp563-572.

Abstract
In this paper texture analysis techniques are used to segment rough surfaces into regions of homogeneous texture. The performance of three rough surface classifiers was assessed and compared. The classifiers differ in their discrimination as well as their input and computational requirements. Experiments were used to identify the failure modes of the classifiers and to identify which classifier is best suited to a particular task. A series of guidelines for the choice of classifier are presented and justified.

Recovery of Fingerprints using Photometric Stereo
IMVIP2001, Irish Machine Vision and Image Processing Conference, 5-7 September 2001, pp192-199.

Abstract
We demonstrate the use of photometric stereo to enhance the recovery of both visible and plastic fingerprints. Examples are given that show the recovery of visible prints from rough surfaces and a plastic print from a patterned surface. This approach improves the quality of the print by reducing the effects of variations in the background surface. It provides a cheap and effective way to improve the quality of the image before enhancement and recognition algorithms are applied.

Segmentation of Machined Surfaces
IMVIP2001, Irish Machine Vision and Image Processing Conference, 5-7 September 2001, pp200-207.

Abstract
This paper reports a preliminary study into the segmentation of diffusely reflective machined surfaces into regions of homogeneous texture. A published technique that uses photometric stereo and Gabor filters is tested using surfaces of known type and roughness from a surface finish comparator. We found the technique to be effective at segmenting this class of surface and to be rotation invariant. We conclude that this technique is useful for segmenting visibly rough  diffusely reflective surfaces that have been milled, ground or shape turned.

Surface rotation invariant texture classification using photometric stereo and surface magnitude spectra
BMVC2000, British Machine Vision Conference, 11-14 September 2000, Bristol, Vol. 2, pp486-487.

Abstract
Abstract-Many image-rotation invariant texture classification approaches have been presented previously. This paper proposes a novel scheme that is surface-rotation invariant. It uses magnitude spectra of the partial derivatives of the surface derived using photometric stereo.  Unfortunately the partial derivative operator is directional (it acts as a directional filter of the surface height map). It is therefore not suited to direct use as a rotation invariant feature. We present a simple frequency domain method of removing these directional artefacts. Polarograms (polar functions of spectra) are extracted from resulting spectra and classification is performed using the cross-correlation function.
A proof for the removal directional artefacts from partial derivative spectra is provided. Preliminary results obtained using the classification scheme on real textures are presented.

On the use of gradient-space eigenvalues for rotation invariant texture classification
ICPR 2000, 15th International Conference on Pattern Recognition, Barcelona, 3-8 September 2000, pp 943-946.

Abstract
Abstract-Many image-rotation invariant texture classification approaches have been presented previously. This paper proposes a novel scheme that is surface-rotation invariant. It uses the eigenvalues of a surface?s gradient-space distribution as its features. Unlike the partial derivatives, from which they are computed, these eigenvalue features are invariant to surface rotation.
First we show that a simple classifier using a single isotropic feature (grey-level standard deviation) is not invariant to surface rotation. Then a practical surface rotation invariant classifier that uses photometric stereo to estimate surface derivatives is developed.
Results for both classifiers are presented.

The response of texture features to illuminant rotation
ICPR 2000, 15th International Conference on Pattern Recognition, Barcelona, 3-8 September 2000, pp 955-958.

Abstract
Abstract-Rotation of the illuminant source about a subject  textured surface can cause catastrophic failure of texture classification schemes. This is due to the variation of texture feature output that can occur when the illuminant direction is varied. This paper uses theory and experiment to show that the outputs of linear texture filters, and their features, are sinusoidal functions of the illuminant's tilt angle.

Model-based investigation of directional characteristics of sidescan sonar
PSIP'99, Physics in Signal and Image Processing, ENST, Paris 18-19 January 1999

Abstract
Abstract-The characteristics of sidescan sonar images are dependent on the orientation of the towfish trajectory relative to the seabed. However, this effect is frequently overlooked in the automated analysis of sidescan imagery. This paper investigates a frequency domain model that predicts these directional characteristics. It then uses simulation to show that the model is reasonable even in cases where assumptions inherent in its derivation are violated. The impact of this directional filtering effect on the output of an isotropic texture feature is presented and the variation shown to be dramatic. Not surprisingly, variation of towfish trajectory, between training and classification sessions, is shown to have disastrous consequences for a conventional texture classifier.

A Model-Based Technique for the Classification of Textured Surfaces with Illuminant Direction Invariance.
British Machine Vision Conference 1997, Vol.2, pp 470-479

Abstract
In this paper, the authors present a novel scheme for the reduction of texture misclassification due to variation of the illuminant direction between the training and classification stages. The approach is model-based, using photometric techniques to form a surface description. This description is rendered under the illumination conditions at which classification will take place to simulate training data which is appropriate to the classification task. This paper uses a feature set of twelve Gabor filters and the similarity of the simulated and actual features will be assessed. The classification accuracy of the model-based technique is assessed against that of a classifier trained under only one illumination condition and another classifier retrained for each illumination condition.

Compensation of illumination tilt variation for texture classification
Fifth International Conference on image processing and its applications. 4-6 July 1995, pp.767-761

Abstract
This paper uses theory and laboratory experiment, to show that directional illumination used during the image acquisition process, acts as a directional filter of three dimensional texture. It is shown that the directional characteristics of image texture are not intrinsic to the physical texture being imaged, as they are affected by the direction of the illumination. The implications of this to texture classification are then investigated using a set of Laws' [1] operators. Finally, a scheme for the compensation of effects caused by changes in illuminant orientation is proposed and evaluated.

Illumination: A directional filter of texture?
British Machine Vision Conference 1994 Vol.2 pp.449-458.

Abstract
This paper uses theory, simulation, and laboratory experiment, to show that directional illumination, used during the image acquisition process, acts as a directional filter of three dimensional texture. It is shown that the directional characteristics of image texture are not intrinsic to the physical texture being imaged, as they are affected by the direction of the illumination. This result has important implications for texture classification schemes: as many use directional characteristics for discrimination purposes. Variation of illuminant direction is shown to significantly affect a common texture measure : the Laws' L5E5 operator [1]."