National Ict Australia Limited

Antonio Robles-Kelly

Cong Phuoc Huynh

Lin Gu

Johannes Jordan

Jun Zhou

Ahmed Sohaib

Lawrence Mutimbu

Nariman Habili

Pattaraporn Khuwuthyakorn

This disclosure relates to processing colour images to reconstruct hyperspectral images from the colour images. An image processor determines an output hyperspectral image by determining for each point of the output hyperspectral image a combination of multiple hyperspectral prototype components to correspond the output hyperspectral image to the input image. Each of the multiple hyperspectral prototype components comprises multiple component values associated with respective wavelengths and each of the multiple component values is based on multiple points of training image data associated with the wavelength of that component value. Since the component values are based on multiple points of the training image, relationships between points in the training image, such as texture, can be used to reconstruct the hyperspectral image data, which leads to a more robust and more accurate reconstruction.

Spectral reconstruction

The disclosure concerns estimating an illumination spectrum of a pixel of hyperspectral or multispectral image data. A processor determines an optimised illumination spectrum for a first point based on an initial illumination spectrum and based on associations between the first point and multiple second points of multiple instances of the image data, wherein each of the multiple instances is associated with a refined amount of image information. It is possible to determine the illumination spectrum per pixel of the original image data without clustering pixels or restrictive assumptions but instead by utilising the associations for the statistical inference on the illumination spectrum.

Pixel wise illuminant estimation

This disclosure concerns the determination of electronic multispectral or hyperspectral images, such as by a processor in a camera. The processor receives first image sensor signals that are associated with a first wavelength and with multiple first imaging elements at respective first imaging element locations. The processor determines a spline approximation to the first image sensor signals across the first imaging element locations and determines the multispectral or hyperspectral image data by interpolating the spline approximation at the pixel location. Finally, the image data is stored on a data store. Using splines to interpolate the image sensor signals outperforms other methods, which means increased image quality due to more accurate interpolation. More specifically, using splines leads to a more accurate spectrum at each point and reduced spectral error compared to other methods.

Determining multispectral or hyperspectral image data

This disclosure concerns the determination of improved exposure parameters of an image capturing device, such as a multi-spectral or hyper-spectral camera. A processor receives or determines image values for multiple points of an image and receives first one or more exposure parameters that were used to generate the image values. The processor then determines a distribution of the image values. For each point of the image data the processor then determines an enhanced value by equalizing the distribution of the image values. Finally, the processor determines the one or more improved exposure parameters of the image capturing device, such that the one or more improved exposure parameters adjust the received or determined image values towards the enhanced image values. When the exposure parameters are used for capturing a further image, the further image will be enhanced.

Automatic exposure control for cameras

The disclosure concerns processing hyperspectral or multispectral images. Image data comprises a sampled image spectrum represented by first values for each pixel location representative of an intensity associated with a wavelength index. A processor determines for each pixel location second values based on a measure of similarity between pixel locations with respect to the first values such that two pixel locations that 5 are similar with respect to the first values are also similar with respect to the second values. The processor then stores for each pixel location the determined one or more second values associated with that pixel location on a data store. This way, the image data is made suitable for applications, such as clustering or displaying, while pixels that are similar in the input image are also similar in the output data. This means that a 10 structure between the pixels in the input image is preserved.

Processing hyperspectral or multispectral image data

This disclosure concerns processing of electronic images, such as hyperspectral or multispectral images. A processor performs a method for estimating an illumination spectrum of an image which defines an input spectrum for each point of the image. The processor determines for each point of a first set of points a measure of variation in relation to the input spectrum of that point. The processor then selects one point from the first set based on the measure of variation. The processor determines a cluster of points based on the input spectrum of the selected point and determines an estimate for the illumination spectrum based on the input spectra of points in the cluster. Since the processor selects a point based on the measure of variation and then determines a cluster based on that point, it performs better than using other methods where the cluster is determined based on a random starting point, for example.

Image clustering for estimation of illumination spectra

The disclosure concerns processing of electronic images, such as hyperspectral or multispectral images () to determine an output color value for a component spectrum of a decomposed multispectral or hyperspectral image. A processor () first determines or accesses the component spectrum and a first color value for the component spectrum and receives an input color value for a pixel location in the image, wherein the component spectrum contributes to the pixel. The processor () then determines the output color value for the component spectrum based on the first color value for the component spectrum, the input color value and a utility score and stores the output color value for the component spectrum on a datastore (). It is an advantage that the processor () receives a color value for a pixel location but the output color value is not exactly the received color value but one that is determined based on a utility score.

Determining color values in hyperspectral or multispectral images

The disclosure concerns processing of electronic images, such as hyperspectral, or multispectral images. In particular, but is not limited to, a method, software and computer for estimating shape information or a photometric invariant of a location of image of a scene. The image data () indexed by wavelength &#x3bb; and polarization filter angle &#x3b8;. For each wavelength &#x3bb; index, a polarization angle &#x3c6; is estimated from the image data () by the processor (). The processor () then also estimates the shape information (such as azimuth &#x3b1;, such as zenith &#x3b8;, or surface normal) or photometric invariants (such as refractive index) based on the estimated polarization angle &#x3c6; for each wavelength index &#x3bb;. Greater accuracies can be achieved in the estimated shape information and/or photometric invariants by using wavelength-indexed data. Further, surface information or photometric invariant can be estimated based upon polarization in a single-view hyperspectral or multi-spectral imagery. Further, by relying on the polarization angle for the estimation, the method is insensitive to changes in illumination power and direction.

Shape and photometric invariants recovery from polarisation images

The disclosure concerns processing of electronic images, such as hyperspectral or multispectral images. Processing of the images includes object recognition and image enhancement in a way that harnesses the information available from the image based on the wavelength indexed spectral data these types of images provide. Illumination spectrum of such an image is estimated (FIG.) using a cost function based on a dichromatic reflection model and a constraint term. This method may be performed on sub-images of the image. A method for selecting these sub-images (FIG.) is also disclosed. A method of determining photometric parameters of the image given the estimated illumination spectrum (FIG.) is also disclosed.

Illumination spectrum recovery

The disclosure concerns processing of electronic images, such as hyperspectral, multispectral or trichromatic images. In particular, but is not limited to, a method, software and computer for estimating parameters of a reflectance model applied to an image is disclosed. Examples of processing of the images using the estimated parameters includes material recognition, re-coloring and re-shading of objects represented in the image. That is, a computer implemented method is provided of estimating one or more of photogrammetric parameters,surface shapeand index of refractionrepresented in a reflectance image having one or more known illumination directionsand a known viewing direction, the method comprising optimizing () the difference between the reflectance image and a reflectance model, the reflectance model being based on surface shape; the material index of refractionand a set of photogrammetric parameters.

The disclosure concerns processing of electronic images, such as hyperspectral or multispectral images. In particular, but is not limited to methods, software and computer systems for determining underlying spectra of an image of a scene. The image data comprises for each pixel location a sampled image spectrum that is a mixture of plural reflectance spectra. Processordetermines or accesses plural hyperplanes that each have plural linearly independent basis vectors. Each hyperplane represents an estimate of one of the plural reflectance spectra. The processorthen determines for each pixel location, a contribution of the plural basis vectors of each hyperplane to the image spectrum of that pixel location. The processordetermines or accesses plural hyperplanes and not plural endmembers directly. Hyperplanes are two-dimensional while endmembers are only one-dimensional. As a result, hyperplanes carry more information, such as the illumination spectrum and are therefore of greater use. The decomposition of the image data into hyperplanes is possible without knowing the illumination spectrum. This decomposition allows for a range of new applications such as endmember extraction and compact representation.

Decomposing hyperspectral or multispectral image data

The invention concerns the compact representation of a reflectance spectrum of a material. For example, for in compression, identification and comparison of reflectance spectrum data of multiple materials. The compressed representation interpolating a spline curve to the reflectance spectrum data, the spline curve having a set of control points, a knot vector, and representing wavelength and reflectance as functions of an independent parameter (). Then removing one or more knots from the knot vector that minimise a cost function in a parameter domain of the spline curve based on the wavelength function (). Aspects of the invention include a method, software, a computer system and the compact representation itself.

Compact representation of a reflectance spectrum

The disclosure concerns hyperspectral and multispectral image. The disclosure concerns the computation of a descriptor for such an image. The descriptor can be used in a range of image post-processing applications, such as classification, recognition, tracking and localization. Aspects of the invention include a computer implemented method and a computer system. To computer the descriptor for a set of wavelength indexed bands, a band representation of the radiance measurements for that band based on a probability distribution function is computed. Then a wavelength-dependent cross-correlation between the band representations is computed. Finally, a descriptor based on the computed cross-correlation between the band representations is computed. It is an advantage that the descriptor captures a discriminative and descriptive representation of scene captured in the hyperspectral or multispectral image.

Growing community of inventors

Eveleigh, Australia

Antonio Robles-Kelly