Sony Corporation

Apple Inc.

Su Wang

Akira Nakamura

Shengyang Dai

Xun Xu

Jun Yokono

Takeshi Ohashi

Alok Deshpande

Gijesh Varghese

Wu Cheng

Morten Poulsen

Tak Shing Wong

Dian Gong

Xutao Lv

Farhan A Bagai

Hao Sun

An adaptive approach to image bracket determination and a more memory-efficient approach to image fusion, which are designed to generate low noise and high dynamic range (HDR) images in a wide variety of capturing conditions, are described. An incoming preview image stream may be obtained from an image capture device. When a capture request is received, an analysis may be performed on an image from the preview image stream that has a predetermined temporal relationship to the image capture request. Based on the analysis, a set of images (and their respective capture parameters, e.g., exposure time) may be determined for the image capture device to capture. As the determined set of images are captured, they may be registered and fused in a memory-efficient manner that, e.g., places an upper limit on the overall memory footprint of the registration and fusion operations—regardless of how many images are captured in total.

Adaptive image bracket determination

Automated nuclei area/number estimation utilizes a two-stage estimation framework-area estimation first followed by number estimation. After determining area information, each local patch's shape features are able to be extracted to define a local voting rule. The resulting voting score determines the strength of each local voting peak. The number of voting peaks is exactly the number of nuclei.

Automated nuclei area/number estimation for IHC image analysis

A manifold-aware ranking kernel (MARK) for information retrieval is described herein. The MARK is implemented by using supervised and unsupervised learning. MARK is ranking-oriented such that the relative comparison formulation directly targets on the ranking problem, making the approach optimal for information retrieval. MARK is also manifold-aware such that the algorithm is able to exploit information from ample unlabeled data, which helps to improve generalization performance, particularly when there are limited number of labeled constraints. MARK is nonlinear: as a kernel-based approach, the algorithm is able to lead to a highly non-linear metric which is able to model complicated data distribution.

Manifold-aware ranking kernel for information retrieval

For digital pathology imaging, intelligent processing, such as automatic recognition or content-based retrieval, is one significant benefit that drives the wide application of this technology. Before any intelligent processing on pathology images, every image is converted into a feature vector which quantitatively capture its visual characteristics. An algorithm characterizing pathology images with statistical analysis of local responses of neural networks is described herein. The algorithm framework enables extracting sophisticated textural features that are well adapted to the image data of interest.

Characterizing pathology images with statistical analysis of local neural network responses

Content-based retrieval of digital pathology images (DPI) is a fundamental component in an intelligent DPI processing and management system. One key issue of content-based DPI retrieval is how to represent an image as a feature vector, capturing its key information that is most relevant to the goal of retrieval. A unified framework of extracting information of different levels for DPI, namely low level color information, middle level texture information and high level diagnostic information is described herein. Such information from all the levels are integrated to the end of content-based DPI retrieval.

Combining information of different levels for content-based retrieval of digital pathology images

Systems and methods for implementing a hierarchical image recognition framework for classifying digital images are provided. The provided hierarchical image recognition framework utilizes a multi-layer approach to model training and image classification tasks. A first layer of the hierarchical image recognition framework generates first layer confidence scores, which are utilized by the second layer to produce a final recognition score. The provided hierarchical image recognition framework permits model training and image classification tasks to be performed more accurately and in a less resource intensive fashion than conventional single-layer image recognition frameworks. In some embodiments real-time operator guidance is provided for an image classification task.

Systems and methods for digital image analysis

An integrated interactive segmentation with spatial constraint method utilizes a combination of several of the most popular online learning algorithms into one and implements a spatial constraint which defines a valid mask local to the user's given marks. Additionally, both supervised learning and statistical analysis are integrated, which are able to compensate each other. Once prediction and activation are obtained, pixel-wised multiplication is conducted to fully indicate how likely each pixel belongs to the foreground or background.

Integrated interactive segmentation with spatial constraint for digital image analysis

Content-based retrieval of digital pathology images (DPI) is a fundamental component in an intelligent DPI processing and management system. The fundamental procedure of the retrieval is evaluating the similarity between the query image and every image in the database with some distance function, and sorting of the latter based on their distances to the query. A novel approach to optimally combine a set of existing distance functions into a stronger distance that is suitable for retrieving DPI in a way respecting human perception of image similarity is described herein.

Discriminative distance weighting for content-based retrieval of digital pathology images

Fluorescence in situ hybridization (FISH) enables the detection of specific DNA sequences in cell chromosomes by the use of selective staining. Due to the high sensitivity, FISH allows the use of multiple colors to detect multiple targets simultaneously. The target signals are represented as colored dots, and enumeration of these signals is called dot counting. Using a two-stage segmentation framework guarantees locating all potential dots including overlapped dots.

Flourescent dot counting in digital pathology images

An interactive segmentation framework for 3-D teeth CT volumetric data enables a user to segment an entire dental region or individual teeth depending upon the types of user input. Graph cuts-based interactive segmentation utilizes a user's scribbles which are collected on several 2-D representative CT slices and are expanded on those slices. Then, a 3-D distance transform is applied to the entire CT volume based on the expanded scribbles. Bony tissue enhancement is added before feeding 3-D CT raw image data into the graph cuts pipeline. The segmented teeth area is able to be directly utilized to reconstruct a 3-D virtual teeth model.

Graph cuts-based interactive segmentation of teeth in 3-D CT volumetric data

Systems and methods for implementing a superpixel boosted top-down image recognition framework are provided. The framework utilizes superpixels comprising contiguous pixel regions sharing similar characteristics. Feature extraction methods described herein provide non-redundant image feature vectors for classification model building. The provided framework differentiates a digitized image into a plurality of superpixels. The digitized image is characterized through image feature extraction methods based on the plurality of superpixels. Image classification models are generated from the extracted image features and ground truth labels and may then be used to classify other digitized images.

Superpixel-boosted top-down image recognition methods and systems

Systems and methods for implementing a superpixel image segmentation technique using a boundary preserving distance metric are disclosed. The disclosed technique segments a digital image into superpixels comprising contiguous pixel regions sharing similar characteristics. Superpixel image segmentation techniques presented herein utilize a boundary preserving distance metric. A boundary preserving distance metric presented herein measures the similarity between two pixels of a digital image at least partially based on a boundary probability values of the two pixels and surrounding pixels.

Superpixel segmentation methods and systems

Systems and methods for implementing a multi-label image recognition framework for classifying digital images are provided. The provided multi-label image recognition framework utilizes an iterative, multiple analysis path approach to model training and image classification tasks. A first iteration of the multi-label image recognition framework generates confidence maps for each label, which are shared by the multiple analysis paths to update the confidence maps in subsequent iterations. The provided multi-label image recognition framework permits model training and image classification tasks to be performed more accurately than conventional single-label image recognition frameworks.

Digital image analysis utilizing multiple human labels

Systems and methods for implementing a multi-step image recognition framework for classifying digital images are provided. The provided multi-step image recognition framework utilizes a gradual approach to model training and image classification tasks requiring multi-dimensional ground truths. A first step of the multi-step image recognition framework differentiates a first image region from a remainder image region. Each subsequent step operates on a remainder image region from the previous step. The provided multi-step image recognition framework permits model training and image classification tasks to be performed more accurately and in a less resource intensive fashion than conventional single-step image recognition frameworks.

Digital image analysis using multi-step analysis

Methods and systems disclosed herein provide the capability to automatically process digital pathology images quickly and accurately. According to one embodiment, an digital pathology image segmentation task may be divided into at least two parts. An image segmentation task may be carried out utilizing both bottom-up analysis to capture local definition of features and top-down analysis to use global information to eliminate false positives. In some embodiments, an image segmentation task is carried out using a 'pseudo-bootstrapping' iterative technique to produce superior segmentation results. In some embodiments, the superior segmentation results produced by the pseudo-bootstrapping method are used as input in a second segmentation task that uses a combination of bottom-up and top-down analysis.

Growing community of inventors

San Jose, CA, United States of America

Su Wang