Sony Corporation

Sony Electronics Inc

Sony Group Corporation

Sony Corporation of America

Ming-Chang Liu

Ximin Zhang

Liangyin Yu

Mark Allen Robertson

Ko-Kai Albert Huang

Bi Song

Chen-Rui Chou

Kuang-Man Huang

Yoshihiro Murakami

Yutaka Yoneda

Toru Kurata

Ikuo Tsukagoshi

Gazi Ali

Xiaojing Chen

Su-Hui Chiang

Liyu Gong

Chuen-Chien Lee

Wei-Jen Li

Albert Huang

Seiji Kobayashi

Alexander Berestov

Shigeatsu Yoshioka

Tatsumi Sakaguchi

Peng Lin

Yoichi Mizutani

Xun Xu

Masato Kajimoto

Quan Yuan

Wei Ji

Michael Zordan

Joshua Scott Hobson

Da Li

Ahmad Khodayari-Rostamabad

A framework that includes a feature extractor and a cluster component for clustering is described herein. The framework supports (1) offline image-based unsupervised clustering that replaces time-consuming manual gating; (2) online image-based single cell sorting. During training, one or multiple cell image datasets with or without ground truth are used to train feature extractor, which is based on a neural network including several convolutional layers. Once trained, the feature extractor is used to extract features of cell images for unsupervised cell clustering and sorting. In addition, additional datasets may be used to further refine the feature extractor after it has been trained.

Framework for image based unsupervised cell clustering and sorting

An efficient and robust high-speed neural networks for cell image classification is described herein. The neural networks for cell image classification utilize a difference between soft-max scores to determine if a cell is ambiguous or not. If the cell is ambiguous, then the class is classified in a pseudo class, and if the cell is not ambiguous, the cell is classified in the class corresponding to the highest class score. The neural networks for cell image classification enable a high speed, high accuracy and high recall implementation.

Efficient and robust high-speed neural networks for cell image classification

A method for automatically estimating cellularity in a digital pathology slide image includes: extracting patches of interest from the digital pathology slide image; operating on each patch using a trained first deep convolutional neural network (DCNN) to classify that patch as either normal, having an estimated cellularity of 0%, or suspect, having a cellularity roughly estimated to be greater than 0%; operating on each suspect patch using a second DCNN, trained using a deep ordinal regression model, to determine an estimated cellularity score for that suspect patch; and combining the estimated cellularity scores of the patches of interest to provide an estimated cellularity for the digital pathology slide image at a patch-by-patch level.

Automatic estimation of tumor cellularity using a DPI AI platform

An image-based classification workflow uses unsupervised clustering to help a user identify subpopulations of interest for sorting. Labeled cell images are used to fine-tune the supervised classification network for a specific experiment. The workflow allows the user to select the populations to sort in the same manner for a variety of applications. The supervised classification network is very fast, allowing it to make real-time sort decisions as the cell travels through a device. The workflow is more automated and has fewer user steps, which improves the ease of use. The workflow uses machine learning to avoid human error and bias from manual gating. The workflow does not require the user to be an expert in image processing, thus increasing the ease of use.

Classification workflow for flexible image based particle sorting

A method for super-resolution of block-compressed texture is provided. The method includes receiving a first texture block of a first block size. Based on application of a first block compression (BC) scheme on the received first texture block, coded-texture values are generated in a compressed domain. Further, a first machine learning model is applied on the generated coded-texture values to generate super-resolution coded-texture values in the compressed domain. The generated super-resolution coded-texture values are processed to generate a second texture block of a second block size. The second block size is greater than the first block size.

Super-resolution of block-compressed texture for texture mapping applications

A method of tumor detection and segmentation accepts a first Whole Slide Image (WSI) having a first resolution; creates a corresponding second WSI having a second resolution lower than the first resolution; applies an adaptive thresholding technique to the second WSI to create a background removal mask background; applies the mask to the first WSI to provide a third WSI with extracted patches, characterized by a third resolution, greater than the second resolution and lower than the first resolution; uses a first machine learning system on the third WSI to create a heat map at the third resolution, indicating a subset of the patches likely to include one or more clusters of tumor cells; and uses a second machine learning system on the first WSI and the heat map to segment each patch in a corresponding output image at the first resolution, outlining one or more corresponding clusters.

Tumor detection and segmentation for DPI AI platform routes

Implementations generally relate to determining the position of a tool tip of a surgical instrument and the orientation of the surgical instrument. In some implementations, a method includes receiving at least one image frame of a plurality of image frames. The method further includes detecting a surgical tool in the at least one image frame. The method further includes determining a position of a tip of the surgical tool. The method further includes determining an orientation of the surgical tool.

Surgical tool tip and orientation determination

System for performing fully automatic brain tumor and tumor-aware cortex reconstructions upon receiving multi-modal MRI data (T1, T1c, T2, T2-Flair). The system outputs imaging which delineates distinctions between tumors (including tumor edema, and tumor active core), from white matter and gray matter surfaces. In cases where existing MRI model data is insufficient then the model is trained on-the-fly for tumor segmentation and classification. A tumor-aware cortex segmentation that is adaptive to the presence of the tumor is performed using labels, from which the system reconstructs and visualizes both tumor and cortical surfaces for diagnostic and surgical guidance. The technology has been validated using a publicly-available challenge dataset.

System of joint brain tumor and cortex reconstruction

An electronic device that handles recognition of mental behavioral, affect, emotional, mental states, mental health, or mood-based attributes based on deep neural networks (DNNs), stores a set of EEG signals and a set of bio-signals associated with a subject. The electronic device trains a plurality of first recognition models on a training set of EEG signals and a training set of bio-signals associated with different training subjects. The electronic device trains a second recognition model on a feature vector from output layers of the plurality of first recognition models. The electronic device estimates a plurality of dependency or relationship data by application of the trained plurality of first recognition models on the set of EEG signals and bio-signals. The electronic device identifies a mental behavioral attribute of the subject by application of the trained second recognition model on the plurality of signals and their relationship data.

Electronic device for recognition of mental behavioral attributes based on deep neural networks

A framework combines vision and sample-efficient reinforcement-learning based on guided policy search for autonomous driving. A controller extracts environmental information from vision and is trained to drive using reinforcement learning.

Vision-based sample-efficient reinforcement learning framework for autonomous driving

A method for automatic organ segmentation in CT images comprises in a first step, rough region segmentation; in a second step, coarse organ segmentation; and in a third step, refinement of organ segmentation. The organ may be a liver. Rough region segmentation may comprise applying standard anatomical knowledge to the CT images. Coarse segmentation may comprise identifying organ voxels using a probabilistic model. Refinement of organ segmentation may comprise refinement based on intensity, followed by refinement based on shape. Apparatuses configured to carry out the method are also disclosed.

Automatic liver segmentation in CT

Implementations generally relate to determining tool handedness for surgical videos. In some implementations, a method includes receiving at least one image frame of a plurality of image frames. The method further includes detecting one or more objects in the at least one image frame. The method further includes classifying the one or more objects into one or more tool classifications, where the one or more objects are tools. The method further includes determining, for each tool, if the tool is assistive or non-assistive. The method further includes determining, for each tool, a handedness of the tool.

Tool handedness determination for surgical videos

Blind legacy video artifact reduction reduces ghost artifacts and Y/C artifacts. Ghost artifacts are reduced using edge correlation and an IIR filter. Y/C artifacts are reduced using frequency domain filtering. The result is much cleaner video content.

Blind legacy video artifact reduction

Implementations generally relate to detecting dominant tools in surgical videos. In some implementations, a method includes receiving at least one image frame. The method further includes detecting one or more objects in the at least one image frame. The method further includes classifying the one or more objects into one or more tool classifications, where the one or more objects are tools. The method further includes determining a handedness of the one or more tools. The method further includes determining a dominant tool from the one or more tools based at least in part on the one or more classifications of the one or more tools and based at least in part on the handedness of the one or more tools.

Dominant tool detection system for surgical videos

A method for 2D feature tracking by cascaded machine learning and visual tracking comprises: applying a machine learning technique (MLT) that accepts as a first MLT input first and second 2D images, the MLT operating on the images to provide initial estimates of a start point for a feature in the first image and a displacement of the feature in the second image relative to the first image; applying a visual tracking technique (VT) that accepts as a first VT input the initial estimates of the start point and the displacement, and that accepts as a second VT input the two 2D images, processing the first and second inputs to provide refined estimates of the start point and the displacement; and displaying the refined estimates in an output image.

Growing community of inventors

San Jose, CA, United States of America

Ming-Chang Liu