Second Sight Medical Products, Inc.

Doheny Eye Institute

Cortigent, Inc.

Avraham I Caspi

Robert Jay Greenberg

Arup Roy

Matthew J McMahon

Kelly Hobart McClure

Jessy D Dorn

Francesco Merlini

Chunhong Zhou

Neil Hamilton Talbot

Jerry Ok

Brian V Mech

Ione Fine

Pishoy Maksy

Alan Matthew Horsager

Scott Greenwald

Fatima Anaflous

Gregoire Cosendai

Ashish Ahuja

Devyani Nanduri

Gislin Dagnelie

Paul Gibson

Mark D Welch

Greg Seifert

Uday Patel

The present invention is a system for detecting, tracking and recognizing human faces in a visual prosthesis. In a visual prosthesis, the input camera is always higher resolution than the electrode array providing percepts to the subject. It is advantageous to detect, track and recognize human faces. This information can be provided to the subject by highlighting the face in the visual scene, providing auditor or vibratory notice that a human face is in the visual scene, looking up the face in a database to identify and annunciate the name of the person in the visual scene, or otherwise communicate the identity of the person, like providing a custom vibratory pattern corresponding to known individuals (like custom ring tones associated with caller ID) or automatically zooming in on a face to aid the subject in identifying the face.

Face detection tracking and recognition for a visual prosthesis

The present invention is a depth sensing visual prosthesis. By depth sensing objects within an image, items located at a certain depths from the user can be removed to declutter an image. The resulting image presents only objects within a range of depth from the visual prosthesis user. Depth sensing can be accomplished in various ways including structured light, time of flight, or preferably, a stereo camera pair.

Depth filter for visual prostheses

The present invention is a method of processing a video image in an electronic video processor, including the steps of receiving an input image having an input field of view, generating a processed image from the input image, and having an output field of view smaller than the input field of view, searching for a predetermined pattern within the input image, providing an indication when the predetermined pattern is found in the input image, zooming the processed image to the input field of view and highlighting the predetermined pattern in the processed image in response to the indication.

Pattern detection and location in a processed image

The present invention is an improved system for use of eye tracking including spatial mapping percepts in a visual prosthesis by presenting an electrically induced precept through a visual prosthesis, requesting a subject look to the direction of the percept and tracking their eye movement. Eye movement is both faster and more accurate than asking a visual prosthesis user to point to the location of a percept. This method can be beneficial in a retinal prosthesis, but is particularly useful in a cortical visual prosthesis where visual cortex does not match the retinotopic map. Methods are presented for calibrating an eye tracker. Eye tracking hardware may also be used for blanking video information base on the subject's natural blink reflex.

Uses for eye tracking in a visual prosthesis

The present invention is a fitting system with a graphical interface with specific interface screens for specific functions. Methods and devices for fitting a visual prosthesis are described. In one of the methods, threshold levels and maximum levels for the electrodes of the prosthesis are determined and a map of brightness to electrode stimulation levels is later formed. A fitting system for a visual prosthesis is also discussed, together with a computer-operated system having a graphical user interface showing visual prosthesis diagnostic screens and visual prosthesis configuration screens.

Visual prosthesis fitting

Systems and methods for detecting and displaying obstacles to visually impaired persons are disclosed. Images from a captured scene are analyzed to detect obstacles. A V-Disparity algorithm is employed to detect the ground plane and remove it from the images, thereby enhancing the capacity of retinal implants to display only obstacles. The system includes cameras to capture images; a processor with memory to detect obstacles in the captured images, to remove a ground plane, and to calculate a map of pixels representing the obstacles; and a retinal implant configured to receive the map of pixels and display the obstacles to a system user by electrically stimulating retinal nerves. The method includes synchronizing captured images and calculating a disparity map between the captured images to detect obstacles and remove a ground plane from processed images. A map of pixels representing the obstacle is transmitted to a retinal implant. Depth information may derive from stereo cameras, a time of flight camera or a structured light camera.

Methods and systems for detecting obstacles for a visual prosthesis

The present invention is a system for detecting, tracking and recognizing human faces in a visual prosthesis. In a visual prosthesis, the input camera is always higher resolution than the electrode array providing percepts to the subject. It is advantageous to detect, track and recognize human faces. Then information can be provided to the subject by highlighting the face in the visual scene, providing auditor or vibratory notice that a human face is in the visual scene, looking up the face in a database to state the name of the person in the visual scene, otherwise communication id like providing a custom vibratory pattern corresponding to known individuals (like custom ring tones associated with caller ID) or automatically zooming in on a face to aid the subject in identifying the face.

The present invention is a method of processing a video image in an electronic video processor including the steps of receiving an input image having an input field of view, generating a processed image from the input image, and having an output field of view smaller than the input field of view, searching for a predetermined pattern within the input image, providing an indication when the predetermined pattern is found in the input image, and zooming the processed image to the input field of view and highlighting the predetermined pattern in the processed image in response to the indication.

The present invention is an improved fitting and training system for a visual prosthesis. Fitting a visual prosthesis through automated means is challenging and fitting a visual prosthesis manually is tedious for clinician and patent, and provides great opportunity for error. A hybrid of computer controlled and manual fitting provides effective, efficient and controlled fitting process. The process includes testing a group of electrodes in random order by providing a prompt followed by stimulation and the patient responding if they saw a percept. After each set, a maximum likelihood algorithm is used to determine the next stimulation level, or if further stimulation is needed for each electrode.

Hybrid fitting for a visual prosthesis

Face detection, tracking, and recognition for a visual prosthesis

The present invention is generally directed to visual neural stimulation and more specifically to improved usability of a visual prosthesis, and a visual prosthesis structure easily adaptable to the eye or the brain. They system includes a pattern recognition component, and zoom component combined with an indication component for indicating the location of the pattern, such as a face, in a zoomed out image.

Pattern detection and location indication for a visual prosthesis

A visual prosthesis and a method of operating a visual prosthesis are disclosed. Neural stimulation through electrodes is controlled by spatial maps, where a grouped or random association is established between the data points of the acquired data and the electrodes. In this way distortions from the foveal pit and wiring mistakes in the implant can be corrected. Moreover, broken electrodes can be bypassed and a resolution limit can be tested, together with testing the benefit the patient receives from correct spatial mapping.

Spatial mapping for a visual prosthesis

A method of editing a video configuration file downloadable to or from a video processing unit of a fitting system for a visual prosthesis is shown. The visual prosthesis has a plurality of electrodes and the video configuration file defines mapping of a video signal captured from a camera of the visual prosthesis to an electrical signal for the electrodes. The editing controls a brightness map for an individual electrode or electrode groups, together with a temporal stimulation pattern to which an individual electrode or electrode groups are assigned. A related computer-operated system is also disclosed.

Growing community of inventors

Rehovot, Israel

Avraham I Caspi