Gm Global Technolgoy Operations, Inc.

Apple Inc.

Focal Cool, LLC

Moshe Laifenfeld

Igal Bilik

Tal Kaitz

Cristiano L Niclass

Alexander Pokrass

Shingo Mandai

Nadav Lavi

Kobi Jacob Scheim

Eli Tzirkel-Hancock

Thierry Oggier

Andrei Kolin

Eitan Hirsh

Steven Porter Hotelling

Jerrold Von Hauck

Lynn Richard Youngs

Ivan Krstic

Anup K Sharma

Jonathan Pokrass

Thorsten Gernoth

Feng Tang

Thomas P Mensch

Michael Ross Malone

Matthew C Waldon

Lucia Elena Ballard

Tal Philosof

Gregory Guyomarc'h

Mark N Gamadia

Petr Kostka

Deepti S Prakash

Christopher G Zeleznik

Gideon Littwin

Zeev Roth

Ziv Hendel

Lucie Kucerova

Etai Littwin

Timothy J Talty

Andrew David Fernandez

Eilon Riess

Timothy D Julson

Abhishek Dhanda

Vyacheslav Berezin

Gill Rafael Tsouri

Pavan Kumar Anasosalu Vasu

Or Nahir

Mendy Shoval

Vyshakh Sanjeev

Sagy Zino

David Silver

Alexander Shpunt

Mina A Rezk

Scott M Reilly

Yingjun Bai

Shahar Villeval

Andrew Kenneth John McMahon

Kelsey Yukio Ho

Kevin M Baltes

Igal Kotzer

Andrew J Sutton

Todd J Nilsen

Vladimir Suplin

Ronen Akerman

Oriel Bergig

Gary W Taraski

Yohay Buchbut

Oz Barak

Shaun S Marshall

Harish Venkataraman

Susan Ann Thompson

Rahmi Hezar

Richard E Bills

Gary Chung

Gregory M Castillo

Bernhard Buettgen

Mark Andrew Steffka

Douglas M Kidd

Eviatar Tron

Jason R Holloway

Jason D Mudge

Scott W Piper

Yuval Polevoy

Omer Cohen

Denise R Merrill

Jennifer Ellen Akers

Kyle Stever

Ravi Ramachandran

Darshan Shrestha

Doron Shinbox

Zhongmin Wang

Connor Henley

Pavan Kumar Ansosalu Vasu

Alexandra H Chau

Nir Ben Zadok

Ron Sokolovsky

Inna Stainvas

Andrew Kenneth John Mcmahon

Imaging apparatus includes a camera, which includes an autofocus mechanism. A range sensor includes a transmitter, which includes multiple emitters configured to transmit optical pulses at different angles toward the scene, and a receiver, to receive and sense respective times of flight of the optical pulses that are reflected from the scene. A photodetector outputs a signal indicative of an intensity of reflections from the scene. Control circuitry drives the emitters in alternation to transmit the optical pulses, estimates a range to a target in the scene responsively to the times of flight sensed by the receiver, detects a difference in the signal output by the photodetector in response to the pulses transmitted by different ones of the emitters, and sets the autofocus mechanism responsively to the estimated range as long as the detected difference is no greater than a predefined threshold difference.

Detection of close-range specular targets

Sensing apparatus includes a radiation source, which emits pulses of optical radiation toward multiple points in a target scene. A receiver receives the optical radiation that is reflected from the target scene and outputs signals that are indicative of respective times of flight of the pulses to and from the points in the target scene. Processing and control circuitry selects a first pulse repetition interval (PRI), a second PRI, greater than the first PRI, and a third PRI, greater than the second PRI, from a permitted range of PRIs, drives the radiation source to emit sequences of the pulses at the first PRI, the second PRI, and the third PRI, and processes the signals output by the receiver in response to the first, second, and third sequences of the pulses in order to compute respective depth coordinates of the points in the target scene.

Selection of pulse repetition intervals for sensing time of flight

Depth sensing apparatus includes a radiation source, which emits a first array of beams of light pulses through a window toward a target scene. Objective optics image the target scene onto a second array of sensing elements, which output signals indicative of respective times of incidence of photons. A first calibration, which associates the beams in the first array with respective first locations on the second array onto which the beams reflected from the target scene are imaged, is used in processing the signals in order to measure respective times of flight of the light pulses. A second calibration indicates second locations on which stray radiation is incident on the second array due to reflection of the beams from the window. Upon detecting a change in the second locations relative to the second calibration, the first calibration is corrected so as to compensate for the detected change.

Depth sensor calibration using internal reflections

A method of building a moving average histogram of photon times of arrival includes, for each time interval in first and second subsets of time intervals, latching a time reference corresponding to a time of receipt of an avalanche timing output signal of a single-photon avalanche diode (SPAD), and advancing a count stored at a memory address corresponding to the latched time reference. The memory address corresponds to a range of time references. The method further includes reading and clearing a first set of counts after the first subset of time intervals; phase-shifting the sequence of time references with respect to a set of memory addresses after the first subset of time intervals; reading and clearing a second set of counts after the second subset of time intervals; and building the moving average histogram using at least the first and second sets of counts.

SPAD-based photon detectors with multi-phase sampling TDCs

Imaging apparatus () includes a radiation source (), which emits pulsed beams () of optical radiation toward a target scene (). An array () of sensing elements outputs signals indicative of respective times of incidence of photons on the sensing elements. Objective optics () form a first image of the target scene on the array of sensing elements. An image sensor () captures a second image of the target scene. Processing and control circuitry () is configured to process the second image so as to detect a relative motion between at least one object in the target scene and the apparatus, and which is configured to construct, responsively to the signals from the array, histograms of the times of incidence of the photons on the sensing elements and to adjust the histograms responsively to the detected relative motion, and to generate a depth map of the target scene based on the adjusted histograms.

Enhanced depth mapping using visual inertial odometry

Various embodiments disclosed herein include techniques for determining autofocus for a camera on a mobile device. In various instances, a depth imaging system (e.g., a time-of-flight autofocus system (ToF-AF system)) is used to determine distance of a subject in order to determine autofocus for a camera. In some instances, however, the ToF-AF system may be unable to detect a subject that is very close to the camera when the objects are below a minimum detectable distance of the ToF-AF system. In such instances, an existing IR detector outside of the ToF-AF system may be implemented to measure reflected signals from the ToF-AF system. A power ratio may be determined from the reflected signals and used to determine information about the distance of the subject from the camera.

Near distance detection for autofocus

Imaging apparatus () includes a radiation source (), which emits pulsed beams () of optical radiation toward a target scene (). An array () of sensing elements () output signals indicative of respective times of incidence of photons in a first image of the target scene that is formed on the array of sensing elements. An image sensor () captures a second image of the target scene in registration with the first image. Processing and control circuitry () identifies, responsively to the signals, areas of the array on which the pulses of optical radiation reflected from corresponding regions of the target scene are incident, and processes the signals from the sensing elements in the identified areas in order measure depth coordinates of the corresponding regions of the target scene based on the times of incidence, while identifying, responsively to the second image, one or more of the regions of the target scene as no-depth regions.

Calibration of a depth sensing array using color image data

Techniques are disclosed relating to biometric authentication, e.g., facial recognition. In some embodiments, a device is configured to verify that image data from a camera unit exhibits a pseudo-random sequence of image capture modes and/or a probing pattern of illumination points (e.g., from lasers in a depth capture mode) before authenticating a user based on recognizing a face in the image data. In some embodiments, a secure circuit may control verification of the sequence and/or the probing pattern. In some embodiments, the secure circuit may verify frame numbers, signatures, and/or nonce values for captured image information. In some embodiments, a device may implement one or more lockout procedures in response to biometric authentication failures. The disclosed techniques may reduce or eliminate the effectiveness of spoofing and/or replay attacks, in some embodiments.

Biometric authentication techniques

Various embodiments disclosed herein include techniques for determining autofocus for a camera on a mobile device. In some instances, depth imaging is used to assist in determining a focus position for the camera through an autofocus process. For example, a determination of depth may be used to determine a focus position for the camera. In another example, the determination of depth may be used to assist another autofocus process.

Camera autofocus using time-of-flight assistance

An optical sensing device includes a light source, which emits one or more beams of light pulses toward a target scene at respective angles about a transmit axis of the light source. A first array of single-photon detectors output electrical pulses in response to photons that are incident thereon. A second array of counters count the electrical pulses output during respective count periods by respective sets of one or more of the single-photon detectors. Light collection optics form an image of the target scene on the first array along a receive axis, which is offset transversely relative to the transmit axis, thereby giving rise to a parallax shift as a function of distance between the target scene and the device. Control circuitry sets the respective count periods of the counters, responsively to the parallax shift, to cover different, respective time intervals following each of the light pulses.

Close-range measurement of time of flight using parallax shift

Sensing apparatus includes a radiation source, which emits pulses of optical radiation toward multiple points in a target scene. A receiver receives the optical radiation that is reflected from the target scene and outputs signals that are indicative of respective times of flight of the pulses to and from the points in the target scene. Processing and control circuitry selects a first pulse repetition interval (PRI) and a second PRI, greater than the first PRI, from a permitted range of PRIs, drives the radiation source to emit a first sequence of the pulses at the first PRI and a second sequence of the pulses at a second PRI, and processes the signals output in response to both the first and second sequences of the pulses in order to compute respective depth coordinates of the points in the target scene.

Depth sensing apparatus includes a radiation source, which is configured to emit a first plurality of beams of light pulses toward a target scene. An array of a second plurality of sensing elements is configured to output signals indicative of respective times of incidence of photons on the sensing element, wherein the second plurality exceeds the first plurality. Light collection optics are configured to image the target scene onto the array of sensing elements. Processing and control circuitry is coupled to receive the signals from the array and is configured to search over the sensing elements in order to identify, responsively to the signals, respective regions of the array on which the light pulses reflected from the target scene are incident, and to process the signals from the identified regions in order determine respective times of arrival of the light pulses.

Calibration of depth sensing using a sparse array of pulsed beams

Techniques for the modification of at least part of a target image (e.g., scene objects within the target image), e.g., to make the target image appear that it was captured at a different time (e.g., a different time of day, different time of year) are disclosed. This 'dynamic re-timing' of the target image may be achieved by finding one or more source images including the same (or similar) scene depicted in the target image (but, e.g., captured at different times), extracting stylistic elements from the source image(s), and then modifying at least part of the target image in a realistic fashion (e.g., not altering the geometry of objects in the target image), based on one or more extracted stylistic elements from the source image(s). Three-dimensional modeling of scene objects may allow a more realistic-looking transfer of the extracted stylistic elements onto scene objects in the target image to be achieved.

Dynamic image re-timing

Growing community of inventors

Haifa, Israel

Moshe Laifenfeld