Essential Products, Inc.

Google Inc.

Matthew J Hershenson

Andrew E Rubin

Xiaoyu Miao

David John Evans, V

Xinrui Jiang

Jason Sean Gagne-Keats

Joseph Anthony Tate

Rebecca Schultz Zavin

Michael Kolb

Joe Freeman Britt, Jr

Eugene Koh

Richard Scott Bartlett

David John Evans V

Paul Heninwolf

Robert Dale Newberry, Jr

Certain aspects of the technology disclosed herein integrate a camera with an electronic display. An electronic display includes several layers, such as a cover layer, a color filter layer, a display layer including light emitting diodes or organic light emitting diodes, a thin film transistor layer, etc. The camera can include a plurality of noncontiguous pixel regions disposed beneath the display layer and be configured to capture a plurality of images corresponding to the plurality of noncontiguous pixel regions. In some embodiments, the plurality of images can form a composite image with depth.

System comprising a display and a camera that captures a plurality of images corresponding to a plurality of noncontiguous pixel regions

Certain aspects of the technology disclosed herein integrate a camera with an electronic display. An electronic display includes several layers, such as a cover layer, a color filter layer, a display layer including light emitting diodes or organic light emitting diodes, a thin film transistor layer, etc. In one embodiment, the layers include a substantially transparent region disposed above the camera. The substantially transparent region allows light from outside to reach the camera, enabling the camera to record an image. In another embodiment, the color filter layer does not include a substantially transparent region, and the camera records the light from the outside colored by the color filter layer. According to another embodiment, while none of the layers include a substantially transparent region, the layers are all substantially transparent, and the camera disposed beneath the layers records light reaching the camera from outside.

Display device comprising a touch sensor formed along a perimeter of a transparent region that extends through a display layer and exposes a light sensor

Various embodiments concern sensors and other components that can be disposed beneath a variable transparency layer of a mobile device. By modifying how much voltage is applied to the variable transparency layer, a component, such as a camera, can be readily hidden when not in use. More specifically, the variable transparency layer may be substantially opaque when the camera is not in use and at least partially transparent when the camera is in use and ready to capture an image. The opacity level of the variable transparency layer can be modified by a voltage source that is electrically coupled to the variable transparency layer. The various levels of opacity could also enable the variable transparency layer to act as an electronic aperture for the camera.

Increasing display size by placing optical sensors beneath the display of an electronic device

Some embodiments include a mobile device with a light sensor overlaid under a display (e.g., a touchscreen). The mobile device can identify a command to capture an image. The mobile device can adjust at least a target portion of an opaqueness adjustable region of the display directly over the light sensor. The opaqueness adjustable region is capable of transforming from substantially opaque to substantially transparent. The mobile device can capture the image utilizing at least the light sensor while the target portion of the opaqueness adjustable region is substantially transparent.

Mobile device with display overlaid with at least a light sensor

In one embodiment, a light sensor, such as a camera, records an image through the surface with the residue to produce a stained image. A processor associated with the camera identifies object outlines within the image using a machine learning model, and smooth the colors within the object outlines. In another embodiment, the light sensor is placed beneath a dual-mode region of a display containing the residue. The dual-mode region can be opaque and function as part of the display, or can be transparent and allow environment light to reach the light sensor. Initially, the processor determines the position of the residue by causing the dual-mode region to display a predetermined pattern, while the light sensor records the predetermined pattern. Using the determined position of the residue, the processor corrects the pixels within the residue in the recorded image, by interpolating the values of the pixels outside of the residue.

Sensing light through a surface with a residue

Various embodiments concern techniques for intelligently switching between multiple sources of USB signals. More specifically, user devices are described that include a physical USB port for receiving a USB connector and one or more wireless transceivers that communicate with an accessory. The wireless transceiver(s) may communicate with the accessory using a USB-based protocol (e.g., Wireless USB). The user devices described herein can intelligently switch between these different sources of USB signals so that USB signals can be simultaneously or sequentially received from a peripheral (via the USB port) and an accessory (via the wireless transceiver(s)). In some embodiments, a switching routine is executed (e.g., by a processor or signal switch) that determines which peripheral and/or accessory is connected to a user device at a given point in time.

Systems and techniques for intelligently switching between multiple sources of universal serial bus signals

Various embodiments concern a wireless accessory bus for a user device that enables accessories to be easily and securely attached to the user device. More specifically, the wireless accessory bus enables data to be wirelessly transmitted between the accessory and the user device when the accessory and the user device are located within close proximity to one another (e.g., when the accessory is securely attached to the wireless accessory bus). Power could also be wirelessly transferred from the user device to the accessory (or vice versa). In some embodiments, the user device includes fastening component(s) (e.g., magnets) that allow the user device and the accessory to be magnetically secured to one another. The accessory typically enables the user device to readily utilize a new functionality or an improvement to an existing functionality.

Accessories for electronic user devices having wireless accessory buses

Presented here are manufacturing techniques to create an irregularly shaped electronic display, including a hollow within which a sensor, such as a camera, can be placed. The manufacturing techniques enable the creation of the hollow anytime during the manufacturing process. The resulting electronic display occupies the full side of the mobile device, with the sensors placed within and surrounded by the display.

Hollowed electronic display

Variable transparency layers for electronic devices

A mobile device can include electronic circuitry and a chassis shell adapted to protect the electronic circuitry. The chassis shell includes an opening exposing a tray inside the mobile device. The tray can include an external card slot and a patterned wall. The tray can be disposed within a cavity of a support structure of the mobile device. An input/output (I/O) component, such as a speaker, and a card reader can also be disposed within the cavity. The I/O component can communicate via at least a hole in the patterned wall (e.g., a speaker grill).

Conjoining functional components of a mobile device

Various embodiments concern fingerprint sensors for mobile devices. By modifying one or more layers of a display assembly, a fingerprint sensor can be positioned directly beneath the display. For example, the air gap within the backlight structure could be replaced by an optically clear adhesive that allows waves emitted by a fingerprint sensor to properly propagate through the display assembly. In some embodiments, a fingerprint sensor is set within a recess in the display assembly. For example, a depression could be created in the protective substrate, and the fingerprint sensor could be set within the depression using an optically clear adhesive. The functionality of a fingerprint sensor could also be replicated by increasing the density of touch-sensing elements in certain areas of a display assembly that utilizes in-cell technology.

Fingerprint sensors for mobile devices

Disclosed herein are various embodiments of an imaging device having a wide field of view configured to connect to another device. According to one embodiment, the wide field of view camera can include an array of cameras for recording a wide view (e.g., 360&#xb0; view) of the surrounding environment and generate a 360&#xb0; image of the surrounding environment. The wide field of view camera can be configured to connect and/or attach to another device, including a mobile device. Embodiments include a wired or wireless connection mechanism to facilitate communication between the wide field of view camera and another device. The connection mechanism may enable transmission of data associated with the wide field of view camera to another device. Embodiments include an attachment mechanism to fasten the wide field of view camera to another device.

Wide field of view camera for integration with a mobile device

Certain aspects of the technology disclosed herein include generating a virtual sound field based on data from an ambisonic recording device. The ambisonic device records sound of a surrounding environment using at least four microphones having a tetrahedral orientation. An omnidirectional microphone having an audio-isolated portion can be used to isolate sound from a particular direction. Sound received from the plurality of microphones can be used to generate a virtual sound field. The virtual sound field include a dataset indicating a pressure signal and a plurality of velocity vectors. The ambisonic recording device can include a wide angle camera and generate wide angle video corresponding to the virtual sound field.

Growing community of inventors

Mountain View, CA, United States of America

Matthew J Hershenson