Apple Inc.

Zillow, Inc.

Zillow Group, Inc.

Google Inc.

Mfib Holdco, Inc.

Qi Shan

Li Guan

Alex Colburn

Ivaylo Boyadzhiev

Aditya Sankar

Steven Maxwell Seitz

Carlos Hernandez Esteban

Ramin Mehran

Jeffrey P Bigham

Samuel C White

Carlos E Guestrin

Shreyas V Joshi

Mingzhe Li

Xiaoyi Zhang

Ian M Richter

Alexandre Da Veiga

Amit Jain

Joshua M Susskind

Miguel Angel Bautista Martin

David Guera Cobo

Bryan M Perfetti

Robert Alex Colburn

Emilien Dupont

Fareeha Irfan

Calibration of eye tracking is improved by collecting additional calibration pairs when user is using apps with eye tracking. A user input component is presented on a display of an electronic device, detecting a dwelling action for user input component, and in response to detecting the dwelling action, obtaining a calibration pair comprising an uncalibrated gaze point and a screen location of the user input component, wherein the uncalibrated gaze point is determined based on an eye pose during the dwelling action. A screen gaze estimation is determine based on the uncalibrated gaze point, and in response to determining that the calibration pair is a valid calibration pair, training a calibration model using the calibration pair.

On-the-fly calibration for improved on-device eye tracking

Various implementations disclosed herein include devices, systems, and methods for synthesizing an environment based on an image. In various implementations, a device includes a non-transitory memory and one or more processors coupled with the non-transitory memory. In various implementations, a method includes determining an engagement score that characterizes a level of engagement between a user and a representation of a subject included in an image. In some implementations, the method includes, in response to the engagement score satisfying an engagement threshold, obtaining stored information regarding the subject, and synthesizing an environment based on the image and the stored information regarding the subject.

Generating a three-dimensional environment based on an image

Various implementations disclosed herein include devices, systems, and methods that determine a wrist measurement or watch band size using depth data captured by a depth sensor from one or more rotational orientations of the wrist. In some implementations, depth data captured by a depth sensor including at least two depth map images of a wrist from different angles is obtained. In some implementations, an output is generated based on inputting the depth data into a machine learning model, the output corresponding to circumference of the wrist or a watch band size of the wrist. Then, a watch band size recommendation is provided based on the output.

Touchless wrist measurement

The subject technology provides a framework for learning neural scene representations directly from images, without three-dimensional (3D) supervision, by a machine-learning model. In the disclosed systems and methods, 3D structure can be imposed by ensuring that the learned representation transforms like a real 3D scene. For example, a loss function can be provided which enforces equivariance of the scene representation with respect to 3D rotations. Because naive tensor rotations may not be used to define models that are equivariant with respect to 3D rotations, a new operation called an invertible shear rotation is disclosed, which has the desired equivariance property. In some implementations, the model can be used to generate a 3D representation, such as mesh, of an object from an image of the object.

Neural rendering

Various implementations disclosed herein include devices, systems, and methods that provide measurements of objects based on a location of a surface of the objects. An exemplary process may include obtaining a three-dimensional (3D) representation of a physical environment that was generated based on depth data and light intensity image data, generating a 3D bounding box corresponding to an object in the physical environment based on the 3D representation, determining a class of the object based on the 3D semantic data, determining a location of a surface of the object based on the class of the object, the location determined by identifying a plane within the 3D bounding box having semantics in the 3D semantic data satisfying surface criteria for the object, and providing a measurement of the object, the measurement of the object determined based on the location of the surface of the object.

Automatic measurements based on object classification

Techniques are described for automated operations involving capturing and analyzing information from an interior of a house, building or other structure, for use in generating and providing a representation of that interior. Such techniques may include using a user's mobile device to capture visual data from multiple viewing locations (e.g., video captured while the mobile device is rotated for some or all of a full 360 degree rotation at each viewing location) within multiple rooms, capturing data linking the multiple viewing locations, analyzing each viewing location's visual data to create a panorama image from that viewing location, analyzing the linking data to determine relative positions/directions between at least some viewing locations, creating inter-panorama links in the panoramas to each of one or more other panoramas based on such determined positions/directions, and providing information to display multiple linked panorama images to represent the interior.

Capture, analysis and use of building data from mobile devices

Various implementations disclosed herein include devices, systems, and methods that obtain a three-dimensional (3D) representation of a physical environment that was generated based on depth data and light intensity image data, generate a 3D bounding box corresponding to an object in the physical environment based on the 3D representation, classify the object based on the 3D bounding box and the 3D semantic data, and display a measurement of the object, where the measurement of the object is determined using one of a plurality of class-specific neural networks selected based on the classifying of the object.

Techniques are described for using computing devices to perform automated operations to generate mapping information using inter-connected images of a defined area, and for using the generated mapping information in further automated manners. In at least some situations, the defined area includes an interior of a multi-room building, and the generated information includes a floor map of the building, such as from an automated analysis of multiple panorama images or other images acquired at various viewing locations within the building&#x2014;in at least some such situations, the generating is further performed without having detailed information about distances from the images' viewing locations to walls or other objects in the surrounding building. The generated floor map and other mapping-related information may be used in various manners, including for controlling navigation of devices (e.g., autonomous vehicles), for display on one or more client devices in corresponding graphical user interfaces, etc.

Use of automated mapping information from inter-connected images

Automated mapping information generation from inter-connected images

Techniques are described for automated operations involving acquiring and analyzing information from an interior of a house, building or other structure, for use in generating and providing a representation of that interior. Such techniques may include using a user's mobile device to capture video data from multiple viewing locations (e.g., 360&#xb0; video at each viewing location) within multiple rooms, and capturing data linking the multiple viewing locations (e.g., by recording video, acceleration and/or other data from the mobile device as the user moves between the two viewing locations), creating a panorama image for each viewing location, analyzing the linking information to model the user's travel path and determine relative positions/directions between at least some viewing locations, creating inter-panorama links in the panoramas to each of one or more other panoramas based on such determined positions/directions, and providing information to display multiple linked panorama images to represent the interior.

Connecting and using building data acquired from mobile devices

Growing community of inventors

Mercer Island, WA, United States of America

Qi Shan