Samsung Electronics Co., Ltd.

Apple Inc.

Arun Rakesh Yoganandan

Benjamin R Blachnitzky

Aaron Mackay Burns

Nicolai Philip Georg

Adam G Poulos

Chang Long Zhu Jin

Kumi Akiyoshi

Benjamin Hylak

Yuchang Hu

Stacie Lynn Hibino

Melinda Yang

Anette L Freiin Von Kapri

Mark Schlager

Jue Wang

Zachary Z Becker

Nan-Wei Gong

Michelle Chua

Jordan A Cazamias

David M Schattel

Tobias Rick

Henry N Holtzman

Kristen Kator

Jee Hoon Kim

Kyle T McGill

Jae Chung

Jung Won Hong

Kumi Akivoshi

Christopher L Nolet

Samantha Koire

Tais C Mauk

Chang Long Zhujin

Jung-Won Hong

In one implementation. a method for dynamically selecting an operation modality for a physical object. The method includes: obtaining a user input vector that includes at least one user input indicator value associated with one of a plurality of different input modalities; obtaining tracking data associated with a physical object; generating a first characterization vector for the physical object, including a pose value and a user grip value, based on the user input vector and the tracking data, wherein the pose value characterizes a spatial relationship between the physical object and a user of the computing system and the user grip value characterizes a manner in which the physical object is being held by the user; and selecting, based on the first characterization vector, a first operation modality as a current operation modality for the physical object.

Method and device for dynamically selecting an operation modality for an object

A method is performed at an electronic device with a non-transitory memory, one or more processors, and a display. The method includes obtaining positional information regarding an auxiliary device. The method includes determining an anchor point of the auxiliary device on a volumetric computer-generated object. Determining the anchor point includes setting the anchor point to a first position of the auxiliary device in accordance with a determination that dimensions of the volumetric computer-generated satisfy a first criterion. Determining the anchor point further includes setting the anchor point to a second position of the auxiliary device in accordance with a determination that the dimensions of the volumetric computer-generated object satisfy a second criterion different from the first criterion. The first position is different from the second position. The method includes anchoring, based on the positional information, the volumetric computer-generated object to the anchor point of the auxiliary device.

Anchoring to multiple positions of an auxiliary device

A method is performed at an electronic device with one or more processors, a non-transitory memory, a display, an extremity tracking system, and a communication interface provided to communicate with a finger-wearable device. The method includes displaying a computer-generated object on the display. The method includes obtaining finger manipulation data from the finger-wearable device via the communication interface. The method includes determining a multi-finger gesture based on extremity tracking data from the extremity tracking system and the finger manipulation data. The method includes registering an engagement event with respect to the computer-generated object according to the multi-finger gesture.

Multi-finger gesture based on finger manipulation data and extremity tracking data

A method includes displaying a plurality of computer-generated objects, and obtaining finger manipulation data from a finger-wearable device via a communication interface. In some implementations, the method includes receiving an untethered input vector that includes a plurality of untethered input indicator values. Each of the plurality of untethered input indicator values is associated with one of a plurality of untethered input modalities. In some implementations, the method includes obtaining proxy object manipulation data from a physical proxy object via the communication interface. The proxy object manipulation data corresponds to sensor data associated with one or more sensors integrated in the physical proxy object. The method includes registering an engagement event with respect to a first one of the plurality of computer-generated objects based on a combination of the finger manipulation data, the untethered input vector, and the proxy object manipulation data.

Object engagement based on finger manipulation data and untethered inputs

A method is performed at an electronic device with one or more processors, a non-transitory memory, a display, and an extremity tracker. The method includes obtaining extremity tracking data via the extremity tracker. The method includes displaying a computer-generated representation of a trackpad that is spatially associated with a physical surface. The physical surface is viewable within the display along with a content manipulation region that is separate from the computer-generated representation of the trackpad. The method includes identifying a first location within the computer-generated representation of the trackpad based on the extremity tracking data. The method includes mapping the first location to a corresponding location within the content manipulation region. The method includes displaying an indicator indicative of the mapping. The indicator may overlap the corresponding location within the content manipulation region.

Mapping a computer-generated trackpad to a content manipulation region

A method includes displaying a plurality of computer-generated objects, including a first computer-generated object at a first position within an environment and a second computer-generated object at a second position within the environment. The first computer-generated object corresponds to a first user interface element that includes a first set of controls for modifying a content item. The method includes, while displaying the plurality of computer-generated objects, obtaining extremity tracking data. The method includes moving the first computer-generated object from the first position to a third position within the environment based on the extremity tracking data. The method includes, in accordance with a determination that the third position satisfies a proximity threshold with respect to the second position, merging the first computer-generated object with the second computer-generated object in order to generate a third computer-generated object for modifying the content item. The method includes displaying the third computer-generated object.

Merging computer-generated objects

A method includes, while displaying a computer-generated object at a first position within an environment, obtaining extremity tracking data from an extremity tracker. The first position is outside of a drop region that is viewable using the display. The method includes moving the computer-generated object from the first position to a second position within the environment based on the extremity tracking data. The method includes, in response to determining that the second position satisfies a proximity threshold with respect to the drop region, detecting an input that is associated with a spatial region of the environment. The method includes moving the computer-generated object from the second position to a third position that is within the drop region, based on determining that the spatial region satisfies a focus criterion associated with the drop region.

Targeted drop of a computer-generated object

Content management includes receiving an image of one or more display devices with respective first content displayed. For at least one display device, a display device identification (ID) is retrieved. A change to the first content displayed at the at least one display device is caused, based on the display device ID of the at least one display device, upon interaction with the image.

Configuration and operation of display devices including content curation

In one implementation, a method for visually indicating positional/rotational information of a finger-wearable device. The method includes: determining a set of translational values and a set of rotational values for the finger-wearable device, wherein the finger-wearable device is worn on a finger of a user; displaying, via a display, a visual representation of a location of the finger-wearable device based on the set of translational values; generating a visual representation of a grasp region of the user based on the set of translational values and the set of rotational values; and concurrently displaying, via the display, the visual representation of the grasp region with the visual representation of the location of the finger-wearable device.

Method and device for positional/rotational information of a finger-wearable device

A method includes receiving a voice input at an electronic device. An ambiguity of the voice input is determined. The ambiguity is resolved based on contextual data. The contextual data includes at least one of: an image, a non-voice input comprising a gesture, a pointer of a pointing device, a touch, or a combination thereof.

System and method for context-based interaction for electronic devices

An electronic device includes a touchscreen display and at least one processor. The at least one processor is configured to present a dynamic user interface on the touchscreen display. The dynamic user interface includes a first input region forming a dial. The first input region is configured to receive user input related to a first viewing direction associated with a 360&#xb0; video. The at least one processor is also configured to detect the user input. In addition, the at least one processor is configured to transmit the user input to a display device to control presentation of the 360&#xb0; video on the display device. The dynamic user interface may further include a second input region, where the second input region is configured to receive second user input related to a second viewing direction associated with the 360&#xb0; video.

360&#xb0; video viewer control using smart device

A hover touch controller device includes a touch sensor having a touch surface and a proximity sensor. The touch sensor provides two-dimensional position information on when and where a user's finger touches the touch surface. The proximity sensor provides three-dimensional position information on pre-touch events. The pre-touch events corresponding to the user's finger hovering over the touch surface within some maximum depth. The hover touch controller device further includes a processor. The processor determines from the three-dimensional information a hover point projected on the touch surface and determines from the two-dimensional information a touch point on the touch surface. The processor communicates the hover point and the contact point to a display device. This can include correcting for any perceived user interaction issues associated an offset between the hover point and the touch point.

Error correction for seamless transition between hover and touch sensing

A hover touch controller device includes a touch sensor having a touch surface with a first aspect ratio and a proximity sensor. Information on when and where a user touches the touch surface is detected. Additionally, information on hover-input events is detected. The hover input events correspond to a user's finger hovering over the touch surface within some maximum depth. The hover touch device further includes a controller. The processor communicates three-dimensional spatial information to a Graphical User Interface (GUI). The GUI generates visualizations based on the hover events and the touch events on a display having an interactive surface with a second aspect ratio. The processor further corrects for any issues associated with supporting a variety of GUI designs. This can include correcting for a difference between the first aspect ratio and the second aspect ratio.

Dynamically adaptive sensing for remote hover touch

A controller includes a touch sensor and a proximity sensor. Information on when and where a user touches the surface of the touch sensor is detected. Additionally, information on pre-touch events is detecting. The pre-touch events correspond to a user's finger hovering over the touch sensor within some maximum depth. A display device generates visualizations based on the hover events and the touch events. This can include providing visual feedback in a hover mode where a user's finger is, relative to the touch sensor. Additionally, various forms of navigation information may be provided in the hover mode, as well as other control information.

Growing community of inventors

San Francisco, CA, United States of America

Arun Rakesh Yoganandan