Multinarity Ltd

Sightful Computers Ltd

Doron Assayas Terre

Tamir Berliner

Tomer Kahan

Orit Dolev

Dori Peleg

Oded Noam

Aviv Burshtein

Eli Elhadad

Amit Knaani

Amir Eshel

Systems, methods, and computer program products are provided for performing operations for presenting content in three dimensional (3D) environments. Performing operations for presenting content in 3D environments includes: requesting content for virtual presentation in an environment of an extended reality appliance; receiving the content for the virtual presentation in the environment; receiving with the content, a plurality of tags, each being associated with a portion of the content, wherein a particular portion of the content associated with a particular tag includes an inanimate virtual object; for each tag, selecting, based on the respective tag, a 3D placement requirement for the portion of the content associated with the respective tag; and for each portion of the content, determining, based on a layout of the environment and the selected 3D placement requirement, a location in the environment for virtual placement of the respective portion of the content.

Presenting extended reality content in different physical environments

A method for augmenting a physical display with an extended reality display. A processor receives first signals representative of a first object fully presented on a physical display, second signals representative of a second object having a first portion presented on the physical display and a second portion extending beyond a border of the physical display, and third signals representative of a third object initially presented on the physical display and subsequently wholly moved beyond the border of the physical display. In response to receiving the second signals, the second portion is presented via a wearable extended reality appliance in a virtual space while the first portion is presented on the physical display. In response to receiving the third signals, the third object is wholly presented via the wearable extended reality appliance in the virtual space following the third object having been wholly presented on the physical display.

Extending working display beyond screen edges

Systems, methods, and computer program products are provided for facilitating presentation of content in a three dimensional (3D) environment. Facilitating presentation of content in a 3D environment includes: receiving an indication of device settings of an extended reality appliance; receiving an indication of content requested for presentation via the extended reality appliance; based on the device settings, selecting 3D placement requirements for the requested content, the 3D placement requirements including at least one of: a 3D physical background, a visual span, virtual object positioning protocols, 3D anchoring requirements, 3D content size requirements, or ambient illumination requirements; transmitting the selected 3D placement requirements and the requested content, the transmitted content including at least one tag for associating portions of the content with at least some of the 3D placement requirements to thereby enable the extended reality appliance to display the content in a 3D environment.

Protocol for facilitating presentation of extended reality content in different physical environments

Systems, methods, and non-transitory computer readable media including instructions for presenting location-based content are described. Presenting location-based content includes obtaining an indication of a current physical location of an extended reality appliance; providing the indication to a first server that maps physical locations to a plurality of content addresses; receiving from the first server, at least one specific content address associated with the current physical location; using the at least one specific content address to access a second server; receiving content, associated with the current physical location, from the second server; and presenting the content via the extended reality appliance, while the extended reality appliance is in the current physical location.

Location-based virtual resource locator

Systems, methods, and non-transitory computer readable media including instructions for customizing location-based content presentation are described. Customizing location-based content presentation includes accessing a group of content display mode rules specifying a format impacting 3D presentation; accessing stored selections unique to a wearer of a wearable extended reality appliance associating content display mode rules with physical locations; receiving a first indication that the wearable extended reality appliance is at the first physical location; based on the received first indication and the accessed stored selections, displaying first location-based content at the first physical location according to the first content display mode rule; receiving a second indication that the wearable extended reality appliance is at the second physical location; and based on the received second indication and the accessed stored selections, displaying second location-based content at the second physical location according to the second content display mode rule.

User-customized location based content presentation

Methods, systems, apparatuses, and computer-readable media are provided for moving a virtual cursor along two traversing virtual planes. In one implementation, the computer-readable medium includes instructions to cause a processor to generate, via a wearable extended reality appliance, a display including a virtual cursor and virtual objects located on a first virtual plane that traverses a second virtual plane overlying a physical surface; receive a first two-dimensional input reflective of an intent to select a first virtual object located on the first virtual plane; cause a first cursor movement, toward the first virtual object, along the first virtual plane; receive a second two-dimensional input reflective of an intent to select a second virtual object appearing on the physical surface; and cause a second cursor movement, toward the second virtual object, including a partial movement along the first virtual plane and a partial movement along the second virtual plane.

Coordinating cursor movement between a physical surface and a virtual surface

Systems, methods, and computer program products are provided for performing operations for presenting content in three dimensional (3D) environments. Performing operations for presenting content in 3D environments includes: requesting content for virtual presentation in an environment of an extended reality appliance; receiving the content for presentation in the environment; receiving with the content, a plurality of tags, each of the plurality of tags being associated with a portion of the content; capturing, using an image sensor, a layout of the environment; for each tag, selecting, based on the tag, a 3D placement requirement for the content portion associated with the tag; and for each content portion, determining, based on the layout of the environment and the 3D placement requirement selected for the respective content portion, a location in the environment for virtual placement of the respective content portion.

A system and method for preventing virtual collisions may include presenting, via an extended reality appliance, a first virtual object docked to a first movable physical object and a second virtual object docked to a second movable physical object. The movement of the first and second physical objects may be tracked and an impending collision between the first virtual object and the second virtual object may be determined. Priority rules establishing that the first physical object has priority over the second physical object may be accessed. And based on the priority rules, favoring the first virtual object over the second virtual object at the time of the impending collision.

Managing virtual collisions between moving virtual objects

Systems, methods, and non-transitory computer readable media including instructions for selectively operating a wearable extended reality appliance are disclosed. Selectively operating the wearable extended reality appliance includes establishing a link between a wearable extended reality appliance and a keyboard device; receiving sensor data from at least one sensor associated with the wearable extended reality appliance, the sensor data being reflective of a relative orientation of the wearable extended reality appliance with respect to the keyboard device; based on the relative orientation, selecting from a plurality of operation modes a specific operation mode for the wearable extended reality appliance; identifying a user command based on at least one signal detected by the wearable extended reality appliance; and executing an action responding to the identified user command in a manner consistent with the selected operation mode.

Interpreting commands in extended reality environments based on distances from physical input devices

Systems, methods, and non-transitory computer readable media for virtually extending a physical keyboard are disclosed. In one implementation, a non-transitory computer readable medium contains instructions that cause a processor to: receive image data representing a keyboard placed on a surface from an image sensor associated with a wearable extended reality appliance; determine that the keyboard is paired with the wearable extended reality appliance; receive an input for causing a display of a virtual controller with the keyboard; display, via the wearable extended reality appliance, the virtual controller in a first location on the surface, with original spatial orientation relative to the keyboard; detect a movement of the keyboard to a different location on the surface; and in response to the detected movement, present the virtual controller in a second location on the surface, with a subsequent spatial orientation relative to the keyboard that corresponds to the original spatial orientation.

Systems and methods for extending a keyboard to a surrounding surface using a wearable extended reality appliance

Methods, systems, apparatuses, and non-transitory computer-readable media are provided for enabling gesture interaction with invisible virtual objects. In one implementation, the computer-readable medium includes instructions to cause a processor to receive image data captured by at least one image sensor of a wearable extended reality appliance in a field of view; display a plurality of virtual objects in a portion of the field of view; receive a selection of a specific physical object; receive a selection of a specific virtual object; dock the specific virtual object with the specific physical object; when the specific physical object and the specific virtual object are outside the portion of the field of view such that the specific virtual object is invisible to a user of the wearable extended reality appliance, receive a gesture input indicating interaction with the specific virtual object; and cause an output associated with the specific virtual object.

Gesture interaction with invisible virtual objects

Systems, methods, and non-transitory computer readable media configured for modifying the display of virtual objects docked to a movable input device are provided. In one implementation, the computer readable medium may be configured to receive image data from an image sensor, the image data representing an input device placed at a first location on a supporting surface. At least one processor may generate a presentation of at least one virtual object in proximity to the first location. Operations may include docking the at least one virtual object to the input device. The processor may determine that the input device is in a second location on the support surface. The virtual object presentation may be modified based on the second location. Operations may further include determining that the input device is in a third location removed from the support surface and modifying the virtual object presentation accordingly.

Evolving docking based on detected keyboard positions

An integrated computational interface device may include a portable housing having a key region and a non-key region; a keyboard associated with the key region of the housing; and a holder associated with the non-key region of the housing. The holder may be configured for selective engagement with and disengagement from the wearable extended reality appliance, such that when the wearable extended reality appliance is selectively engaged with the housing via the holder, the wearable extended reality appliance is transportable with the housing.

Systems and methods for extending working display beyond screen edges

Systems, methods, and non-transitory computer readable media for controlling perspective in an extended reality environment are disclosed. In one embodiment, a non-transitory computer readable medium contains instructions to cause a processor to perform the steps of: outputting for presentation via a wearable extended reality appliance (WER-appliance), first display signals reflective of a first perspective of a scene; receiving first input signals caused by a first multi-finger interaction with the touch sensor; in response, outputting for presentation via the WER-appliance second display signals to modify the first perspective of the scene, causing a second perspective of the scene to be presented via the WER-appliance; receiving second input signals caused by a second multi-finger interaction with the touch sensor; and in response, outputting for presentation via the WER-appliance third display signals to modify the second perspective of the scene, causing a third perspective of the scene to be presented via the WER-appliance.

Systems and methods for controlling virtual scene perspective via physical touch input

Systems, methods, and non-transitory computer readable media containing instructions for causing at least one processor to perform operations to enable cursor control in an extended reality space are provided. In one implementation, the processor is configured to perform operations comprising receiving from an image sensor first image data reflecting a first region of focus of a user of a wearable extended reality appliance; causing a first presentation of a virtual cursor in the first region of focus; receiving from the image sensor second image data reflecting a second region of focus of the user outside the initial field of view in the extended reality space; receiving input data indicative of a desire of the user to interact with the virtual cursor; and causing a second presentation of the virtual cursor in the second region of focus in response to the input data.

Growing community of inventors

Holon, Israel

Doron Assayas Terre