Shopify Inc.

Brennan Letkeman

Russ Maschmeyer

Eric Andrew Florenzano

Diego Macario Bello

Daniel Beauchamp

James Harold Hall, Jr

Adam Debreczeni

Sarah Hurtgen

Hettige Ray Perera Jayatunga

Neil Leonard Padgett

James Lepp

Bradley Joseph Aldridge

Andrew Nhat-Nam Ngo

A computer system maintains low-rank adaptation (LoRA) models, where each LoRA model includes a set of weights configured to modify parameters of a large-language model (LLM) to cause the LLM to generate text having a corresponding property. The computer system presents a set of manipulable user-interface controls that allow configuration of properties of LLM-generated text. Output of the LLM is modified using LoRA models that are selected based on a state of the user-interface controls as manipulated. A preview is provided of LLM output corresponding to the current state of the user-interface controls during presentation and manipulation thereof. To provide this preview, the computer system iteratively provides a prompt to the LLM and outputs the output of the LLM responsive to that prompt for each iteration. For each iteration, the LLM output is modified using the LoRA models selected based on the current state of the user-interface controls as manipulated.

Composable low-rank adaptation models for defining large-language model text style

A computer-implemented method is disclosed. The method includes: determining a first position of a real display device in a real-world environment; receiving a request to display virtual information at a second relative position with respect to the first position in an AR version of the real-world environment; responsive to receiving the request: determining a displayable area associated with the second relative position; and causing the virtual information to be overlaid on the displayable area in the AR version of the real-world environment.

Methods for calibrating augmented reality scenes

To improve user experience when interacting with AR content within an AR environment, the AR content may be overlaid over a proxy object in a real-world space. Differences in dimension between the proxy object and the virtual model may be such that the object is larger than the virtual model, which may result in portions of the object appearing to protrude from behind the virtual model, decreasing user enjoyment. In some embodiments, an AR system for the overlay of AR content on a proxy object and concealment of the proxy object may be implemented. The system may overlay a virtual model to a proxy object, and then conceal any remaining visible portions of the proxy object from the visual field of a device displaying the AR environment. The system may overlay the virtual model so that any remaining visible portion of the proxy object is a single continuous region.

Systems and methods for overlay of virtual object on proxy object and concealment of proxy object

A computer-implemented is disclosed. The method includes: generating an augmented reality (AR) scene that includes a virtual 3D representation of a product and a view of a first graphical user interface; monitoring user interactions with the virtual 3D representation of the product based on detected gestures of the user; determining modifications to the virtual 3D representation of the product based on the monitored user interactions; presenting, in the AR scene, a modified 3D representation of the product; converting the modified 3D representation of the product to a 2D image; and causing the 2D image to be displayed at a defined location of the first graphical user interface in AR.

Systems and methods for editing content items in augmented reality

A computer-implemented method includes: obtaining a three-dimensional (3D) representation of a first real-world environment; identifying a real-world object of interest in a second real-world environment, the first real-world environment different from the second real-world environment; determining a first position in the 3D representation of the first real-world environment corresponding to the real-world object of interest; and generating an augmented reality (AR) version of the first real-world environment for presentation in the second real-world environment using the 3D representation of the first real-world environment and based on positioning the real-world object of interest in the first position in the AR version of the first real-world environment.

Systems and methods for modeling real-world objects in virtual scenes

A system and method are provided for updating electronic content by selectively replacing virtual 3D objects being displayed in an electronic environment. The method includes in one example, presenting augmented reality (AR) content, the presented AR content showing a set of objects, at least some of the objects of the set of objects being rendered 3D objects; receiving an input identifying one of the objects of the set of objects; and responsive to the input, providing updated AR content, the updated AR content replacing at least one of the objects with a different rendered 3D object, wherein the replaced at least one of the objects is determined based on the identified one of the objects of the set of objects.

System and method for updating electronic content by selectively replacing virtual 3D objects

Methods and systems for providing a dynamic product presentation are disclosed. In one example, a method comprises providing, by a processor, a three-dimensional representation of a product in a virtual environment for display on a customer device; and responsive to the processor identifying a surface in a camera feed of the customer device having a dimensionality suitable for the product, generating, by the processor, an augmented media containing an augmented reality representation of a three-dimensional model for the product on the surface.

Augmented reality enabled dynamic product presentation

The present disclosure provides AR systems and methods. The computer-implemented method comprises displaying, on a mobile device, a rendered virtual object in an augmented reality (AR) scene, and detecting, using an input device of the mobile device, a gesture having a detected speed. The method further includes identifying the gesture, wherein the gesture is identified as a first command to implement a first function related to the virtual object responsive to the detected speed of the gesture being less than a speed threshold, and the gesture is identified as a second command to implement a second function related to the virtual object responsive to the detected speed of the gesture being greater than the speed threshold. The second command and second function are different from the first command and the first function, respectively. The identified gesture is then processed.

System and method to manipulate virtual model based on physical parameters of gesture input

Coin

In virtual reality (VR) and augmented reality (AR), eye tracking may be performed to determine the user's gaze direction. The gaze direction may be used to enhance user interaction. However, when a user gazes in a particular direction, it could sometimes be the case that there are multiple items located in that gaze direction, each at a different depth. The gaze of direction alone might not be indicative of the item at which the user is looking. Therefore, in some embodiments, to try to further enhance user interaction, a gaze depth of the gaze may be determined. Some embodiments are directed to performing eye tracking to detect a gaze depth of a human's gaze and modifying a user interface (UI) responsive to a change in the gaze depth.

Systems and methods for responsive user interface based on gaze depth

Methods and systems for providing a dynamic product presentation are disclosed. In one example, a method comprises identifying, by a processor, a first surface in a first view of a camera feed from a customer device; obtaining, by the processor, a three-dimensional model of a product that corresponds to the first surface; providing, by the processor, an augmented media containing an overlay of a first augmented reality representation of the three-dimensional model of the product in the first view; and responsive to the processor identifying a second surface in a second view of the camera feed from the customer device, revising, by the processor, the augmented media to contain a second augmented reality representation of the three-dimensional model of the product on the second surface in the second view.

To interact with and manipulate a virtual object associated with an AR experience displayed on the screen of a user device (e.g., a smartphone), the user must first learn or discover the particular gestures that correspond with a given function, but such gestures may not always be intuitive. The user may find it difficult or bothersome to learn/remember the specific gesture(s) needed to achieve a specific result. Moreover, there are only a limited number of gestures that can be performed using the screen of the device, often leading to gestural conflict. In some embodiments, a dual-interface AR system may be implemented to provide an AR experience for a user. The dual-interface AR system utilizes two user interfaces and distributes the input gestures between them, allowing the user to dynamically switch between the two and to use the interface that may be most intuitive to the user.

Growing community of inventors

Calgary, Canada

Brennan Letkeman