Shopify Inc.

Russ Maschmeyer

Brennan Letkeman

Eric Andrew Florenzano

Diego Macario Bello

Daniel Beauchamp

Hettige Ray Perera Jayatunga

Adam Debreczeni

Sarah Hurtgen

James Harold Hall, Jr

Neil Leonard Padgett

James Lepp

Andrew Nhat-Nam Ngo

A computer system maintains low-rank adaptation (LoRA) models, where each LoRA model includes a set of weights configured to modify parameters of a large-language model (LLM) to cause the LLM to generate text having a corresponding property. The computer system presents a set of manipulable user-interface controls that allow configuration of properties of LLM-generated text. Output of the LLM is modified using LoRA models that are selected based on a state of the user-interface controls as manipulated. A preview is provided of LLM output corresponding to the current state of the user-interface controls during presentation and manipulation thereof. To provide this preview, the computer system iteratively provides a prompt to the LLM and outputs the output of the LLM responsive to that prompt for each iteration. For each iteration, the LLM output is modified using the LoRA models selected based on the current state of the user-interface controls as manipulated.

Composable low-rank adaptation models for defining large-language model text style

Methods and systems for prompting a large language model (LLM) to generate a description of an object with indications of any unsubstantiated information are disclosed. A prompt is generated to a LLM to generate a description of an object, where the prompt includes one or more object attributes to include in the generated description. The prompt also includes an instruction for the LLM to annotate any portions of the generated description that are, involve, and/or include unsubstantiated information according to a defined format. The prompt is provided to the LLM and the generated description is received. The generated description is parsed to identify, based on the defined format, one or more annotated portions indicating unsubstantiated information. The generated description is presented for display via a user device.

Methods and systems for generation of text using large language model with indications of unsubstantiated information

A computer-implemented method is disclosed. The method includes: determining a first position of a real display device in a real-world environment; receiving a request to display virtual information at a second relative position with respect to the first position in an AR version of the real-world environment; responsive to receiving the request: determining a displayable area associated with the second relative position; and causing the virtual information to be overlaid on the displayable area in the AR version of the real-world environment.

Methods for calibrating augmented reality scenes

To improve user experience when interacting with AR content within an AR environment, the AR content may be overlaid over a proxy object in a real-world space. Differences in dimension between the proxy object and the virtual model may be such that the object is larger than the virtual model, which may result in portions of the object appearing to protrude from behind the virtual model, decreasing user enjoyment. In some embodiments, an AR system for the overlay of AR content on a proxy object and concealment of the proxy object may be implemented. The system may overlay a virtual model to a proxy object, and then conceal any remaining visible portions of the proxy object from the visual field of a device displaying the AR environment. The system may overlay the virtual model so that any remaining visible portion of the proxy object is a single continuous region.

Systems and methods for overlay of virtual object on proxy object and concealment of proxy object

A computer-implemented is disclosed. The method includes: generating an augmented reality (AR) scene that includes a virtual 3D representation of a product and a view of a first graphical user interface; monitoring user interactions with the virtual 3D representation of the product based on detected gestures of the user; determining modifications to the virtual 3D representation of the product based on the monitored user interactions; presenting, in the AR scene, a modified 3D representation of the product; converting the modified 3D representation of the product to a 2D image; and causing the 2D image to be displayed at a defined location of the first graphical user interface in AR.

Systems and methods for editing content items in augmented reality

A system and method are provided for displaying objects in electronic user interfaces. The method includes receiving, via a user interface, a first input defining a manipulation to be applied to a three-dimensional (3D) object corresponding to a two-dimensional (2D) image displayed in the user interface. The method also includes determining an initial positioning of a corresponding 3D model to be applied when loading an updated 3D model to replace the 2D image. The method also includes manipulating, in the user interface, the corresponding 3D object according to the first input.

System and method for manipulation of 3D objects in electronic user interfaces

A computer-implemented method includes: obtaining a three-dimensional (3D) representation of a first real-world environment; identifying a real-world object of interest in a second real-world environment, the first real-world environment different from the second real-world environment; determining a first position in the 3D representation of the first real-world environment corresponding to the real-world object of interest; and generating an augmented reality (AR) version of the first real-world environment for presentation in the second real-world environment using the 3D representation of the first real-world environment and based on positioning the real-world object of interest in the first position in the AR version of the first real-world environment.

Systems and methods for modeling real-world objects in virtual scenes

A system and method are provided for updating electronic content by selectively replacing virtual 3D objects being displayed in an electronic environment. The method includes in one example, presenting augmented reality (AR) content, the presented AR content showing a set of objects, at least some of the objects of the set of objects being rendered 3D objects; receiving an input identifying one of the objects of the set of objects; and responsive to the input, providing updated AR content, the updated AR content replacing at least one of the objects with a different rendered 3D object, wherein the replaced at least one of the objects is determined based on the identified one of the objects of the set of objects.

System and method for updating electronic content by selectively replacing virtual 3D objects

Methods and systems for providing a dynamic product presentation are disclosed. In one example, a method comprises providing, by a processor, a three-dimensional representation of a product in a virtual environment for display on a customer device; and responsive to the processor identifying a surface in a camera feed of the customer device having a dimensionality suitable for the product, generating, by the processor, an augmented media containing an augmented reality representation of a three-dimensional model for the product on the surface.

Augmented reality enabled dynamic product presentation

A system and method are provided for displaying objects in electronic user interfaces. The method includes receiving, via a user interface, a first input defining a rotation to be applied to each of a plurality of three-dimensional (3D) objects displayed in the user interface. The method also includes manipulating, in the user interface, each of the plurality of 3D objects in tandem according to the first input, the manipulating comprising rotating each of the plurality of 3D objects individually in tandem according to the rotation to be applied.

System and method for tandem manipulation of 3D objects in electronic user interfaces

In virtual reality (VR) and augmented reality (AR), eye tracking may be performed to determine the user's gaze direction. The gaze direction may be used to enhance user interaction. However, when a user gazes in a particular direction, it could sometimes be the case that there are multiple items located in that gaze direction, each at a different depth. The gaze of direction alone might not be indicative of the item at which the user is looking. Therefore, in some embodiments, to try to further enhance user interaction, a gaze depth of the gaze may be determined. Some embodiments are directed to performing eye tracking to detect a gaze depth of a human's gaze and modifying a user interface (UI) responsive to a change in the gaze depth.

Systems and methods for responsive user interface based on gaze depth

Methods and systems for providing a dynamic product presentation are disclosed. In one example, a method comprises identifying, by a processor, a first surface in a first view of a camera feed from a customer device; obtaining, by the processor, a three-dimensional model of a product that corresponds to the first surface; providing, by the processor, an augmented media containing an overlay of a first augmented reality representation of the three-dimensional model of the product in the first view; and responsive to the processor identifying a second surface in a second view of the camera feed from the customer device, revising, by the processor, the augmented media to contain a second augmented reality representation of the three-dimensional model of the product on the second surface in the second view.

Growing community of inventors

Berkeley, CA, United States of America

Russ Maschmeyer