Wevo, Inc.

Alexander Barza

Dustin R Garvey

Frank Chiang

Alexa Stewart

Janet Muto

Nitzan Shaer

Hannah Sieber

Charlie Hoang

Shannon Walsh

Jon Andrews

Jessica Yau

Melany Carvalho

Kelly Lyons

Ruthie McCready

Sara Peters

Kim Coccoluto

Everett Granger

Julia McCarthy

Andrea Paola Aguilera García

Rick Alarcon Sisniegas

Shachar Koresh

Laryssa Costa De Souza

Techniques are described herein for using artificial intelligence (AI) and machine learning (ML) to automate, accelerate, and enhance various aspects of comparative user experience testing. Embodiments interface with generative language models to compare user experiences and summarize the results of the comparison. In some embodiments, automated systems and programmatic processes access a series of analysis contexts, where a context includes a collection of message content fragments. The systems and processes may use the message content fragments for a given context to construct a dialogue with a generative language model to compare separate user experiences based on the results of a set of user experience tests. The output of the generative language model at one stage of the analysis may be combined with content fragments for another context to craft a dialogue at another stage of the analysis and/or to perform additional analyses of the user experiences.

Systems and methods for automating comparative user experience test analyses

One or more embodiments relate to a method for refining usability testing through the systematic collection and analysis of user click paths within an online interface. By tracking and filtering these click paths based on one or both of publicly observable attributes and non-publicly observable attributes, obtained in part through user responses to screening questions, the method generates distinct, aggregated click paths. These paths are then subjected to in-depth analysis that yields insights and metrics regarding user behavior and usability. Some embodiments further extend to the presentation of an interactive representation of these paths within a testing interface that allows researchers to explore and extract actionable findings.

Usability click tracking with navigable click paths

Operations of a prompt management system are disclosed. The operations may include: receiving a prompt for performing a set of tasks, assigning an agent group that includes a plurality of agents to perform a set of roles associated with a dataset in support of the set of tasks, causing the plurality of agents to perform the set of roles using a first machine-learning model, receiving a set of role results from the plurality of agents responsive to performing the set of roles, performing the set of tasks using at least a second machine-learning model, and providing a task result for display on a user interface device. The set of tasks may include executing an operation on the set of role results using the second machine-learning model, and generating a task result that includes a product of the operation executed on the set of role results.

Prompt management systems for moderating generative artificial intelligence operations

Techniques are described herein for producing machine-generated findings given a set of user experience test results. In some embodiments, the system generates the findings using an artificial intelligence and machine learning engine. The findings may highlight areas that are predicted to provide the most insight into optimizing a product's design. A finding may be generated based on all or a subset of the test result elements, including qualitative and/or quantitative data contained therein. A finding may summarize a subset of the UX test results that are interrelated. A finding may link a summary to one or more references extracted from the set of test results to show support for the machine-generated insights in the underlying raw test data. Machine-generated findings reports may provide near instantaneous guidance for optimizing product designs while removing extraneous information from a vast quantity of raw test result data.

Scalable systems and methods for discovering and summarizing test result facts

Techniques are described herein for selecting, curating, normalizing, enriching, and synthesizing the results of user experience tests. In some embodiments, a system identifies a qualitative element within a result set for a user experience test. The system then selects a machine learning model to apply based on one or more attributes associated with the user experience test and generates a predicted visibility, quality, and/or relevance for the qualitative element. Based on the prediction, the system generates a user interface that curates a set of results of the user experience test.

Scalable systems and methods for curating user experience test results

Techniques are described for producing machine learning models to generate findings associated with user experiences with products and/or services. In some embodiments, a training process receives a set of findings from one or more user experience tests, where a finding includes a summary and a set of one or more references supporting the summary. The training process further identifies a supplemental set of one or more references that were not included in the initial finding to support the summary. The training process trains a machine learning model, such as a neural or generative language model, based on the first set of one or more references and the second set of one or more references to generate summaries from a subset of sampled references based at least in part on the first set of one or more references and the second set of one or more references.

Robust systems and methods for training summarizer models

Growing community of inventors

Cambridge, MA, United States of America

Alexander Barza