Citibank, N.a.

Chamindra Desilva

Vishal Mysore

Ramkumar Ayyadurai

Daniel Lewandowski

Philip Davies

Stuart Murray

Biraj Krushna Rath

Payal Jain

Mariusz Saternus

Tariq Husayn Maonah

Shardul Malviya

Samantha Cory

Deepak Jain

Wayne Liao

James Myers

Sofia Rahman

Prashant Praveen

Christopher Tucker

Miriam Silver

Prithvi Narayana Rao

William Franklin Cameron

Pramod Goyal

Manjit Rajaretnam

Ganesh Prasad Bhat

Joshua Adam Goldman

Julisia Jackson

Venkata Uttam Kumar Chunduri

The systems and methods disclosed herein receive alphanumeric characters defining operative boundaries for expected model use cases, along with operational data. The expected model use cases share common attributes, which are used by a first AI model to construct observed model use cases from the operational data. Each observed model use case includes features such as a text-based description, expected input and output,? AI model(s) generating the expected output from the input, and/or data supporting the AI models. For each observed model use case, a second AI model maps the alphanumeric characters and features to a risk category, selecting from multiple risk categories based on the level of risk associated with the features. The system identifies criteria for the observed model use case within the alphanumeric characters and generates gaps by comparing the criteria with the features of the observed model use case.

Identifying and remediating gaps in artificial intelligence use cases using a generative artificial intelligence model

The systems and methods disclosed herein obtain a set of alphanumeric characters defining constraints for agents and the agents' operational data. Each agent uses an output from a first set of artificial intelligence (AI) models and predefined objectives to autonomously generate proposed actions for execution on software application(s). For each agent, a second set of AI models evaluates the agent by identifying gaps in the proposed actions by comparing them with the expected actions. Using a third set of AI models and the identified gaps, the systems modify the proposed actions by adding, altering, or removing actions from the proposed actions.

Validating autonomous artificial intelligence (AI) agents using generative AI

The systems and methods disclosed herein generate responses using data retrieved by validating vector embeddings using hash signatures. An output generation request is received via a computing device and includes an input that includes a content set and a command set. The content set includes data chunks, which refers to text, audio, image, and/or video data. A subset of the data chunks that fail to associate with existing hash signatures are selected. Each selected data chunk is validated against predefined constraints that define the operative boundaries of a guideline set and subsequently assigned a unique hash signature to indicate a degree of satisfaction of the data chunk with the guidelines. Using an artificial intelligence (AI) model, a response to the output generation request is generated in accordance with the data chunks, where each data chunk is associated with a corresponding unique hash signature.

Validating artificial intelligence model outputs using hash signatures and chunk-level access controls

The technology evaluates the compliance of an AI application with predefined vector constraints. The technology employs multiple specialized models trained to identify specific types of non-compliance with the vector constraints within AI-generated responses. One or more models evaluate the existence of certain patterns within responses generated by an AI model by analyzing the representation of the attributes within the responses. Additionally, one or more models can identify vector representations of alphanumeric characters in the AI model's response by assessing the alphanumeric character's proximate locations, frequency, and/or associations with other alphanumeric characters. Moreover, one or more models can determine indicators of vector alignment between the vector representations of the AI model's response and the vector representations of the predetermined characters by measuring differences in the direction or magnitude of the vector representations.

Validating vector constraints of outputs generated by machine learning models

The systems and methods disclosed herein receive alphanumeric characters defining operative boundaries for expected model use cases, along with operational data. The expected model use cases share common attributes, which are used by a first AI model to construct observed model use cases from the operational data. Each observed model use case includes features such as a text-based description, expected input and output, AI model(s) generating the expected output from the input, and/or data supporting the AI models. For each observed model use case, a second AI model maps the alphanumeric characters and features to a risk category, selecting from multiple risk categories based on the level of risk associated with the features. The system identifies criteria for the observed model use case within the alphanumeric characters and generates gaps by comparing the criteria with the features of the observed model use case.

The systems and methods disclosed herein relate to querying data using artificial intelligence models. A generalized model receives an output generation request and partitions it into segments mapped to specific domains, where each domain indicates associated databases and guidelines. The segments are routed to domain-specific models trained on domain-specific data, which generate query fragments by comparing performance metrics and system resource usage metrics. The query fragments are aggregated into an overall query that satisfies guidelines across domains. The systems and methods can include a feedback loop to adjust the domain-specific models using user interactions and performance metrics to dynamically adapt to a skill level or experience of the user.

Querying data using specialized and generalized artificial intelligence models

The systems and methods disclosed herein receives artifacts generated using a first set of models within a multi-model superstructure. The multi-model superstructure includes a second set of models to test the first set of models. The multi-model superstructure dynamically routes the artifacts of the first set of models to one or more models of the second set of models by (i) determining a set of dimensions of the artifacts against which to evaluate the artifacts and (ii) identifying the models in the second set used to test the particular dimension. The second set of models then assesses each artifact against a set of assessment metrics. If an artifact fails to meet one or more assessment metrics, the second set of models generates actions to align the artifact with the set of assessment metrics.

Dynamic multi-model monitoring and validation for artificial intelligence models

The technology evaluates the compliance of an AI application with predefined guidelines. The technology obtains a set of guidelines defining operation boundaries of the AI application and constructs test cases associated with each guideline. Each test case can include a prompt, an expected outcome, and an expected explanation. The technology supplies the prompts to the AI application, receives case-specific outcomes and explanations from the AI application, and compares them with the expected outcomes and expected explanations. A compliance indicator is generated based on the evaluation results, indicating the degree of compliance of the AI application with the guidelines.

Dynamically validating AI applications for compliance

Growing community of inventors

London, United Kingdom

Chamindra Desilva