Artificial intelligence determined emotional state with dynamic modification of output of an interaction application

Abstract

A method including receiving an input from a user for an interaction application that is executing. The method including receiving a first plurality of multimodal cues from a plurality of trackers, wherein each of the plurality of trackers is configured to track a corresponding cue using artificial intelligence. The method including providing the first plurality of multimodal cues to an artificial intelligence (AI) model configured to classify one or more emotions of a user, wherein the AI model generates a first predicted emotion based on the first plurality of multimodal cues. The method including determining a modification to content generated by the interaction application in response to the input, wherein the modification to the content is determined based on the predicted emotion. The method including providing the modification to the interaction application, wherein the interaction application is configured to generate modified content based on the modification.