International Business Machines Corporation

Logan Bailey

Zachary A Silverstein

Jeremy R Fox

Jacob Ryan Jepperson

Melanie Dauber

Su Liu

Fang Lu

Jennifer M Hatfield

Suman Patra

Kelley Anders

Tiberiu (Jim) Suto

Jill S Dhillon

Thomas Jefferson Sandridge

Hamid Majdabadi

Tyler Hansen

Dylan Zucker

David M Cesarano

Alexander Joseph Abell

Kenneth C Accetta

According to one embodiment, a method, computer system, and computer program product for mixed-reality peripheral interfacing is provided. The present invention may include recording, with a camera, video footage of a peripheral held by a user participating in a mixed-reality environment; extracting, by a processor, peripheral data from the video footage; extracting interface data from the video footage; extracting input data entered into the peripheral by the user; responsive to determining that one or more keybindings do not match the peripheral, rebinding one or more keybindings of the peripheral based on the peripheral data; displaying a virtual representation of the peripheral to the user within the mixed-reality environment based on the peripheral data; and displaying a virtual representation of the user interacting with the virtual representation of the peripheral within the mixed-reality environment based on the interface data and the input data.

Dynamic augmented reality and screen-based rebinding based on detected behavior

A method for persisting a robotic process automation (RPA) session receives, through an RPA authentication engine, a software application site being interacted with by an end user. The RPA authentication engine deploys a bot running the RPA session on the software application site. The RPA authentication engine identifies login credentials of the end user associated with the software application site. The RPA authentication engine receives confirmation of an initial login to the software application site by the end user. The RPA authentication engine iteratively interfaces with the software application site using the login credentials to persist the RPA session.

Robotic process automation log-in method for authentication

Embodiments detect a task keyword related to an offloaded task of a task-giver, train a machine learning (ML) model using the task keyword, determine that the task keyword is similar to a historical task by utilizing the trained ML model, record a user input in response to determining that the task keyword is similar to the historical task, freeze an action execution during a pseudo action before a finality action is executed, provide the recorded user input to the task-giver and requesting approval from the task-giver, and execute the offloaded task using robotic process automation (RPA) in response to receiving approval from the task-giver.

Ad-hoc pseudo completion task

A computer-implemented method for document concatenation and ontological structuring. The method includes identifying a downloaded file and determining whether the downloaded file is associated with one or more saved files. Based on determining that the downloaded file is associated with the one or more saved files, developing an ontological structure storing the downloaded file and the one or more saved files.

Document concatenation and ontological structuring based on similarities

In an approach for providing a user with a set of information on a feature of a comparable product and enabling the user to assess the feature by overlaying the user's product with a digital twin of the comparable product, a processor identifies a first product in a surrounding of the user by comparing an image of the first product to one or more images stored in a product corpus. A processor searches, based on the first product, the product corpus for one or more products within a family of the first product. A processor selects a second product from the product corpus based in part on a set of product comparison preferences. A processor overlays the first product with a digital twin of the second product. A processor simulates a feature associated with the second product that is different from a feature associated with the first product.

Augmented reality overlay of feature and versioning improvement

In several approaches, a processor feeds an interaction by a user into a recurrent neural network (RNN) model. A processor identifies, using the RNN model trained to identify a specific workflow based on the interaction, an in-process workflow of the user. A processor predicts, using the RNN model, a next interaction of the user based on a next expected step in the in-process workflow. A processor performs at least one optimization function related to the next interaction of the user including at least one of autofocusing on a next user interface (UI) element associated with the next expected step in the in-process workflow, snapping a text cursor to the next UI element associated with the next expected step in the in-process workflow, and providing a UI prompt with a link to the next UI element associated with the next expected step in the in-process workflow.

UI workflow optimization based on expected next UI interaction

A method, computer system, and a computer program product for multi-content interactions is provided. The present invention may include detecting a user engaging with a first content and a second content simultaneously. The present invention may also include determining that the user is more engaged with the second content relative to the first content. The present invention may further include executing a content preserving action to enable the user to reengage with the first content without missing any segments of the first content.

Multi-content interactions

Aspects of the present disclosure relate to save context capturing. An indication of a save of a version of software can be received. Contextual data associated with a user managing the version of the software can be associated with the save of the version of software, the contextual data captured during a timeframe temporally related to the save. The association between the contextual data and the save of the version of the software can be stored in a software versioning repository. The save of the version of the software and the associated contextual data can be retrieved in response to a predetermined condition or a request.

Save context capturing

According to one embodiment, a method, computer system, and computer program product for importing player settings is provided. The present invention may include requesting one or more player settings responsive to receiving a command from a user to import one or more player settings associated with a player; receiving the one or more player settings; performing a hardware check to determine if the player settings are compatible with one or more user hardware elements; overwriting one or more user settings with the revised player settings based on the hardware check; and displaying a message to the user with one or more results of the overwrite.

Multiplayer E-gaming importable controls and settings

An embodiment intercepts a first notification intended for a user. An embodiment identifies, by analyzing a currently occurring interaction including the user, a lull in the interaction. An embodiment delivers during the lull, the first notification to the user.

Context based user notification adjustment and management

Optimizing a user experience during transition to an updated user interface includes using a computer processor to capture a user experience of a user's interaction with a program based on migration of the user interface from a current version of a page layout associated with the user to a new version of the page layout associated with the user. Disparity events experienced by the user while transitioning from the current version of the page layout to the new version of the page layout are analyzed, and a recommendation report of changes to the user interface is generated and sent to a developer associated with the new version of the page layout. An updated page layout reflecting modifications may then be received based on the recommendation report.

Software user experience transition assistance

Contextualization of digital content includes generating a corpus of metadata associated with a user, including technical information about devices and applications that execute on the devices, and interaction data describing usage of the devices and applications that execute on the devices, orienting the metadata to ontological elements, monitoring user consumption of digital content on the device and identifying content elements, comparing the identified content elements to the ontological elements, identifying a target content element to contextualize for the user with selected metadata of the corpus, and modifying the digital content for graphical presentation to the user, the modifying providing added information, associated with the target content element, in the digital content, the added information including the selected metadata of the corpus.

Contextualization of digital content

Described is a method to provide feedback and status updates during virtual collaborative sessions by monitoring, via a primary device associated with a user, a user status for the user participating in a collaborative session with one or more other users. In response to determining a user status change for the user is detected, the method can determine a response for the user to provide to the one or more other users based on a user profile and a context for the user status change, wherein the status change indicates an unavailability of the user. In response to determining the response is required for the user to provide the one or more other users, the method can send the response to a user device associated with each of the one or more other users.

Digital assistant feedback and status updates

Execution of application programming interface (API) calls from a non-adjacent augmented reality device includes monitoring network traffic in a network including one or more smart devices connected to the network. Data input corresponding to user interactions with a physical environment is captured from the one or more smart devices connected to the network. A history of API calls stored in the one or more smart devices regarding the user interactions with the physical environment are accessed, based on the network traffic monitoring and the captured data input, changes in the physical environment are identified and mapped to the history of API calls. A handshake is generated between an augmented reality device and at least one of the one or more smart devices to execute an API call from the augmented reality device to the at least one smart device based on the mapping.

Executing application programming interface calls in an augmented reality environment

An embodiment establishes an interaction database based at least in part on interaction data representative of user engagement with a set of features of an application. The embodiment measures user engagement with each feature of the set of features. The embodiment calculates a familiarity score for each feature of the set of features based at least in part on measured user engagement. The embodiment constructs a subset of features of the set of features, wherein the subset of features is characterized by low familiarity score. The embodiment generates at least one recommendation to utilize at least one feature of the subset of features. The embodiment assesses a focal state of a user based on a focal state indicator metric. The embodiment displays the recommendation on a user device at an optimal time, wherein the optimal time is based at least in part on the focal state of the user.

Growing community of inventors

Atlanta, GA, United States of America

Logan Bailey