Retrocausal, Inc.

Microsoft Technology Licensing, LLC

Andrey Konin

Muhammad Zeeshan Zia

Quoc-Huy Tran

Sateesh Kumar

Sanjay Haresh

Michael Bleyer

Yuri Pekelny

Amol Ashok Ambardekar

Pedro Urbina Escos

Ning Xu

Sheng Kai Tang

William Douglas Guyman

Harpreet Singh Sawhney

Awais Ahmed

Hamza Khan

Shahram Najam Syed

Muhammad Shakeeb Hussain Siddiqui

Bilha-Catherine W Githinji

A ML-based method and system for optimizing industrial processes, is disclosed. The ML-based method includes: obtaining image data and video data associated with objects, from video capturing units installed on industrial floors; obtaining positioning information of the objects from positioning systems configured on the objects; computing trajectories for the objects, using a coordinate transformation module based on streams corresponding to the image data, the video data, and the positioning information; combining the trajectories associated with the objects, into a common coordinate frame, using the coordinate transformation module; determining data associated with tracks, and metadata, for the objects, using the coordinate transformation module; and combining information associated with the data associated with the tracks, and the metadata, to predict optimized industrial processes, using a combiner of a spatio-temporal reasoning engine with a machine learning (ML) model.

Machine learning based systems and methods for optimizing industrial processes by analyzing layouts of environments

A ML-based method and system for optimizing job efficiency based on generation of predetermined motion time system (PMTS) representations for tasks associated with jobs, is disclosed. The ML-based method includes: obtaining description data corresponding to the tasks associated with the jobs, in representations comprising at least one of: text representations, image representations, video representations, and CAD representations; generating the PMTS representations of the tasks associated with the jobs, based on the obtained description data corresponding to the tasks associated with the jobs, using a machine learning model; generating insights corresponding to the tasks associated with the jobs, based on PMTS representations of the tasks associated with the jobs, using the machine learning model; and converting one PMTS representation to another PMTS representation, using the machine learning model.

Machine learning based systems and methods for optimizing job efficiency based on generation of predetermined motion time system (PMTS) representations for tasks associated with jobs

A system and method for determining sub-activities in videos and segmenting the videos is disclosed. The method includes extracting one or more batches from one or more videos and extracting one or more features from set of frames associated with the one or more batches. The method further includes generating a set of predicted codes and determining a cross-entropy loss, temporal coherence loss and a final loss. Further, the method includes categorizing the set of frames into one or more predefined clusters and generating one or more segmented videos based on the categorized set of frames, the determined final loss, and the set of predicted codes by using s activity determination-based ML model. The method includes outputting the generated one or more segmented videos on user interface screen of one or more electronic devices associated with one or more users.

System and method for determining sub-activities in videos and segmenting the videos with little to no annotation

A system and method for generating industrial process in a computing environment is disclosed. The system receives multimedia input from users, analyzes multimedia input, to determine semantics associated with multimedia inputs, using at least one of natural language and symbolic processing techniques, and deep learning technique. Furthermore, the system determines process specifications and descriptions in industrial process, using at least one of a dataset and library comprising representations of industrial processes in plurality of configurations, using machine learning (ML) models. Additionally, the system combines a plurality of types of process specifications and descriptions. Further, the system generates industrial processes corresponding to the multimedia inputs, based on combining the plurality of types of process specifications and descriptions. Furthermore, the system outputs the generated industrial processes, on at least one of display of a user device, and external devices.

System and method for generating industrial processes in a computing environment

A system and method for optimizing industrial assembly process in an industrial environment is disclosed. A system operates on artificial intelligence (AI) based conversational/GUI platform, where it receives user commands related to industrial assembly process improvement queries. By analyzing received user commands, system identifies type of industrial assembly process mentioned by extracting relevant keywords or other attributes. Using trained AI-based classification table, system determines performance attributes associated with identified type of process. The system leverages various sources such as domain knowledge, organization-specific knowledge bases, data from tools/internet-based services, and statistical measurements from industrial environment. Based on results from language model, system generates a response to user command, which can include insights on performance level of industrial assembly or packaging process, recommendations for improvements, and actuation commands for operating machinery in industrial environment. Finally, system outputs generated response on AI-based conversational/GUI platform in multiple representation forms.

System and method for optimizing industrial assembly processes

A system and method for learning human activities from video demonstrations using video augmentation is disclosed. The method includes receiving original videos from one or more data sources. The method includes processing the received original videos using one or more video augmentation techniques to generate a set of augmented videos. Further, the method includes generating a set of training videos by combining the received original videos with the generated set of augmented videos. Also, the method includes generating a deep learning model for the received original videos based on the generated set of training videos. Further, the method includes learning the one or more human activities performed in the received original videos by deploying the generated deep learning model. The method includes outputting the learnt one or more human activities performed in the original videos.

System and method for learning human activities from video demonstrations using video augmentation

A system and method for correlating video frames in a computing environment. The method includes receiving first video data and second video data from one or more data sources. The method further includes encoding the received first video data and the second video data using machine learning network. Further, the method includes generating first embedding video data and second embedding video data corresponding to the received first video data and the received second video data. Additionally, the method includes determining a contrastive IDM temporal regularization value for the first video data and the second video data. The method further includes determining temporal alignment loss between the first video data and the second video data. Also, the method includes determining correlated video frames between the first video data and the second video databased on the determined temporal alignment loss and the determined contrastive IDM temporal regularization value.

System and method for correlating video frames in a computing environment

A system and method for management and evaluation of one or more human activities is disclosed. The method includes receiving live videos from data sources. The live videos comprises activity performed by human. The activity comprises actions performed by the human. Further, the method includes detecting the actions performed by the human in the live videos using a neural network model. The method further includes generating a procedural instruction set for the activity performed by the human. Also, the method includes validating quality of the identified actions performed by the human using the generated procedural instruction set. Furthermore, the method includes detecting anomalies in the actions performed by the human based on results of validation. Additionally, the method includes generating rectifiable solutions for the detected anomalies. Moreover, the method includes outputting the rectifiable solutions on a user interface of a user device.

System and method for management and evaluation of one or more human activities

A method for object recognition includes, at a computing device, receiving an image of a real-world object. An identity of the real-world object is recognized using an object recognition model trained on a plurality of computer-generated training images. A digital augmentation model corresponding to the real-world object is retrieved, the digital augmentation model including a set of augmentation-specific instructions. A pose of the digital augmentation model is aligned with a pose of the real-world object. An augmentation is provided, the augmentation associated with the real-world object and specified by the augmentation-specific instructions.

Real-world object recognition for computing device

A system for building computational models of a goal-driven task from demonstration is disclosed. A task recording subsystem receives a recorded video file or recorded sensor data representative of an expert demonstration for a task. An instructor authoring tool generates one or more sub-activity proposals; enables an instructor to specify one or more sub-activity labels upon modification of the one or more sub-activity proposals into one or more sub-tasks. A task learning subsystem learns the one or more sub-tasks represented in the demonstration of the task; builds an activity model to predict and locate the task being performed in the recorded video file. A task evaluation subsystem evaluates a live video representative of the task; generates at least one performance description statistics; identifies a type of activity step executed by the one or more actors; provides an activity guidance feedback in real-time to the one or more actors.

System and method for building computational models of a goal-driven task from demonstration

Facilitating input to the computing system by displaying an input area on the palm of a human hand, and allowing easy input mode changes using gestures of that hand. Computer vision is used to detect the palm of a human hand. Augmented reality is used to display the input area on that palm of the display human hand. Computer vision may then be used to detect when the other human hand of that user interfaces with the input area. The input area has multiple input modes that each define how input from the input human hand is interpreted by the computing system. In response to the computer vision detected an input mode changing gesture of the display human hand, the computing system changes the input mode of the input area so as to change how input provided by the input human hand is interpreted by the computing system.

Mode-changeable augmented reality interface

Techniques are provided to rank a set of candidate hologram placement locations based on a defined set of rules or criteria. An environment's spatial mapping is initially accessed. This spatial mapping describes the environment in a 3D representation. By analyzing this spatial mapping, an unordered set of candidate hologram placement locations is generated. These candidate hologram placement locations are locations that are identified from within the environment as being locations that are potentially suitable for hosting a hologram. To be selected, each of the candidate hologram placement locations is required to satisfy certain minimum criteria. Thereafter, these candidate hologram placement locations are then ranked based on a set of pre-defined hologram placement criteria to generate a set of ranked candidate hologram placement locations. This set of ranked candidate hologram placement locations is then exposed by causing the set to be accessible during a hologram placement event.

Growing community of inventors

Redmond, WA, United States of America

Andrey Konin