Sri International

Sarnoff Corporation

L-3 Communications Corporation

Visteon Global Technolgoies, Inc.

David Sarnoff Research Center, Inc.

Imax Corporation

Rakesh Kumar

Harpreet Singh Sawhney

Supun Samarasekera

Keith James Hanna

James Russell Bergen

Taragay Oskiper

Manoj Aggarwal

Han-Pang Chiu

Zhiwei Zhu

Ying Shan

Stephen Charles Hsu

Feng Han

Vincent Paragano

Aydin Arpa

Oleg Naroditsky

Bogdan C Matei

Mikhail Sizintsev

Ryan Villamil

Yanlin Guo

Raia Hadsell

Hai Tao

Bogdan Calin Mihai Matei

Garbis Salgian

Zachary Seymour

Niluthpol C Mithun

Jayan Eledath

Vlad Branzoi

Tao Zhao

Jeffrey Lubin

Peter Jeffrey Burt

Girish Acharya

Michael John Wolverton

Sang-Hack Jung

Jiangjian Xiao

Glenn Murray

Hui Ting Cheng

Mayank Bansal

David Kalokitis

Padmanabhan Anandan

David R Patterson

Philip Miller

Douglas Fremont Dixon

Gary A Gendel

Jane Asmuth

Steve Hsu

Varun Murali

Janet Kim

David Nister

Wenyi Zhao

Yuichi Ikeda

Bhaskar S Ramamurthy

Michal Irani

Necip Fazil Ayan

Jayakrishnan Kumar Eledath

James R Carpenter

John R Fields

Curtis R Carlson

Michael Henry Brill

Lambert Ernest Wixson

David James Hirvonen

Karan Sikka

John Benjamin Southall

Yi Yao

Subhodev Das

Tamaki Horii

Richard Patrick Wildes

Yi Tan

Faroog Abdel-kareem Ibrahim

Massimiliano De Zambotti

Fiona C Baker

Jonathan D Brookshire

Shmuel Peleg

Ian M Colrain

Hidefumi Takenaka

Michael Raymond Piacentino

Aaron Spaulding

Abhinav Rajvanshi

Doug Dixon

Louise Yarnall

Thomas Germano

Charles Fielding Finch Karney

Ryan Cekander

Vivek Verma

Supun Samarasakera

Wen-Yi Zhao

Norman Winarsky

Bodgan Matei

William Brian Lehman

Thomas Edward Klinedinst

Barbara Levienaise-Obadia

Xun Zhou

Mike Tinker

Kevin Williams

Janet Yonga Kim Knowles

Gregory Drew Kessler

Targay Oskiper

Vince Paragano

Lu Wang

Supun Samarasekara

Sujit Kuthirummal

A method, apparatus and system for object detection in sensor data having at least two modalities using a common embedding space includes creating first modality vector representations of features of sensor data having a first modality and second modality vector representations of features of sensor data having a second modality, projecting the first and second modality vector representations into the common embedding space such that related embedded modality vectors are closer together in the common embedding space than unrelated modality vectors, combining the projected first and second modality vector representations, and determining a similarity between the combined modality vector representations and respective embedded vector representations of features of objects in the common embedding space to identify at least one object depicted by the captured sensor data. In some instances, data manipulation of the method, apparatus and system can be guided by physics properties of a sensor and/or sensor data.

Physics-guided deep multimodal embeddings for task-specific data exploitation

A method, apparatus and system for efficient navigation in a navigation space includes determining semantic features and respective 3D positional information of the semantic features for scenes of captured image content and depth-related content in the navigation space, combining information of the determined semantic features of the scene with respective 3D positional information using neural networks to determine an intermediate representation of the scene which provides information regarding positions of the semantic features in the scene and spatial relationships among the semantic features, and using the information regarding the positions of the semantic features and the spatial relationships among the semantic features in a machine learning process to provide at least one of a navigation path in the navigation space, a model of the navigation space, and an explanation of a navigation action by the single, mobile agent in the navigation space.

System and method for efficient visual navigation

A method, machine readable medium and system for semantic segmentation of 3D point cloud data includes determining ground data points of the 3D point cloud data, categorizing non-ground data points relative to a ground surface determined from the ground data points to determine legitimate non-ground data points, segmenting the determined legitimate non-ground and ground data points based on a set of common features, applying logical rules to a data structure of the features built on the segmented determined non-ground and ground data points based on their spatial relationships and incorporated within a machine learning system, and constructing a 3D semantics model from the application of the logical rules to the data structure.

Fully automated multimodal system architecture for semantic segmentation of large-scale 3D outdoor point cloud data

A method, apparatus and system for artificial intelligence-based HDRL planning and control for coordinating a team of platforms includes implementing a global planning layer for determining a collective goal and determining, by applying at least one machine learning process, at least one respective platform goal to be achieved by at least one platform, implementing a platform planning layer for determining, by applying at least one machine learning process, at least one respective action to be performed by the at least one of the platforms to achieve the respective platform goal, and implementing a platform control layer for determining at least one respective function to be performed by the at least one of the platforms. In the method, apparatus and system despite the fact that information is shared between at least two of the layers, the global planning layer, the platform planning layer, and the platform control layer are trained separately.

Artificial intelligence-based hierarchical planning for manned/unmanned platforms

Techniques for augmenting a reality captured by an image capture device are disclosed. In one example, a system includes an image capture device that generates a two-dimensional frame at a local pose. The system further includes a computation engine executing on one or more processors that queries, based on an estimated pose prior, a reference database of three-dimensional mapping information to obtain an estimated view of the three-dimensional mapping information at the estimated pose prior. The computation engine processes the estimated view at the estimated pose prior to generate semantically segmented sub-views of the estimated view. The computation engine correlates, based on at least one of the semantically segmented sub-views of the estimated view, the estimated view to the two-dimensional frame. Based on the correlation, the computation engine generates and outputs data for augmenting a reality represented in at least one frame captured by the image capture device.

Augmenting reality using semantic segmentation

Methods and apparatuses for tracking objects comprise one or more optical sensors for capturing one or more images of a scene, wherein the one or more optical sensors capture a wide field of view and corresponding narrow field of view for the one or more images of a scene, a localization module, coupled to the one or more optical sensors for determining the location of the apparatus, and determining the location of one more objects in the one or more images based on the location of the apparatus and an augmented reality module, coupled to the localization module, for enhancing a view of the scene on a display based on the determined location of the one or more objects.

Augmented reality vision system for tracking and geolocating objects of interest

A computing system includes a vision-based user interface platform to, among other things, analyze multi-modal user interactions, semantically correlate stored knowledge with visual features of a scene depicted in a video, determine relationships between different features of the scene, and selectively display virtual elements on the video depiction of the scene. The analysis of user interactions can be used to filter the information retrieval and correlating of the visual features with the stored knowledge.

Real-time human-machine collaboration using big data driven augmented reality technologies

A method, apparatus and system for visual localization includes extracting appearance features of an image, extracting semantic features of the image, fusing the extracted appearance features and semantic features, pooling and projecting the fused features into a semantic embedding space having been trained using fused appearance and semantic features of images having known locations, computing a similarity measure between the projected fused features and embedded, fused appearance and semantic features of images, and predicting a location of the image associated with the projected, fused features. An image can include at least one image from a plurality of modalities such as a Light Detection and Ranging image, a Radio Detection and Ranging image, or a 3D Computer Aided Design modeling image, and an image from a different sensor, such as an RGB image sensor, captured from a same geo-location, which is used to determine the semantic features of the multi-modal image.

Semantically-aware image-based visual localization

During GPS-denied/restricted navigation, images proximate a platform device are captured using a camera, and corresponding motion measurements of the platform device are captured using an IMU device. Features of a current frame of the images captured are extracted. Extracted features are matched and feature information between consecutive frames is tracked. The extracted features are compared to previously stored, geo-referenced visual features from a plurality of platform devices. If one of the extracted features does not match a geo-referenced visual feature, a pose is determined for the platform device using IMU measurements propagated from a previous pose and relative motion information between consecutive frames, which is determined using the tracked feature information. If at least one of the extracted features matches a geo-referenced visual feature, a pose is determined for the platform device using location information associated with the matched, geo-referenced visual feature and relative motion information between consecutive frames.

Collaborative navigation and mapping

Computer aided inspection systems (CAIS) and method for inspection, error analysis and comparison of structures are presented herein. In some embodiments, a CAIS may include a SLAM system configured to determine real-world global localization information of a user in relation to a structure being inspected using information obtained from a first sensor package, a model alignment system configured to: use the determined global localization information to index into a corresponding location in a 3D computer model of the structure being inspected; and align observations and/or information obtained from the first sensor package to the local area of the model 3D computer model of the structure extracted; a second sensor package configured to obtain fine level measurements of the structure; and a model recognition system configured to compare the fine level measurements and information obtained about the structure from the second sensor package to the 3D computer model.

Computer aided inspection system and methods

A method for providing a real time, three-dimensional (3D) navigational map for platforms includes integrating at least two sources of multi-modal and multi-dimensional platform sensor information to produce a more accurate 3D navigational map. The method receives both a 3D point cloud from a first sensor on a platform with a first modality and a 2D image from a second sensor on the platform with a second modality different from the first modality, generates a semantic label and a semantic label uncertainty associated with a first space point in the 3D point cloud, generates a semantic label and a semantic label uncertainty associated with a second space point in the 2D image, and fuses the first space semantic label and the first space semantic uncertainty with the second space semantic label and the second space semantic label uncertainty to create fused 3D spatial information to enhance the 3D navigational map.

Multi-modal data fusion for enhanced 3D perception for platforms

Techniques are disclosed for improving navigation accuracy for a mobile platform. In one example, a navigation system comprises an image sensor that generates a plurality of images, each image comprising one or more features. A computation engine executing on one or more processors of the navigation system processes each image of the plurality of images to determine a semantic class of each feature of the one or more features of the image. The computation engine determines, for each feature of the one or more features of each image and based on the semantic class of the feature, whether to include the feature as a constraint in a navigation inference engine. The computation engine generates, based at least on features of the one or more features included as constraints in the navigation inference engine, navigation information. The computation engine outputs the navigation information to improve navigation accuracy for the mobile platform.

Semantic visual landmarks for navigation

A computing system for virtual personal assistance includes technologies to, among other things, correlate an external representation of an object with a real world view of the object, display virtual elements on the external representation of the object and/or display virtual elements on the real world view of the object, to provide virtual personal assistance in a multi-step activity or another activity that involves the observation or handling of an object and a reference document.

Augmented reality virtual personal assistant for external representation

Embodiments of the present invention generally relate to computer aided rebar measurement and inspection systems. In some embodiments, the system may include a data acquisition system configured to obtain fine-level rebar measurements, images or videos of rebar structures, a 3D point cloud model generation system configured to generate a 3D point cloud model representation of the rebar structure from information acquired by the data acquisition system, a rebar detection system configured to detect rebar within the 3D point cloud model generated or the rebar images or videos of the rebar structures, a rebar measurement system to measure features of the rebar and rebar structures detected by the rebar detection system, and a discrepancy detection system configured to compare the measured features of the rebar structures detected by the rebar detection system with a 3D Building Information Model (BIM) of the rebar structures, and determine any discrepancies between them.

Computer aided rebar measurement and inspection system

A method and apparatus for training and guiding users comprising generating a scene understanding based on video and audio input of a scene of a user performing a task in the scene, correlating the scene understanding with a knowledge base to produce a task understanding, comprising one or more goals, of a current activity of the user, reasoning, based on the task understanding and a user's current state, a next step for advancing the user towards completing one of the one or more goals of the task understanding and overlaying the scene with an augmented reality view comprising one or more visual and audio representation of the next step to the user.

Method and apparatus for mentoring via an augmented reality assistant

Growing community of inventors

West Windsor, NJ, United States of America

Rakesh Kumar