Purdue Research Foundation

Other

Tata Consultancy Services Limited

Zimmer Gmbh

Karthik Ramani

Vinayak Raman Krishnamurthy

Ke Huo

Ayan Sinha

Cecil Piya

Tianyi Wang

Chiho Choi

Joon Hee Choi

Sang Ho Yoon

Yuanzhi Cao

Fengming He

Xun Qian

Thirumalai N Devanathan

Devadatta Madhukar Kulkarni

Ana Villanueva Pérez

F Vinayak

Raja Jasti

Zhuangying Xu

Kuiyang Lou

Subramaniam Jayanti

Natraj Iyer

Jiantao Pu

Subramaniam Chidambaram

William Z Bernstein

Devarajan Ramanujan

Jeffery Tew

Hank Huang

Michael John McCoy

Jamie Tan

Nainesh Rathod

You Wu

Sangpil Kim

Hyung-gun Chi

Chan Woo Chung

Mark Stephen Duvall

Hairong Liu

Daniel Edward Woolard

Gaoping Huang

Alexander J Quinn

Zhengzhe Zhu

Ziyi Liu

Jingyu Shi

Rahul Jain

Aliasgar Ganiji

Nikhil Vishwanath Joglekar

Mahendra Babu Arugundram Hrikrishna Moorthy

Terrell Kendall Glenn

Caleb Carithers

Ananya Ipsita

A learning system is disclosed that leverages intention-driven causality to enhance skill learning. The learning system enables an author to easily develop mixed reality (MR) tutorial content for performing a task that advantageously captures causal relationships between steps and, thus, enables such causality to be conveyed to the novice user when learning how to perform the task. To this end, the learning system leverages a novel hierarchical representation of causality and intention alongside a systematic workflow suitable for designing skill learning content. By preserving and presenting causal information to the novice user, the user can better understand not only the steps required to perform a task, but also why each step is performed.

Visualizing causality in mixed reality for manual task learning

A machine task tutorial system is disclosed that utilizes augmented reality to enable an expert user to record a tutorial for a machine task that can be learned by different trainee users in an adaptive manner. The machine task tutorial system advantageously utilizes an adaptation model that focuses on spatial and bodily visual presence for machine task tutoring. The machine task tutorial system advantageously enables adaptive tutoring in the recorded-tutorial environment based on machine state and user activity recognition. The machine task tutorial system advantageously utilizes AR to provide tutorial recording, adaptive visualization, and state recognition. In this way, the machine task tutorial system supports more effective apprenticeship and training for machine tasks in workshops or factories.

Adaptive tutoring system for machine tasks in augmented reality

An augmented reality (AR) interaction authoring system is described. The AR interaction authoring system is configured to support the real-time creation of AR applications to support AR-enhanced toys. The design of the AR interaction authoring system enables bidirectional interactions between the physical-virtual space of toys and AR. The AR interaction authoring system allows intuitive authoring of AR animations and toys actuations through programming by demonstration, while referring to the physical toy for a contextual reference. Using a visual programming interface, users can create bidirectional interactions by utilizing users' input on toys to trigger AR animations and vice versa. A plug-and-play IoT toolkit is also disclosed that includes hardware to actuate common toys. In this way, users can effortlessly integrate toys into the virtual world in an impromptu design process, without lengthy electronic prototyping.

Authoring systems and methods for enabling bidirectional binding of augmented reality with toys in real-time

An augmented reality system for developing and providing augmented reality learning experiences is disclosed. The augmented reality system advantageously combines augmented reality with the capabilities of cloud technology to provide a pull-based collaborative model, in which students and instructors collaborate by uploading, sharing, and downloading augmented reality learning content. The augmented reality system enables students to improve the augmented reality learning content by adding contributions to the original augmented reality learning content that was created by an instructor.

Augmented reality platform for collaborative classrooms

A method and system for localizing a plurality of stationary devices, such as Internet of Things (IoT devices), arranged in an environment is disclosed. A mobile device is configured to survey an environment to generate a three-dimensional map of the environment using simultaneous localization and mapping (SLAM) techniques. The mobile device and the stationary devices are equipped with wireless transceivers, such as Ultra-wideband radios, for measuring distances between the devices using wireless ranging techniques. Based on the measured distances, the mobile device is configured to determine locations of the stationary devices in a reference frame of the three-dimensional map. In some embodiments, the determined locations can be used to enable a variety of spatially aware augmented reality features and interactions between the mobile device and the stationary device.

System and method for spatially mapping smart objects within augmented reality scenes

Disclosed is a visual and spatial programming system for robot navigation and robot-IoT task authoring. Programmable mobile robots serve as binding agents to link stationary IoT devices and perform collaborative tasks. Three key elements of robot task planning (human-robot-IoT) are coherently connected with one single smartphone device. Users can perform visual task authoring in an analogous manner to the real tasks that they would like the robot to perform with using an augmented reality interface. The mobile device mediates interactions between the user, robot(s), and IoT device-oriented tasks, guiding the path planning execution with Simultaneous Localization and Mapping (SLAM) to enable robust room-scale navigation and interactive task authoring.

Augmented reality interface for authoring tasks for execution by a programmable robot

An augmented reality (AR) application authoring system is disclosed. The AR application authoring system enables the real-time creation of freehand interactive AR applications with freehand inputs. The AR application authoring system enables intuitive authoring of customized freehand gesture inputs through embodied demonstration while using the surrounding environment as a contextual reference. A visual programming interface is provided with which users can define freehand interactions by matching the freehand gestures with reactions of virtual AR assets. Thus, users can create personalized freehand interactions through simple trigger-action programming logic. Further, with the support of a real-time hand gesture detection algorithm, users can seamlessly test and iterate on the authored AR experience.

System and method for authoring freehand interactive augmented reality applications

A system and method for authoring and performing Human-Robot-Collaborative (HRC) tasks is disclosed. The system and method adopt an embodied authoring approach in Augmented Reality (AR), for spatially editing the actions and programming the robots through demonstrative role-playing. The system and method utilize an intuitive workflow that externalizes user's authoring as demonstrative and editable AR ghost, allowing for spatially situated visual referencing, realistic animated simulation, and collaborative action guidance. The system and method utilize a dynamic time warping (DTW) based collaboration model which takes the real-time captured motion as inputs, maps it to the previously authored human actions, and outputs the corresponding robot actions to achieve adaptive collaboration.

System and method for embodied authoring of human-robot collaborative tasks with augmented reality

Some examples provides a set of frameworks, process and methods aimed at enabling the expression and exploration of free-form and parametric 3D shape designs enabled through natural interactions with a hand-held mobile device acting as a controller for 3D virtual objects. A reference plane in a virtual space generated by the location of the mobile device may be used to select a 3D virtual object intersected by the reference plane. Positioning of the mobile device may also be used to control a pointer in the virtual space. In an example, the orientation of the mobile device may be detected by an accelerometer or gyroscope. In example, the position of the mobile device may be detected by a position sensor.

Defining and deforming 3D virtual objects using cross-sectional shapes

A method for hand pose identification in an automated system includes providing map data of a hand of a user to a first neural network trained to classify features corresponding to a joint angle of a wrist in the hand to generate a first plurality of activation features and performing a first search in a predetermined plurality of activation features stored in a database in the memory to identify a first plurality of hand pose parameters for the wrist associated with predetermined activation features in the database that are nearest neighbors to the first plurality of activation features. The method further includes generating a hand pose model corresponding to the hand of the user based on the first plurality of hand pose parameters and performing an operation in the automated system in response to input from the user based on the hand pose model.

Method and system for hand pose detection

A system and method for authoring and implementing context-aware applications (CAPs) are disclosed. The system and method enables users to record their daily activities and then build and deploy customized CAPs onto augmented reality platforms in which automated actions are performed in response to user-defined human actions. The system and method utilizes an integrated augmented reality platform composed of multiple camera systems, which allows for non-intrusive recording of end-users' activities and context detection while authoring and implementing CAPs. The system and method provides an augmented reality authoring interface for browsing, selecting, and editing recorded activities, and creating flexible CAPs through spatial interaction and visual programming.

System and method for authoring human-involved context-aware applications

Robot navigation and robot-IoT interactive task planning using augmented reality

A sensing system includes a base material, a plurality of electrodes, a capacitive sensing channel and a controller. The stretchable base material has a resistance distribution that changes in response to being mechanically deformed as a result of a human body contact. The base material has a rebound elasticity. The electrodes are attached to a perimeter of the base material, the capacitive sensing channel is attached to the base material. The controller is operatively connected to the plurality of electrodes and the capacitive sensing channel. The controller is configured to measure instantaneous voltage measurements from the plurality of electrodes, and determine whether the base material is mechanically deformed based on the instantaneous voltage measurements using a support vector machine classifier.

Flexible touch sensing system and method with deformable material

A method of operating an augmented reality (AR) system includes capturing images of a first real-world workspace using a camera of a first head mounted AR device of the AR system being worn by a first user, processing the images using a first processor of the AR system to identify physical objects in the first real-world workspace and detect 3D positions of the identified physical objects in a 3D space corresponding to the first real-world workspace, rendering virtual objects representing the identified physical objects on the display of the first head mounted AR device at the respective 3D positions for the identified physical objects, manipulating a first one of the virtual objects using at least one hand-held controller of the AR system in a manner that mimics a performance of a first procedural task using the physical object associated with the first one of the virtual objects, recording the manipulation of the first one of the virtual objects that mimics the performance of the first procedural task as first augmented reality content, and storing the first augmented reality content in a memory of the AR system.

System and method for generating asynchronous augmented reality instructions

Manipulating 3D virtual objects using hand-held controllers

Growing community of inventors

West Lafayette, IN, United States of America

Karthik Ramani