University of Central Florida Research Foundation, Inc.

The University of North Carolina at Chapel Hill

University of Florida Research Foundation, Incorporated

Gregory Francis Welch

Gerd Bruder

Henry Fuchs

Arjun Nagendran

Mindi Anderson

Ryan Schubert

Andrei State

Ryan McMahan

Joseph LaViola, Ii

Francisco Guido-Sanz

Laura Gonzalez

Zubin Choudhary

Karen Aroian

Jason Eric Hochreiter

Steven Talbert

Patricia Weinstein

Kangsoo Kim

Remo Pillat

Kelly Allred

Kurtis P Keller

Jan-Michael Frahm

Richard D Gilson

Nizam Razack

Anselmo A Lastra

Hassan Foroosh

Thomas G Bishop

Mingsong Dou

Andrew Nashel

Matthew Gottsacker

Mark Alan Livingston

John Turner Whitted

Salam Daher

Desiree A Diaz

Feng Zheng

Austin Erickson

Mary Lou Sole

Ryan Edward Schubert

Peter Lincoln

Nahal Norouzi

The invention provides a system for generating spatially explicit auditory cues for a recipient. It includes a processor to receive real-time location data of multiple physical entities relative to the recipient's spatial position and visual orientation. An audio generation module transforms this data into distinct audio signals using Head-Related Transfer Functions (HRTFs) to simulate perceived direction and distance. The output device, such as a headset or vehicle speakers, presents these audio signals spatially to allow perception of entities' relative locations. The system can adjust volume based on entity distance, integrate Doppler effects to indicate motion, and mute entities outside predefined distance boundaries. It also supports above-ground and ground assets, with altitude information converted to audible cues. The system monitors the recipient's orientation in real-time to adjust the HRTFs accordingly.

Spatially explicit auditory cues for enhanced situational awareness

A patient simulation system for healthcare training is provided. The system includes one or more interchangeable shells comprising a physical anatomical model of at least a portion of a patient's body, the shell adapted to be illuminated from within the shell to provide one or more dynamic images viewable on the outer surface of the shells; wherein the system comprises one or more imaging devices enclosed within the shell and adapted to render the one or more dynamic images on an inner surface of the shell and viewable on the outer surface of the shells; one or more interface devices located about the patient shells to receive input and provide output; and one or more computing units in communication with the image units and interface devices, the computing units adapted to provide an interactive simulation for healthcare training. In other embodiments, the shell is adapted to be illuminated from outside the shell.

Physical-virtual patient system

The invention pertains to methods for monitoring the operational status of a home automation system through extrinsic visual and audible means. Initial training periods involve capturing image and audio data representative of nominal operation, which is then processed to identify operational indicators. Unsupervised machine learning models are trained with these indicators to construct a model of normalcy and identify expectation violations in the system's operational pattern. After meeting specific stopping criteria, real-time monitoring is initiated. When an expectation violation is detected, contrastive collages or sequences are generated comprising nominal and anomalous data. These are then transmitted to an end user, effectively conveying the context of the detected anomalies. Further features include providing deep links to smartphone applications for home automation configuration and the use of auditory scene analysis techniques. The invention provides a multi-modal approach to home automation monitoring, leveraging machine learning for robust anomaly detection.

Audiovisual detection of expectation violations in disparate home automation systems

One or more cameras capture objects at a higher resolution than the human eye can perceive. Objects are segmented from the background of the image and scaled to human perceptible size. The scaled-up objects are superimposed over the unscaled background. This is presented to a user via a display whereby the process selectively amplifies the size of the objects' spatially registered retinal projection while maintaining a natural (unmodified) view in the remainder of the visual field.

Spatial positioning of targeted object magnification

The present invention is a system to manage interrupt notifications on an operating system based on the characteristics of content in which an end user is currently immersed or engaged. For example, relatively high bitrate video throughput is indicative of corresponding high information depth and more action occurring in the scene. For periods of high information depth, interrupt notifications are deferred until the information depth falls into a relative trough. Additional embodiments of the invention process scene transitions, technical cues, dialog and lyrics to release queued interrupt notification at optimal times. A vamping process is also provided when interrupt notification are released to keep the end user prescient to the background application in which they were engaged prior to the interrupt notification coming into focus.

Intelligent digital interruption management

An anatomical feature simulation unit is a physical device designed to help simulate an anatomical feature (e.g., a wound) on an object (e.g., a human being or human surrogate such as a medical manikin) for instructing a trainee to learn or practice treatment skills. For the trainee, the simulation looks like a real body part when viewed using an Augmented Reality (AR) system. Responsive to a change in the anatomic state of the object (e.g., bending a knee or raising of an arm) not only the spatial location and orientation of the anatomical feature stays locked on the object in the AR system, but the characteristics of the anatomical feature change based on the physiologic logic of changing said anatomical state (e.g., greater or less blood flow, opening or closing of a wound).

Adaptive visual overlay for anatomical simulation

A system for translating text streams of alphanumeric characters into preconfigured, haptic output. Text strings are parsed against a grammar index to locate assigned haptic or vibratory output. This may include speech-to-text, chat messaging, or text arrays of any kind. When a match is located, a standardized haptic output pattern is invoked through a haptic device. A device affordance module adapts the haptic output pattern to the capabilities of the target haptic device.

Grammar dependent tactile pattern invocation

Virtual humans exhibit behaviors associated with inputs and outputs of an autonomous control system for medical monitoring of patients. To foster the awareness and trust, the virtual humans exhibit situational awareness via apparent (e.g., rendered) behaviors based on inputs such as physiological vital signs. The virtual humans also exhibit situational control via apparent behaviors associated with outputs such as direct control of devices, functions of control, actions based on high-level goals, and the optional use of virtual versions of conventional physical controls. A dynamic virtual human who continually exhibits awareness of the system state and relevant contextual circumstances, along with the ability to directly control the system, is used to reduce negative feelings associated with the system such as uncertainty, concern, stress, or anxiety on the part of real human patients.

Medical monitoring virtual human with situational awareness

This invention relates to tracking of hand-held devices and vehicles with respect to each other, in circumstances where there are two or more users or existent objects interacting in the same share space (co-location). It extends conventional global and body-relative approaches to 'cooperatively' estimate the relative poses between all useful combinations of user-worn tracked devices such as HMDs and hand-held controllers worn (or held) by multiple users. Additionally, the invention provides for tracking of vehicles such as cars and unmanned aerial vehicles.

Augmentation of relative pose in co-located devices

One or more cameras capture objects at a higher resolution than the human eye can perceive. Objects are segmented from the background of the image and scaled to human perceptible size. The scaled-up objects are superimposed over the unscaled background. This is presented to a user via an augmented reality display whereby the process selectively amplifies the size of the objects' spatially registered retinal projection while maintaining a natural (unmodified) view in the remainder of the visual field.

Intelligent object magnification for augmented reality displays

A system for remote tactile telepresence wherein tactile telepresence sessions are time-coded along with audiovisual content wherein playback is heard, seen, and felt. Embodiments associate motion capture with the tactile profile so that remote, haptic recipients may see renderings of objects (e.g., hands) imparting vibrotactile sensations.

Visual-tactile virtual telepresence

Dynamic online and real-time virtual humans exhibit behaviors associated with inputs and outputs of an autonomous control system. To foster the awareness and trust, the virtual humans exhibit situational awareness via apparent (e.g., rendered) behaviors based on inputs such as direct system measures, functions of the measures, estimates of state based on measures, and estimates of state based on a priori constraints. The virtual humans also exhibit situational control via apparent behaviors associated with outputs such as direct control of devices, functions of control, actions based on high-level goals, and the optional use of virtual versions of conventional physical controls. A dynamic virtual human who continually exhibits awareness of the system state and relevant contextual circumstances, along with the ability to directly control the system, is used to reduce negative feelings associated with the system such as uncertainty, concern, stress, or anxiety on the part of real humans.

Autonomous systems human controller simulation

A system for remote tactile telepresence wherein an array of predefined touch gestures are abstracted into cataloged values and invoked either by pattern matching, by assigned name or visual indicia. A local and remote cache of the catalog reduces latency even for complicated gestures as only a gesture identifier needs to be transmitted to a haptic output destination. Additional embodiments translate gestures to different haptic device affordances. Tactile telepresence sessions are time-coded along with audiovisual content wherein playback is heard, seen, and felt. Another embodiment associates motion capture associated with the tactile profile so that remote, haptic recipients may see renderings of objects (e.g., hands) imparting vibrotactile sensations.

Low latency tactile telepresence

Methods, systems, and computer readable media for minimal-latency tracking and display for matching real and virtual worlds in head-worn displays are disclosed. According to one aspect, a method for minimal-latency tracking and display for matching real and virtual worlds in head-worn displays includes calculating a desired image, calculating an error image as the difference between the desired image and an image currently being perceived by a user, identifying as an error portion a portion of the error image having the largest error, updating a portion of a projected image that corresponds to the error portion, and recalculating the image currently being perceived by a user based on the updated projected image.

Minimal-latency tracking and display for matching real and virtual worlds in head-worn displays

This invention relates to tracking of user-worn and hand-held devices with respect to each other, in circumstances where there are two or more users interacting in the same share space. It extends conventional global and body-relative approaches to 'cooperatively' estimate the relative poses between all useful combinations of user-worn tracked devices such as HMDs and hand-held controllers worn (or held) by multiple users. For example, a first user's HMD estimates its absolute global pose in the coordinate frame associated with the externally-mounted devices, as well as its relative pose with respect to all other HMDs, hand-held controllers, and other user held/worn tracked devices in the environment. In this way, all HMDs (or as many as appropriate) are tracked with respect to each other, all HMDs are tracked with respect to all hand-held controllers, and all hand-held controllers are tracked with respect to all other hand-held controllers.

Growing community of inventors

Heathrow, FL, United States of America

Gregory Francis Welch