Gm Global Technolgoy Operations, Inc.

Keith A Fry

Daniel E Rudman

Shane McCutchen

Denis R Burke

Danilo Gurovich

Marco T Carnevale

Mukesh Gupta

Matthew N Hovey

Joseph M Goree, Ii

A system and method of providing augmented reality (AR) remote assistance to a vehicle user of a vehicle. The method includes: sending live video data to a remote assistant application at a remote computer from a portable wireless device (PWD) that is local to the vehicle; receiving the annotation information at the mobile device from the remote assistant application, wherein the annotation information includes annotation graphic information and annotation location information, wherein the annotation graphic information identifies an annotation graphic; and displaying on the electronic display an AR video that includes the annotation graphic located at the annotation display location and over a portion of the live video data, wherein the AR video is updated as the field of view of the electronic camera changes so that the annotation graphic as displayed within the AR video appears fixed to a location within the field of view of the electronic camera.

Augmented reality (AR) remote vehicle assistance

Methods and systems are provided for facilitating stops for vehicles en route to a common destination. A request is received from a first vehicle, the request pertaining to a desire to make a particular type of stop en route to a destination. Information pertaining to the request is transmitted to one or more additional vehicles that are also en route to the destination. Feedback is obtained from the first vehicle, the second vehicle, or both, as to a desired location for the stop.

Methods and systems for facilitating stops for vehicles en route to a common destination

A method of managing speech interfaces to computer-based services includes beginning a first speech session that is carried out in a vehicle over a short-range wireless connection between a vehicle occupant and a mobile device; detecting an initiation of a second speech session while the first speech session is being carried out; determining an assigned priority level of the first speech session relative to an assigned priority level of the second speech session based on a first speech context identified in the first speech session and a second speech context identified in the second speech session; and when the assigned priority level of the second speech session has a higher priority than the assigned priority level of the first speech session, carrying out a session-appropriate action on the first speech session.

Managing in vehicle speech interfaces to computer-based cloud services due recognized speech, based on context

A method of providing hands-free services using a mobile device having wireless access to computer-based services includes receiving speech in a vehicle from a vehicle occupant; recording the speech using a mobile device; transmitting the recorded speech from the mobile device to a cloud speech service; receiving automatic speech recognition (ASR) results from the cloud speech service at the mobile device; and comparing the recorded speech with the received ASR results at the mobile device to identify one or more error conditions.

Centralized speech logger analysis

There is provided a communication system that includes a vehicle and a mobile device, and a method uses that system to provide vehicle owner's manual information. The method includes the steps of: receiving a digital image of a portion of a vehicle; identifying an object of the vehicle in the digital image; associating the object with at least one reference object image stored in an object library comprising a plurality of reference object images; displaying owner's manual information based on the association; and storing at least a portion of the digital image in the object library as a new reference object image for future identifications.

Providing vehicle owner's manual information using object recognition in a mobile device

Computer-implemented methods, systems and apparatus are disclosed for monitoring and reporting notification preferences. A processor of a vehicle determines whether any vehicle events specified in notification preferences have occurred. The notification preferences can be defined by an owner of the vehicle and specify vehicle events that the owner wants to be monitored and reported. When the processor determines that any vehicle event specified in the notification preferences has occurred, the processor can store a record of the vehicle event (and any other vehicle events that occur) in a vehicle event detection log, and then regularly or periodically generate a notification report that includes records for each of the vehicle events that are stored in the vehicle event detection log. The notification report can then be communicated to a remote computer located outside the vehicle. In addition, some vehicle events can trigger generation and communication of an alert message to the remote computer.

Methods, systems and apparatus for determining whether any vehicle events specified in notification preferences have occurred

A mobile voice platform for providing a user speech interface to computer-based services includes a mobile device having a processor, communication circuitry that provides access to the computer-based services, an operating system, and one or more applications that are run using the operating system and that utilize one or more of the computer-based services via the communication circuitry. The mobile voice platform includes at least one non-transient digital storage medium storing a program module having computer instructions that, upon execution by the processor, receives speech recognition results representing user speech that has been processed using automated speech recognition, determines a desired computer-based service based on the speech recognition results, accesses a remotely-stored service interface associated with the desired service, initiates the desired service using the service interface, receives a service result from the desired service, and provides a text-based service response for conversion to a speech response to be provided to the user.

Mobile voice platform architecture with remote service interfaces

A method of providing hands-free services using a mobile device having wireless access to computer-based services includes carrying out a completed speech session via a mobile device without any physical interaction with the mobile device, wherein the speech session includes receiving a speech input from a user, and obtaining from a cloud service a service result responsive to the speech input, and providing the service result as a speech response presented to the user.

Speech-based user interface for a mobile device

A method of providing hands-free services using a mobile device having wireless access to computer-based services includes establishing a short range wireless connection between a mobile device and one or more audio devices that include at least a microphone and speaker; receiving at the mobile device speech inputted via the microphone from a user and sent via the short range wireless connection; wirelessly transmitting the speech input from the mobile device to a speech recognition server that provides automated speech recognition (ASR); receiving at the mobile device a speech recognition result representing the content of the speech input; determining a desired service by processing the speech recognition result using a first, service-identifying grammar; determining a user service request by processing at least some of the speech recognition result using a second, service-specific grammar associated with the desired service; initiating the user service request and receiving a service response; generating an audio message from the service response; and presenting the audio message to the user via the speaker.

Services identification and initiation for a speech-based interface to a mobile device

Methods and systems are provided for associating vehicles en route to a common destination. A request is received from a first vehicle as to an identification of one or more additional vehicles en route to the common destination. The first vehicle is associated with the additional vehicles as part of a caravan. Information is provided as to each of the vehicles in the caravan.

Methods and systems for associating vehicles en route to a common destination

A method of providing hands-free services using a mobile device having wireless access to computer-based services includes establishing a short-range wireless connection between a mobile device and a vehicle; receiving speech in the vehicle that is communicated to a mobile voice platform resident on the mobile device via the short-range wireless connection; parsing the received speech using the mobile voice platform to identify vehicle-specific vocabulary; when vehicle-specific vocabulary exists, identifying at least one vehicle-specific vocabulary term that affects vehicle operation; identifying at least one vehicle function of an inter-related group of vehicle functions that is associated with the identified vehicle-specific vocabulary; determining whether to access one or more cloud services based on the identity of the vehicle function; generating a text-based command reflecting audible control of the affected vehicle function; and sending the text-based command from the mobile device to the vehicle over the short-range wireless connection.

Growing community of inventors

Rochester Hills, MI, United States of America

Keith A Fry