Sensory, Inc.

Sensory Circuits, Inc.

Todd F Mozer

Forrest S Mozer

Pieter J Vermeulen

Bryan Pellom

Jonathan Shaw

Erich Adams

Michael C Mozer

Jonathan Welch

John-Paul Hosom

Jeff Rogers

Thomas North

Pieter J Verneulen

Pieter Vermeulen

Techniques for sanitizing personally identifiable information (PII) from audio and visual data are provided. For example, in a scenario where the data comprises an audio signal with speech uttered by a speaker S, these techniques can include removing, obfuscating, or transforming speech related and non-speech related audio cues in the audio signal that can be used to trace the identity of S, while allowing the content of S's speech to remain recognizable. As another example, in a scenario where the data comprises an image or video in which a person P appears, these techniques can include removing, obfuscating, or transforming P's visible biological features and visual indicators of P's location, belongings, or personal data in the image/video, while allowing the general nature of the footage to remain discernable. Through this PII sanitization process, the privacy of individuals portrayed in the audio or visual data can be preserved.

Sanitizing personally identifiable information (PII) in audio and visual data

Techniques for sanitizing personally identifiable information (PII) from audio and visual data are provided. For instance, in a scenario where the data comprises an audio signal with speech uttered by a person P, these techniques can include removing/obfuscating/transforming speech-related PII in the audio signal such as pitch and acoustic cues associated with P's vocal tract shape and/or vocal actuators (e.g., lips, nasal air bypass, teeth, tongue, etc.) while allowing the content of the speech to remain recognizable. Further, in a scenario where the data comprises a still image or video in which a person P appears, these techniques can include removing/obfuscating/transforming visual PII in the image or video such as P's biological features and indicators of P's location/belongings/data while allowing the general nature of the image or video to remain discernable. Through this PII sanitization process, the privacy of individuals portrayed in the audio or visual data can be preserved.

Smart listening modes for supporting quasi always-on listening on an electronic device are provided. In one embodiment, the electronic device can determine that a user is likely to utter a voice trigger in order to access the always-on listening functionality of the electronic device. In response to this determination, the electronic device can automatically enable the always-on listening functionality. Similarly, the electronic device can determine that a user is no longer likely to utter the voice trigger in order to access the always-on listening functionality of the electronic device. In response to this second determination, the electronic device can automatically disable the always-on listening functionality.

Smart listening modes supporting quasi always-on listening

Techniques for implementing dynamic volume adjustment by a virtual assistant are provided. In one embodiment, the virtual assistant can receive a voice query or command from a user, recognize the content of the voice query or command, process the voice query or command based on the recognized content, and determine an auditory response to be output to the user. The virtual assistant can then identify a plurality of criteria for automatically determining an output volume level for the response, where the plurality of criteria including content-based criteria and environment-based criteria, calculate values for the plurality of criteria, and combine the values to determine the output volume level. The virtual assistant can subsequently cause the auditory response to be output to the user at the determined output volume level.

Dynamic volume adjustment for virtual assistants

Techniques for automatically triggering video surveillance using embedded voice, speech, or sound recognition are provided. In one embodiment, a computer system can receive an audio signal captured from an area to be monitored via video surveillance. The computer system can further recognize, via an embedded recognition component, a voice, speech phrase, or environmental sound in the audio signal, and can determine that the recognized voice, speech phrase, or environmental sound corresponds to a predefined trigger condition. The computer system can then automatically transmit a signal to one or more video capture devices to begin video recording of the area.

Triggering video surveillance using embedded voice, speech, or sound recognition

Techniques for unobtrusively verifying the identity of a user of a computing device are provided. In one embodiment, the computing device can establish one or more verification models for verifying the user's identity, where at least a subset of the one or more verification models is based on enrollment data that is collected in an unobtrusive manner from the user. The computing device can then verify the user's identity using the one or more verification models.

Unobtrusive verification of user identity

Techniques for performing low-fidelity always-on A/V monitoring are provided. In one embodiment, an always-on A/V monitoring system can record audio or video footage of an area of interest on a continuous basis while operating in a low-fidelity recording mode, where the recorded audio or video footage has a quality level that is sufficient to detect one or more events that have meaning to the system or a user, but is insufficient to recognize details with respect to the area of interest that would be considered private to an individual appearing in, or associated with, the recorded audio or video footage.

Low-fidelity always-on audio/video monitoring

Techniques for implementing unobtrusive training for speaker verification are provided. In one embodiment, an electronic device can receive a plurality of voice samples uttered by one or more users as they interact with a voice command-and-control feature of the electronic device and, for each voice sample, assign the voice sample to one of a plurality of voice type categories. The electronic device can further group the voice samples assigned to each voice type category into one or more user sets, where each user set comprises voice samples likely to have been uttered by a unique user. The electronic device can then, for each user set: (1) generate a voice model, (2) issue, to the unique user, a request to provide an identity or name, and (3) label the voice model with the identity or name provided by the unique user.

Unobtrusive training for speaker verification

Techniques for analyzing long-term audio recordings are provided. In one embodiment, a computing device can record audio captured from an environment of a user on a long-term basis (e.g., on the order of weeks, months, or years). The computing device can store the recorded audio on a local or remote storage device. The computing device can then analyze the recorded audio based one or more predefined rules and can enable one or more actions based on that analysis.

Analysis of long-term audio recordings

Techniques for leveraging a combination of audio-based and vision-based cues for voice command-and-control are provided. In one embodiment, an electronic device can identify one or more audio-based cues in a received audio signal that pertain to a possible utterance of a predefined trigger phrase, and identify one or more vision-based cues in a received video signal that pertain to a possible utterance of the predefined trigger phrase. The electronic device can further determine a degree of synchronization or correspondence between the one or more audio-based cues and the one or more vision-based cues. The electronic device can then conclude, based on the one or more audio-based cues, the one or more vision-based cues, and the degree of synchronization or correspondence, whether the predefined trigger phrase was actually spoken.

Using combined audio and vision-based cues for voice command-and-control

Techniques for leveraging multiple biometrics for enabling user access to security metadata are provided. In one embodiment, a computing device can receive first and second biometric identifiers from a user. The computing device can further determine, via a multi-biometric authentication system, that the user's identity can be verified using the first biometric identifier, but cannot be, or has not been, verified using the second biometric identifier. In response to this determination, the computing device can provide information to the user for facilitating verification of the user's identity using the second biometric identifier.

Leveraging multiple biometrics for enabling user access to security metadata

In one embodiment the present invention includes a method comprising receiving an acoustic input signal and processing the acoustic input signal with a plurality of acoustic recognition processes configured to recognize the same target sound. Different acoustic recognition processes start processing different segments of the acoustic input signal at different time points in the acoustic input signal. In one embodiment, initial states in the recognition processes may be configured on each time step.

Systems and methods for hands-free voice control and voice search

Techniques for implementing face-based authentication with situational adaptivity are provided. In one embodiment, a computing device can create an enrollment template for a user, the enrollment template being derived from one or more enrollment images of the user's face and being usable by a face-based authentication system to authenticate the user's identity. The computing device can further determine a first set of metadata associated with the enrollment image(s) and can store the first set of metadata with the enrollment template. At a later time (e.g., an authentication event), the computing device can capture an input image of the user's face, determine a second set of metadata associated with the input image, and calculate a computational distance between the input image and the enrollment template, the calculating taking into account a degree of difference between the first and second sets of metadata. Finally, the user can be authenticated based on the distance.

Face-based authentication with situational adaptivity

In one embodiment, a method includes receiving an acoustic input signal at a speech recognizer. A user is identified that is speaking based on the acoustic input signal. The method then determines speaker-specific information previously stored for the user and a set of responses based on the recognized acoustic input signal and the speaker-specific information for the user. It is determined if the response should be output and the response is outputted if it is determined the response should be output.

Background speech recognition assistant using speaker verification

In one embodiment, a method receives an acoustic input signal at a speech recognizer configured to recognize the acoustic input signal in an always on mode. A set of responses based on the recognized acoustic input signal is determined and ranked based on criteria. A computing device determines if the response should be output based on a ranking of the response. The method determines an output method in a plurality of output methods based on the ranking of the response and outputs the response using the output method if it is determined the response should be output.

Growing community of inventors

Los Altos Hills, CA, United States of America

Todd F Mozer