Microsoft Technology Licensing, LLC

Intel Corporation

University of Washington

Intel-ge Care Innovations LLC

Matthai Philipose

Paramvir Bahl

Peter Bodik

Nissanka Arachchige Bodhi Priyantha

Ganesh Ananthanarayanan

Stephen Edward Hodges

Alastair Wolman

Sharad Agarwal

Joshua R Smith

John Helmes

Haichen Shen

Tsuwang Hsieh

William Noah Schilit

Ivan Jelev Tashev

Uttam K Sengupta

Yun-Cheng Ju

Lin Zhong

Anthony George LaMarca

Craig D Chambers

Beth Teresa Logan

Lenin Ravindranath Sivalingam

Donald Frank Brinkman, Jr

Susan Jane Eggers

Shuayb M Zarar

Jaeyeon Jung

Max Artemov

Haoyu Zhang

Chien-Chun Hung

Robert LiKamWa

Seungyeop Han

Tanzeem K Choudhury

Jong Hwan Ko

Markus Mock

Suvamsh Shivaprasad

Brian K Grant

Donald Frank Brinkman

Robert Likamwa

Systems, methods, and computer-readable storage devices are disclosed for improved real-time audio processing. One method including: constructing a deep neural network model, including a plurality of at least one-bit neurons, configured to output a predicted label of audio data, the plurality of at least one-bit neurons arranged in a plurality of layers, including at least one hidden layer, and being connected by a plurality of connections, each connection having at least a one-bit weight, wherein one or both of the plurality of at least one-bit neurons and the plurality of connections have a reduced bit precision; receiving a training data set, the training data set including audio data; training the deep neural network model using the training data set; and outputting a trained deep neural network model configured to output a predicted label of real-time audio data.

Systems, methods, and computer-readable media for improved real-time audio processing

A method can include classifying, using a compressed and specialized convolutional neural network (CNN), an object of a video frame into classes, clustering the object based on a distance of a feature vector of the object to a feature vector of a centroid object of the cluster, storing top-k classes, a centroid identification, and a cluster identification, in response to receiving a query for objects of class X from a specific video stream, retrieving image data for each centroid of each cluster that includes the class X as one of the top-k classes, classifying, using a ground truth CNN (GT-CNN), the retrieved image data for each centroid, and for each centroid determined to be classified as a member of the class X providing image data for each object in each cluster associated with the centroid.

Querying video data with reduced latency and cost

Described herein is a mechanism for creating a dynamic video highlight from a plurality of video segments. A metadata collection agent collects metadata comprising attributes about a video, segments within the video where one or more events occur, attributes about the creator of the video and so forth. The metadata is collected and used to create highlight video definitions comprising a set of metadata attribute-value pairs. The highlight video definitions can be created in an interactive fashion by presenting a user interface allowing selection of a combination of attribute-value pairs to include/exclude segments from the definition and/or manual selection of custom segments. The highlight video definitions can be stored and/or shared among users. The video highlight definitions are utilized to instantiate one or more video players to play the video segments in an identified order without assembling the video segments into a separate video.

Dynamic video highlight

An imaging sensor system includes a control system that helps save energy, computing, and communication resources by capturing evaluation images with a gating image sensor and processing the evaluation images to determine whether there is a region of interest (ROI) present that warrants capturing a subsequent image with a gated image sensor for additional image processing. That is, the gating image sensor and preliminary processing operates as a gate for turning on the gated sensor for the additional image capturing and processing. The additional image captured by the gated sensor may include only the ROI, which is provided to an image processing algorithm for further image processing. In addition, a gating sensor may include infrared sensing to help determine whether the ROI includes thermal data indicative of human features.

Multimodal image sensing for region of interest capture

A method to record video with a video camera while respecting bystander privacy includes acquiring sensory data separate from the video, parsing the sensory data for evidence of a human being in a field of view of the video camera, and recording video with the video camera if no human being is detected in the field of view, based upon the sensory data.

Video capture with privacy safeguard

A classification system classifies different aspects of content of an input image stream, such as faces, landmarks, events, and so forth. The classification system includes a general classifier and at least one specialized classifier template. The general classifier is trained to classify a large number of different aspects of content, and a specialized classifier can be trained based on a specialized classifier template during operation of the classification system to classify a particular subset of the multiple different aspects of content. The classification system determines when to use the general classifier and when to use a specialized classifier based on class skew, which refers to the temporal locality of a subset of aspects of content in the image stream.

Dynamic classifier selection based on class skew

Wearable audio accessories for computing devices are described. In one embodiment the wearable audio accessory provides a speech based interface between the user and a nearby computing device for the performance of user-initiated or computing device initiated microtasks. Information is provided to the user via a loudspeaker and the user can provide input via a microphone. An audio sensing channel within the accessory continuously monitors the audio signal as detected by the microphone and in various embodiments will trigger more complex audio processing based on this monitoring. A wireless communication link is provided between the accessory and the nearby computing device. To mitigate any delay caused by the switching between audio processing techniques, the audio accessory may include a rolling buffer which continuously stores the audio signal and outputs a delayed audio signal to the audio processing engines.

Wearable audio accessories for computing devices

Various technologies described herein pertain to performing video analytics. The approaches set forth herein support live video analytics at scale with approximate and delay-tolerant processing. Video streams can be captured by multiple cameras and continuously streamed to a video analytics computing system; the video streams can be received at the video analytics computing system. Multiple video analytics queries can be executed on the video streams. The multiple video analytics queries can be concurrently executed by the video analytics computing system on the video streams as the video streams are continuously streamed to the video analytics computing system. The multiple video analytics queries can be executed utilizing resources of the video analytics computing system allocated between the multiple video analytics queries. Execution of the multiple video analytics queries can return respective results for the multiple video analytics queries. The results for the multiple video analytics queries can be outputted.

Live video analytics at scale

A global manager communicates with various local managers to receive and process video queries. The video queries identify components that process live video streams, placement options for where the components of the video query may be executed, and various video query plans. The video query plans include options such as framerate and video quality. As the global manager processes the video queries, the global manager determines an initial set of video query configurations that identify a video query plan and placement option for each component of a given video query. Using the initial set of video query configurations, the global manager then determines an optimal set of video query configurations for the received set of video queries. The global manager communications instructions to the local managers to execute the components of the video queries using the video query plans and placement options from the optimal set of video query configurations.

Processing live video streams over hierarchical clusters

The subject disclosure is directed towards energy saving mechanisms of image sensor circuitry (e.g., in a camera). Image quality data, such as provided by an application, is processed to make energy consumption of image sensor circuitry more proportional to output image quality by controlling the operation of one or more controllable power saving mechanisms of the image sensor circuitry. Power saving mechanisms may include a frequency controlled clock, the ability to turn off unneeded components, an inter-frame standby mode that puts the image sensor circuitry into a standby mode between capturing sequential frames, selectable parallel analog chains having different energy usage properties and column circuitry that allows turning off circuitry corresponding to unneeded columns of the sensor array.

Energy-proportional image sensor

A development system is described for facilitating the development of a spoken natural language (SNL) interface. The development system receives seed templates from a developer, each of which provides a command phrasing that can be used to invoke a function, when spoken by an end user. The development system then uses one or more development resources, such as a crowdsourcing system and a paraphrasing system, to provide additional templates. This yields an extended set of templates. A generation system then generates one or more models based on the extended set of templates. A user device may install the model(s) for use in interpreting commands spoken by an end user. When the user device recognizes a command, it may automatically invoke a function associated with that command. Overall, the development system provides an easy-to-use tool for producing an SNL interface.

Facilitating development of a spoken natural language interface

Low frequency harmonics detected by an accelerometer in a wrist-mounted device are used to infer ambulation. The fundamental rhythm or pitch is extracted from gathered accelerometer data and the determined pitch and associated properties are used to infer whether a subject is walking or not walking.

Method and apparatus to detect ambulation

Growing community of inventors

Seattle, WA, United States of America

Matthai Philipose