Shutterstock, Inc.

Samsara Inc.

Nathan Hurst

Yan Wang

Kevin Lai

Cole Jurden

Akshay Raj Dhamija

Kevin Scott Lester

Margaret Irene Finch

Manor Lev-Tov

David Chester

Shirish Nair

Rohit Annigeri

Rachel Demerly

Heath Hohwald

Sharan Srinivasan

Brian Westphal

Vaibhav Malpani

Jose Cazarin

Sung Chun Lee

Jing Wang

Michael Steven Ranzinger

Alexander Chavez

Kenshiro Nakagawa

Maisie Wang

Abner Ayala

Douglas Boyle

Evan Welbourne

Howard Yu

Shaurye Agarwal

Justin Levine

Olamide Akintewe

Jovanna Bubar

Aravindh Ramesh

John Charles Bicket

Sven Eberhardt

Narendran Rajan

Lucas Doyle

Suryakant Kaushik

Saurabh Tripathi

Ivan Stoev

Mike Ranzinger

Jason Liu

Pang Wu

Phil Ammirato

Allen Lu

Robert Koenig

Marc Clifford

Shiva Bala

Shaurye Aggarwal

Maya Krupa

Harry Lu

Sanjit Biswas

Yangyong Zhang

An Applicant Assistant System may receive, from a user device, a user input indicating a query or task associated with information displayed in a user interface. The device may determine one or more components associated with the user interface and access component metadata of the determined one or more components, the component metadata for each of the one or more components indicating one or more of a fact, a data link, or an action. A prompt comprising at least some of the component metadata, at least some context information, and an indication of one or more available response elements may be provided to a large language model (LLM). The LLM may response with one or more response elements that may be processed by the System, such as to cause updates to the user interface on the user device.

Artificial intelligence application assistant

Techniques are presented for detecting when drivers drive while drowsy. In some implementations, a drowsiness model is trained with data associated with inward videos and outward videos captured during a trip. The inward videos capture the inside of the cabin with the driver, and the outward videos capture the view in front of the vehicle in the direction of travel. Further, a device at the vehicle periodically calculates a drowsiness scale index value that indicates the level of drowsiness of the driver. Calculating the drowsiness scale index value includes obtaining a set of inward frames from the inward videos; for each inward frame, creating a face image by cropping the inward frame; obtaining a set of outward frames from the outward videos; calculating inward embeddings of the face images and outward embeddings of the outward frames; and calculating, by the drowsiness model, the drowsiness scale index value.

Drowsy driving detection

Techniques are presented for the detection and management of collision warning (CW) events. A training dataset comprising videos of vehicle collisions and non-collisions, sensor readings, environmental conditions, and more is utilized to train a CW classification model for detecting potential collision events in vehicles. A backend CW classification model, with greater computational resources, employs a more complex neural network to review CW events received by the Behavioral Monitoring System (BMS) based on video data, achieving higher precision and reducing false positives. The CW model is installed in vehicles for real-time detection, while the backend model is deployed at the BMS. The BMS validates detected CW events, filters out false positives, and streamlines the review process for fleet administrators and customers. Additional BMS filtering operations include assessing non-proximity-related CW events and camera impairments, with the filtered CW events presented for review in the safety inbox.

Forward collision warning

Methods, systems, and computer programs are presented for monitoring tailgating when a vehicle follows another vehicle at an unsafe distance. A method for enhancing a Following Distance (FD) machine learning (ML) model is disclosed. The method includes providing a management user interface (UI) for configuring FD parameters, followed by receiving FD events. A UI for manual FD annotation and another for customer review of filtered FD events are also provided. Annotations and customer review information are collected to improve the training set for the FD ML model. The FD model is then trained with the new data and downloaded to a vehicle. Once installed, the FD model is utilized to detect FD events within the vehicle, thereby enhancing the vehicle's safety and performance in driving scenarios by improving the accuracy and reliability of FD event predictions or detections.

Monitoring the safe distance between vehicles while driving

A computer-implemented method for machine learning model operation can include, by one or more processors executing program instructions: providing a first training dataset comprising a plurality of images and associated object detection labels, providing a second training dataset comprising a plurality of images and associated classification labels, and providing a machine learning model comprising a model backbone, an object detection task head, and a classification task head. The method can further include training the machine learning model by training the object detection task head using the first training dataset and training the classification task head using the second training dataset. The method can further include deploying the trained machine learning model that includes the trained model backbone and the trained object detection task head but does not include the trained classification task head.

Multi-task machine learning model for event detection

Methods, systems, and programs are presented for detecting impaired views in monitoring cameras. One method includes training a rotation classifier with unsupervised learning utilizing a first set of images. The rotation classifier is configured to receive an input image and generate a rotation feature embedding for the input image. In addition, the method includes training an impairment classifier with supervised learning utilizing a second set of images, impairment labels for each of the second set of images, and the rotation feature embedding, generated by the rotation classifier, for each of the second set of images. The method further includes accessing a vehicle image captured by a camera on a vehicle, and providing the vehicle image to the impairment classifier as input, and the impairment classifier outputs a camera impairment from a set of camera impairment categories. Further, the vehicle image and the camera impairment are presented on a user interface.

Detection of camera with impaired view

Methods, systems, and computer programs are presented for the management of lane-departure (LD) events. One method includes training a classifier for LD events and loading the classifier into a vehicle. LD events are detected based on outward images using the classifier, while the turn signal is monitored to prevent false triggers. If an LD event is detected, rules are checked for alerting the driver and deciding whether to alert the driver or not. Subsequently, additional rules are checked for reporting the event and deciding whether to report the event to a Behavior Monitoring System (BMS) or to discard it. The method also includes a solid line departure model that identifies crossing dashed, solid-white, and solid-yellow lanes, delaying alerts and event generation until a significant portion of the vehicle crosses over the lane. The model also outputs a confidence score reflecting the amount of vehicle deviation from the driving lane.

Lane departure monitoring

A method for receiving a first user query from a user for searching an item, forming a first filter based on the first user query, and forming a first filtered item collection is provided. The method includes predicting a new query based on the first user query and a historical query log, forming a second filter for the new query, and applying the second filter to the first filtered item collection to form a second filtered item collection. Further, associating an item score to each of a plurality of items in the first and second filtered item collections, sorting the plurality of items in the first and second filtered item collections according to the item score associated to each of the plurality of items, and providing, to a user display, an item in the plurality of items in the first or second filtered item collections according to a sorting order.

Image search using intersected predicted queries

Methods for predicting network access probability of data files accessible over a computer network are provided. In one aspect, a method includes generating a primary data vector for a media file based on a stored data representation of the file, and providing the data vector for the file to an algorithm that uses past interaction information for at least one other media file from a collection of media files having a degree of similarity with the media file above a threshold similarity value. The method also includes receiving, as an output of the algorithm, a marketability score for the media file, the score indicative of a likelihood that a user will download the media file. Systems and machine-readable media are also provided.

Architecture for predicting network access probability of data files accessible over a computer network

Various aspects of the subject technology relate to systems, methods, and machine-readable media for user drawing based image search. These aspects include an image retrieval system using a convolutional neural network trained to identify how users draw semantic concepts and using an image search engine to search against images having a similar concept. The aspects include mapping between concepts of the user drawing space and concepts of the image space such that images associated with the same concept are identified. For each input user drawing, the drawing is first processed through a concept classifier to identify a corresponding concept, and then through a feature extractor to form a corresponding feature vector. The results from the concept classifier and the feature extractor may be combined to search against a collection of images having a similar concept to determine a listing of images ranked by visual and semantic similarity to the input.

User drawing based image search

Various aspects of the subject technology relate to systems, methods, and machine-readable media for language translation based on image search similarities. These aspects include an image retrieval system using a convolutional neural network that is trained to identify a correlation between an image and a language term, and using an image search engine to search against images corresponding to visual words that are responsive to a given search query in a given spoken language. These aspects include access to interaction probability data that identifies user interaction probabilities for the visual words to determine a correlation between the input language terms of the search query and the rate at which users interact with images of a corresponding visual word that is responsive to the search query. The system then provides a prioritized listing of images that is responsive to the given search query based on the identified user interaction probabilities.

Language translation based on search results and user interaction data

A computer-implemented method is provided for retrieving an image from a user in a desired format and for detecting a compression efficiency for the image. When the compression efficiency is above a pre-selected threshold the computer-implemented method includes obtaining a saliency representation of the image, capturing a feature description of a non-salient portion of the image, flattening the non-salient portion in a new image, storing the new image in a selected format in a memory and storing a background descriptor for the image in the memory.

Image file compression using dummy data for non-salient portions of images

Growing community of inventors

Seattle, WA, United States of America

Nathan Hurst