Gm Global Technolgoy Operations, Inc.

Roy Uziel

Dan Levi

Oded Bialer

Noa Garnett

Herein, a technology that facilitates the optimization of vision-language (VL) based classifiers with text embeddings is discussed. The technology includes tuning the VL-based classifier employing a pre-trained image encoder of a visual-language model (VLM) for imaging embedding of pre-classified images and a pre-trained textual encoder of the VLM for textual embedding of a set of differing textual sentences. The technology further includes determining an optimized set of differing textual sentences of a superset of textual sentences. The optimized set of differing textual sentences has a minimal classification loss of the VL-based classifier when classifying the pre-classified images.

Classification by vision-language model with optimized text embeddings

A system for classifying a road crossing intention of a pedestrian includes a processor including a pretrained image encoder generating an image embedding based upon an input image. The system further includes a remote server receiving the image embedding. The remote server device further references a plurality of pretrained image and text embeddings each corresponding to either a positive road crossing intention or a negative road crossing intention. The remote server device further determines a plurality of proximity values evaluating whether the input image is closer to the positive road crossing intention or the negative road crossing intention, evaluating the image embedding against each of the pretrained embeddings. The remote server device further classifies a road crossing intention of the pedestrian based upon the plurality of proximity values. The system further includes generates a road crossing intention output based upon the road crossing intention of the pedestrian.

System and method for pedestrian road crossing intention detection

A system for analyzing images includes a processing device includes a receiving module configured to receive an image, and an analysis module configured to apply the received image to a machine learning network and classify one or more features in the received image, the machine learning network configured to propagate image data through a plurality of convolutional layers, each convolutional layer of the plurality of convolutional layers including a plurality of filter channels, the machine learning network including a bottleneck layer configured to recognize an image feature based on a shape of an image component, The system also includes an output module configured to output characterization data that includes a classification of the one or more features.

Growing community of inventors

Herzliya, Israel

Roy Uziel