Waymo LLC

Google Inc.

Baidu USA LLC

Baidu.com Times Technology (Beijing) Co. Ltd.

Xin Zhou

Roshni Cooper

Yang Li

Michael R James

Gang Li

Le Kang

Hao Tian

Zhiyu Cheng

Xing Li

John Francis Canny

Zifeng Huang

Mingyu Chen

Yingze Bao

Xuan Zhong

Courtney McCool

Mostafa Dehghani

Daming Lu

Xi Chen

Jeff Chienyu Wang

Alexey Alexeevich Gritsenko

Haomin Liu

Bo He

Jingyu Xin

Dapeng Li

Jingya Xun

Courtney Mccool

The technology relates to using on-board sensor data, off-board information and a deep learning model to classify road wetness and/or to perform a regression analysis on road wetness based on a set of input information. Such information includes on-board and/or off-board signals obtained from one or more sources including on-board perception sensors, other on-board modules, external weather measurement, external weather services, etc. The ground truth includes measurements of water film thickness and/or ice coverage on road surfaces. The ground truth, on-board and off-board signals are used to build the model. The constructed model can be deployed in autonomous vehicles for classifying/regressing the road wetness with on-board and/or off-board signals as the input, without referring to the ground truth. The model can be applied in a variety of ways to enhance autonomous vehicle operation, for instance by altering current driving actions, modifying planned routes or trajectories, activating on-board cleaning systems, etc.

Road condition deep learning model

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training and using machine learning models to generate graphical user interfaces from textual descriptions.

Creating user interface using machine learning

A method includes receiving, via a computing device, a screenshot of a display provided by a graphical user interface of the computing device. The method also includes generating, by an image-structure transformer of a neural network, a representation by fusing a first embedding based on the screenshot and a second embedding based on a layout of virtual objects in the screenshot. The method additionally includes predicting, by the neural network and based on the generated representation, a modeling task output associated with the graphical user interface. The method further includes providing, by the computing device, the predicted modeling task output.

Neural networks based multimodal transformer for multi-task user interface modeling

It is advantageous for a vehicle to detect road wetness or related environmental conditions. This is particularly true for self-driving vehicles, which can then adjust the manner of automated operation of the vehicle to increase safety by reducing speed, braking earlier, adjusting internal estimates of road traction parameters, or adjusting autonomous operation in some other manner. It is difficult to directly measure road wetness (e.g., using spectroscopy or other methods directed at the road surface), however, it is possible to indirectly estimate road wetness based on road noise audio signals detected via one or more microphones disposed on the vehicle. The location of the microphones, the type of post-processing applied to the audio signals, or other factors can be adapted to increase the useful road wetness-related content of such audio signals while reducing the presence of engine noise, road noise, or other confounding signals.

Using audio to detect road conditions

Presented herein are systems, methods, and datasets for automatically and precisely generating highlight or summary videos of content. In one or more embodiments, the inputs comprise a text (e.g., an article) of the key event(s) (e.g., a goal, a player action, etc.) in an activity (e.g., a game, a concert, etc.) and a video or videos of the activity. In one or more embodiments, the output is a short video of an event or events in the text, in which the video may include commentary about the highlighted events and/or other audio (e.g., music), which may also be automatically synthesized.

Generating highlight video from video and text inputs

With rapidly evolving technologies and emerging tools, sports-related videos generated online are rapidly increasing. To automate the sports video editing/highlight generation process, a key task is to precisely recognize and locate events-of-interest in videos. Embodiments herein comprise a two-stage paradigm to detect categories of events and when these events happen in videos. In one or more embodiments, multiple action recognition models extract high-level semantic features, and a transformer-based temporal detection module locates target events. These novel approaches achieved state-of-the-art performance in both action spotting and replay grounding. While presented in the context of sports, it shall be noted that the systems and methods herein may be used for videos comprising other content and events.

Transformer-based temporal detection in video

Described herein are systems and methods that improve the success rate of relocalization and eliminate the ambiguity of false relocalization by exploiting motions of the sensor system. In one or more embodiments, during a relocalization process, a snapshot is taken using one or more visual sensors and a single-shot relocalization in a visual map is implemented to establish candidate hypotheses. In one or more embodiments, the sensors move in the environment, with a movement trajectory tracked, to capture visual representations of the environment in one or more new poses. As the visual sensors move, the relocalization system tracks various estimated localization hypotheses and removes false ones until one winning hypothesis. Once the process is finished, the relocalization system outputs a localization result with respect to the visual map.

Systems and methods for robust self-relocalization in a visual map

Presented herein are systems, methods, and datasets for automatically and precisely generating highlight or summary videos of content. For example, in one or more embodiments, videos of sporting events may be digested or condensed into highlights, which will dramatically benefit sports media, broadcasters, video creators or commentators, or other short video creators, in terms of cost reduction, fast, and mass production, and saving tedious engineering hours. Embodiment of the framework may also be used or adapted for use to better promote sports teams, players, and/or games, and produce stories to glorify the spirit of sports or its players. While presented in the context of sports, it shall be noted that the methodologies may be used for videos comprising other content and events.

Automatically and precisely generating highlight videos with artificial intelligence

Growing community of inventors

Mountain View, CA, United States of America

Xin Zhou