Baidu USA LLC

Baidu Online Network Technology (beijing) Co., Ltd.

Baidu.com Times Technology (beijing) Co. Ltd.

Beijing Baidu Netcom Science and Technology Co., Ltd.

Baidu Online Network Technology Co., Ltd.

Le Kang

Yingze Bao

Mingyu Chen

Zeyu Liu

Jian Wang

Hao Sun

Errui Ding

Shilei Wen

Xubin Li

Chengyue Zhang

Xiao Xi Liu

Zhizhen Chi

Hao Tian

Xin Zhou

Zhiyu Cheng

Xing Li

Hongwu Zhang

Tao Liu

Song Yang

Wei Qi

Zhigang Wang

Yupeng Li

Xi Chen

Daming Lu

Jeff Chienyu Wang

Haofeng Kou

Kuipeng Wang

Yuxuan Luo

Qingze Wang

Xuejun Wang

Jingyu Xin

Dapeng Li

Jingya Xun

Hongtian Yang

Jingxi Lu

Bo He

Xiao Liu

Presented herein are systems, methods, and datasets for automatically and precisely generating highlight or summary videos of content. In one or more embodiments, the inputs comprise a text (e.g., an article) of the key event(s) (e.g., a goal, a player action, etc.) in an activity (e.g., a game, a concert, etc.) and a video or videos of the activity. In one or more embodiments, the output is a short video of an event or events in the text, in which the video may include commentary about the highlighted events and/or other audio (e.g., music), which may also be automatically synthesized.

Generating highlight video from video and text inputs

With rapidly evolving technologies and emerging tools, sports-related videos generated online are rapidly increasing. To automate the sports video editing/highlight generation process, a key task is to precisely recognize and locate events-of-interest in videos. Embodiments herein comprise a two-stage paradigm to detect categories of events and when these events happen in videos. In one or more embodiments, multiple action recognition models extract high-level semantic features, and a transformer-based temporal detection module locates target events. These novel approaches achieved state-of-the-art performance in both action spotting and replay grounding. While presented in the context of sports, it shall be noted that the systems and methods herein may be used for videos comprising other content and events.

Transformer-based temporal detection in video

Presented herein are systems, methods, and datasets for automatically and precisely generating highlight or summary videos of content. For example, in one or more embodiments, videos of sporting events may be digested or condensed into highlights, which will dramatically benefit sports media, broadcasters, video creators or commentators, or other short video creators, in terms of cost reduction, fast, and mass production, and saving tedious engineering hours. Embodiment of the framework may also be used or adapted for use to better promote sports teams, players, and/or games, and produce stories to glorify the spirit of sports or its players. While presented in the context of sports, it shall be noted that the methodologies may be used for videos comprising other content and events.

Automatically and precisely generating highlight videos with artificial intelligence

Described herein are systems and methods for object detection to achieve hard real-time performance with low latency. Real-time object detection frameworks are disclosed. In one or more embodiments, a framework comprises a first CPU core, a second CPU core, and a plurality of shaves. In one or more embodiments, the first CPU core handles general CPU tasks, while the second CPU core handles the image frames from a camera sensor and computation task scheduling. In one or more embodiments, the scheduled computation tasks are implemented by the plurality of shaves using at least one object-detection model to detect an object in an image frame. In one or more embodiments, computation results from the object-detection model with a higher detection probability is used to form an output for object detection. In one or more embodiments, the object-detection models share some parameters for smaller size and higher implementing speed.

Systems and methods for low-power, real-time object detection

Embodiments of the present disclosure provide a method and apparatus for identifying an item. The method includes: acquiring an item image of a to-be-identified item; setting initial position coordinates of the to-be-identified item on the item image; and executing following identifying: inputting the item image and the initial position coordinates into a pre-trained attention module to output an item feature of the to-be-identified item; inputting the item feature into a pre-trained long short-term memory network to output a predicted category and predicted position coordinates of the to-be-identified item; determining whether a preset condition is satisfied; and determining, in response to the preset condition being satisfied, a predicted category of the to-be-identified item outputted by the long short-term memory network a last time for use as a final category of the to-be-identified item.

Method and apparatus for identifying item

The disclosure provides human-tracking methods, systems, and storage media. The method includes: acquiring a plurality of human point clouds of a current frame from a plurality of cameras; generating a total point cloud of the current frame by integrating the plurality of human point clouds of the current frame; acquiring a plurality of human point clouds of a next frame from the plurality of cameras; acquiring a total point cloud of the next frame by integrating the plurality of human point clouds of the next frame; and performing human tracking based on the total point cloud of the current frame and the total point cloud of the next frame.

Human-tracking methods, systems, and storage media

The present disclosure provides a method and a device for recognizing a product, an electronic device and a non-transitory computer readable storage medium, relating to a field of unmanned retail product recognition. The method includes the following. A video taken by each camera in a store is acquired. A recognition is performed on each video to obtain a video segment that a product delivery is recognized and to obtain participated users. The participated users include a delivery initiation user and a delivery reception user. The video segment is inputted into a preset delivery recognition model to obtain a recognition result. The recognition result includes a product delivered and a delivery probability. The product information of products carried by the participated users is updated based on the recognition result.

Method and device for recognizing product

The present disclosure provides a human body recognition method and apparatus, and a storage medium, the method comprising: determining a coordinate of a target person in a three-dimensional space according to images containing the target person collected by at least two cameras; calculating back-projection errors of the target person under different cameras respectively according to the coordinate of the target person in the three-dimensional space; determining whether the cameras have a human body recognition error according to the back-projection errors of the cameras; when a camera has the human body recognition error, performing re-recognition of the target person under the camera by using person re-identification ReID, until the back-projection errors of all the cameras containing the target person are not greater than a preset threshold. The present disclosure can improve accuracy of the human body recognition result effectively.

Human body recognition method and apparatus, and storage medium

A human body tracking method, apparatus, and device, and a storage medium. The method includes: obtaining a current frame image captured by a target photographing device at a current moment; detecting each human body in the current frame image to obtain first position information of the each human body in the current frame image; calculating second position information of a first human body in the current frame image; determining target position information of the each human body in the current frame image according to the second position information of the first human body in the current frame image, the first position information of the each human body in the current frame image, and pedestrian features of all tracked pedestrians stored in a preset list.

Human body tracing method, apparatus and device, and storage medium

Embodiments of the present disclosure provide a method and apparatus for generating information, a device for human-computer interaction, and a computer readable medium. The method may include: acquiring gravity sensing data of a shelf carrying an item; and identifying, in response to determining that the item on the shelf is taken based on the gravity sensing data, the taken item based on the gravity sensing data and an acquired image of the taken item, and generating order information of the taken item.

Method and apparatus for generating information, and device for human-computer interaction

A method, apparatus, device, and storage medium for predicting the number of people of a dense crowd, including: converting a first image, in which the number of people is to be determined, into a corresponding first thermodynamic chart according to a thermodynamic chart conversion model; and determining the number of people in the first image according to the first thermodynamic chart, wherein the thermodynamic chart conversion model is obtained by training according to a pre-marked second image and a thermodynamic chart corresponding to each second image, thereby achieving prediction of the number of people of a dense crowd, improving the accuracy in predicting the number of people of the dense crowd while improving management efficiency.

Method, apparatus, device, and storage medium for predicting the number of people of dense crowd

A method, apparatus and a system for human body tracking processing, where an apparatus for video collection processing in the system has a built-in intelligent chip, and before uploading video data to a cloud server, the intelligent chip performs a pre-processing on the video data, retains a key image frame and performs a human body detection and a tracking processing on the key image frame by using human body detection tracking algorithm to acquire a first human body detection tracking result. Afterwards, the intelligent chip sends the first human body detection tracking result to the cloud server, so that the cloud server performs a human body re-identification algorithm processing and/or three-dimensional reconstruction algorithm processing on the first human body detection tracking result to acquire a second human body detection tracking result.

Method, apparatus and system for human body tracking processing

Described herein are systems and methods that allow for dense depth map estimation given input images. In one or more embodiments, a neural network model was developed that significantly differs from prior approaches. Embodiments of the deep neural network model comprises more computationally efficient structures and fewer layers but still produces good quality results. Also, in one or more embodiments, the deep neural network model may be specially configured and trained to operate using a hardware accelerator component or components that can speed computation and produce good results, even if lower precision bit representations are used during computation at the hardware accelerator component.

Stereo camera depth determination using hardware accelerator

Embodiments of the present disclosure disclose a method and an apparatus for determining a target object and a human-computer interaction system. The method according to one embodiment of the present disclosure comprises: in response to detecting a position change of an item, determining a to-be-detected image frame sequence based on a detection moment when the position change is detected; performing a human body key point detection to a to-be-detected image frame in the to-be-detected image frame sequence; and determining a target object which performs a target operation action to the item based on a detection result of the human body key point detection. This embodiment improves accuracy of the determined target object.

Method and apparatus for determining a target object, and human-computer interaction system

Embodiments of the present application provide an image-based position detection method, an apparatus, a device and a storage medium. Images captured at the same time by a plurality of photographing devices mounted in different orientations are acquired, where the plurality of photographing devices are synchronous in time; a two-dimensional position of a target object in each of the images is detected; and a three-dimensional position of the target object in an actual space is determined based on the two-dimensional position of the target object in each of the images and internal parameters and external parameters of the plurality of photographing devices. The embodiments of the present application implement a three-dimensional positioning solution based on a plurality of cameras, thereby improving the reliability and accuracy of object positioning.

Growing community of inventors

Sunnyvale, CA, United States of America

Le Kang