Contextual querying of content rendering activity

Abstract

In an example aspect, the present disclosure provides for an example method for processing queries over content rendering activity. The example method includes receiving, by a computing system comprising one or more processors, a first input signal of a first modality, the first input signal being obtained using one or more sensors of a client device and providing local context signals associated with a content rendering event on an output device. The example method includes receiving, by the computing system, a second input signal of a second modality different from the first modality. The example method includes generating, by the computing system and based on the first input signal and the second input signal, a content query. The example method includes retrieving, by the computing system and based on the content query, a content item associated with the content rendering event.