Structured Video Documents

Organization Name

Inventor(s)

Francoise Beaufays of Mountain View CA US

Structured Video Documents

This abstract first appeared for US patent application 20250094491 titled 'Structured Video Documents

Original Abstract Submitted

a method includes receiving a content feed that includes audio data corresponding to speech utterances and processing the content feed to generate a semantically-rich, structured document. the structured document includes a transcription of the speech utterances and includes a plurality of words each aligned with a corresponding audio segment of the audio data that indicates a time when the word was recognized in the audio data. during playback of the content feed, the method also includes receiving a query from a user requesting information contained in the content feed and processing, by a large language model, the query and the structured document to generate a response to the query. the response conveys the requested information contained in the content feed. the method also includes providing, for output from a user device associated with the user, the response to the query.