Databricks, inc. (20250005076). Dataflow Graph Processing with Expectations
Dataflow Graph Processing with Expectations
Organization Name
Inventor(s)
Michael Paul Armbrust of Berkeley CA US
Andreas Neumann of Oakland CA US
Mukul Murthy of Berkeley CA US
Jonathan Mio of San Francisco CA US
Dataflow Graph Processing with Expectations
This abstract first appeared for US patent application 20250005076 titled 'Dataflow Graph Processing with Expectations
Original Abstract Submitted
a system for dataflow graph processing comprises a communication interface and a processor. the communication interface is configured receive an indication to generate a dataflow graph, wherein the indication includes a set of queries. the processor is coupled to the communication interface and is configured to: determine dependencies of each query in the set of queries on another query; determine a dag of nodes based at least in part on the dependencies; insert a node in the dag of nodes to generate an updated dag to enforce an expectation; determine a dataflow graph based on the updated dag; and provide the dataflow graph.