Databricks, Inc. patent applications on June 20th, 2024
Patent Applications by Databricks, Inc. on June 20th, 2024
Databricks, Inc.: 1 patent applications
Databricks, Inc. has applied for patents in the areas of G06F16/27 (1), G06F16/2455 (1) G06F16/278 (1)
With keywords such as: tasks, stateful, rebalancing, cluster, spread, performed, executors, processing, across, and such in patent application abstracts.
Patent Applications by Databricks, Inc.
20240202211.STATE REBALANCING IN STRUCTURED STREAMING_simplified_abstract_(databricks, inc.)
Inventor(s): Alexander Balikov of Kirkland WA (US) for databricks, inc., Tathagata Das of New Haven CT (US) for databricks, inc., Karthikeyan Ramasamy of Mountain View CA (US) for databricks, inc.
IPC Code(s): G06F16/27, G06F16/2455
CPC Code(s): G06F16/278
Abstract: a data processing service performs a rebalancing process for rebalancing stateful tasks on a cluster computing system. in one instance, the method for rebalancing stateful tasks is performed such that the per-operator partitions are spread across available executors of a cluster of the cluster computing system with respect to one or more statistics of the tasks. in one instance, the method for rebalancing stateful tasks is also performed such that the total number of stateful tasks are balanced per executor as long as this rebalancing does not imbalance the per-operator placements. in this way, the processing of stateful tasks can be spread across multiple executors in a relatively uniform manner, even though there may be an upfront cost of breaking the local caching on an executor.
