Distributed Worker Pool for Crawling Data Stored in the Cloud

Abstract: systems and methods for operating a scanning system, implemented either on-premises or in a cloud-based service, for crawling and analyzing files stored in one or more data repositories. the scanning system includes a controller, a message broker, and a distributed pool of workers, and, in one embodiment, a method includes receiving, by the controller, policy and configuration data associated with at least one organization; generating, by the controller, job assignments corresponding to files to be analyzed according to the received policy and configuration data; publishing the job assignments to the message broker for parallel distribution among the distributed pool of workers; retrieving and scanning, by at least one worker, the files from the one or more data repositories in accordance with the assigned job; and executing, where required by the policy and configuration data, at least one policy-based action on the files within the data repositories.

Inventor(s): Shankar Vivekanandan, Narinder Paul, Parth Shah, Pratibha Nayak, Sonal Choudhary, Huan Chen

CPC Classification: H04L63/20 ({for managing network security; network security policies in general (filtering policies )})

Search for rejections for patent application number 20250184367