SELECTING OPTIMAL HARDWARE CONFIGURATIONS

Abstract: a data processing service builds a container for a customer to run a trained large language model (llm). the data processing service receives a trained llm and a desired configuration from a user of a client device. based on the desired configuration, the data processing service selects a hardware configuration and structures weights of the trained llm based on the hardware configuration. the data processing service generates a container image reflecting the hardware configuration, registers the container image to a container registry, and generates a container from the container image as well as an application programming interface (api) endpoint for the container. the data processing service deploys the trained llm in the api endpoint using the container such that the trained llm is accessible through api calls.

Inventor(s): Ahmed Bilal, Steven Yikun Chen, Bruce Laurent Fontaine, Daya Shanker Khudia, Chenran Li, Ankit Mathur

CPC Classification: G06F9/44505 ({Configuring for program initiating, e.g. using registry, configuration files})

Search for rejections for patent application number 20250181358