18481803. PROMPT COMPLEXITY FOR LARGE LANGUAGE MODELS (Google LLC)

PROMPT COMPLEXITY FOR LARGE LANGUAGE MODELS

Organization Name

Google LLC

Inventor(s)

Swaroop Mishra of Mountain View CA (US)

Ragha Kotikalapudi of San Jose CA (US)

Obaid Sarvana of Chicago IL (US)

Sahitya Potluri of Sunnyvale CA (US)

YaGuang Li of Sunnyvale CA (US)

Taylor Bos of Santa Clara CA (US)

Steven Zheng of San Bruno CA (US)

Hanzhao Lin of Cupertino CA (US)

Chenkai Kuang of Sunnyvale CA (US)

Heng-Tze Cheng of Mountain View CA (US)

Ed H. Chi of Los Altos CA (US)

Quoc Le of Sunnyvale CA (US)

PROMPT COMPLEXITY FOR LARGE LANGUAGE MODELS

This abstract first appeared for US patent application 18481803 titled 'PROMPT COMPLEXITY FOR LARGE LANGUAGE MODELS

Original Abstract Submitted

Some implementations relate to generating a training and/or evaluation dataset with LLM prompts (e.g., derived from user queries) based on a prompt complexity. An input prompt, for example derived from a user query, is received. The input prompt is decomposed into a prompt tree comprising a plurality of nodes. The plurality of nodes comprise: a plurality of leaf nodes corresponding to simple sub-prompts of the input query; a plurality of branch nodes of sub-prompts each corresponding to multiple simple sub-prompts; and a root node corresponding to the input prompt. A prompt complexity is determined based on a path length of the prompt tree. The prompt complexity is compared to a threshold complexity. If the prompt complexity is above the threshold complexity, the input prompt is included in a set of training prompts and/or a set of evaluation prompts.