18210535. Learning to Personalize Vision-Language Models through Meta-Personalization (Adobe Inc.)

Learning to Personalize Vision-Language Models through Meta-Personalization

Organization Name

Adobe Inc.

Inventor(s)

Simon Jenni of Hägendorf (CH)

Fabian David Caba Heilbron of San Jose CA (US)

Chun-Hsiao Yeh of Emeryville CA (US)

Bryan Russell of San Francisco CA (US)

Josef Sivic of Prague (CZ)

Learning to Personalize Vision-Language Models through Meta-Personalization

This abstract first appeared for US patent application 18210535 titled 'Learning to Personalize Vision-Language Models through Meta-Personalization

Original Abstract Submitted

Techniques for learning to personalize vision-language models through meta-personalization are described. In one embodiment, one or more processing devices lock a pre-trained vision-language model (VLM) during a training phase. The processing devices train the pre-trained VLM to augment a text encoder of the pre-trained VLM with a set of general named video instances to form a meta-personalized VLM, the meta-personalized VLM to include global category features. The processing devices test the meta-personalized VLM to adapt the text encoder with a set of personal named video instances to form a personal VLM, the personal VLM comprising the global category features personalized with a set of personal instance weights to form a personal instance token associated with the user. Other embodiments are described and claimed.