ZERO-SHOT DOMAIN TRANSFER WITH A TEXT-TO-TEXT MODEL

Organization Name

microsoft technology licensing, llc

Inventor(s)

Stephanie Hyland of Harston GB

Aditya Nori of Cambridge GB

Fangyu Liu of Cambridge GB

Fernando Perez Garcia of London GB

Qianchu Liu of Cambridge GB

Hoifung Poon of Bellevue WA US

Javier Alvarez-valle of Cambridge GB

Naoto Usuyama of Sammamish WA US

Ozan Oktay of London GB

Sheng Zhang of Issaquah WA US

Shruthi Jaisimha Bannur of Cambridge GB

Tristan Josef Naumann of Seattle WA US

ZERO-SHOT DOMAIN TRANSFER WITH A TEXT-TO-TEXT MODEL

This abstract first appeared for US patent application 20240256796 titled 'ZERO-SHOT DOMAIN TRANSFER WITH A TEXT-TO-TEXT MODEL

Original Abstract Submitted

example solutions for zero-shot domain transfer with a text-to-text model train a text-to-text model for a target domain using unlabeled in-domain text training data, and concurrently train the model using labeled general-domain task training data. the in-domain training comprises masked language modeling (mlm) training, and the task training comprises both natural language generation (nlg) training and natural language understanding (nlu) training. the nlg training comprises natural language inference (nli) training and the nlu training comprises summarization training. the trained model acquires domain-specific task competency, sufficient to perform a language task within the target domain. suitable target domains include radiology, biomedical, and other medical, legal, and scientific domains. this approach leverages large volumes of general-domain task training data and plentiful unlabeled in-domain text, even as labeled in-domain training data may be unavailable or prohibitively expensive for certain specialized domains.