18817147. Debiasing Pre-trained Sentence Encoders With Probabilistic Dropouts (Oracle International Corporation)

Debiasing Pre-trained Sentence Encoders With Probabilistic Dropouts

Organization Name

Oracle International Corporation

Inventor(s)

Swetasudha Panda of Burlington MA (US)

Ariel Kobren of Cambridge MA (US)

Michael Louis Wick of Lexington MA (US)

Stephen Green of Burlington MA (US)

Debiasing Pre-trained Sentence Encoders With Probabilistic Dropouts

This abstract first appeared for US patent application 18817147 titled 'Debiasing Pre-trained Sentence Encoders With Probabilistic Dropouts

Original Abstract Submitted

Debiasing pre-trained sentence encoders with probabilistic dropouts may be performed by various systems, services, or applications. A sentence may be received, where the words of the sentence may be provided as tokens to an encoder of a machine learning model. A token-wise correlation using semantic orientation may be determined to determine a bias score for the tokens in the input sentence. A probability of dropout that for tokens in the input sentence may be determined from the bias scores. The machine learning model may be trained or tuned based on the probabilities of dropout for the tokens in the input sentence.