20250165801. Systems Methods (Datum Point Labs .)

SYSTEMS AND METHODS FOR FEATURE DROPOUT KNOWLEDGE DISTILLATION

Abstract: embodiments described herein provide systems and methods for knowledge distillation. a system encodes features from a student model and a teacher model to provide principal components. the principal components may be decoded to provide decoded components. logits may also be output by the student and teacher models. a loss function may be computed based on the principal components, decoded components, and logits. parameters of the student model may be updated based on the loss function.

Inventor(s): Bokyeung Lee, Hanseok Ko, Bonhwa Ku

CPC Classification: G06N3/096 (Transfer learning)

Search for rejections for patent application number 20250165801