Qualcomm incorporated (20240412075). KERNEL-GUIDED ARCHITECTURE SEARCH AND KNOWLEDGE DISTILLATION

KERNEL-GUIDED ARCHITECTURE SEARCH AND KNOWLEDGE DISTILLATION

Organization Name

qualcomm incorporated

Inventor(s)

Haijun Zhao of Beijing (CN)

KERNEL-GUIDED ARCHITECTURE SEARCH AND KNOWLEDGE DISTILLATION

This abstract first appeared for US patent application 20240412075 titled 'KERNEL-GUIDED ARCHITECTURE SEARCH AND KNOWLEDGE DISTILLATION

Original Abstract Submitted

a method for compressing of a deep neural network model includes determining an architecture of a teacher model. an initial layer of the teacher model is preserved and included in a student model. repeated layers of the teacher model having a same type are identified. the second of such layers is removed such that the student model includes fewer layers than the teacher model. knowledge distillation is applied to train the student model. the identifying and removal of layers of the same type is repeated to compress the student model.