On the Unreasonable Effectiveness of Knowledge Distillation: Analysis in the Kernel Regime

Rahbar, Arman ; Panahi, Ashkan ; Bhattacharyya, Chiranjib ; Dubhashi, Devdatt ; Chehreghani, Morteza Haghir (2020) On the Unreasonable Effectiveness of Knowledge Distillation: Analysis in the Kernel Regime

PDF
408kB

Official URL: https://arxiv.org/abs/2003.13438

Abstract

Error parsing XML in render_xhtml_field: :1: parser error : xmlParseEntityRef: no name
e two layer non-linear networks in model and regime in (Arora et al., 2019; Du &
                                                                                ^
:1: parser error : xmlParseEntityRef: no name
linear networks in model and regime in (Arora et al., 2019; Du & Hu, 2019; Cao &
                                                                                ^
:1: parser error : xmlParseEntityRef: no name
 network. Intriguingly, we also confirm the lottery ticket hypothesis (Frankle &
                                                                                ^

Item Type:	Article
Source:	Copyright of this article belongs to arXiv
Keywords:	Machine Learning (cs.LG), Machine Learning (stat.ML), FOS: Computer and information sciences, FOS: Computer and information sciences
ID Code:	127750
Deposited On:	13 Oct 2022 11:01
Last Modified:	13 Oct 2022 11:01

Repository Staff Only: item control page