repository
Network Group-based Knowledge Distillation using Online Role Change
| DC Field | Value | Language |
|---|---|---|
| dc.contributor.advisor | 황원준 | - |
| dc.contributor.author | 최준용 | - |
| dc.date.accessioned | 2022-11-29T03:01:14Z | - |
| dc.date.available | 2022-11-29T03:01:14Z | - |
| dc.date.issued | 2022-08 | - |
| dc.identifier.other | 32092 | - |
| dc.identifier.uri | https://dspace.ajou.ac.kr/handle/2018.oak/20919 | - |
| dc.description | 학위논문(석사)--아주대학교 일반대학원 :인공지능학과,2022. 8 | - |
| dc.description.tableofcontents | Ⅰ. Introduction 1 Ⅱ. Related Works 5 Ⅲ. Proposed Method 8 1. Background 8 2. Group-based Knowledge Distillation using Online Role change 9 3. Intensive teaching 12 4. Private teaching 13 5. Group teaching 14 Ⅳ. Experimental Results and Discussion 14 1. Datasets 15 2. Networks 15 3. Implementation Details 15 4. Ablation Study: The number of temporary teachers 16 5. Ablation Study: Type of augmentation for intensive teaching 17 6. Ablation Study: Style of group teaching 18 7. Ablation Study: Network perspective of equal size 19 8. Ablation Study: Performance contribution by individual components 20 9. Ablation Study: t-SNE visualization 21 10. Ablation Study: Frequency of temporary teacher 21 11. Comparison with The State-of-the-art Methods 22 Ⅴ. Conclusion 26 Reference 27 | - |
| dc.language.iso | eng | - |
| dc.publisher | The Graduate School, Ajou University | - |
| dc.rights | 아주대학교 논문은 저작권에 의해 보호받습니다. | - |
| dc.title | Network Group-based Knowledge Distillation using Online Role Change | - |
| dc.type | Thesis | - |
| dc.contributor.affiliation | 아주대학교 일반대학원 | - |
| dc.contributor.alternativeName | Junyong Choi | - |
| dc.contributor.department | 일반대학원 인공지능학과 | - |
| dc.date.awarded | 2022. 8 | - |
| dc.description.degree | Master | - |
| dc.identifier.localId | 1254237 | - |
| dc.identifier.uci | I804:41038-000000032092 | - |
| dc.identifier.url | https://dcoll.ajou.ac.kr/dcollection/common/orgView/000000032092 | - |
| dc.description.alternativeAbstract | In knowledge distillation, since a single, omnipotent teacher network cannot solve all problems, multiple teacher-based knowledge distillations have been studied recently. However, sometimes their improvements are not as good as expected because some immature teachers may transfer the false knowledge to the student. In this paper, to overcome this limitation and take the efficacy of the multiple networks, we divide the multiple networks into teacher and student groups, respectively. That is, the student group is a set of immature networks that require learning the teacher's knowledge, while the teacher group consists of the selected networks that have performed well. Furthermore, according to our online role change strategy, the top-ranked networks in the student group are able to promote to the teacher group at every iteration and vice versa. After training the teacher group using the error images of the student group to refine the teacher group's knowledge, we transfer the collective knowledge from the teacher group to the student group successfully. We verify the superiority of the proposed method on CIFAR-10, CIFAR-100, and ImageNet which achieves high performance. We further show the generality of our method with various backbone architectures such as resnet, wrn, vgg, mobilenet, and shufflenet. | - |
- Appears in Collections:
- Graduate School of Ajou University > Department of Artificial Intelligence > 3. Theses(Master)
- Files in This Item:
- There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.
-
- License
- STATISTICS
- Total Visit :3,548,429
- Total Download :1,795
- Today View :10,794
