model compression 썸네일형 리스트형 [ACM SAC 2025] Advanced Knowledge Transfer: Refined Feature Distillation for Zero-Shot Quantization in Edge Computing https://arxiv.org/abs/2412.19125 (Accepted)Abstract기존 Zero-Shot Quantization(ZSQ, Data-Free Quantizaton) 분야에서는 full-precision(FP) Model로부터 높은 quality의 데이터를 생성하는 데 초점을 두는 연구가 진행되고 있음. 하지만, low-bit(높은 압축률) 환경에서 Quantized Model을 학습할 때는 Quantized Model이 정보 수용량 관련 한계를 갖기 때문에 데이터를 생성하는 기법만으로는 적절한 학습이 이루어지지 않음. 이러한 한계를 개선하기 위해 본 논문에서는 Quantized Model을 효과적으로 학습하기 위한 AKT(Advanced Knowledge Transfer) Meth.. 더보기 [Neurocomputing 2025] A lightweight video anomaly detection model with weak supervision and adaptive instance selection https://www.sciencedirect.com/science/article/pii/S0925231224014693 Neurocomputing (IF: 5.5, Q1) A lightweight video anomaly detection model with weak supervision and adaptive instance selectionVideo anomaly detection is to determine whether there are any abnormal events, behaviors or objects in a given video, which enables effective and inte…www.sciencedirect.com AbstractWeakly supervised vid.. 더보기 [ICML2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache Liu, Zirui, et al. "Kivi: A tuning-free asymmetric 2bit quantization for kv cache." ICML 2024 (Poster) https://icml.cc/virtual/2024/poster/34318 ICML Poster KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV CacheAbstract: Efficiently serving large language models (LLMs) requires batching many requests together to reduce the cost per request. Yet, the key-value (KV) cache, which stores atte.. 더보기 [AAAI2021] Cross-Layer Distillation with Semantic Calibration Chen, Defang, et al. "Cross-layer distillation with semantic calibration." Proceedings of the AAAI conference on artificial intelligence. Vol. 35. No. 8. 2021. (AAAI 21) https://ojs.aaai.org/index.php/AAAI/article/view/16865 AbstractFeature map을 기반으로 지식을 전이하는 기존 feature distillation은 student model을 효과적으로 training시키는 방식임. 하지만, 의미론적 정보(semantic information)는 다양한 layer에 분포하며 이는 부정적인 regualrization.. 더보기 이전 1 다음
