연구동향 분석

심층 신경망의 경량화를 위한 아웃라이어 인지 양자화 기법 = Outlier-Aware Quantization Technique for Lightweight Deep Neural Networks

반근우 2022년

논문상세정보
인용/피인용
' 심층 신경망의 경량화를 위한 아웃라이어 인지 양자화 기법 = Outlier-Aware Quantization Technique for Lightweight Deep Neural Networks' 의 주제별 논문영향력
논문영향력 선정 방법
논문영향력 요약
주제
  • Deep Neural Networks
  • Model Compression
  • Outlier-Aware Quantization
  • Post-Training Quantization
  • Quantization
  • outlier
동일주제 총논문수 논문피인용 총횟수 주제별 논문영향력의 평균
136 0

0.0%

' 심층 신경망의 경량화를 위한 아웃라이어 인지 양자화 기법 = Outlier-Aware Quantization Technique for Lightweight Deep Neural Networks' 의 참고문헌

  • [91] J. Redmon, A. Farhadi, Yolov3: An incremental improvement, arXiv preprint arXiv:1804.02767, 2018.
    [2018]
  • [8] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, L. C. Chen, "Mobilenetv2: Inverted residuals and linear bottlenecks," the IEEE conference on computer vision and pattern recognition, pp. 4510-4520, 2018.
    [2018]
  • [84] W. Zhao, F. Chen, H. Huang, D. Li, W. Cheng, A New Steel Defect Detection Algorithm Based on Deep Learning, Computational Intelligence and Neuroscience, Vol. 2021, pp. 1–13, 2021.
  • [83] X. Dong, C. J. Taylor, T. F. Cootes, Small Defect Detection Using Convolutional Neural Network Features and Random Forests, In Proceedings of the European Conference on Computer Vision, 2018.
    [2018]
  • [82] J. Choi, Z. Wang, S. Venkataramani, P. I. J. Chuang, V. Srinivasan, K. Gopalakrishnan, Pact: Parameterized clipping activation for quantized neural networks, International Conference on Learning Representations, 2018.
    [2018]
  • [81] R. Krishnamoorthi, Quantizing deep convolutional networks for efficient inference: A whitepaper, arXiv preprint arXiv:1806.08342, 2018.
    [2018]
  • [7] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, H. Adam, Mobilenets: Efficient convolutional neural networks for mobile vision applications, arXiv preprint arXiv:1704.04861, 2017.
    [2017]
  • [79] Y. Bengio, N. Léonard, A. Courvile, Estimating or propagating gradients through stochastic neurons for conditional computation, arXiv preprint arXiv:1308.3432, 2013.
    [2013]
  • [77] X. Nie, H. Ding, M. Qi, Y. Wang, E. K. Wong, URCA-GAN: UpSample Residual Channel-wise Attention Generative Adversarial Network for image-to-image translation, Neurocomputing, Vol. 443, pp. 75-84, 2021.
  • [75] A. Horé, D. Ziou, Image quality metrics: PSNR vs. SSIM, 20th international conference on pattern recognition, pp. 2366-2369, 2010.
    [2010]
  • [74] S. Chen, C. Shen, L. Zhang, Y. Tang, Dynamic aggregation for heterogeneous quantization in federated learning, IEEE Transactions on Wireless Communications, Vol. 20, pp. 6804-6819, 2021.
  • [73] R. Hönig, Y. Zhao, R. Mullins, DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning, arXiv preprint arXiv:2111.00465, 2021.
  • [71] Y. Mao, Z. Zhao, G. Yan, Y. Liu, T. Lan, L. Song, W. Ding, Communication Efficient Federated Learning with Adaptive Quantization, arXiv preprint arXiv:2104.06023, 2021.
  • [6] M. Lin, Q. Chen, S. Yan, Network in network, International Conference on Learning Representations, 2014.
    [2014]
  • [66] A. Reisizadeh, A. Mokhtari, H. Hassani, and R. Pedarsani, An exact quantized decentralized gradient descent algorithm, IEEE Trans. Signal Process., vol. 67, no. 19, pp. 4934–4947, Oct. 2019.
  • [65] S. P. Karimireddy, Q. Rebjock, S. U. Stich, and M. Jaggi, Error feedback fixes signsgd and other gradient compression schemes, in Proc. 36th Int. Conf. Mach. Learn., Jun. 2019, pp. 3252–3261.
  • [64] F. Sattler, S. Wiedemann, K. Müller, and W. Samek, Robust and communication-efficient federated learning from non-i.i.d. data, IEEE Trans. Neural Netw. Learn. Syst., vol. 31, no. 1, pp. 3400–3414, Sep. 2020.
  • [63] Y. J. Cho, J. Wang, G. Joshi, Client selection in federated learning: Convergence analysis and power-of-choice selection strategies, arXiv preprint arXiv:2010.01243, 2020.
    [2020]
  • [61] W. Chen, S. Horvath, P. Richtarik, Optimal client sampling for federated learning, arXiv preprint arXiv:2010.13723, 2020.
    [2020]
  • [59] M. Kisantal, Z. Wojna, J. Murawski, J. Naruniec, K. Cho, Augmentation for small object detection, arXiv preprint arXiv:1902.07296, 2019.
    [2019]
  • [52] N. Otsu, threshold selection method from gray-level histogram, IEEE transactions on systems, man, and cybernetics, pp. 62–66, 1979.
    [1979]
  • [50] E. Westphal, H. Seitz, A machine learning method for defect detection and visualization in selective laser sintering based on convolutional neural networks, Additive Manufacturing, Vol. 41, 2021.
  • [4] Y. Li, Y. Chen, X. Dai, D. Chen, M. Liu, L. Yuan, Z. Liu, L. Zhang, N. Vasconcelos, MicroNet: Towards image recognition with extremely low FLOPs, Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 468-477, 2021.
  • [48] G. Chen, H. Wang, K. Chen, Z. Li, Y. Liu, W. Chen, A. Knoll, A survey of the four pillars for small object detection: Mul-tiscale representation, contextual information, super-resolution, and region proposal, in IEEE Transactions on Systems, Man, and Cybernetics: Systems, Vol. 52, no. 2, pp. 936-953, 2020.
  • [44] W. P. Cai, W. J. Li, Weight normalization based quantization for deep neural network compression, arXiv preprint arXiv:1907.00593, 2019.
    [2019]
  • [3] F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, K. Keutzer, SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size, arXiv preprint arXiv:1602.07360, 2019.
    [2019]
  • [39] Y. Nahshan, B. Chmiel, C. Baskin, E. Zheltonozhskii, R. Banner, A. M. Bronstein, A. Mendelson, Loss aware post-training quantization, Machine Learning, Vol. 110, pp. 3245-3262, 2021.
  • [38] M. Nagel, R. A. Amjad, M. V. Baalen, C. Louizos, T. Blankevoort, Up or down? adaptive rounding for post-training quantization, International Conference on Machine Learning, pp. 7197-7206, 2020.
    [2020]
  • [35] R. Banner, Y. Nahshan, E. Hoffer, D. Soudry, ACIQ: analytical clipping for integer quantization of neural networks, International Conference on Learning Representations, 2018.
    [2018]
  • [34] Y. Li, R. Gong, X. Tan, Y. Yang, P. Hu, Q. Zhang, F. Yu, W. Wang, S. Gu, Brecq: Pushing the limit of post-training quantization by block reconstruction, International Conference on Learning Representations, 2021.
  • [30] Y. Li, X. Dong, W. Wang, Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks, International Conference on Learning Representations, 2020.
    [2020]
  • [29] P. Stock, A. Fan, B. Garham, E. Grave, R. Gribonval, H. Jégou, A. Joulin, Training with quantization noise for extreme model compression, International Conference on Learning Representations, 2021.
  • [28] P. Stock, A. Joulin, R. Gribonval, B. Graham, H. Jégou, And the bit goes down: Revisiting the quantization of neural networks, International Conference on Learning Representations, 2019.
    [2019]
  • [23] S. Zhou, Y. Wu, Z. Ni, X. Zhou, H. Wen, Y. Zou, DoReFa-Net: Training low bitwidth convolutional neural networks with low bitwidth gradients, arXiv preprint arXiv:1606.06160, 2016.
    [2016]
  • [1] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556, 2014.
    [2014]
  • [19] M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, Y. Bengio, Binarized Neural Networks, Advances in Neural Information Processing Systems, 2016.
    [2016]
  • [18] F. Li, B, Zhang, B. Liu, Ternary Weight Networks, NIPS Workshop on Efficient Methods for Deep Neural Networks, 2016.
    [2016]
  • [17] C. Zhu, S. Han, H. Mao, W. J. Dally, Trained ternary quantization, International Conference on Learning Representations, 2017.
    [2017]
  • [16] A. Zhou, A. Yao, Y. Guo, L. Xu, Y. Chen, Incremental network quantization: Towards lossless CNNs with low-precision weights, International Conference on Learning Representations, 2017.
    [2017]
  • [14] M. Courbariaux, Y. Bengio, J. P. David, Binaryconnect: Training deep neural networks with binary weights during propagations," Advances in Neural Information Processing Systems, pp. 3123–3131, 2015.
    [2015]
  • [10] S. Han, H. Mao, W. J. Dally, Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding, International Conference on Learning Representations, 2016.
    [2016]
  • ZeroQ : A novel zero shot quantization framework
  • You only look once : Unified , real-time object detection
  • YOLO9000 : Better , faster , stronger
    J. Redmon , A. Farhadi pp . 6517 ? 6525 [2017]
  • XNOR-Net : ImageNet Classification Using Binary Convolutional Neural Networks
  • Value-aware quantization for training and inference of neural networks
    E. Park , S. Yoo , P. Vajda pp . 580-595 [2018]
  • Uniq : Uniform noise injection for non-uniform quantization of neural networks
  • UVeQFed : Universal vector quantization for federated learning ,
  • Towards accurate binary convolutional neural network
    X. Lin , C. Zhao , W. Pan pp . 345 ? 353 [2017]
  • Thinet : A filter level pruning method for deep neural network compression
    J. H. Luo , J. Wu , W. Lin pp . 5058 ? 5066 [2017]
  • Surface Defects Recognition of Wheel Hub Based on Improved Faster R-CNN
  • Small target detection using morphology and modified Gaussian distance function
    J. H. Kim , J. J . Park , S. H. Ahn , D. G. Lee Vol . 9 , pp . 555 ? 560 [2016]
  • Small object detection in optical remote sensing images via modified faster R-CNN
    Ren , C. Zhu , S. Xiao Vol . 8 [2018]
  • Shufflenet : An extremely efficient convolutional neural network for mobile devices
  • Segmentation-based deep-learning approach for surface-defect detection
  • SSD : Single Shot MultiBox Detector
  • Robust quantization : One model to rule them all ,
  • Resource management and fairness for federated learning over wireless edge networks
  • Residual networks for light field image super-resolution
    S. Zhang , Y. Lin , H. Sheng pp . 11046-11055 [2019]
  • Real-Time Small Target Detection Using Local Contrast Difference Measure at Predictive Candidate Region
    J. H. Ban , J. H. Wang , D. Lee , J. H. Yoo , Vol . 22 , pp . 1 ? 13 [2017]
  • QSGD : Communication-efficient SGD via gradient quantization and encoding ,
  • Post-training piecewise linear quantization for deep neural networks
  • Post training 4-bit quantization of convolutional networks for rapid-deployment
  • Position-based scaled gradient for model quantization and pruning ,
    J. Kim , K. Yoo , N. Kwak 33 , pp . 20415-20426 , [2020]
  • Pointer defect detection based on transfer learning and improved cas-cade-RCNN
  • Pit defect detection on steel shell end face based on machine vision
    H. Hu , D. Xu , X. Zheng , B. Zhang pp . 898 ? 902 , [2020]
  • Performance Evaluation of Stochastic Quantization Methods for Compressing the Deep Neural Network Model
    J. Choi , J. Yoo Vol . 25 , No . 9 , pp . 775-781 [2019]
  • Object Detection and Classification of Metal Polishing Shaft Surface Defects Based on Convolutional Neural Network Deep Learning
  • Low-bit quantization of neural networks for efficient inference
  • Low-bit quantization needs good distribution
  • Learning to quantize deep networks by optimizing quantization intervals with task loss ,
  • Learning both weights and connections for efficient neural networks ,
  • Improving neural network quantization without retraining using outlier channel splitting ,
  • Focal loss for dense object detection
  • Fedpaq : A communication-efficient federated learning method with periodic averaging and quantization ,
  • Federated learning : Challenges , methods , and future directions
    T. Li , A. K. Sahu , A. Talwalkar , V. Smith , Vol . 37 , pp . 50-60 , [2020]
  • Faster R-CNN : Towards Real-Time Object Detection with Region Proposal Networks
  • Fast r-cnn
    R. Girshick pp . 1440 ? 1448 [2015]
  • Exploiting Inverse Power of Two Non-Uniform Quantization Method to Increase Energy Efficiency in Deep Neural Networks
    J. Choi , J. Yoo Vol . 47 , No . 1 , pp . 27-35 , [2020]
  • Efficientnet : Rethinking model scaling for convolutional neural networks
    M. Tan , Q . Le Vol . 97 , pp . 6105-6114 [2019]
  • Double quantization for communication efficient distributed optimization
    Y. Yu , J. Wu , and L. Huang pp . 4440 ? 4451 [2019]
  • Densely connected convolutional networks
  • Deformable ConvNets v2 : More deformable , better results
    X. Zhu , H. Hu , S. Lin , J. Dai pp . 9308 ? 9316 [2019]
  • Deformable ConvNet with aspect ratio constrained NMS for object detection in re-mote sensing imagery
  • Defect intelligent identification in resistance spot welding ultrasonic detection based on wavelet packet and neural network
    J. Liu , G. Xu , L. Ren , Z. Qian , L. Ren Vol . 90 , pp . 2581 ? 2588 [2017]
  • Deep residual learning for image recognition
  • Data-free quantization through weight equalization and bias correction ,
  • Communication-efficient learning of deep networks from decentralized data
  • CollaGAN : Collaborative GAN for missing image data imputation
  • Channel pruning for accelerating very deep neural networks ,
    Y . He , X. Zhang , J . Sun pp . 1389 ? 1397 [2017]
  • Bayesian optimized 1-bit cnns
  • An improved faster R-CNN for small object detection
  • Adaptive quantization for deep neural network
    Y. Zhou , S. M. Moosavi-Dezfooli , N. M. Cheung , P. Frossard Vol . 32 , No . 1 , pp . 4596-4604 [2018]
  • Adabits : Neural network quantization with adaptive bit-widths
    Q. Jin , L. Yang , Z. Liao pp . 2146-2156 , [2020]
  • A novel algorithm for defect extraction and classification of mobile phone screen based on machine vision