Research Projects

• Keywords: AI model compression, sparse coding, quantization

• Keywords: binary code, deobfuscation, generative AI

• Keywords: computer vision, sound recognition, on-sensor AI, model compression with resource constraints

Fine dusts (PM10 and PM2.5) are high risk factors against public health in South Korea, and long-term predictions of fine dusts are essential for governmental decision making such as initiating emergency reduction measures. In this, accurate prediction is required to minimize social and economic impacts. 

In this project, we create new long-term (+3 ~ +6 days ahead) PM10 and PM2.5 prediction models based on deep neural nets using meteorological fields and related inputs, with the goal to outperform the current forecasting systems such as WRF and CMAQ that have clear limits in uncertainty accumulation over long prediction horizons.

Publication

• • Development of a Deep Learning-based Midterm PM2.5 Prediction Model Adapting to Trend Changes, Dong Jun Min, Hyerim Kim, Sangkyun Lee, The Transactions of the Korea Information Processing Society, 2024 [pdf]

Model compression is an application of sparse coding, where we "compress" models by excluding many zero values in sparse parameter vectors from storage and compution. In this project, we use L1-norm and its variants as regulaizers to induce various forms of zero-value patterns in parameter tensors in DNNs, expecially in CNNs.

We study sparse coding, a technique to use regularizers to induce certain structure in trained model paramters.  L1-norms are the most popular regularizers, appearing in machine learning & statistics (e.g. LASSO) and signal recovery (e.g. compressed sensing), where elementwise sparsity of parameter vectors leads to discovery of important variables/signals.

We're also interested in an actual implementation of the idea, so that training and testing of DNNs can be performed on embedded systems using model parameters in compressed forms. We study parallel implementations using CUDA and OpenCL backends, on embedded platforms such as Nvidia Jetson and Samsung Exynos 8890.

This project is supported by the Electronics and Telecommunications Research Institute of Korea (ETRI, 2018~2020) and the National Research Foundation of Korea (grant NRF-2018R1D1A1B07051383,  2018~2020) 

Publication:

• • Improving the Robustness of Model Compression by On-Manifold Adversarial Training , Junhyung Kwon and Sangkyun Lee, Future Internet, 2021 [pdf]

  • [SCI: Q2] Robust CNN Compression Framework for Security-Sensitive Embedded Systems, Jeonghyun Lee and Sangkyun Lee, Applied Sciences, 2021 [pdf]

  • [SCI: Q2] Data Quality Measures and Efficient Evaluation Algorithms for Large-Scale High-Dimensional Data, Hyeongmin Cho and Sangkyun Lee, Applied Sciences, 2021 [pdf]

This project has been supported by Samsung SDS (2020~2021)

Patents:

• 악성 코드 탐지 방법 및 그 장치 (Method for detecting malware and apparatus thereof), 출원번호: 10-2021-0147222

Recently it has been shown that ML models can be fooled by creating so-called adversarial examples, modifying data points to maximize the training loss function. Adversarial examples have been studied actively in computer vision and computer security. 

We have investigated recently proposed attack mechanisms against ML models, studying why such attacks are ever possible regarding learning models and theory. We also have investigated available defenses for those attacks, analyzing their potential problems. 

We now investigate malware detection problems, where ML-based detectors are getting more interest due to their capability to prevent zero-day attacks. In our research, we try to build adversarial examples with binary code constraints, to check that if it is possible to obfuscate ML-based malware detectors by modifying malware binary code in a systematic fashion.

This project has been supported by the National Security Research Institute of Korea (grants 2017-125, 2018-150)