Trustworthy and energy-efficient AI solutions are explored by JunKyu Lee, Research Fellow in the Institute for Analytics and Data Science.
I am a new research fellow in the Institute for Analytics and Data Science (IADS). The aim of this blog is to share my research background and plan with potential research partners.
Research Overview
I have a research vision. "Make the world sustainable from massive AI usage!" AI is used everywhere, consuming astronomical energy and producing massive CO2 emissions. It is expected that energy consumption and CO2 emissions from AI usage will increase rapidly. To tackle this issue, my research is focused on minimizing energy consumption and CO2 emissions by co-optimising machine learning models across algorithm, arithmetic and dataflow. My core research skillset includes incorporating numerical linear algebra properties into machine learning frameworks [1, 2], analysing the accuracy and stability of machine learning models with respect to dynamically varying environments [2-5], and providing energy-efficient machine learning solutions based on the findings from the analysis [1, 3, 4, 6].
Research Challenges
It is challenging to deploy high-performance machine learning models on computing devices under limited computational resources and energy (e.g., IoT-based AI). E.g., if you would like to improve the accuracy of a machine learning model, more computational resources and energy are generally required. To improve the resource efficiency of a model, the accuracy and stability of machine learning models can assist in selecting the least sufficient models given the dataset.
Research Plan in IADS
I am thrilled to work on establishing a "Mathematical and Computational Foundation of Energy-Efficient Machine Learning (MACFEEM)" during the Institute Fellowship. If successfully completed, this foundation can lead to significant economic, social, and environmental impacts as shown in Figure 1, since many potentially critical AI applications can be handled within this foundation.
I would like to leverage an excellent cross-disciplinary research environment provided by IADS. For example, I want to incorporate my energy-efficient machine learning research skills into exploring security-aware energy-efficient machine learning with cyber security experts in IADS (e.g., Prof. Haris Mouratidis) since many IoT AI applications require high security and energy efficiency simultaneously. I can also explore data-efficient machine learning methods (e.g., kernel methods) for causal inference tasks with causal inference experts (e.g., Damian Machlanski and Dr Spyros Samothrakis) since the number of training samples is not often sufficient for causal inference tasks.
Research staff in IADS have diverse research backgrounds, centred on data science or machine learning, including computer science, linguistics, social science, life science, political science, etc. I am thrilled to work with them to evaluate the effects of MACFEEM on real life in terms of energy consumption and data efficiency by utilizing the training and test dataset they collected. I hope my accuracy/stability analysis skills can be incorporated into diverse research expertise in IADS to realize part of the IADS goal, "Trustworthy Data Science". During this Fellowship, my research will be supported and guided by IADS Director, Prof. Haris Mouratidis, and mentor, Dr Xiaojun Zhai.
Proposal Plan in IADS
I wish to make a research proposal, “Mathematical and Computational Foundation of Energy-Efficient Machine Learning (MACFEEM)”, which aims to provide energy-efficient AI solutions by incorporating numerical linear algebra properties into a machine learning framework to impact society, the economy, and the environment, as shown in Figure 1. For example, the numerical linear algebra community analyzes the effects of numerical behaviours on accuracy and stability quantitatively, while the machine learning community prefers qualitative understanding. Therefore, the analysis approaches used by the two communities complement each other, and the combined analysis will assist in building the ecosystem towards "Trustworthy AI" in terms of transparency, safety and explainability by clarifying the effects of numerical behaviors (e.g., the perturbation on input) on the accuracy and stability of machine learning. Since most UKRI AI funding calls are related to either Net Zero or Trustworthy AI, I wish to combine MACFEEM with Trustworthy AI with IADS research staff to secure research funding considering Net Zero and Trustworthy AI simultaneously during this Institute Fellowship.
Figure 1. Impact of MACFEEM on Society, the Economy, and the Environment
What am I looking for?
I am exploring cross-disciplinary research opportunities to promote healthcare, manufacturing, security, etc. For example, optimal machine learning algorithms mainly rely on data characteristics (e.g., data quantity, quality, complexity, distribution, and noise, etc.). According to such data characteristics, optimal machine learning models and compute resources can be selected. For example, researchers in life science can explore how to utilize their data for features and labels, while I can explore which algorithm and computing devices can be recommended according to the data characteristics. In this regard, I would like to explore trustworthy energy-efficient AI solutions for real life through cross-disciplinary research approach with the Institute for Social and Economic Research and the Institute of Public Health & Wellbeing. I am also interested in engaging in Knowledge Transfer Partnerships (KTP) to maximize the impacts of my research on industry and society. If you are interested in a collaboration, please feel free to contact me. Thank you so much.
References
[1] J. Lee, D. S. Nikolopoulos, and H. Vandierendonck, "Mixed-Precision Kernel Recursive Least Squares," IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 3, pp. 1284-1298, 2022, doi: 10.1109/TNNLS.2020.3041677.
[2] J. Lee, B. Varghese, and H. Vandierendonck, "ROMA: Run-Time Object Detection To Maximize Real-Time Accuracy," in the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023, pp. 6405-6414.
[3] J. Lee and H. Vandierendonck, "Towards Lower Precision Adaptive Filters: Facts From Backward Error Analysis of RLS," IEEE Transactions on Signal Processing, vol. 69, pp. 3446-3458, 2021, doi: 10.1109/TSP.2021.3086355.
[4] J. Lee, H. Vandierendonck, M. Arif, G. D. Peterson, and D. S. Nikolopoulos, "Energy-Efficient Iterative Refinement Using Dynamic Precision," IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 8, no. 4, pp. 722-735, 2018, doi: 10.1109/JETCAS.2018.2850665.
[5] J. Lee, B. Varghese, R. Woods, and H. Vandierendonck, "TOD: Transprecise Object Detection to Maximise Real-Time Accuracy on the Edge," in 2021 IEEE 5th International Conference on Fog and Edge Computing (ICFEC), 10-13 May 2021 2021, pp. 53-60, doi: 10.1109/ICFEC51620.2021.00015.
[6] J. Lee et al., "Resource-Efficient Convolutional Networks: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques," ACM Comput. Surv., vol. 55, no. 13s, p. Article 276, 2023, doi: 10.1145/3587095.
