New data management solutions are urgently needed
The energy demand of Artificial Intelligence (AI) systems is a growing concern, especially with the increasing use of deep learning and other computationally intensive algorithms. OpenAI used 45 terabytes of data to train the GPT-3 model. Despite its modelling and powerful description capabilities, training the GPT-3 model required a large amount of computing power, data, and capital investment, which translates into significant carbon dioxide emissions (Bavarian, 2022). With the increased use of AI across several domains, from manufacturing to banking, to intelligent transport systems, Europe urgently needs to develop new data management solutions that will harness the transformative potential of the AI whilst contributing to the green transition, meeting the European Green Deal objectives.
We need to start designing AI algorithms that consume less energy.
Energy demand for AI Services
In addition to the training of models, AI systems also face high energy demands related to tasks such as inference, data storage and retrieval, and data centres’ cooling. It is possible to address the issue by creating AI algorithms that are designed to consume less energy. Energy efficiency can be considered the ratio of the energy consumed by an AI service to the output or work produced depending on the nature of the service e.g. energy per prediction, energy per transaction etc. Evaluating if an AI service is energy efficient, requires a multi-faceted approach that considers a range of factors and uses respective metrics. Measuring, for example, the energy required to train an AI model involves instrumenting the hardware or using power models that provide energy consumption estimates based on hardware specifications, collecting data on energy consumption during the training process, normalising the data based on the size and complexity of the model, and reporting the results along with other performance metrics.
To measure the energy consumption, meaning the total energy consumed by the AI service during its operation, energy monitoring software and hardware or power meters can be employed. The respective carbon footprint can be estimated based on the energy consumption of the service and the carbon intensity of the energy sources used (e.g. green renewable power sources vs. conventional ones). Further, the design of energy-efficient AI systems needs to factor in scalability, allowing for scaling up or down based on demand, without requiring significant increase in energy. A key to AI services design is balancing efficiency with performance. A more energy-efficient service should achieve comparable or better performance with less energy consumption, therefore ensuring acceptable levels of accuracy and speed of the service.
Evaluating AI Efficiency
Haj-Ali (2019) introduced a methodology for the fair evaluation of Machine Learning (ML) algorithms with respect to resource consumption, specifically their energy consumption, memory usage, and runtime, proposing a combination of accuracy and resource cost. The proposed methodology involves measuring the performance of ML algorithms using a range of datasets with varying sizes and characteristics, and comparing their resource consumption across different computing platforms. There is significant variability in resource consumption across different ML algorithms and computing platforms, attributed to factors such as algorithm design, implementation, and hardware configuration. Recommendations for improving the energy efficiency of ML algorithms included using simpler models, optimising hyperparameters, and minimising data movement.
The review of techniques for improving the energy efficiency of ML algorithms by Y. Zhang et al. (2021), includes hardware acceleration, model optimisation, and data compression. Data compression techniques, such as weight sharing and sparsity, reduce the size of data processed during training and inference. Common techniques for model optimisation include pruning, quantization, and knowledge distillation, which can reduce the number of parameters in a model and improve energy efficiency. Pruning refers to the process of reducing the size of the model by removing certain parts of it without significantly sacrificing accuracy. By removing unimportant weights, the model is streamlined and can run faster with less computational resources, making it more suitable for deployment on resource-limited devices. Quantization reduces the memory and computation requirements of the model. Knowledge distillation is the technique where a smaller and more computationally efficient model is trained to mimic the behaviour of a larger, more complex model. The idea is to transfer the knowledge or insights learned by the larger model to the smaller model, allowing it to achieve similar performance with fewer parameters and less computation.
Alsedais et al (2020) compared various techniques for improving energy efficiency during the training and inference phases of deep learning, including software optimisations and model compression techniques, such as model parallelism and pipeline parallelism. The authors also highlight the importance of optimising the choice of algorithms and the hyperparameters used in deep learning models to improve energy efficiency.
Schwartz (2019) argues that AI developers have a responsibility to mitigate the environmental impact of their technology and proposed several strategies for achieving “Green AI”, including model compression. Gartner’s report on Emerging Tech Impact Radar 2023, also highlights model compression as a potential solution to overcoming current AI limitations, along with edge AI and synthetic data. Model compression can significantly reduce a model’s size, with negligible performance impact.
The GREEN.DAT.AI project vision
In this context, the GREEN.DAT.AI project aims to channel the potential of AI towards the Europe’s sustainability goals, by developing novel Energy-Efficient Large-Scale Data Analytics Services, ready-to-use in industrial AI-based systems while reducing the environmental impact of data management processes. Core to this shift from AI accuracy to AI efficiency are Federated Learning (FL) mechanisms, which utilise distributed data for inferring information based on decentralised collaborative modelling algorithms. The ambition is to minimise data transfer, achieve faster inference with a shorter reaction time and drive innovation in applications where these parameters are critical. The AI Services Toolbox will include:
- AI-enabled data enrichment
- Incentive mechanisms for data sharing
- Synthetic data generation
- Large-scale learning at the edge/fog
- Federated & Auto ML at the edge/fog
- Explainable AI/Feature Learning with privacy preservation
- Federated & Automatic Transfer Learning
- Adaptive FL for Digital Twin applications
- Automated IoT event-based change detection and forecasting.
The project will also develop a benchmarking and evaluation framework for measuring and comparing the energy efficiency of different AI services, addressing the need for more accurate energy consumption models and the development of energy-aware AI algorithms.
The efficiency of the AI services will be measured in four industries (Smart Energy, Smart Agriculture/Agri-food, Smart Mobility, Smart Banking), leveraging the use of AI-ready data spaces. An AI-ready data space is a data management framework designed to support the use of AI techniques. This data environment enables the efficient processing, analysis, and sharing of data across different organisations in a way that is compatible with AI workflows. The key features of an AI-ready data space include interoperability, data quality, data security and privacy, data discovery, and AI capabilities.
Overall, an AI-ready data space is an important component of an organization’s AI strategy, as it provides a foundation for the effective use of energy-efficient AI techniques in data-driven workflows. Together, these technologies as combined in GREEN.DAT.AI are expected to unlock new potential for existing and future AI applications that address emerging business and societal challenges.
References
- A European strategic long-term vision for a prosperous, modern, competitive and climate neutral economy, A Clean Planet for all, COM(2018) 773 final, Brussels, 28.11.2018
- A. Alsedais et al., A Comprehensive Study of Energy-Efficient Deep Learning: Challenges and Opportunities, 2020
- A. Haj-Ali et al. Towards Energy-Efficient Machine Learning: A Methodology for Benchmarking and Analysis, 2019.
- B. Rana et al., Towards Energy-Efficient Machine Learning: A Survey of Techniques and Metrics, 2021
- M. Bavarian, H. Jun et.al, Efficient training of language models to fill in the middle, Open AI, https://doi.org/10.48550/arXiv.2207.14255, 2022
- M. Zhang, J. Li, A commentary of GPT-3 in MIT Technology Review 2021, Fundamental Research, Volume 1, Issue 6, 2021, Pages 831-833, https://doi.org/10.1016/j.fmre.2021.11.01
- S. Benedict, Energy Efficient Aspects of Federated Learning – Mechanisms and Opportunities. In K.K. Patel, et al. (eds.) Soft Computing and its Engineering Applications. Springer, 2021
- R. Schwartz et al., Green AI, 2019
- T. Nguyen, A. Jump, D. Casey, Gartner Research Excerpt, 2023 Emerging Tech Impact Radar, 2022.
- Y. Zhang et al., Energy-Efficient Machine Learning: A Review, 2021
- GREEN.DAT.AI Project Grant Agreement
About
Author
Learn more about the project
Website: https://greendatai.eu
This project has received funding from the Horizon Europe research and innovation programme under GA 101070416. Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or European Commission. Neither the European Union nor the granting authority can be held responsible for them.
