As more and more enterprises ride the wave of Artificial Intelligence (AI) systems and applications, a need for a structured approach for managing AI projects arises. In the past, enterprises used to deploy few Machine Learning (ML) and AI models to address specific needs such as mining historic datasets for extracting business knowledge. During the last couple of years, enterprises have entered a new era, as they are planning to deploy data-driven systems in almost every aspect of their business operations. In this direction they employ multiple models and algorithms for different applications, yet over a common data infrastructure. Hence, enterprises must monitor the development and deployment of many AI systems, in terms of a variety of properties such as their effectiveness, trustworthiness, security, and economic benefit. This requires a structured monitoring framework, which is commonly known as AI governance.
AI governance relies on a common set of processes for understanding and auditing the AI models used by enterprise. It is aimed at providing a unified, yet integrated approach to scrutinizing the training, the operation and the business benefits of various algorithms. Modern enterprises must create a proper governance framework to ensure the trusted and effective operation of their AI systems.
The Main Aspects of AI Governance
AI Governance frameworks audit AI models and algorithms across their entire lifecycle. In this direction, they must consider the following aspects of AI models’ development and deployment:
- Data Infrastructure and Management: Most AI techniques rely on proper datasets for training ML algorithms. Therefore, enterprises establish scalable, secure and reliable data management infrastructures to support their AI systems. Likewise, AI governance frameworks must ensure the scalability, resilience and reliability of the datasets that are used for building AI systems. For instance, industrial enterprises (e.g., manufacturers, energy operators, oil & gas companies) must monitor and ensure the reliability of the data that are used to train AI algorithms. This is because industrial data tend to be unreliable due to environmental factors in industrial sites, but also due to their fragmentation across multiple heterogeneous systems.
- Algorithms Training: The governance of AI and ML algorithms training is very important for ensuring their accuracy and trustworthiness. Non-properly trained algorithms suffer from different types of biases in their operation, including for example social, racial and gender related biases. Likewise, the training of AI algorithms affects their ability to be used in certain contexts. For instance, it is generally wrong to train an AI algorithm for a specific problem (e.g., employee hiring selection) and used it in another context (e.g.., employee performance assessment). Therefore, a proper AI governance framework must ensure that algorithms are properly trained i.e., using unbiased and statistically balanced data for the problem at hand.
- AI Models Library: AI governance must also audit different models and algorithms against their appropriateness for specific tasks. When building and operating multiple AI systems, organizations must ensure the selection of proper models for the given tasks (e.g., Long Short Term Memory (LSTM) and Recurrent Neural Networks (RNN) are appropriate for predictive maintenance). Hence, AI governance frameworks must keep track of the models used and how they match specific needs.
- Security: The rise of AI systems is providing new opportunities for adversarial attacks, such as poisoning and evasion attacks against deep neural networks. Poisoning attacks (re)train a neural network with adversarial data towards compromising the model’s ability to produce right decisions (e.g., correct classification of patterns). Likewise, evasion attacks try to fool deep neural networks by feeding them with adversarial inputs that the models fail to classify correctly. Thus, an AI governance framework must include measures for ensuring the cyber-resilience of AI models against such security attacks.
- Ethical, Legal and Regulatory Compliance: The misuse of AI systems can lead to unethical decisions (e.g., socially biased decisions), financial loss or even loss of human lives (e.g., in the case of autonomous vehicles). This is the reason why the development, deployment and operation of AI systems will be subject to regulation. For instance, last month the European Commission presented an AI regulation proposal, which will affect the ways European organizations develop and use AI. The proposal makes provisions for considerable regulatory penalties in cases of non-compliance. Overall, AI governance must ensure the legal and regulatory compliance of the AI systems of an enterprise.
- People Dimension: AI Governance frameworks must also consider the impact of AI systems on people, including employees and customers of an enterprise. This include an analysis on the human involvement in the operation of AI systems, including the needs for human oversight and human-robot collaboration. The human aspects of AI are closely related to the assessment of the ethical, transparent and regulatory compliant use of AI.
AI Governance Best Practices
To establish a proper governance framework with the above-listed dimensions, enterprises had better adopt the following best practices:
- Business Objectives First: AI governance is set to optimize the use of AI within an enterprise and the business benefits that stem from it. Hence, it should prioritize the accomplishment of business objectives. The latter should be used to drive audits over processes like AI model selection, training of algorithms, ethical compliance and more.
- AI Projects Portfolio Management: Governance frameworks must be designed in ways that account for the entire AI projects portfolio of the company. As such they should consider all the data of the company, the full set of algorithms used, as well as the different stakeholders’ roles that engage in AI development and use.
- Senior Management Involvement: AI governance cannot be successful without involvement of the top management of a company i.e., C-level executives’ involvement. Senior management engagement demonstrates the importance of governance and the commitment of the company to it. Furthermore, it is the only way to ensure the engagement of all business units, departments and employees affected by AI.
- Setting and Tracking Key Performance Indicators (KPI): Bill Hewlett’s famous quote: “You cannot manage what you cannot measure” applies in the case of AI governance as well. Enterprises must define and track tangible KPIs as part of their AI governance frameworks. KPIs must cover areas like AI systems technical and financial performance, users’ acceptance and satisfaction, potentially biased decisions, and more.
- Explainable AI (XAI): XAI aims at developing “white glass” models that are understandable to humans. It is used to explain how black-box models like deep neural networks operate, which is a key for the acceptance of AI systems by humans. The development and use of XAI systems boost the transparency of AI systems operations and is a good practice for AI governance.
In the era of hyper-automation organizations will have to deploy, operate and manage many AI systems. This management won’t be effective without proper governance. This is the main reason why companies must develop a proper governance framework as part of their investments on expanding and scaling up the use of AI and its impact on business performance. We therefore expect Chief Information Officers (CIOs) and Digital Transformation Manager (DTM) to include AI governance in their agendas and to consider the above-listed guidelines.
