AI Vendor Performance Monitoring and Measurement
Expert-defined terms from the Professional Certificate in Artificial Intelligence Vendor Due Diligence Framework course at London School of International Business. Free to read, free to share, paired with a globally recognised certification pathway.
Artificial Intelligence (AI) #
the simulation of human intelligence processes by machines, especially computer systems. These processes include learning (the acquisition of information and rules for using the information), reasoning (using the rules to reach approximate or definite conclusions), and self-correction.
AI Vendor Performance Monitoring #
the ongoing evaluation and measurement of an AI vendor's performance in delivering products or services that meet the agreed-upon requirements and expectations.
AI Vendor Performance Measurement #
the process of quantifying and evaluating an AI vendor's performance using predefined metrics and criteria.
AI Vendor Due Diligence #
the process of evaluating an AI vendor's capabilities, performance, and risks before entering into a business relationship or contract.
Algorithm #
a set of statistical processing steps. In AI, an algorithm is a set of mathematical instructions that a machine follows to complete a task.
Artificial Neural Network (ANN) #
a computing system inspired by the human brain's biological neural networks. ANNs consist of input and output layers, as well as (in most cases) a hidden layer consisting of units that transform the input into something the output layer can use.
Big Data #
extremely large data sets that may be analyzed computationally to reveal patterns, trends, and associations, especially relating to human behavior and interactions.
Computer Vision #
a field of AI that trains computers to interpret and understand the visual world.
Deep Learning #
a subset of machine learning in AI that has networks capable of learning unsupervised from data that is unstructured or unlabeled. Also known as deep neural learning or deep neural networking.
Machine Learning (ML) #
a method of data analysis that automates analytical model building. It is a branch of artificial intelligence based on the idea that systems can learn from data, identify patterns, and make decisions with minimal human intervention.
Natural Language Processing (NLP) #
a field of AI that focuses on the interaction between computers and human language. It involves the ability of a computer program to understand human language as it is spoken.
Performance Metrics #
quantifiable measures used to evaluate the quality, effectiveness, and efficiency of an AI vendor's products or services.
Predictive Analytics #
the use of data, statistical algorithms, and machine learning techniques to identify the likelihood of future outcomes based on historical data.
Quality Assurance (QA) #
the process of ensuring that products and services meet specified requirements and standards.
Regression Testing #
the process of testing changes to computer programs to ensure that previously developed and tested programs still function correctly after a change.
Return on Investment (ROI) #
a performance measure used to evaluate the efficiency of an investment or to compare the efficiency of a number of different investments.
Risk Management #
the process of identifying, assessing, and prioritizing uncertainties in investment decisions and taking coordinated and economical actions to minimize potential losses.
Robotic Process Automation (RPA) #
the use of software with artificial intelligence (AI) and machine learning capabilities to handle high-volume, repetitive tasks that previously required humans to perform.
Supervised Learning #
a type of machine learning in AI where the AI is trained on a labelled dataset, allowing it to learn the relationship between inputs and outputs and generalize that relationship to new data.
Training Data #
the data used to train AI models. The quality and quantity of training data can significantly impact an AI model's performance.
Unsupervised Learning #
a type of machine learning in AI where the AI is trained on an unlabelled dataset, allowing it to learn patterns and relationships in the data without any pre-existing labels or guidance.
Vendor Management #
the process of selecting, evaluating, and overseeing the work of outside contractors, consultants, and vendors.
Vendor Performance Evaluation #
the process of assessing an AI vendor's performance against predefined criteria, including quality, timeliness, and cost.
Vendor Risk Management #
the process of identifying, assessing, and prioritizing potential risks associated with working with an AI vendor and taking coordinated and economical actions to minimize those risks.
In the context of the Professional Certificate in Artificial Intelligence Vendor… #
This process is crucial for ensuring that the AI vendor's products or services meet the agreed-upon requirements and expectations. Effective AI Vendor Performance Monitoring and Measurement require a thorough understanding of the AI vendor's capabilities, performance, and risks, as well as the ability to identify and mitigate potential issues before they become major problems.
To ensure successful AI Vendor Performance Monitoring and Measurement, it is ess… #
These metrics should be aligned with the organization's overall AI strategy and objectives and should be regularly reviewed and updated as necessary. Effective performance metrics might include measures of accuracy, completeness, timeliness, and efficiency, among others.
Once performance metrics have been established, it is essential to put in place… #
This process should include regular status updates, progress reports, and meetings with the AI vendor to review performance and address any issues or concerns. It is also essential to establish a process for addressing underperformance, including the possibility of escalation or termination of the vendor relationship if necessary.
In addition to monitoring and measuring performance, it is essential to establis… #
This process should include regular risk assessments, the identification of potential risks and vulnerabilities, and the development and implementation of risk mitigation strategies. Effective risk management can help ensure that the organization's AI strategy and objectives are not compromised by issues related to the AI vendor's performance or behaviour.
Finally, it is essential to establish a process for ongoing vendor management, i… #
Effective vendor management can help ensure that the organization's AI strategy and objectives are fully realized, and that the benefits of working with an AI vendor are maximized.
In summary, AI Vendor Performance Monitoring and Measurement are critical compon… #
Effective performance monitoring and measurement require a thorough understanding of the AI vendor's capabilities, performance, and risks, as well as the ability to establish clear performance metrics, regularly monitor and measure performance, manage risks, and optimize the vendor relationship over time. By following best practices in AI Vendor Performance Monitoring and Measurement, organizations can ensure that their AI strategies and objectives are fully realized, and that the benefits of working with an AI vendor are maximized.