7 AI Improvement Best Practices & Case Studies in 2024 (2024)

AI and ML models inevitably degrade over time, regardless of the level of sophistication of the models or the diversity of the dataset, unless they are regularly updated or re-trained.1

Here, we outline best practices and case studies for maintaining or improving the quality of AI models to help developers and digital transformation leaders ensure consistent performance.

7 ways to improve your AI model

Explained in 3 different categories:

Feed more data

Adding new and fresh data is one of the most common and effective methods of improving the accuracy of your machine-learning model. Research2 has shown a positive correlation between dataset size and AI model accuracy (See figure below).

Therefore, expanding the dataset that is used for model retraining can be an effective way to improve AI/ML models. Make sure that the data changes according to the environment in which it is deployed. It is also important to follow proper data collection quality assurance practices.

1. Data Collection

Data collection/harvesting can be used to expand your dataset and feed more data into the AI/ML model. In this process, fresh data is collected to re-train the model. This data can be harvested through the following methods:

• Private collection
• Automated data collection
• Custom crowdsourcing

Check this quick read to learn more about these data collection methods.

You can also work with an AI data service to obtain relevant datasets without the hassle of gathering data. Check out these articles to find the right data collection service for you AI project:

2. Synthetic data

This is another way of feeding more data into an AI/ML model. As the name implies, synthetic data is data that is created artificially rather than by real events. It is generatedbyusingalgorithms and is used for a variety of tasks, such as testing new AI/ML models, model validation, and re-training AI models. Synthetic data can be used to meet specific dataset requirements that can not be fulfilled with existing (real) datasets. Check this quick read to learn more about how synthetic data can be generated.

Case study: More data for chatbots3

A chatbot for IT support struggled with understanding and classifying user questions accurately. To improve its performance, 500 IT support queries were rewritten into multiple variations across seven languages. This additional data helped the chatbot recognize different question formats, enhancing its ability to respond more effectively over time.

Improve the data

Improving the existing data can also result in an improved AI/ML model.

Now that AI solutions are tackling more complex problems, better and more diverse data is required to develop them.For instance, research4 about a deep-learning model that helps object detection systems understand the interactions between two objects, concludes that the model is susceptible5 to dataset bias and requires a diverse datasets to produce results.

Improvements can be achieved through:

3. Enriching the data

Expanding the dataset is one of many ways to improveAI. Another important way of enhancingAI/ML models is enriching the data. This simply means that the new data that is collected to expand the dataset must be processed before feeding into the model. This can also mean improving the annotation of the existing dataset. Since new and improved labeling techniques are developed, they can be implemented on the existing or newly gathered dataset to improve model accuracy.

4. Improving data quality

Improving the quality of the data is also necessary for improving AI/ML models. Using a data centric approach can also help in improving the development process of AI. The quality of the data can be improved by maintaining the quality during the collection phase. You can also check this quick read to learn about how the quality of existing data can be improved.

5. Leveraging data augmentation

Some people might confuse augmented data with synthetic data; however, both the terms have some differences. Augmented data refers to the addition of information to an existing dataset, while synthetic data is generated artificially to stand in for real data. augmented data is often used to improve the accuracy of predictions or models, while synthetic data is commonly used for testing and validation.

Check out this quick read to learn more about the different techniques of data augmentation.

Case study: Speech recognition data improvement6

Challenge: The speech recognition system for car infotainment struggled to understand diverse voice commands.

Solution: Thousands of voice recordings from different regions were collected, transcribed, and analyzed to improve recognition accuracy. This improvement in the voice dataset helped train the system to respond better to various commands and pronunciations.

Improve the algorithm

Sometimes, the algorithm that was initially created for the model needs to be improved. This can be due to different reasons, including a change in the population that the model is deployed on.

For instance, studies show that patients with lower income levels have a greater health risk as compared to patients with higher income levels. Suppose a deployed AI/ML algorithm that evaluates the patient’s health risk and does not include the income level parameter is suddenly exposed to data of patients with lower income levels. In that case, it is unlikely to produce fair evaluations.

Therefore, upgrading the algorithm and adding new parameters to it can be an effective way to improve model performance. The algorithm can be improved in the following ways:

6. Improve the architecture

There are a few things that can be done in order to improve the architecture of an algorithm. One way is to take advantage of modern hardware features, such as SIMD7 instructions or GPUs. Additionally, data structures and algorithms can be improved through the use of cache-friendly data layouts and efficient algorithms. Finally, algorithm developers can exploit recent advances in machine learning and optimization techniques.

7. Feature re-engineering

Feature re-engineering of an algorithm is the process of improving the algorithm’s features in order to make it more efficient and effective. This can be done by modifying the algorithm’s structure or by tweaking its parameters.

Case study: Deepmind8

Google DeepMind made significant improvements to its AI models by optimizing their architecture and re-engineering various components for better performance. For example, the Gemini model was built with a multimodal architecture, enabling it to handle tasks across text, audio, and images more effectively. Additionally, PaLM 2 was enhanced by using a compute-optimal scaling approach and dataset improvements to enhance reasoning tasks. These architectural upgrades allowed for greater accuracy and adaptability

Recommendations on how to approach AI/ML model improvement

This section highlights 2 recommendations on what to do before implementing the 3 AI improvement techniques mentioned in the article:

Monitor performance

You can only improve something by knowing its areas to improve. This can be done by monitoring the features of the AI/ML model. However, if all the model features can not be monitored, only a selected number of key features can be observed to study variations in their output that can impact the model performance.

Hypothesis generation

Prior to selecting the right method, we recommend performing hypothesis generation. This is a pre-decisional process that structures the decision process and narrows down the options. This process involves gaining domain knowledge, studying the problem the AI/ML model is facing, and narrowing down readily available options that can tackle the identified issues.

FAQs

Further reading

• 4 Steps and Best Practices to Effectively Train AI

External resources

• 1. Vela, D., Sharp, A., Zhang, R., Nguyen, T., Hoang, A., & Pianykh, O. S. (2022). Temporal quality degradation in AI models. Scientific Reports, 12(1), 1-12.
• 2. Adey, Brenton. (2021). Investigating ML Model Accuracy as Training Size Increases. Accessed: 22/Nov/2022.
• 3. Training Data for Chatbots. Clickworker. Accessed: 11/Sep/2024.
• 4. Liu, N., Li, S., Du, Y., Tenenbaum, J., & Torralba, A. (2021). Learning to compose visual relations. Advances in Neural Information Processing Systems, 34, 23166-23178.
• 5. Anyverse (February 28, 2022), “More complex deep learning models require more complex data.” Retrieved: 22/Nov/2022.
• 6. Speech recognition training data for software development. Clickworker. Accessed: 11/Sep/2024.
• 7. Single instruction, multiple data. Wikipedia. Nov 2, 2022. Accessed: 5/Dec/2022
• 8. 2023: A Year of Groundbreaking Advances in AI and Computing. Deepmind. Accessed: 11/Sep/2024.