By Anbu Muppidathi, President and CEO of Qualitest
In software development or technology adoption, it is a proven fact that fixing issues are costlier than preventing them. With AI adoption at the forefront of Investor and business owners minds, they should be deeply invested in not only the adoption but the quality of engineering in AI because it directly impacts the risk and return profile of their investments. Understanding and monitoring the engineering quality in AI adoption can provide several benefits that help investors make better investment decisions.
Let’s break out the basics of software testing. Software testing is the process of evaluating and verifying that a software product or application does what it’s meant to do. The benefits of testing include preventing bugs, reducing development costs and improving overall performance for a company. In traditional software development, engineering quality in early stages of software development lifecycle is the foundation for quality engineering and it is focused mainly on customer requirements and experience, design, and QA automation.
In a typical Continuous Integration/Continuous Delivery (CI/CD) pipeline, designing quality is a task we complete even before the requirements are signed off. Quality architecture and the underneath automation strategy, tools etc. are all in place, by the time the system architecture is ready. Continuous automation that helps the scrum team pass every one of the quality gates, will be in place, so that the correctness, completeness, consistency, performance, security and any other specific CX attributes are guaranteed at the scrum level.
Well, that is all good for traditional application development. However, when it comes to AI modeling or AI platform adoption, there are additional quality gates we need to establish because the quality attributes are substantially more than the ones that we need for typical applications. Validating the trustworthiness and safety of AI produced code/applications are fundamentally important. Businesses cannot risk their brand by pushing AI applications without verifying these additional quality attributes.
Here’s how we can approach this problem.
- Start with data, build the model for accuracy: The importance of data in AI/ML models is far greater because the models learn, use and generate data. Wrong or poor-quality data will make the model quality unacceptable. Improving the data quality over a set period is a bad idea. Necessary quality gates to validate data health are an absolute necessity throughout the model development, model deployment and model operations activities.
- Training models and data for ethical bias: Unlike traditional applications, AI models/applications learn its behavior throughout its lifecycle. Early identification of potential ethical and bias violations should be identified and prioritized by leaders and investors to prevent unintended consequences and maintain public trust. Most of the time, it is not the AI algorithm, but the training data that can help put necessary guardrails around these challenges. Training, Tuning and Testing (TTT) should all come together in constantly watching for patterns of potential deviations, so that necessary alarms can go off for fixing the model as and when the red flags are raised.
- Continuous model monitoring for safety: It’s not good enough to test the models for the required performance, especially when it comes to safety. The AI models/applications should be constantly monitored for the acceptable levels of performance. The teams should be very clear about what is "fault tolerance" so the system continues with new learning and what is a "grinding halt" so we stop the AI from functioning. Unless these events are designed at the unit level, implementing them during the model rollout won't be possible.
- Ground Truth for validation: Every AI model/application should be validated with Ground Truth, which translates to real facts and truths collected from the real-world environment. Making the necessary arrangements to collect the ground truth and use it to test the models/applications at the unit level will avoid fixing costly errors at later stages of development/deployment. While ground truth helps in accuracy and consistency of AI model performance, synthetic data can help in performance of the models.
- Expanding the coverage for Continuous automation: It is important to expand the coverage of Continuous automation to include model and training data assurance, ethical/bias pattern identification logic, performance and security vulnerabilities and above all AI-produced code properly flagged in the lifecycle so that one can address any legal implications of copyright/IP violations. Expanding the scope of Continuous automation to include model and training data assurance, ethical/bias pattern identification, and security vulnerabilities is critical in addressing legal implications for investors and company stakeholders.
- Setting model operations for faster feedback loop: It is vital to speed up incorporating customer inputs - in addition to the QA feedback look - in the AI model/applications. No matter how much testing we might have done; customer inputs can give a lot more insights about our models/applications. Processing the customer inputs faster in the scrum cycle will help address the issues faster.
- Full picture view versus AI Component view: Corporation will have the mix of AI and non-AI application components in the IT that are responsible for the business process. As we fast develop and adopt AI, the integration of these AI/Non-AI components should not be overlooked. Remember, it is not the functionality, but the performance, security, bias, safety that are all important to be validated in the integrated model. When we test the integration, the scope should include these granular aspects at the overall systems level.
- One-off versus adoption at scale: The plan that we have for the pilot AI-application or model development need not work when we adopt AI at scale. Constantly question the quality management plan, every time an AI-application or platform is adopted. Periodic validation of the quality strategy is a must.
- Injecting human supervision throughout the lifecycle: By now, we have concluded the fact that without human supervision, AI can do nothing good. While we hyper-automate the SDLC processes with AI-powered tools, injecting human supervision to the necessary review and decision-making processes in every one of the quality gates is an absolute necessity. Without which, we will let the model learn unwanted behaviors or we may miss to avoid a potential disaster.
- Experience is the final answer: After all is said and done, it is the customer experience that ultimately certifies if our AI model/application is trustworthy. Involving customers early and throughout the AI models/application development, monitoring and operations is a smart way to keep the model quality at a higher level all the time. Customer insights provide valuable perspectives that traditional testing might overlook, facilitating faster issue resolution.
When it comes to AI adoption or AI model development, the objective of quality engineering additionally takes on “trustworthiness” and “safety.” Engineering quality for trust and safety requires intelligent quality gates in the AI adoption/model development activities. We know that AI solutions/platforms present 'black boxes' to be validated. As listed in this article, a methodical approach brings a method to the madness.
Understanding and monitoring the engineering quality in AI adoption provides several benefits that help investors make better investment decisions. It allows them to assess the risk factors, growth potential, and ethical considerations associated with AI technologies, while ultimately guiding them towards investments that align with their financial objectives and values.
About the author:
Anbu Muppidathi is the President and CEO of Qualitest. A technology veteran with more than 30 years of experience in digital transformation and technology modernization, Anbu has world-class operational and go-to-market expertise. Before joining Qualitest, Anbu most recently served as Global Head of Cognizant’s Enterprise Cloud Application Services. Prior to that, while running Cognizant’s Quality Engineering and Assurance practice between 2014 and 2018, he more than doubled the company’s testing revenue to $2.2B in annual sales with a team of 35,000 professionals while improving its analyst rankings to the leader status.
The views and opinions expressed herein are the views and opinions of the author and do not necessarily reflect those of Nasdaq, Inc.