What Laboratory Quality Systems need to consider

Let me start by telling you a story about a recent interaction with AI.

My husband and I regularly ask our little friend Google to tell us what the weather will be like for the day. She happily advises on current and forecast temperature, humidity, and likelihood of rain throughout the day. All important things when you’re preparing for the day ahead. Except, one day, my husband asked the question and received one answer. I subsequently asked the same question to the same device and received a completely different answer. I’m talking different temperatures by 5^oC, and a different humidity reading.

When asked the same question again by both my husband and me, she delivered the same original answers. When my husband challenged her, she came back even more emphatically with the same answer she had initially given him and said the reason for the difference was because she uses sources that can change the information minute by minute! (I’ll have to talk to the BOM about how rapidly they update their readings from the weather stations.)

By this time, it was clear that my husband was getting frustrated. Some might unkindly say it’s because we have a female Google and women can be fickle…….

What does this show? It shows how confident AI can be, even in spite of a challenge to its output, and how, up until this point, we were willing to accept Google as always telling the truth about something as simple as the weather. There is a risk in that!

 

The Age of AI

I recently saw the following post on social media.

Artificial intelligence has rapidly moved from novelty to normality.

What was once experimental is now embedded in everyday work. Reports are drafted using AI tools. Procedures are summarised automatically. Data is analysed faster than ever before. Decisions that once required human review are increasingly being supported, or influenced, by algorithms.

For quality professionals and laboratories, this creates both opportunity and risk. We previously wrote about the topic of Risk and AI a couple of years ago and made some bold predictions. Now it’s time for an update, as AI has evolved and organisations have wider usage of the technology.

The challenge is not whether AI will be used. In many cases, it already is. The challenge is understanding how it changes the nature of control, competence, decision-making, and trust within a system.

This growing focus on governance and control has also led to the development of ISO/IEC 42001, which provides a framework for managing artificial intelligence systems responsibly within organisations.

Much like earlier management system standards, the emphasis is not purely on technology itself, but on how organisations identify risks, establish controls, maintain oversight, and demonstrate accountability.

Let’s first look at some facts about AI.

 

Fact 1: AI Does Not Remove Risk

One of the more interesting aspects of AI adoption is that it is often promoted as a way to reduce human error.

In some situations, this is true. AI can improve consistency, process large amounts of information quickly, identify patterns, and automate repetitive activities. Used appropriately, these capabilities can strengthen systems and improve efficiency. And it’s very confident in giving you its answers!

At the same time, AI introduces new forms of risk that are less familiar and sometimes less visible.

Traditional quality systems tend to assume that decisions are being made by people who can explain their reasoning, demonstrate competence, and apply judgement in context. AI changes this dynamic.

The risk is not simply that AI may produce incorrect outputs. The greater risk is that people may place unwarranted confidence in those outputs without properly understanding how they were generated.

 

Fact 2: AI changes the problem from Human Error to Systemic Error

Historically, many quality systems focused heavily on reducing individual mistakes.

Training, supervision, procedures, and competency frameworks were all designed around improving human consistency and reliability.

AI changes the nature of the problem.

Instead of isolated human error, organisations may face systemic error occurring at scale. An incorrect assumption embedded within an AI-generated workflow can be repeated consistently across multiple activities. A poorly interpreted result can influence decision-making long before anyone realises there is an issue.

The danger is not inconsistency. The danger is highly efficient consistency applied to flawed reasoning.

 

Fact 3: AI and the Illusion of Authority

One of the most significant risks associated with AI is the way it presents information.

AI-generated responses are often delivered with clarity and confidence, regardless of whether the information is accurate. This creates an illusion of authority that can be difficult to challenge, particularly when outputs appear plausible or professionally written.

In laboratories and quality systems, this matters.

An incorrect calculation can often be identified through verification. A persuasive but flawed interpretation may be far more difficult to detect.

The issue is not that AI “lies” in the human sense. The issue is that language models generate responses based on patterns rather than understanding. Confidence is not the same as validity.

Quality systems have traditionally relied on traceability, verification, review, and evidence. A significant underpinning principle in Standards such as ISO/IEC 17025, ISO 15189 and ISO/IEC 17020 is that of competence. These principles become even more important in environments where AI is used.

With these facts in mind, what does the age of AI mean for quality management systems and conformance with standards such as ISO/IEC 17025, ISO/IEC 17020 and ISO 15189?

 

Competence in an AI-Supported Environment

I’m sorry, Dave, I’m afraid I can’t do that

This creates an interesting question for quality systems:

What does competence look like when AI is involved?

Traditionally, competence has focused on an individual’s ability to perform work correctly. Increasingly, competence may also involve understanding aspects such as:

• when AI tools are appropriate
• what their limitations are
• how outputs should be verified
• where human judgement remains essential

A person does not need to understand the underlying mathematics of machine learning to use AI responsibly. They do, however, need enough understanding to recognise uncertainty, challenge outputs where necessary, and avoid treating AI-generated information as inherently reliable.

This is not fundamentally different from how we should approach any other form of equipment or software validation. We start with confirming that the equipment or software has been installed and configured according to the manufacturer’s specifications. Then we test whether the equipment or software operates correctly within its specified ranges. Finally, we verify that the equipment performs as intended under actual conditions of use, meeting user requirements. The principles of control still apply. The context has simply become more complex.

 

The Risk of Gradual Dependence

Judgment Day Doesn’t Arrive All at Once

Another challenge is that reliance on AI often develops gradually.

At first, AI may be used for small administrative tasks. Over time, organisations begin relying on it for drafting procedures, summarising investigations, preparing reports, analysing data, or supporting technical decisions.

Each individual use may appear low risk. Collectively, however, they can alter how knowledge is developed and retained within the organisation.

There is a difference between using AI as a tool and outsourcing critical thinking to it.

This distinction may become increasingly important over time.

 

Risk-Based Thinking Still Applies

The introduction of AI does not replace existing quality principles. If anything, it reinforces them.

Risk-based thinking remains highly relevant.

This approach aligns closely with ISO/IEC 42001, which applies management system principles to the governance of AI systems. The standard places significant emphasis on areas such as risk assessment, human oversight, transparency, competence, monitoring, and continual improvement.

Many of these concepts will already feel familiar to laboratories and organisations operating under standards such as ISO/IEC 17025, ISO 15189, or ISO/IEC 17020.

Questions labs and organisations may need to consider include:

• What decisions are being influenced by AI?
• How are outputs verified before use?
• What level of human oversight is appropriate?
• What happens if the AI output is incorrect?
• How is competence maintained when tasks become automated?

These are not purely technical questions. They are also system questions.

The same principles used to assess risk in other areas of the lab can still be applied here. The challenge is ensuring that organisations recognise AI as part of the system rather than treating it as something external to it.

 

Trust, Verification, and Control

Quality systems ultimately depend on trust.

Trust in results. Trust in processes. Trust in decisions. Trust that systems are functioning as intended.

AI changes how that trust is established.

In traditional systems, trust was often built through demonstrated competence and repeatable process control. In AI-supported systems, trust may increasingly depend on verification, oversight, transparency, and the ability to critically assess outputs.

Blind trust in AI is no more appropriate than blind trust in any other uncontrolled process.

The principles of review, validation, and independent verification remain essential.

 

The Human Element Still Matters

Despite rapid technological change, quality systems remain fundamentally human systems.

People still define requirements, interpret context, assess uncertainty, make decisions, and determine what level of risk is acceptable.

AI may support these activities, but it does not remove responsibility for them.

In many ways, the age of AI may increase the importance of critical thinking rather than reduce it.

The organisations that adapt most effectively are unlikely to be those that automate everything blindly. They are more likely to be the organisations that integrate AI thoughtfully, maintain strong oversight, and understand where human judgement continues to matter most.

 

Before We Hand the Keys to Skynet

AI is not simply a new tool. It represents a shift in how information is generated, interpreted, and trusted.

For laboratories and quality systems, the challenge is not resisting AI or embracing it uncritically. The challenge is understanding how to integrate it in a way that maintains confidence, control, and accountability.

As organisations increasingly integrate AI into operational and decision-making processes, standards such as ISO/IEC 42001 are likely to become increasingly relevant in demonstrating structured governance and responsible use of AI technologies.

The principles underlying good quality systems have not disappeared.

If anything, they have become more important. Remember:

Trust, But Verify. Even HAL.

 

If your organisation is considering how AI fits within its quality system, MAS Management Systems can assist in applying practical risk-based thinking, governance, and control principles to emerging technologies in a structured and pragmatic way.

 

FAQs

What is AI risk management?

AI risk management is the process of identifying, assessing, and controlling risks associated with the use of artificial intelligence systems within an organisation.

What is ISO/IEC 42001?

ISO/IEC 42001 is an international management system standard that provides a framework for the governance, oversight, and responsible use of artificial intelligence.

Why is AI governance important?

AI governance helps organisations ensure that AI systems are used responsibly, transparently, and in a way that aligns with legal, ethical, and operational requirements.

Can AI create new risks for quality systems?

Yes. AI can introduce risks related to inaccurate outputs, lack of transparency, overreliance on automation, data quality issues, and reduced human oversight.

How does AI affect competence?

As AI becomes more common, competence increasingly includes understanding when AI tools are appropriate, how outputs should be verified, and where human judgement remains essential.

Do laboratories need to validate AI outputs?

Laboratories should verify and review AI-generated outputs before use, particularly where those outputs influence technical decisions, reports, investigations, or quality records.

What is the relationship between ISO 17025 and ISO 42001?

Both standards use management system principles such as risk-based thinking, competence, continual improvement, and governance. ISO 42001 applies these principles specifically to AI systems.