Like many people, I have been following the growing debate around artificial intelligence and, in particular, the warnings about how rapidly it is becoming embedded in everyday life and business decision-making.

https://youtu.be/i30jVPqQeOM?si=6EtPZyZ6dIeIujk5

The attached video is one of the more thought-provoking examples I have seen recently. It considers what could happen as AI systems become increasingly sophisticated, autonomous and influential. Some of the scenarios discussed sit at the more extreme end of the debate, and none of us can know with certainty where this technology will ultimately take us.

It did, however, make me think about a much more immediate question for the insurance industry.

Are we sufficiently conscious of the point at which AI moves from supporting professional judgement to influencing, or potentially replacing, it?

Insurance has always been a business built around information, risk, evidence and judgement. At underwriting stage, information is gathered, risk assessed, terms applied and a premium calculated. When a claim occurs, another assessment begins. What happened? Does the policy respond? Is the information accurate? Does the available evidence support the account being presented? Are there inconsistencies? Is there an indemnity issue? Does something warrant further investigation?

However sophisticated the technology becomes, these remain important questions and, ultimately, somebody has to take responsibility for the answers.

Processing a claim is not the same as deciding one

There is clearly enormous potential for AI within claims. A sufficiently capable system could review an FNOL, policy wording, underwriting information, photographs, previous claims, correspondence, repair documentation and other available information in a fraction of the time it would take a person. It could compare accounts, identify patterns, highlight inconsistencies and direct a claims professional towards matters requiring closer examination.

That is exactly the type of capability we should be exploring.

But there is an important distinction between a system saying, “There are three aspects of this claim that warrant further consideration,” and a system saying, “This claim should be declined.”

The first assists professional judgement. The second begins to substitute for it.

What happens, for example, when one of the fundamental pieces of information being analysed is wrong? Perhaps a question was misunderstood, an FNOL was poorly recorded, important context is missing, a document is inaccurate or an image has been manipulated. The system may process the information perfectly and still arrive at the wrong conclusion because the information upon which its reasoning depends is flawed.

The risk may therefore not be AI producing an obviously ridiculous answer. The greater risk may be AI producing a highly persuasive wrong answer.

As systems become better, faster and more convincing, there is also a danger that human oversight becomes increasingly passive. A claims professional might technically remain “in the loop”, but if experience teaches them that the machine is usually right, how long before reviewing its recommendation becomes little more than approving it?

An anomaly is not a conclusion

This is particularly relevant to fraud and indemnity. Those involved in claims investigation know that questionable claims rarely arrive with a single definitive piece of evidence proving something is wrong. More often there are fragments: a timeline doesn’t quite work, two accounts conflict, metadata doesn’t correspond, an unexpected relationship emerges, financial circumstances require clarification or previous claims activity raises another question.

Individually, those matters may mean nothing. Collectively, they may justify further enquiries.

AI should become exceptionally good at identifying those connections, particularly across quantities of information that would be extremely difficult for an individual to review manually. But it must not turn an indicator into a conclusion.

An anomaly isn’t fraud. It is a reason to ask another question.

The same applies to indemnity. Identifying something that may engage a policy term and establishing that the circumstances actually justify an indemnity decision are very different things. Context, evidence and proper consideration of the policy wording remain essential.

Finding the right balance

We have had to consider these questions ourselves while developing our RTC service and the SaaS technology we built to support it. We wanted to harness the advantages AI could bring to early claims investigation without allowing the technology to replace the investigation or become the ultimate decision-maker. Via many conversations, this is something insurers are keen to avoid, as are we.

From our perspective, the service allows the policyholder to provide their account, the interview is recorded and the evidence remains capable of human review. Technology assists us with elements of the process, research, and analysis, helping to identify matters that may require attention, while the investigative process provides context, scrutiny and professional judgement. Our current RTC approach combines AI-assisted processes with conventional investigative outputs, including investigation reports and witness signed statements.

Crucially, the output is intended to inform a claims decision, not become the claims decision.

In creating the software to manage the claims, we very quickly established mandatory guardrails. AI is not permitted to access, view or retain client or personal data. That separation was designed into the architecture rather than treated simply as a procedural instruction. This is a must for clients and policyholders to have conficdence in the service. For me, that illustrates an important principle for the wider adoption of AI in insurance. We shouldn’t only be asking what can the technology do? We should also decide, in advance, what should it never be allowed to do?

The evidence is changing as well

There is another side to this. AI isn’t developing solely within insurance companies. Customers, suppliers, representatives and inevitably those seeking to exploit the claims process will have access to increasingly sophisticated technology too. Generated documents, manipulated images and artificially constructed information are likely to become progressively more convincing.

Autonomous vehicles offer another glimpse of the same challenge. As vehicles increasingly make decisions themselves, investigating an accident may eventually require us to establish not simply what the person behind the wheel did, but what the technology did, what information it received and why it responded as it did.

The principle extends well beyond motor insurance. Connected homes, wearable devices and digitally controlled equipment will generate increasing quantities of evidence surrounding insured events. Claims professionals may therefore have more information available than ever before, while simultaneously facing a greater challenge in establishing what can be trusted and what that information actually proves.

Keeping responsibility human

Perhaps the important debate isn’t whether AI will replace claims handlers, investigators or underwriters. A better question is what we should allow AI to do exceptionally well and which decisions should remain meaningfully human.

Let technology process information, identify anomalies, compare accounts, highlight potential policy issues and show us things we might otherwise have missed. Then let an appropriately experienced person ask the next question, challenge the evidence, investigate where necessary and make an informed decision.

Because ultimately an insurer may need to explain not simply what decision was reached, but why and how.

“The system decided” is unlikely to be a particularly satisfactory explanation to a policyholder, regulator, ombudsman or court.

AI will become more capable and claims processes will inevitably become more automated. Used properly, that should make claims handling considerably better. The opportunity is not to choose between technology and human expertise, but to combine the strengths of both while establishing clear boundaries around each.

Perhaps the principle is therefore relatively simple:

Automate the process where it improves it. Use AI to inform, research, analyse and challenge. But never automate away the human responsibility for the decision.

If you have a view on what AIU should or shouldnt do, we love to hear from you.

David Booker CEO

©2026 DLB Investigations Ltd