Why AI Fails on Messy Data (and Why Waiting Won't Fix It)
Most businesses see little ROI from AI and blame messy data. Here's why cleaning it first rarely works, and what successful companies do instead.
Most businesses get little ROI from AI, and messy data usually takes the blame. Cleaning everything up first rarely works. The businesses getting results build systems that handle the routine cleanup, flag what is ambiguous, and send the judgment calls to a person who knows the business.
Every day I talk to businesses of all sizes about how they are approaching AI in their operations. The responses vary dramatically, from small business owners who see it as little more than a search engine to portfolio managers pushing large-scale automation across the board. There are founders anticipating and solving problems that have yet to be created, business owners who reject the idea entirely, the list goes on.
Far and away the most common user I come across looks something like this. They have some AI integrated into their business, usually native in a tool they already use. They have played around with it a little and see some potentially tantalizing uses. Then it falls flat in its ability to consistently deliver reliable outcomes, they become increasingly skeptical of its effectiveness, and they go back to doing their job.
Sound familiar?
Why isn't AI delivering ROI for most businesses?
Most businesses add AI tools without designing them around a specific problem in their operation, so the results are inconsistent and the ROI never shows up.
Every new tool promises cheap, high-volume output somewhere around the level of your best people, yet very few companies are showing consistent ROI, whether in revenue or time saved. We are several years into this experiment and have seen very little of the transformational business design you would expect.
Businesses know they will get left behind if they do not figure it out. They know there must be some way to use this technology, and they can see the proof that it is working for someone. They just do not know how to get from an interesting conversation with a chatbot to something genuinely valuable for them and their employees.
Is messy data the reason AI isn't working?
When I ask what went wrong, the answer usually comes back to the data. And they are not wrong. Take a law firm running mass arbitration. They might be filing claims for thousands of individual clients against the same company, and every one of those clients came in through an intake form, an email, or a phone call. The same person signs up twice under slightly different names. Account numbers are missing or mistyped. The details that decide whether someone even has a valid claim are buried in a text description, a screenshot, or a PDF of a receipt. Ask an AI tool to sort through that and pick out the eligible claims, and it will either guess or get stuck. Neither is acceptable when you are filing legal claims.
Should you clean up your data before using AI?
Cleaning up your data first usually stalls the project, and buying another tool rarely fixes the underlying problem. Businesses tend to go one of two ways after the first attempt falls flat.
Some decide to clean up their data first and come back to AI later. It sounds responsible, but structuring all that information is time consuming, and most businesses do not have the time or resources to do it on their own. Nobody owns the project, it slides down the priority list, and the data keeps getting messier while they wait.
Others decide the answer must be another tool. They keep spinning the wheel, hopeful that adding something else to their tech stack will finally get the magic result. Most AI spend right now is going to new tools. Those tools use models that require tokens, which means more spend. Very little of it produces a return, and the cycle continues. If you want a clearer picture of where the money goes, see what AI actually costs a small business.
Either way, the team decides AI does not work for them and goes back to doing things the old way. All the while, a competitor is using AI to automate their intake and take on more clients than ever. Or they have given their marketing team a fast, adaptable setup that is producing results without adding headcount. Whatever it may be, there is a growing divide between the businesses getting results and those that are not.
What do businesses getting results from AI do differently?
Businesses getting results from AI design the system around a specific problem in their actual operation, mess and all, instead of switching a tool on and hoping for the best.
The businesses on the right side of that divide are not any cleverer than the rest. They did not magically happen upon the right tool. They did not stumble into an opportunistic intern with Claude Code. Their data is not any cleaner. But clearly they are doing something differently, and they have the results to show for it.
If we assume businesses are using mostly the same tools and have access to generally similar resources, what accounts for their widely varying results? The answer is implementation.
AI is not a silver bullet. Somewhere in the middle of all the outlandish claims exists a highly useful tool that is nothing more than the next step in the technological world we have built. Treat it as such and you get measurable results. Expect it to just work with little plan or infrastructure and it becomes useless quickly.
How do you make AI work with messy data?
Good implementation means building for the mess instead of waiting for it to go away. A well-designed system does not need perfect data. It needs clear instructions for what to do when the data is not perfect, and it handles three jobs:
- Clean up what is mechanical, like formatting and obvious duplicates.
- Flag what is ambiguous, like two records that might be the same client or a missing date.
- Route the judgment calls to a person who knows the business.
For the law firm, that means duplicate sign-ups get merged automatically, claims missing an account number get flagged for someone to chase down, and an attorney only reviews the borderline cases instead of every file. The same pattern applies to any firm handling high volumes of documents, which is why we see it so often in professional services. If you are not sure where to start, what to automate first is a good place to look.
Do you need an expert to implement AI?
Yes, in most cases. Access to the tools is the easy part, and almost everyone has it now. Knowing how to build them around your data and your operation is what most businesses are missing.
When QuickBooks arrived, businesses did not all of a sudden stop hiring accountants. Anyone could buy the software, but owning it did not make you good at accounting. Plenty of owners who tried to do it themselves ended up paying an accountant to untangle the mess. The businesses that got the most out of it put it in the hands of someone who knew what they were doing. AI is no different.
My recommendation to all the businesses I talk to is to stop throwing money at each new tool and instead find someone who knows how to use the technology for the specific outcomes they are looking to achieve. Here is why certification matters when choosing an AI consultant. And whoever you bring in, ask them one question first: what happens when the data is wrong? If they have a clear answer, you are talking to the right person.
Frequently asked questions
Can AI work with messy data?
Yes, if the system is designed for it. A well-designed system cleans up mechanical problems like formatting and duplicates, flags ambiguous records, and routes judgment calls to a person who knows the business.
Should I clean up my data before implementing AI?
Waiting for clean data usually stalls the project. Cleanup is time consuming, rarely has a clear owner, and the data keeps getting messier while you wait. It is more effective to build a system that handles imperfect data from the start.
Why do most AI tools fail to deliver ROI?
Most businesses add tools without designing them around a specific problem in their operation. Without a plan for their own data and workflows, the tools produce inconsistent results and teams stop using them.
What should I ask before hiring someone to implement AI?
Ask what happens when the data is wrong. A good answer describes how the system catches bad or ambiguous data and gets it in front of the right person.
Find out where messy data is holding your AI back.
The right fix depends on how your business collects and handles information today. A free assessment will show you where AI can take on the routine work and where a person should stay in the loop.