In 2013, MD Anderson Cancer Center partnered with IBM to build one of the most ambitious clinical AI projects in history. Watson would analyze patient records, cross-reference the latest research, and recommend cancer treatments. The institution committed $62 million. The technology was state of the art. The team was world class.
Four years later the project was shut down. During testing, Watson produced treatment recommendations that oncologists and IBM themselves flagged as not ready for clinical use. The system never treated a single patient. The AI wasn't malfunctioning. It was doing exactly what it was built to do. The problem was that the clinical protocols it learned from weren't standardized across physicians. Different doctors approached the same diagnoses differently, and Watson learned from all of them. Nobody asked whether the process was stable enough to automate before they automated it.
Sixty-two million dollars. Four years. And the question that would have caught it could fit on an index card.
A July 2025 MIT study looked at 300 enterprise AI deployments. Ninety-five percent delivered no measurable financial return. Not disappointing returns. Zero. MD Anderson is not an outlier. It is the pattern. The difference between the five percent who got it right and everyone else wasn't the tool. It was what leaders asked before they bought it.
Six questions. The first three are the baseline. Every leader should be asking them. The second three are the ones almost nobody is asking, and they're where most of the $30 to $40 billion in failed pilots went wrong.
The Core Questions
1. What specific problem are we solving, and is it systematic and repeatable?
IBM Senior Research Scientist Marina Danilevsky described what she kept seeing in enterprise AI rollouts: organizations decided to use AI first, then figured out what for. Step one: we're going to use AI tools. Step two: what should we use them for? That sequence is backwards, and it's expensive.
The discipline is to start with a problem so specific you can describe it in one sentence. Consider literature reviews in medical research. A research team needs to gather every relevant study on a specific clinical question, filtered by date, methodology, patient population, and outcome measure. Right now that work typically gets handed to a medical student because it is time-consuming and follows a predictable pattern. The student spends days on a task that requires almost no clinical judgment, just careful adherence to a search protocol. That is a problem AI solves well. The inputs are consistent, the process is repeatable, and the output is measurable. Every research institution doing this manually today is leaving time and money on the table.
But here's what most leaders skip: the problem has to be systematic and repeatable for the investment math to close. A one-time inefficiency doesn't justify AI infrastructure. The ROI compounds when a problem occurs hundreds of times per week, follows recognizable patterns, and has consistent inputs and outputs. Variability kills returns. If you can't describe the pattern, you don't have a use case yet.
2. Is the underlying process stable and clean enough to automate?
AI doesn't fix broken processes. It accelerates them. Watson is the clearest example of this, but it's not the only one. The pattern repeats across industries: a team identifies an inefficiency, buys a tool to address it, and discovers six months later that the inefficiency wasn't the real problem. The process underneath it was inconsistent, the data going into it was messy, or the people involved weren't following the same steps. AI doesn't know the difference between a clean process and a broken one. It learns from whatever it's given and executes at scale.
Before you automate anything, ask whether everyone involved in the process would describe it the same way. If the answer is no, you have a process problem, not an AI opportunity. Fix the process first. Then automate it.
3. Who owns the outcome, and what is the outcome metric?
Before the tool gets purchased, someone's name needs to be on the outcome. Not the vendor's. Not a steering committee. A specific leader whose performance review six months from now will reflect whether this worked. If you can't answer that question in under five seconds, the accountability structure isn't there yet, and without it the investment will drift.
That leader also needs to define what "working" means in terms of outcomes, not activity. The MIT study found a consistent pattern in failed deployments. Organizations measured activity instead of outcomes, they saw increased logins and AI use and mistook the adoption for optimization. The successful five percent measured business outcomes: time to diagnosis, revenue per rep, customer retention. A late 2024 Gallup poll found that only fifteen percent of U.S. employees report their organization has communicated a clear AI strategy. Without a clear strategy, there is no clear metric. Without a clear metric, nobody knows whether the investment worked until it's too late to fix it.
The Questions Nobody's Asking
4. What is our actual ceiling?
Most leaders never map where the time in their workflow actually goes before buying an AI tool. That gap matters more than most realize. If sixty percent of your workflow is overhead and only forty percent is the actual task you are trying to automate, you cannot get more than a 2.5x improvement no matter how good the AI is. The math doesn't care about the model. It cares about what surrounds it. Jeff Dean, head of Google DeepMind, made this point directly: even a model that ran infinitely fast would only get you a two to three times improvement end to end. Everything else gets absorbed by the systems around it.
Radiology is a clear example of this playing out in real time. Hospitals invested heavily in AI diagnostic tools to speed up image reading. The AI read scans faster than any radiologist. Institutions that measured radiologist reading time saw real improvement and declared success. But institutions that measured time from symptom to diagnosis saw almost none. The bottleneck was never image reading. It was getting images into the system in the right priority order, and getting results back to the ordering physician in a format they could act on quickly. The AI was fast. Everything surrounding it wasn't built for that speed. Leaders bought a solution to the wrong problem because they never mapped where the time actually went.
Before you invest, time a full workflow from trigger to outcome. Map where the hours go. Identify the constraint that will limit your ceiling. Then ask whether the investment addresses that constraint, or optimizes something that was never the bottleneck.
5. Can our infrastructure support this at the speed AI needs, or does it just technically allow it?
There is a difference between a system that can technically be accessed by AI and a system that AI can actually use at the speed it needs to work. Your existing software might have a connection point for AI to plug into, but that connection might move at human speed, designed for someone clicking through a screen, not for a system processing hundreds of requests per minute. Most leaders only think about one side of this. They ask whether AI can connect to their systems, but they don't ask whether the humans in the process can keep up with the AI on the other end. A workflow has two potential bottlenecks: the connection between your people and your systems, and the connection between your AI and your systems. For a process to actually perform, neither one can be the weak link. An AI that processes in seconds is useless if a human still has to manually review and approve every output before anything moves. The ceiling is always set by whichever bottleneck you haven't fixed yet.
Consider two workflows side by side. The first is a weekly or quarterly leadership brief. An AI pulls metrics and updates from multiple systems, compiles them into a single formatted document, and delivers it for review. The AI-to-system connection is fast, the manual data gathering is eliminated, and the only human touchpoint is at the end where judgment is actually needed. That is a workflow AI improves.
The second is the automated abnormal lab alert system used in hospitals. The AI flags an out-of-range result and routes the alert to the ordering physician. It works exactly as designed. But in a teaching hospital during residency rotations, the ordering physician may have already rotated off the service. The alert goes to someone who is no longer responsible for that patient. The AI processed correctly. The human system around it was not built to receive the output. The bottleneck was never the technology. It was the structure wrapped around it, and nobody fixed that before the tool went live.
6. Are we automating a task, a judgment call, or theatre?
Tasks and judgment calls are different categories. They require different implementation strategies, different oversight structures, and different accountability. A task has defined inputs, a predictable process, and a measurable output. Scheduling an interview based on calendar availability is a task. Deciding whether a candidate is the right fit for the team is a judgment call. Automating a task without human oversight is reasonable. Automating a judgment call without human oversight is a liability.
The practical difference is where you place the human in the workflow. For tasks, the human belongs at the design stage, building the rules and reviewing outcomes periodically. For judgment calls, the human belongs inside the process, not reviewing after the fact but making the call in real time with AI as a support tool rather than the decision maker. Before any AI implementation, your team should be able to answer one question clearly: is a human approving this before it has consequences, or after? If the answer is after, and the work involves judgment, that is the design flaw worth fixing before you build anything else.
But there is a third category worth naming: theatre. Work that exists because the organization agreed to perform it, not because it actually produces value when examined. Status reports nobody reads, approval chains that exist because they once solved a problem that's gone, review meetings that produce no decisions. If you automate theatre, you get faster theatre. The ROI is zero because the value was always zero. Before you build the business case for an AI investment, ask an honest question about the process itself: is this real work that AI can improve, or is it a performance the organization has agreed to keep staging?
MD Anderson had the technology, the funding, and one of the best medical teams in the world. What they didn't have was someone asking the right questions before the project started. That's not a technology problem. It's a leadership problem. And it's one you can solve before the next budget meeting.
The organizations getting real AI returns aren't smarter or better resourced. They just stopped long enough to ask.
The Short Version: Ninety-five percent of enterprise AI deployments delivered zero measurable ROI in a July 2025 MIT study. The difference between that majority and the five percent who got it right wasn't the tool. It was six questions asked before the investment was made: whether the problem is systematic and repeatable, whether the process underneath is stable, who owns the outcome and how it gets measured, what the actual ceiling is given their environment, whether their infrastructure can support AI at the speed it needs, and whether they're automating real work or automating theatre.
Try This Now: Use your taxes as a test case. Write down every step involved in completing them, from the moment you decide it is time to start, to the moment you file. Most people discover the list is longer than expected. Now sort each step into one of three categories: is this a repeatable task with a predictable output, is this a judgment call that requires your specific knowledge, or is this friction that exists for no good reason. The actual calculation is a task. AI handles it well. The itemize vs. standard deduction decision is a judgment call that depends on your situation. The two hours spent hunting for a missing form is friction, and no AI investment fixes that until you fix the document organization problem underneath it. Run the same exercise on any process in your organization before you buy a tool to improve it. The goal is not to find reasons AI won't work. It is to find exactly where it will.
_Carlo DelDonno, MD, Army Captain — If this was useful, forward it to one person. That's the whole ask._
Pro Tier: The Questions That Keep Experts Up at Night
Most AI due diligence stops at the six questions above. That's enough to avoid the obvious mistakes. These three are the questions that determine whether an organization is stronger two years after the investment than it was before it.
7. Does this create dependency or compound capability in our people?
There is a meaningful difference between an AI tool that does the work for your people and one that makes your people better at the work. Most leaders don't ask which one they're buying.
The evidence on this is uncomfortable. A clinical study found that endoscopists who regularly used AI for polyp detection became measurably worse at finding polyps when the AI was turned off. Detection rates dropped from 28% to 22%. The AI was performing correctly the entire time. The physicians using it were quietly losing the skill the AI was supplementing.
Aviation documented the same pattern. FAA research spanning more than a decade found that long-haul pilots exposed to high levels of flight automation showed degradation in manual flying ability, situational awareness, and cognitive agility under pressure. The FAA responded by publishing Aviation Circular AC 120/123, explicitly recommending more manual flying to keep expertise alive.
Before you deploy, ask: in two years, will the people using this tool be better at the underlying work, or will they have lost the ability to do it without the tool?
8. Does this AI fit into your current technology stack, or does it require adopting other technology to function?
Research on enterprise AI deployments found that legacy system integration adds 25 to 35 percent to base implementation costs on average. A separate analysis found that for every dollar of direct technology investment, organizations spend up to ten dollars on intangibles: process redesign, retraining, and organizational adaptation. The tool is the smallest part of what you are actually buying.
Before you sign anything, ask the vendor directly: what does this tool need that we don't currently have? If they can't answer that clearly, you have found your first red flag.
9. Is the AI learning from data that already had blind spots?
AI doesn't create bias. It industrializes whatever bias already exists in the data it learned from. A 2021 peer-reviewed study published in Patterns by Cell Press documented this: skin cancer diagnostic algorithms trained predominantly on images from lighter-skinned patients showed significantly lower accuracy when tested on darker-skinned patients. The AI was performing exactly as trained. The training data had a gap that nobody examined before deployment.
This is not a technology problem. It is a data lineage problem, and it requires a human decision before the model is ever built. What population does this data represent? Who is underrepresented? Where did the humans who generated this data have systematic blind spots?
AI is not a finished product. The models available today are the worst they will ever be. Every organization that builds on a strong foundation now will be positioned to compound those advantages as the technology improves.
_Carlo DelDonno, MD, Army Captain_
Sources
1. MD Anderson / IBM Watson — $62 million project cost and shutdown. University of Texas System Audit Report, 2016. Reported by IEEE Spectrum, JNCI (Oxford Academic), and Medscape. 2. MIT Study — 95% of enterprise AI deployments delivered zero measurable ROI. MIT Sloan Management Review, July 2025. Study of 300 enterprise AI deployments. 3. Marina Danilevsky — AI-first vs. problem-first sequencing. IBM Senior Research Scientist. 4. Gallup Poll — 15% of employees report a clear AI strategy. Gallup Workplace Survey, late 2024. 5. Jeff Dean — Amdahl ceiling for AI workflows. Head of Google DeepMind, GTC 2025. 6. Radiology AI — bottleneck shift. Mastrianni M, et al., FDA / arXiv, Nov 2025; Academic Radiology, Oct 2025. 7. Endoscopist deskilling study — Budzyń K, Romańczyk M, Kitala D, et al., The Lancet Gastroenterology and Hepatology, August 2025. 8. Pilot automation and skill degradation. FAA Aviation Circular AC 120/123; Human Factors, Vol. 56, No. 8, Dec 2014. 9. Enterprise AI integration costs — McKinsey Global Institute and Gartner, 2024. 10. AI bias in dermatology — Obermeyer Z, et al., Patterns (Cell Press), 2021. PMC8515002.