Mary Meeker published her first major AI report in five years last May. 340 slides. The kind of document that gets passed around in every boardroom and quoted in every strategy deck for the next twelve months.
One number got most of the attention: 800 million weekly active ChatGPT users. Fastest technology adoption in history. AI is everywhere.
That number is real. It just doesn't tell you what you need to know.
Adoption and value are two different things. Right now most organizations are tracking one and assuming it means the other.
- 95% of gen AI pilots fail to move the needle on profitability (MIT NANDA Initiative, 2025) - 42% of companies abandoned the majority of AI initiatives in 2025 (S&P Global, 2025) - 74% of companies struggle to achieve and scale AI value (BCG, 2024)
Those three numbers sit alongside Meeker's 800 million users figure. Same year. Same industry. Meeker's report doesn't explain the gap, but it's worth understanding because it tends to show up in two ways.
The first is that people aren't actually using the tools. The rep pastes notes into ChatGPT once a week so the usage report looks green. The engineer quietly turned off Copilot after the third bad output. The dashboard shows adoption while the work hasn't changed at all.
The second is harder to catch because everyone involved is genuinely trying. The tool gets deployed. People use it. And it still doesn't work, because AI that performs well in a controlled demo tends to struggle when it meets the actual environment it's supposed to run in. Legacy systems, inconsistent data, processes that were built long before any of this existed. Meeker's report actually names this directly: models are getting smarter, but the production environments most organizations are dropping them into look nothing like the benchmark conditions they were built for.
IBM Watson is the most expensive example of that second problem on record.
Cautionary Case: IBM Watson Health — $4 Billion, Sold for Parts
After Watson beat two Jeopardy champions in 2011, IBM announced it would become an AI doctor. Commercial products within 18 to 24 months. A partnership with Memorial Sloan Kettering. A promise that cancer treatment was about to change.
Watson was trained on data from elite New York hospitals and then deployed to treat patients in completely different populations around the world. The data didn't transfer. A 2016 audit found that MD Anderson Cancer Center had spent $62 million on the project before canceling it. By 2018, more than a dozen IBM partners had quietly walked away from their oncology projects.
IBM sold Watson Health in 2022 for roughly what it had spent on acquisitions alone. Over $5 billion in, out for parts.
The INSEAD case study on the collapse identified three contributing factors: societal expectations outran the actual technology, the salesforce oversold what the product could do, and the doctors who were supposed to use it day-to-day were never genuinely involved in designing it. The tool didn't fit how they worked, so it didn't get used. One analyst at Damo Consulting described it plainly: "IBM suffered from its own self-inflicted wounds, letting the marketing get ahead of itself."
Success Case: Johnson & Johnson — 900 Experiments, Then the Hard Question
J&J spent three years encouraging wide AI experimentation across the company. By the end of that period they had nearly 900 use cases running. A lot of activity and a lot of green dashboards.
Then their CIO Jim Swanson ran the numbers and found that 10% to 15% of those use cases were delivering 80% of the value. Swanson told the Wall Street Journal the broad early approach was a necessary maturation process. But he didn't keep defending the portfolio because it looked good on paper. He cut it. Governance moved from a centralized board to the business units closest to the actual problems.
The results got specific. A 2.6-fold increase in clinical trial enrollment at AI-recommended sites. A supply chain that rerouted chemotherapy shipments during a 2024 hurricane before a shortage could develop. An internal talent platform that increased employee placements by 8% year over year. A quality management system that reduced data delivery lag from 24 hours to under 10 minutes.
J&J measured AI success by revenue impact, cost reduction, and patient outcomes rather than by adoption rates or system performance. That shift in how they kept score is what separated their results from the industry average.
Three things worth carrying out of this
1. Measure outcomes, not activity. Logins and usage rates tell you how many people opened the tool. They don't tell you whether anything changed because of it. J&J's CIO wasn't asking whether people were using the tools. He was asking what the tools were actually producing.
2. The people doing the work have to be part of building the solution. IBM's doctors were handed a tool that didn't fit how they worked. When something doesn't fit your actual workflow, you find ways around it rather than through it. J&J eventually moved ownership to the teams closest to the problems, and that's when the results showed up.
3. Experimentation is a phase, not a permanent state. J&J's CIO framed the early broad approach as a necessary maturation process, and it probably was. But maturation implies an end point. If your organization has been running broad AI pilots for two or three years without seriously culling what isn't working, that's worth examining honestly.
What This Means on Tuesday
You don't need 900 use cases to have this problem. You might have three, or one tool your team installed six months ago that nobody mentions anymore.
BCG's 2024 State of AI research found that organizations actually extracting value made twice the investment in people alongside the technology and focused AI on core business processes first, not on the peripheral things that are easy to demo but far from the actual work.
The dashboard tells you what's deployed. Finding out what's real is a different job, and it requires someone willing to ask the uncomfortable version of the question.
The Short Version: Mary Meeker's report shows 800 million weekly AI users. Separate research shows 95% of pilots aren't delivering real results. IBM spent over $5 billion on Watson Health and sold it for parts when the tool never fit the environment it was deployed into. J&J ran 900 experiments, found that 15% were delivering 80% of the value, and cut the rest. The organizations winning right now aren't the ones with the most tools deployed. They're the ones willing to ask what's actually working.
Try This Now: Pick one AI tool your team is currently using. Ask three people on your team separately to describe the last time it actually changed how they made a decision or saved them meaningful time. If you get three different answers, or silence, you're measuring activity rather than impact. That's your starting point. I did this with my own workflow recently and found two tools I hadn't opened in three weeks. The dashboard said otherwise.
Next week we get into what Meeker's data says about tacit knowledge — the things AI genuinely can't touch — and why that matters more for your career right now than most people realize.
Carlo MD, CPT — Still figuring it out, still sharing what works.
_If this was useful, forward it to one person. That's the whole ask._
Sources
1. Mary Meeker, _Trends: Artificial Intelligence_, Bond Capital, May 2025. bondcap.com/reports/tai 2. MIT NANDA Initiative, 2025. Cited in Baytech Consulting, "The Great AI Pullback," September 2025. 3. S&P Global, 2025. Cited in Medium / Simple AI, "The AI Implementation Paradox," June 2025. 4. BCG, "AI Adoption in 2024: 74% of Companies Struggle to Achieve and Scale Value," October 2024. 5. Tona et al., "The Deployment of AI to Infer Employee Skills: Insights From Johnson & Johnson's Digital-First Workforce Initiative," _Information Systems Journal_, Wiley, April 2025. 6. ODSC Team, "Johnson & Johnson Finds 15% of AI Projects Drive 80% of Value," Open Data Science, April 2025. 7. Chief AI Officer, "How Johnson & Johnson Generated $500 Million From 900 AI Projects," January 2026. 8. Klover.ai, "Johnson & Johnson's AI Strategy," July 2025. 9. J&J MedTech, "4 Ways Tech and AI Are Transforming the MedTech Innovation Product Lifecycle," 2025. 10. Stat News / Casey Ross, "How IBM's Watson Went From the Future of Health Care to Sold Off for Parts," January 2022. 11. Stat News, "4 Lessons from IBM's Failure to Transform Medicine With Watson Health," March 2021. 12. INSEAD Publishing, "Challenges in Commercial Deployment of AI: Insights from the Rise and Fall of IBM Watson." 13. Paddy Padmanabhan / Damo Consulting, cited in Stat News, March 2021. 14. Wall Street Journal, Jim Swanson interview, cited in ODSC / PYMNTS, April 2025.