Everybody already agrees that experimentation matters. Move fast, try things, find out what customers actually want faster than the other guy, that part isn’t controversial anymore, especially now, with budgets tighter and everyone getting asked to do more with less. You don’t need me to convince you a faster feedback loop beats a slower one.
What I keep noticing is a level below that. It’s not that the AI companies are experimenting faster than everyone else. It’s what they’re using the experimentation for, and the scale they’re running it at, that nobody’s had access to before.
ChatGPT has more than 900 million weekly active users, by OpenAI’s own count, some reporting puts it past a billion by now, and those users are sending something like 2.5 billion prompts a day. Sit with that second number for a second. A prompt isn’t a click. It isn’t a funnel step or a session length. It’s a person typing, in their own words, the exact task they were trying to get done, unprompted, with no product manager standing over their shoulder asking leading questions. That’s the highest-resolution voice-of-customer data that’s ever existed, and it shows up 2.5 billion times a day.
And it’s not just OpenAI running this playbook. Every serious player in the space, Anthropic, Google, the rest, is doing some version of the same thing: ship it broad, ship it constantly, and treat every single session as a data point instead of an afterthought.
Same story on the developer side. GitHub Copilot has something like 20 million users writing code against it, and GitHub doesn’t have to guess what to build next. They watch what gets kept and what gets thrown out, broken down by language, by editor, refreshed continuously. That’s not a beta program with a hundred people filling out a survey once a quarter. That’s a running structural read on what developers actually want, at a scale no traditional user-research team could ever touch.
The part that’s easy to miss is that none of this is an accident. These companies aren’t stumbling into a data goldmine because they happened to grow fast. They’re built, on purpose, to ship broadly, cheaply, and constantly, specifically because every interaction is itself a data point about what to build next. The product is also the research instrument. That’s a genuinely different posture than shipping a feature and sending out an NPS survey twice a year to see how it landed.
Now, more data doesn’t automatically mean better decisions. I’ve watched teams sit on a genuinely massive pile of usage data and do nothing useful with it, because nobody owned the job of turning “here’s what four hundred million people did this week” into an actual, funded, prioritized decision. Scale without a translation step is just a bigger haystack. The companies actually pulling ahead here aren’t just the ones collecting the most data, they’re the ones who built an org function whose entire job is closing that loop, fast, on a cadence that keeps up with how fast the data is arriving.
That’s the real shift. It’s not “move fast and experiment,” everybody already says that. It’s “build the product so that using it produces a continuous, structured record of what a billion people actually want,” and then build the muscle to act on that record before it goes stale. Most companies have gotten decent at the first half. Almost none of them have built the second.