Author - Rich Chapman
Date - 17/02/2026

Its a hard truth that has happened to my clients or the business I was working in.

These days most CRO and Experimentation practitioners are very clued up and experienced in the discipline. We do not need "here is how to write a hypothesis" or "data grounded experimentation works best" rhetoric anymore, we know. So if we're doing all these things correctly, why do experimentation programmes keep stalling or failing completely if the people running the programmes know how do it, and how to do it well? I have seen talented well set up teams with experienced UX designers, web developers, marketing department and external agencies all on point working together, fail. Why?

There is a simple answer on the surface (which obviously is more complex), people. I have spoken about experimentation culture for years now and how it's needed to be embedded into an organisation for CRO and Experimentation to succeed.

But this means changing mindsets of stakeholders and crucially, leadership. Modern day organisations and leadership are data led. Decision making culture has become "I need the data to make a decision", which translates to "I want proof before i commit". This has lowered risk in decision making significantly, a good thing. So if experimentation can prove outright if something works or does not... why the problem? It is wrapped up in the same line of thinking and emotional response as, "if I spend money, I have to generate a return". Common business practice since commerce was a thing is to measure profit from any investment laid out by a business, ROI. No one is going to sign budget expense off without some steer on what the return could be, right? Their emotional response would likely be negative and apprehension. So therein lies the problem.

‍ROI-Led vs Learning-Led Leadership.

There are key differences between traditional business models and an organisation with experimentation at its core, two contrasting styles, one fits with experimentation and one does not. So we have to ask, are our leaders open to learning? More importantly are they open to failing? If you ask any business director or shareholder if they are open to failure, the response will very likely be no.

‍ROI-led leadership style:

Asks “What revenue will this test deliver?"
Kills ideas that don’t promise immediate uplift
Stops programmes after 2–3 inconclusive tests - This is a waste of money and resources
Places pressure on single tests to win
Cuts budget during early pressure
Employs one person to be responsible for CRO, without the necessary authority to make change

Test-and-learn leadership style:‍

Asks “What revenue will this test deliver?"
Kills ideas that don’t promise immediate uplift
Stops programmes after 2–3 inconclusive tests - This is a waste of money and resources
Places pressure on single test to win
Cuts budget during early pressure
Employs one person to be responsible for CRO also without the necessary authority to make change

The irony? The learning-led approach drives far greater long-term ROI and loss aversion.

The Hard Truth

With ROI and profit the constant driver and thought process, You cannot outsource experimentation culture or values.

• You can hire agencies (like us :D)
• You can buy tools
• You can recruit talent

But if the C-suite doesn’t believe in test-and-learn as an operating philosophy, the programme will stall. I am not blaming senior leaders here, I know I may sound like that, some people call it HiPPO bashing*. It is simply down to a habitual business led, short term/quick win profit mindset which has served them very well, not mixing well with experimentation thought leadership. It is unintentional, it is our job now to change the narrative if we want experimentation to succeed, culture is set in the boardroom and not at team operating level.

Link experimentation to strategy KPIs and not ROI/quick wins. Measure differently using learning, test velocity. Protect the programme during losing streaks and report on loss aversion(they will happen, its normal). Get people on board, tell them how running an experiment and help their job and their KPIs, this is the best way to get buyin across a business.

A one liner to take away with you - Curiosity must come from the top.

*No HiPPOs were harmed in the writing of this blog.