Is a CRO retainer worth it for a AU$2M to AU$20M store? A decision table by revenue, traffic and developer capacity, with the sessions each size needs to test.
A CRO retainer is worth it for a store doing about AU$2M to AU$20M a year online when two things hold: the margin from a realistic lift beats the fee, and nobody in-house has the time to research, build and measure fixes every month. A one-off audit is the better buy when you have a developer with spare capacity, a store under about AU$1.5M a year, or a rebuild coming. The numbers that decide it are your revenue, your sessions and your developer hours.
Most Australian stores in that range cannot A/B test small changes in a useful time. A retainer for them has to do something other than run tests: ship fixes the research makes clear, measure each one against a baseline, and keep finding the next one. This guide gives the decision table, the traffic arithmetic and the evidence for ongoing work over a one-off report.
What does each option buy?
An audit buys a diagnosis, a retainer buys diagnosis plus monthly building and measuring, and an in-house hire buys one person's time. The price ranges below are from Australian providers' own pages, read on 20 November 2026.
| Option | Published price | What you get | What you still need |
|---|---|---|---|
| One-off audit | AU$699 to AU$14,500 once | Research and a ranked fix list | A developer to build it, and someone to measure each fix |
| Audit, then build by the hour | Audit plus about AU$140 to AU$185 an hour | The list, built in scoped pieces | Someone to decide the order and measure the results |
| Monthly retainer | AU$1,100 to AU$18,000 a month | Ongoing research and fixes; A/B tests where traffic allows | Time to approve changes |
| In-house specialist | About AU$80,000 a year plus super | One person full time | A developer, and a testing tool if you test |
Sources and the full tables are in our CRO cost guide. The cheapest retainers are marketing retainers with CRO as one inclusion, so compare the hours that reach your store.
Which one fits your store?
Your revenue sets whether a retainer can pay for itself, your traffic sets whether changes can be tested, and your developer capacity sets whether an audit's list will get built. Read across the row that fits.
| Store | Traffic | In-house developer time | Buy | Why |
|---|---|---|---|---|
| Under AU$1.5M a year | Under about 52,000 sessions a month | Any | A do-it-yourself first pass or a AU$699 to AU$1,500 audit | A AU$60,000 retainer needs more than a 10% lift to break even |
| AU$1.5M to AU$5M | 52,000 to 175,000 sessions a month | Spare capacity most weeks | An audit, built by your developer | You can ship the list; retest in six months |
| AU$1.5M to AU$5M | 52,000 to 175,000 | None spare | A retainer that ships fixes and measures each one | Break-even lift 3% to 10%; tests only on large changes |
| AU$5M to AU$20M | 175,000 to 700,000 | A team with a backlog | A retainer that ships, plus testing on high-traffic pages | Break-even lift 0.75% to 3%; 10% changes testable in a month site-wide |
| AU$5M to AU$20M | 175,000 to 700,000 | A team with spare capacity | An audit, or a testing programme your team builds for | You have the builders; buy the research and test design |
| Any size, rebuild within six months | Any | Any | Put the money into the rebuild's plan | Fixes to the old store are thrown away at launch |
| Any size, one known leak | Any | Any | A project for that leak | A retainer is overhead for a single fix |
Traffic bands assume a AU$120 average order and a 2% conversion rate; replace them with your own. Break-even assumes a AU$60,000 year and a 40% gross margin. Brands paying AU$10,000 to AU$15,000 a month need two to three times the lift shown: a AU$10M store at AU$180,000 a year needs 4.5%.
Other providers' pages say much the same about the order of things. Knight Studios' answer to "Do I need an audit or ongoing CRO?" is "Start with the audit. Ongoing CRO is worth it once you have the traffic to keep testing." Cart Forensics says an audit pays off fastest when there is traffic to convert "and someone ready to ship the changes", and advises stores about to rebuild to put the effort into the new build. A rebuild can be the right fix: The Flower Room moved off BigCommerce in July 2026 and online orders rose 94% (22 September to 19 November 2026 against the same dates in 2025, including a new pickup channel) (case study).
Can your store A/B test, or should a retainer ship and measure?
Below about AU$5M a year, most stores cannot A/B test small changes in a useful time, so a retainer should ship clear fixes and measure them against a baseline. The table uses the standard two-proportion sample size at 95% confidence and 80% power, the same formula and figures as our CRO cost guide.
| Annual online revenue | Sessions a month | Orders a month | Smallest lift a 4-week test can detect, site-wide | Same, on a page that sees a quarter of sessions | Months to detect a 10% lift, site-wide |
|---|---|---|---|---|---|
| AU$1M | 34,722 | 694 | 23% | 49% | 4.6 |
| AU$2M | 69,444 | 1,389 | 16% | 34% | 2.3 |
| AU$5M | 173,611 | 3,472 | 10% | 21% | 0.9 |
| AU$10M | 347,222 | 6,944 | 7% | 14% | 0.5 |
| AU$20M | 694,444 | 13,889 | 5% | 10% | 0.2 |
Assumptions: AU$120 average order, 2% conversion rate, every session in the test split evenly between two versions. The quarter-of-sessions column keeps the 2% rate for simplicity; a product-page test has a higher baseline and needs somewhat fewer sessions, a checkout test far fewer sessions but a narrower audience. The sessions column for AU$2M works out as AU$2,000,000 ÷ 12 ÷ AU$120 ÷ 2% = 69,444.
A detectable lift of 16% is large. A 5% change needs about 630,000 sessions, which takes a AU$2M store about nine months to confirm by test. Optimizely's own definition of a real experiment needs at least 1,000 visitors per version, which any of these stores can reach; reaching an answer you can trust takes far more.
So below the testing line, a retainer should:
- Fix what is broken without testing it: a button that does nothing, a script error at checkout, a delivery cost hidden until the last step.
- Ship changes the research supports and measure revenue per visitor against the same weeks before the change and the same weeks a year earlier, to separate the fix from the season.
- Save A/B tests for large changes on high-traffic pages, sized before the test starts.
- Report each week what shipped and what the number did.
Our guide to A/B testing on Shopify covers the tools and how to size a test.
Why does ongoing CRO beat a one-off audit?
Ongoing work beats a one-off report because most ideas fail, gains fade, and the store keeps changing after the audit is delivered. An audit's list is a set of bets taken at one moment. A retainer keeps placing bets, drops the losers and builds on the winners.
Most ideas lose
Optimizely's analysis of more than 127,000 experiments run on its platform from 2018 to 2023 found that about 12% won on their primary metric and 88% did not. Optimizely values an experiment as win rate times uplift: a 10% chance of a 10% lift is worth a 1% expected impact. A one-off list of ten uncertain ideas, built once, can expect about one to work. A retainer that ships ten a quarter for a year gets four times the attempts.
That 12% applies to ideas uncertain enough to test. A broken button or a checkout error is a near-certain fix, which is why the first month of a retainer usually ships those before anything else.
Gains fade
The same Optimizely data found that most experiment uplifts decay to 80% of their first-month value after a year, with revenue holding 91%. Its report adds that improving performance "requires continually changing the system by which you research, ideate, and develop experiments". A one-off audit has no one to notice the fade.
Learning builds on itself
Koning, Hasan and Chatterji tracked A/B testing adoption across a large sample of high-technology start-ups and found that among firms that adopted it, performance improved by 30% to 100% after a year of use (Management Science, 2022). They attribute the effect to firms using testing beyond one-off incremental changes, as "part of a broader strategy of experimentation". Those firms launched more new products, scaled promising ideas and dropped bad ones faster. The study covers start-ups and measures performance mostly through web traffic, not store revenue, so treat it as evidence for the method; it does not forecast what your store will gain.
The store keeps changing
An audit describes the store on the day it was done. New apps, theme updates, a new range, a sale or a new ad channel change what shoppers see after it is delivered. Microsoft Clarity keeps recordings for 30 days, so the evidence behind an audit is gone a month later unless someone keeps reading it. Contentsquare's 2026 benchmark found conversion rates fell 5.1% year on year across its 6,500 sites, so a conversion rate fixed once does not stay fixed.
When is a one-off audit the better buy?
An audit is the better buy when you can build its list yourself, when the store is too small for a retainer to pay, or when a rebuild is coming. It is also the right first step before most retainers, because it shows whether the store has enough fixable leaks to justify monthly fees.
An audit works when four things are true:
- A developer, in-house or on call, has time to build the list within a quarter.
- Someone owns the result and will measure each fix against the weeks before it.
- The store is not about to change platform or theme.
- You will come back to it: book a second audit six to twelve months later.
If two of those are missing, the audit's list tends to stall, and the fee bought a document. What to look for in the audit itself is in our guide to CRO audits.
What should a CRO retainer deliver each month?
A retainer should deliver shipped changes and a measured number every month. Before you sign, ask for these in writing:
| What | When | How you check it |
|---|---|---|
| A baseline number, usually revenue per visitor | Before any change ships | Your last 30 days, in your own analytics |
| The first shipped fix | Within the first week or two | The change is live in your theme |
| A written log of what shipped | Weekly | Each entry dated, with the page and the reason |
| The number against the baseline | Monthly | Same weeks before the change, and a year earlier |
| Tests, where the traffic allows | When a change is large enough to detect | Sample size calculated before launch |
| Exit terms | At signing | Notice period, and what stays in your store |
How to compare providers on these is in our guide to choosing a CRO agency.
How do you lower the risk of a retainer?
Lower the risk by starting with a credited audit, keeping the first term short, writing the number down and reading the guarantee's conditions. Each removes a way the fee can be spent without a result.
Our own terms are one example. The paid audit costs AU$2,500 to AU$4,000 plus GST; start the retainer within 90 days and the full fee comes off the first month, and if you don't, you keep the audit and owe nothing further. The retainer starts from AU$5,000 a month, with a first term of 90 days, then month to month with 24 hours' notice. Before we start, we agree one number measured against your last 30 days; if we haven't hit it at 90 days, we work until we hit it, for free. The conditions are on the guarantee page.
We say no to stores with fewer than 20,000 visits a month or under AU$1.5M a year online, because changes cannot be measured fairly at that size. For them, the audit or the one-hour check is the right spend.
Sources
- Optimizely: Evolution of Experimentation (2023 ebook, 127,000+ experiments, 2018 to 2023) - about 12% win on the primary metric; uplifts decay to 80% after a year, revenue 91%; expected impact = win rate × uplift; true experiments need 1,000+ visitors per variation
- Koning, Hasan and Chatterji, Experimentation and Start-up Performance: Evidence from A/B Testing, Management Science, 2022 - performance improves 30% to 100% after a year of use
- Microsoft Clarity FAQ - data retained 30 days
- Contentsquare 2026 Digital Experience Benchmark (99 billion sessions, 6,500+ sites) - conversion rates down 5.1% year on year
- Provider prices and statements, each read on the provider's own page on 20 November 2026 and named without links: Knight Studios (CRO), Cart Forensics (CRO audit), and the 16 providers in our CRO cost guide
- Sample sizes: two-proportion formula at 95% confidence (two-sided) and 80% power; break-even lift = yearly fee ÷ (yearly online revenue × gross margin)
- The Flower Room case study - online orders +94%, 22 September to 19 November 2026 against the same dates in 2025, after leaving BigCommerce
Questions
Is a CRO retainer worth it?
A CRO retainer is worth it when the extra gross margin from the lift beats the fee and nobody in-house has time to ship fixes every month. On a AU$60,000 year at a 40% gross margin, a AU$2M store needs a 7.5% relative lift to break even and a AU$5M store 3%. Below about AU$1.5M a year online the lift needed passes 10%, and a one-off audit or a do-it-yourself first pass is usually the better buy.
What is the difference between a CRO audit and a CRO retainer?
A CRO audit is a one-off diagnosis: research into your store's data that ends in a ranked list of fixes, usually built by your own developer. A CRO retainer is monthly work that keeps researching, ships fixes, runs tests where the traffic allows and measures each change against a baseline. Australian audits publish at AU$699 to AU$14,500 once; retainers at AU$1,100 to AU$18,000 a month.
Can a small online store run A/B tests?
A store doing AU$2M a year, at a AU$120 average order and a 2% conversion rate, has about 69,000 sessions a month. In a four-week test across the whole site it can detect a lift of about 16% or more at 95% confidence and 80% power; a test on a page that sees a quarter of those sessions needs a lift of about 34%. Small changes cannot be tested at that size, so they are shipped and measured against a baseline instead.
Why do CRO gains need ongoing work?
Most ideas do not win: Optimizely found about 12% of 127,000+ experiments won on their primary metric (2018 to 2023), so results come from many attempts. Gains also fade, with most experiment uplifts decaying to 80% of their first-month value after a year in the same data. Koning, Hasan and Chatterji found start-ups that adopted A/B testing improved performance 30% to 100% after a year of use, which they put down to learning that builds over time.
Should I get a CRO audit before a retainer?
Yes, if the provider credits it. An audit tells you whether the store has enough fixable leaks to justify a retainer, and sets the baseline the retainer is measured on. Studio Slate's audit costs AU$2,500 to AU$4,000 and is credited in full to the first month if you start the retainer within 90 days; if you don't, you keep the audit.



