It’s becoming a common practice within organisations for any new programmes to be tested before a broader rollout.
Across industries, from retailers to restaurants to hotels and more, companies will choose a few markets or customers that get to try out a new concept while the company tracks programme profitability and customer satisfaction.
This phenomenon of testing allows organisations to not only react quickly to consumer demands, but also allows them to do so in a low risk manner.
Traditionally confined to industries like manufacturing and software development, where testing primarily implies quality assurance, closed-loop testing is now making an appearance in consumer-facing industries.
In this context, closed-loop testing means that after an idea has been tested in-market, it can be rapidly analysed and insights about where the idea worked best should inform a targeted rollout (e.g. only rolling the programme out to customers who responded profitably).
These learnings should then be used to generate additional testable hypotheses, allowing a company to perfect a programme over time through testing. This is the future of testing: a continuous cycle of hypothesis generation, testing, and optimisation.
The closed-loop model works particularly well for certain types of activities – similar, repeatable and high volume tests that are often executed at the store or customer-level (for example, daily email promotions, coupons and direct mail).
For example, suppose an apparel retailer that frequently runs promotions sends out a weekly email advertising a ‘20 € Off Denim’ offer to 10% of their customers as a test. The data about which customers received the email is automatically loaded into the company’s test analysis software.
Soon thereafter, the software quickly analyses the campaign to understand whether the promotion drove incremental sales, and which areas or individual customers responded best. The software can then model exactly which customers are predicted to generate incremental margin, and then the retailer can deliver the offer to the remaining subset of customers who will respond profitably.
Don’t just analyse
An interesting benefit of closed-loop testing is that once enough tests have been run, the results can be leveraged to conduct “meta-analysis” across campaigns.
Simply put, meta-analysis is a method of gaining broader insights by analysing results across independent tests. With so many points of contact between organisations and consumers nowadays, it is crucial that organisations understand how to optimise each one, in order to have the greatest effect.
The same goes for promotional campaigns – for instance, when conducting analysis across all promotional campaigns run in the past year, an apparel retailer might find that younger customers respond best to “BOGO” offers, and customers in suburban areas respond best to sales promoting multiple items.