Why every business needs to run experiments - and how to go about doing it

YOU’RE CHAIRING A meeting with your team. A new product you have recently launched is not getting any traction.

You and your team are discussing options and there’s a lot of disagreement about what to do. Eventually, you all agree that the price needs to be discounted.

Next decision: how much of a discount? One person suggests 10%, while another disagrees – arguing a ’2 for 1′ offer would be much more effective.

There is a good case for both options and, at this point, you’re not sure how to choose the most effective option to boost sales.

In our experience, discussions of this nature are common to all SMEs and larger businesses too. And the ultimate decisions are often based on intuition or the convincing argument of a particularly persuasive manager.

This creates a real problem. If SMEs don’t experiment and scientifically compare options, they can never know for sure which of these options, if any, is actually effective in increasing sales.

If the product is simply discounted and sales increase, without experimentation methods it will be impossible to know whether pricing was the driver or if there was some other factor – like a change in the weather for, say, a garden centre.

If your sales are now being driven by a spell of clear skies and not the discount, you’re reducing profit margins for no reason. So, what should businesses do instead?

You should carry out simple experiments to test whether changes to existing products, processes, or prices produce the desired changes in sales, customer satisfaction, or attendance.

There are five main steps in running a business experiment:

1. Define the problem you wish to tackle

For example, customers aren’t responding to your most recent email promoting a new product, which is on a promotion.

2. Identify possible solutions

These could be something as simple as: putting a time limit on the promotion; changing the timing of your email; changing the channel you promote through; using your customers’ own names at the start of the email.

Because you don’t know which of these will work best, you should run an experiment.

3. Segment the customer base

Ideally this would be done in a random way across your entire customer base to have one ‘control group’ who experience no change in their interactions with your business and another group who receive the new form of communication.

By assigning customers to these two groups in a random way, it reduces the likelihood that other factors, such as the customer’s age, address or level of income, being the cause of a boost in responses, rather than the change in the communication itself.

4. Implement and measure

Send out your original communication to your control group and the new communication with the time limit included.

Tools such as Mailchimp are really useful for this, as you can measure the response rates from your communications and compare the impact of each communication.

Through this you can determine if people responded differently to the new communication and measure the impact it had on responses to your email promotion.

5. Refine and scale up

Now you’ve found out what works, you can roll it out across your company and keep on refining these communications to continue optimising each message with your customers.

What are the benefits of running experiments?

There are three main benefits of running even quite simple experiments:

1. Learning what works

Ever had a great idea that hasn’t turned out as you planned? With experimentation, your business can understand what is working and why.

You may expect a new training programme for sales staff to boost sales, but without experimenting you may never find out if it was the training programme that lead to the boost in sales or a roll out of new price promotions.

2. Saving money and resources

Rather than throwing good money after bad, businesses can target and test out products and services before rolling them out. It can also prevent the rollout of products that backfire.

3. Encouraging innovation and improving procedural fairness

Experimentation provides a democratic way of determining what a business should do next. In the example at the start of this article, managers couldn’t agree on the best way to proceed.

It allows both ideas to be trialled and (usually) provides a definitive answer as to what should be done, improving fairness and encouraging innovation as multiple parties can have their say.

I have never run an experiment before, where should I start?

Businesses running their first experiment should choose a ‘quick win’ project to generate support for experimentation within the business and learn experimental methods.

These experiments usually consist of making small changes to existing operations or communications. For example, a business might experiment with changing the content of an email subject line, or indeed an email itself, on click-through and purchasing rates.

Mailchimp provides a very simple platform for businesses to get comfortable with experimentation, allowing you to send alternative versions of emails to different groups of customers and even handles the randomisation element for you.

While there are significant differences between running an email experiment and a more considered experiment, for example, on pricing, it can be useful for businesses to start small and understand the underlying process.

Donal Cronin is a director at Carr Communications, which recently launched a behavioural economics and sciences unit.

If you want to share your opinion, advice or story, email opinion@fora.ie.