The Risks of Skipping Alienation Testing

Evolving with consumer tastes is challenging. The market demands change, and consumers crave novelty. But with every product update, companies risk alienating long-time devotees.

Take the example of Oreo. This century-old brand is adored by kids (and big kids) around the world, but by the 1990’s, Nabisco saw the world changing and decided they needed to change, too. 

Their original filling recipe (or as they officially call it, “stuf”) used pig lard. But that meant millions of people around the world who don’t eat pig or animal products couldn’t enjoy the cookie that so many others loved. Plus, consumers of the 1990's were not the consumers of the 1910's, and there was a risk that even Oreo fans might begin to see pig lard as a less-than-ideal ingredient.

So in 1997, they debuted a new and improved Oreo to the world–one that is completely vegan! Now, millions more consumers can enjoy the cookie–and those that already loved it, love it even more.

Image via foodbeast.com

This is where alienation testing proves its worth. Change is necessary, but so is understanding how your updated product performs with real consumers. Whether it is a package redesign or formula update, even small changes can ignite a firestorm. Alienation testing is your best bet for achieving confidence in updates to a winning product’s continued viability and mitigating the loss of millions of dollars in product development or advertising.

What is "alienation testing"?

Alienation testing is a research method designed to assess the impact that potential product changes may have on your existing consumer base. Companies can understand potential risks and benefits by estimating the number of customers who might discontinue their usage or purchases due to the proposed changes.

Who is alienation testing for?

Whether you’re new to the shelf or a decades-old product, alienation testing should be a key part of your market research. Highlight Customer Success Manager Elizabeth Hogan explains that “alienation testing is particularly important for brands with hero and adjacent products that have loyal and repeat customer bases. It's imperative to ensure that even the slightest improvements, adjustments, or even tweaks to the formula or packaging will not create risks to the customer base.”

"It's imperative to ensure that even the slightest improvements, adjustments, or even tweaks to the formula or packaging will not create risks to the customer base."

- Elizabeth Hogan, Highlight Customer Success Manager

The need for alienation testing extends far beyond keeping pace with consumer preferences. “Adjustments may be rooted in changes in availability or development of raw materials, redistribution of human capital, or even disruptions to the existing supply chain,” explained Elizabeth, “but in the end they all result in changes to the core product, and brands are wise to consider alienation testing a core step in rolling out such changes.”

Identify new formulas your customers crave

Two types of alienation testing: monadic testing versus sequential monadic testing

There are two kinds of alienation testing:

In “monadic” testing, participants only receive one product to provide feedback on: either the “control” or the “test” (i.e., the new version).

In “sequential monadic”, or comparison testing, however, participants receive both the "control" and "test" products, and are tasked with testing both versions in an intentionally randomized order.

An example of sequential monadic testing a brand might use to gauge the risk of alienation with a new formula

When it comes to alienation testing specifically, Highlight recommends sequential monadic testing. That’s because with a sequential monadic approach to alienation testing, you’re able to get both the control and the test in the hands of your testers, quickly and efficiently to determine preference between two products. And we know that speedy access to the data you need for innovation, retailer conversations, investor pitches and beyond is crucial.


An added benefit of using Highlight’s owned community of testers (who we call “Highlighters”) is that brands have the ability to hypertarget existing users of your current product. If you want to get as granular as, say, consumers of your coffee who drink non-dairy milk three times a week with half a teaspoon of sugar–you can do so!

Case study: How a leading American food company used alienation testing to successfully update their iconic children’s lunch offering

Parents everywhere are becoming more conscious of their children’s sugar intake. In response to these evolving consumer preferences, a leading American food company set out to update their beloved children’s lunch product with healthier drinks formulated with the plant-based zero calorie sweetener stevia. 

For their insights team, it was crucial to understand not just the reaction to and satisfaction with the new beverage formula compared to the original, but also how two flavors of the new version performed against each other.

By targeting kids aged six to ten who already use the lunch kit once a month within the Highlighter community, they were able to get the data they needed to determine the viability of the new formula, and which flavor of the formula to go to market with.

How to create a questionnaire for alienation testing

Composing a good survey is not as easy as it might seem–but it is key to getting valid results.

Take, for example, a single questionnaire being implemented across multiple international cultures. The answers to the same seemingly simple question, “How would you rate this product?” become much harder to parse through. Respondents based in the US, for example, may tend to reply much more positively than respondents based in, for example, France–but both may be just as likely to purchase that product.

This particular example points to the importance of incorporating demographic and psychographic questions into your survey or filtering for your preferred traits and behaviors before implementing your questionnaire. (When using Highlight's owned community of in-home testers, you can filter based on 25 different demographic and psychographic categories.)

But that’s just one example of a thousand and one ways a survey can fail to serve your purpose–or worse, give you faulty data. Start with these best practices for questionnaire design to avoid the most common pitfalls.

Five best practices for survey and questionnaire design

1. Ask only the questions that you really need answered. As anyone who has had to fill out multiple forms at the doctor’s office or file their taxes can tell you, paperwork fatigue is real. The same applies to your survey respondents. Be as succinct as you can to safeguard the quality of your data.
   
   
   
2. Make sure your multiple choice options do not overlap and include all possible outcomes. For example, if you ask “How many times a month do you purchase this product?” a common error is providing options like “1-2”, “2-4”, or “4-6”. In this scenario, if a respondent buys the product twice a month, it’s unclear which option they should select. And respondents that buy the product seven or more times a month have no option at all!
   
   
   
3. Don’t use leading language that could introduce bias. For example, if you ask someone to give your product a rating, simply ask “How would you rate…?” Don’t introduce wording like “We’re committed to formulating tastier products. How would you rate…?” as that may be perceived as an appeal for a higher rating and subtly influence respondents to tell you what they think you want to hear.
   
   
   
4. Use a balanced scale when designing rating questions so that you don’t end up with data that skews more positive or negative. If your options included three positive options but only two negative options, then your data will be biased towards the positive.
   
   
   
   
5. Focus on close-ended questions. Ultimately, surveys are intended to be a quantitative research method, and there are better methods for capturing qualitative feedback, such as video testimonials (which you can also access through the Highlight platform).

Can I test multiple variables in an alienation test? 

Depending on the product, you may want to test a number of different key metrics. For example, a new formula for a snack may require evaluating aroma, texture, sweetness or other sensory metrics. By comparing the scores, decision-makers can determine if the test product's score significantly differs from the control's, and make informed decisions on what changes, if any, may be needed for the test product.

Find the formulas customers–old AND new–love

Using alienation testing for informed decision-making and risk mitigation

There’s no need to risk the loss of millions of dollars when tried-and-true processes like alienation testing are an accessible option. 

By identifying the specific factors that could alienate their customers, companies can make informed decisions about which changes to implement and gain data-based clarity directly from the consumer of the risks associated with your product update. 

When a company releases a product that flops, they don’t just risk the cost of that product and all the development, marketing, and distribution behind it. They risk damage to their brand reputation, too–a loss that will be deeply felt and can be much harder to recover from. 

Don’t leave your brand’s good name up to guesswork. Now that you know the ins and outs of alienation testing, it’s time to put your knowledge to use. Learn more about how you can start gathering consumer insights to insure your brand name and set your products–and future products–up for long-term success.

How Highlight’s product intelligence platform leverages alienation testing and other strategic methods to optimize and refine your products

If you’re trying to understand what product changes or new prototypes have potential, how you can refine your product positioning, or how you can mitigate alienation impact, Highlight can help. Our mission is to help companies build better products for people and the planet–and when it comes to alienation testing, that means giving you the data you need to make better innovation decisions.

If you need answers to questions like:

• Will this change to formula/packaging help attract new consumers?
• What kind of impact will this change have on my loyal shoppers?
• How can we optimize our product communication to capture the physical product experience?

The Highlight platform enables your team to:

• Identify which new prototype to move into further development by testing new options against your in-market product
• Gain in-context qualitative data from video, photos, or even in-depth interviews to better understand where to refine your packaging or messaging
• Use quantitative alienation evaluation to assess risk and potential impact to purchase intent

Attract new fans and retain loyal ones