A Culture of Experimentation: Increasing User Engagement and Retention

3 min read

Engagement meets Retention
This is a deep dive on one of the topics from our new eBook, Digital Behavior Design!

Behavior change is a science, and that means we need to experiment if we want to make progress increasing engagement and retention.

Experimentation is not just a tool for learning, it’s an imperative for any business that is going to survive in a changing world. And no world changes faster (and more capriciously) than consumer apps. The app you and your team built might be the center of your entire world, but for your users, it’s just another app on their phone vying for their attention amidst dozens of others competing for use.

Experimenting – on the content in your app and structure of your user experience – will be one of your most powerful tools in ensuring your app stays novel and interesting for your users.

Strategies for Fostering Experimentation in Your Organization

It’s one thing to say your organization values experimentation. But how do you actually embed ‘experimentation’ as a cultural value?

The first step is to define what that looks like in practice. This may sound obvious, but it’s important that your team know that experimentation an essential piece of the company culture and how to do it. Turn every employee into the CEO of their domain, who will think critically, act boldly, learn from failure and keep pushing forward.

Below are some strategies to foster a culture of experimentation and ownership in your organization.


Update Your Values

If experimentation is valuable to your organization or department, consider listing it in your company values. Most companies values include: integrity, leadership, accountability, passion, commitment etc. which are fantastic values. However, these words do not on their own communicate a culture of experimentation. Some ideas for values that promote experimentation:

  • Challenge the status quo.
  • Think critically, act boldly.
  • Iterate and innovate.
  • Fail fast.

Never be Satisfied

Great products are never done, and neither are their teams. Truly great products are constantly evolving, solving new problems and staying relevant for their customers. In neuroscience, we learn that when we get used to something, it loses its intrigue and we become immune to things that used to delight us. Continuous product design and development to re engage and delight users is essential to staying relevant, exciting, and enticing.

Push your team to constantly look at the product with a critical eye. As product owners, we should be our own hardest critics identifying areas of improvement ongoing. Do user research, user testing and QA ongoing and push your team to never be satisfied with the current state.

Things to look for:

  • New problems your users face
  • Areas of the app that seem outdated or out of sync with the product
  • Things that do not fit in with the holistic view of the product
  • Features that are not getting good traction with your users
  • Fluff features that may be the result of feature creep and are a distraction more than a benefit

Reframe Success and Failure

Fear of failure is something that we all deal with. It is especially challenging in a professional setting where employees feel their credibility is at stake. Reframe success and failure for you team to alleviate some of this emotional resistance.

Everyone believes things that aren’t true, and often our ‘failures’ are nothing more than having the courage to check if a belief is true.
In an experimentation culture, success lies in having the idea, formulating the hypothesis and running the experiment to test it. We can’t control or predict the outcome, and more often than not we fail. Reframing success as having an idea, developing it and following through with it not only helps to motivate your team but it puts success in your control. And hey, if the experiment pays off thats an added bonus!

Experimentation at Boundless Mind

To ensure that you’re doing more than guessing and checking as you change things, you need to understand both the underlying domain model of what you’re experimenting on (human behavior), and have the right tools to be rigorous about the experiment you’re conducting. These both matter because without an understanding of the phenomenon you’re experimenting on, you won’t be able to make any new predictions as a consequence of your findings. And without experimental rigor, you can’t know whether or not the data you’re collecting accurately represents real life, or is experimental error. It will all have been a waste of time.

For example, at Boundless Mind, we help teams experimentally optimize positive reinforcement in their user experience. We chose this approach – positive reinforcement – because the paradigm for the domain model we’re working on (human behavior and human habits) predicted where we should experiment first: reinforcement. The frequency of a behavior is modified most effectively by the pattern and rhythm of reinforcement that we receive for behavior. We didn’t have to derive that from first principles: by understanding the underlying phenomenon that we’re interested in experimenting on, we could select an appropriate method and route to intervention (also, why we passed on working on Push Notifications first: the domain model of human psychology suggests that triggering a behavior alone is not particularly effective at habit formation).

So when beginning to experiment yourself, what is the underlying model that describes what you’re trying to change? How do you know this to be the case? And what work has already been done in the field to flesh out the paradigms that describe that phenomenon? If you’re working on smartphone apps, chances are your underlying phenomenon are going to be human psychology and neuroscience. If you haven’t yet, check out our Required Reading list for some of the books that helped us better make sense of how and why users do what they do (no PhD required!)

We built a rigorous experimentation engine to try new and improved ways of personalizing delight for users AND do so in a way that let us continue to predict and learn. We’re fortunate to have backgrounds in experimental sciences, so we could build this in-house. But for teams that aren’t interested in going back to grad school – or hiring out a science team to work with your engineers, there are great tools online for experimentation. Personally, we like Optimizely and Taplytics. Once you have a great framework selected, we’d strongly suggest brushing up a little on experimental design to help you make sure that you’ve constructed your experiment well.

Want to learn more about behavior design? Reserve a copy of our upcoming ebook!

Lindsey Meredith