The problem with activity & engagement (and what to measure instead)

Get out your work gloves, it's time to give your metrics context by tying them to a customer process.
  •   4 min read

Page views are up. Page views are down. So what?

It’s fair to say that for most apps, no activity at all is definitely a bad sign. But can we say that some activity is definitely a good sign? Not exactly.

In-app activity signals, like sign ins, page views, clicks and other action events, don’t necessarily equate to anything positive. It’s entirely possible, for example, that your customer logged in for the third time this week because they have not been able to accomplish the task they wanted to accomplish and they are showing their boss just how bad your app is so that they can justify the switch to a new system.

The problem is that while engagement is correlated with positive outcomes for customers (you have to “engage” with the software in order to produce a positive outcome), it is also correlated with negative outcomes (frustration and churn, for example). In other words, engagement doesn’t necessarily lead to customers finding value– at least not directly, or in all cases.

So what does produce positive outcomes?

Work! Specifically, work within a process that leads, quite deliberately, to the customer finding value or realizing a positive outcome at the end.

Measuring work within a process that leads to a successful outcome is different than measuring individual engagement metrics in the way we have been measuring them since forever.

The main difference is giving context to our metrics. There’s a world of difference between measuring work for work’s sake vs. measuring work as progress toward a specific outcome. Work is still activity, but it’s activity with an outcome-oriented context.

To better illustrate what we’re talking about here, let’s walk through a hypothetical example taken from real life: Mowing the lawn.

To start, the positive outcome we’re looking for is a freshly mown lawn – that’s success for our users. Any other outcome is negative, or at least subpar. So what work (activity) needs to happen to get that lawn mowed?

The user needs to:

  • Go into the shed, because that’s where the mower is.
  • Get the battery (or gas can, but we’re going electric) off the charger and plug it into the mower.
  • Check to make sure the mower is ready (possibly empty the bag if they were lazy last time).
  • Move the mower to the lawn.
  • Mow the lawn (and actually finish the whole thing).

Traditionally, we’d look at a few “key” actions within this process in order to judge engagement levels and therefore the overall health of our Lawn Mowing Platform™. We’d count the number of Shed Entries, Battery Pickups, Bag Empties, and Total Lawn Mows and display those totals into some kind of Lawn Activity Dashboard.

Then we’d say something useless like, “guess what Boss, Shed Entries were up 15% last month!”

And when our boss brings up the fact that Total Lawn Mows actually declined a little bit we’ll say, “oh don’t worry about that—we’ve got lots of positive engagement with the shed, which as we all know is a very good indicator because that’s where the lawn mower is.”

Let’s get real about user progress

In reality, a user entering the shed means next to nothing unless they also get the battery, plug it in, check the mower, and move the mower out of the shed with them when they exit. If we link all these activities together we get a much better indication of lawn mowing intent then we would simply measuring them all as discrete, unrelated data points. All of those actions, that work, are part of a step in a process: Lawn Mowing Prep.

Once you’ve defined Lawn Mowing Prep as a set of actions that are done together, you can start measuring it in a useful way. Not just counting the number of times someone completed Lawn Mowing Prep, but looking into deeper questions:

On average, how long does Lawn Mowing Prep take? 5 minutes? If someone has taken 20 minutes to complete that step, or stopped early, how likely are they to go on to successfully mow the lawn today? Maybe a longer time indicates there was a problem. Like the battery wasn’t charged and they left the shed to wait. How many users started but did not complete the work involved with Lawn Mowing Prep this week?

Going even further, how many people don’t complete the process in a strictly linear fashion? How many enter the shed, get the battery, leave the shed, re-enter later, check the mower, leave again, come back and finally get the mower out?

Doing things the old way, that scenario leads to inflated engagement metrics (3 Shed Entries!) with no sense of the cause. But when you’re measuring activity within a process AND you’re measuring that process at an individual user level you can easily see that this particular user is just procrastinating their lawn mowing work (maybe we need to smooth out the process?).

What’s the takeaway here?

Start thinking about activity and engagement in terms of the role those actions play in achieving a positive outcome within your app or business. What work are your users doing within your app? What process do they follow? What process should they follow? Planning and mapping the ideal process that moves a user toward a positive outcome is the first step to customer experience enlightenment.

Enjoy this article? Show it some love!

Share on twitter
Share on facebook
Share on linkedin

Get the newsletter

  • Talk with our team

    Complete the form below and we’ll be in touch!

  • This field is for validation purposes and should be left unchanged.
  • Sign up to receive Chassi news and CXE insights

  • By signing up you agree to our Terms of Use. You may opt-out at any time.
  • This field is for validation purposes and should be left unchanged.

By browsing this website, you accept the use of cookies, as they help us optimize your experience. Learn more.