Measuring the Mechanics in Gamification

If you can’t measure it, it doesn’t matter. At least, so they say. While it’s probably not true in every circumstance, being able to measure things does provide value, and—in a business context—it is almost essential in all cases. So, let’s take a look at how we might go about that in a gamification context using methods recommended by Janaki Kumar and Mario Herger in their book, Gamification at Work: Designing Engaging Business Software. If you can take readings of what’s going on in your projects, you’ll be far more likely to get results that put your work where it should be—ahead.

The Sales Manager

We’d like you to try this little thought experiment. You have just been made a sales manager at your company. Your first task is to conduct staff performance reviews. You decide to get the first two reviews done that day. You’re going to interview the two “star performers”, David and Kelly, in order to try to get an idea of what to expect as excellent performance within your team.

David arrives promptly and in a suit. You ask him to tell you about his achievements this year. He says, “I sold more than I did last year. I won a lot of new business—oh, and I’ve been salesman of the month twice.” Then he stops you and says he has to rush to a client meeting.

Author/Copyright holder: Richard Masoner / Cyclelicious. Copyright terms and licence: CC BY-SA 2.0

Success isn’t just about doing things right; it’s about communicating those things in the right way, too.

Kelly also arrives on time and presents herself before you, immaculate and confident. You ask her the same question. She replies: “I made $23 million in sales at an average margin of 14%. I brought eight new clients on board, increasing my accounts to 27 major accounts. I’ve been salesperson of the month twice—the first time for landing our largest account, which gives us an annual spend of $5 million now, and the second time, for the most sales in a month, in which I achieved $4.5 million of revenue.” She too then excuses herself and rushes off.

So, that’s the height of all the material you’re going to get from these interviews so far. You ponder for a moment on what you’ve heard. As you do so, you notice a motivational quote on your screen…

“Be personal. Be relevant. Be specific.”
—Seth Godin, management consultant and best-selling author

Which of the two team members did you learn more from?—Kelly, right? That’s because Kelly was specific about what she’d achieved; she laid out the metrics that she should be measured by. That’s exactly what you need to do for gamification projects. Without metrics, you would end up like David, talking in circles about success, but not really communicating anything of value. David may well have been able to match Kelly in what he had done—they both made salesperson of the month twice, after all—but you’d never have worked that out from his scant, vague facts. If you can talk about your projects in the way Kelly does, then you’re more likely to impress the people who provide you with the budget for gamification in the first place.

Mechanics depend on Metrics

Your metrics will vary from project to project. Only you can determine what the relevant key-performance-indicators (KPIs) will be for your gamification exercise. When you decide on a metric, you also need to decide how to measure it. If you can’t measure something, you can’t report on it without sounding like David in our example above.

Author/Copyright holder: jonny goldstein. Copyright terms and licence: CC BY 2.0

Metrics will vary from project to project, but it is vital to have metrics so that you can measure progress towards your goals.

Common Metrics

There’s no definitive list of metrics for projects but here are some examples that might get you thinking along the right lines.

Metric – Engagement (Who is Using the System and How Much?)

Possible Measurements: Average number of actions per player, median number of actions across the player base, number of players performing those actions, the number of repeat (returning) players, the progress of each player through the system, customer satisfaction (both players and business sponsors)

In a project, your finger should be on the pulse of many of these measurements; play-testing is especially valuable as you can investigate factors such as player progress right in front of you.

Metric – Time (What’s Happening as the System Matures?)

Possible Measurements: Retention of players, frequency of play, decrease in response time to system demands, timeliness

Again, assessing the figures as to who’s staying on the journey and for how long is a vital activity. If, for instance, a slowdown in system response time is putting people off, you should find out the size of the slowdown (in seconds) and figure out how to compensate by tweaking your design without sacrificing quality.

Metric – Return on Investment (What’s the Gamification Exercise worth to the business?)

Possible Measurements: Numbers of active players, increase in performance, reduction in costs, increase in sales, increase in customer retention

These are bigger-picture issues, but you can measure them more easily than anything else you’ll encounter. These are the figures that will represent the value of your work at first glance, so you’ll want them to be high.

Metric – Quality (How Good is the Gamification Considered to Be?)

Possible Measurements: Community or Player Ratings, Accuracy of Data Captured

Now we’re really talking feedback. If the players love it, that’s all the better; the question is, how sure are you as to how many of them love it/like it/find it OK? Many shades of grey can creep in here, so measure it well and put in the time to figure out where your work rates on the scale. Does “good” start at 72%, or should that still be “average” territory? What exactly do the players mean in their feedback?

Author/Copyright holder: Windell Oskay. Copyright terms and licence: CC BY 2.0

Quality is a very useful metric; get things right, and players will love you for it.

It can be tempting to incorporate as many metrics as possible. Don’t. Try to focus on a small number of key metrics that make your reporting concise, the data capture for reporting simple and which will give the team a very clear indication of the issues they need to address as well as where they have done a great job.

The right metrics will enable you to demonstrate the value of your project whenever you need to.

The Take Away

So as to measure the effectiveness of a gamification project and make that effectiveness meaningful to those who invest in it, you will need to develop some metrics. Some common metrics used to measure effectiveness include engagement, return on investment, quality, and time. However, you can develop bespoke metrics to suit the needs of the project, and you should not feel constrained by standard metrics when they don’t meet your needs. It can seem labour- and time-intensive to get this part of your project worked out, but it’s essential. Measured results are useful facts; subjective terms are only impressions. Taking down a player’s results clearly and with a sharp eye on figures will enable you to study what you need to do objectively, and an objective result is the bottom line in business.