I recently chaired a session on ‘Going from company vision to Actionable Metrics’ at the Stoos Stampede conference in Amsterdam. In that session I tried to show some ideas on making the link from an overall company vision, through different approaches to achieve that vision, to concrete actionable metrics allowing teams within a company to autonomously pursue steps towards making that vision a reality. I’m not sure I succeeded in all of that in the session, so I’m trying again in this post…
Autonomy
A goal of a lean enterprise is to ensure that the people doing the work have all the information, knowledge and skills necessary to make decisions in their day-to-day work. For a lean knowledge organisation that means that people don’t just need to know their own work field well, they need to be able to relate the decisions they make every day to the longer-term goals and vision of the organisation.
Much has been said about supporting high levels of motivation and customer focus within companies. Especially in larger companies this is quite hard to sustain, which is not surprising with works such as Dan Pink’s Drive emphasising the importance of autonomy for the knowledge worker. Ensuring the right information, and a quick feedback loop for knowledge workers is key to motivated, high performing people.
Networks
Such autonomy can’t easily be achieved in a classically structured hierarchical organisation. The siloes inherent in that type of structure are natural barriers. Barriers to the autonomy of action where the distribution of necessary skills and knowledge over separate departments is an impediment to producing work and serving the customer. Barriers as well to the autonomy of reaction where the feedback loop on whether an action was in any way effective in reaching the goals of the organisation or not is too long, or absent.
An organizational structure much more compatible with that goal of autonomy, is that of a network organisation. The basic concept of a network organisation is that of independently working cross-functional teams that gather each other’s support where necessary but generally are able to make their own decisions. Enabling them to make their own decision is the subject of this post. These are the type of organizations that the Stoos Network is considering as the preferred replacement for today’s dysfunctions.
Actionable Metrics
The Lean Startup concept of Actionable Metrics (in order to create Validated Learning) is a great way to give a team the necessary autonomy to work independently towards the right goals. In a startup those metrics can be very directly linked with the goals of the company. In larger organisations there is need for a clear link between the overall company vision and Actionable Metrics that are usable on the team level.
**An actionable metric is one that ties specific and repeatable actions to observed results. **— Ash Maurya, http://www.ashmaurya.com/2010/07/3-rules-to-actionable-metrics/
In this post I’ll be using an Effect Map as the method to link the vision to specific metrics, but other methods exist of course. During the session, Catherine Louis mentioned GQM as a method designed determine which metrics to use. This paper gives some more background on GQM. The GQM method seems mostly concerned with determining the right metric for any given goal or problem, and can as such be very useful within the type of context I’m talking about. Another approach at determining the metrics you need is the A3 method.
The nice thing about Effect Maps is that they are very inclusive, and involve different roles and functions in their creation. This fits well with the multi-functional teams in our target organisation. They’re also easy to scale, using a diverge and merge facilitation process, so you can easily work on this with larger groups with full participation.
We’re on a mission from…
The first point of order is determining why we’re here. Not in a metaphysical way. I don’t really have the patience for that. In a ‘What are we trying to do as a company?’ way. A company vision and mission statements should provide us with a good starting point here. A vision statement could be “A literate future, “; with a mission statement of “More readers, more books.”;
This is of course very generic, and a subsequently generated Effect Map could go all over the place:
Effect Map Example
One thing we always need to add to the ‘Why?’ part of the Effect Map is a concrete, measurable, goal. This could be, in this case, encourage people to read more books, going from a current estimate of 100 books in a ’lifetime’ (30 years, apparently, in the poll we got that figure from), to 1000.
Our company could encourage people to read more books in many different ways. The Effect Map shows various directions: working through publishers, changing business models, working with public libraries, promote reading in schools, making books cheaper, working with writers directly instead of through publishers, and some ways of helping people find the right books through technology.
Since we have a technology company, those last seem more relevant to start with. A larger company would probably start exploring some of the other possibilities as well, and perhaps be able to integrate those with the technology work. That could mean incorporating different sales models into the e-reader software. Or creating a 2nd hand e-book market in there. Or something. Plenty of opportunities!
Getting to actionable metrics
How do we go from such a generic goal (people read 10x more books in their lifetime!) to some actionable metrics that can be used by the multi-functional teams that our network organization comprises of? These teams need to be able to use those metrics in their day-to-day decision making. They need to be able to devise experiments, prioritise work, and navigate towards products and solutions without the type of top-down supervision that characterizes the more traditional organization.
First of all you need a baseline. Say we have a product through which people can read books: e-reader software (I told you we were a tech company). From that software we could gather statistics on the number of books people read. To do this well, we’d probably need to track this relative to how long customers have been using our software so we don’t get skewed figures from early enthousiasm (for instance). The term to look for is cohort testing. In our example, it turns out people are buying, on average, one book every three months. To get to the goal of 10x more books, we should then improve this to 3 books a month! This is already a shorter term, and thus more helpful, goal.
Pirates
To get to more useful figures, we need to turn to Dave McClure’s Pirate Metrics. Pirate Metrics are all about the funnel of attracting customer interest, keeping them, and selling to them: Acquisition, Activation, Retention, Referral and Revenue, or AARRR. Just by looking at customers through this lens gives a useful perspective. Our goal is phrased as getting people to read (on average) 10x more books. This could be approached as a matter of increasing retention (more books per customer), but also as on of Acquisition/Activation (getting more customers). That last one only works if we don’t take them away from other sources of books, of course. Can you think of a way to measure that? Certainly combined with increasing retention it would still give a new positive effect.
This would give us two main variables to pay attention to: Retention and Acquisition. We should, as a matter of course, be paying attention to at least the first three (AAR) of these metrics, and most companies will have a natural tendency to also track the last R… But tracking what the result of specific actions are on Retention and Acquisition should have our focus for now.
Pirate Metrics
Splitting Metrics
But wait! In the Effect Map we had come up with two high-level feature ideas that would help us reach our goals: ‘Social Reading’ and ‘Better Book Recommendations’. Should both these ideas work with the same metrics?
Interesting question. On the one hand, I’d expect to be tracking all the pirate metrics in a well established application. But. The whole idea here is that you focus. So while we should keep a global eye on the whole (I’ll get back to that later), the experiments we’re conducting should focus on the change of a particular (set of) variables.
For our examples:
- Social Reading - This is mostly about having existing readers getting each other interested in other books. That would be Retention. Secondary would be getting new customers in by sharing outside the app, which would be Referral. It’s important to note that distinction, as this has a direct influence on the priority of hypotheses to try.
- Recommendations - This is also mostly about Retention. Existing readers should be getting more relevant recommendations, and thus but more books. The second level would be Activation. People who visit our shop already, but haven’t bought anything yet, should also be getting better recommendations and thus be prompted to buy.
This is consistent with the way we defined our goals, focusing on existing readers. That means it gives a decided focus to our development work. Phrasing our goals a little differently might increase our attention to new customer acquisition, but we’re not doing that. Consciously diving down into our metrics makes these kind of choices explicit, and that’s A Good Thing.
Absolutely Relative
So how would our teams take these more metrics towards specific hypotheses? First, we’d establish a baseline for retention. That could be
- When people buy a book, the average time between this purchase and the previous one is 92 days
Then we can start measuring this over time. A nice, always visible chart on a big screen in the development teams’ rooms would be a great idea.
This is a useful metric, as we can measure it day-by-day. It can also be calculated in time based cohorts, as well as feature based cohorts, so we can compare normal changes over time with changes caused directly by our new features.
Ok, now we can get started. “Social Reading”; is quite a broad concept. Our imaginative team of developers and product people can brainstorm-up quite a large cloud of ideas that fall within that scope, and they might have a collective gut-feeling on which ones of those would be most effective. They might have used another Effect Mapping exercise to generate ideas, and dot-voted on the most plausible ones. Or not.
The question they should be asking themselves is:
- What would be the simplest way, costing the least effort, to prove that this idea can indeed prove effective in decreasing the average time between purchases?
If that’s not what they’re asking, they might as well be asking their company if it was feeling lucky, inc.
So for any ideas they generated, they should be thinking about this questions: how can it help disprove (or prove) that the “Social Reading”; idea is plausible?
From the long list (or effect map) of ideas that they generated (sharing quotes, sharing notes, rating books, publishing ‘reading lists’, embedding shared things on blogs, embedding on facebook or twitter, etc.) they pick one item. In this case that item might be a very basic “If a user can easily share he’s reading a book on twitter, this will trigger a shorter time between purchases”;.
Now there are some problems with this one. Most important of all is that we don’t limit our audience, so we don’t know if people receiving the tweet will be existing customers. That’s Ok, though. It simply means we’re also testing for referral. Having an ‘internal’ audience might be more effective. But it would probably require a much larger up-front investment to create a communication channel between just our customers, an as such would be a less efficient way to test the hypothesis.
Another problem might be that we’re not helping the customer to share parts of the book, or anything, so the content of the tweet will probably be unspecific. We want more!
Stop!
Hold on. Take it easy. Hold your horses. We were looking for the simplest way to validate our hypothesis. How did we get into a discussion on all the cool features that should be in there? This feature, that feature, estimations (of both effort and expected value), discussions about opinions about hypotheticals…
If you want to know whether some tweets, to an audience that probably includes some existing customers, about a specific book, have some impact on sales then what you should do is write a few tweets. About some books. With an account that’s probably already there, from one of the people in the team. That probably already has other users of the service among its followers.
We all know that this is what should be done, that this is what the whole Lean Startup idea professes: Do The Simplest Thing. But even (or particularly?) in a bigger enterprise we need to put our money where our mouth is. And more importantly, avoid putting too much money where our mouth is and focus on getting that (in)validation of our most important hypothesis.
Giving the reins to the team
Taking those minimal steps is an important part of the overall process. It also seems to be one of the most difficult parts. Like developers needing time and practice to get used to working in the small steps of Test Driven Development. Like the Product Owners needing practice to split their requirements up into small enough chunks to be practical within a short sprint. Doing the absolute minimum work required to invalidate a hypothesis is probably the most difficult skill (or discipline?) to master from the Lean Startup mindset.
You can’t make it work without, though.
Especially in larger organisations where, by simple imperative of the size of the organisation, the involvement in individual projects, products and teams from the people setting the overall direction is much less than in a small startup!
The collaborative construction of Effect Maps ties together our organisation with a common vision and goal. Our carefully crafted and continuously tuned set of actionable metrics give teams clear direction within their level of influence to achieve.
To ensure that the organisational leadership doesn’t need to feel nervous about progress towards their goals, it is crucial that we fail as fast as is possible. And adjust. And try the next idea.
All Together Now
So organizational leadership can comfortably sleep at night in the knowledge that the full intellect and energy of their entire company is being put to work in the pursuit of truth, happiness and organizational goals while continuously self-correcting by the application of validated learning.
What more could they want?
There is one step still missing in this particular example, though. The metrics gathered for the specific experiments provide the very specific data needed for validated learning on the team level. The broader metrics that those are built on are still necessary for the bigger picture.
In our example that means that the targeted cohort testing done in each team is only one slice of the whole. The same (pirate) numbers are being gathered for a much broader cohort over longer periods of time to check whether the organisation as a whole is on the right track. Since that broader cohort would include the entire customer base, it will capture the combined results of all the teams.
Summary
In this article I’ve tried to illustrate, using a simple example, how longer term organizational goals can be made measurable in the short term, and can be used to provide the direction and purpose for teams to work independently and with full autonomy towards a shared organizational purpose.
Can you capture your organization’s vision in goals? What end-result metrics will you introduce? Can you refrain from cost metrics and focus on new value delivery? Go on. Do it.