Yammer analytics – what should I measure?

Laurence Lock Lee from Swoop analytics

Most internal communications people
 are a bit lost when it comes to metrics and internal social networks like Yammer.
I interviewed Laurence Lock Lee, the co-founder & Chief 
Scientist at Swoop Analytics, about ESNs and measurement.
(A bit too much to read in one go? Download a PDF version of the post).

1. Let’s say I’m thinking about launching
 Yammer and I work in internal communications. What should I measure?

You really need to be clear about what your executive sponsors are looking for. The common response is ‘increased employee engagement’. For me ‘Engagement’ means open and transparent conversations. Therefore I would advocate that the breadth and depth of conversations on Yammer should be a priority over simpler activity measures.

2. What if want to launch Yammer and I’m responsible for a change management initiative. Let’s for argument’s sake say I want all our experts to start doing sales. I want to use Yammer as a way to offer peer support. What should I measure?

An important capability of Yammer is to ‘deepen’ the support for front line sales staff by bringing internal experts into the conversation. Internal experts are usually happy to support sales staff, but the onus is on sales to use Yammer to ask for help. I would suggest the best way to start is to have frontline sales staff establish some sales specific groups and seed the discussions with their current questions and challenges, inviting internal experts to join the groups and conversations. You should then aim to measure and monitor the breadth and depth of the conversation threads and who is participating (or not). For specific challenges I would recommend the use of hash tags, which can then be used to measure and monitor the degree of engagement.

2. I have a Very Important Message
 I need to get across to all staff. Is there a way for me to measure the impact of that message?

This depends on the message. If the message is about a new emergency evacuation procedure for the building, then a simple measure of % read may be enough (if you use Yammer you will have to use O365 analytics to get reads though). More likely, if the message is about say, a proposed HR policy around ‘working from home’, then read statistics are not enough. Here again the breadth and depth of the conversation around the message is key. For sure, the message should be hash tagged to help monitor this. And it can have an effect. Last year we interviewed David Thodey, the former CEO of Australia’s largest Telco Telstra, about their use of Yammer. He noted that policy messages were one area that Yammer significantly changed the way they developed them.

3. Senior management want me to prove
 the business value of our Yammer network/enterprise social network. What do I do?

A common question and not a simple one to answer. Firstly, I would respond by asking them “what would a successful use of Yammer look like for them?” You can waste a lot of time chasing down specific use cases that can be formalised into an ROI calculation, only to find that they are still not convinced (I’ve been there). What can have an impact is a simple business story or anecdote (even if it is not yours). The trigger for David Thodey to become a Yammer zealot was a simple story about how a young service desk technician was able to rapidly respond to a customer about an issue with an outdated mobile phone, when he posted the issue on Yammer, and was flooded with responses, many from outside the formal support centre.

What can have an impact is a simple business story or anecdote (even if it is not yours).

4. On Swoop (software from Laurence’s company) you categorise people
 in different collaborator personas (catalyst, broadcaster, engager..). Is the goal to move people towards a certain type of persona, for example away from observer or broadcaster towards responder or engager? Or is it a tool for community managers who can
 then identify people who can help them in various ways?

Our goal is to help people improve their own collaborative behaviours, by showing them how they are currently interacting. This is a personal thing, that’s why we secure the personal tab. We do have a nominal pecking order of Engager, Catalyst, Responder, Broadcaster, Observer, based on academic research and our own applied research. But different contexts will call for different behaviours, so any of the positive personas (Engager, Catalyst, Responder) are worthwhile targets, based on your personal situation. Community and team leaders can also gain value by looking at the persona compositions of their teams or communities. Obviously too many observers and/or broadcasters is not a good situation to be in. Targeted coaching by leaders can help here, but real change has to start with the individual, and that’s the model we are pursuing.

5. You talk a lot about the social
 cohesion metric. What does it consist of and why is it so important?

Our ‘Social Cohesion’ measure is a simple proxy for ‘reciprocity’. It simply measures the degree to which your interactions are reciprocated by others in your network e.g. you reply to my posts and I reply to yours. Aggregating individual results to the whole enterprise means that we have the ability to assess social cohesion at all levels of an organisation. In network science, reciprocity is a fundamental and key indicator of trust and therefore performance. One of the early insights we gained from our Yammer benchmarking data is how dangerous it is to rely on simple activity measures, when what you are after is better collaboration.

In network science, reciprocity is a fundamental and key indicator of trust and therefore performance.

6. My goal is to increase collaboration/knowledge
 sharing between departments. How would I measure that on Swoop?

Swoop has the ability to accept directory/profile data into its analytics (in fact we actively encourage it). When this data is available we show on the Enterprise tab the degree to which interactions are spanning departmental boundaries. On the Group tab, community leaders can assess the diversity of their reach across the formal lines of business. The same with the Topic tab, where we can see how much a tagged topic is pervading the formal organisation; which is good for assessing the engagement around the important messages that I spoke of earlier. We are not just limited to departments. If we are provided with other profile attributes like geographic location, gender, job roles etc..we can see how interactions span these attributes as well.

7. Is Swoop more about measuring collaboration
 than it is about measuring communications? If so, why?

Absolutely. That said, we would argue that ‘communications’ should be two-way, not just one-way. We often see that Yammer is a flow on from an Intranet project. Traditional Intranets were all about one-way publishing. Yammer adds a ‘Social’ dimension to a traditional Intranet, meaning interactions and then hopefully collaboration. The reason for this is that information itself generates no explicit value unless it is acted on. And today these actions need to be increasingly collaborative, rather than individual. So an effective measure of collaboration will in fact predict performance in our view.

8. If you could teach Swoop users
 one thing, what would it be?

While community managers are very important to us, our goal is the whole enterprise. We aren’t there yet, but our message to prospective end users of Swoop is that “we can help you improve your collaborative behaviour to become a more productive member of your teams and communities; and therefore be acknowledged for the value you bring to them”

9. In addition to working at Swoop
 you also do social network analysis where you identify key informal influencers within an organisation. This is important in change management. Could internal communications people do a similar analysis before rolling out an ESN to find out who could drive
 ESNs forward?

Yes, Swoop emerged from over a decade of social network analysis projects and management consulting www.optimice.com.au. A number of those projects were conducted ahead of a major enterprise wide implementation, to as you say, identify those potential influential staff that could smooth the way for the implementation, by being ‘enrolled’ in the change ahead of the implementation. Our experience has been that when organisations can identify the informal influencers at all levels in the organisation, and then enrol them in the change activity; change will happen faster and more effectively. This case study speaks to this.

When organisations can identify the informal influencers at all levels in the organisation, and then enrol them in the change activity; change will happen faster and more effectively.

10. Where are ESN analytics solutions headed?

I think we are still suffering from a more ‘quantity’ than ‘quality’ approach, with the majority of vendors looking to drive more data into the ubiquitous ‘data warehouse’. While we are starting to see more acknowledgement of personal productivity analytics in ESN analytics solutions, I would suggest they address more ‘process’ elements than the required deeper ‘behavioural’ elements, that we think are more important.

As we are seeing with Yammer, I think we will see ESN’s become more integrated with the more general ‘Office’ enterprise software and incumbent analytics. This could potentially weaken the focus on collaboration, which is the acknowledged strength of ESNs. For example, typical office analytics emphasise personal productivity; how can I waste less time in meetings, read and answer less emails and IMs, find that document faster. While on the surface there is nothing wrong with this, it doesn’t directly address the need to improve collaboration. Perhaps increasing your own personal productivity is only at the expense of someone else? I know that many of our clients are frustrated by the quality of analytics that come ‘out of the box’. While this is a good thing for Swoop, I think it is important that as we move forward with  Enterprise software that vendors start to appreciate that analytics should no longer be the sole purview of management and that in today’s economy, change has to energised from the bottom up; and that is where our analytics need to be.

Perhaps increasing your own personal productivity is only at the expense of someone else?

11. Since more and more organisations want their employees to be collaborative, HR are trying to figure out how to reward it (as you know many current incentive systems accidentally reward information hoarding – why share information if that information helps your colleague get that promotion or bonus you wanted!). Some HR people have noticed that internal social networks offer a way to get data on who is being collaborative and reward that. What do you think about this development?

We are seeing a growing number of firms abandoning the annual performance review in favour of a continuous feedback and coaching approach. One of the nice things about analysing social networks is that it is difficult to game. For example in Swoop we identify influential leaders through their networking patterns, of which a key component is their ability to attract responses from others.

Another measure we use, which we have drawn from academic research, is the give-receive balance.

Another measure we use, which we have drawn from academic research, is the give-receive balance. Essentially it is related to your ability to balance the contributions you make with those you receive (our engager persona). While you can directly impact how much you post, reply and like, you can only impact what you receive by the nature of what you give. Social networks therefore take the politics associated with hierarchies out of the performance equation. I do have some concerns about HR ‘formalising’ a process of rewarding key players in the social network, simply because it could destabilise what in essence is a natural process, with interventions drawn from top down management thinking. The situation I would like to see emerge is more related to the way holocracies are working, where formal rewards are determined by the people you interact and work with the most. Today’s line managers would need to transform from ‘organiser/controller’ to ‘coach and connector’ and assist teams undertake these performance assessment tasks.

If you have questions to Laurence he is @llocklee on Twitter.

Download this post as PDF.

– Virpi

PS. Swoop have a standing offer to Yammer users for a free two week trial plus membership of their benchmarking programme – check out their site for details. You will get a free assessment of how your organisation compares with others in their database. It is completely anonymous.

One thought on “Yammer analytics – what should I measure?

  1. This post made my week. Thanks Virpi and Laurence. 🙂
    We are in the process of defining what to measure in our future social collaboration network and your lessons learned helped us a lot.

    Thanks again!

© 2013 All Rights Reserved. Business Goes Social ®