How We Measure Community Engagement Activity

Originally published Mar 30, 2016 in our internal Jive community, Spark, at McGraw-Hill Education, when I introduced new community analytics developed using Jive’s Data Export Service (DES) data. This is where I introduced the new terms I came up with to describe community activity: Value-Gained Activity and Value-Added Activity.

Introduction

Spark engagement analytics were new as of April 2016 and continue to evolve as we learn better how to work with the data and design ways to make it useful and actionable. As such, these analytics are subject to change without notice. If you find a need for a specific measure, please let Ted Hopton know so we can continue to make it available for you even if we move away from it in our general Spark reporting.

Engagement Activities

To better understand how people are engaging in Spark, we break engagement activities into six categories, five of which are grouped into Value-Gained and Value-Added Activities.

In simplest terms, value-gained activities are those that take from the community and value-added activities are those that give to the community. Both giving and taking are desirable: we need people to give (create value in Spark) so that other people can take what they need (gain value from Spark).

Value-Gained Activities

Activities which provide value for the person performing them are called value-gained activities. The actor is the one who benefits by gaining something of value from Spark.

  • Consumption — Activities where members viewed or downloaded content

Value-Added Activities

Activities which provide value for others are called value-added activities. Other people in the community benefit by gaining something of value from the contributions of the actor. There are four categories of value-gained activities:

  • Participation – Activities where people interacted on content in the community by either liking, commenting, sharing, rating, etc.
  • Curation – Activities where people help organize community content to make content / conversations easier to find. We break curation into two types:
    • Personal Curation – Activities that organize how the individual finds/receives content (e.g., bookmarking, following)
    • Community Curation – Activities that make it easier for the rest of the community to find content (e.g., tags, move)
  • Collaboration (creating, uploading or editing content)
  • Outcomes (marking content with labels such as Official, Final, etc.)

Participation & Collaboration Matter Most

While it is useful to understand all of the ways that people add value in Spark, it’s important to recognize which activities are most important: Participation and Collaboration. Participation activity is a significant driver of value and engagement in Spark, and Collaboration is the essential act of creating content in Spark, without which there would be nothing from which to gain value. Curation activity, and to a lesser degree Outcomes activity, add “noise” to our data when we are trying to focus on what matters most, so many of our reports omit them to make it easier to see the trends related to Participation and Collaboration.

Expert / Asset Location

After we break engagement activity into value-gained and value-added activities, one category of engagement activities is left over: Expert / Asset Location — Activities where members search for either answers (content) or experts (people) to get their task accomplished. Locating needed resources is a vital engagement activity, so it is included in the Engagement Index, but it is neither a value-added activity (the community does not benefit from searches) nor a value-gained activity (the act of searching does not produce value — actually finding what you need does). Searching is a step in the process of gaining value, not an indicator that value was gained.

If we had a way to measure successful searches, then we could include that as a value-gained activity. Currently, we can only measure how many searches were conducted, and this data is not helpful in measuring value. For example, if we improve the findability of content such that fewer searches per person are needed to locate it, then the number of searches would decrease. But if more people use search more often to locate valuable content, then the number of searches would increase. Tracking the number of searches does not give us insight into the value people are gaining from using Spark.

Instead, consumption activity is the result of people locating needed resources, since people had to find content or profiles and then chose to view them. In addition to the search engine, locating includes other navigation, as well as Spark presenting the content to users on the home page or in streams or email notifications. The end result is what represents value gained: users chose to view particular content.

Value-Gained Index (VGI)

The Value-Gained Index focuses solely on value-gained activity, where the user benefits by gaining something of value from Spark. Value-Gained Index = All Spark value-gained activities / unique value-gaining users.

Value-Added Index (VAI)

The Value-Added Activity Index focuses solely on value-added activity, where the user contributes to the betterment of the community. Value-Added Activity Index = All Spark value-added activities / unique value-adding users. We rarely report on the VAI because we focus, instead, on the Participation + Collaboration Index.

Participation + Collaboration Index (PCI)

The Participation + Collaboration Index (PCI) focuses on what matters most in value-added activity (see above, Participation & Collaboration Matter Most). Participation + Collaboration Index (PCI) = (Participation activity + Collaboration activity) / unique participators and collaborators.

Participation Index (PI)

The Participation Index (PI) measures Participation. Participation Index (PI) = Participation activity / unique participators.

Collaboration Index (CI)

The Collaboration Index (CI) measures Collaboration. Collaboration Index (CI) = Collaboration activity / unique collaborators.

Using Spark Engagement Analytics

The reports are delivered in Tableau Server, which allows viewers to interact directly with the reports to filter the results, change the time-scale and drill-down on any segment to view the underlying data (click the Revert or Undo buttons to back up from any changes). Each provides a different lens through which we can view the activity taking place in Spark.

Leave a Reply

Your email address will not be published. Required fields are marked *