Getting Data from the Jive Data Export Service – Not Fun

Originally published Dec 14, 2014 in JiveWorks. It’s important to note that with endlessly patient guidance from Dirk McNealy I eventually mastered getting data from the Jive DES and have found it to be extremely valuable. But it’s still not fun. Much too hard to use. I’ve added in updates to the original post that reflected my understanding as of October 2017. I’ve learned much more since then, which I’ll post about at another time.

I’ve been looking forward to using the new Jive Data Export Service ever since Josh Richau first started telling me about it. It sounded like a terrific resource that would be easy to use. I even attended the JW14 session by John Hungerford and Dirk McNealyJive Data Export Service – Turbo-Charge Your Jive Integration with Analytics – THURSDAY, OCT 23RD 4:30 PM – 5:30 PM to learn more. I also read and re-read Using the V2 Jive Data Export Service (Analytics) in the hope that it would walk me through what I needed to know.

We just upgraded to Jive Cloud in November 2014 and this week is the first chance I have had to dig into the Jive DES. I’m sorry to say that my initial experience with the Jive DES has been underwhelming in terms of outcomes and overwhelming in terms of difficulty. Perhaps I just need simpler directions, which I would be grateful to follow. Here are the issues I am encountering and would love to have help resolving:

  1. I thought I would be able to connect directly to the Jive Analytics database with a tool such as Tableau. Is that possible, and if so, how is it done? [Update: it can’t be done.]
  2. I thought all of the data was going to be in one giant table, but the Jive DES Explorer offers four separate download options. So I am challenged to join tables, after all. [Update: it is actually all one giant table, but the DES Explorer portal forces you to access it as if it were four tables, unhelpfully in my view.]
  3. I need to be able to filter my data based on attributes in our user profiles. The most valuable attributes are our custom profile fields, of course, containing valuable data such as Business Unit. These fields are not part of the data set I can get from Jive DES. It looks like I’m going to have to build some way to match on username and look up in some other table the profile data I need. [Update: Indeed, I did have to build my own database to create and keep updated two separate tables from other sources than DES, and then join them with DES records in Tableau. Talk about not fun…]
  4. The 50k row limit really sucks. It’s a huge time suck to have to download file after file of daily data to try to make sure no records will be omitted. Since there are four different types of file, that means four downloads for each day’s data. Not fun and not a productive use of my time. [Update: Jive removed the 50k row limit.]
  5. Because of the 50k row limit, I can’t get usable data for Content. There are more than 50,000 rows of data for a single day in the Content file, but the smallest increment of data I can select is one day. I suppose I could filter on specific activities and then piece it all back together, but see above about not fun and not a productive use of my time. I have a pretty small community (7-8000 users) so I can’t imagine how large communities would get any usable data. [Update: Jive removed the 50k row limit.]
  6. The Explorer user interface is not user-friendly. To filter data before you download it, you don’t select from a list of possible terms — you have to type each term in individually. Again, not fun and not a productive use of my time. [Update: I stopped using the Explorer portal and rely strictly on scripts which people much smarter than me created and I have learned to modify to meet my needs.]
  7. There is a way to use REST to get the data, but how I can do that on a Mac and return tens of thousands of records in a CSV format is not documented anywhere that I can see. It appears to be something developers could do, but I’m not a developer. I’m just the person who really needs to use the data. [Update: I still don’t know how to use REST, really, but after attending a workshop on using it specifically to access this data, and armed with great crib sheets, tips and a kick-ass script written by Dane Slutzky, I can now get DES data using REST.]

I would love to learn that I have simply missed the wonderful user manual that explains all of this and makes it easy to get value from this service I’ve been hearing such positive things about for so long. I would love to have someone offer training in how to do this. I would really love to have Jive make it easy to get my data using tools like Excel or Tableau without hiring a developer.

Leave a Reply

Your email address will not be published. Required fields are marked *