Craft Data; Big Data for Refined Tastes

I spent a lovely afternoon recently in beautiful Buffalo, NY at The Blue Monk Bar. The Blue Monk is a craft beer specialty bar. They had 32 beers on draft at the time of my visit and I had heard of maybe 3. This was my idea of an afternoon in heaven. The beers were all unique (to me anyway), all had imaginative and creative names, and the few that I tested were all perfect for my tastes. Thinking about this experience and my job as a Big Data evangelist for NetApp it struck me that there were parallels between the beer market and big data (and honestly, if you can’t compare your profession to beer or bacon – what’s the point?). I’d like to introduce you to the concept of Craft Data.


How did I get here? I spend my time thinking and talking about Big Data, Analytics specifically. I have noticed some (increasing) Big Data Fatigue when talking to my customers. Gartner says Big Data is in the Trough of Disillusionment already. We still, as an industry, do not have a consistent or agreed upon definition of Big Data. So, I had to think about some new way to talk about Big Data to my customers when they say, “I don’t want to hear about Big Data.” My thought process was:


Mass breweries produce beer in, well, mass quantities (duh) and focus on Volume and Velocity. Variety? You want the regular or the light version?




Given the success of mega brewers and the money in big beer there is a place for mass-produced beer (NASCAR!) but certain tastes require something different.


Craft breweries produce beer in much less quantity (specialized) and focus on quality and Value. Your tastes drive you to a beer with oysters, chili's, or called “Moose Drool” or “Old Chub”?  Then craft beers are for you.


Thinking about Big Data as related to the beer market, I saw a correlation with Big Data and the special cuts of data that come from Big Analytics specifically. The vision in my head was that there is a place for Big Data but certain tastes require a craft view of data. And by view, I don’t mean just visualizations or dashboards (while very important) for humans. A craft view of data can also mean machine-machine data in my eyes.


So, what am I getting at? What is craft data? At a very high level:


  • Craft Data is the production of Big Data for refined tastes and views
  • Craft Data pushes the envelope on what can be done with Big Data
  • Craft Data requires imagination and artistry, Big Data is easy from a Volume and Velocity component
  • Craft Data provides a vehicle for organizations to reinvent and reinvigorate Big Data


An example of what I am talking about could be in the Utilities space, specifically in electricity with the Smart Grid. Big Data in this space is the volume and velocity from smart meters on every home in the United States sending every bit of information on electricity back to the utilities data centers. Craft data could be the real-time reporting of electricity use back to an individual consumer’s mobile device enabling the consumer to do real-time throttling of their electrical usage.


Like craft beer, craft data is made up of unique, hard to find ingredients. Sure, the same basic ingredients, but wildly different recipes. The craft of data (like beer) is about taking the core recipe (the core big data) and creating something special, something almost one of a kind. Craft data is about the art of the data, not just the production of it.


This is still very much a work-in-progress in my mind. Am I just renaming (poorly) the visualization/dashboard market with this concept (which I worry about)? Is the notion of defining a subset of Big Data a good idea? Do we need a universally accepted definition of the Big Data market before we can start defining new concepts underneath it? Is this interesting and requires more fleshing out? While I enjoy this beet-root infused beer, tell me what you think.


Cool W J Peter!

Good w/e