I had the opportunity to sit in on a presentation by Arka Mukherjee, CEO and Founder of Global IDs, at the Data Governance Winter Conference in Ft. Lauderdale in December 2012. The session had an intrigued title that captured my interest, “Merging Big Data with Enterprise Data.” Arka acknowledged that most of Global IDs use cases are around “small data” today. However, the company works with several Fortune 1000 companies including a large telecommunications operator.
Arka provided some useful background information on big data from the perspective of Global IDs:
- Volume – Most of the big data use cases deal with petabytes versus megabytes of data
- Format – Big data is primarily in unstructured format
- Sparsity – Big data has very low signal to noise ratio, sort of like “searching for a needle in a needle stack”
- NoSQL – Big data involves NoSQL database platforms like Cassandra, Hadoop, MongoDB, Neo4j, etc.
Arka went on to emphasize that big data requires some unique skills:
- Data modeling with ontologies
- Data analytics with R
- Data movement in and out of NoSQL DBs
- Data governance on Resource Description Framework (RDF) stores
Arka mentioned the following use cases around linking big data with enterprise master data:
- Linking product master data with product safety websites
- Linking enterprise reference data with Linked Open Data, which is the biggest integration project linking thousands of government databases with RDF
- Linking supplier master data with news websites
- Linking enterprise master data with third party data like market data, demographics, credit reports from Acxiom and D&B
Global IDs provides “extreme data governance” capabilities in large and complex data environments. They claim to take the problem away from humans, and let machines handle these processing-intensive tasks. Arka mentioned that a recent client had a million databases, and the average client will look at three to four million attributes.
An interesting company that I will definitely want to learn more about.