March 20, 2013

Big Data: Releasing the Power of Data to Drive Business Value

James Giese UWEBC Communications Director

Enterprises are capturing an ever-growing amount of data of all types, often now referred to as big data. These terabytes or even petabytes of data can be analyzed to provide new business insights.

Russ Lankenau, Solutions Architect, MapR Technologies, made a presentation on several big data use cases during the UWEBC's IT Peer Group meeting on March 20, 2013.

Lankenau discussed how to provide business value with big data and demonstrated some of the new tools available to analyze the data. He stressed that big data is more than huge volume and variety; it is an opportunity to find insights in new and emerging types of data, both structured and unstructured.

“Big data is not new, but the tools to collect and analyze large data sets are,” said Lankenau. He then presented a variety of use cases illustrating how organizations can obtain business value from big data analysis.

All of these use cases had similar characteristics: lots of data, data that was structured, semi-structured, and unstructured; and varied systems interoperating, such as Hadoop, Storm, Solr, and MPP. Lankenau pointed out that the use cases highlighted how distributed data processing can provide new sources of revenue, lower costs, and generate profits; but, good opportunities to do so usually start with a single focus.

• Use Case 1: Proactive Monitoring of Servers by a Managed Services Company. The data sources for this use case were server telemetry data, monitoring logs, and network flow data. The techniques used were pattern recognition; proactive monitoring; and early alert delivery of interruptions in services. Applying the analytic techniques the company already had in place with distributed processing allowed them to monitor their servers in real time to detect changes in behavior. Such real-time monitoring gave the organization an opportunity to offer new services and increase revenue.
• Use Case 2: Telecommunications Company with an ETL Offload Case. The company was running a large monthly dataset of customer records. By integrating big data technologies with its current data warehouse, the company was able to improve its data warehouse capabilities and lower the costs of its monthly Extract, Transform, and Load (ETL) offload case. 
• Use Case 3: Large Waste and Recycling Company to Track Idle Alerts. A large waste and recycling company had truck geo-location data captured at 5-second intervals for more than 20,000 trucks as well as geographic boundary information for landfills. The company used Hadoop, MapReduce, and Storm to develop route optimization and issue alerts when trucks were idle. The company was then able to decrease costs with the new truck routing and alert information.
• Use Case 4: Online Ancestry Records Company. Ancestry.com has huge amounts of raw data: birth, death certificates, census information, military records, immigration records and is now collecting snips of user-submitted DNA samples. The company also collects search behavior activity. The company uses record linking, search relevance, clickstream behavior, and DNA matching services. By applying Hadoop, the company was able to provide multiple views on top of a single large set of data with ad hoc queries of the structured and unstructured data. By doing so, it was able to provide a new service and increase revenue.

“All these examples started from a single focus that wanted to utilize a big data set. A good focus is to start with one very specific use case and then expand techniques and knowledge from that, “said Lankenau , “The key to discovering new insights from big data is to use both your structured and unstructured data together, while integrating existing analytics into your business processes.”