THE FUTURE IS NOW

New developments for processing unstructured data are arriving on the scene almost daily, with one of the latest and most significant coming from the social networking site Twitter. Making sense of its massive database of unstructured data was a huge problem—so huge, in fact, that it purchased another company just to help it find the value in its massive data store. The success of Twitter revolves around how well the company can leverage the data that its users generate. This amounts to a great deal of unstructured information from the more than 200 million accounts the site hosts, which generates 230 million Twitter messages a day.

To address the problem, the social networking giant purchased BackType, the developer of Storm, a software product that can parse live data streams such as those created by the millions of Twitter feeds. Twitter has released the source code of Storm, making it available to others who want to pursue the technology. Twitter is not interested in commercializing Storm.

Storm has proved its value for Twitter, which can now perform analytics in real time and identify trends and emerging topics as they develop. For example, Twitter uses the software to calculate how widely Web addresses are shared by multiple Twitter users in real time.

With the capabilities offered by Storm, a company can process Big Data in real time and garner knowledge that leads to a competitive advantage. For example, calculating the reach of a Web address could take up to 10 minutes using a single machine. However, with a Storm cluster, that workload can be spread out to dozens of machines, and a result can be discovered in just seconds. For companies that make money from emerging trends (e.g., ad agencies, financial services, and Internet marketers), that faster processing can be crucial.

Like Twitter, many organizations are discovering that they have access to a great deal of data, and those data, in all forms, could be transformed into information that can improve efficiencies, maxi-mize profits, and unveil new trends. The trick is to organize and analyze the data quickly enough, a process that can now be accomplished using open source technologies and lumped under the heading of Big Data.

Other examples abound of how unstructured, semistructured, and structured Big Data stores are providing value to business segments. Take, for example, the online shopping service LivingSocial, which leverages technologies such as the Apache Hadoop data processing platform to garner information about what its users want.
The process has allowed LivingSocial to offer predictive analysis in real time, which better services its customer base. The company is not alone in its quest for squeezing the most value out of its unstructured data. Other major shopping sites, shopping comparison sites, and online versions of brick-and-mortar stores have also implemented technologies to bring real-time analytics to the forefront of customer interaction.

However, in that highly competitive market, finding new ways to interpret the data and process them faster is proving to be the critical competitive advantage and is driving Big Data analytics forward with new innovations and processes. Those enterprises and many others learned that data in all forms cannot be considered a commodity item, and just as with gold, it is through mining that one finds the nuggets of value that can affect the bottom line.

Taken from : Big Data Analytics: Turning Big Data into Big Money