Very recently, when it has been extensively reported in the media that Microsoft researcher David Rothschild has accurately forecasted this year’s 20 Oscar winners out of 24 with the help of his predictive model, our paths have once again crossed with big data in a fascinating way. But then, it’s a glimpse into a smorgasbord of transformational capabilities that it brings. For businesses, the lingering buzz around big data doesn’t ring hollow as they are unlocking unprecedented value by applying it. An Accenture survey has captured this pervasive sentiment; 94% of companies have admitted that their experience with big data has proved to be an exact match for their expectations. As such, related investments are coming in thick and fast. Capgemini Consulting predicts that three years hence organizational spending on big data technologies will end up at $114 billion.
On this splendid note, let’s have a retrospect of major developments in the concerned space in 2014. Also, we will look at some of the success stories, and the hurdles that have got businesses in a sweat. Finally, a short lowdown on the best practices that make the tryst with big data a meaningful and plain sailing affair.
The mass exodus of companies from a physical set-up to a cloud based one has been a significant event last year. This has happened primarily because a host of data processing technologies are now fully operational on cloud platforms. Sensing the opportunity, leading vendors have spared no effort to drive companies towards cloud. Offerings such as Salesforce’s Wave, IBM’s Bluemix, or Amazon’s Redshift have gained a healthy stream of customers. In fact, a hybrid architecture that combines the cloud and on-premises will form the backbone of data crunching in near future.
The open-source and distributed big data solution Hadoop has emerged as the default analytic framework for a large number of enterprises. In a sense, it is the data OS for them. Moreover, products that promise SQL-on-Hadoop are making life easier for enterprise users. These are light on company coffers as running them do not require the services of experts with specialized knowledge.
Storing data in Hadoop is not the end in itself. It’s just the beginning, as analysis of the data stacked there is what generates valuable insights. An array of advanced processing engines in 2014, whether open-source Spark or other commercial alternatives that are designed to manage data with multiple attributes, have enabled companies to get real benefits.
With more and more companies deciding to dabble in big data, demand for skilled data professionals has soared in 2014. This has encouraged education providers to come up with various industry-ready training programs on the subject.
Data lakes are basically repositories that can store data in native formats for parsing at a later time. Since the concept is an effective tool in combating data fidelity and integration issues, companies are finding it useful.
Big data has a lot of use cases in an enterprise environment. According to the Accenture survey, 85% of users think that it has phenomenal power to change the workings of a business. They have identified customer relationship, product development, and management of business operations as the areas where the impact of big data will be felt the most. Almost replicating this tone, a QuinStreet enterprise report says, to 72% of companies the biggest advantage of big data lies in fast and spot-on decision making.
Here’s a few instances of successful implementation of big data:
That big data is a boon to enterprises is no secret. However, the process of adapting to a big data-driven culture is not free from challenges.
Practices that move things in the right way
The best anodyne to fight the pain generating from the issues discussed above is to develop a dedicated big data strategy.
To conclude, a ‘separate the wheat from the chaff’ approach is needed to discover the truth in big data i.e., enterprises must clearly understand what information to keep and what to discard.