Amid all of the predictions and hypothesis about the future of BI in 2015, my favourite developing trend is the convergence of Big Data and Business Intelligence. Three things we saw in 2014 are driving this:
1. The Hortonworks IPO. While some view the IPO of Hortonworks as the peak of Hadoop hype, it also marks the turning point for Hadoop as a technology mature enough to be adopted by large companies and government organizations. The IPO is a sign of insider confidence that they will close a number of big enterprise software deals over the next several quarters. This means that innovation is slowing down but it also marks the start of a wave of big data projects that are more than pilots and experiments. The expectation now is that real business value will be delivered. Hortonworks was the first to IPO, but we will also see lots of news in 2015 from the two other leading Hadoop companies, Cloudera and MapR.
2. The enterprise software vendor push for Hadoop integration. The big enterprise software vendors missed no occasion in 2014 to promote their own proprietary solutions as the perfect complement to Hadoop. IBM is the vendor to watch here, as they have a huge install base of Cognos and SPSS customers who will be interested in adopting Hadoop. I think that all of the legacy BI vendors will do something noteworthy in 2015 in their attempt to gain big data mindshare. For example, Microsoft announced last week the acquisition of Revolution Analytics, a software vendor that sells commercial support for the open source statistics programming language R. R is a popular choice among data scientists for working with big data sets sourced from Hadoop. (Also interesting to note is that Microsoft is an investor in Hortonworks.) Oracle, HP and EMC are also working to blur the lines between their legacy BI/DW products and big data software.
3. The emergence of the Chief Data Officer. While the cost of collecting and storing massive amounts of data has dropped, it is not free, and sitting on a virtual mountain of data introduces a new set of problems and opportunities. Hence, the emerging role of the Chief Data Officer (CDO) in large organizations that are not technology companies. One of the key challenges of the CDO is to merge data together (perhaps dumping into the data lake?), which means marrying legacy databases and data warehouses with newer and bigger data repositories. The business will not want to give up their sophisticated BI systems, honed over many years (or even decades) but at the same time they will need newer technologies to integrate and to access fresher and broader data sets.