Top 10 Trends in Big Data
"Big data" is no longer just a buzzword. Researchers at Forrester have "found that, in 2016, almost 40 percent of firms are implementing and expanding big data technology adoption. Another 30 percent are planning to adopt big data in the next 12 months."
Similarly, the Big Data Executive Survey 2016 from NewVantage Partners found that 62.5 percent of firms now have at least one big data project in production, and only 5.4 percent of organizations have no big data initiatives planned or underway.
Researchers say the adoption of big data technologies is unlikely to slow anytime soon. IDC predicts that the big data and business analytics market will increase from $130.1 billion this year to more than $203 billion in 2020. "The availability of data, a new generation of technology, and a cultural shift toward data-driven decision making continue to drive demand for big data and analytics technology and services," said Dan Vesset, group vice president, analytics and information management. "This market is forecast to grow 11.3 percent in 2016 after revenues reached $122 billion worldwide in 2015 and is expected to continue at a compound annual growth rate (CAGR) of 11.7 percent through 2020."
While it's clear that the big data market will grow, how organizations will be using their big data is a little less clear. New big data technologies are entering the market, while use of some older technologies continues to grow. This slideshow covers ten top trends that will likely shape the big data market in 2017 and beyond.
Big Data Trends
Experts expect machine learning, predictive analytics, IoT and edge computing to have a big impact on big data projects in 2017 and beyond.
Open source applications like Apache Hadoop, Spark and others have come to dominate the big data space, and that trend looks likely to continue. One survey found that nearly 60 percent of enterprises expect to have Hadoop clusters running in production by the end of this year. And according to Forrester, Hadoop usage is increasing 32.9 percent per year.
Experts say that in 2017, many enterprises will expand their use of Hadoop and NoSQL technologies, as well as looking for ways to speed up their big data processing. Many will be seeking technologies that allow them to access and respond to data in real time.
One of the technologies that companies are investigating in an attempt to speed their big data processing is in-memory technology. In a traditional database, the data is stored in storage systems equipped with hard drives or solid state drives (SSDs). In-memory technology stores the data in RAM instead, which is many, many times faster. A report from Forrester Research forecasts that in-memory data fabric will grow 29.2 percent per year.
Several different vendors offer in-memory database technology, notably SAP, IBM, Pivotal.
Image Source: Micron Technology
As big data analytics capabilities have progressed, some enterprises have begun investing in machine learning (ML). Machine learning is a branch of artificial intelligence that focuses on allowing computers to learn new things without being explicitly programmed. In other words, it analyzes existing big data stores to come to conclusions which change how the application behaves.
According to Gartner machine learning is one of the top 10 strategic technology trends for 2017. It noted that today's most advanced machine learning and artificial intelligence systems are moving "beyond traditional rule-based algorithms to create systems that understand, learn, predict, adapt and potentially operate autonomously."
Image Source: MapR
Predictive analytics is closely related to machine learning; in fact, ML systems often provide the engines for predictive analytics software. In the early days of big data analytics, organizations were looking back at their data to see what happened and then later they started using their analytics tools to investigate why those things happened. Predictive analytics goes one step further, using the big data analysis to predict what will happen in the future.
