Big data is a misnomer. It reduces the realities of the continuously growing deluge of data to exactly this aspect: the deluge, the chaos and, last but not least, the volume aspect. In other words: anything that refers to this dimension of ‘big’.
At least, that’s still how many people see it, ever since the days we started talking about big data and despite the fact that the aspect of volume is less important in reality. Although we know that the outcomes, the challenges and opportunities of unstructured data and big data analytics are all far more important than the volume dimension (velocity, variety, value, purpose and action matter more), each single day new research is published to emphasize how much big data there really is.
Focus on the big data industry: alive and well but changing
Given the link between the cloud and big data, AI and big data analytics and the data and analysis aspects of the Internet of Things (IoT) with a clear connection between analytics, AI and IoT, it isn’t really a surprise that, just as is the case with IoT, AI, cloud and so forth there is quite some hype regarding these predictions on the growth of the big data universe.
We like numbers, don’t we? And we seem to like anything that’s really big. If big data is big, so is the industry that is built around it. From big data software and services to infrastructure and analytics the big data industry is alive and well. Yet, at the same time it’s changing.
Big data analytics: an increasing role in the rapidly growing BDA market
A huge (bigger than big) industry is that of big data and business analytics, where big data (analytics) is gradually getting a bigger piece of the pie.
Acording to IDC the “Worldwide Big Data and Business Analytics Market” or BDA, so analytics alone, is poised to grow from $130.1 billion this year to over $203 billion in 2020 (forecast published on October 3rd, 2016), among others driven by a shift towards a data-driven mindset and the increasing importance of data strategy.
In the words of IDC’s Dan Vesset: “The availability of data, a new generation of technology, and a cultural shift toward data-driven decision making continue to drive demand for big data and analytics technology and services”.
By the way, you read that number right: over $200 billion or $0.203 Trillion in just four years from now. To put it in perspective: also according to IDC, worldwide IT spending is expected to reach $2.7 Trillion in 2020. Or in other words: the BDA industry alone is good for over 7.5 percent of all IT spending, which, among others, also includes telecommunications, services, cloud, mobility, smartphones, consumer IT, and storage. Imagine that.
True, BDA is certainly not just big data analytics but, then again, as big data continues to mature, IDC expects its share of the larger Business Analytics market to increase.
The growth of the overall big data technology and services market
This growing role of big data in the BDA market was mentioned by IDC end 2015 when the company predicted that by 2019 the worldwide big data technology and services market was growing to $48.6 Billion in 2019. The big data technology and services market is expected to reach $57 billion by 2020.
Obviously it’s not that easy to define what the big data technology and services market exactly is. In more than one IT spending category or type of applications, big data is an important piece of today’s reality, even if that category/application covers more than just big data. Think about APM or Application Performance Management, for instance, which is about ALL applications, including big data performance monitoring. Or think about data centers, the cloud, cybersecurity, the list goes on.
If we look at the overal big data industry (security, services, storage infrastructure, networking, data center infrastructure, discovery tools, big data applications, analytics and so on), it’s clear that the big data industry has at least four more golden years ahead of it. And if you are planning your career now you might want to consider moving in that enormous Big Data and Business Analytics space.
Yet, the maturation of big data also means that the industry is changing and so is the way businesses look at it.
Big data investments: looking ahead
On October 4th, Gartner published a press release with big data investments numbers and predictions that made the big data industry react in no time.
A survey, conducted in June 2016 among 199 members of Gartner’s Research Circle, shows that although big data investments continue to rise, the investments also showed signs of contracting.
According to the survey 48 percent of companies invested in big data in 2016, an increase with 3 percent in comparison with 2015. Yet at the same time the percentage of companiess planning to invest in big data in the next two years dropped from 31 percent to 25 percent.
The expected weakening of big data investments made the industry frown. Several actors said that there were no signs whatsoever that organizations would spend less than before, well on the contrary. Critics, among others, pointed to the low numbers of participants.
However, Gartner’s findings represent realities facing the big data industry over the next few years, which should be taken into account.
Big data challenges to solve as the market matures
Gartner’s Nick Heudecker gave different possible explanations for the findings. We look at a few of them and add our take with some additional comments and observations.
A more holistic view
For starters, there is too much focus on big data as a separate effort, rather than looking at how it is used in a holistic way with all the consequences of such a purpose-driven and integrated approach.
Getting beyond the pilot stages
Secondly, several respondents who have invested or planned to invested in big data often remain stucked at the pilot stage and only a small percentage said having deployed their big data project to production.
A matter of priorities
The just mentioned explanation obviously can be linked with the first one but there is also a third reason: it appears that big data projects tend to get less spending priority than other IT initiatives.
The ROI challenge
Next, there is the fact that many big data projects don’t have a tangible ROI which can be determined upfront, as Nick Heudecker put it. This was certainly a point that was debated a lot. However, it is pretty clear that if you stay stuck in the pilot stage and don’t look at big data projects in a holistic way, you can’t really measure ROI as in many cases you don’t even have a clear plan. This might seem obvious but it is a pain point and always has been.
The common business sense issues in big data
When it boils down to technological innovations it seems that we tend to forgot essential and basic questions such as the goal, the way to get there and so on.
Moreover, we also see the issues of isolated efforts with a focus on the technology instead of the common sense business aspects popping up time and time again. The exact same phenomenon is happen in the space of the Internet of Things and it never has been any different before. And, indeed, we see it clearly in regards to big data as well.
Business leadership as a must
This brings us to another issue: a lack of effective business leadership or involvement in data initiatives and pilots with ad-hoc technologies and infrastructure.
Not even the most fierce critic of Gartner’s finding can ignore this, again, simple business fact, which we see happening each time when new technologies are being adopted as well. The business needs to drive the decisions and that means leadership.
If the business leadership isn’t involved (enough), in today’s reality that means almost guaranteed failure. It goes for big data projects, it goes for IoT projects, it goes for any digital transformation or simple digitization project. Again: business drives investments, everywhere.
When big data becomes mainstream
Last but not least there is the statement by Nick Heudecker on the evolutions of big data as a term and practice.
We quote, “Another reason could be that the big data initiative is a part of a larger funded initiative. This will become more common as the term “big data” fades away, and dealing with larger datasets and multiple data types continues to be the norm.”
What’s your big data purpose?
For us, this is as crucial a point as the others. Organizations should focus on realizing business projects, leveraging the technologies and big data they have and need, to get the results they want, with a clear ROI.
And, increasingly, big data or whatever we call it, is a part of that. But you don’t do big data project or a ‘xyz technology’ project for that matter.
You want to drive more revenues, enhance customer experience, save costs, create new business models, find new sources of revenue, etc.
Big data lessons from digital transformation – change ahead
There is too much focus on big data as a separate effort
This requires that holistic perspective instead of a separate effort. If there is one thing we should remember about digital transformation and even digitization it is exactly this focus on goals, steering away from all too much focus on the technologies, looking at challenges and opportunities in a holistic way (with a clear leadership and beyond silos) and using common sense.
It shouldn’t come as a surprise that this isn’t happening at many organizations, also in ‘big data projects’ (again, a misnomer, and indeed as it becomes mainstream, it will fade away).
This is a responsibilty for everyone: IT, the business but certainly also the “big data industry” which in many cases tends too focus to much on big data in its narratives, rather than looking at the individual context of each business project and the broader reality and purpose in which big data solutions fit.
This big data industry will continue to grow, the increasing focus on big data analytics is a good sign about the direction it’s heading towards. However, as we come closer to 2020, the industry will change and in some areas investments will drop while new ones will join the ‘big data’ industry reality. And it’s not just about organizations being ‘unable’ to (fully) leverage the potential of (big) data. It’s not just about the lack of business focus which also lives in the big data industry.
It’s also about changes in the broader big data space as such. But that’s for a next article.
Top image: Shutterstock – Copyright: Photon photo – All other images are the property of their respective mentioned owners.
