By John Mertic
I spent some time reviewing the latest Gartner Magic Quadrant for Business Intelligence and Analytics Platforms in preparation for my time at the Gartner Data and Analytics Summit last week. Overall, I’m really excited to see vendors overall scoring higher in ‘Ability to Execute’; Gartner toughly judges this so seeing the general shift upwards is great to see.
While the piece is clearly targeted towards buyers of these tools – I wanted to take a critical eye on the positioning of vendors in relation to their interoperability with Big Data and Hadoop tools. After all, it was a mere decade ago that all of data was covered by a single Gartner analyst. Enter the age of Big Data; with that variability, velocity, and volume has come a cornucopia of products, strategies, and opportunities for answering the data question.
In the same way, BI and Analytics has come from being purely the realm of “data at rest” to become cohesive with “data in motion”. It’s no surprise then to see two “pure play big data” BI vendors, Datameer and ZoomData, joining ClearStory which joined the MQ last year – cementing the enterprise production need of valuable data insights. And with a tip of the hat to the new breed of open source trailblazers such as Hortonworks, they heavily leverage Hadoop and Spark as not just another data source but instead a tool to better process data – letting them focus on their core competency of delivering business insights.
However, what really struck me was the positioning of data governance as a whole in this report – let’s dig into that more.
Data governance and discovery is being pushed farther out
If you’d compare the 2016 report to the 2017 report – you’d immediately notice this line from 2016…
By 2018, smart, governed, Hadoop-based, search-based and visual-based data discovery will converge in a single form of next-generation data discovery that will include self-service data preparation and natural-language generation.
By 2020, smart, governed, Hadoop/Spark-, search- and visual-based data discovery capabilities will converge into a single set of next-generation data discovery capabilities as components of modern BI and analytics platforms.
Two year delay in just a year is something of note – clearly there is a continual gap in converging the technologies. This aligns with what our members and end-users in our UAB mention as well – the lack of a unified standard here is hurting adoption and investment.
Governance no longer considered a critical capability for a BI vendor
This really stood out to me in light of the point above – is sounds like Gartner believes that governance will need to happen at the data source versus the access point. It’s a clear message that better data management needs to happen in the data lake – we can’t secure at the endpoints for true enterprise production deployment. This again supports the needs of driving standards in the data security and governance space.
I recently sat down with IBM Analytics’ WW Analytics Client Architect Neil Stokes on our ODPi Member Conversations podcast series and the discussion of data lakes was a very present one. To listen to this podcast, visit ODPi Youtube.
I’m reminded of the HL Mencken quote “For every complex problem there is an answer that is clear, simple, and wrong.” Data governance is hard, and not ever going to be something one vendor will solve in a vacuum. That’s why I’m really excited to see the output of both our BI and Data Science SIG and Data Security and Governance SIG in the coming month. Starting the conversation in the context of real world usage, looking at both the challenges and opportunities, is the key to building any successful product. Perhaps this work could be the catalyst for smarter investment and value adds as these platforms continue to grow and become more mature.