Predictable Hybrid Hadoop Blog Series – Crossing the Chasm

By March 20, 2017Blog

In the previous blog in this series, we outlined some of the important ways that running Hadoop in production – especially in enterprise-wide production – differs from point solutions and PoCs.

As a leading downstream community of big data vendors, users and platform providers, ODPi is focused on tackling the security, governance, lifecycle management and application portability needed to run Hadoop at scale.

A classic way to think about technology maturity is Geoffrey Moore’s Chasm model. In our new white paper, we plot key Hadoop milestones against the technology adoption curve (see image below) and argue that the things the ODPi community are focused on are essential to continuing the adoption of this transformative technology.

An adaptation of Everett Roger’s famous S diffusion of innovations curve, the Chasm model argues that users on the left of the chasm are fundamentally different from those on the right. The chasm separates users by adoption trigger/motivation: on the left it’s all about competitive advantage at nearly any cost, on the right it’s about continuity of operations and keeping up with the Joneses.

As awesome as this model is, it has sometimes been co-opted. One way this happens is by applying it to a Product, when in fact it needs to apply to a Category. This is one reason why we are so bullish about our work at ODPi – we explicitly acknowledge that the only way Hadoop and associated Big Data solutions can cross the chasm to mainstream adoption is by working together to define category-wide – NOT vendor-specific – answers to questions like lifecycle management, security and governance, application portability – these are the things that address early and late majority users’ interest in stability and operational continuity.

When thinking about what it really means for a technology to be a platform, we like the way Sam Ghod puts it:

A platform abstracts away a messy problem so you can build on top of it. Platforms do this by delivering portability and extensibility.

With ODPi Releases 1.0 and 2.0 in place, we invited Application Vendors to self-certify that their applications work unmodified across multiple ODPi Runtime Compliance Hadoop Distros. As of this writing, twelve applications from leading vendors like SAS, IBM and DataTorrent have completed the self-certification.

We believe that savvy Enterprise CDOs, CIOs, CTOs and Chief Information Security Officers (CISOs) should carefully consider the platform independence that ODPi’s Interoperable Apps program delivers before making their Hadoop platform choices. If one of your preferred vendors isn’t listed either as an Interoperable App or as a Runtime Compliant Platform, let that vendor know that it matters to you.

In 2017, we’re heads down adding to our existing specifications and creating new workstreams through our Special Interest Groups. We invite you to get involved. If you are a twitter user, be sure to follow @odpiorg and participate in our ongoing polls.