Hadoop Summit San Jose 2016 Wrap-up

By July 13, 2016Blog

We’re Making Good on our Pledge to Open the Big Data Ecosystem

As part of the industry convergence on San Jose, ODPi members and Linux Foundation staffers used Hadoop Summit to share our common commitment to grow Apache Hadoop and Big Data through a set of Specifications.

HS16SJPic1.jpg

.@vVineet @ScottCGrayIBM @hornpolish & @smiti_sharma sharing “War Stories: Making Software Work w/ Hadoop

HS16SJPic2.jpg

@ODPiOrg booth at Hadoop Summit – those rocket footballs were a hit!

HS16SJpic3.jpg

@IBMBigData booth before the show opened – Can you find the ODPi Rocket?

HS16SJpic4.jpg

@CaskData captured plenty of attention with their focus on Applications and Insights, not Infrastructure and Integration

HS16SJpic5.jpg

@Altiscale ready for the rush of attendees looking for Big Data as a Service

It was terrific seeing ODPi members and sharing ideas at the conference. And the conference sessions couldn’t have been more on point. In the words of Ben Markham from ODPi member Xiilab:

I particularly loved the session about Apache Nifi and how to build a smart home, as this is related to Xiilab and also something I’d personally love to do. The sheer amount of data that needs to be processed in order to make an efficient smart home is amazing, and it speaks to why we’re all so passionate about this industry!

Before describing the significant milestone achieved at Hadoop Summit, first let me provide a short recap on ODPi’s progress to date.

ODPi published its first Runtime Specification in March to specify how HDFS, YARN, and MapReduce components should be installed and configured. The Runtime specification also provides a set of tests for validation to make it easier to create big data solutions and data-driven applications.

  • The purpose?
    Increases consistency for ISVs and End Users when building on top of, integrating with, and running Hadoop.

  • Why?
    Because consistency around things like how APIs are exposed and where .jar files are located reduces engineering effort on low-value activities like maintaining compatibility matrices, so that more effort can go into building the features that customers care about.

That’s the promise and commitment ODPi and its members made to the industry when we published the Runtime Spec.  

At Hadoop Summit, ALL FIVE ODPi members that ship Apache Hadoop distributions announced that they achieved ODPi Runtime Compliance.

runtime_compliant_image.PNG

Cool – so how exactly does that Open the Big Data Ecosystem?

Two of the Distros that achieved Runtime Compliance, Hortonworks and IBM Big Insights, collectively partner with several hundred of the biggest Big Data ISVs and IHVs.

Altiscale, a cloud Big Data as a service company, Infosys, which supports many government clients around the world with their Hadoop distro and custom Big Data apps on top of it, and ArenaData, who is making a name for themselves bringing Hadoop and Big Data to more Russian and Eastern European businesses, also achieved Runtime Compliance.

Thanks to ODPi, today ANY of the applications that run on Hortonworks or IBM Big Insights can, WITH SIGNIFICANTLY LESS UP FRONT AND ONGOING engineering cost, support Altiscale, ArenaData and Infosys.

Pivotal lit the way by describing on their blog how Pivotal HDB was installed on the ODPi reference implementation and on one of the ODPi Runtime Compliant distributions with no modifications to standard installation steps.

That’s called Opening the Big Data Ecosystem!

Now it’s your turn to show your support for an Open Big Data Ecosystem

Tweet why YOU think Hadoop and Big Data need standards.

Share a challenge you’ve faced, maybe an engineering effort that just took way longer than it should have, or a customer support ticket that by rights should have taken minutes but instead took hours.

Be sure to tag @odpiorg and include the hashtag #ODPi4Standards in your tweet and you’ll be entered to win one of TEN $25.00 Visa Gift cards. Read contest rules here.*

*Eligibility Criteria: 10 people, tweeting 7/14/2016 – 7/18/2016, with constructive #ODPi4Standards feedback + @ODPiOrg tag or RT will win a $25 Visa gift card.