Category

Announcements

ODPi Publishes First Runtime Specification and Test Suite To Simplify and Expedite Development of Data-Driven Applications

By | Announcements

Leveraging Key Apache Hadoop® Components, ODPi Members Advance First Industry Standard Deployment Model for Enterprise Big Data Solutions  

SAN FRANCISCO, March 28, 2016–ODPi, a nonprofit organization accelerating the open ecosystem of big data solutions, today announced the first release of the ODPi Runtime Specification and test suite to ensure applications will work across multiple Apache Hadoop® distributions.

Designed to make it easier to create big data solutions and data-driven applications, the ODPi Runtime Specification is the first release from the industry-backed organization. While the Hadoop ecosystem is rapidly innovating, a certain degree of diversity and complexity are actually impeding adoption. Founded last year, more than 25 ODPi members are focused on simplification and standardization within the big data ecosystem and further advancing the work of the Apache Software Foundation.

Descending from Apache Hadoop 2.7, the Runtime Specification features HDFS, YARN, and MapReduce components and is part of the common reference platform ODPi Core.

“The turbulent big data market needs more confidence, more maturity, and less friction for both technology vendors and consumers alike,” said Nik Rouda, senior big data analyst at Enterprise Strategy Group (ESG). “ESG research found that 85% of those responsible for current Hadoop deployments believed that ODPi would add value.”

Key ODPi Runtime Specification Technical Features

The ODPi test framework and self-certification also aligns closely with the Apache Software Foundation by leveraging Apache BigTop for comprehensive packaging, testing, and configuration. Additionally, more than half the code in the latest Big Top release originated in ODPi.

All ODPi Runtime-Compliance tests are linked directly to lines in the ODPi Runtime Specification. To assist with compliance, in addition to the test suite, ODPi also provides a reference build.

The published specification also includes rules and guidelines on how to incorporate additional, non-breaking features, which are allowed provided source code is made available through relevant Apache community processes.

What’s Next for ODPi

The ODPi Operations Specification to help enterprises improve installation and management of Hadoop and Hadoop-based applications will be available later this year.  The Operations Specification covers Apache Ambari, the ASF project for provisioning, managing, and monitoring Apache Hadoop clusters.

“ODPi complements the work done in the Apache projects by filling a gap in the big data community in bringing together all members of the Hadoop ecosystem,” said John Mertic, senior manager of ODPi. “Our members – Hadoop distros, app vendors, solution providers, and end-users – are fully committed to leveraging Apache projects and utilizing feedback from real-world use cases to provide industry guidance on how Hadoop should be deployed, configured, and managed. We will continue to expand and contribute to innovation happening inside the Hadoop ecosystem.”

Comments from Members

Ampool

“With its broader, flexible approach to standardizing the Hadoop stack, ODPi is particularly attractive to smaller companies, such as Ampool. Instead of spending testing/qualification cycles across different distributions and respective versions, the reference implementation would really help reduce both the effort and risk of Hadoop integration for us.” – Milind Bhandarkar, Ph.D, founder and CEO, Ampool

DataTorrent

“ODPi will simplify developing and testing applications that work across distros and hence lower the cost of building Hadoop-based big data applications. For example, DataTorrent will be able to certify RTS installation and runtime for ODPi and know it will work with multiple platform providers.” – Thomas Weise, Apache Apex (incubating) PPMC member and architect/co-founder, DataTorrent

Hortonworks

“At Hortonworks, we aim to speed Hadoop adoption through ecosystem interoperability rooted in open source so enterprise customers can reap the benefits of increased choice with more modern data applications and solutions. As a founding member, we are pleased to see ODPi’s first release become available to the ecosystem and look forward to our continued involvement to accelerate the adoption of modern data applications.” – Alan Gates, co-founder, Hortonworks

IBM

“Big Data is the key to enterprises welcoming the cognitive era and there’s a need across the board for advancements in the Hadoop ecosystem to ensure companies can get the most out of their deployments in the most efficient ways possible. With the ODPi Runtime Specification, developers can write their application once and run it across a variety of distributions – ensuring more efficient applications that can generate the insights necessary for business change.” – Rob Thomas, vice president of product development, IBM Analytics

Linaro

“Linaro recognizes the importance of ODPi’s work to promote and advance the state of Apache Hadoop and Big Data technologies for the enterprise while minimizing fragmentation and redundant effort. Linaro’s own focus is similar to this in developing open source software for the ARM ecosystem and it makes perfect sense that where these two areas intersect that Linaro and ODPi should work together to ensure ARM is fully supported and that fragmentation is minimized across the industry.” – Martin Stadtler, director of the Linaro Enterprise Group (LEG)

Pivotal

“It was a little over a year ago that ODPi was formed, and we have already proved beneficial to upstream ASF projects (Hadoop, Bigtop, Ambari). There’s a need for a stable enterprise-grade platform that is managed as an industry asset to benefit all of the companies driving value from Hadoop and big data. This is why the first release of the ODPi Runtime Specification and test suite is so exciting. It is a big step toward realizing our goal of accelerating the delivery of business outcomes through big data solutions by driving interoperability on an enterprise-ready core platform.” – Roman Shaposhnik, director of Open Source at Pivotal, Apache Hadoop and Bigtop committer and ASF member

SAS

“As a founding member, SAS’s support of the Open Data Platform Initiative demonstrates our ongoing commitment to developing innovative applications and solutions for our customers that are compatible with the Hadoop ecosystem. ODPi enables us to remain committed to ensuring our applications work with and exploit the Hadoop distribution of our customers’ choice, while being able to bank on the stability and quality expected in demanding business environments.” – Craig Rubendall, vice president of platform R&D, SAS

Additional Resources

About ODPi

ODPi is a nonprofit organization committed to simplification and standardization of the big data ecosystem with a common reference platform called ODPi Core. As a shared industry effort, ODPi members represent big data technology, solution provider and end user organizations focused on promoting and advancing the state of Apache Hadoop® and big data technologies for the enterprise. For more information about ODPi, please visit: http://www.odpi.org.

###

Media Contact:

Natasha Woods
ODPi
(415) 312-5289
pr@odpi.org

ODPi Doubles Membership, Announces Technical Milestones and Open Governance Structure

By | Announcements

Industry-wide effort to advance open standards for Apache Hadoop® attracts diverse representation from Big Data ecosystem, brings enterprise requirements and technical team to bear

NEW YORK (O’Reilly’s Strata Conference) and BUDAPEST (Apache: Big Data Conference), September 28, 2015 – ODPi, a nonprofit organization accelerating the delivery of Big Data solutions by powering a well-defined platform called ODPi Core, today announced new members, technical milestones, its formal governance structure and that it will be hosted at The Linux Foundation as a Collaborative Project.

The explosion of data and the requirements to store and process information has resulted in a variety of Big Data solutions. ODPi brings industry leaders together to accelerate the adoption of Apache Hadoop® and related Big Data technologies and make it easier to rapidly develop applications. This will be done through integration and standardization of a common reference platform that enables users to realize business results more quickly. ODPi will integrate a variety of upstream Apache projects, working across the Apache ecosystem to create a downstream reference platform on top of which new Big Data solutions can be built.

Membership investments in this effort have nearly doubled since ODPi was announced in February. Members to date represent a diverse group of Big Data solution providers and end users such as Altiscale, Ampool, Capgemini, CenturyLink, DataTorrent, EMC, GE, Hortonworks, IBM, Infosys, Linaro, NEC, Pivotal, PLDT, SAS Institute Inc, Splunk, Squid Solutions, SyncSort, Telstra, Teradata, Toshiba, UNIFi, VMware, WANdisco, Xiilab, zData and Zettaset.

“ODPi is a useful downstream project for the community to work on a common reference platform and set of technologies around Hadoop,” said Jim Zemlin, executive director at The Linux Foundation. “We’ve seen this model work with open source technologies experiencing rapid growth and know it can increase adoption and open up opportunities for innovation on top of an already strong Hadoop community.”

Technical milestones include the release of an initial ODPi core specification and reference implementation that simplifies upstream and downstream qualification efforts and has been created by developers from across the Big Data landscape. More than 35 maintainers from 25 companies are dedicated to this ongoing work to start. The planned ODPi Certification Program is also underway. The goal of ODPi Certification Programs will be to ensure consistency and compatibility across the Big Data ecosystem. To learn more about this technical progress, please visit: https://github.com/odpi or visit the ODPi.org website at http://ODPi.org

ODPi uses an open governance model that is led by a community of developers who will form a Technical Steering Committee (TSC) based on expertise and value of contribution. All members will have an equal vote on ODPi Core decisions, regardless of investment level, ensuring equality among all participants and an industry-wide consolidation of enterprise requirements. ODPi will also elect a Board of Directors responsible for the financial, legal and promotional aspects of ODPi.

ODPi will be featured at Strata+Hadoop World New York, Sept 29-October 1 where it will demonstrate the vision of one application running on many ODPI member distributions. An ODPi panel will be featured at Apache: Big Data in Budapest September 28-September 30, 2015

For more information about the ODPi, please visit: http://ODPi.org

ODPi is a Linux Foundation Collaborative Project, which are independently funded software projects that harness the power of collaborative development to fuel innovation across industries and ecosystems. By spreading the collaborative DNA of the largest collaborative software development project in history, The Linux Foundation provides the essential collaborative and organizational framework so project hosts can focus on innovation and results. For more information about Linux Foundation Collaborative Projects, please visit: http://collabprojects.linuxfoundation.org/

Member Comments

Altiscale
“The ODPi has made tremendous progress in a short amount of time. This rapid growth attests to the demand for standards in the rapidly evolving Hadoop ecosystem. Both customers and third party application providers can see greater value from Hadoop when they are confident that their developed solutions will run as broadly as possible,” said Raymie Stata, CEO of Altiscale, former CTO of Yahoo!, and core member of the ODPi Technical Working Group. “The Technical Working Group has been moving along speedily on Hadoop core, and we are already looking ahead to bring more projects into the standard specification.”

Ampool
“With the widespread adoption and importance of the Hadoop ecosystem within the enterprise, the ODPi standard platform is a very important and timely effort, and Ampool is excited to be participating. With ODPi, it will be clear what standards and level of support are expected for platforms, configuration, security, and interoperability in Hadoop distributions. We are confident that the ODPi effort will provide a solid base for further growth of Hadoop as the foundation of next generation data infrastructure,” said Milind Bhandarkar, Founder & CEO, Ampool Inc.

DataTorrent
“Our membership in ODPi demonstrates our commitment to spurring adoption and fostering support for the larger Hadoop ecosystem. As one of the earliest members to ODPi, we strongly believe in participating in a common framework to ensure that every enterprise has access to DataTorrent RTS for unified batch and streaming platform, no matter what Hadoop distribution,” said Phu Hoang, cofounder and CEO, DataTorrent.

EMC Corporation
“EMC is keenly interested in helping our customers build standardized environments for big data workloads. These environments are good for customers and good for the industry,” said Kelly Kanellakis, Senior Director, Business Operations, EMC Corporation. “ODPi builds exactly that through its efforts to create interchangeable Hadoop environments by working with customers and vendors in a collaborative way.”

GE
“GE Software is committed to advancing the Hadoop ecosystem to support the industrial requirements of managing, processing and extracting insights from big data at scale,” said Vince Campisi, CIO, GE Software. “The creation of a common platform certification and governance process under the ODPi is enabling us to more effectively deliver industrial-strength apps to our customers to tackle their big data challenges with confidence. With the help of ODPi we can achieve this at a low cost, while enabling our customers to also benefit from the productivity gains that the Industrial Internet has to offer. It is a win-win for everyone.”

Hortonworks
“At Hortonworks, we believe innovation happens not in isolation but in collaboration. We aim to speed Hadoop adoption through ecosystem interoperability rooted in open source so enterprise customers can reap the benefits of increased choice with more big data applications and solutions. As a founding ODPi member, we are pleased to see its significant strides toward these goals, all under an open and transparent governance model,” said Shaun Connolly, vice president of corporate strategy, Hortonworks.

IBM
“The power and appeal of open source innovation for technologies such as Hadoop is undeniable, especially when it comes to the growing volumes of data generated by social media, mobile devices, and machine-to-machine sensors,” said Beth Smith, General Manager, Analytics Platform, IBM Analytics. “In a recent report, IDC estimated that only 30 percent of clients have adopted Hadoop. Adoption is being constrained by complexity and a lack of standardization. IBM is fully committed, working with this community, to help drive speed-to-innovation for consistency and standardization in the development of smart business apps and accelerate the use of analytics across every business in a fundamental way.”

Infosys
“Through its platinum sponsorship of the ODPi Infosys is working with industry leaders to promote and advance the state of Apache Hadoop® and other enterprise big data technologies. Infosys also wants to grow the adoption of big data technologies in the enterprise by making significant improvements in areas like development and deployment tools, performance and security and is contributing these enhancements back to the community. One of our first contributions to the ODPi is the ODPi reference deployer that our team has built,” said Navin Budhiraja, SVP, Head of Architecture and Technology at Infosys Limited. “Infosys Information Platform (IIP), our open source data analytics platform, supports the ODPi core and the extensive use of open source in IIP reflects the commitment of Infosys to ODPi and the open source community. In addition, multiple other strategic initiatives at Infosys, such as our industry solutions in Banking, Aeronautics and Retail, the Infosys Automation Platform (IAP), and the Industrial Internet Consortium (IIC) testbeds for predictive maintenance are powered by IIP.”

Pivotal
“ODPi accelerates the delivery of Big Data solutions by providing a well-defined platform called ODPi Core, enabling enterprises to build transformative, personalized applications with data at their core. Pivotal’s technology and software development expertise transforms good companies into great software companies. Our collaboration with ODPi will usher a new era of open source Big Data solutions central to an enterprise customer’s digital transformation journey, by arming them with the ability to use data to foster meaningful engagement with their customers,” said Gavin Sherry, Vice President and CTO, Data, Pivotal

Teradata”Teradata is committed to accelerating enterprise adoption of Hadoop. ODPi plays an important role by testing and certifying the Apache Hadoop core. ODPi makes implementations easier and enterprise class –as well as enabling more software tools to work with Hadoop. We have been impressed by the progress ODPi has made towards these goals in such a short amount of time,” said Justin Borgman, VP/GM, Teradata Center for Hadoop.

UNIFi
“UNIFi is excited to be a part of the ODPi. We believe establishing standards and best practices to this rapidly innovating / evolving ecosystem of technology components will be a great step to enabling enterprises to become more data driven. Focusing on delivering value to the business is the promise of this movement and this consortium is furthering that goal,” said Sean Keenan, cofounder and vice president of products at UNIFi.

VMware
“VMware products and services help our customers deliver a consistent environment for building, running and managing any application including big data workloads. ODPi’s work on common, open source infrastructure to advance and accelerate big data adoption aligns well with VMware’s goal,” said Mark Lohmeyer, Vice President Products, Cloud Platform BU, VMware. “VMware is looking forward to participating in this pan-industry effort and advancing Big Data technologies for everyone.”

Xiilab
“Xiilab provides services and software based on Big Data. Our goal is for our users to feel there is no limit to what their application can accomplish. We hope to bring Hadoop and open source to the hands of the community so that the innovation may broaden the achievements of many. Being the first member from South Korea, we wish to have a symbiotic relationship with the community that will help deliver an accessible service to users across the globe,” said Xiilab CEO and President, Woo Young Lee.

zData Inc.
“As a Big Data solution provider, zData Inc. sees the inherent need for unification and collaborative supervision of this rapidly changing platform ecosystem. zData’s largest challenge in providing Hadoop Managed Services has been to successfully deliver a predictable level of service to our customers and partners while still keeping up with the latest community innovations from many disparate open source projects. The ODPi has the opportunity to become the unifying factor, pulling together resources across the entire eco-system, from vendors to customers, to focus on standardizing and unifying these Open Source technologies.”

Zettaset
“Security is still considered a major barrier to broader adoption of Hadoop in the enterprise. To address that problem, Zettaset is providing customers with a proven, commercial-grade, standards-compliant encryption solution which is performance-optimized for Big Data architectures encompassing Hadoop, NoSQL, and other databases while delivering the highest levels of data protection,” said CEO Jim Vogt of Zettaset. “Zettaset and its advanced big data security solutions are aligned with the ODPi for one simple reason.  With a standardized Hadoop distribution built around ODPi, customers now have a much more consistent and predictable technology foundation supported by a wide choice of long-standing systems vendors as well as specialized application providers like Zettaset.  Technology platform stability inherently reduces risk, giving customers greater confidence to deploy Hadoop as a mainstream solution in the enterprise.”

About The Linux Foundation The Linux Foundation is a nonprofit consortium dedicated to fostering the growth of Linux and collaborative software development. Founded in 2000, the organization sponsors the work of Linux creator Linus Torvalds and promotes, protects and advances the Linux operating system and collaborative software development by marshaling the resources of its members and the open source community. The Linux Foundation provides a neutral forum for collaboration and education by hosting Collaborative Projects, Linux conferences including LinuxCon, and generating original research and content that advances the understanding of Linux and collaborative software development. More information can be found at www.linuxfoundation.org.

###

The Linux Foundation and Linux Standard Base are trademarks of The Linux Foundation. Linux is a trademark of Linus Torvalds.

Media Contact
Jennifer Cloer
The Linux Foundation
503-867-2304
Jennifer@Linuxfoundation.org