Category

Announcements

Attunity and ING Join ODPi To Further Open Big Data Ecosystem

By | Announcements

New members support efforts to advance data governance and data science approaches

Berlin, Germany – April 16, 2018 – DataWorks Summit — ODPi, a nonprofit organization accelerating the open ecosystem of big data solutions, today announced that Attunity and ING have joined the initiative to advance data governance and data science approaches.

Many vendors have focused on productizing Apache Hadoop® as a distribution, which led to inconsistency that increased the cost and complexity for application vendors and end-users to  fully embrace Apache Hadoop. Founded in 2015, ODPi is an industry effort to accelerate the adoption of Apache Hadoop and related big data technologies. ODPi’s members aim to accelerate Apache Hadoop adoption through a neutral, industry-wide approach to data governance and data science. Together, they are supporting the mission of creating an open data ecosystem through collaboration with subject matter experts and data platform and tools vendors.

The Big Data market has, in part due to efforts by ODPi and its members, achieved the desired simplification of the Apache Hadoop landscape. However, barriers to broader and more rapid enterprise Hadoop adoption exist and can benefit from a neutral, industry-wide approach to data governance and data science,” said John Mertic, director of program management, ODPi. “We are thrilled to have Attunity and ING on board as ODPi members to help us further these industry-wide approaches.”

The new ODPi members will join a diverse and growing group of members that include well-known Apache Hadoop software companies, service providers and end users, as well as a rapidly growing community.

ING Information Architect and Application Developer, Maryna Strelchuk, and ODPi Director of Program Management, John Mertic, will be co-presenting at DataWorks Summit on The rise of big data governance: Insight on this emerging trend from active open source initiatives.

About the newest members:

Attunity is a leading provider of modern data integration and Big Data management software solutions that enable availability, delivery, and management of data across heterogeneous enterprise platforms in organizations worldwide. Its flagship solution, with change data capture technology, offers real-time data integration and ingestion across all databases, data warehouses, Hadoop and the cloud. Leading businesses choose Attunity to enable data lakes for real-time analytics, and ultimately, maximize the value of their IT and data investments.

“Attunity is excited to become a member of ODPi, helping to set a vision and technology ecosystem for metadata management that will benefit enterprises building modern data architectures,” said Itamar Ankorion, Chief Marketing Officer at Attunity. “Attunity shares ODPi’s belief that automated discovery and maintenance of metadata has to be an integral part of all modern data integration tools like ours that access, change and move information. We look forward to being part of ODPi’s efforts to standardize, support and accelerate growth of the Big Data Ecosystem.”

ING is a global financial institution with a strong European base, offering banking services. We draw on our experience and expertise, our commitment to excellent service and our global scale to meet the needs of a broad customer base, comprising individuals, families, small businesses, large corporations, institutions and governments. Our customers are at the heart of what we do.

“ING decided to become a member of ODPi to help drive standardization around open metadata,” said Ferd Scheepers, Chief Information Architect at ING. “Analytics is one of our strategic priorities, and we believe that standardization of metadata is a key enabler to be successful with analytics. ODPi as an independent group plays a key role in helping standardization across vendors, for ING the key reason to join and support ODPi.”

Additional Resources

About ODPi

ODPi is a nonprofit organization committed to simplification and standardization of the big data ecosystem with a common reference platform called ODPi Core. As a shared industry effort, ODPi members represent big data technology, solution provider and end user organizations focused on promoting and advancing the state of Apache Hadoop® and big data technologies for the enterprise. For more information about ODPi, please visit: http://www.ODPi.org

###

Media Contact:

Natasha Woods

ODPi

(415) 312-5289

pr@odpi.org

ODPi Launches Apache Bigtop Grant Fund Program

By | Announcements

Bigtop “Test Drive” Grant Program to further enterprise-wide production of Apache Hadoop

SAN FRANCISCO, June 13, 2017 – ODPi, a nonprofit organization accelerating the open ecosystem of big data solutions, today announced Apache™ Bigtop “Test Drive” Grant Program, a new grant funding program designed to increase developer involvement in the Apache Software Foundation (ASF) project. Through the program, ODPi is investing $50,000 to fund developer work with the world’s top Apache and big data developers and architects to expand Bigtop’s functionality and usability.

To apply to participate in the Bigtop “Test Drive” Grant Program, submit proposals here by Friday, July 14 at 11:59pm PST.

“By launching this program, we benefit the enterprise end-user, developers, and ultimately build a stronger relationship with the Apache Software Foundation and the big data communities hosted by it,” said Roman Shaposhnik, VP of Technology for ODPi and Apache Hadoop and Bigtop committer and ASF member. “We encourage the community to apply for a Bigtop grant and work alongside ODPi member companies IBM, SAS, Hortonworks, Splunk, GE and others as we strengthen and extend Apache Bigtop.”

Apply to Apache Bigtop “Test Drive” Grant Program

The ODPi Technical Steering Committee (TSC) is looking for Apache Hadoop ecosystem developers and/or big data practitioners building solutions for in-house or external clients to help further the features in Bigtop. This might be software development, developing new teaching materials, documenting best practices, standardizing APIs or doing research. The program will allow developers to hone their skills, build relationships with leading big data companies and earn monetary compensation. Beyond financial support, the TSC can also provide administrative support, promotion and some collaboration tools.

The program will run for six months, beginning August 1, 2017 and ending February 2, 2018. All participants will be given a dedicated mentor from the TSC and will be required to report their progress to the TSC on a monthly basis (first Thursday of each month). The TSC reserves the right to remove participants from the program.

Applicants will need to write a two to four-page proposal that describes the Bigtop problem they want to solve and the funding needed to solve the problem. The TSC will review all proposals and accepted program applicants will be announced Monday, July 31, 2017. Details regarding the proposal, including the submission process, can be found at here.

The ODPi investment means a lot to the Bigtop community,” said Evans Ye, Bigtop’s PMC Chair. “It marks a new milestone that the project not only supports distro vendors, but also, at a higher level, enterprises looking to increase their use of hybrid big data. I believe we’ll have great synergy because ODPi and Bigtop are both committed to making the big data ecosystem more open, connected and relevant.”

A comprehensive packaging, testing, and configuration suite of the leading open source big data components, Apache Bigtop supports a wide range of components/projects, including Hadoop, HBase and Spark. Grants will be awarded for work in the following functional areas: building, continuous integration (CI), testing, deployment, supported platform coverage, and list of supported big data components to expand the platform and set of tools for building standardized big data deployments. By enhancing automation and CI, extending the testing functionality and improving deployment, Bigtop will provide the big data operational predictability that enterprises require.

“In order to confidently expand Apache™ Hadoop® to enterprise-wide production use, businesses need to know that their preferred big data stacks will run predictably and Apache Bigtop can provide this confidence,” said John Mertic, Director, ODPi. “By launching Bigtop “Test Drive” Grant Program, ODPi is helping to add Bigtop functionality that will make the project even more valuable to enterprise end-users.”

Additional Resources

About ODPi

ODPi is a nonprofit organization committed to simplification and standardization of the big data ecosystem with a common reference specification. As a shared industry effort, ODPi members represent big data technology, solution provider and end user organizations focused on promoting and advancing the state of Apache Hadoop® and big data technologies for the enterprise. For more information about ODPi, please visit: http://www.ODPi.org

The Linux Foundation, ODPi and edX Announce New, Free Intro to Apache Hadoop Course

By | Announcements

Massive Open Online Course (MOOC) to provide students with basic knowledge of the leading Big Data processing software

 SAN FRANCISCO – March 30, 2017 – The Linux Foundation, the nonprofit advancing professional open source management for mass collaboration, today announced its newest massive open online course (MOOC) is available for registration. The course, LFS103x – Introduction to Apache Hadoop, is offered through edX, the nonprofit online learning platform launched in 2012 by Harvard University and Massachusetts Institute of Technology (MIT). This free course will begin in early June.

This is the fifth edX MOOC offered by The Linux Foundation. Its first course, Intro to Linux, has reached more than 800,000 students globally and continues to grow in registrations. The others are Intro to Cloud Infrastructure Technologies, Introduction to OpenStack and Introduction to DevOps: Transforming and Improving Operations.

Apache Hadoop is an open source project used for distributed processing of large sets of data. It is used by organizations large and small around the world to manage and analyze the massive amounts of data being created every single second of every day. A large number of additional open source software projects exist that can be installed and run on top of or alongside it to provide additional functionality. According to 451 Research, Hadoop will advance at a 38 percent compound annual growth rate (CAGR) through 2020 and reach $4.4 billion in revenue by 2020.

At the same time, the demand for individuals who have experience managing this platform is also accelerating. According to the IT Skills and Certifications Pay Index research from Foote Partners, “the need for big data skills also continues to lead to pay increases – about 8 percent over the last year,” making this an ideal time for individuals to start a career managing Big Data with Apache Hadoop.

“As innovation across the Hadoop landscape continues to skyrocket, we’re thrilled to provide accessible, vendor-neutral education for the Big Data community,” said ODPi’s Director, John Mertic. “ODPi is committed to reducing ecosystem complexity and, with Roman Shaposhnik leading this ‘Introduction to Apache Hadoop’ edX course, we look forward to sharing insights that make Hadoop manageable for organizations of all sizes.”

LFS103x is taught by Hadoop experts from The Linux Foundation’s ODPi project, which is committed to simplification and standardization of the big data ecosystem with common reference specifications and test suites. Shaposhnik, VP of Technology for ODPi at The Linux Foundation and the course instructor, is also a committer on Apache Hadoop, co-creator of Apache Bigtop, and contributor to various other Hadoop ecosystem projects. He is also an Apache Software Foundation member and a former Chair of Apache Incubator.

Students in the course will learn:

  • The origins of Apache Hadoop and its big data ecosystem
  • Deploying Hadoop in a clustered environment of a modern day enterprise IT
  • Building data lake management architectures around Apache Hadoop
  • Leveraging the YARN framework to effectively enable heterogeneous analytical workloads on Hadoop clusters
  • Leveraging Apache Hive for an SQL-centric view into the enterprise data lake
  • An introduction to managing key Hadoop components (HDFS, YARN and Hive) from the command line
  • Securing and scaling your data lakes in multi-tenant enterprise environments

“In today’s high-tech world, more data is created every day and increasingly organizations need professionals qualified to analyze it,” said edX CEO and MIT Professor Anant Agarwal. “We are pleased to again partner with The Linux Foundation to increase access to in-demand education, helping to bring Apache Hadoop expertise into the Big Data industry.”

The course includes six chapters, each with a short graded quiz at the end. A final exam is also required in order to complete the course. Students may take the complete course at no cost, or add a verified certificate of completion for $99.

For more information on The Linux Foundation’s training and certification programs, please visit:  http://training.linuxfoundation.org.

 About The Linux Foundation

The Linux Foundation is the organization of choice for the world’s top developers and companies to build ecosystems that accelerate open technology development and commercial adoption. Together with the worldwide open source community, it is solving the hardest technology problems by creating the largest shared technology investment in history. Founded in 2000, The Linux Foundation today provides tools, training and events to scale any open source project, which together deliver an economic impact not achievable by any one company. More information can be found at www.linuxfoundation.org.

About ODPi

ODPi is a nonprofit organization committed to simplification and standardization of the big data ecosystem with a common reference specification. As a shared industry effort, ODPi members represent big data technology, solution provider and end user organizations focused on promoting and advancing the state of Apache Hadoop®and big data technologies for the enterprise. For more information about ODPi, please visit: http://www.odpi.org.

###

Media Contact:

Natasha Woods

ODPi

(415) 312-5289

pr@odpi.org

ODPi Grows Its Membership With International Set of Data-Driven Companies

By | Announcements

China Mobile, High Octane, Innovyt and LizardFS join mission to solidify ODPi as the essential Open Source companion for enterprise-wide production Hadoop

SAN JOSE – Strata + Hadoop World, March 14, 2017 – ODPi, a nonprofit organization accelerating the open ecosystem of big data solutions, today announced China Mobile, High Octane, Innovyt and LizardFS have joined as members to help companies unlock value from disparate data and continue advancing the standardization of Apache™ Hadoop® and related big data solutions.

According to Brian Hopkins, VP & principal analyst at Forrester Research, a revenue model built by the firm – which included speaking to hundreds of firms and pouring over three years of survey data with nearly 10,000 responses – conservatively forecasted that insights-driven businesses would earn about $400 billion in 2016; however, by 2020 they will be making over $1.2 trillion a year due to an astonishing compound annual growth rate between 27 percent and 40 percent.

“ODPi is committed to serving the big data ecosystem by facilitating standardization and ensuring Hadoop reaches its full potential as an enterprise-wide production big data platform,” said John Mertic, Director, ODPi. “We’re thrilled to have a breadth of companies from China, Belgium, Poland and the U.S. join our efforts in validating big data as a sustainable, long-term area of investment for organizations worldwide.”

These new members bring the ODPi network of collaborative big data startups, enterprise service providers and software-driven end users to more than 35 companies – signifying the ecosystem’s growing desire to make Hadoop accessible and ready for every organization around the world.

About the newest members:

China Mobile is the biggest telecom operator in the world. The open source offerings extended by China Mobile Software Technology, including Hadoop product and professional services, serve more than 826 million customers.

“Becoming a member of ODPi will not only make customers’ choices among various Hadoop distributions far easier, but it will also help to build their confidence in China Mobile’s Hadoop product,” said Shaoling Sun, Executive Vice President of China Mobile Software Technology. “As industry compliance is the top priority of our big data products, we hope to soon have a unified, standard Hadoop version of the interface built on the ODPi reference specification.”

High Octane is a Belgium-based consulting firm, focused on companies’ enterprise architecture to help strategize, materialize and execute on their big data visions.

“High Octane is proud to join ODPi, as the organization is close to the upstream projects and focuses on a long-term operational view,” said Philippe Back, founder of High Octane. “As long-time believers in open source software, ODPi is a great vendor-neutral venue for us to share our Hadoop insights and learn from the ecosystem feedback provided by end users and SIGs from the field.”

Innovyt is a big data consulting company specializing in advanced analytics, cloud and data science focused on ensuring its customers can implement modern data-driven applications using big data technologies. Its team is committed to building a set of solution-centric frameworks that will expedite the implementation of leading platforms, like Hadoop.

“ODPi’s initiative to create interoperability and compliance for production Hadoop are incredibly meaningful – as these will provide our customers with standards, best practices and a common language,” said Vineet Kumar, founder of Innovyt. “We look forward to learning and contributing to open standards through our partnership with ODPi.”

LizardFS – an open-source Distributed File System licensed under GPLv3 – was developed and distributed by Skytechnology in Warszawa, Poland. The scalable, fault-tolerant and highly-available file system ensures security by storing all data in many replicas spread across all available nodes and can be used to build affordable storage cluster.

“Our membership to ODPi will enable LizardFS to take part in creating an industry standard for big data solutions and resolving challenges the industry faces, especially as it relates to the storage of data,” said Simon Haly, CSO of LizardFS. “We’re eager to join the initiative’s efforts and look forward to the direction ODPi gives companies just starting their journey in this thriving ecosystem.”

Additional Resources

About ODPi

ODPi is a nonprofit organization committed to simplification and standardization of the big data ecosystem with a common reference specification. As a shared industry effort, ODPi members represent big data technology, solution provider and end user organizations focused on promoting and advancing the state of Apache Hadoop® and big data technologies for the enterprise. For more information about ODPi, please visit: http://www.odpi.org

###

Media Contact:

Natasha Woods

ODPi

pr@odpi.org

ODPi Publishes First Operations Specification To Provide Developers Consistency Across Application Management Tools

By | Announcements

Leveraging Apache Ambari, Apache Hive and Hadoop Compatible File System support, ODPi 2.0 Release Standardizes Deployment Model for Enterprise Big Data Solutions   

Seville, Spain – Apache: Big Data Europe, November 14, 2016 — ODPi, a nonprofit organization accelerating the open ecosystem of big data solutions, today announced the availability of ODPi 2.0, which includes the first release of the ODPi Operations Specification and the Runtime Specification 2.0, to standardize the development model for big data solution and application providers and help enterprises improve installation and management of Hadoop-based applications.

With more than 30 members, including recently announced DriveScale, Redoop, and Xavient Information Systems, ODPi is focused on simplification and standardization within the big data ecosystem and further advancing the work of the Apache Software Foundation. Designed to make it easier to create big data solutions and data-driven applications, ODPi adds Apache Hive and Hadoop Compatible File System support (HCFS) as part of the ODPi Runtime Specification 2.0. Additionally, the ODPi 2.0 release includes Operations Specification 1.0, which provides standard guidelines for application management tools serving as reference platforms; including Apache Ambari.

“With the release of the first Operations Specification, ODPi is moving standardization forward for Apache projects in a pragmatic, fluid way that embraces developer input,” said John Mertic, Director of ODPi. “ODPi specifications are based input from developers and enterprises and how they are actually big data technologies in production environment and address real issues they’ve encountered. Our technical team developed this latest release knowing that the SQL layer, backend storage, and how applications should be installed, managed and configured in a Apache Hadoop cluster are important to them. We’ll continue to iterate on previous releases and seek industry input to ensure that we are tackling the critical issues that benefit the wider big data ecosystem.”

Key ODPi Operations Specification 1.0 Technical Features

The ODPi Operations Specification 1.0 provides standard guidelines for application management tools, with Apache Ambari as a reference platform, the Apache Software Foundation project for provisioning, managing, and monitoring Apache Hadoop clusters. By providing common expectations in guidelines, developers are able to create data-driven applications for all management tools used by platform providers. For big data solution and application providers, this minimizes the complexity, cost and training needed to build big data applications.

ODPi community worked closely with the Apache Ambari community to develop the Operations Specification, ensuring backward compatibility with the standardization and alignment with the community’s needs. ODPi community further designed this spec such that other management tools could attain compliance.

Similar to Spark, Ambari is a rapidly changing project. In working on the latest release, ODPi’s technical team collected substantial Ambari institutional knowledge, which they’ve contributed to Ambari. The reference manual will help developers more easily write an application for Ambari to manage their applications.

Key ODPi Runtime Specification 2.0 Technical Features

ODPi Runtime Specification 2.0 adds Apache Hive and Hadoop Compatible File System support (HCFS) components to Yarn, MapReduce and HDFS from ODPi Runtime Specification 1.0. HCFS support will enable storage and cloud vendors to leverage ODPi standards, empowering them to use their native storage solutions as part of an ODPi Runtime Compliant Hadoop Platform and reduce the incompatibilities that end users face. By including Apache Hive, ODPi will reduce SQL query inconsistencies across Hadoop Platforms. ODPi based its work on Hive version 1.2 and has included core functionality that will continue to behave in a standard way for future versions of Apache Hive. For more on this addition, read ODPi technical steering committee chair Alan Gates’ blog.

ODPi Compatibility and Interoperability

Several Apache Hadoop platform and big data solution and application providers; including Ampool, Hortonworks, IBM, Pivotal, and SAS have committed to testing against ODPi 2.0 to become ODPi Compliant and ODPi Interoperable. They have the ability to test against both the Operations Specification 1.0 and Runtime Specification 2.0 separately; offering greater simplicity for big data solution and application providers. This option provides end-users greater choice and flexibility by fostering an open big data ecosystem that transcends traditional vendor alliances.

For more on how ODPi is helping enterprises boost the value they get from Hadoop and Big Data, read Rouda’s recent whitepaper and listen to the accompanying webinar.

Comments from Members

Ampool

“Since the founding of ODPi, Ampool has been committed to helping to drive standardization both in the organization and in how our data services interoperate with multiple Hadoop platforms. With the release of the first operations specification, we’re looking forward to submitting Ampool’s Active Data Store for compliance in the coming months.” – Milind Bhandarkar, Ph.D., Founder and CEO, Ampool

IBM

“There is a major shift occurring on how data is treated within their organization. Fundamentally, it is no longer about the persistent stores, data in Hadoop, data in operational database and real-time streaming. It is about how that data is accessed in trust and used within an organization. By working with ODPi and committing to provide these organizations with a compliant platform they can count on and interoperable software that sits on top of Hadoop, including IBM Big SQL, IBM SPSS Analytic Server, IBM Big Replicate, and others, we are helping our customers build their businesses.” – Ritika Gunnar, Vice President of Offerings, IBM Analytics

Pivotal

“Complying with the latest version of the ODPi specification simplifies how Apache HAWQ can query the vast quantities of data in the popular Apache Hive format, and allows us to seamlessly integrate configuration and administration through Apache Ambari. ODPi is allowing us to roll out compatibility features with the Apache Hadoop ecosystem at a much faster pace.” – Jacque Istok, Head of Data Engineering, Pivotal Software

SAS

“As an ODPi member, we are reinforcing our commitment to ensuring that SAS applications work with and exploit the Hadoop distribution of our customers’ choice – while being able to bank on the stability and quality expected in demanding business environments. The availability of ODPi 2.0 allows us to more efficiently support our customers, while also enhancing the installation and management of the SAS application within their Hadoop environment.” – Craig Rubendall, Vice President of Platform R&D, SAS

Additional Resources

About ODPi

ODPi is a nonprofit organization committed to simplification and standardization of the big data ecosystem with a common reference specification. As a shared industry effort, ODPi members represent big data technology, solution provider and end user organizations focused on promoting and advancing the state of Apache Hadoop® and big data technologies for the enterprise. For more information about ODPi, please visit: http://www.odpi.org.

###

Media Contact:

Natasha Woods

ODPi

(415) 312-5289

pr@odpi.org

DriveScale, Redoop, and Xavient Information Systems Join ODPi To Create Interoperable Big Data Ecosystem

By | Announcements

New members rally around compliance efforts to help organizations optimize and streamline Apache Hadoop implementations

Seville, Spain – Apache: Big Data Europe, November 14, 2016 – ODPi, a nonprofit organization accelerating the open ecosystem of big data solutions, today announced DriveScale, Redoop, and Xavient Information Systems have joined the organization to advance the simplification and standardization of Apache™ Hadoop® and related technologies.

With the incredible growth of Hadoop – forecasted to surpass $16 billion by 2020 – and related Apache project evolution, it has become difficult for developers to keep up with the pace of innovation. To make adoption of the platform more appealing to organizations looking to invest in big data technologies, ODPi members are committed to simplifying development and compatibility testing for applications.

“ODPi’s work to ensure interoperability of applications across a wide range of commercial Hadoop platforms is gaining momentum thanks to ongoing membership growth,” said John Mertic, director of program management, ODPi. “Hadoop has become a crucial part of any enterprise’s big data strategy. We are helping to mitigate complexity in the Hadoop ecosystem by facilitating standardization across big data technologies – ultimately spearheading newer and greater innovations.”

ODPi membership has grown to include more than 30 companies – encompassing industry-leading Apache Hadoop software companies, big data startups, enterprise service providers and end users.

About the newest members:

DriveScale provides a smarter way to build infrastructure for scale-out systems like Hadoop. Its composable data center architecture is provided via a set of on-premises and SaaS tools that coordinate between multiple levels of infrastructure. With DriveScale, companies can more easily support Hadoop deployments of any size, as well as other modern application workloads.

“Enterprises have varying big data needs that require flexible and interoperable platform components,” said Gene Banman, CEO of DriveScale. “Becoming a member of ODPi will allow us to better maximize data center efficiency for Hadoop with interoperability for enterprise-grade deployments.”

Redoop is a big data platform founded in China, devoted to ensuring enterprises reap the benefits of big data technology. Revolving around Hadoop Common to ensure a corresponding distributed system, Redoop provides the underlying optimization, system management, and data management to help enterprises build their own tailored data systems.

“Redoop believes that ODPi’s mission of creating a more open and interoperable ecosystem for current and potential Apache Hadoop users is meaningful to the entire big data community,” said XiaoJun Tong, founder of Redoop. “We look forward to joining the initiative and collaborating with its members for the benefit of our customers.”

Xavient is a global IT consulting and software services company, focusing on transforming business ideas into effective solutions. Xavient was among the first five solution and application providers to pledge a commitment to the ODPi Interoperable Compliance Program for its DiP (Data Ingestion Platform) – a real-time data analysis application – ensuring its ability to successfully run on multiple ODPi Runtime Compliant Platforms.

“Xavient is committed to providing customers with tailored capabilities and solution flexibility and making our real-time data analysis solution interoperable with ODPi Compliance Program applications was a natural next step,” said Neeraj Sabharwal, head of cloud, data & analytics at Xavient Information Systems. “We are thrilled to become a member of ODPi to contribute to an open big data ecosystem that transcends big vendor agendas.”

Additional Resources

About ODPi

ODPi is a nonprofit organization committed to simplification and standardization of the big data ecosystem with a common reference specification. As a shared industry effort, ODPi members represent big data technology, solution provider and end user organizations focused on promoting and advancing the state of Apache Hadoop® and big data technologies for the enterprise. For more information about ODPi, please visit: http://www.odpi.org.

###

Media Contact:

Natasha Woods

ODPi

(415) 312-5289

pr@odpi.org

Big Data Applications Now ODPi Interoperable, Provide Increased Simplicity To Hadoop Ecosystem

By | Announcements

Leading industry companies first to comply with big data industry standard; ODPi Runtime Specification 2.0 released with Apache Hive and HCFS

New York City, N.Y. – Strata + Hadoop World, September 27, 2016 — ODPi, a nonprofit organization accelerating the open ecosystem of big data solutions, today announced that solution and application providers DataTorrent, IBM, Pivotal, SAS, Syncsort, WANdisco, and Xavient have committed to the ODPi Interoperable Compliance Program. This makes it easier for enterprises to choose and adopt big data technologies and ensures these applications are interoperable across a wider range of commercial Apache™ Hadoop® Platforms.

“The implementation of complex environments is a major obstacle to organizations implementing big data. The new ODPi Interoperable Compliance Program is a big step in the right direction to help organizations mitigate the risk associated with Hadoop implementations,” said John L. Myers, Managing Research Director of Business Intelligence at Enterprise Management Associates. “Adding Apache Hive and Hadoop Compatible File System (HCFS) to the Runtime Specification 2.0 will enable the initiative to make continued progress toward the interoperability of these components and it will make Hadoop more appealing to the organizations looking to make the leap into big data initiatives.”

A dozen ODPi Interoperable applications have been tested to successfully run on multiple ODPi Runtime Compliant Platforms. Currently, Altiscale, ArenaData, Hortonworks, IBM, and Infosys Apache Hadoop Platforms are ODPi Runtime Compliant, offering greater simplicity for big data solution and application providers. ODPi programs give end-users greater choice and flexibility by fostering an open big data ecosystem that transcends traditional vendor alliances.

Additionally, ODPi announced today that the ODPi Runtime Specification 2.0 will add Apache Hive and Hadoop Compatible File System support (HCFS). These components join Yarn, MapReduce and HDFS from ODPi Runtime Specification 1.0. By including Apache Hive, which read, write, and manage large datasets residing in distributed storage, ODPi will reduce SQL query inconsistencies across Hadoop Platforms. ODPi will base its work on Hive version 1.2 and ensure there is core functionality that will continue to behave in a standard way for future versions of Apache Hive. HCFS support will enable storage and cloud vendors to leverage ODPi standards, letting them leverage their native storage solutions as part of an ODPi Runtime Compliant Hadoop Platform and reduce the incompatibilities that end-users face.

“As our work continues to complement the Apache Software Foundation, ODPi is helping the Hadoop ecosystem become more valuable to those who are testing and building big data applications. Through a common specification, we are enabling developers to easily write applications that sit on top of big data stacks, lowering the costs of interoperability across systems. These compliant applications should need little to no re-engineering to run on other ODPi Runtime Compliant platforms,” said John Mertic, Director of ODPi. “For enterprises, the benefits are increased efficiency, flexibility and smoother maintenance.”

ODPi Interoperability Guidelines and Testing

The ODPi Interoperable Compliance Program is an ongoing commitment between Apache Hadoop platforms, big data solution and application providers, and ODPi to ensure that the best practices outlined by the ODPi Specification Releases are adhered to for the benefit of enterprises; providing them flexibility of deployment and choice of solution. The ODPi Interoperable and ODPi Runtime Compliance programs guarantee that any software or application deemed ODPi Interoperable can run on any of the ODPi Runtime Compliant Apache Hadoop platforms. This makes operating a heterogeneous big data environment easier for enterprises.

Enterprises can easily obtain details regarding which versions of an application are ODPi Interoperable and which Apache Hadoop platforms are compliant by visiting ODPi For End Users.

Current ODPi Interoperable Applications include:

Big data solution or application providers Product/Version
SAS Base SAS v9.4m3
SAS SAS/Access Interface to Hadoop Version v9.4m3
WANdisco WANdisco Fusion 2.9; Always-On Enterprise Class Availability and Performance for Hadoop
Syncsort Syncsort DMX-h 9.0
DataTorrent DataTorrent Real Time Streaming
IBM IBM SPSS Analytic Server
IBM IBM Big SQL
IBM IBM Big Replicate
IBM IDR for Apache Hadoop
Pivotal Pivotal HDB powered by Apache HAWQ
Xavient DiP (Data Ingestion Platform)

Comments from Members

IBM

“ODPi is providing a common platform to develop big data apps, enabling interoperability across different distributions and application offerings. In that spirit, IBM is ensuring many of our Apache Hadoop related offerings are interoperable including IBM Big SQL, IBM SPSS Analytic Server, IBM Big Replicate, and others. By supporting interoperable applications, the value to clients of ODPi Compliant distributions is being enhanced.” – Ritika Gunnar, Vice President of Offerings, IBM Analytics

Pivotal

“Working with our customers, Pivotal developed Pivotal HDB based on Apache HAWQ, a fully SQL-92 compliant, native SQL engine running on Hadoop. Compatibility and standardization across multiple Hadoop distributions is among the many reasons why we focus on ODPi Compliant offerings, allowing us to more rapidly test, deploy, and innovate.” – Jacque Istok, Head of Data Engineering at Pivotal Software

SAS

“SAS is pleased to announce that we are the first software vendor to achieve the distinction of ODPi Interoperable. By declaring that SAS interfaces with Apache Hadoop in demonstrable, standard ways, we can reduce our customers’ risk, simplify testing complexity and speed time to value for anyone building or deploying SAS applications.” – Craig Rubendall, Vice President of Platform R&D at SAS and ODPi board member

Syncsort

“We are thrilled to be able to provide our customers with Syncsort DMX-h 9.0 as an ODPi Interoperable solution. Interoperability helps to streamline the development and testing process, which, in turn allows us to focus our resources on supporting our customers’ big data requirements.” – Arnie Farrelly, Vice President of Big Data Global Product Support and Services at Syncsort

WANdisco

“ODPi has eliminated significant risk for both developers and end users, by providing a set of standards that makes it possible for an ecosystem of interoperable, enterprise class big data applications to evolve and flourish. WANdisco (LSE:WAND) is extremely pleased to have achieved ODPi Interoperable compliance for Fusion 2.9 and become part of that ecosystem. Our patented replication technology reduces risk even further, by ensuring ODPi Compliant Hadoop Platforms deliver continuous availability and performance, benefitting every application deployed on them.”  – Jim Campigli, COO and Co-Founder at WANdisco

Xavient

“Customer-driven requirements resulted in the creation of DiP(Data Ingestion Platform), which is based on Hadoop and leverages many open source technologies like Kafka, Spark Streaming, Storm and components like HDFS, Hive, Zeppelin and HBase. We are focused on ODPi Compliant offerings because it provides us with a platform to test and deploy DiP on multiple Hadoop versions with an efficient testing process. DiP is a co-dev opportunity, so ODPi gives a structure to test the tool based on customer investment and use case.” – Neeraj Sabharwal, Director Cloud and Big Data at Xavient Information System

Additional Resources

About ODPi

ODPi is a nonprofit organization committed to simplification and standardization of the big data ecosystem with a common reference specification. As a shared industry effort, ODPi members represent big data technology, solution provider and end user organizations focused on promoting and advancing the state of Apache Hadoop® and big data technologies for the enterprise. For more information about ODPi, please visit: http://www.odpi.org.

###

Media Contact:

Natasha Woods

ODPi

(415) 312-5289

pr@odpi.org

Apache™ Hadoop® Distributions Now ODPi Runtime Compliant

By | Announcements

Altiscale, ArenaData, Hortonworks, IBM, and Infosys first to comply with industry standard for big data applications

SAN FRANCISCO, June 27, 2016ODPi, a nonprofit organization accelerating the open ecosystem of big data solutions, today announced that the Apache™ Hadoop® distributions products from Altiscale, ArenaData, Hortonworks, IBM, and Infosys are now ODPi Runtime Compliant. This ensures that their distributions provide big data application vendors a consistent set of base level expectations as defined by ODPi Runtime specification.

“Our members represent all key roles in the big data ecosystem – Apache Hadoop distros, ISVs, system integrators, solution providers and end users – and have brought their real-world experience in helping to align enterprise demands with the developer community,” said John Mertic, director of program management for ODPi. “Having Altiscale, ArenaData, Hortonworks, IBM, and Infosys declare compliance with the ODPi Runtime Specification is a strong step toward simplifying and standardizing the big data ecosystem to accelerate the delivery of business outcomes.”

ODPi’s work complements the Apache Software Foundation and helps companies use Apache Hadoop more effectively. The organization’s aim is to develop a cross-industry standard that enables developers to easily write applications that sit on top of big data stacks, lowering the costs of interoperability across systems.

ODPi Compliant Guidelines and Testing

Descending from Apache Hadoop 2.7, the Runtime Specification covers HDFS, YARN, and MapReduce components. It specifies how Apache components should be installed and configured and provides a set of tests for validation to make it easier to create big data solutions and data-driven applications.

To become ODPi compliant, the Apache Hadoop distributions submitted test results through a GitHub repository for specific products. To assist with compliance, ODPi provides a test suite and a reference build. The ODPi validation test suite guarantees that a distribution of Apache Hadoop from a compliant vendor or an end-user deployment of Apache Hadoop complies with the ODPi-defined specifications. The test framework and self-certification aligns closely with the Apache Software Foundation by leveraging Apache BigTop for comprehensive packaging, testing, and configuration.

Participation in the ODPi is open to anyone, with all work being done is public on GitHub. Developers can join the conversation on the mailing lists or Slack channel, as well as give feedback on the spec or provide tests for validating ODPi compliance.

What’s Next for ODPi

The ODPi Operations Specification to help enterprises improve installation and management of Apache Hadoop and Apache Hadoop-based applications will be available later this year. The Operations Specification leverages Apache Ambari as a reference project. Apache Ambari is an ASF project for provisioning, managing, and monitoring Apache Hadoop clusters.

Comments from Members

Altiscale

“The Apache Hadoop ecosystem is both expanding and evolving rapidly, which not only makes it hard for app developers to keep up with the wide variety of projects, it also makes it challenging for enterprise customers to commit to a Hadoop platform and agree on the capabilities that they need. The ODPi standard specification addresses these issues, providing a solid foundation and peace of mind to developers and enterprise customers. At Altiscale, we are excited to announce that the Altiscale Data Cloud is now officially ODPi Runtime compliant.” – Mike Maciag, COO at Altiscale

ArenaData

“As a small company running Apache Hadoop distributions and looking to expand in the Russian market, becoming ODPi Runtime compliant will help us provide current and future customers reassurance that their developing and testing applications will work across multiple platform providers.” – Dmitry Komarov, head of sales and marketing at ArenaData

EMC Corporation

“The compliance of various distributions and application vendors on the ODPi runtime is a milestone. It provides standardization and portability for developers as well as ISVs and cloud administrators. Ultimately the goal is to provide increased efficiency and lower maintenance impact. Our customers can expect much more flexibility in their deployments and choice of solutions.” – Smiti Sharma,  Product Lead, Virtustream Cloud Services, EMC

SAS

“With Apache Hadoop distributions becoming ODPi Runtime compliant, we are able to focus on developing innovative applications and solutions for our customers that are compatible with the Hadoop ecosystem.” – Craig Rubendall, vice president of platform R&D, SAS

Hortonworks

“We are proud to become ODPi Runtime compliant, which aligns with our Hortonworks Data Platform certification program. The ODPi common reference specs should lower the investment for application builders and increase the number of applications in the ecosystem, allowing us to provide greater value to our customers.” – Alan Gates, co-founder, Hortonworks

IBM

“ODPi is providing a common specification to develop big data apps, opening up choices for developers by enabling interoperability across different distribution and application offerings. Having this platform to count on, starting with a small, specific and consistent packaging model that lives within the ecosystem, is extremely valuable to application developers. This will be a key differentiator for IBM to serve customers, which is why we have become ODPi Runtime compliant.” – Ritika Gunnar, Vice President of Offerings, IBM Analytics

Infosys

“Infosys is focused on growing the adoption of big data technologies in the enterprise by making significant improvements in areas like development and deployment tools, performance and security. By becoming ODPi Runtime compliant, we are providing a compatible solution to our enterprise clients in Banking, Aeronautics and Retail.” – Navin Budhiraja, SVP, Head of Architecture and Technology at Infosys Limited

Additional Resources

About ODPi

ODPi is a nonprofit organization committed to simplification and standardization of the big data ecosystem with a common reference specification. As a shared industry effort, ODPi members represent big data technology, solution provider and end user organizations focused on promoting and advancing the state of Apache Hadoop® and big data technologies for the enterprise. For more information about ODPi, please visit: http://www.odpi.org.

###

Media Contact:

Natasha Woods
ODPi
(415) 312-5289
pr@odpi.org

ODPi Becomes Gold Sponsor of The Apache Software Foundation

By | Announcements

asf

Sponsorship cements ODPi’s support for the upstream Apache® Hadoop® development community

Vancouver, B.C. – Apache: Big Data, May 11, 2016 — ODPi, a nonprofit organization accelerating the open ecosystem of big data solutions, today announced it is a gold sponsor of The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 open source projects and initiatives. ODPi joins existing ASF sponsors: Hortonworks, IBM, Pivotal, and WANDisco, who are also ODPi members, in the industry effort to advance open source projects for the big data ecosystem.

Founded in February 2015, ODPi has always strived to build upon the innovative work of the ASF to help create a thriving and expanding Big Data ecosystem built around the success of Apache® Hadoop®. ODPi is committed on simplifying this growing ecosystem through the integration and standardization of a common reference platform called the ODPi Core, hence focusing on how to bring and deploy the different Apache projects together.

“We are pleased to welcome ODPi to the ASF Sponsorship program and their support of Apache big data projects,” said ASF Vice Chairman Greg Stein. “As analysts project that enterprises will fully embrace the Apache Hadoop ecosystem, Sponsor support is even more vital to our success in spearheading industry-leading innovations that are developed in a trusted, community-driven environment.”

ODPi has followed Apache big data projects, included Apache Hadoop in its Runtime Spec and Apache Bigtop in its test suite out this past March, as well as Apache Ambari in its Operation Spec due out this summer. Additionally, we are evaluating for future specs several other projects critical to advancing big data applications or providing the greatest impact on the greatest number of big data deployments. Guided by community and industry input, as a new crop of big data projects developed at the ASF mature, ODPi plans to incorporate additional upstream projects into ODPi Core.

“We are excited to sponsor the Apache Software Foundation to ensure ODPi is supporting development of the many projects in the Hadoop ecosystem,” said John Mertic, director of ODPi. “ODPi complements the work done in the Apache projects by focusing on easing integration and standardization for downstream application vendors and end-users that build upon Apache Hadoop. We believe our focus on the downstream Hadoop ecosystem helps oxygenate the big data market and stimulate growth.”

Additional Resources

About ODPi

ODPi is a nonprofit organization committed to simplification and standardization of the big data ecosystem with a common reference platform called ODPi Core. As a shared industry effort, ODPi members represent big data technology, solution provider and end user organizations focused on promoting and advancing the state of Apache Hadoop® and big data technologies for the enterprise. For more information about ODPi, please visit: http://www.odpi.org.

“Apache”, “Ambari”, “Apache Ambari”, “Bigtop”, “Apache Bigtop”, “Hadoop”, and “Apache Hadoop” are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

###

Media Contact:

Natasha Woods
ODPi
(415) 312-5289
pr@odpi.org

4C Decision, ArenaData, and AsiaInfo Join ODPi To Promote Open Big Data Ecosystem

By | Announcements

New members support efforts to create open standards for Apache Hadoop to accelerate delivery of data-driven applications

SAN FRANCISCO, April 26, 2016ODPi, a nonprofit organization accelerating the open ecosystem of big data solutions, today announced that 4C Decision, ArenaData, and AsiaInfo, have joined the initiative to advance efforts to create a common reference specification called ODPi Core.

Many vendors have focused on productizing Apache Hadoop® as a distribution, which has led to inconsistency that increases the cost and complexity for application vendors and end-users to  fully embrace Apache Hadoop. Founded last year, ODPi is an industry effort to accelerate the adoption of Apache Hadoop and related big data technologies. ODPi’s members aim to streamline the development of analytics applications by providing a common specification with reference implementations and test suites.

“A wide range of ISVs are showing broad support and interest in ODPi’s work to build a common ODPi Core specification,” said John Mertic, director of program management, ODPi. “By continuing to add members, ODPi is raising the bar for standardization and interoperability across Apache Hadoop and big data technologies, which will enable organizations to more easily build enterprise-class applications and innovate higher up in the stack.”

ODPi is integrating a variety of upstream Apache projects, working across the Apache ecosystem to create a downstream reference specification to enable new Big Data solutions to be built.

This model work has worked with countless open source technologies experiencing rapid growth and ODPi aims to leverage this same model to increase adoption and open up opportunities for innovation on top of an already strong Apache Hadoop community.

The three new ODPi members will join a diverse and growing group of members that includes well-known Apache Hadoop software companies, service providers and end users, as well as a rapidly growing community of more than 35 maintainers from 30 companies.

About the newest members:

4C Decision is a full stack data, analytics, and technology service provider that helps companies establish best practices to store, manage and analyze their data efficiently. 4C Decision applies their extensive business intelligence experience across various industries to ensure their data meets the industry standards, such as UMDP (Uniform Mortgage Data Program) and Government Policy, such as HMDA (Home Mortgage Disclosure Act).

“4C Decision strives to provide business value for our customers by applying the most cutting edge technologies for big data, allowing them to foresee their next business decisions,” said Cupid Chan, managing partner at 4C Decision. “We see ODPi as a vital component to the big data ecosystem that will enable the future of business. We are excited to improve our client solutions by leveraging standardized ODPi components.”

ArenaData is the first open analytical platform leveraging 100 percent open components to replace legacy platforms or SNMP databases that are unable to scale at the speed modern business requires. ArenaData enables enterprises – that require an affordable solution for data gathering and analysis – to transform their ideas into new solutions and to unlock the potential of Big Data.

“As a small company running Apache Hadoop distributions and looking to expand in the Russian market, we believe ODPi’s work to bring standardization to Apache Hadoop will be very beneficial to the entire ecosystem,” said Dmitry Komarov, head of sales and marketing at ArenaData. “We look forward to collaborating with ODPi members and advancing an open ecosystem, which will help drive speed-to-innovation for our customers and beyond.”

AsiaInfo is a leading provider of Internet-based systems and software for the telecommunications industry. With more than 14,000 employees worldwide, the company provides a full suite of business and operations support systems and professional services to solve the business challenges of the future. Through its innovative BSS/OSS, Big Data, and security software products, AsiaInfo is driving the transformation of the telecoms industry into the new business Internet era.

Additional Resources

About ODPi

ODPi is a nonprofit organization committed to simplification and standardization of the big data ecosystem with a common reference platform called ODPi Core. As a shared industry effort, ODPi members represent big data technology, solution provider and end user organizations focused on promoting and advancing the state of Apache Hadoop® and big data technologies for the enterprise. For more information about ODPi, please visit: http://www.odpi.org

###

Media Contact:

Natasha Woods
ODPi
(415) 312-5289
pr@odpi.org