Category

ODPi Egeria

ODPi Egeria project news and updates

ODPi Member Spotlight: Interview with Ferd Scheepers, Chief Information Architect, ING

By Blog, ODPi Egeria

The success that ODPi has achieved as a nonprofit organization committed to simplification and standardization of the big data ecosystem is driven by the dedication of our member organizations and individuals. The ODPi Member Spotlight series interviews key ODPi contributors for a conversation exploring why they participate in ODPi, seeking to learn more about the individuals whose efforts are accelerating the development of today’s Big Data ecosystem, standards and solutions.


We recently spoke with Ferd Scheepers, Chief Information Architect for ING, to discuss his involvement with ODPi. In his role as ING’s global Chief Information Architect, Ferd has driven ING’s journey to becoming a data driven company for the last 5 years, defining ING’s Data Lake architecture for information management. He is championing the Apache Atlas and ODPi open metadata initiatives, and took time to share his vision, ideas and what motivates his contributions to ODPi–along with insight on how ING benefits from being an active member.

Tell me about your job position and what you are responsible for at ING?

For the last five years, I have been working as the global Chief Information Architect of ING. In this role, I am responsible for creating the Information Architecture for ING, which is becoming more and more important as we pursue the ambition of becoming a true data-driven organisation. We have created the ING data lake architecture, which is the main vehicle for ING to implement a fully metadata-driven data landscape, where all the data in the organisation is known. By known we mean not only where the data is, but also the data quality, the meaning, the owner of the data, and the full lineage from where the data comes to life, to any place the data is consumed, either by ING employees or by external parties like our regulators.

What is your involvement with ODPi? Tell us about the role you’ve played, your contributions, goals, and interests.

We got involved with ODPi in early 2017. At that time we had started together with IBM and Hortonworks to drive an Open Metadata initiative to define a set of open metadata standards, and build both a reference implementation for an Open Metadata compliant Metadata repository and the Open Metadata Highway. The Open Metadata Highway is a set of (Open Metadata Repository) Services that let different metadata repositories talk to each other in order to exchange metadata. On top of OMRS there is a set of (Open Metadata Access) Services, that enable dedicated applications or UIs specific for different personas in the organisation to consume services from the entire metadata landscape.

ODPi as an existing vendor-neutral organisation came in the picture as the most logical home for this open standard. Apache Atlas was chosen for the reference implementation for an Open Metadata compliant metadata repository, and the Open Metadata Highway was developed as the Egeria project within ODPi.

Why does ING see value in this work that ODPi is providing a vendor-neutral home for?

When ING got involved in driving this Open Metadata initiative, we knew that making such an initiative succeed requires several things. A willingness of several vendors to join together to make it a success. At least one company (preferably more) that represents the consumers of these vendor solutions, to explain the need for such a standard from a consumer perspective. And an open, vendor-neutral and respected community to be a home for the standard.

IBM and Hortonworks were involved from day one representing the vendors. ING took on the role to be the catalyst to bring them together. Not just as a voice of the customer, we decided to sit in the driving seat and have a full team contribute to developing this open standard. ODPi already being a very active group in steering the standardisation around Hadoop distributions seemed the logical choice for a home for the work that we were doing. Both because ODPi already had most of the facilities we needed, and because many of the vendors we wanted to join in this initiative were already a member.

ING also became a full member of ODPi in 2017 to support the valuable work ODPi is doing. We very much value the platform ODPi offers for developing the open standard, but more importantly, we value the community of vendors it brings us, and the exposure we get from ODPi to get the open standard known within a bigger community of both vendors and consumers.

What benefit has ING recognized from its membership with ODPi? What value do you expect to see from your participation?

Our participation in ODPi has already given us the platform to develop the ODPi Egeria open metadata standard. A full team from ING has been actively building the standard on the ODPi infrastructure. As a member, we also get to co-steer the direction of the open metadata initiative, and we benefit from the marketing initiatives from ODPi.

Through the community, we have now also involved SAS in the open metadata initiative, and we are talking to others. We expect ODPi to help us get this initiative known even more, both within the vendor community and with the consumer community.

Once the standard is mature, we see a role for ODPi in validating compliance to the standard, by delivering a test suite. ODPi will also deliver a set of value packs on top of the standard, like a GDPR pack, something we also see a lot of value in.

ODPi Egeria - Project Objectives

ODPi Egeria – Project Objectives

Tell us what excites you the most personally in regards to the technical work being done in ODPi?

Being a real nerd, I love to develop a new standard by really building it from scratch. Unfortunately, I can’t spend all my days coding anymore, so I am limited to reading some of the code that was developed, and to help drive the architecture and design for Egeria.

Building this standard, in my opinion, will be a game changer for the data industry, once we have a way to govern all data in all systems through the metadata, it will take the maturity of data management and governance to a whole new level. Imagine banks like ING delivering data to our regulators through a set of open formats, with the open metadata format on top. Our regulators having full lineage on where the data originated. It would solve all the challenges companies have today on proving that they are in control.

Companies exchanging data will be able to see where their data is being used, and supply usage agreements with that data in an open format. Data being available everywhere with the full metadata, every data consumer understanding what data they look at, the quality, the definitions, in any technology they use. Imagine customers being able to see exactly where their data is, who has access to it, what consent they have given.

Data privacy by design will truly become feasible through such a standard. And we will not stop at the traditional data landscape, it also extends to APIs, events, all the ways data is made accessible. I believe this standard is the beginning of a transformation in data management, and I think it is a very exciting project to work on.

ODPi Announces New Egeria Conformance Program to Advance Open Metadata Exchange Between Vendor Tools

By Announcements, ODPi Egeria

SAN FRANCISCO, February 11, 2019 – ODPi, a nonprofit Linux Foundation project, accelerating the open ecosystem of big data solutions, today announced the ODPi Egeria Conformance Program, which ensures vendors who ship ODPi Egeria in their product offerings are delivering a consistent set of APIs and capabilities, such that data governance professionals can easily build an enterprise-wide metadata catalog that all their data tools can easily leverage.

Egeria is one of the open source projects under the ODPi umbrella. ODPi aims to be a standard for simplifying, sharing and developing an open big data ecosystem.

“Open metadata and governance is incredibly valuable IT operating environments. The ODPi Egeria ecosystem is taking a big step today aimed at fulfilling the promise of delivering useful metadata exchange capabilities and vendors are beginning to sign up to the standards,” said John Mertic director of program management, ODPi. “By adopting ODPi Egeria standards and implementation as the core of your metadata management and governance program, an organization is able to future-proof their investments and be able to adopt the best-of-breed tools for their business.”

Open metadata and governance is a key part of the standardization of IT operating environments. If software and data components can be described in a common way, including the relationships between them, and annotated with governance requirements then it becomes much easier to automate deployments and optimize workload deployments. These are valuable outcomes for any company dealing with big data.

The ODPi Egeria Conformance program makes it possible for vendors to test their products to ensure their conformance to project standards and provides exclusive marks to use  in customer facing support materials. Conformance is accomplished through a self-testing program.

The Conformance program has been designed to aid businesses who are dealing with metadata and will quickly see the benefits of adding Egeria conformance to the list of requirements for new software tool purchases. Both IBM and SAS, leading vendors of data governance tools who have contributed to ODPi Egeria since it’s inception, have committed to ship ODPi Egeria Conformant products in 2019. Many more vendors are evaluating ODPi Egeria and will announce their conformance at a later date.

ODPi Egeria, a new project from ODPi launched in August 2018, supports the free flow of metadata between different technologies and vendor offerings. Egeria enables organizations to locate, manage and use their data more effectively. In addition, it provides governance features that smooth over the gaps between different vendor offerings enabling organization to have a complete and highly automated governance program

“ODPi Egeria brings much-needed standards to the world of metadata management and governance,” said Jay Limburn, IBM Distinguished Engineer and Director of Offering Management, Unified Governance and Integration Products. “The work aligns well with our unified governance strategy and we look forward to continuing our work with ODPi to deliver products based on ODPi Egeria and the ODPI Egeria Conformance Program.”

“The ODPi Egeria technology is advancing rapidly due to the support of companies such as IBM, ING and SAS. The project is less than a year old and already it is being embedded in key products,” said Mandy Chessell, lead for the ODPi Egeria project. “The launch of the conformance program is the next phase in its maturity, enabling vendors to advertise that their software can collaborate in the ODPi Egeria ecosystem. By delivering the conformance suite as open source, we are enabling organizations to verify that any technology they are considering purchasing will operate correctly in an ODPi Egeria ecosystem,”

“As a maintainer of the ODPi Egeria project, we are thrilled to see the next step in its maturity, with the ODPi Egeria Conformance program,” said Craig Rubendall, Vice President, Platform Research and Development, SAS.  “This program is critical to ensure the consistency and quality of the solutions integrating with and leveraging the ODPi Egeria open metadata standards. As SAS rolls out products that have this support we can be confident it is being done in a way that ensures the interoperability goals set by ODPi Egeria.”

Additional Resources

About ODPi
ODPi is a nonprofit organization committed to simplification and standardization of the big data ecosystem. As a shared industry effort, ODPi members represent big data technology, solution provider and end user organizations focused on promoting and advancing the state of big data technologies for the enterprise. For more information about ODPi, please visit: http://www.ODPi.org

About The Linux Foundation

The Linux Foundation is the organization of choice for the world’s top developers and companies to build ecosystems that accelerate open technology development and commercial adoption. Together with the worldwide open source community, it is solving the hardest technology problems by creating the largest shared technology investment in history. Founded in 2000, The Linux Foundation today provides tools, training and events to scale any open source project, which together deliver an economic impact not achievable by any one company. More information can be found at www.linuxfoundation.org.

The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our trademark usage page: https://www.linuxfoundation.org/trademark-usage.

Media Contact
Nancy McGrory
The Linux Foundation
nmcgrory@linuxfoundation.org

Managing Privacy in the GDPR-era

By Blog, Events, ODPi Egeria

 

Now that the EU General Data Protection Regulation (GDPR) is in full effect, businesses both large and small have made changes to be fully compliant, regardless of where they are located. The changes include more regulation for how companies collect data, how they store it, keep it safe from hackers and use it in their day-to-day activities. Some people think GDPR as ‘giving the power over data back to the user’. GDPR replaced old data privacy laws that were set up in 1995 and that have been obsolete for some time now.

But what does this mean for the consumer?

According to this Marketing Week article, consumers don’t understand how brands use their data. In fact, 48% of consumers still don’t understand where and how organizations use their personal data. This is up from 31% when the research was last conducted two years ago.

Only 7% feel they have a good understanding of how companies use their data, with 45% saying they “somewhat understand,” but 18% believe businesses treat people’s personal data in an honest and transparent way.

This is where ODPi comes in. ODPi’s Data Governance initiative aims to create an open data governance ecosystem through collaboration with data governance subject matter experts and data platform and tools vendors. On Thursday, July 12, ODPi is hosting a webinar focused on managing privacy.

Mandy Chessell, distinguished engineer and master inventor at IBM, will share best practices for how IBM manages data that keeps individuals’ privacy respected and is compliant with new regulations on data privacy such as the EU GDPR.

Attendees will learn:

  • The life cycle of a digital service as it is developed, sold, enhanced and used. This life cycle breaks the work into six stages. Each stage describes the roles and the activities involved to ensure data privacy.
  • The types of artifacts that need to be collected about a digital service and the methods used to develop it.
  • How these artifacts link together in an open metadata repository (data catalog).

Click to learn more or to register for the webinar.

The Rise of Big Data Governance: Strata Data Conference and DataWorks Summit Sessions, Webinar, RedGuide and More!

By Blog, Events, ODPi Egeria

Each of today’s most forward-thinking enterprises have been forced to face similar data challenges: the reliance on real-time data to better serve their customers and, subsequently, the requirement of complying with regulations to protect that data, such as the EU’s General Data Protection Regulation (GDPR).

ODPi Data Governance PMC is working to create a neutral, industry-wide approach to data governance. Together, they are supporting the mission of creating an open data ecosystem through collaboration with subject matter experts and data platform and tools vendors.

Below please find upcoming speaking sessions, Meetups, webinars and a RedGuide meant to further the discussion and work of Data Governance.

March 6–8, 2018

Strata Data Conference

San Jose, CA

The rise of big data governance: Insight on this emerging trend from active open source initiatives

Speakers:

 Maryna Strelchuk (ING)

 John Mertic (ODPi)

Time: 1:50pm–2:30pm

Date: Wednesday, March 7, 2018

https://conferences.oreilly.com/strata/strata-ca/public/schedule/detail/64048

John Mertic and Maryna Strelchuk detail the benefits of a vendor-neutral approach to data governance, explain the need for an open metadata standard, and share how companies like ING, IBM, Hortonworks, and more are delivering solutions to this challenge as an open source initiative. The solution to this emerging challenge is a tricky one. For companies like ING, this data governance challenge has been met with metadata, a consistent view across a large heterogeneous ecosystem, and collaboration with an active open source community.

—————————-

April 16-19, 2018

DataWorks Summit

Berlin, Germany

The rise of big data governance: Insight on this emerging trend from active open source initiatives

Speakers:

 Ferd Scheepers (ING)

 John Mertic (ODPi)

https://dataworkssummit.com/berlin-2018/

Attendees will understand the role of metadata, the need for a cross-technology view on metadata, the role of Apache Atlas as a reference implementation, and the role of ODPi in offering value-added services, such as certification.

ODPi Data Governance PMC

Hosted by:

 Mandy Chessell (IBM)

https://dataworkssummit.com/berlin-2018/bofs/

This Birds of Feather (BoFs) sessions, hosted by IBM, ING, ODPi, and Hortonworks will include discussions around the ODPi Data Governance PMC. Come and share your experiences, challenges, future interests.

—————————-

April 26, 2018 at 9am PST/ 12pm EST

ODPi Webinar

Speakers: Mandy Chessell (IBM), John Mertic (ODPi)

Topic – Discussion of the IBM Redguide “The Journey Continues: From Data Lake to Data-Driven Organization”, an overview of the ODPi Data Governance PMC and a look at what’s to come this year.

Sign up here: https://www.odpi.org/projects/data-governance-pmc 

Check @ODPi on Twitter for details soon!

—————————-

Download Now!

The Journey Continues: From Data Lake to Data-Driven Organization

Written by Mandy Chessell (IBM), Ferd Scheepers (ING), Maryna Strelchuk (ING), Ron van der Starre (IBM), Seth Dobrin (IBM), and Daniel Hernandez (IBM)

http://www.redbooks.ibm.com/Abstracts/redp5486.html?Open  

This IBM Redguide™ publication looks back on the key decisions that made the data lake successful and looks forward to the future. It proposes that the metadata management and governance approaches developed for the data lake can be adopted more broadly to increase the value that an organization gets from its data. Delivering this broader vision, however, requires a new generation of data catalogs and governance tools built on open standards that are adopted by a multi-vendor ecosystem of data platforms and tools.

Work is already underway to define and deliver this capability, and there are multiple ways to engage. This guide covers the reasons why this new capability is critical for modern businesses and how you can get value from it.

Social Media Auto Publish Powered By : XYZScripts.com