Who would have guessed? Not all metadata repositories are the same!
There is a lot more to metadata than first meets the eye! When I started working with the Egeria team, I thought metadata would be straightforward. But I was limiting my thinking to data: there’s also – types, date created, owner, relationships and so on. The reality is there is a wealth of metadata for each element of data. The Egeria type system defines metadata types; currently, there are almost 500 types that describe elements of data.
The Egeria open standard is designed to support metadata from all applications and datastore repositories. This requires the Egeria type system to be comprehensive. Consider the metadata you have in a relational database, which is very different from that stored in a data transformation tool or a DevOps pipeline. This is why Egeria has such a large type system as it needs to cover a multitude of metadata types and operations.
The different repositories that Egeria can connect to do not necessarily each have to support the whole type system – some only use a subset of the types. They also might only support a subset of the operations. This is why Egeria needs a way to capture and describe the level of type and operation support that each repository connector provides.
Introducing the Conformance Test Suite (CTS)
We, therefore, created a Conformance Test Suite (CTS) to enable you to test your metadata repository. The CTS is helpful both during the development of a connector and for conformance testing for any metadata repository to determine whether it conforms to the Egeria metadata collection interface.
The CTS will help you to determine the extent to which your particular repository can store and manage the Egeria type system. The CTS tests whether instances of these metadata types can be created, read, updated, deleted and searched. These are all operations that are defined in the metadata collection API.
Some repositories will not support all the Egeria metadata types or have the full range of capabilities defined in the Egeria metadata collection API. This is because Egeria is a universal metadata exchange standard and seeks to cover all metadata and metadata capabilities.
What do we Test?
When a repository is tested we need to ensure it behaves “properly” according to the Egeria protocol. For example, when metadata is added, updated and so on, the repository should generate expected results. In the situation where a metadata type or operation is not supported then that repository should give an appropriate response to show the feature is not supported. Not supporting all metadata types and operations 100% is neither a good nor a bad thing. But responding to all tests with the appropriate protocol is essential.
The Egeria protocol is defined by the types and events described in the open metadata specification.
So how does the Conformance Test Suite work?
To use Egeria with a new kind of metadata repository, you will need a metadata connector to connect your metadata repository to the Open Metadata and Governance (OMAG) server platform. If one does not exist you can create one. See the blog article “Implementing and Open Metadata Connector” written by Chris Grote. The CTS will then allow you to test the extent to which your connector and repository support the Egeria type system and operations.
The diagram below shows the basic configuration you will need in place to run the Conformance Test Suite (CTS). You will need two Open Metadata and Governance Servers (OMAG servers). The first OMAG server runs the CTS and collates the results. The second OMAG server is configured to run with a local connection to the metadata repository that you are testing.
How to Run the CTS Repository Conformance Workbench
The CTS Repository Conformance Workbench provides the Conformance Tests that will be run against the new metadata repository.
- Configure the CTS server to run the repository conformance workbench, which can be achieved by posting to the following URL:
http://localhost:8080/open-metadata/admin-services/users/garygeeke/servers/<name of CTS server>/conformance-suite-workbenches/repository-workbench/repositories/<name of server to test>
(assuming that the admin-services serverURLRoot is set to localhost:8080)
- It is best practice to start the CTS Workbench OMAG server first. The CTS server should be configured to use the default Egeria repository.
- Next, start the Test OMAG server, this should be configured to use the repository to be tested for conformance.
- When the Test OMAG server starts it will register with the other servers in the cohort. The CTS server will see this registration and return its own registration details.
- The CTS server will now recognize the Test Server as identified in the workbench configuration. The Conformance Tests will now automatically run as defined in the CTS Repository Conformance Workbench.
- The Conformance test results are stored in memory within the CTS server and can be written to a Test Report file on request. To generate the report file you can POST to the following URL:
For more details on running this process refer to the technical deep-dive blog “The ODPi Egeria Open Metadata Conformance Suite – Repository Workbench“.
These are generated in JSON files, however, for the repositories we have considered in this release here are our findings for:
- The JanusGraph metadata repository.
- The In-memory metadata repository used for testing and demos.
- The Apache Atlas repository proxy.
- The IBM Information Governance Catalog repository proxy.
- FULL means the repository fully supports this profile
- PARTIAL means the repository supports some of this profile
- NONE means the repository does not support this profile
- NA means this profile is not tested in the current release
Running a demo of the Conformance Test Suite
If you would like to see exactly how to run the Conformance Test Suite then follow the tutorial in the demo Jupyter Notebook. To access the Egeria Tutorials you will need to run the Egeria demo which has been preconfigured in a Docker container, details of how to do this can be found in the Blog “Getting started with Egeria notebooks using docker“. Once you have run this you will be able to launch the Jupyter notebooks and run the Conformance Suite notebook.
Alternatively, you can run this natively, refer to the CTS technical blog for more details on running the CTS
Get in touch and find out how to contribute to the Egeria Project
To Join the Egeria Project or Slack Channel, check out the following links.
- Contribute to ODPi Egeria
- Contact the team via slack – join here & go to #egeria-discussions. We’d love to hear what you think!