Data map API
This page is under review
The mappings package is one of the 5 main component categories in the information model.
The data map APIs are a set of APIs provided by the information model server to aid with the transfer of data between two data sets, where the source and destination data sets conform to a data model. Either the source or the destination data models (or both) may be implemented as either relational or graph or both.
The APIs cover two main types of activities:
- The retrieval of mapping files for use by a transform module client, set in the context of a known source type and destination type.
- The mapping of values between source and destination concepts terms or codes, set in the context of known source properties and destination properties obtained from the mapping files generated as above.
Background and use cases
The use of a mapping server enables the separation of concern between the maintenance of data maps and the application that transforms data.
The mapping server is used by a transform module operating as a client via the mapping APIs.
The mapping server does not have an application that generates run time mapping code. The server assumes that the transform client is able to map any X to any Y, but may not know what to map to what and what to do along the way. The transform client module can be designed to be generic in respect of X and Y, as long as it can support the technology that X and Y exists in.
The main challenge in transforming data, is not so much the software, as the knowledge of what maps to what and how. This knowledge usually resides in the domain of informatics. The conventional approach of writing down this knowledge in documents or spreadsheets is helpful but insufficient. Instead it would be much better if this knowledge was presented in both a machine readable and human readable form. Even better, if the mappings can be used to parameterise a generic mapping application.
The philosophy of the Discovery Information model is to form a bridge between the two technical worlds of informatics and programming in a way that none technical people can also visualise. Thus the approach to mappings has one level of abstraction from the technology to operate the mappings and, unlike many integration solutions is specifically designed not to impose a technology selection on the client.
The approach draws on the ideas inherent in the W3C R2RML and ORM open source systems such as Hibernate.
The APIs are designed to be used as an assistance to a code based ORM like transform, and can thus be used for some or all of the process, depending on the requirements of the transform client. In other words, the use of the APIs is a pick and mix approach, making it possible to introduce it into currently operating transformations without disruption.
Rather than simply providing a map, the mapping server operates like a SatNav providing both a map and a set of instructions as to how to use it, but keeps the control of the process within the code space of the transformation module. It uses similar constructs to the data set definition language i.e. a query syntax, albeit simplified.
There are 2 main APIs: Get map, and Get Mapped value.
There are 3 main uses of the get map API:
- Relational to object mapping (MR2O) . Obtain a map which describes how to create or reference target objects from a set of source tables and fields, and how to map their values.
- Object to relational mapping (MO2R). Obtain a map which describes how to create or reference a target database from a set of source objects and how to map their values.
- Object to Object mapping (MO2O). Obtain a map which describes how to create or reference a target objects from a set of source objects and how to map their values.
The different approaches to the above maps is designed to address the problem of object relational impedance mismatch which is the problem of moving to and from rows/columns to types/subtypes;/properties.
There is one main use of the Get mapped value API
- For a codeable value in the source (e.g. a code or text), when taken together with the context of that value, obtain the common information model target concept (or code).
General approach to mapping
Map to and from common model
Unlike a generic approach to mappings, which normally support many to many mappings, Discovery maps from a source only to the common information model's data model, and from the common data model to a target i.e. two mapping steps.
The benefit of this approach is that only one outbound map is required for each target (from the common model) and only one inbound map for each source (to the common model). The common data model acts as the hub that can "clean" and organise the data in a standardised way.
This approach also enables mapping clients to make use of either or both mapping steps separately or grouped depending on the use case. For example it would be quite common to use the output from a relational source to the common data model target, as a means of eventually mapping to FHIR, whether or not the mapping server supports the full relational to FHIR 2 step map. This is because perhaps a currently implemented mapping process that map to and from FHIR, may only require a few additional IM facilities, such as what to do with new publisher fields and what to put in a generic FHIR name/ value pair extension.
Context
Context is used throughout the mapping model. Context indicates the context in which a particular map operates. The same map set in a slightly different context may operate in a different way. For example a mapping of X, from system A from provider B may be different from the mapping of X from system A, from provider C. Context differentiates the two.
Context is used by the server in two ways:
- As a means of identifying slight differences between seemingly similar maps i.e. a different context, thus avoiding the wrong map.
- As a means of converging slightly different maps to a common mapping context i.e. initially different contexts converging to the same context to enable re-usability of maps.
Context is used by the client in order to short cut requests via the API. Having first defined context, it is then presented by the mapping server as a token for use by the the client, to enable re-use of the same context as a short cut to improved performance and to make coding easier. The context token is then an exchange token that tells the mapping server that a particular map request is operating within a particular context. Alternatively the client could send enough information each time to the server to provide the context for a map, so in this sense the context token is a convenience rather than a necessity.
The underlying approach to authoring the maps themselves is described in the article Map maker manager which describes the fundamental concepts involved in making the maps for the server to deliver.
Mapping document
A mapping document is an information model document subtype returned by the Mapping server on request from a transform client via the mapping REST API.
The mapping server contains a repository of Mapping documents, previously authored by the map authors.
A requester must first know the nature of the source and target they are requesting the map for. The description of the source and the target are both provided in a Map start/end object that contains a set of identifiers that provide sufficient context for the map server to know what map to retrieve.
A mapping document contains a Domain map (class Map domain) which includes the source and target terminals for the map. The source and target terminals consist of identifiers for the source and target context properties.
The following example request body requests the map between the Barts trust, Cerner Millennium, CDS schema and the discovery information model
{ "fromSource":{
"provider": {
"value":"H12345",
"display": "BartsNHSTrust",
"codeScheme":"https://FHIR.nhs.uk/ods"},
"system":{
"value":":CM_System_CernerMillenium",
"codeScheme":"https://DiscoveryDataService.org/InformationModel"},
"schema": {
"value": "CM_Schema_CDS",
"codeScheme": "https://DiscoveryDataService.org/InformationModel"}},
"toTarget":{
"system": {
"value":":CM_System_DiscoveryCore",
"codeScheme": "https://DiscoveryDataService.org/InformationModel"},
"schema": {
"value":"CM_Schema_DiscoveryInformationModel_V1",
"codeScheme": "https://DiscoveryDataService.org/InformationModel"}} }
The mapping document response also contains the actual source and target map domain object as well as the relevant maps, together with the context of the domain map. In the example the domain map context is between Barts CDS and Discovery IM.
{"mapContext":":MDC_BartsCDS2IM"}
If the client knows that the map is provider independent and they do not need to pass in the provider, then they need not populate the source provider property. If they do not know whether it is relevant or not, and they populate the property with the provider identifier, then the mapping server may choose to ignore that property if not relevant and the response will not include the provider.
If the client knows the map context it simply needs to pass in the domain map context as part of the request.
Data Map
A mapping document contains one or more data maps between a source and a target. The source may be relational or in object form , and the target may be relational or object form depending on the map domain parameters as described above.
In this article the working assumption will be that the mapping is MR2O i.e. mapping relational to information model objects.
A data map uses a set of clauses in the order of FROM , WHERE, CREATE | REFERENCE which indicate that from a source table or field, perhaps with some other criteria, that the transform will either create an object, property or relationship or reference an object already created previously in another process.
Data maps may be nested so that maps that have created objects as dependent (child objects) can go on to populate the properties. Also CREATE clauses can have nested data maps in order to go on to populate objects created as targets of relationships.
Many maps are simple one to one maps. However, the following example shows a part of a map that takes a flat CDS table row and creates two encounters, one of which is a sub-encounter to the other,
{ "map": {
"from": { "table": "cds_inpatient"},
"create": { "object": ":DM_HospitalInpEntry", "as": "inpatient"}, //From the cds row create an encounter
"map": [ {
"from": { "field": "spell_number"}, "as": "spellnumber"
"create": { "relationship": { // trigger the creation of sub encounter
"relationshipType": ":RM_isComponentOf",
"createTarget": { "object": ":CM_HospitalInpAdmitEncounter","as": "admission"} } } },
{
"from": {
"field": "administrative_category_code"},
"create": {
"fieldContext": ":DMAP_cds_acc", //context used later for mapping the field value
"property": "DM_adminCategoryonAdmission", // create property in admission sub-encounter
"value": {
"valueFunction": "mapConceptValue"}}}]}}}} // use map concept value to populate from field
Value maps
Use of context dependency
For a source value to be understood, it is necessary to provide some context to it. All values are therefore set in some form of context. This context is provided by the data maps.
Sometimes, when mapping text or a value, it is necessary to provide additional context from another field.
For example, in a source system, the word 'negative, a value of the field 'result text' associated with a code value '12345', a value in the field 'test', which is a field of the table of 'clinical events', used by the system 'Cerner Millennium', in the hospital 'Barts NHS trust', may mean something completely different to 'negative' with the code '12345' set in another hospital, even with the same system.
The map therefore indicates whether a dependent field value should be sent via the API when asking for a map in context.
{ "map": {
"from": {
"field": "RESULT_TXT"
},
"create": {
"fieldContext": ":DMAP_CDE_RESULT_TXT",
"property": ":DM_ObservedConcept",
"value": {
"valueFunction": "$mapConceptValue", //function to map the result text value
"dependentField": "EVENT_CODE" //indicates that the event code should be sent in