Data map API
The mappings package is one of the 5 main component categories in the information model.
The data map APIs are a set of APIs provided by the information model server to aid with the transfer of data between two data sets, where the source and target data sets conform to a data model. Either the source or the destination data models (or both) may be implemented as either relational or graph or both.
The APIs cover two main types of activities:
- The retrieval of mapping files for use by a transform module.
- The mapping of properties and values
The extent to which these are used by a transform module is determined by the degree of knowledge already built into a transform and the nature of the transformation software. It may be used as a human readable documentation. It may be used to enhance a current transform. It may also be used to drive a generic transformation engine.
Background and use cases
The Discovery information model supports a consistent approach to the presentation of maps using a mapping syntax heavily influenced by the W3C R23ML language and the ORM approach as defined by source systems such as Hibernate.
The use of a mapping server enables the separation of concern between the maintenance of data maps themselves and the applications that transform data.
The mapping server can be used by a transform module operating as a client via the mapping APIs.
Unlike many transform tools, the mapping server does not provide run time mapping code. The server assumes that the transform client is technically able to map X to Y, but may not know how what the map actually is. Use of the mapping server enables the transform client module to operate using a generic mapping process as long as the module can support the technology that X and Y data exists in.
The main challenge in transforming data, is not so much the software, as the knowledge of what maps to what and how. This knowledge usually resides in the domain of informatics. The conventional approach of writing down this knowledge in documents or spreadsheets is helpful but insufficient. Instead it would be much better if this knowledge was presented in both a machine readable and human readable form. Even better, if the mappings can be used to parameterise a generic mapping application.
There are many different mapping techniques in operation. In particular object relational mapping tools based on this separation already exist and there are a scattering of standards in this area. Thus, The approach draws on the ideas inherent in the W3C R2RML and ORM open source systems such as Hibernate.
The APIs are designed to be used as an assistance to a code based ORM like transform, and can thus be used for some or all of the process, depending on the requirements of the transform client. In other words, the use of the APIs is a pick and mix approach, making it possible to introduce it into currently operating transformations without disruption.
The mapping server operates like a SatNav by providing both a map and a set of instructions as to how to use it. It nevertheless keeps the control of the process within the code space of the transform application. It uses similar constructs to the data set definition language i.e. a query syntax, albeit simplified.
There are 3 main APIs: Get map, Get mapped property, and get Mapped value.
There are 3 main uses of the get map API:
- Relational to object mapping (MR2O) . Obtain a map which describes how to create or reference target objects from a set of source tables and fields, and how to map their values.
- Object to relational mapping (MO2R). Obtain a map which describes how to create or reference a target database from a set of source objects and how to map their values.
- Object to Object mapping (MO2O). Obtain a map which describes how to create or reference a target objects from a set of source objects and how to map their values.
The different approaches to the above maps is designed to address the problem of object relational impedance mismatch which is the problem of moving to and from rows/columns to types/subtypes;/properties.
There is one main use of the Get mapped value API
- For a codeable value in the source (e.g. a code or text), when taken together with the context of that value, obtain the common information model target concept (or code).
General approach to mapping
Discovery maps from a source only to the common information model's data model, and from the common data model to a target i.e. two mapping steps, often combined into one from the client perspective.
The benefit of this approach is that only one outbound map is required for each type of target (map from the common model) and only one inbound map for each source (map to the common model). The common data model acts as the hub that can "clean" and organise the data in a standardised way.
This approach also enables mapping clients to make use of either or both mapping steps separately or grouped depending on the use case. For example, currently implemented mapping process that map to and from FHIR, may only require a few additional IM facilities, such as what to do with new provider fields and what to put in a generic FHIR name/ value pair extension.
Use of Context
Context is used throughout the mapping model. Context indicates the context in which a particular data map artefact operates. The same map set in a slightly different context may operate in a different way. For example a mapping of X from system A from provider B, may be different from the mapping of X from system A from provider C. Context differentiates the two.
Context is used by the server in two ways:
- As a means of identifying slight differences between seemingly similar maps i.e. a different context, thus avoiding the wrong map.
- As a means of converging slightly different maps to a common mapping context i.e. initially different contexts converging to the same context to enable re-usability of maps.
Context is used by the client in order to short cut requests via the API. Having first defined context, it is then presented by the mapping server as a token for use by the the client, to enable re-use of the same context as a short cut to improved performance and to make coding easier. The context token is then an exchange token that tells the mapping server that a particular map request is operating within a particular context. Alternatively the client could send enough information each time to the server to provide the context for a map, so in this sense the context token is a convenience rather than a necessity.
There are two ways for the client to handle context:
- Provide the context with the API request. The mapping server reads the context and works out which mapping node to use to return the mapped property or value
- Provide the context identifier with the API request. The client already knows the context of a request and therefore can send it back in. Context operates as a sort of toke
Client provided context may use some or all of the full context information depending on whether the client knows the mapping server's mapping rules. For example, every GP practice using an EMIS Web system has the same context for all observations (as every EMIS GP system uses the same table structure. Thus the client does not need to send in the provider ID. However, if in doubt this can be sent in, as the mapping server will know to ignore it.
In this example the request shows the full context definition of a field from an "Admitted patient care" table which requires a value map for the '1' in the field "administration_category_code"
{ "MapDataRequest": {
"sourceContext": {
"provider": {
"value": "H123344",
"display": "BartsNHSTrust",
"codeScheme": "\"https://fhir.nhs.uk/ValueSet/ods\""
},
"system": {
"value": "CernerMillenium",
"codeScheme": "https://DiscoveryDataService.org/InformationModel"
},
"schema": "CDS",
"table": "APC"
},
"field": {
"name": "administrative-category_code",
"value": 1} }}
If the client knows the context the request could be as follows:
{ "MapDataRequest": {
"sourceContext": {
"context":"/CDS/APC"},
"field": {
"name": "administrative-category_code",
"value": 1} }}
The underlying approach to authoring the maps themselves is described in the article Map maker manager which describes the fundamental concepts involved in making the maps for the server to deliver.
Get map
One of the two main map APIs involves requesting a map for the purposes of mapping from a source schema to a target schema.
The request - response exchange involves the use of parts of a 'mapping document' which is a JSON or XML document that contains a subset of the the information model (which itself can be considered as one huge document).
A mapping document instance, like all the information model content, conforms to a subset of a single overarching class model available as a W3C XML schema or UML class. Thus a mapping document is both human and machine readable. It can be used as a means of controlling the flow of a generic map processor.
The mapping server contains a repository of Mapping documents, generated by the map server from a set of map nodes previously authored by a map maker.
Map domain
A mapping document contains a map domain, which provides the information required to map from source to target. The combination of source description, mapping process and target definition can be referred to as a map domain.
To request a map, A requester must first know the nature of the source and target they are requesting the map for. The description of the source and the target are both provided in a Map start/end object that contains a set of identifiers that provide sufficient context for the map server to know what map to retrieve.
A mapping document contains a Domain map (class Map domain) which includes the source and target terminals for the map. The source and target terminals consist of identifiers for the source and target context properties.
A mapping request may contain both source context and target context. If no target context is included, it is assumed that the target is the core information model.
The following is part of the body of a request for a mapping file for Barts Trust CDS schema to the information model
{ "fromSource":{
"provider": {
"value":"H12345",
"display": "BartsNHSTrust",
"codeScheme":"https://FHIR.nhs.uk/ods"},
"system":{
"value":":CM_System_CernerMillenium",
"codeScheme":"https://DiscoveryDataService.org/InformationModel"},
"schema": {
"value": "CM_Schema_CDS",
"codeScheme": "https://DiscoveryDataService.org/InformationModel"}}}
If the client knows that the map is provider independent and they do not need to pass in the provider, then they need not populate the source provider property. If they do not know whether it is relevant or not, and they populate the property with the provider identifier, then the mapping server may choose to ignore that property if not relevant, and the response will not include the provider.
The mapping document response also contains the actual source and target map domain object as well as the relevant maps, together with the contexts of the source and target.
In the example the domain map context is between Barts CDS and FHIR Care connect
{"fromSource" : {
"context": "/CDS/Cerner/Barts"}},
{"toTarget" : {
"context":"/DiscoveryDataService.irg/InformationModel"}}
If the client knows the map context it simply needs to pass in the source and target context as part of the request.
Data Maps
A mapping document contains one or more data maps between a source and a target. The source may be relational or in object form , and the target may be relational or object form, depending on the map domain parameters as described above.
In this article the working example will be based on MR2O i.e. mapping relational to information model objects.
A data map uses a set of clauses in the order of CREATE FROM , WHERE which indicate that an object or property should be created from a source table or field, with perhaps with some other criteria, the transform will either create an object, property or relationship or reference an object already created previously in another process.
Data maps may be nested so that maps that have created objects as dependent (child objects) can go on to populate the properties of the child objects. Also CREATE clauses can have nested data maps in order to go on to populate objects created as targets of relationships.
Many maps are simple one to one maps. However, the following example shows a part of a map that takes a flat CDS table row and creates two encounters, one of which is a sub-encounter to the other.
"map": {
"from": {
"table": "cds_inpatient"
},
"create": {
"context": "String",
"object": {
"class": " :DM_HospitalInpEntry",
"idGenerator": {
"auto": "true"
}
},
"as": "@inpatient"
},
"map": [
{
"from": {
"field": "spell_number"
},
"create": [
{
"object": {
"class": ":CM_HospitalInpAdmitEncounter>",
"idGenerator": {
"auto": "true"
}
},
"as": "@admission"
},
{
"relationship": {
"relationshipType": ":RM_isComponentOf",
"referenceTarget": "@inpatient"
}
}
]
},
{
"from": {
"field": "administrative_category_code"
},
"create": {
"context": "/CDS/APC/administrative-category_code",
"property": {
"name": "DM_adminCategoryonAdmission",
"value": {
"valueMap": [
{
"sourceValue": "01",
"targetValue": ":CM_AdminCat01"
},
{
"sourceValue": "02",
"targetValue": ":CM_AdminCat02"
},
{
"sourceValue": "03",
"targetValue": ":CM_AdminCat03"
},
{
"sourceValue": "04",
"targetValue": ":CM_AdminCat04"}]}}}}]}}}
To walk through the above example, a set of instructions:
From the table "cds_inpatient", create an object of the class "hospital inpatient entry" (which is a sub class of encounter) object , assign to variable @inpatient
From the field "spell_number", : 1. Create an object of class "hospital inpatient admission" (which is also a subclass of encounter) assign variable @admission
2. Create a relationship 'subcomponent of,' linking to the @inpatient object
3. From the field 'administrative_category_code' create property of adminCategoryOnAdmission (noting the context)
From the above map the client also knows to be able to use the /Map/Barts/Cerner/CDS/Inpatient/accc context identifier with the MapConceptValue API to map the code
Get mapped property
This API returns a target property from a source property set in the context of a source provider source system, schema, table
Typically this might be used by a transform client when ascertaining what to do with unknown source properties that have not been "hard wired" into the target when otherwise a transform already exists. Typically these would be implemented as extensions or triple tables in the target.
In this example, the client already knows the context identifier of a CDS schema and admitted patient care table
{"MapDataRequest": {
"sourceContext": {
"context": "/CDS/APC"
},
"field": {
"name": "administrative-category_code"}}}
Get mapped value
This API returns a target value from a source value set in the context of a source provider source system, schema, table/class, field or property.
Typically this might be used by a transform client when ascertaining what to do with unknown source values that have not part of the mapping domain. Typically this is used when new codes are required or where the value is from a legacy classification or text.
In this example the client is seeking the mapped value for a field value of 1. The term is unknown (i.e. is not in a look up table from the source)
{"MapDataRequest": {
"sourceContext": {
"context": "/CDS/APC"
},
"field": {
"name": "administrative-category_code",
"value": 1
}
}
Sometimes, when mapping text or a value, it is necessary to provide additional context from another field.
For example, in a source system, the word 'negative, a value of the field 'result text' associated with a code value '12345', a value in the field 'test', which is a field of the table of 'clinical events', used by the system 'Cerner Millennium', in the hospital 'Barts NHS trust', may mean something completely different to 'negative' with the code '12345' set in another hospital, even with the same system.
In this case a dependent field value is sent with the value nested within the dependent field
{ "sourceContext": {
"context": "/CDE/CLEVE"
},
"field": {
"name": "Event_Code",
"value": "3562720",
"term": "SARS Cov2 PMR",
"field": {
"name": "Event_Result",
"term": "Positive"
}
}