Data map API

The mappings package is one of the 5 main component categories in the information model.

The data map APIs are a set of APIs provided by the information model server to aid with the transfer of data between two data sets, where the source and target data sets conform to a data model. Either the source or the destination data models (or both) may be implemented as either relational or graph or both.

The APIs cover two main types of activities:

1. The retrieval of mapping files for use by a transform module for the full transform
2. The mapping of properties and values to aid with existing transforms.

The extent to which these are used by a transform module is determined by the degree of knowledge already built into a transform, and the nature of the transformation software. It may simply be used as a human readable documentation. It may be used to enhance a current transform. It may also be used to drive a generic transformation engine.

Background and use cases

The Discovery information model supports a consistent approach to the presentation of maps using a mapping syntax heavily influenced by the W3C R23ML language and the ORM JPA approach as used by open source systems such as Hibernate.

The use of a mapping server enables the separation of concern between the maintenance of data maps themselves and the applications that transform data. As well as the technical separation it separates the business of map production from the business of transformation.

The mapping server can be used by a transform module operating as a client via the mapping APIs.

Unlike many transform tools, the mapping server does not provide run time mapping code. The server assumes that the transform client is technically able to map X to Y, but may not know what the map actually is. Use of the mapping server enables the transform client module to operate using a generic mapping process as long as the module can support the technology that X and Y data exists in.

The main challenge in transforming data, is not so much the software, as the knowledge of what maps to what and how. This knowledge usually resides in the domain of informatics. The conventional approach of writing down this knowledge in documents or spreadsheets is helpful but insufficient. Instead it would be much better if this knowledge was presented in both a machine readable and human readable form. Even better, if the mappings can be used to parameterise a generic mapping application.

There are many different mapping techniques in operation. In particular object relational mapping tools based on this separation already exist and there are a scattering of standards in this area. Thus, The approach in the Discovery mapping service draws on the ideas inherent in the W3C R2RML and ORM open source systems such as Hibernate.

The APIs are designed to be used as an assistance to a code based ORM like transform, and can thus be used for some or all of the process, depending on the requirements of the transform client. In other words, the use of the APIs is a pick and mix approach, making it possible to introduce it into currently operating transformations without disruption.

The mapping API provides 3 levels of mapping

1. Full schema mapping. The mapping server operates like a SatNav by providing both a map and a set of instructions as to how to use it. It nevertheless keeps the control of the process within the code space of the transform application. It uses similar constructs to the data set definition language i.e. a query syntax, albeit simplified.
2. Property mapping. The server maps a class/ property or table/ field to a property concept. This is useful for things like FHIR when working out the nature of an extension or key value pair
3. Value mapping. The means by which a legacy code or piece of text, is mapped to a core ontology concept, the mapping dependent on the context of the legacy code or term.

The above correspond to the 3 main APIs: Get map, Get mapped property, and get Mapped value.

Use of the two mapping APIs

There are 3 main uses of the get map API:

• Relational to object mapping (MR2O) . Obtain a map which describes how to create or reference target objects from a set of source tables and fields, and how to map their values.
• Object to relational mapping (MO2R). Obtain a map which describes how to create or reference a target database from a set of source objects and how to map their values.
• Object to Object mapping (MO2O). Obtain a map which describes how to create or reference a target objects from a set of source objects and how to map their values.

The different approaches to the above maps is designed to address the problem of object relational impedance mismatch which is the problem of moving to and from rows/columns to types/subtypes;/properties.

The main use of the Get mapped property API is for assigning the common data model properties when processing legacy data from non standard sources.

There is one main use of the Get mapped value API which is for a codeable value in the source (e.g. a code or text), when taken together with the context of that value, obtain the common information model target concept (or code).

General approach to mapping

Choice of mapping steps

Discovery maps from a source only to the common information model's data model, and from the common data model to a target i.e. two mapping steps, often combined into one from the client perspective.

The benefit of this approach is that only one outbound map is required for each type of target (map from the common model) and only one inbound map for each source (map to the common model). The common data model acts as the hub that can "clean" and organise the data in a standardised way.

This approach also enables mapping clients to make use of either or both mapping steps separately or grouped depending on the use case. For example, currently implemented mapping process that map to and from FHIR, may only require a few additional IM facilities, such as what to do with new provider fields and what to put in a generic FHIR name/ value pair extension.

The approach to the Discovery syntax is in line with the information modelling language i.e uses class definitions to define the language syntax which provides explicit labelling of the constructs. In line with the modelling language a pure token based approach is also supported.

Use of Context

Context is used throughout the mapping model. Context indicates the context in which a particular data map artefact operates. The same map set in a slightly different context may operate in a different way. For example a mapping of X from system A from provider B, may be different from the mapping of X from system A from provider C. Context differentiates the two.

Context is used by the server in two ways:

• As a means of identifying slight differences between seemingly similar maps i.e. a different context, thus avoiding the wrong map.

• As a means of converging slightly different maps to a common mapping context i.e. initially different contexts converging to the same context to enable re-usability of maps.

Context is used by the client in order to short cut requests via the API. Having first defined context, it is then presented by the mapping server as a token for use by the the client, to enable re-use of the same context as a short cut to improved performance and to make coding easier. The context token is then an exchange token that tells the mapping server that a particular map request is operating within a particular context. Alternatively the client could send enough information each time to the server to provide the context for a map, so in this sense the context token is a convenience rather than a necessity.

There are two ways for the client to handle context:

1. Provide the context with the API request. The mapping server reads the context and works out which mapping node to use to return the mapped property or value
2. Provide the context identifier with the API request. The client already knows the context of a request and therefore can send it back in. Context operates as a sort of toke

Client provided context may use some or all of the full context information depending on whether the client knows the mapping server's mapping rules. For example, every GP practice using an EMIS Web system has the same context for all observations (as every EMIS GP system uses the same table structure. Thus the client does not need to send in the provider ID. However, if in doubt this can be sent in, as the mapping server will know to ignore it.

In this example the request shows the full context definition of a field from an "Admitted patient care" table which requires a value map for the '1' in the field "administration_category_code"

{
 "MapColumnValueRequest": {
  "Provider": {
   "CodeScheme": "\"https://fhir.nhs.uk/ValueSet/ods\"",
   "Value": "H123344",
   "Display": "BartsNHSTrust"
  },
  "System": {
   "CodeScheme": "https://DiscoveryDataService.org/InformationModel",
   "Value": "CernerMillenium"
  },
  "Schema": "CDS",
  "Table": "APC",
  "Column": "administrative-category_code",
  "Value": {
   "CodeScheme": ":CM_NHS-DataDictionary",
   "Value": 1
  }
 }
}

If the client knows the context the request could be as follows:

{
 "MapColumnValueRequest": {
  "Context":"Barts/Cerner/CDS/APC/administration_classification_code",
  "Value": {
   "CodeScheme": ":CM_NHS-DataDictionary",
   "Value": 1
  }
 }
}

The underlying approach to authoring the maps themselves is described in the article Map maker manager which describes the fundamental concepts involved in making the maps for the server to deliver.

Map requests

One of the two main map APIs involves requesting a map for the purposes of mapping from a source schema to a target schema.

The request - response exchange involves the use of JSON or XML conforming to the mapping syntaxe which is a subset of the the information model language. JSON or token based syntax is supported.

A request instance, like all the information model content, conforms to a subset of a single overarching modelling language. Thus a mapping document is both human and machine readable. I

The mapping server contains a repository of Maps, generated by the map server from a set of map nodes previously authored by a map maker.

Map Request

Map domain class

A map request is a request for map between a source schema and target schema. The combination of source description, mapping process and target definition for a full schema can be referred to as a map domain.

To request a map, a requester must first know the nature of the source and target they are requesting the map for. The description of the source and the target are both provided in structure that provides sufficient context for the mapping server to identify the source and target map required i.e. contains a set of identifiers that provide sufficient context for the map server to know what map to retrieve.

A map response contains a Domain map (class Map domain) which includes the source and target context for the map. The source and target terminals consist of identifiers for the source and target context properties.

A mapping request contains both source context and target context. If no target context is included, it is assumed that the target is the core information model.

The following is part of the body of a request for a mapping file for Barts Trust CDS schema to the information model

{
 "MapRequest": {
  "Provider": {
   "CodeScheme": "\"https://fhir.nhs.uk/ValueSet/ods\"",
   "Value": "H123344",
   "Display": "BartsNHSTrust"
  },
  "System": {
   "CodeScheme": "https://DiscoveryDataService.org/InformationModel",
   "Value": "CernerMillenium"
  },
  "Schema": "CDS",
  "Target" :"https://DiscoveryDataService.org/InformationModel/Core"
 }
}

If the client knows that the map is provider independent and they do not need to pass in the provider, then they need not populate the source provider property. If they do not know whether it is relevant or not, and they populate the property with the provider identifier, then the mapping server may choose to ignore that property if not relevant, and the response will not include the provider.

The mapping document response also contains the actual source and target map domain object as well as the relevant maps, together with the contexts of the source and target.

In the example the domain map context is between Barts CDS and FHIR Care connect

{
 "MapRequest": {
  "Context" :"Barts/Cerner/CDS"
  "Target" :"https://DiscoveryDataService.org/InformationModel/Core"
 }
}

If the client knows the map context it simply needs to pass in the source and target context as part of the request.

Map table column request

A mapping document contains one or more data maps between a source and a target. The source may be relational or in object form , and the target may be relational or object form, depending on the map domain parameters as described above.

The only difference in syntax between Object to object mapping and relational to object/ relational mapping is the terminology used. Tables columns, joins foreign keys, versus objects and properties and relationships.

In this example the client is requesting the map for a particular column from an admitted patient care column. The client already knows the context identifier for the schema and the table.

 "MapColumnRequest": {
"Context": "Barts/Cerner/CDS/APC",
  "Column": "administrative-category_code"
 }

Map column value request

This API returns a target value from a source value set in the context of a source provider source system, schema, table/class, field or property.

Typically this might be used by a transform client when ascertaining what to do with unknown source values that have not part of the mapping domain. Typically this is used when new codes are required or where the value is from a legacy classification or text.

In this example the client is seeking the mapped value for a field value of 1. The term is unknown (i.e. is not in a look up table from the source)

{
 "MapColumnValueRequest": {
  "Provider": {
   "CodeScheme": "\"https://fhir.nhs.uk/ValueSet/ods\"",
   "Value": "H123344",
   "Display": "BartsNHSTrust"
  },
  "System": {
   "CodeScheme": "https://DiscoveryDataService.org/InformationModel",
   "Value": "CernerMillenium"
  },
  "Schema": "CDS",
  "Table": "APC",
  "Column": "administrative-category_code",
  "Value": {
   "CodeScheme": ":CM_NHS-DataDictionary",
   "Value": 1
  }
 }
}

Sometimes, when mapping text or a value, it is necessary to provide additional context from another field.

For example, in a source system, the word 'negative, a value of the field 'result text' associated with a code value '12345', a value in the field 'test', which is a field of the table of 'clinical events', used by the system 'Cerner Millennium', in the hospital 'Barts NHS trust', may mean something completely different to 'negative' with the code '12345' set in another hospital, even with the same system.

In this case a dependent field value is sent with the value nested within the dependent field. In this example the mapping client is seeking the concept to represent a "positive" result for a test code.

{
 "MapColumnValueRequest": {
  "Context": "/CDE/CLEVE/ResultCode",
  "DependentColumnValue": {
   "Column": "Event_Code",
   "Value": {
    "Value": 3562720
   }
  },
  "Value": {
   "Value": "Positive"
  }
 }
}

Map Response

A data map uses a set of clauses in the order of CREATE FROM , WHERE which indicate that an object or property should be created from a source table or field, with perhaps with some other criteria, the transform will either create an object, property or relationship or reference an object already created previously in another process.

Data maps may be nested so that maps that have created objects as dependent (child objects) can go on to populate the properties of the child objects. Also CREATE clauses can have nested data maps in order to go on to populate objects created as targets of relationships.

Many maps are simple one to one maps. However, the following example shows a part of a map that takes a flat CDS table row and creates two encounters, one of which is a sub-encounter to the other.

"map": {
     "from": {
      "table": "cds_inpatient"
     },
     "create": {
      "context": "String",
      "object": {
       "class": " :DM_HospitalInpEntry",
       "idGenerator": {
        "auto": "true"
       }
      },
      "as": "@inpatient"
     },
     "map": [
      {
       "from": {
        "field": "spell_number"
       },
       "create": [
        {
         "object": {
          "class": ":CM_HospitalInpAdmitEncounter>",
          "idGenerator": {
           "auto": "true"
          }
         },
         "as": "@admission"
        },
        {
         "relationship": {
          "relationshipType": ":RM_isComponentOf",
          "referenceTarget": "@inpatient"
         }
        }
       ]
      },
      {
       "from": {
        "field": "administrative_category_code"
       },
       "create": {
        "context": "/CDS/APC/administrative-category_code",
        "property": {
         "name": "DM_adminCategoryonAdmission",
         "value": {
          "valueMap": [
           {
            "sourceValue": "01",
            "targetValue": ":CM_AdminCat01"
           },
           {
            "sourceValue": "02",
            "targetValue": ":CM_AdminCat02"
           },
           {
            "sourceValue": "03",
            "targetValue": ":CM_AdminCat03"
           },
           {
            "sourceValue": "04",
            "targetValue": ":CM_AdminCat04"}]}}}}]}}}

To walk through the above example, a set of instructions:

From the table "cds_inpatient", create an object of the class "hospital inpatient entry" (which is a sub class of encounter) object , assign to variable @inpatient

From the field "spell_number",  : 1. Create an object of class "hospital inpatient admission" (which is also a subclass of encounter) assign variable @admission

2. Create a relationship 'subcomponent of,' linking to the @inpatient object

3. From the field 'administrative_category_code' create property of adminCategoryOnAdmission (noting the context)

From the above map the client also knows to be able to use the /Map/Barts/Cerner/CDS/Inpatient/accc context identifier with the MapConceptValue API to map the code

{  "sourceContext": {
   "context": "/CDE/CLEVE"
  },
  "field": {
   "name": "Event_Code",
   "value": "3562720",
   "term": "SARS Cov2 PMR",
   "field": {
    "name": "Event_Result",
    "term": "Positive"
   }
  }