EMIS GP published data transformation and mapping
To see the latest Compass V2 schema, please see Discovery Remote Subscriber Database Familiarisation#Data_Mapping
This article describes the mapping of EMIS published data into the intermediate FHIR store.
Location and organisation
Admin_Location
Each record maps to a FHIR Location resource.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| LocationGuid | Unique ID | Location::Id* | location | id |
| LocationName | Location name | Location::Name | location | name |
| LocationTypeDescription | Type of location | Location::Type::Text | location | type_desc |
| ParentLocationGuid | Parent location | Location::PartOf::Reference * | location | managing_organization_id |
| OpenDate | Open Date | Location::primarycare-activeperiod-extension::ValuePeriod::Start | - | - |
| CloseDate | Close Date | Location::primarycare-activeperiod-extension::ValuePeriod::End | - | - |
| MainContactName | Main contact name | Location::primarycare-location-main-contact-extension::String | - | - |
| FaxNumber | Fax number | Location::contactPoint::fax (use=work, system=fax) | - | - |
| EmailAddress | Email adress | Location::contactPoint::email (use=work, system=email) | - | - |
| PhoneNumber | Phone number | Location::contactPoint::phone (use=work, system=phone) | - | - |
| HouseNameFlatNumber | Address name/flat number | Location::Address::Line | - | - |
| NumberAndStreet | Address street | Location::Address::Line | - | - |
| Village | Address village | Location::Address::Line | - | - |
| Town | Address town | Location::Address::City | - | - |
| County | Address county | Location::Address::District | - | - |
| Postcode | Type of organisation | Location::Address::PostalCode | location | Postcode |
| Deleted | Indicates whether the location has been deleted | <If true, FHIR resource is deleted> | - | - |
| ProcessingId | Sequentual identifier indicating the order to process the items | ** | - | - |
Admin_Organisation
Each record maps to a FHIR Organization resource.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| OrganisationGuid | Unique ID | Organization::Id* | organization | id |
| CDB | EMIS customer number | Organization::Identifier::Value (use=secondary, system=cdb-number) | - | - |
| OrganisationName | Name of organisation | Organization::name | organization | Name |
| ODSCode | ODS code | Organization::Identifier::Value (use=official, system=ods-organization-code) | organization | ods_code |
| ParentOrganisationGuid | Link to Admin_Organisation.OrganisationGuid | Organization::PartOf::Reference * | organization | parent_organization_id |
| CCGOrganisationGuid | Link to Admin_Organisation.OrganisationGuid | Organization::PartOf::Reference * | - | - |
| OrganisationType | Type of organisation | Organization::Type*** | organization | type_desc |
| OpenDate | Date organisation opened | Organization::primarycare-activeperiod-extension::ValuePeriod::Start | - | - |
| CloseDate | Date organisation closed | Organization::primarycare-activeperiod-extension::ValuePeriod::End | - | - |
| MainLocationGuid | Unique ID referencing Dimension.Location | Organization::primarycare-mainlocation-extension::ValueReference* | - | - |
| ProcessingId | Sequentual identifier indicating the order to process the items | ** | - | - |
Admin_OrganisationLocation
Each record is used to provide extra information for FHIR Location resources generated from the Admin_Location transform.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| OrganisationGuid | Link to Admin_Organisation.OrganisationGuid | ManagingOrganization* | - | - |
| LocationGuid | Link to Admin_Location.LocationGuid | <used to match to the corresponding FHIR Location> | - | - |
| IsMainLocation | Flag to indicate if this is the main location | <used to order the ManagingOrganization references> | - | - |
| Deleted | Indicates whether the link has been deleted | <not mapped, since the Admin_Location deleted field is used> | - | - |
| ProcessingId | Sequentual identifier indicating the order to process the items | ** | - | - |
Patient and users
Admin_Patient
Each record maps to a single FHIR Patient resource and a FHIR EpisodeOfCare resources. Note that if the patient is deducted and subsequently re-registers, then a second EpisodeOfCare resource is created that all future updates are mapped to.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| PatientGuid | Unique ID | Patient::Id*
EpisodeOfCare::Id* |
patient | id |
| OrganisationGuid | Link to Admin_Organisation.OrganisationGuid | EpisodeOfCare ::ManagingOrganization::Reference * | patient | organization_id |
| UsualGpUserInRoleGuid | Link to Admin_UserInRole.UserInRoleGuid | EpisodeOfCare ::CareManager::Reference *
Patient::CareProvider::Reference * |
- | - |
| Sex | Sex of patient | Patient::Gender | patient | gender_concept_id |
| DateOfBirth | Date of birth | Patient::BirthDate | patient | date_of_birth |
| DateOfDeath | Date of death | Patient::Deceased | patient | date_of_death |
| Title | Title | Patient::Name::Prefix (use=official) | patient | title |
| GivenName | Forename | Patient::Name::Given | patient | first_names |
| MiddleNames | Middle name | Patient::Name::Given | patient | first_names |
| Surname | Surname | Patient::Name::Family | patient | last_name |
| DateOfRegistration | Date of registration | EpisodeOfCare ::Period::Start
Also used as part of the EpisodeOfCare Id |
episode_of_care
registration_status_history |
date_registered
start_date |
| NhsNumber | NHS number | Patient::Identifier::Value (use=official, system=nhs-number) | patient | nhs_number |
| PatientNumber | Patient number | Patient::Identifier::Value (use=secondary, system=patient-number) | patient | person_id |
| PatientTypeDescription | Patient type | EpsisodeOfCare:: primarycare-patient-registration-type-extension::ValueCodeableConcept*** | episode_of_care | registration_type_concept_id |
| DummyType | Is a dummy patient | Patient::patient-is-test-patient-extension::ValueBoolean (extension is only created if value is true) | - | - |
| HouseNameFlatNumber | House name, flat no | Patient::Address::Line | patient_address | address_line_1 |
| NumberAndStreet | Number and street | Patient::Address::Line | patient_address | address_line_2 |
| Village | Village | Patient::Address::Line | patient_address | address_line_3 |
| Town | Town | Patient::Address::City | patient_address | city |
| County | County | Patient::Address::District | patient_address | address_line_4 |
| Postcode | Postcode | Patient::Address::PostalCode | patient_address | postcode |
| ResidentialInstituteCode | Residential institute code | Patient::primarycare-patient-residential-institute-code-extension::ValueString | - | - |
| NHSNumberStatus | PDS trace indicator | Patient::primarycare-nhs-number-verification-status-extension::ValueCodeableConcept
(this is either blank or “Verified” in all known cases; if any other value is ever received, an error will be thrown) |
- | - |
| CarerName | Name of carer | Patient::Contact::Name::Text | ||
| CarerRelation | Carer relationship to patient | Patient::Contact::Relationship::CodeableConcept | - | - |
| PersonGuid | Globally unique GUID for the patient | <not mapped> | - | - |
| DateofDeactivation | Date that the patient's registration became inactive | EpisodeOfCare ::Period::End | episode_of_care
registration_status_history |
date_registered_end
end_date |
| Deleted | Indicates whether the patient has fallen out of scope | <If true, all FHIR resources for this patient are deleted> | - | - |
| SpineSensitive | Flag to indicate the patient has been marked as sensitive | Patient::primarycare-patient-spine-sensitive-extension::ValueBoolean | - | - |
| IsConfidential | Flag to indicate patient is marked as confidential | Patient::primarycare-confidential::ValueBoolean (extension only created if value is true) | - | - |
| EmailAddress | Email address | Patient::Telecom::Value (use=home, system=email) | patient_contact | type_concept_id - value |
| HomePhone | Home phone | Patient::Telecom::Value (use=home, system=phone) | patient_contact | type_concept_id - value |
| MobilePhone | Mobile phone | Patient::Telecom::Value (use=mobile, system=phone) | patient_contact | type_concept_id - value |
| ExternalUsualGPGuid | Unique ID for External Usual GP | Patient::CareProvider::Reference * | - | - |
| ExternalUsualGP | External Usual GP Name | Patient::CareProvider::Reference * | - | - |
| ExternalUsualGPOrganisation | Link to Admin_Organisation.OrganisationGuid | patient | registered_practice_organization_id | |
| ProcessingId | Sequentual identifier indicating the order to process the items | ** | - | - |
Admin_UserInRole
Each record maps to a FHIR Practitioner resource.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| UserInRoleGuid | Unique ID | id* | practitioner | id |
| OrganisationGuid | Link to Admin_Organisation.OrganisationGuid | Role::ManagingOrganisation::Reference * | practitioner | organization_id |
| Title | Title | Name::Prefix | practitioner | name |
| GivenName | Forename | Name::Given | practitioner | name |
| Surname | Surname | Name::Family | practitioner | name |
| JobCategoryCode | Job category code | Role::Role::Code | practitioner | role_code |
| JobCategoryName | Job category name | Role::Role::Display | practitioner | role_desc |
| ContractStartDate | Date contract started | Role::Period::Start | - | - |
| ContractEndDate | Date contract ended | Role::Period::End | - | - |
| ProcessingId | Sequentual identifier indicating the order to process the items | ** | - | - |
Agreements_SharingOrganisation
Each record represents the state of the Emis-DDS sharing agreement and is not mapped to FHIR.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| OrganisationGuid | Unique ID | <not mapped> | - | - |
| IsActivated | Has the organisation activated their agreement | <not mapped> | - | - |
| LastModifiedDate | Date/Time of last modification | <not mapped> | - | - |
| Disabled | Has the organisation been disabled | <If true, and transform not configured to whitelist this organisation as deleted, then the transform will fail> | - | - |
| Deleted | Has the organisation been deleted | <not mapped> | - | - |
Appointments
Appointment_Session
Each record maps to a FHIR Schedule resource.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| AppointmentSessionGuid | Unique ID | Id* | schedule | id |
| Description | Name of session | Comment | schedule | name |
| LocationGuid | Link to Admin_Location.LocationGUID | primarycare-location-extension::ReferenceValue * | schedule | location |
| SessionTypeDescription | Session type | Type::Text | schedule | type |
| SessionCategoryDisplayName | Session category name | Comment | - | - |
| StartDate | Start date | PlanningHorizon::Start | schedule | start_date |
| StartTime | Start time | PlanningHorizon::Start | - | - |
| EndDate | End date | PlanningHorizon::End | - | - |
| EndTime | End time | PlanningHorizon::End | - | - |
| Private | Indicates whether the session is marked as private | <not mapped> | ||
| OrganisationGuid | Link to Admin_Organisation.OrganisationGuid | <not mapped> | schedule | organization_id |
| Deleted | Indicates whether the appointment session has been deleted | If true, FHIR resource is deleted | - | - |
| ProcessingId | Sequentual identifier indicating the order to process the items | ** | - | - |
Appointment_SessionUser
Each record is used to provide extra information for FHIR Practitioner resources generated from the Appointment_Session transform.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| SessionGuid | Link to Appointment_Session.AppointmentSessionGuid | <used to match to the corresponding FHIR Schedule> | appointment | schedule_id |
| UserInRoleGuid | Link to Admin_UserInRole.UserInRoleGuid | Actor::Reference * | appointment | practitioner_id |
| Deleted | Indicates whether the session user has been removed from the session | <if true, this “actor” is removed from the FHIR Schedule> | - | - |
| ProcessingId | Sequentual identifier indicating the order to process the items | ** | - | - |
Appointment_Slot
Each record maps to both a FHIR Appointment a FHIR Slot resource.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| SlotGuid | Unique ID | Appointment::Id *
Slot::Id * |
appointment | id |
| AppointmentDate | Date of appointment | Appointment::Start
Slot::Slot |
appointment | start_date |
| AppointmentStartTime | Time of appointment | Appointment::Start
Slot::Slot |
appointment | date_time_sent_in |
| PlannedDurationInMinutes | Planned duration of appointment in minutes | Appointment::End
Slot::End <number of minutes is added to start time to derive end time> |
appointment | planned_duration |
| PatientGuid | Link to Admin_Patient.PatientGuid | Appointment::Participant::Reference * | appointment | patient_id |
| SendInTime | Time patient was sent in | Appointment::primarycare-appointment-sent-in-extension::DateTimeValue
Appointment::Status (=Arrived) |
appointment | date_time_sent_in
appointment_status_concept_id |
| LeftTime | Time patient left | Appointment::primarycare-appointment-left-extension::DateTimeValue
Appointment::Status (=Fulfilled) |
appointment | date_time_left
appointment_status_concept_id |
| DidNotAttend | Did the patient attend Y/N | Appointment::Status (=NoShow) | appointment | appointment_status_concept_id |
| PatientWaitInMin | How long the patient waited for from their time of arrival | Appointment::primarycare-appointment-wait-extension::DurationValue | appointment | patient_wait |
| AppointmentDelayInMin | How long the patient waited for from their appointment time | Appointment::primarycare-appointment-delay-extension::DurationValue | appointment | patient_delay |
| ActualDurationInMinutes | Actual duration of appointment in minutes | Appointment::MinutesDuration | appointment | actual_duration |
| OrganisationGuid | Link to Admin_Organisation.OrganisationGuid | <not mapped> | appointment | organization_id |
| SessionGuid | Link to Appointment_Session.AppointmentSessionGuid | Slot::Schedule::Reference * | appointment | schedule_id |
| DnaReasonCodeId | Link to ClinicalCode.CodeId | Appointment::primarycare-appointment-dna-reason-extension::CodeableConceptValue (code and term looked up via Coding_ClinicalCode content) | - | - |
| Deleted | Indicates whether the appointment has been deleted | <If true, both FHIR resources are deleted> | - | - |
| ProcessingId | Sequentual identifier indicating the order to process the items | ** | - | - |
Care Record
CareRecord_Consultation
Each record maps to a FHIR Encounter resource.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| ConsultationGuid | Unique ID | Id *Ca | encounter | id |
| PatientGuid | Link to Admin_Patient.PatientGuid | Patient::Reference * | encounter | patient_id |
| OrganisationGuid | Link to Admin_Organisation.OrganisationGuid | ServiceProvider::Reference * | - | - |
| EffectiveDate | Clinically effective date | Period::Start | encounter | clinical_effective_date |
| EffectiveDatePrecision | Date accuracy | Period::Start | encounter | date_precision_concept_id |
| EnteredDate | Entered date | primarycare-recorded-date-extension::ValueDateTime | encounter | date_recorded |
| EnteredTime | Entered time | primarycare-recorded-date-extension::ValueDateTime | encounter | date_recorded |
| ClinicianUserInRoleGuid | Link to Admin_UserInRole.UserInRoleGuid | Participant::Reference * | encounter | practitioner_id |
| EnteredByUserInRoleGuid | Link to Admin_UserInRole.UserInRoleGuid | primarycare-recorded-by-extension::ValueReference * | encounter | practitioner_id |
| AppointmentSlotGuid | Link to Appointment_Slot.SlotGuid | Appointment::Reference * | encounter | appointment_id |
| ConsultationSourceTerm | Source of Consultation | primarycare-encounter-source::ValueCodeableConcept::Text | encounter | non_core_concept_id |
| ConsultationSourceCodeId | Link to ClinicalCode.CodeId | primarycare-encounter-source::ValueCodeableConcept **** | encounter | non_core_concept_id |
| Complete | Y/N | primarycare-encounter-incomplete::ValueBoolean (extension only set if value is false) | - | - |
| Deleted | Indicates whether the consultation has been deleted | <If true, FHIR resource is deleted> | - | - |
| IsConfidential | Flag to indicate observation is marked as confidential | Patient::primarycare-confidential::ValueBoolean (extension only created if value is true) | - | - |
| ProcessingId | Sequentual identifier indicating the order to process the items | ** | - | - |
CareRecord_Diary
Each record is mapped to a FHIR ProcedureRequest resource.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| DiaryGuid | Unique ID | Id * | procedure_request | id |
| PatientGuid | Link to Admin_Patient.PatientGuid | Subject::Reference * | procedure_request | patient_id |
| OrganisationGuid | Link to Admin_Organisation.OrganisationGuid | <not mapped> | - | - |
| EffectiveDate | Clinically effective date | ScheduledDateTime | procedure_request | clinical_effective_date |
| EffectiveDatePrecision | Date accuracy | ScheduledDateTime | procedure_request | date_precision_concept_id |
| EnteredDate | Date entry was created | OrderedOn | procedure_request | date_recorded |
| EnteredTime | Time entry was created | OrderedOn | procedure_request | date_recorded |
| ClinicianUserInRoleGuid | Link to Admin_UserInRole.UserInRoleGuid | Performer::Reference * | procedure_request | practitioner_id |
| EnteredByUserInRoleGuid | Link to Admin_UserInRole.UserInRoleGuid | Orderer::Reference * | procedure_request | practitioner_id |
| CodeId | Link to ClinicalCode.CodeId | Code **** | procedure_request | non_core_concept_id |
| OriginalTerm | The numeric value for result observations | Code::Text | procedure_request | non_core_concept_id |
| AssociatedText | Free text | Notes | - | - |
| DurationTerm | Duration free text | primarycare-procedure-request-schedule-text-extension::StringValue | - | - |
| LocationTypeDescription | Location type description | primarycare-procedure-request-location-extension::StringValue | - | - |
| Deleted | Indicates whether the diary entry has been deleted | <If true, FHIR resource is deleted> | - | - |
| IsConfidential | Flag to indicate observation is marked as confidential | Patient::primarycare-confidential::ValueBoolean (extension only created if value is true) | - | - |
| IsActive | Indicates whether the diary entry is currently active | Status (=Requested) | procedure_request | status_concept_id |
| IsComplete | Indicates whether the diary entry is complete | Status (=Completed) | procedure_request | status_concept_id |
| ConsultationGuid | Link to CareRecord_Consultation.ConsultationGuid | Encounter::Reference *
(the target Encounter resource is also updated with a reference back to this resource) |
procedure_request | encounter_id |
| ProcessingId | Sequentual identifier indicating the order to process the items | ** | - | - |
CareRecord_Observation
Care record observations are a high level entity type. CareRecord_Observation contains data which is mapped to multiple FHIR resource types. Target FHIR profiles depend on content.
Each record maps to one of a FHIR DiagnosticReport, Observation, Procedure, Condition, AllergyIntolerance, FamilyMemberHistory, Immunization, Specimen, ReferralRequest or DiagnosticOrder. The algorithm used to determine the target type is listed following the standard map
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| ObservationGuid | Unique ID | DiagnosticReport::Id * | observation | id |
| Observation::Id * | observation | id | ||
| Procedure::Id * | observation | id | ||
| Condition::Id * | observation | id | ||
| AllergyIntolerance::Id * | allergy_intolerance | id | ||
| FamilyMemberHistory::Id * | observation | id | ||
| Immunization::Id * | observation | id | ||
| Specimen::Id * | observation | id | ||
| ReferralRequest::Id * | referral_request | id | ||
| DiagnosticOrder::Id * | diagnostic_order | id | ||
| PatientGuid | Link to Admin_Patient.PatientGuid | DiagnosticReport::Subject::Reference* | observation | patient_id |
| Observation::Subject::Reference * | observation | patient_id | ||
| Procedure::Subject::Reference * | observation | patient_id | ||
| Condition::Patient::Reference * | observation | patient_id | ||
| AllergyIntolerance::Patient::Reference * | allergy_intolerance | patient_id | ||
| FamilyMemberHistory::Patient::Reference * | observation | patient_id | ||
| Immunization::Patient::Reference * | observation | patient_id | ||
| Specimen::Subject::Reference * | observation | patient_id | ||
| ReferralRequest::Patient::Reference * | referral_request | patient_id | ||
| DiagnosticOrder::Subject::Reference * | diagnostic_order | patient_id | ||
| OrganisationGuid | Link to Admin_Organisation.OrganisationGuid | <not mapped> | - | - |
| EffectiveDate | Clinically effective date | DiagnosticReport::EffectiveDateTime | observation | clinical_effective_date |
| Observation::EffectiveDateTime | observation | clinical_effective_date | ||
| Procedure::PerformedPeriod::Start | observation | clinical_effective_date | ||
| Condition::OnsetDateTime | observation | clinical_effective_date | ||
| AllergyIntolerance::Onset | allergy_intolerance | clinical_effective_date | ||
| FamilyMemberHistory::Date | observation | clinical_effective_date | ||
| Immunization::Date | observation | clinical_effective_date | ||
| Specimen::Collection::Collected::ValueDateTime | observation | clinical_effective_date | ||
| ReferralRequest::Date | referral_request | clinical_effective_date | ||
| DiagnosticOrder::Event::DateTime | diagnostic_order | clinical_effective_date | ||
| EffectiveDatePrecision | Date accuracy | DiagnosticReport::EffectiveDateTime | observation | date_precision_concept_id |
| Observation::EffectiveDateTime | observation | date_precision_concept_id | ||
| Procedure::PerformedPeriod::Start | observation | date_precision_concept_id | ||
| Condition::OnsetDateTime | observation | date_precision_concept_id | ||
| AllergyIntolerance::Onset | allergy_intolerance | date_precision_concept_id | ||
| FamilyMemberHistory::Date | observation | date_precision_concept_id | ||
| Immunization::Date | observation | date_precision_concept_id | ||
| Specimen::Collection::Collected::ValueDateTime | observation | date_precision_concept_id | ||
| ReferralRequest::Date | referral_request | date_precision_concept_id | ||
| DiagnosticOrder::Event::DateTime | diagnostic_order | date_precision_concept_id | ||
| EnteredDate | Date entry was created | DiagnosticReport::primarycare-recorded-date-extension::DateTimeValue or | observation | date_recorded |
| Observation::primarycare-recorded-date-extension::DateTimeValue | observation | date_recorded | ||
| Procedure:: primarycare-recorded-date-extension::DateTimeValue | observation | date_recorded | ||
| Condition::DateRecorded | observation | date_recorded | ||
| AllergyIntolerance::RecordedDate | allergy_intolerance | date_recorded | ||
| FamilyMemberHistory:: primarycare-recorded-date-extension::DateTimeValue | observation | date_recorded | ||
| Immunization:: primarycare-recorded-date-extension::DateTimeValue | observation | date_recorded | ||
| Specimen:: primarycare-recorded-date-extension::DateTimeValue | observation | date_recorded | ||
| ReferralRequest::primarycare-recorded-date-extension::DateTimeValue | referral_request | date_recorded | ||
| DiagnosticOrder::primarycare-recorded-date-extension::DateTimeValue | diagnostic_order | date_recorded | ||
| EnteredTime | Time entry was created | DiagnosticReport::primarycare-recorded-date-extension::DateTimeValue or | observation | date_recorded |
| Observation::primarycare-recorded-date-extension::DateTimeValue | observation | date_recorded | ||
| Procedure:: primarycare-recorded-date-extension::DateTimeValue | observation | date_recorded | ||
| Condition::DateRecorded | observation | date_recorded | ||
| AllergyIntolerance::RecordedDate | allergy_intolerance | date_recorded | ||
| FamilyMemberHistory:: primarycare-recorded-date-extension::DateTimeValue | observation | date_recorded | ||
| Immunization:: primarycare-recorded-date-extension::DateTimeValue | observation | date_recorded | ||
| Specimen:: primarycare-recorded-date-extension::DateTimeValue | observation | date_recorded | ||
| ReferralRequest::primarycare-recorded-date-extension::DateTimeValue | referral_request | date_recorded | ||
| DiagnosticOrder::primarycare-recorded-date-extension::DateTimeValue | diagnostic_order | date_recorded | ||
| ClinicianUserInRoleGuid | Link to Admin_UserInRole.UserInRoleGuid | DiagnosticReport::primarycare-diagnostic-report-filed-by-extension::ValueReference * | observation | practitioner_id |
| Observation::Performer::Reference * | observation | practitioner_id | ||
| Procedure::Performer::Reference * | observation | practitioner_id | ||
| Condition::Asserter::Reference * | observation | practitioner_id | ||
| AllergyIntolerance::Recorder::Reference * (note, this FHIR field should not have been used for this source field, but it’s too late to change without converting DDS content) | allergy_intolerance | practitioner_id | ||
| FamilyMemberHistory:: primarycare-family-member-history-reporter-extension::ValueReference * | observation | practitioner_id | ||
| Immunization::Performer::Reference * | observation | practitioner_id | ||
| Specimen::Collection::Collector::Reference * | observation | practitioner_id | ||
| ReferralRequest::Requester or Recipient::Reference * | referral_request | practitioner_id | ||
| DiagnosticOrder::Orderer::Reference * | diagnostic_order | practitioner_id | ||
| EnteredByUserInRoleGuid | Link to Admin_UserInRole.UserInRoleGuid | primarycare-recorded-by-extension::ValueReference *
(same extension used for all resources) |
- | - |
| ParentObservationGuid | If this observation has a parent code the link to the ParentObservationGuid is provided here | parent-resource::ValueReference *
(same extension used for all resources, and the target resource is also updated with a reference back to this one) |
observation | parent_observation_id |
| CodeId | Link to Coding_ClinicalCode.CodeId | DiagnosticReport::Code **** | observation | core_concept_id
non_core_concept_id |
| Observation::Code **** | observation | core_concept_id
non_core_concept_id | ||
| Procedure::Code **** | observation | core_concept_id
non_core_concept_id | ||
| Condition::Code **** | observation | core_concept_id
non_core_concept_id | ||
| AllergyIntolerance::Substance **** | allergy_intolerance | core_concept_id
non_core_concept_id | ||
| FamilyMemberHistory::Condition::Code **** | observation | core_concept_id
non_core_concept_id | ||
| Immunization::VaccinationCode **** | observation | core_concept_id
non_core_concept_id | ||
| Specimen::Type **** | observation | core_concept_id
non_core_concept_id | ||
| ReferralRequest::ServiceRequested::CodeableConcept **** | referral_request | core_concept_id
non_core_concept_id | ||
| DiagnosticOrder::Item::Code **** | diagnostic_order | core_concept_id
non_core_concept_id | ||
| ProblemGuid | Link to CareRecord_Problem.ObservationGuid | primarycare-condition-partofproblemepisode-extension::ValueReference *
(same extension used for all resources, and the target Condition resource is also updated with a reference back to this one) |
- | - |
| AssociatedText | Free text | DiagnosticReport::Conclusion | - | - |
| Observation::Comments | - | - | ||
| Procedure::Notes | - | - | ||
| Condition::Notes | - | - | ||
| AllergyIntolerance::Note | - | - | ||
| FamilyMemberHistory::Note | - | - | ||
| Immunization::Note | - | - | ||
| Specimen::Collection::Comment | - | - | ||
| ReferralRequest::Description | - | - | ||
| DiagnosticOrder::Note | - | - | ||
| ConsultationGuid | Link to CareRecord_Consultation.ConsultationGuid | DiagnosticReport::Encounter::Reference * | observation | encounter_id |
| Observation::Encounter::Reference * | observation | encounter_id | ||
| Procedure::Encounter::Reference * | observation | encounter_id | ||
| Condition:: Encounter::Reference * | observation | encounter_id | ||
| AllergyIntolerance::encounter-associatedEncounter::ReferenceValue * | allergy_intolerance | encounter_id | ||
| FamilyMemberHistory:: encounter-associatedEncounter::ReferenceValue * | observation | encounter_id | ||
| Immunization::Encounter::Reference * | observation | encounter_id | ||
| Specimen::encounter-associatedEncounter::ReferenceValue * | observation | encounter_id | ||
| ReferralRequest::Encounter::Reference * | referral_request | encounter_id | ||
| DiagnosticOrder::Encounter::Reference * | diagnostic_order | encounter_id | ||
| (the target Encounter resource is also updated with a reference back to this one) | ||||
| Value | The numeric value for result observations | <not mapped for DiagnosticReport> | - | - |
| Observation::ValueQuantity::Value | observation | result_value | ||
| <not mapped for Procedure> | - | - | ||
| <not mapped for Condition> | - | - | ||
| <not mapped for AllergyIntolerance> | - | - | ||
| <not mapped for FamilyMemberHistory> | - | - | ||
| <not mapped for Immunization> | - | - | ||
| <not mapped for Specimen> | - | - | ||
| <not mapped for ReferralRequest> | - | - | ||
| <not mapped for DiagnosticOrder> | - | - | ||
| NumericUnit | Unit | <not mapped for DiagnosticReport> | - | - |
| Observation::ValueQuantity::Unit | observation | result_value_units | ||
| <not mapped for Procedure> | - | - | ||
| <not mapped for Condition> | - | - | ||
| <not mapped for AllergyIntolerance> | - | - | ||
| <not mapped for FamilyMemberHistory> | - | - | ||
| <not mapped for Immunization> | - | - | ||
| <not mapped for Specimen> | - | - | ||
| <not mapped for ReferralRequest> | - | - | ||
| <not mapped for DiagnosticOrder> | - | - | ||
| ObservationType | Type of observation | <not mapped> | - | - |
| NumericRangeLow | Low Range | <not mapped for DiagnosticReport> | - | - |
| Observation::ReferenceRange::Low | observation_additional | property_id, json_value | ||
| <not mapped for Procedure> | - | - | ||
| <not mapped for Condition> | - | - | ||
| <not mapped for AllergyIntolerance> | - | - | ||
| <not mapped for FamilyMemberHistory> | - | - | ||
| <not mapped for Immunization> | - | - | ||
| <not mapped for Specimen> | - | - | ||
| <not mapped for ReferralRequest> | - | - | ||
| <not mapped for DiagnosticOrder> | - | - | ||
| NumericRangeHigh | High Range | <not mapped for DiagnosticReport> | - | - |
| Observation::ReferenceRange::High | observation_additional | property_id, json_value | ||
| <not mapped for Procedure> | - | - | ||
| <not mapped for Condition> | - | - | ||
| <not mapped for AllergyIntolerance> | - | - | ||
| <not mapped for FamilyMemberHistory> | - | - | ||
| <not mapped for Immunization> | - | - | ||
| <not mapped for Specimen> | - | - | ||
| <not mapped for ReferralRequest> | - | - | ||
| <not mapped for DiagnosticOrder> | - | - | ||
| DocumentGuid | Unique ID for associated attachment | primarycare-external-document-extension::ValueString
(same extension used on all resource types) |
- | - |
| Deleted | Indicates whether the clinical observation has been deleted | <If true, FHIR resource is deleted> | - | - |
| IsConfidential | Flag to indicate observation is marked as confidential | primarycare-confidential::ValueBoolean
(extension only created if value is true, and same extension used on all resource types) |
- | - |
| ProcessingId | Sequentual identifier indicating the order to process the items | - | - | - |
Observation content dependent rules
The target resource type is decided by the following rules, using the code and 'CodeType' from the Coding_ClinicalCode file (looked up using the CodeId from CareRecord_Observation) and the Value field from CareRecord_Observation.
| Code type Rule | Target profile |
| If the Value is empty, and the CodeType is one of (Biochemistry, Cyology_Histology,Haematology,Immunology,Microbiology,Radiology, Health_Management) | DiagnosticReport |
| If the Value is not empty OR the CodeType is one of
(Biochemistry,Biological_Values,Cytology_Histology, Haematology, Health_Management, Immunology,Microbiology,Radiology,Symptoms_Findings, Procedure (note, the codes is this category aren't actually "procedures"), Administration_Documents_Attachments, Body_Structure //dental structures,Care_Episode_Outcome, Dental_Finding,Diagnostics, Discharged_From_Service,EMIS_Qualifier,Ethnicity,HMP,Intervention_Category, Intervention_Target, KC60,;Marital_Status, Nationality,Nursing_Problem, Nursing_Problem_Domain, Obsteterics_Birth,Person_Health_Social, Planned_Dental, Problem_Rating_Scale, Reason_For_Care, Referral_Activity, Referral_Rejected, Referral_Withdrawn, Regiment, Religion, Trade_Branch, Unset) |
Observation
|
| If the CodeType is Conditions_Operations_Procedures and the Code is in the original READ 2 categories of (Preventive procedures, "Operations, procedures, sites", Other therapeutic procedures) | Procedure |
| If the CodeType is Conditions_Operations_Procedures and the Code is in the oringal READ 2 categories of (Infectious and parasitic diseases, Neoplasms,Endocrine, nutritional, metabolic and immunity disorders, Diseases of blood and blood-forming organs, Mental disorders, Nervous system and sense organ diseases, Circulatory system diseases, Respiratory system diseases, Digestive system diseases, Genitourinary system diseases, Skin and subcutaneous tissue diseases, Musculoskeletal and connective tissue diseases) | Condition
|
| Else If the CodeType is Conditions_Operations_Procedures | Observation |
| If the CodeType is any of (Allergy_Adverse_Drug_Reations, Allergy_Adverse_Reations) | AllergyIntolerance |
| If the CodeType is any of (Dental_Disorder, Dental_Procedure) | Condition |
| If the CodeType is (Family_History) | FamilyMemberHistory |
| f the CodeType is (Immunisations) | Immunization |
| If code type is (Investigation_Requests) | DiagnosticOrder |
| If code type is Pathology_Specimen | Specimen |
| If code type is Referral | Referral request |
There is a mismatch between EMIS observations and FHIR resource profiles
| Code type Rule | Target profile |
|
IF target profile IS Condition IF CSV Care Record observation ID is in CSV Care Record Problem ID |
Combine problem and Condition into problem onset profile |
|
In addition to the above mappings R1: IF the target resource select from above is not condition R2: IF the CSV CareRecord_Observation ID is in CSV CareRecord_Problem record ID |
Combine problem and other into Condition with "problem" profile |
| R2: ELSE | Observation |
| R1: Else : See R3 | |
|
R3: If the CSV Care Record Observation has PROBLEM ID R4: IF the CSV care record Observation ID is NOT the PROBLEM ID R5: IF the CSV care record CODE = the linked problem ID CODE |
Condition with episode indicator = review |
| (R3,R4,R5 : Else | No change |
CareRecord_ObservationReferral
This file is used to provide additional data for CareRecord_Observation records, populating the extra data onto FHIR ReferralRequest resources.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| ObservationGuid | Unique ID | <used to match to the CareRecord_Observation record> | referral_request | id |
| PatientGuid | Link to Dimension.Patient.PatientGuid | <used to match to the CareRecord_Observation record> | referral_request | patient_id |
| OrganisationGuid | Link to Admin_Organisation.OrganisationGuid | <not mapped> | ||
| ReferralTargetOrganisationGuid | Link to Admin_Organisation.OrganisationGuid | Recipient::Reference * | referral_request | recipient_organization_id |
| ReferralUrgency | Urgency of referral | Priority::CodeableConcept *** | referral_request | referral_request_priority_concept_id |
| ReferralServiceType | Service type | Type::CodeableConcept *** | referral_request | referral_request_type_concept_id |
| ReferralMode | Mode of referral | primarycare-referral-request-send-mode-extension::CodeableConcept *** | referral_request | mode |
| ReferralReceivedDate* | Date referral was received | <field is always empty> ***** | - | - |
| ReferralReceivedTime* | Time referral was received | <field is always empty> ***** | - | - |
| ReferralEndDate* | Date referral was ended | <field is always empty> ***** | - | - |
| ReferralSourceId* | Link to ClinicalCode.CodeId | <field is always empty> ***** | - | - |
| ReferralSourceOrganisationGuid* | Link to Admin_Organisation.OrganisationGuid | Requester::Reference * | referral_request | requester_organization_id |
| ReferralUBRN | Unique Booking Reference Number | Identifier::Value (use=official, system= ubrn) | - | - |
| ReferralReasonCodeId* | Link to ClinicalCode.CodeId | <field is always empty> ***** | - | - |
| ReferringCareProfessionalStaffGroupCodeId* | Link to ClinicalCode.CodeId | <field is always empty> ***** | - | - |
| ReferralEpisodeRTTMeasurementTypeId* | Link to ClinicalCode.CodeId | <field is always empty> ***** | - | - |
| ReferralEpisodeClosureDate* | Episode end date | <field is always empty> ***** | - | - |
| ReferralEpisodeDischargeLetterIssuedDate* | Discharge letter issued date | <field is always empty> ***** | - | - |
| ReferralClosureReasonCodeId* | Link to ClinicalCode.CodeId | <field is always empty> ***** | - | - |
| ProcessingId | Sequentual identifier indicating the order to process the items | ** | - | - |
CareRecord_Problem
This file is used to provide additional data for CareRecord_Observation records, populating the extra data onto FHIR Condition resources. Any FHIR Condition matched to by this file uses the “problem” FHIR profile rather than the standard “condition” profile.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| ObservationGuid | Unique ID | <used to match to the CareRecord_Observation record> | observation | id |
| PatientGuid | Link to Admin_Patient.PatientGuid | <used to match to the CareRecord_Observation record> | observation | patient_id |
| OrganisationGuid | Link to Admin_Organisation.OrganisationGuid | <not mapped> | - | - |
| ParentProblemObservationGuid | If this observation has a parent code the link to the ParentObservationGuid is provided here | primarycare-problem-related-extension::Target::ReferenceValue * | observation | parent_observation_id |
| Deleted | Indicates whether a problem has been deleted | <If true, the FHIR condition is downgraded from having the “problem” profile to the “condition” profile> | - | - |
| Comment | Problem comment (Free text) | <field not mapped as it’s always empty, but if data is ever detected, then an error will be raised so this can be implemented> | - | - |
| EndDate | Problem end date | AbatementDate | observation | problem_end_date |
| EndDatePrecision | Date accuracy | AbatementDate | - | - |
| ExpectedDuration | Expected duration of problem | primarycare-problem-expectedduration-extension::IntValue | - | - |
| LastReviewDate | Date Problem last reviewd | primarycare-problem-lastreviewed-extension::Date::DateValue | - | - |
| LastReviewDatePrecision | Date accuracy | primarycare-problem-lastreviewed-extension::Date::DateValue | - | - |
| LastReviewUserInRoleGuid | Link to Admin_UserInRole.UserInRoleGuid | primarycare-problem-lastreviewed-extension::Performer::ReferenceValue * | - | - |
| ParentProblemRelationship | Relationship to parent problem | primarycare-problem-related-extension::Type::StringType *** | - | - |
| ProblemStatusDescription | Status of the problem | AbatementBoolean <problem status is either “Active Problem” or “Past Problem”> | - | - |
| SignificanceDescription | Problem significance | primarycare-problem-significance-extension::CodeableConceptValue *** | observation_additional | property_id, value_id |
| ProcessingId | Sequentual identifier indicating the order to process the items | ** | - | - |
Coding_ClinicalCode
This file provides reference data on clinical codes, needed by other files. The content is loaded into a database table so records can be looked up. No FHIR resource is directly created from this file.
Coding_DrugCode
This file provides reference data on drug codes, needed by other files. The content is loaded into a database table so records can be looked up. No FHIR resource is directly created from this file.
Prescribing_DrugRecord
Each record in this file is mapped to a FHIR MedicationStatement resource.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| DrugRecordGuid | Unique ID | Id * | medication_statement | id |
| PatientGuid | Link to Admin_Patient.PatientGuid | Patient::Reference * | medication_statement | patient_id |
| OrganisationGuid | Link to Admin_Organisation.OrganisationGuid | <not mapped> | - | - |
| EffectiveDate | Clinically effective date | DateAsserted | medication_statement | clinical_effective_date |
| EffectiveDatePrecision | Date accuracy | DateAsserted | medication_statement | date_precision_concept_id |
| EnteredDate | Date entered | primarycare-recorded-date-extension::DateTimeValue | medication_statement | date_recorded |
| EnteredTime | Time entry was created | primarycare-recorded-date-extension::DateTimeValue | medication_statement | date_recorded |
| ClinicianUserInRoleGuid | Link to Admin_UserInRole.UserInRoleGuid | InformationSource::Reference * | medication_statement | practitioner_id |
| EnteredByUserInRoleGuid | Link to Admin_UserInRole.UserInRoleGuid | primarycare-recorded-by-extension::ReferenceValue * | medication_statement | practitioner_id |
| CodeId | Link to DrugCode.CodeId | Medication **** | medication_statement | core_concept_id |
| Dosage | Dosage | Dosage | medication_statement | dose |
| Quantity | Quantity | primarycare-medication-authorisation-quantity-extension::QuantityValue | medication_statement | quantity_value |
| QuantityUnit | Unit | primarycare-medication-authorisation-quantity-extension::QuantityValue | medication_statement | quantity_unit |
| ProblemObservationGuid | Link to CareRecord_Observation.ProblemGuid | ReasonForUseReference::Reference *
(target Condition resource is also updated with a reference back to this one) |
- | - |
| PrescriptionType | Type of prescription | primarycare-medication-authorisation-type-extension::CodeableConceptValue *** | medication_statement | authorisation_type_concept_id |
| IsActive | Is this still active | Status | medication_statement | is_active |
| CancellationDate | Date of Cancellation | primarycare-medication-authorisation-cancellation-extension::DateValue | medication_statement | cancellation_date |
| NumberOfIssues | No. of times drug has been issued on current authorisation | primarycare-medication-authorisation-numberofrepeatsissued-extension::IntValue | - | - |
| NumberOfIssuesAuthorised | No. of issues authorised | primarycare-medication-authorisation-numberofrepeatsallowed-extension::IntValue | - | - |
| IsConfidential | Flag to indicate observation is marked as confidential | Patient::primarycare-confidential::ValueBoolean (extension only created if value is true) | - | - |
| Deleted | Indicates whether the drug has been deleted | <If true, FHIR resource is deleted> | - | - |
| ProcessingId | Sequentual identifier indicating the order to process the items | ** | - | - |
Prescribing_IssueRecord
Each record in this file is mapped to a FHIR MedicationOrder resource.
| Publisher | DDS Core | Compass V2 | ||
|---|---|---|---|---|
| Field Name | Data/Coding Type | FHIR | Table | Field |
| IssueRecordGuid | Unique ID | Id * | medication_order | id |
| PatientGuid | Link to Admin_Patient.PatientGuid | Patient::Reference * | medication_order | patient_id |
| OrganisationGuid | Link to Admin_Organisation.OrganisationGuid | <not mapped> | medication_order | organization_id |
| DrugRecordGuid | Link to Prescribing_DrugRecord.DrugRecordGuid | primarycare-medication-order-authorisation-extension::ValueReference * | medication_order | medication_statement_id |
| EffectiveDate | Clinically effective date | DateWritten
(this date is also used to populate the primarycare-medication-authorisation-firstissuedate-extension::ValueDate and primarycare-medication-authorisation-mostrecentissuedate-extension::ValueDate extensions on the FHIR MedicationStatement) |
medication_order | clinical_effective_date |
| EffectiveDatePrecision | Date accuracy | DateWritten | medication_order | date_precision_concept_id |
| EnteredDate | Entered date | primarycare-recorded-date-extension::DateTimeValue | medication_order | date_recorded |
| EnteredTime | Time entry was created | primarycare-recorded-date-extension::DateTimeValue | medication_order | date_recorded |
| ClinicianUserInRoleGuid | Link to Admin_UserInRole.UserInRoleGuid | Prescriber::Reference * | medication_order | practitioner_id |
| EnteredByUserInRoleGuid | Link to Admin_UserInRole.UserInRoleGuid | primarycare-recorded-by-extension::ReferenceValue * | medication_order | practitioner_id |
| CodeId | Link to DrugCode.CodeId | Medication **** | medication_order | core_concept_id |
| Dosage | Dosage | DosageInstruction::Text | medication_order | dose |
| Quantity | Quantity | DispenseRequest::Quantity::Value | medication_order | quantity_value |
| QuantityUnit | Unit | DispenseRequest::Quantity::Unit | medication_order | quantity_unit |
| ProblemObservationGuid | Link to CareRecord_Observation.ProblemGuid | Reason::Reference *
(target Condition resource is also updated with a reference back to this one) |
- | - |
| CourseDurationInDays | Number of days the course was due to last (where known) | DispenseRequest.ExpectedSupplyDuration.Value | medication_order | duration_days |
| EstimatedNhsCost | Estimated NHS cost | primarycare-medication-order-estimatednhscost-extension::DecimalValue | medication_order | estimated_cost |
| IsConfidential | Flag to indicate observation is marked as confidential | Patient::primarycare-confidential::ValueBoolean (extension only created if value is true) | - | - |
| Deleted | Indicates whether the drug issue has been deleted | <If true, FHIR resource is deleted> | - | - |
| ProcessingId | Sequentual identifier indicating the order to process the items | ** | - | - |
*Source EMIS identifiers are translated to DDS UUIDs using a persisted mapping.
**ProcessingId columns are used to sequence records in the SFTP Reader application, but aren’t used in the FHIR transform.
***EMIS free-text mapped to value set for FHIR.
****Code and Term looked up from CodeId using Coding_ClinicalCode or Coding_DrugCode content.
******EMIS specification states that these fields will be empty.
Prescribing_DrugRecord End Dates
The Prescribing_DrugRecord file contains both a Boolean active indicator (IsActive) and the medication end date (CancellationDate). Within EMIS Web, it’s the IsActive indicator that is used to determine whether medication is active or not, the CancellationDate just providing additional information for ended medication. Analysing the received data has highlighted that there are a number of cases where the CancellationDate doesn’t match what the IsActive flag states:
1) IsActive = true but a CancellationDate is present; 1) IsActive = false but a CancellationDate is absent.
From the dates of affected data, these appear to be related to either a service moving to Emis Web from another GP system, and are an artefact of that transfer.
To ensure consistency in the FHIR MedicationStatements created from this file, the Cancellation date will only be carried over if the DrugRecord is non-active. If no cancellation date is present, it will be derived from the medication start and course length (found from the associated Prescribing_IssueRecord file).
Prescribing_DrugRecord End Dates
The Prescribing_DrugRecord file contains both a Boolean active indicator (IsActive) and the medication end date (CancellationDate). Within Emis Web, it’s the IsActive indicator that is used to determine whether medication is active or not, the CancellationDate just providing additional information for ended medication. Analysing the received data has highlighted that there are a number of cases where the CancellationDate doesn’t match what the IsActive flag states:
- IsActive = true but a CancellationDate is present;
- IsActive = false but a CancellationDate is absent.
From the dates of affected data, these appear to be related to either a service moving to Emis Web from another GP system, and are an artefact of that transfer.
To ensure consistency in the FHIR MedicationStatements created from this file, the Cancellation date will only be carried over if the DrugRecord is non-active. If no cancellation date is present, it will be derived from the medication start and course length (found from the associated Prescribing_IssueRecord file).
Special Considerations
The following sections describe all the special cases that the transform needs to support.
Version Support
The two versions of the specification supplied by EMIS are 5.1 and 5.3.1. The test pack data, although similar to the 5.1 specification has a number of differences, mostly column names being spelled incorrectly (e.g. “clinican” rather than clinician).
The extract data doesn’t declare its version in any of the files.
Because of the above two points, the transform will attempt to determine the version from the files and fields present, ensuring that all files/fields are consistent with a known format. For example, if a field contains a mis-typed column called “clinican” then it will validate that all other files/fields match the test pack format. If a known format cannot be matched, an error will be thrown. This will ensure that changes to the format (which are generally not announced) do not slip through unnoticed and relevant code changes can be implemented to support the changes.
Deleting Entire Records
Many of the files have a “Deleted” column to indicate that CareRecord_Observation, Prescrtiing_DrugRecord etc. should be deleted. The Admin_Patient file has a similar field, but we have determined that if this is set to true, the entire patient record should be deleted, not just data derived from the Admin_Patient record itself. Accordingly, the transform will support this, and treat an Admin_Patient delete as an instruction to delete all FHIR resources for the patient (within the scope of that service).
Deleting CareRecord_Problem Records
Problem records are represented by content in two files, CareRecord_Observation and CareRecord_Problem, both of which have a “Deleted” column. If an update is received with just “Deleted” true in CareRecord_Problem (and not in CareRecord_Observation), then the FHIR Condition resource should not be deleted, but should just be down-graded to being a non-problem Condition (i.e. change the profile URL).
If a CareRecord_Observation update is received with “Deleted” true, then the FHIR Condition should be deleted.
Deleting Deducted and Deceased Records
The data sharing agreement Emis have implemented for the DDS extracts includes active patients and non-active patients who were deducted or deceased in the last year. We have discovered that this rule is not only applied when a bulk extract is generated but is also applied to each daily extract.
For example, a bulk extract is received 01/01/2017 containing an active patient. If that patient leaves the practice on 01/02/2017, the delta extract the day after, on 02/02/2017, will inform of the deduction with an updated Admin_Patient record. A year later, on 02/02/2017 (or within a few days), a further update will be received for that patient with the “delete” field set to true.
To ensure that the DDS maintains patient records more than a year after deduction or death, the transform will ignore delete instructions for deducted or deceased patients except in the case where the Agreements_SharingOrganisation file (which represents the sharing agreement) shows that the agreement has been disabled (although extracts with disabled agreements are only processed if it is explicitly configured to do so for that organisation).
New Ethnicity and Marital Status Observations
If a patient has their ethnicity or marital status changed in Emis Web but no other demographic altered, the extract will contain the new CareRecord_Observation(s) but no Admin_Patient record.
The transform will ensure that these new ethnicities and marital statuses are added to the FHIR Patient resource even though no Admin_Patient record processing will be done for the patient.
Admin_Patient and Patient Re-registrations
The Admin_Patient file only contains one record per patient, giving the registration and deduction date (if present) of their most recent registration at the service. This means that if a patient is deducted and subsequently re-reigsters at the service, the Admin_Patient record is simply “updated” with the new registration date, and has the deduction date removed, overwriting any trace of their previous registration.
To avoid losing the history of past registrations, which affects analytics run using a past reference date, the transform will factor in the registration start date when creating EpisodeOfCare resources. Each distinct registration start date will result in a distinct EpisodeOfCare resource.
Missing Clinical and Drug Codes
Occasionally, when EMIS release new clinical and drug codes into Emis Web, they fail to release the same update to their extract servers. This means that once clinicians start using these new codes, the extracts contain content that refers to these new codes (e.g. CodeId on CareRecord_Observation) but no corresponding Coding_ClinicalCode or Coding_DrugCode record was ever received.
Since Coding_ClinicalCode and Coding_DrugCode are needed to look up Read2 and DM+D codes and terms, the transform should fail if it detects that this has happened, reporting an error so the issue can be raised with EMIS.
When this has previously happened, EMIS have generally corrected the issue within a few days, and the next extracts received contain the missing Coding_ClinicalCode and Coding_DrugCode records.
Although processing for the affected service(s) is halted while there are missing codes (which also prevents processing of the subsequent extracts for those services), any extract processed for any unaffected service will resolve the issue for the affected service, meaning that processing can be resumed.
“Disabled” Extracts
The Agreements_SharingOrganisation file represents the Emis-DDS sharing agreement. It has a “disabled” field which is intended to indicate if the service ever manually disables the sharing agreement, which may happen if the practice is closing.
However, on a semi-regular basis (around ten times per month) a service’s extract will come through showing the sharing agreement as disabled when the service hadn’t disabled it. After investigation, it was believed that if an error occurred during the generation of the Emis extract, then this would result in this field being wrongly set to true. Typically, whatever error occurred would be fixed automatically on the next day (Emis seemingly being unaware this is happening) and the extract generation would resume.
Prior to April 2019, when the above problem happened, the Admin_Patient file would also include a “delete” for every patient, and when the extract was fixed it would contain a complete re-bulk of the data. To prevent the DDS inadvertently processing this mass delete of data (which would then reflect the delete to all data subscribers), the transform detects the disabled sharing agreement and halts processing data for that service. To allow processing of data when a service has legitimately disabled the sharing agreement, a whilelist of allowed disabled practices is maintained in the application configuration. Only services on the whitelist will have extracts processed if their sharing agreement shows as disabled.
To prevent the major disruption of deleting all data for a service only to re-add it the next day, when the fixed extract was received, the extract files for the date the problem happened (which will not have been processed) were replaced with new ones without the delete statement and with the sharing agreement changed to be not disabled. Additionally, a routine was run on previously received files and the re-bulk to detect any clinical records (e.g. CareRecord_Observation) that had legitimately been deleted on the day of the problem. These new files were then processed by the transform, processing the small number of actual deletes, if any, and then the re-bulk files were processed. The generic FHIR filing code supports re-processing data already saved, detecting duplicates and ignoring them, so processing this bulk data didn’t cause a problem (except in the time taken to do so).
Towards the end of April 2019, EMIS seemingly changed their extract software (although no communication was received to say so). From this date onwards, when a service goes into the disabled state, no delete is received for any patient data. When the feed resumes, the next extract simply contains all the changes since the previous valid extract – so only the sharing agreement file needs to be corrected to allow processing to continue.
Non-Patient (Admin) Resources
Each of the extract feeds from EMIS contains multiple services, up to around fifty. When each feed was first started, the first extract files received contained all the non-patient data, such as Organisations (in Admin_Organisation) and Practitioners (in Admin_UserInRole). When new services are added to a feed, these non-patient files aren’t re-sent.
However, DDS requires each service to have its own set of non-patient data, so that it can be co-located in the same FHIR database. So the transform much retain the non-patient data for subsequent processing as and when new services are added to the extract feed. This is done using the EMIS Admin Cache, which is a table used to store the non-patient data on a per-extract feed basis. When a new service is added to the extract, this is detected and all non-patient data for that extract feed is transformed and saved before any actual data for the new service.
New Children for Pre-existing Observations
Although not common, it’s possible to receive CareRecord_Observation records that refer to ParentObservationGuid records that were received in a previous extract. Since ParentObservationGuid relationships are maintained in both child and parent resource (i.e. the child has a Reference to the parent and the parent has a Reference to the child) the transform must update any pre-existing parents with References to any new children.
New Children for Pre-existing Consultations
It’s possible to receive CareRecord_Observation and CareRecord_Diary records that refer to CareRecord_Consultation records received previously. Since ConsultationGuid relationships are maintained in both Encounter and child resources (i.e. the Encounter has References to its child resources, and the child resources have an Encounter reference), the transform must ensure that pre-existing Encounter resources are updated with any newly received child References.
New Children for Pre-existing Problems
When new observations or medications are recorded in Emis Web and linked to a problem, the subsequent update included the new CareRecord_Observation, Prescribing_DrugRecord or Prescribing_IssueRecord records with the ProblemObservationGuid set, but will not include any details on the problem itself (unless the problem was amended in some other way). Problem-child linkage is represented in both Condition and child resources (i.e. the Condition has References to its child resources, and the child resource have a Condition reference). The transform must ensure that the pre-existing FHIR Condition is updated with any newly received child References.
New IssueRecords for Pre-existing DrugRecords
If an existing, active repeat medication template is issued in Emis Web, the subsequent extract will contain a new record in Prescribing_IssueRecord, but no corresponding update to the Prescribing_DrugRecord file. The transform must handle this, not only saving the new FHIR MedicationOrder, but updating the “last issue date” extension on the linked FHIR MedicationStatement.
Updates to Pre-existing Consultations
If an aspect of a previously saved consultation is changed, the subsequent update will include the updated CareRecord_Consultation record. However, the extract won’t contain any linked records that weren’t directly updated themselves. In this scenario, the transform should ensure that any child References on the previously saved consultation are carried over onto the new, updated version.
Updates to Pre-existing Problems
When a “problem” is recorded in EMIS Web, this new information is represented in both the CareRecord_Observation and CareRecord_Problem files, the former containing the common observation fields (date, code etc.) and the latter file having problem-specific fields. From that point on, if any of the problem-specific fields is amended, the subsequent extract may only contain an update in the CareRecord_Problem file, without an update to CareRecord_Observation. So the transform must support an update to a Condition resource (with the “problem” profile) that is only present in the CareRecord_Problem file.
The reverse, however, is not the case. If one of the common observation fields is amended in EMIS Web, the subsequent extract will contain both CareRecord_Observation and CareRecord_Problem records for the problem.
In both cases, the list of child resources on the pre-existing Condition resource should also be carried over to the new, updated version.
Blood Pressures
EMIS Web represents BP recordings using three CareRecord_Observation records: observations each with numeric value for the systolic and diastolic, both linked to a common parent observation.
In addition to the normal transform rules for CareRecord_Observation records, which will result in three FHIR Observations, with links between parent and children, the transform should also duplicate the numeric values in the Component structure on the parent Observation.
Read Code Standardisation
Whenever a FHIR resource is created or updated that contains a Read2 code (e.g. FHIR observation) the Read2 code is standardised to be in the five-character format used in TRUD files and other suppliers (e.g. Vision). This means that any code shorter than five characters is padded to five with dots. So “C10” is stored as “C10..”.
Ignoring Bulks
In a number of cases, multiple bulks of services were received before the migration of DDS to AWS. To reduce the amount of processing needed to be done to load the data into the AWS-based DDS, only the later bulk was processed, the earlier one (and any deltas before the subsequent re-bulk) being ignored. This was managed on a service-by-service basis, with the date of the second bulk being configured against the service ODS code and any patient-related data (e.g. Admin_Patient, CareRecord_Observation) before this date being ignored by the transform. Non-patient-related files for this period are still processed, so updates to staff, organisations etc. aren’t missed, as these files aren’t re-bulked when patient data is.
Appointment Cancellations
The Appointment_Slot file contains all the details of appointment slots, plus any details of appointments booked into those slots. If an appointment is cancelled, the following update will contain an updated Appointment_Slot record, but simply without any appointment details in it (or with different appointment details if the slot was rebooked before the extract).
The transformer must detect that a slot record has a different PatientGuid to the previous extract(s) and if so, mark the previously saved FHIR Appointment resource as cancelled.
Detecting Problem Reviews
EMIS Web stores Problem reviews with a new Observation record with the same code as the Problem itself and linked to the Problem. Within Emis Web this is flagged as a review, so it is distinguishable from a new diagnosis. The review flag isn’t present in the extract, so it is necessary to infer the review status from the data available.
When a CareRecord_Observation record is received, if it links to a Problem (i.e. ProblemGuid isn’t empty) and the ReadCode for the new Observation is the same as that for the Problem (looked up using Coding_ClinicalCode content) then the new Observation will be flagged as a review, using the primarycare-problem-review extension.
Note that this will correctly not flag the initial Observation as a review, because the Observation that defines the Problem isn’t linked to the Problem in the same way (i.e. the ProblemGuid is empty but the CareRecord_Problem file links back to the Observation).
Mapping to FHIR Value Sets
There are a number of fields in the Emis extract that are mapped to FHIR fields that are value sets. In these cases, the mappings are performed using the files listed below. In each case, if a code is found that isn’t in the relevant mapping file, then an exception is thrown and the transform halted.
Mapping Ethnicity Codes
If the CodeType of a CareRecord_Observation is “Ethnicity”, then the record is used to populate the Ethnicity extension on the Patient resource (in addition to creating an Observation FHIR resource). The Patient extension requires the ethnicity to be mapped to a value set, which is done using these mappings: https://github.com/endeavourhealth/Transforms/blob/master/src/main/resources/EmisEthnicityMap.csv
Mapping Marital Status Codes
If the CodeType of a CareRecord_Observation is “Marital_Status”, then the record is used to populate the MaritalStatus field on the Patient resource (in addition to creating an Observation FHIR resource). The Patient resource requires the marital status to be mapped to a value set, which is done using these mappings: https://github.com/endeavourhealth/Transforms/blob/master/src/main/resources/EmisMaritalStatusMap.csv