Welcome to the London Discovery Data Service knowledge base: Difference between revisions

From Discovery Data Service
Jump to navigation Jump to search
m (fixed typo)
 
(80 intermediate revisions by 5 users not shown)
Line 1: Line 1:
{{DISPLAYTITLE:Welcome to the London Discovery Data Service knowledge base}}
__TOC__


The articles describe and define Discovery Data Service functionality and technology; the architecture that underpins the data service software, the components that make up the service, the testing and assurance processes, and the resources, technologies, and software that are used to host, develop, and support the service.
These articles describe the London Discovery Data Service; the architecture that underpins the data service, the components that make up the service, the testing and assurance processes, and the resources, open source technologies, and software that are used to host, develop, and support the service.  


= Discovery overview =
=London Discovery Data Service status=
 
* The data service is live.
* It currently stores and processes the data from 39 million linked organisational patient records in the UK.
* It contains long term records on 15.7 million unique UK citizens who have been resident in London at some point in the last 30 years (including people who have left or died).
* Data on individuals are held in a single common data model with data values represented by a super ontology composed of the world's leading health ontologies.
* Data in Discovery is submitted by health providers who provide primary, secondary, and urgent care services to patients or clients.
* Data is kept up to date in close to real time - at worst within 24 hours; see [[Latency of extract data feeds]] for more details.
 
= London Discovery overview =


These articles provide background to the ideas, philosophies and approaches that have resulted in the Discovery Data Service. They cover the original objectives, the approach to collaboration and shared tenancy, and the information governance rules that underpin the way privacy and sharing are balanced.
These articles provide background to the ideas, philosophies and approaches that have resulted in the Discovery Data Service. They cover the original objectives, the approach to collaboration and shared tenancy, and the information governance rules that underpin the way privacy and sharing are balanced.


*[[What_is_Discovery|What is Discovery]]? - This article describes what Discovery is and how it is conceptualised at a very high level.
*[[What_is_Discovery|What is Discovery]]? - This article describes what Discovery is and how it is conceptualised at a very high level.
*[[Case studies]] - An overview of projects that access and utilise data from Discovery; information includes the applicant, project summary, and the latest status (Development/Live).
*[[Case studies]] - An overview of some initial projects that access and utilise data from Discovery; information includes the applicant, project summary, and the latest status (Development/Live).
*[[Release news]] - The latest information for all Discovery releases and changes.
*[[Latest news]] - The latest information for all Discovery Data Service releases and changes, plus known issues and updates from system suppliers and other third parties.
*[[Current_data_sets|Current data sets]] - the broad types of data received from various systems.
*[[Current_data_sets|Current data sets]] - the broad types of data received from various systems.
*[[Current published data]] - provides the latest data publisher information by sector, geographical area, service, and system provider.
*[[Current published data]] - provides the latest data publisher information by sector, geographical area, service, and system provider.


= Information model related topics =
= Information model topics =


These articles provide information about the services, structure, and functionality that operate using the Information Models; one of the fundamental tenets of the Discovery Data Service. This is the starting point for those interested in health data and the information derived from it, and why Discovery may be different from other approaches:
This is the starting point for those interested in health data and the information derived from it, and why Discovery may be different from other approaches:


*[[Discovery health information model|Discovery Information model]] - An overview of the model(s), the purpose, and type of content. This does not include the actual content
The [https://wiki.endeavourhealth.org/index.php?title=Welcome_to_the_Endeavour_Health_knowledge_base health information model services] are maintained as open source by the Endeavour health charitable trust. They use standard W3C semantic web languages to model health data, concepts, sets, and queries, thus maintaining compatibility with the main stream technologies.
*[[Health Information modelling language]] - The language used to build the various components of the information models
*[[Health_data_model_content|Discovery health data model ]] - a very high level overview of the types of data held within the Discovery data stores.
*[[Information_model_service|Information model services]] - the technical services , such as APIs and apps that enable an model instance to be used or to implement an EHR based on a model
*[[Information modelling language|Discovery information modelling language]] - describes the machine and human readable syntax used to build the information models and for computers to process or manipulate data


= Operational and interoperability topics =
= Publishers and subscribers =
These articles provide information about how systems can interoperate with Discovery and provide further links into test APIs, schemas, and other documentation associated with data formats for sending and receiving data.
These articles provide some brief high level information for organisations that send data to Discovery and those that use that data.


*[[Secure_publication_to_the_DDS|Secure publication to the DDS]] - lists the technical interfaces in place for receiving data from publishers.  
* [[Publishers]] - includes technical interface information, FHIR mapping, system requirements, and how to make changes to publishing services.
*[[Discovery Remote Subscriber Database Familiarisation]] - helps users of the DDS Remote Subscriber Database to test, understand, and maximise the benefits of the system (includes [[Discovery Remote Subscriber Database Familiarisation#Data_Mapping|data mapping and schema information]]).
* [[Subscribers]] - includes set up, data mapping, and schema information.
*[[Remote_Subscriber_Database_(RSD)_AWS_hosting_setup_guide|Remote Subscriber Database (RSD) AWS hosting setup guide]] - provides information for subscribers who want to set up their data in AWS.
*[[National data opt-out]] - includes information on the NHS national data opt-out and how this is applied in the Discovery Data Service.
*[[Generic_publisher_system_data_requirements|Generic publisher system data requirements]] - provides a set of guides for systems that are publishing data through the Discovery Data Service.   
*[[Publishers_and_mapping_to_the_commmon_model|Publishers and mapping to the common model]] - provides the specification for how supplier system information is mapped to the common model, using FHIR as the intermediary.
*[[Changes to a publishing service]] - describes what you need to do to make changes to a publishing service.


= Services and applications =
= Services and applications =


These articles provide information about the services and applications provided by the Discovery Data Service and the underlying functionality.
These articles provide information about the services and applications provided by the Discovery information services.
 
* [[Discovery Query|Sextant]] - A data set distribution service with an option to post the data directly into the recipients database.
*[[Monitoring and alerting]] - provides an overview of the monitoring and alert systems in place.
 
*[[Data Sharing Manager|Data Sharing manager]]- DSM provides a visual representation of data that is being shared and processed and by which organisations, regions, and/or services.


*[[UPRN address match application|UPRN address matching application -]] a web based application that matches single or batches of hand entered address to authoritative addresses and assigns a unique property reference number.
*[[UPRN address match application|ASSIGN- UPRN address matching application -]] a web based application that matches single or batches of hand entered address to authoritative addresses and assigns a unique property reference number.
*[[Information model manager]] - an application designed to provide a view of the common information model, and support the authoring of the content of the model.
*[[Information_model_service|Information model services]] - the technical services that enable the common information model to be used.
*[[FHIR Get Structured Record API]] - returns a complete patient record in a structured format following GP and Care Connect standards.
*[[FHIR Get Structured Record API]] - returns a complete patient record in a structured format following GP and Care Connect standards.
*[[HL7v2 API]] - Send HL7v2 ADT and ORU (clinical events) data to Discovery.
*[[HL7v2 API]] - Send HL7v2 ADT and ORU (clinical events) data to Discovery.
Line 45: Line 52:
*Data set distribution service - the service that distributes sets of data for populations of patients where the definition is complex or where the subscriber is hosting a copy of the data permanently.
*Data set distribution service - the service that distributes sets of data for populations of patients where the definition is complex or where the subscriber is hosting a copy of the data permanently.
*[[NHS 111 Discovery frailty flagging]] / [[Frailty algorithm]]
*[[NHS 111 Discovery frailty flagging]] / [[Frailty algorithm]]
*[[GitHub repositories]] - descriptions and information relating to the application source code.


= Architectures =
= Architectures =
These articles provide information about the overall high level software architecture models providing a descriptions and visualisations of the Discovery Data Service and information service components and how they interact.
These articles provide information about the overall high level software architecture models providing a descriptions and visualisations of the Discovery Data Service and information service components and how they interact.


*The [[Data Storage Architecture]] illustrates how data is stored within the service and the various implementation options available for organisations that wish to use some or parts of the services.
* The [[Data Storage Architecture]] illustrates how data is stored within the service and the various implementation options available for organisations that wish to use some or parts of the services.
*[[Identity Authentication Authorisation|Identity, authentication and authorisation]] specifies the approach taken by Discovery for accessing utilities and controlling some aspects of access to application functionality and some data resources, after all of the other locks have been opened.
*[[Identity Authentication Authorisation|Identity, authentication and authorisation]] specifies the approach taken by Discovery for accessing utilities and controlling some aspects of access to application functionality and some data resources, after all of the other locks have been opened.
*[[Application zone architecture]] illustrates how the data is accessed in different places and in different ways according to different use cases  
*[[Application zone architecture]] illustrates how the data is accessed in different places and in different ways according to different use cases  
*[[Pseudonymisation]] describes how Discovery generates pseudonymised data.
*[[Pseudonymisation]] describes how Discovery generates pseudonymised data.
*[[Data linking]] describes how Discovery links several patient records into a single person/citizen record.
*[[Data linking]] describes how Discovery links several patient records into a single person/citizen record.
<br />
= Publisher system known issues =
This section documents the known long standing problems relating to systems that publish data to Discovery. The issues as categorised by supplier and system. These issues need to be taken into account when configuring the services to process the data. The issues do not include short term glitches or errors.
*[[EMIS_known_issues|EMIS]]
*[[TPP_known_issues|TPP]]


= Application help =
= Application help =
Line 68: Line 67:
*[[Data_Sharing_Manager|Data Sharing Manager]]&nbsp;-&nbsp;Data Sharing Manager provides a visual representation of data that is being shared and processed and by which organisations, regions, and/or services.
*[[Data_Sharing_Manager|Data Sharing Manager]]&nbsp;-&nbsp;Data Sharing Manager provides a visual representation of data that is being shared and processed and by which organisations, regions, and/or services.
*:You can also amend or cancel data sharing and processing agreements in real time.
*:You can also amend or cancel data sharing and processing agreements in real time.
*[[Integrated Patient Care Record Viewer|Integrated Patient Care Record Viewer]] - ***currently under construction***
*[[Record Viewer]] - Record Viewer shows a combined view of an individual's patient records, using data from different source organisations.


= Wiki tips =
= Wiki tips =
Line 77: Line 76:
*[[Creating a DDS wiki watchlist]]
*[[Creating a DDS wiki watchlist]]
*[[Creating a new user]]
*[[Creating a new user]]
*[https://en.wikipedia.org/wiki/Help:List Wikipedia Help:Lists]
*[[Test page]]

Latest revision as of 12:20, 12 September 2023

These articles describe the London Discovery Data Service; the architecture that underpins the data service, the components that make up the service, the testing and assurance processes, and the resources, open source technologies, and software that are used to host, develop, and support the service.

London Discovery Data Service status

  • The data service is live.
  • It currently stores and processes the data from 39 million linked organisational patient records in the UK.
  • It contains long term records on 15.7 million unique UK citizens who have been resident in London at some point in the last 30 years (including people who have left or died).
  • Data on individuals are held in a single common data model with data values represented by a super ontology composed of the world's leading health ontologies.
  • Data in Discovery is submitted by health providers who provide primary, secondary, and urgent care services to patients or clients.
  • Data is kept up to date in close to real time - at worst within 24 hours; see Latency of extract data feeds for more details.

London Discovery overview

These articles provide background to the ideas, philosophies and approaches that have resulted in the Discovery Data Service. They cover the original objectives, the approach to collaboration and shared tenancy, and the information governance rules that underpin the way privacy and sharing are balanced.

  • What is Discovery? - This article describes what Discovery is and how it is conceptualised at a very high level.
  • Case studies - An overview of some initial projects that access and utilise data from Discovery; information includes the applicant, project summary, and the latest status (Development/Live).
  • Latest news - The latest information for all Discovery Data Service releases and changes, plus known issues and updates from system suppliers and other third parties.
  • Current data sets - the broad types of data received from various systems.
  • Current published data - provides the latest data publisher information by sector, geographical area, service, and system provider.

Information model topics

This is the starting point for those interested in health data and the information derived from it, and why Discovery may be different from other approaches:

The health information model services are maintained as open source by the Endeavour health charitable trust. They use standard W3C semantic web languages to model health data, concepts, sets, and queries, thus maintaining compatibility with the main stream technologies.

Publishers and subscribers

These articles provide some brief high level information for organisations that send data to Discovery and those that use that data.

  • Publishers - includes technical interface information, FHIR mapping, system requirements, and how to make changes to publishing services.
  • Subscribers - includes set up, data mapping, and schema information.
  • National data opt-out - includes information on the NHS national data opt-out and how this is applied in the Discovery Data Service.

Services and applications

These articles provide information about the services and applications provided by the Discovery information services.

  • Sextant - A data set distribution service with an option to post the data directly into the recipients database.
  • Monitoring and alerting - provides an overview of the monitoring and alert systems in place.
  • Data Sharing manager- DSM provides a visual representation of data that is being shared and processed and by which organisations, regions, and/or services.
  • ASSIGN- UPRN address matching application - a web based application that matches single or batches of hand entered address to authoritative addresses and assigns a unique property reference number.
  • FHIR Get Structured Record API - returns a complete patient record in a structured format following GP and Care Connect standards.
  • HL7v2 API - Send HL7v2 ADT and ORU (clinical events) data to Discovery.
  • Get health record - the service that obtains a fully structured health record about a person, in real time via a standards based API.
  • Record publication service - the service that publishes data from health records for specific purposes, often immediately on receipt.
  • Data set distribution service - the service that distributes sets of data for populations of patients where the definition is complex or where the subscriber is hosting a copy of the data permanently.
  • NHS 111 Discovery frailty flagging / Frailty algorithm
  • GitHub repositories - descriptions and information relating to the application source code.

Architectures

These articles provide information about the overall high level software architecture models providing a descriptions and visualisations of the Discovery Data Service and information service components and how they interact.

  • The Data Storage Architecture illustrates how data is stored within the service and the various implementation options available for organisations that wish to use some or parts of the services.
  • Identity, authentication and authorisation specifies the approach taken by Discovery for accessing utilities and controlling some aspects of access to application functionality and some data resources, after all of the other locks have been opened.
  • Application zone architecture illustrates how the data is accessed in different places and in different ways according to different use cases
  • Pseudonymisation describes how Discovery generates pseudonymised data.
  • Data linking describes how Discovery links several patient records into a single person/citizen record.

Application help

  • Data Sharing Manager - Data Sharing Manager provides a visual representation of data that is being shared and processed and by which organisations, regions, and/or services.
    You can also amend or cancel data sharing and processing agreements in real time.
  • Record Viewer - Record Viewer shows a combined view of an individual's patient records, using data from different source organisations.

Wiki tips