What is Discovery: Difference between revisions

From Discovery Data Service
Jump to navigation Jump to search
No edit summary
No edit summary
Line 2: Line 2:
Discovery can be conceptualised via 3 axes; The Discovery idea, the Discovery collaboration, and the actual set of systems that make up the Discovery data service.
Discovery can be conceptualised via 3 axes; The Discovery idea, the Discovery collaboration, and the actual set of systems that make up the Discovery data service.


== The Discovery idea ==
= The Discovery idea =


The idea behind discovery is that if health related data is brought together at the level of the individual, and stored together at the level of a medium size residential population, that great benefits to health can acccrue as a result of algorithms operating on indvidual's data or on the population as a whole. 
The idea behind discovery is that if health related data is brought together at the level of the individual, and stored together at the level of a medium size residential population, that great benefits to health can accrue as a result of algorithms operating on individuals' data or on the population as a whole. 


This idea is specialised to the need to bring data from many different providers and provider systems, and convert or map that data to ''common information model, ''in order that all the data can be accessed by a single language, as if it was designed as a single system. The main difference between Discovery and an organisational management system is that it sources data from many disparate domains and makes sense of the data, whether the data was originally recorded in a standard or non standard form. In other words, all the useful data entered over many years, becomes accessible in a modern and coherent form. 
This idea is specialised to the need to bring data from many different providers and provider systems, and convert or map that data to ''common information model, ''in order that all the data can be accessed by a single language, as if it was designed as a single system. The main difference between Discovery and an organisational management system is that it sources data from many disparate domains and makes sense of the data, whether the data was originally recorded in a standard or non standard form. In other words, all the useful data entered over many years, becomes accessible in a modern and coherent form. 
Line 10: Line 10:
Examples of the way in which this data can be used include;
Examples of the way in which this data can be used include;


At an individual level. When an NHS 111 call handler receives a call from a citizen, the ciitizen's primary and swcondary care data is immediately analysis to detect whether they have fraility, thus resulting in directing the call to a clinician in order to avoid a dangerous trip to hospital.
At an individual level:
<pre><span style="font-family:Georgia,serif;">When an NHS 111 call handler receives a call from a citizen, analyse the primary and secondary care data in real time to detect whether they have frailty, thus resulting in directing the call to a clinician in order to avoid a dangerous trip to hospital</span>
</pre>


At a population level. Using evidence from primary care, acute care, detect the small subset of people with Asthma who appear to have a profile suggesting that they are at risk of death or prolonged hospitalisation.
At a population level:
<pre><span style="font-family:Georgia,serif;">Using evidence from primary care, acute care, detect the small subset of people with Asthma who appear to have a profile suggesting that they are at risk of death or prolonged hospitalisation</span></pre>


== The Discovery collaboratives ==
= The Discovery collaborations =


An idea by itself is not enough. A Discovery collaborative is any local or regional group of providers and commissioners who are willing to work together to establish and ''own''&nbsp;and control their Discovery Data Service. A collaboration applies the following philosophies;
An idea by itself is not enough. A Discovery collaborative is any local or regional group of providers and commissioners who are willing to work together to establish and ''own''&nbsp;and control their Discovery Data Service. A collaboration applies the following philosophies;


#All technicologies, specfications and doecumentation is open source and available for all, including those not within the collaborative. This protects the collaborative against itself and provides the basis for other collaboratives to contribute in a mutually beneficial way.  
#All technologies, speciations and documentation is open source and available for all, including those not within the collaborative. This protects the collaborative against itself and provides the basis for other collaborative to contribute in a mutually beneficial way.  
#All collaboratives are of unlimited size. There is no limit to the members of a collaborative and collaboratives do not attempt to preclude others from joining. Having said that collaborations by the nature of the population coverage of a particular data service, will tend to be local or regional.&nbsp;  
#All collaborative are of unlimited size. There is no limit to the members of a collaborative and collaborative do not attempt to preclude others from joining. Having said that collaborations by the nature of the population coverage of a particular data service, will tend to be local or regional.&nbsp;  
#The collaboration must apply the information governance policies common to Discovery. These include the retention of data controller status by providers, until data is transferred to subscribers as a result of consent by the providers, and the proper handling of identifiable and de-identified data  
#The collaboration must apply the information governance policies common to Discovery. These include the retention of data controller status by providers, until data is transferred to subscribers as a result of consent by the providers, and the proper handling of identifiable and de-identified data  
#&nbsp;
 
= The Discovery data service =
 
Having an idea and a collaboration is not enough. The Discovery data service is the set of systems and human operatives that deliver access to the data whenever needed.
 
The underlying paradigm of the service is known as pub/sub, or more precisely, 'Publish, store and queue, Subscribe'.&nbsp;In this paradigm the following summarises the process which is often referred to as a 'Pipeline'.
 
*All health related events are 'published' to the data service by the provider, either from their system, or through middleware, in whatever form they are able, in as near real time as possible.&nbsp;
*The Data service receives&nbsp;the data, (having checked that the data processing agreements are in place), transforms to a common format, inks at the level of the citizen, using the trusted identifier and stores the data. At this point the data is controlled by the publisher i.e. the Discovery Data service operates on behalf of its publishers, who themselves are part of a collaborative.
*A second stage of processing occurs to 'map' the original data content to the common information model, including the mapping to the Discovery ontology and the Discovery data model. The original data as published is maintained, the initial transform of the data is maintained and full provenance of the data is also maintained, uncluding any provenance of the data made available by the publishers themselves
*Sharing agreements and data sharing projects form the barrier between the publisher side and the subscriber side. NOTHING passes over the divide unless the publishers consent. The publishers remain the data controllers and they still have full control.
*Subscribers, having registered an interest in accessing some data from some population (cohort) of citizens, subject to agreement, either receive&nbsp;the data automatically, or access the data on request. Data may be accessed at the level of a single person, or a population.
*Data is made available either via standard APIs and message formats or via pragmatic approaches such as CSV or simple JSON.
*The service supports modern strong authentication and authorisation standards to ensure security from end to end.
*All data is encrypted in transit AND in storage.
*&nbsp;
 
&nbsp;
 
&nbsp;
 
&nbsp;

Revision as of 08:45, 5 May 2020

Discovery can be conceptualised via 3 axes; The Discovery idea, the Discovery collaboration, and the actual set of systems that make up the Discovery data service.

The Discovery idea

The idea behind discovery is that if health related data is brought together at the level of the individual, and stored together at the level of a medium size residential population, that great benefits to health can accrue as a result of algorithms operating on individuals' data or on the population as a whole. 

This idea is specialised to the need to bring data from many different providers and provider systems, and convert or map that data to common information model, in order that all the data can be accessed by a single language, as if it was designed as a single system. The main difference between Discovery and an organisational management system is that it sources data from many disparate domains and makes sense of the data, whether the data was originally recorded in a standard or non standard form. In other words, all the useful data entered over many years, becomes accessible in a modern and coherent form. 

Examples of the way in which this data can be used include;

At an individual level:

<span style="font-family:Georgia,serif;">When an NHS 111 call handler receives a call from a citizen, analyse the primary and secondary care data in real time to detect whether they have frailty, thus resulting in directing the call to a clinician in order to avoid a dangerous trip to hospital</span>

At a population level:

<span style="font-family:Georgia,serif;">Using evidence from primary care, acute care, detect the small subset of people with Asthma who appear to have a profile suggesting that they are at risk of death or prolonged hospitalisation</span>

The Discovery collaborations

An idea by itself is not enough. A Discovery collaborative is any local or regional group of providers and commissioners who are willing to work together to establish and own and control their Discovery Data Service. A collaboration applies the following philosophies;

  1. All technologies, speciations and documentation is open source and available for all, including those not within the collaborative. This protects the collaborative against itself and provides the basis for other collaborative to contribute in a mutually beneficial way.
  2. All collaborative are of unlimited size. There is no limit to the members of a collaborative and collaborative do not attempt to preclude others from joining. Having said that collaborations by the nature of the population coverage of a particular data service, will tend to be local or regional. 
  3. The collaboration must apply the information governance policies common to Discovery. These include the retention of data controller status by providers, until data is transferred to subscribers as a result of consent by the providers, and the proper handling of identifiable and de-identified data

The Discovery data service

Having an idea and a collaboration is not enough. The Discovery data service is the set of systems and human operatives that deliver access to the data whenever needed.

The underlying paradigm of the service is known as pub/sub, or more precisely, 'Publish, store and queue, Subscribe'. In this paradigm the following summarises the process which is often referred to as a 'Pipeline'.

  • All health related events are 'published' to the data service by the provider, either from their system, or through middleware, in whatever form they are able, in as near real time as possible. 
  • The Data service receives the data, (having checked that the data processing agreements are in place), transforms to a common format, inks at the level of the citizen, using the trusted identifier and stores the data. At this point the data is controlled by the publisher i.e. the Discovery Data service operates on behalf of its publishers, who themselves are part of a collaborative.
  • A second stage of processing occurs to 'map' the original data content to the common information model, including the mapping to the Discovery ontology and the Discovery data model. The original data as published is maintained, the initial transform of the data is maintained and full provenance of the data is also maintained, uncluding any provenance of the data made available by the publishers themselves
  • Sharing agreements and data sharing projects form the barrier between the publisher side and the subscriber side. NOTHING passes over the divide unless the publishers consent. The publishers remain the data controllers and they still have full control.
  • Subscribers, having registered an interest in accessing some data from some population (cohort) of citizens, subject to agreement, either receive the data automatically, or access the data on request. Data may be accessed at the level of a single person, or a population.
  • Data is made available either via standard APIs and message formats or via pragmatic approaches such as CSV or simple JSON.
  • The service supports modern strong authentication and authorisation standards to ensure security from end to end.
  • All data is encrypted in transit AND in storage.
  •