What is Discovery: Difference between revisions

From Discovery Data Service
Jump to navigation Jump to search
No edit summary
(9 intermediate revisions by 2 users not shown)
Line 4: Line 4:
= The Discovery idea =
= The Discovery idea =


Discovery is based on the hypothesis that; If health related data is brought together at the level of the individual, and stored together at the level of a medium size residential population, and made available via a common information model, and used for individual and population based decisioon support, great benefits to health can accrue.
Discovery is based on the hypothesis that; If health related data is brought together at the level of the individual, and stored together at the level of a medium size residential population, and made available via a common information model, and used for individual and population based decision support, great benefits to health can accrue.


Two small examples of the way in which this data can be used include;
Two small examples of the way in which this data can be used include;
Line 10: Line 10:
At an individual level: <span style="color:#8e44ad;">When an NHS 111 call handler receives a call from a citizen,&nbsp; if a&nbsp;&nbsp;system could analyse&nbsp;the primary and secondary care data in real time, and&nbsp;detect whether they have frailty, alert the call handler that a clinician should be consulted, a dangerous trip to hospital may be avoided.</span>
At an individual level: <span style="color:#8e44ad;">When an NHS 111 call handler receives a call from a citizen,&nbsp; if a&nbsp;&nbsp;system could analyse&nbsp;the primary and secondary care data in real time, and&nbsp;detect whether they have frailty, alert the call handler that a clinician should be consulted, a dangerous trip to hospital may be avoided.</span>


At a population level: <span style="color:#8e44ad;">Using data from primary care and acute care combined, if a system coult determine&nbsp;the small number of people with Asthma who appear to have a profile suggesting that they are at risk of death or prolonged hospitalisation, proactive intervention would save lives.</span>
At a population level: <span style="color:#8e44ad;">Using data from primary care and acute care combined, if a system could determine&nbsp;the small number of people with Asthma who appear to have a profile suggesting that they are at risk of death or prolonged hospitalisation, proactive intervention would save lives.</span>


= Discovery or Discovery like collaborations =
= Discovery or Discovery like collaborations =


An idea by itself is not enough. A Discovery collaborative is any local or regional group of providers and commissioners who are willing to work together to establish and control their local Data Service, and apply the following philosophy that;
An idea by itself is not enough. A Discovery collaborative is any local or regional group of providers and commissioners who are willing to work together to establish and control their local Data Service, and apply the following philosophies&nbsp;that;


*All technologies, speciations and documentation must be&nbsp;open source and available for all, including those not within the collaborative. Intellectual property must be shared for the greater good.  
*All technologies, specifications and documentation must be&nbsp;open source and available for all, including those not within the collaborative. Intellectual property must be shared for the greater good.
*Providers must be willing to share, subject to good governance, security and protection of privacy.  
*Providers must be willing to share, subject to good governance, security and protection of privacy.  


= Discovery data services =
= Discovery data services =
[[File:DDS overview.png|right|frameless|786x786px]]


Having an idea and a collaboration is not enough. The Discovery data service is the set of systems and human operatives that deliver access to the data whenever needed.
Having an idea and a collaboration is not enough. The Discovery data service is the set of systems and human operatives that deliver access to the data whenever needed.


The underlying paradigm of the Discovery service is known as pub/sub, or more precisely, 'Publish, store and queue, Subscribe'.&nbsp;In this paradigm the following summarises the process which is often referred to as a 'Pipeline'.
The underlying paradigm of the data&nbsp;service is known as pub/sub, or more precisely, 'Publish, Store,&nbsp;Queue and Subscribe'. The following summarises the process of data through the system from publisher to subscriber.


*All health related events are 'published' to the data service by the provider, either from their system, or through middleware, in whatever form they are able, in as near real time as possible.&nbsp;  
*All health related events are 'published' to the data service by the provider, either from their system, or through middleware, in whatever form they are able, in as near real time as possible.&nbsp;  
*The Data service receives&nbsp;the data, (having checked that the data processing agreements are in place), transforms to a common format, inks at the level of the citizen, using the trusted identifier and stores the data. At this point the data is controlled by the publisher i.e. the Discovery Data service operates on behalf of its publishers, who themselves are part of a collaborative.  
*The Data service receives&nbsp;the data, (having checked that the data processing agreements are in place), transforms to a common format, links at the level of the citizen, using the trusted identifier and stores the data. At this point the data is controlled by the publisher i.e.. the Discovery Data service operates on behalf of its publishers, who themselves are part of a collaborative.  
*A second stage of processing occurs to 'map' the original data content to the common information model, including the mapping to an inclusive&nbsp;ontology and common data model. The original data as published is retained, the initial transform of the data is audited,&nbsp;&nbsp;and full provenance of the data is also maintained, uncluding any provenance of the data made available by the publishers themselves.  
*A second stage of processing occurs to 'map' the original data content to the common information model, including the mapping to an inclusive&nbsp;ontology and common data model. The original data as published is retained, the initial transform of the data is audited,&nbsp;&nbsp;and full provenance of the data is also maintained, including any provenance of the data made available by the publishers themselves.  
*Sharing agreements and data sharing projects form the barrier and enabler between the publisher side and the subscriber side. NOTHING passes over the divide unless the publishers consent. The publishers remain the data controllers and they still have full control.  
*Sharing agreements and data sharing projects form the barrier and enabler between the publisher side and the subscriber side. NOTHING passes over the divide unless the publishers consent. The publishers remain the data controllers and they still have full control until the data passes over.  
*Subscribers, having registered an interest in accessing some data from some population (cohort) of citizens, subject to agreement, either receive&nbsp;the data automatically, or access the data on request. Data may be accessed at the level of a single person, or a population.  
*Subscribers, having registered an interest in accessing some data from some population (cohort) of citizens, subject to agreement, either receive&nbsp;the data automatically, or access the data on request. Data may be accessed at the level of a single person, or a population.  
*Data is made available either via standard APIs and message formats or via pragmatic approaches such as CSV or simple JSON.  
*Data is made available either via standard APIs and message formats or via pragmatic approaches such as CSV or simple JSON.  

Revision as of 14:03, 16 November 2020

Discovery can be conceptualised via 3 axes; The Discovery idea, the Discovery collaboration, and the actual set of systems that make up the Discovery data service.

The Discovery idea

Discovery is based on the hypothesis that; If health related data is brought together at the level of the individual, and stored together at the level of a medium size residential population, and made available via a common information model, and used for individual and population based decision support, great benefits to health can accrue.

Two small examples of the way in which this data can be used include;

At an individual level: When an NHS 111 call handler receives a call from a citizen,  if a  system could analyse the primary and secondary care data in real time, and detect whether they have frailty, alert the call handler that a clinician should be consulted, a dangerous trip to hospital may be avoided.

At a population level: Using data from primary care and acute care combined, if a system could determine the small number of people with Asthma who appear to have a profile suggesting that they are at risk of death or prolonged hospitalisation, proactive intervention would save lives.

Discovery or Discovery like collaborations

An idea by itself is not enough. A Discovery collaborative is any local or regional group of providers and commissioners who are willing to work together to establish and control their local Data Service, and apply the following philosophies that;

  • All technologies, specifications and documentation must be open source and available for all, including those not within the collaborative. Intellectual property must be shared for the greater good.
  • Providers must be willing to share, subject to good governance, security and protection of privacy.

Discovery data services

DDS overview.png

Having an idea and a collaboration is not enough. The Discovery data service is the set of systems and human operatives that deliver access to the data whenever needed.

The underlying paradigm of the data service is known as pub/sub, or more precisely, 'Publish, Store, Queue and Subscribe'. The following summarises the process of data through the system from publisher to subscriber.

  • All health related events are 'published' to the data service by the provider, either from their system, or through middleware, in whatever form they are able, in as near real time as possible. 
  • The Data service receives the data, (having checked that the data processing agreements are in place), transforms to a common format, links at the level of the citizen, using the trusted identifier and stores the data. At this point the data is controlled by the publisher i.e.. the Discovery Data service operates on behalf of its publishers, who themselves are part of a collaborative.
  • A second stage of processing occurs to 'map' the original data content to the common information model, including the mapping to an inclusive ontology and common data model. The original data as published is retained, the initial transform of the data is audited,  and full provenance of the data is also maintained, including any provenance of the data made available by the publishers themselves.
  • Sharing agreements and data sharing projects form the barrier and enabler between the publisher side and the subscriber side. NOTHING passes over the divide unless the publishers consent. The publishers remain the data controllers and they still have full control until the data passes over.
  • Subscribers, having registered an interest in accessing some data from some population (cohort) of citizens, subject to agreement, either receive the data automatically, or access the data on request. Data may be accessed at the level of a single person, or a population.
  • Data is made available either via standard APIs and message formats or via pragmatic approaches such as CSV or simple JSON.
  • The service supports modern strong authentication and authorisation standards to ensure security from end to end.
  • All data is encrypted in transit AND in storage