DDS-UI dashboards

This section explains the internal dashboards used to monitor data flows in to and out of DDS. These dashboards have been built into an application called DDS-UI which, in addition to these dashboards, provides many back-end DDS management utilities, such as those for setting up new publisher feeds.

Included

The DDS-UI dashboards monitor the majority of DDS components that are responsible for the processing of published data, transforming raw data into the standardised FHIR-based intermediary model, and then transforming again for the individual subscriber databases. The dashboards cover almost every element of this data flow, although there are currently some exceptions to this, explained in the Exceptions section.

Not included

This section does not include:

• Slack alerts that report progress or to raise alerts about errors or failures; in many cases these alerts are simply to draw attention to an issue that the DDS-UI dashboards will display.

• Any aspect of hardware or infrastructure; these metrics are separately captured and monitored by the infrastructure team who have their own configured alerts to detect issues.

• Other applications that form part of the DDS, such as Data Sharing Manager or User Manager.

Overview

The DDS is made up of a large number of applications, running on many individual servers, interacting with many databases, all with different purposes. Due to this wide range of components, multiple dashboards have been implemented, to each monitor a specific part of the DDS.

This diagram shows a high-level component-based diagram of the DDS (only including components related to the processing of data).

Each box represents one or more instances of an application, API or process.

Some of the major DDS databases are also shown.

Component	Description
File-based Publishers	Represents all DDS publishers providing flat-file data to via an SFTP push or pull (or via the DDS Uploader application in the case of TPP
SFTP Reader	Set of DDS applications that receives flat-file data, performs some initial processing (such as decryption) and posts the data into the Messaging API for onwards transformation.
MLLP Publishers	Represents the two DDS publishers sending HL7v2 ADT real-time messages via MLLP.
HL7 Receiver	DDS application that receives HL7v2 ADT messages over MLLP, performs some processing, and then posts those into the Messaging API for onwards transformation.
HL7v2 Rest Publishers	Represents the DDS publishers sending HL7v2 ADT and ORU real-time messages via a REST API.
HL7v2 Receiver	DDS application that receives HL7v2 ADT and ORU messages via REST, and then posts them into the Messaging API for onwards transformation.
Messaging API	API that acts as the gateway into the DDS transformation pipeline for all published data.
Inbound Transformation	Represents the set of Queue Reader applications used to transform raw published data into the standardised FHIR format and then store that in the core FHIR record databases.
Outbound Transformation	Represents the set of Queue Reader applications used to transform the FHIR data into the CSV for DDS subscriber databases. For internally hosted subscriber databases, the CSV data is written directly to them. For remotely hosted databases, the CSV data is written to a staging database for remote sending.
UPRN API	API used in the outbound transformation process to look up various attributes relating to addresses.
Information Model API	API used in the outbound transformation process to look up concept identifiers for various clinical coding schemes.
Remote Sending	Represents the process of writing the staged CSV data for remote subscriber databases (output of the outbound transformation process) to the DDS SFTP server, and that data then being collected by the Remote Filer application and applied to the remote subscriber databases.
National Reference Updates	Represents various processes run regularly (typically monthly or quarterly) to update DDS reference databases with new content from national sources, such as TRUD or ONS.
Local Code Updates	Represents processes run regularly (daily) to push new local codes received from EMIS and TPP into the Information Model.
Scheduled Report Extracts	Represents various clinical extracts generated from internal subscriber databases.
Frailty API	API provided to Adastra (via Redwood Technologies) to allow discovery of if a patient is considered “frail”.
Know Diabetes API	Feed to the Know Diabetes API, sending patient record updates to the remotely hosted API.
Clinical Dashboards	Represents the processing of data from the internal subscriber databases into the format used by the DDS clinical dashboards.

The following sections describe each dashboard, and provide a similar diagram, highlighting the components that the specific dashboard monitors.