TPP known issues

TPP Gateway

Due to an ongoing issue with the TPP Gateway Due and the need to rebulk several configurations, the Discovery Data Service is not receiving daily deltas (or is receiving delayed deltas) for the following areas:

• Hammersmith and Fulham
• Ealing South
• Hounslow Great West
• Central London

North West London is investigating this issue and doing everything they can to resolve this issue as quickly as possible.

Updated: 19 October 2020

TPP delta - missing data

A corrupted TPP delta for "TPP_YDDH3_07Y_FHH", which covers HOUNSLOW FELTHAM, was received from the TPP Gateway, resulting in missing data for the following 21 practices:

E85625, E85060, E85126, E85716, E85015, E85059, E85040, E85693, E85058, E85007, E85658, E85746, E85692, E85004, E85113, E85605, E85683, E85001, E85600, E85030, and E85735

The TPP SRCode file has supplied us with a significant amount of the lost data already, but the DDS will still be missing data from the specific date that isn’t in the SRCode file (e.g. new patient registrations, vaccinations, medications). A full rebulk has been requested for each of these practices in order to obtain the missing data and will be processed as soon as it is made available to Discovery. We will provide further updates in due course.

Updated: 6 October 2020

SRCode re-bulks

Under GPIT Futures, TPP offer an IM1 Bulk Extract to extract data from SystmOne. TPP call this their ‘Strategic Reporting’ extract, or just ‘SR’ for short. This is their only mechanism for extracting data on an organisation-level (i.e. all patients at an organisation).

TPP offer other mechanisms to extract data, but these are used on a patient-by-patient basis. Any third-party getting data from SystmOne for entire organisations is using their Strategic Reporting extract (although they may not be using it through IM1 Pairing integration).

The TPP Strategic Reporting extract works by generating the extracts in the SystmOne data centre, then downloading the extract files to a nominated SystmOne Gateway client, which are then available for the extracting organisation to utilise.

For DDS, the specific flow is:

Generate extract (SystmOne data centre) > Download (to SystmOne Gateway) > Upload (to DDS) > Process into DDS

The SystmOne Strategic Reporting extract contains many different files, each representing a different type of clinical concept; for example, a file for repeat medications, or a file for patient addresses. When a SystmOne organisation first starts publishing data to DDS the first Strategic Reporting extract will contain all existing data for all patient records at the organisation - a ‘bulk’ extract. The following day, and each day after that, only the changes from the previous extract are included in the files, making them considerably smaller - ‘delta’ extracts. Extracting, downloading and processing a bulk extract, which may contain more than 10GB of data, is obviously considerably slower and more resource intensive than processing a daily delta, which is typically less than 20MB.

The SRCode file is the largest file in the Strategic Reporting extract, containing all CTV3/Snomed coded items, including numeric values, codes entered during consultations, data entered into new and old-style templates. SRCode typically contains as much data as the other files put together. So if an extracted SystmOne organisation's bulk extract was 10GB, their SRCode file will be 5GB of that.

TPP have a semi-regular issue where they are unable to generate the expected daily deltas of the SRCode files for their Strategic Reporting organisations. Rather than a daily delta of around 20MB of data, the extract delivered to third parties contains a full bulk of the SRCode file, plus the expected daily deltas of the other files. So rather than a 20MB daily extract, a 5GB extract is delivered. Because the SRCode file makes up a huge proportion of the full data set, when this issue occurs, it impacts on the time taken to generate the extracts and the time taken to download them to the SystmOne Gateway (the time taken to upload and process them in to DDS isn't hugely affected). This issue seemingly affects many (if not all) users of the TPP Strategic Reporting system, not just DDS.

We have been informed that TPP did an update to SystmOne (WE 19/09/2020) and as a consequence of this the issue with SRCode re-bulks has occurred again since then.

This issue has happened ten times so far in 2020, on and around the below dates:

• 2020-04-02
• 2020-05-03
• 2020-05-07
• 2020-05-31
• 2020-06-07
• 2020-06-26
• 2020-07-06
• 2020-08-23
• 2020-08-28
• 2020-09-17

Updated: 21 September 2020