INSUBCONTINENT EXCLUSIVE:
A third party audit of a controversial patient data-sharing arrangement between a London NHS Trust and Google DeepMind appears to have
skirted over the core issues that generated the controversy in the first place.
The audit (full report here) — conducted by law
firmLinklaters — of the Royal FreeNHSFoundation Trust acute kidney injury detection app system, Streams, which was co-developed with
Google-DeepMind (using an existing NHS algorithm for early detection of the condition), does not examine the problematic 2015
information-sharing agreement inked between the pair which allowed data to start flowing.
&This Report contains an assessment of the data
protection and confidentiality issues associated with the data protection arrangements between the Royal Free and DeepMind
It is limited to the current use of Streams, and any further development, functional testing or clinical testing, that is either planned or
It is not a historical review,& writes Linklaters, adding that: &It includes consideration as to whether the transparency, fair processing,
proportionality and information sharing concerns outlined in the Undertakings are being met.&
Yet it was the original 2015 contract that
triggered the controversy, after it was obtained and published by New Scientist, with the wide-ranging documentraising questions over the
broad scope of the data transfer; the legal bases for patients information to be shared; and leading to questions over whether regulatory
processes intended to safeguard patients and patient data had been sidelinedby the two main parties involved in the project.
InNovember
2016the pair scrapped and replaced the initial five-year contract with a different one — which put in place additional information
governance steps.
They also went on to roll out the Streams app for use on patients in multiple NHS hospitals— despite the UK data
protection regulator, the ICO, having instigated an investigation into the original data-sharing arrangement.
And just over a year agothe
ICO concluded that the Royal FreeNHSFoundation Trust had failed to comply with Data Protection Law in its dealings with Google DeepMind.
The
audit of the Streams project was a requirement of the ICO.
Though, notably, the regulator has not endorsed Linklaters report
On the contrary, it warns that it seeking legal advice and could take further action.
In a statementon its website, the ICO deputy
commissioner for policy, Steve Wood, writes: &We cannot endorse a report from a third party audit but we have provided feedback to the Royal
We also reserve our position in relation to their position on medical confidentiality and the equitable duty of confidence
We are seeking legal advice on this issue and may require further action.&
In a section of the report listing exclusions, Linklaters
confirms the audit does not consider: &The data protection and confidentiality issues associated with the processing of personal data about
the clinicians at the Royal Free using the Streams App.&
So essentially the core controversy, related to the legal basis for the Royal Free
to pass personally identifiable information on 1.6M patients to DeepMind when the app was being developed, and without people knowledge or
consent, is going unaddressed here.
And Wood statement pointedly reiterates that the ICO investigation &found a number of shortcomings in
the way patient records were shared for this trial&.
&[P]art of the undertaking committed Royal Free to commission a third party audit
They have now done this and shared the results with the ICO
What important now is that they use the findings to address the compliance issues addressed in the audit swiftly and robustly
We&ll be continuing to liaise with them in the coming months to ensure this is happening,& he adds.
&It important that other NHS Trusts
considering using similar new technologies pay regard to therecommendations we gave to Royal Free, and ensure data protection risks are
fully addressed using a Data Protection Impact Assessment before deployment.&
While the report is something of a frustration, given the
glaring historical omissions, it does raise some points of interest — including suggesting that the Royal Free should probably scrap a
Memorandum of Understanding it also inked with DeepMind, in which the pair set out their ambition to apply AI to NHS data.
This is
recommended because the pair have apparently abandoned their AI research plans.
On this Linklaters writes: &DeepMind has informed us that
they have abandoned their potential research project into the use of AI to develop better algorithms, and their processing is limited to
execution of the NHS AKI algorithm… In addition, the majority of the provisions in the Memorandum of Understanding are non-binding
The limited provisions that are binding are superseded by the Services Agreement and the Information Processing Agreement discussed above,
hence we think the Memorandum of Understanding has very limited relevance to Streams
We recommend that the Royal Free considers if the Memorandum of Understanding continues to be relevant to its relationship with DeepMind
and, if it is not relevant, terminates that agreement.&
In another section, discussing the NHS algorithm that underpins the Streams app, the
law firm also points out that DeepMind role in the project is little more than helping provide a glorified app wrapper (on the app design
front the project also utilized UK app studio, ustwo, so DeepMind can&t claim app design credit either).
&Without intending any disrespect
to DeepMind, we do not think the concepts underpinning Streams are particularly ground-breaking
It does not, by any measure, involve artificial intelligence or machine learning or other advanced technology
The benefits of the Streams App instead come from a very well-designed and user-friendly interface, backed up by solid infrastructure and
data management that provides AKI alerts and contextual clinical information in a reliable, timely and secure manner,& Linklaters
writes.
What DeepMind did bring to the project, and to itsother NHS collaborations, is money and resources — providing its development
resources free for the NHS at the point of use, and stating (when asked about its business model) that it would determine how much to charge
the NHS for these app ‘innovations& later.
Yet the commercial services the tech giantis providing to what are public sector organizations
do not appear to have been put out to open tender.
Also notably excluded in the Linklaters& audit: Any scrutiny of the project vis-a-vis
competition law, public procurement law compliance with procurement rules, and any concerns relating to possible anticompetitive
behavior.
The report does highlight one potentially problematic data retention issue for the current deployment of Streams, saying there is
¤tly no retention period for patient information on Streams& — meaning there is no process for deleting a patient medical history
once it reaches a certain age.
&This means the information on Streams currently dates back eight years,& it notes,suggesting the Royal Free
should probably set an upper age limit on the age of information contained in the system.
While Linklaters largely glosses over the
chequered origins of the Streams project, the law firm does make a point of agreeing with the ICO that the original privacy impact
assessment for the project &should have been completed in a more timely manner&.
It also describes it as &relatively thin given the scale of
the project&.
Giving its response to the audit, health data privacy advocacy group MedConfidential — an early critic of the DeepMind
data-sharing arrangement — is roundly unimpressed, writing: &The biggest question raised by the Information Commissioner and the National
Data Guardian appears to be missing — instead, the report excludes a &historical review of issues arising prior to the date of our
appointment&.
&The report claims the ‘vital interests& (i.e
remaining alive) of patients is justification to protect against an &event [that] might only occur in the future or not occur at all&… The
only ‘vital interest& protected here is Google&s, and its desire to hoard medical records it was told were unlawfully collected
The vital interests of ahypotheticalpatient are not vital interests of an actual data subject (and the GDPR tests are demonstrably
unmet).
&The ICO and NDG asked the Royal Free to justify the collection of 1.6 million patient records, and this legal opinion explicitly
provides no answer to that question.&
