Google AI has access to huge haul of NHS patient data

https://www.newscientist.com/article/2086454-revealed-google-ai-has-access-to-huge-haul-of-nhs-patient-data/

by AlexandrB • 10 years ago

124 64 10 years ago

DanBC4• 10 years ago

People in the UK who want to complain about this could try the ICO who strongly regulate health information.

They could also look at any advertising material and report that to ASA (if it meets the criteria for being regulated).

That trust has a Caldicott Guardian who will be responsible - legally - for keeping patient data safe. I would have liked a quote from them, although I guess that quote would be something like "No patient identifiable data has been shared with DeepMind". That would be scary, I have no doubt that DeepMind would do a very good job of de-anonymising data, but I know that Google would have to be monumentally stupid to try that.

There are other data projects happening in the NHS - care.data (that dot isn't a typo!) is one that got a lot of attention. That allowed (after some fuss) people to opt-out. (It didn't allow people to specifically opt in to show their support, which is something I would have done.)

I'm a bit wary of Vice's reporting here. They don't seem to know what they're talking about (there's nothing about controls over patient data in the NHS, for example); they don't seem to have approached the Trust involved; they haven't done a good job of explaining what's going on.

There are some really bad failures or data protection in the NHS (especially around mass email! People using CC instead of BCC to a group of people using an HIV clinic, for example) and there are some historic abuses (selling data to insurance companies) that led to changes in the law.

So I don't know if this is terrible and deserving of anger, or okay and poorly reported, or a good thing with misleading reporting.

nezumi• 10 years ago

Terrible reporting. Improving health outcomes by AI analysis of patient data is a much bigger prize - morally and commercially - than anything which could be achieved through ad targeting. Google is far too smart to squander such an opportunity by abusing patients' trust.

buro92• 10 years ago

The big problem is the precedent it sets for data access.

What are the criteria for who gets access? What are the constraints of that access?

This story covers the latter being blown apart, the constraints were poorly defined and implemented and thus even if the criteria is well defined access to far more data was made possible.

I'm sure that few patients desire an end to research, or would argue that such access isn't a good thing... but what of the insurance industry? Should they have access? Would the NHS be able to define and enforce those constraints?

Perhaps that's an obvious no.

What then of an insurer partnering with a medical research company, from the viewpoint of "This costs insurance a lot of money, we'd like to fund a way to reduce that financial exposure".

The grey areas emerge immediately.

If we cannot control access to patient data, data that would be trivial to either strip anonymity or just to have in aggregate enough to still produce net-negatives (i.e. correlated by post code would reveal enough with little extra work)... and if we cannot define and enforce the constraints of access... then we really shouldn't be sharing what is highly sensitive and personal information that was originally only disclosed between a patient and a Doctor under the premise that what is shared is covered by the explicit and implicit confidentiality of that conversation.

It's always worth remembering:

Data was acquired under doctor patient confidentiality.

If we considered that data to have a licence, it is the most restrictive licence possible. One could consider what has happened here as a re-licensing without permission. Such an act could have a chilling effect on the relationship between the doctor and patient.

chris_va4• 10 years ago

You are making some implicit assumptions that they data access isn't highly controlled.

I have seen a few of these sorts of deals killed because of data access concerns, and/or computation requirements ("you can have access to anonymized data, but you have to run your code in a sandbox on our health servers").

And, this is why we have legislation.

superuser2• 10 years ago

Nope. To set a precedent it would have to precede. Giving de-identified medical records to researchers is a long-standing, well-established and regulated process. The only interesting thing here is that it's Google and not some PhD's university lab.

Here's HHS on what HIPAA has to say about this: [0]

[0] http://www.hhs.gov/hipaa/for-professionals/privacy/special-t...

halhodson3• 10 years ago

not sure if you saw my comment downstream? would encourage you to read original piece for a more nuanced presentation of the information - https://www.newscientist.com/article/2086454-revealed-google...

happy to address criticism

aub3bhat5• 10 years ago

Hi Hal I thought the article should have compared and contrasted with other government run large data sharing programs such as CMS qualified entity or AHRQ HCUP.

imglorp• 10 years ago

It so happens Google has the perfect means at its disposal for de-anonymizing large swaths of such data: trillions of user location records, calendar appointments, emails, and texts. It's not too hard to put all that together to match a specific encounter record, for example.

jtmcmc• 10 years ago

which would both violate their contract and also be illegal.

sp332• 10 years ago

Google is not a monolithic entity. You'd have to trust the individual researchers who have access to the data, and we don't even know who they are. And the NHS didn't ask their patients for permission before handing this data over, so whether you trust them or not is irrelevant, they get your data anyway.

jlarocco• 10 years ago

That seems incredibly naive. They utilize every other bit of information they collect, why wouldn't they utilize this data?

Google is a corporation. It can't have good intentions of its own. It's the thousands of employees who will potentially be working with and handling the data that you need to worry about.

dgacmu• 10 years ago

Google places the most comprehensive controls on PII of any place I've seen, including hospital environments subject to HIPAA. (Mostly because, unlike the hospitals, they have the technical clue how to enforce it properly. The hospitals... are still learning about computer security, and it's not their forte: http://arstechnica.com/security/2016/03/two-more-healthcare-... )

Getting access to private information in Google is hard - my experience as a researcher here is that there's a strong incentive to find an open-source or non-PII dataset before touching user data. I'll go through my year here without ever touching even the most innocuous PII data.

It's very unlikely to me that thousands of people will have access to this data. It's much more likely that a small handful will, and that they'll be supported by others with no access whatsoever. From the article, in fact:

"The agreement clearly states that Google cannot use the data in any other part of its business. The data itself will be stored in the UK by a third party contracted by Google, not in DeepMind’s offices. DeepMind is also obliged to delete its copy of the data when the agreement expires at the end of September 2017."

From an incentive perspective, the potential value-add of abusing the data is tiny compared to the potential costs and loss of user trust. Google's very aware of how important it is to maintain user trust -- http://www.techradar.com/us/news/internet/google-we-have-a-c...

Corporations don't have brains, but they have cultures, and Google's culture -- composed of those thousands of engineers -- is quite fanatical about protecting user privacy. It's one of the non-technical things that's impressed me most during my time here.

The risk with a company like Google is if the economic winds and culture changes, but that's a long-term process, and is also the reason for legally-binding contracts to do things like delete the data (see above).

tl;dr: Google has the technical means to protect the confidential data better than almost any other agency, including from its own employees. The most important question to ask is whether the NHS structured the data sharing in a way that provides for long-term protection, and (IANAL!) it sounds like it from the article.

Source: I'm a professor who deals with our IRB occasionally, have colleagues doing joint CS-medical research, pushed patients around a hospital in a younger life, and am on sabbatical for the year at Google.

ocdtrekkie2• 10 years ago

This is probably one of the reasons some may be opposed to nationalized healthcare... The government making a widespread decision to send your medical data somewhere without your permission.

halhodson3• 10 years ago

but we don't really have the consent-at-scale technology for anything else, do we? tbh it's just this specific NHS Trust that made the call in this instance.

Personally, I see potential in ResearchKit to solve the consent problem re medical data

lbhnact• 10 years ago

ResearchKit may be part of the solution for some things, although the auth and data management will be barriers for commercial providers for the near future.

I have a small team that is working with leaders in the space[1] to help many of the major EMR vendors support an open-standards based approach to medical record sharing.

We'll be seeking public feedback when some of the preliminary work is ready, but are very excited to get input from the community as we make progress.

[1] https://dbmi.hms.harvard.edu/news/more-power-patients

CaptSpify• 10 years ago

Although that is a valid concern, we can have nationalized health-care without the government sending data elsewhere.

Those two aren't inclusive of each other.

ocdtrekkie2• 10 years ago

True, but when the data is in private hands, the government sets regulations, like HIPAA here in the US, to regulate those companies' actions. Meanwhile, the government can and often does do what it wants with what it has.

DanBC4• 10 years ago

The government

In this case the government would be the Department of Health. They've had no involvement with this.

"The NHS" would be NHS England, and they didn't set this up, although they might sta.rt being involved to check the controls.

The local NHS, the Clinical Commissioning Group, didn't set this up.

So we're talking about one hospital trust

cromwellian2• 10 years ago

I think this situation nicely juxtaposes positive externality with private rights.

As an individual, you may wish to hoard all of your personal medical information. Doing so may provide marginal benefit or protection against some theoretical harms. However, much of medical science relies on large population studies. Is knee surgery worth it? Are breast exams? Do COX-2 inhibitors increase heart attacks? Enormous amounts of good could be done by having large datasets of entire patient histories available for analysis by all, assuming they could not be de-anonymized (if not, then the datasets have to be analyzed under contracts)

Whenever I visit the doctor and I am given a form asking if my data can be sent to some group to participate in a study, I always answer yes. Never once have I had any negative repercussions from doing so, and it is my hope that the data was used to publish scientific papers that added to net knowledge of humanity. However, I have to wonder if asking me for permission actually interferes with random sampling. Are people who give permission vs people who refuse statistically more likely to have other behaviors that may influence the results?

rtkwe• 10 years ago

assuming they could not be de-anonymized

That's where the problem is anonymizing data is really difficult and it takes only a few bits of info to associate that data with a name. It takes only 3 pieces of information to uniquely identify 87% of people, Zip code, birthday, and sex.

http://arstechnica.com/tech-policy/2009/09/your-secrets-live...

http://dataprivacylab.org/projects/identifiability/paper1.pd...

cromwellian2• 10 years ago

Right, which justifies the government signing closed source deals to analyze this data, with the threat of legal action for violations. They can stipulate that the results of such analysis should be made public.

Houshalter• 10 years ago

On the other hand anonymization destroys useful data (along with already poor record systems.) There was a Kaggle competition with medical data that had been so thoroughly damaged that 5 year old boys were listed as pregnant.

njs123452• 10 years ago

It's difficult - I have more of a fairness problem with this (such a deal would probably not be available to researchers or smaller companies) than a data privacy concern. As others point out downstream, I think it's not really in Google's interests to use the data maliciously and I have more faith in their opsec than that of any NHS trust.

cronjobber• 10 years ago

Nice try.

If this was about positive externalities, Google tech could have found its way into the NHS.

Instead, as always and every time, data found a way into Google.

Funny how that works.

Silhouette• 10 years ago

And that's your choice. Unfortunately, someone else committed suicide because they didn't ask for the help they needed for their mental health issues because of justified concern that their condition would leak and it would lead to discrimination.

There are many reasons privacy is important. Among the most important is the ability to have completely open discussions with clinical professionals about any medical issues you have, without having to worry that anyone has any ulterior motives or that anything you say could come back to haunt you later.

It is difficult enough for some people to talk about some conditions even with their own doctor in the privacy of the doctor's office. Probably some valuable benefits would result if we dropped privacy altogether and allowed widespread analysis of clinical data. However, it is an absolute certainty that serious damage is done both when patient confidentiality is perceived to be threatened (through patients not raising issues in the first place) and when patient confidentiality is in fact compromised (through discrimination of various kinds, much of which may be based on misconceptions).

AlexandrB2• 10 years ago

Whenever I visit the doctor and I am given a form asking if my data can be sent to some group to participate in a study, I always answer yes. Never once have I had any negative repercussions from doing so, and it is my hope that the data was used to publish scientific papers that added to net knowledge of humanity.

I've grown cynical. My first thought on seeing this kind of form is that a third party will be making money from the information I provide somehow - whether it's through patented drugs or treatments or through pay journals like those owned by Elsevier. Neither I, nor anyone I know, will get to benefit from from the information I provide without spending money (often exorbitant amounts of it). This is the flip side of privatizing everything. Why should I lift a finger to help for-profit entities when they will not do the same to help me?

pavel_lishin• 10 years ago

Why should I lift a finger to help for-profit entities when they will not do the same to help me?

But if releasing your data leads to research that can save lives, isn't that worth doing even if someone gets rich in the process? I'd much rather there exist a million-dollar treatment that can save my life at the cost of living in debt for the next twenty years, rather than die in six months because the research was never done.

(Yes, this is a false dichotomy - there should be a better way of administering treatments and doing medical research.)

savanaly• 10 years ago

It's going to sound like I'm telling you how to live your life and how to feel and I really don't mean it that way but I don't know any other way to express my viewpoint so I'll just power through it:

The idea that someone, somewhere is making a profit on something you provided should make you happy, not sad. To make a profit, they had to be paid, and to be paid they normally have to have provided something that someone wanted enough to part with cash. That cash changed hands is not the main point of the transaction (it's a net zero for society, someone gained cash and someone lost it), the main point is rather that something of value was created.

You threw out the example of pay journals like Elsevier. It is true that for cases like that, there may be a market inefficiency that means not much value is created. But I think those are edge cases and relatively rare. Even if our worst fears about Elsevier and other rent seeking journals are true, it would still be the case that you're helping rather than hurting by providing anonymized personal data.

Bluestrike22• 10 years ago

You--as an individual--receive zero benefits from your medical data when it's kept private. You can't consume it, you can't use it, and you can't trade it. It has no inherent value on its own. Were researchers forced to pay you for access to your de-identified data, it'd be a number so infinitesimal as to be all but meaningless.

You aren't giving up something of value to you. There's zero opportunity cost involved. The value is added after your data is collected, collated, analyzed, and used by researchers. Individually, it's meaningless. Together, it's priceless.

Will someone eventually profit? Sure, but they'll do so because society benefits from the work they've done. And eventually, the cost will decrease. Would you rather have drug treatments that exist in the first place even if someone profits from them or nothing? Those are pretty much your options. For all of its flaws, the current research model at least works. The only other option, besides doing nothing and letting people die from diseases we could eventually treat, is some sort of public financing and that comes with a host of problems. Would anyone really be so foolish as to want to subject medical research to the congressional budget process, even if there's an abstraction layer between congress and the researchers? That means tradeoffs, and lots of them: for instance, Drug option A would be pursued, and option B ignored because we're already funding A even if B might be more promising later on.

We already see it with federal applied research when critical work is jeopardized for the sake of political grandstanding. Every so often, someone will trot out a cherry-picked grant they don't like and wave it around like a red flag in front of a bull. Drug companies might be bad enough, but the politicians would be worse.

aub3bhat5• 10 years ago

You are correct there are already laws and systems in place. E.g. here is an interesting paper in New England Journal of Medicine using California data. Note: Not published by me but someone who I work with.

http://www.nejm.org/doi/full/10.1056/NEJMoa1311485

adamsea• 10 years ago

It's an interesting question for sure. There are issues of consent which, especially for poor or marginalized people, should be addressed. Like Henrietta Lacks...

I think in this case and others like it the family, at minimum, has a right to know. "...it was not until 1973, when a scientist called to ask for blood samples to study the genes her children had inherited from her, that Ms. Lacks’s family learned that their mother’s cells were, in effect, scattered across the planet. Some members of the family tried to find more information. Some wanted a portion of the profits that companies were earning from research on HeLa cells. They were largely ignored for years."

http://www.nytimes.com/2013/08/08/science/after-decades-of-r...

ggggtez• 10 years ago

As an expert in machine learning, I don't see how someone would expect this to work. Actually to predict a disease, you need true positives and true negatives. If you only had access to the true positive data, it would be a lot harder to predict accurately.

askldfhjkasfhd• 10 years ago

Can you explain what you mean a little more? It seems like by having complete medical records, there would be a great deal you could infer statistically.

Do you mean that it's not sufficient to have a medical record that doesn't indicate knee issues? That you would need a medical record that confirms there are no knee issues?

throwanem4• 10 years ago

If you want a reliable result, yes.

Consider: I may have a medical record that doesn't indicate knee issues because I have correctly been diagnosed with no knee issues. Or I may have such a record because I have been incorrectly so diagnosed, despite the presence of knee issues. Or I may have such a record because I haven't been to a doctor about my knee issues in the years since they've developed, and so no opportunity has arisen for my knee issues to be documented. Simply from a medical record reflecting no evidence of knee issues, you cannot know which of these, if any, is true.

As I mentioned in a reply to your neighbor comment, clinical studies are designed to exclude such confounders as these - and that's a considerable part of why such studies are so expensive to design and carry out. It's very difficult to see how such exclusion could be achieved with data of the quality which seems to be involved here.

chris_va4• 10 years ago

Seems pretty simple, really.

1000 patients come in with symptoms that look like cancer at year 0.

100 actually get diagnosed with cancer at some point between year 0 and year 5.

Presumably, the remaining 900 didn't have cancer at year 0.

throwanem4• 10 years ago

"Seems". You can't know what fraction of your remaining 900 had cancer but weren't diagnosed with it for any of a wide variety of reasons, from death through misadventure, through false negative, to simple loss to followup. Clinical studies are designed to exclude such confounding outcomes. It's very difficult to see how any study of this data could be designed to do likewise.

VeejayRampay• 10 years ago

The National Health Service (NHS) is the publicly funded healthcare system for England (source: Wikipedia)

Putting that here because I was confused about what NHS was in the first place (I'm French).

njs123452• 10 years ago

The world's 5th largest employer (just after McDonald's)..

arethuza• 10 years ago

There isn't really a single NHS - there are four National Health Services for the four parts of the UK (England, Scotland, Wales and Northern Ireland):

https://en.wikipedia.org/wiki/National_Health_Service

grillvogel• 10 years ago

best part of this is when we reach the skynet robot uprising they'll know exactly how to kill us the fastest

aub3bhat5• 10 years ago

Why are people astonished?

For more than a decade researchers have been able to get access to large deidentified datasets. As a PhD student I have access to data on 150 Million visits by 45 Million patients. In some ways the data and access I have is superior to that of Google & NHS (since UK is a tiny country in comparison to USA), and I am just a PhD student. Though I have been working on it for last 5 years.

Recently there is a new qualified entity program run by CMS which provides access to Medicare data.

You can read more about my research and see the demo of the system in my past submissions and at

http://www.computationalhealthcare.com

Also for the actual govenment program http://www.ahrq.gov/research/data/index.html

Actually when it comes to Medical information its a much much more complex problem legally. This paper gives a good overview on issue of "Patient ownership of data" http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1857986

I am by no means minimizing the concenrs that people have. But I think the article paints Google in negative light while ignoring current standard practices. I wish the discussion would be more rooted in the facts about how such data sharing systems work currently.

fweespee_ch5• 10 years ago

Why are people astonished?

Because politicians keep making promises in regards to privacy that they don't keep in regards to "confidential" information.

Similarly, most of the medical value from such information [e.g. Frequency of X within a given population fitting certain characteristics] would likely deanonymize people.

DanBC4• 10 years ago

No politician was involved in this decision.

superkamiguru• 10 years ago

There is no equivalent of HIPPA in the UK?

aub3bhat5• 10 years ago

HIPPA stands for Health Insurance Portability and Accountability Act. The "Privacy" component of HIPPA is very limited in scope. Even with HIPPA the data being shared would be consistent with definition of "Limited" dataset.[0] At least in United States there is significant precedent for sharing such data, including even for marketing purposes as decided by Supreme Court case (IMS Health vs Sorrell).

[0]http://www.hopkinsmedicine.org/institutional_review_board/hi...

Google AI has access to huge haul of NHS patient data

Related Stories