Before other deCODEme customers get too irate about errors in data for which they have paid almost $1000, the bug affects only a tiny portion of the results presented. Most importantly, the disease-risk summaries provided by deCODEme seem to be based on the correct genetic information.

“seem to be” is the operative terminology, indeed. As is typical in security / quality-control settings, the question here is, if the software can make such a large mistake, what about all the smaller mistakes it’s making that aren’t so obviously detectable?

Seems to me that before we start trusting these genomic tests for clinical purposes, we’ll want to make sure our genomes are read multiple times, ideally using different technologies. 99.99% accuracy sounds great until you realize you’re dealing with millions of data points, each one of which could be significant.

(And I’m not even touching on whether genomic data is sufficiently predictive, given current knowledge, to be clinically relevant, which as Zak Kohane points out in the article, isn’t a given.)

This weekend, though, I received an unpleasant surprise. I got a spam email sent to ben-healthengage. HealthEngage is a health web site I tried out a few days months ago to explore how some companies are working on device connectivity. I’m 99% certain I haven’t used that email address anywhere else (why would I?) So, is HealthEngage leaking email addresses in some way, either because they’re selling them or because they’re not protecting them very well and spam crawlers are picking them up somewhere?

Either way, it’s a little bit disconcerting: this is a health-data web site, and its members surely worry about their privacy.

Harvard’s own Donald Berwick explains to the New York Times that it’s time to empower patients (see the original Health Affairs article):

Some examples of this new model of care? Shared decision-making would be mandatory in all areas of care, with patient preference occasionally putting evidence-based care “in the back seat.” Patients and families would participate in the design of health care processes and services and would be a part of daily rounds. Medical records would belong not to clinicians but to patients, who would no longer have to get permission to look at them or call the doctor for lab results.

Read the full interview, it’s brief and highly worthwhile. I completely agree with Dr. Berwick.

Meanwhile, in New Jersey, a proposed state law wants to fine anyone who sells software that has anything to do with health data if it hasn’t been certified by CCHIT. CCHIT is a single entity that would get to certify all health software. CCHIT is also pushing to be the lone certification authority for all stimulus-funded work. So, as if health IT wasn’t already painful enough to deal with, now we’re going to move towards a certification monopoly? Say goodbye to:

• iPhone apps that let you track your kids’s vaccines for $4, and really most small iPhone medical apps in general, as they clearly won’t be able to afford the certification fee,
• storing your health data online at Google Health, Microsoft HealthVault, or Indivo/Dossia.
• open-source medical software. As hard as Fred Totter is working to get CCHIT to see the free/open-source point of view, there’s simply no incentive for a certification authority to spend time on a distributed community where it’s unclear who will pay the certification fee.

No matter how well-intentioned and knowledgeable the folks at CCHIT are, creating a certification monopoly shows a lack of understanding of how these things really work. Once the monopoly is in place, where is the motivation for CCHIT to be efficient, responsive to new healthcare models, adaptable to new software methodologies? In addition, what is the certification really worth when the vendors are paying for it anyways? We’ve seen this conflict before in the election world: the “Independent Testing Authorities” are paid by vendors to certify voting machines. At least there, there’s mild competition. How much do you think that certification really means in terms of voting security/privacy/safety? Here’s a hint: all the voting machines that were found to be laughingly insecure by the Berkeley and Princeton teams had been certified by Independent Testing Authorities.

Now, the question on everyone’s mind should be “ok, but how do we ensure that there’s some kind of oversight for health software?” A good and very important question, which I’ll try to answer in a future blog post. But for now, let’s be clear: we need more patient involvement, not less. We need new software that will enable this patient involvement, not old software with half-baked web interfaces tacked on as an afterthought. The last thing we need is a government-mandated certification monopoly. Even if they asked Dr. Berwick to run it, it would be a bad idea, because the incentives are all wrong. Innovation/disruption, which we so desperately need, comes from the new, small players, the ones that simply won’t be viable if they have to pay an upfront certification tax, both in dollars and process.

What we easily forget – because our bodies and brains do this for us automatically – is the need to establish a sufficiently strong, but dampened, feedback loop. If you overcompensate for the weight, then reverse the overcompensation when you realize you overshot, and so on and so forth, the pendulum keeps swinging further and further away from the stable equilibrium point. If you underreact, then you drop the ball. The same goes for information system design: proper feedback loops are crucial for quality control.

A recent Boston Globe article describes how the BIDMC shipped claims data (the codes doctors send to your insurance company for billing purposes) to Google Health, and how some patients discovered that this data was really, really odd. The reaction by the BIDMC has been nothing short of exemplary in working to rectify the situation, except maybe for the fact that billing codes were shipped in the first place, something a number of folks in the industry already knew to be problematic.

But I think this whole discussion misses the important point that e-patient Dave makes:

Then imagine that one day you were allowed to see the records, and you found out there were a whole lot of errors, and the people carefully guarding your data were not as on top of things as everyone thought.

The point here is that putting the patient into the loop – the data feedback loop – is not just a convenience for the patient, it’s a very real way to debug a health record. So instead of knocking Google Health or Microsoft HealthVault or Dossia or Indivo, the question to ask here is: what’s the quality of hospital medical records if patients are left out of the feedback loop? Sure the Google Health errors would likely have been far less egregious if the BIDMC hadn’t shipped billing codes, but that’s a distraction. The core issue remains: what is the feedback loop that ensures a patient’s health record is reliable?.

The Boston Globe, for all of its good reporting, misses the point and hints at overreaction by removing the feedback:

In the meantime, said Tang, who was recently appointed to a new committee advising the Obama administration on health technology, the risks to patients need to be studied further. “Probably for some patients it’s a net benefit, and for others it’s a risk,” he said.

The solution is not a continued paternalistic attitude that claims some patients are ill equipped to deal with this data. As the healthcare system gets more complicated, more specialized, and more fragmented, the most promising and feasible quality-control feedback loop is the patient. Sure, for a little while, the data that filters down to PCHRs will be problematic. But at least it will be checkable by the patient, and that’s quality control you simply won’t get from a hospital medical record divorced from patient oversight. Over time, personally controlled health records are likely going to become a far more reliable source of data than the hospital medical records themselves.

Finally, a feedback loop needs closure. One important technical challenge for personally controlled health records will be ensuring that patients can make corrections to their data in a way that filters back to the hospital’s record. And not just by being featured in the Boston Globe.

But now I don’t have to do an introductory post, because Steven Pinker did it already in the NY Times, much more beautifully and informatively than I could ever have done. If you’re at all interested in the topic, his article is a must-read.

My favorite is his prediction near the end, with which I completely agree:

People who have grown up with the democratization of information will not tolerate paternalistic regulations that keep them from their own genomes, and early adopters will explore how this new information can best be used to manage our health. There are risks of misunderstandings, but there are also risks in much of the flimflam we tolerate in alternative medicine, and in the hunches and folklore that many doctors prefer to evidence-based medicine. And besides, personal genomics is just too much fun.

Peel and EPIC ask:

There are, however, privacy concerns surrounding this new tool.

[...]

In the aggregate, the data reveals useful trends and should be available for appropriate uses. But if disclosed and linked to a particular user, there could be adverse consequences for education, employment, insurance, and even travel. The disclosure of such information could also have a chilling effect on Internet users who may be reluctant to seek out important medical information online if they are concerned that their search histories will be revealed to others… If Google has found a way to ensure that aggregate data cannot be reidentified, it should publish its results.

So this is clearly a stunt meant to scare people who somehow haven’t yet realized that Google has search logs.

If there’s a privacy problem “surrounding this new tool”, then it should be evident from the tool itself. Since data is aggregated at the State level, and since the output is simply an estimate of flu activity for the whole State, there is no privacy risk to speak of. And, Google tells you in detail how FluTrends works.

Of course, Google does have access to your individual search records. So does Yahoo. If they don’t handle that data securely, or if they report individual data to outside entities, then yes there is a privacy problem, potentially a very large privacy problem. But that is completely independent of Flu Trends.

And it’s not like this aggregate data analysis is a new thing for Google: they’ve been analyzing and publishing trends for a while.

I’m all for privacy advocacy, and I do believe that Google needs to improve its commitment to privacy in general, with respect to anonymization of data, disclosure of data resale, and more. But I’m not so sure these privacy advocacy stunts are a good idea, especially on issues where privacy is actually well handled.

UPDATE: I see that Fred Totter is commending Peel and EPIC on this action, saying it reassures him. Interesting. But why is this reassuring? Surely, Google could have been mining data before Flu Trends? What is it about releasing this tool, with its detailed disclosures and explanations, that somehow tickles the privacy bone? Worth a second blog post soon’ish, I think.

In the coming era of personal genomics — when we all can decode our genes cheaply and easily — political candidates may be pressed to disclose their own DNA, like tax returns or lists of campaign contributors, as voters seek new ways to weigh a leader’s medical and mental fitness for public office.

Totally agreed. It’s not “may,” in my opinion, it’s inevitable. I think by 2016, it will be part of the Presidential Election discussion. If genetic testing is widespread, and there’s any history of mental illness or heart disease in a candidate’s family, then the press will come looking, and refusal to disclose will be seen as admission of a problem.

This is the issue that privacy advocates bring up regularly but that fails to resonate with people, even though it really should: the mere availability of data, even if locked with a password, threatens one’s privacy. It creates expectations of due release in certain conditions. Employers might consider it normal due diligence to ask if you’re genetically pre-disposed to anger and aggression when they hire you, just like it was perfectly acceptable, 30 years ago, to ask a woman if she planned on having kids anytime soon.

The only protection we have is legal and societal. It needs to become legally forbidden to ask for this kind of information, and it needs to become socially unacceptable, too. The Genetic Information Non-discrimination Act (GINA) takes us in the right direction on this front, though it’s likely not the last word.

So this is a (harsh) criticism of some folks on the Left and their attitude towards Science. It’s about how Science can be inconvenient, not just to the oil companies, but also to the hippies, yuppies, to you, and to me. Science has an annoying tendency not to follow any ideology, and a true scientist is one who can accept a failed hypothesis when confronted with the facts. You don’t get to pick and choose your evidence.

Let’s start with Bill Maher, a fantastic comedian whom I watched regularly until an insane pseudo-medicine rant a few weeks ago that turned my stomach. Bill argued that the reason we get sick and need drugs is because of “the swamp” of bad food (i.e. meat) that we eat:

I would never get the flu on a plane
[...]
You all look at me like I’m crazy…

Yes, Bill, you are crazy. If you’re sitting next to someone with the flu on the plane, no matter how little meat and much tofu you’ve been eating over the last few years, you’re going to get the flu, and if it’s a bad case of the flu, you might die. That’s how biology and germs work. The idea that somehow we have evolved into perfect beings who never get sick, that germs wouldn’t themselves evolve and find ways to reproduce and survive, is preposterous. This truth, supported by hundreds of years of medical evidence, might be inconvenient to folks at PETA who like to exaggerate the factory farming issue (which is a real issue). But it’s true nevertheless.

Bill continues:

What about the idea of side-effects? We see ads about drugs all the time on the news.
[...]
If you’re taking these drugs and you get these side-effects, that’s not the cure.

Of course drugs have side effects. That’s why drugs are typically not prescribed without good cause, because there’s always a downside. Doctors will prescribe a drug to you when they believe the positive effect of the drug significantly outweighs the potential negatives. This is clear and obvious to anyone with medical training or to someone, like me, who just asks their primary care physician a few simple questions. Bill: couldn’t you at least ask a doctor about this before going on your rant?

Interestingly, Bill’s rant here indirectly makes a good point: non-doctors, like Bill Maher, are ill-equipped to understand these drug ads. The magazine-based drug ads seem even worse to me: have you read the 2, sometimes 4 pages of fine print that follow a drug ad? Does anyone, really? Isn’t this a sign that maybe we’re targeting these ads at the wrong audience? That’s an issue worth discussing, Bill. Stick to the policy, and please leave the science to the scientists.

Vaccines

In medical school, my wife was shown graphs of childhood diseases and resulting deaths over the years. The trend was unmistakable: Republican administrations de-fund vaccination programs, diseases rise sharply, Democrats re-fund them, diseases drop. But again, this is not a post about Republicans. This is about how Democrats can be just as bad. The latest hip thing is the belief that vaccines cause other problems, in particular that some mercury-based preservatives (thimerosal) in vaccines (the MMR vaccine especially) cause Autism.

This theory was pushed by an RFK Jr. piece in Salon and Rolling Stone a few years ago. Sadly, the piece is full of exaggerations and statements taken out of context. But the story is back in the news now, because of a recent ruling by the vaccine court, which agreed to pay the medical costs for the family of a little girl who developed autism-like symptoms after receiving a number of vaccination shots.

Except… here are the facts that her parents agree with: the little girl has an extremely rare underlying (and at the time, undiagnosed) disorder, 9 vaccine shots were administered at once because she had missed some earlier (a practice against which most doctors strongly warn), and her disease is “autism-like”, but it’s not autism. In other words, we know what happened to this little girl: she had an extremely rare condition which, when combined with the excessive vaccination shots, triggered a sequence of events which led to brain damage. Blaming the vaccine here is a bit like blaming an antibiotic because some people are allergic. Should you test for and monitor the possibility of an allergic reaction? Of course. Should you forgo antibiotics altogether, letting thousands of people die from TB, pneumonia, etc..? I hope not.

The evidence against the MMR-autism link is extremely strong. The most compelling is that, since the mercury-based preservative was removed from vaccines in 2001, the rate of autism diagnosis has continued to increase at the same “alarming” rate in the US and two other countries. Experts agree that the increase in autism diagnosis is not an actual increase in the disease, rather it is an increase in the medical community’s understanding of what constitutes autism-spectrum diseases. We’ve always had Autism, we just didn’t know how to diagnose it.

Unfortunately, like most negative evidence, this kind of result is particularly difficult for most people to process (even scientists, who can get very emotionally attached to their hypotheses). We still don’t have an actual explanation for Autism, and we’d like someone to blame. The idea that we know nothing about how this disease comes about is unsettling. We don’t know what causes Autism, and that’s extremely inconvenient. But we do know it’s not vaccines. That’s the harsh truth, as best as Science can tell.

And the vaccination situation is getting worse. The New York Times reports on a trend in California to not vaccinate kids at all. This is, in no uncertain terms, insane and irresponsible. These parents actually hold “measles parties” to have their kids infect one another so they gain disease resistance the “natural way.” Like Bill Maher, these parents are forgetting that “the natural way” means certain death for a sizeable chunk of the population. The purpose of vaccines is to get the natural immunity without that pesky “side-effect” of death.

In the case of vaccines, this insane reasoning finds comfort in the particular state of the world we’re in: most kids are vaccinated, which means the viral pool is low and even unvaccinated kids don’t get infected, so parents don’t see the downside. But with more unvaccinated kids, the pool will increase, and the damage will spread even to the vaccinated kids (because vaccines are not 100% reliable), and more dramatically to the kids too young to get vaccinated.

A number of pharmaceutical companies have misled and continue to mislead us. But that doesn’t mean that all drugs and medical interventions are bad. It just means one should be cautious. The danger of some of the new drugs is that there isn’t enough research to substantiate their benefit. The danger of not vaccinating is that there’s overwhelming research that proves the enormous benefits of vaccination, both individually and to the population at large.

It’s about the research. It’s about the science. It doesn’t always fit some nice consistent ideology. It’s inconvenient. But it’s the truth. And since we live in a world where the truth has real consequences, we have to learn to deal with the inconsistencies in our mental models and be good scientists. Otherwise, we have no leg to stand on when criticizing folks with different ideologies who twist science to their liking.

Dossia was established by major U.S. employers Applied Materials, BP America Inc., Cardinal Health, Intel Corporation, Pitney Bowes Inc. and Wal-Mart to create a Web-based system that will enable employees to gain access to their own personal health data, which is now largely inaccessible to them. Dossia will use a Web-based infrastructure to empower individuals to manage their own health care, improve communications with their doctors, and provide more complete and accurate information for healthcare providers than the current system, which continues to be fragmented and still partially paper-based.

[...]

Dossia also announced a collaboration with Children’s Hospital Boston to provide strategic and technological expertise and guidance in creating, deploying and operating the Dossia infrastructure. For more than a decade, researchers in the Children’s Hospital Informatics Program (CHIP), based at Children’s Hospital Boston and affiliated with Harvard Medical School and with the Harvard-MIT Division of Health Sciences and Technology have been leaders in pioneering and promoting personal control of health information as a key to improving consumer health management and outcomes, and in developing rigorous, privacy-protective methods of ensuring patient control over their own medical information.

And on that note, back to work…