Category Archives: policy

the responsibility we have as software engineers

Posted on by

I had the chance to chat this week with the very awesome Kate Heddleston who mentioned that she’s been thinking a lot about the ethics of being a software engineer, something she just spoke about at PyCon Sweden. It brought me back to a post I wrote a few years ago, where I said:

There’s this continued and surprisingly widespread delusion that technology is somehow neutral, that moral decisions are for other people to make. But that’s just not true. Lessig taught me (and a generation of other technologists) that Code is Law

[…]

In 2008, the world turned against bankers, because many profited by exploiting their expertise in a rapidly accelerating field (financial instruments) over others’ ignorance of even basic concepts (adjustable-rate mortgages). How long before we software engineers find our profession in a similar position? How long will we shield ourselves from the responsibility we have, as experts in the field much like experts in any other field, to guide others to make the best decision for them?

Well, I think that time has come.

Everyone uses software, very few people understand it. What seems obvious to a small elite group is completely opaque to the majority of the world. This gap is incredibly hard for us, the software engineering elite, to see. A few examples:

  • The Radiolab Podcast did a wonderful piece – Trust Engineers – where they explored the case of Facebook running experiments on its newsfeed. For non-engineers, there’s an incredible feeling of breached trust upon realizing that a set of flesh-and-blood humans have that much control over the algorithm that feeds them daily information. (And, for that matter, to most researchers used to interacting with an IRB, there’s complete shock at what Facebook did.) For most engineers, including a number of very good and ethical people at Facebook, it’s surprising that this is even an issue.
  • A couple of years ago, a friend of a friend – who happens to be a world-renowned physician and research scientist – asked me: “Ben, can the system administrators at work read my email? Even if they don’t have my password?” The answer is yes and yes. This is obvious to us engineers, so much so that we don’t even think twice about it. To a non-engineer, even an incredibly smart person, this is absolutely non-obvious.
  • A close friend, another very smart person, was discussing something with his young child recently, and I overheard “if you don’t know, ask the computer, the computer knows and it’s always right.” Where do I begin?

We, software engineers, have superpowers most people don’t remotely understand. The trust society places in us is growing so rapidly that the only thing that looks even remotely similar is the trust placed in doctors. Except, most people have a pretty good idea of the trust they’re placing in their doctor, while they have almost no idea that every time they install an app, enter some personal data, or share a private thought in a private electronic conversation, they’re trusting a set of software engineers who have very little in the form of ethical guidelines.

Where’s our Hippocratic Oath, our “First, Do No Harm?”

I try very hard to think about this in my own work, and I try to share this sense of duty with every engineer I mentor and interact with. Still, I don’t have a good answer to the core question. Yet it feels increasingly urgent and important for us to figure this out.

Letter to President Obama on Surveillance and Freedom

Posted on by

Dear President Obama,

My name is Ben Adida. I am 36, married, two kids, working in Silicon Valley as a software engineer with a strong background in security. I’ve worked on the security of voting systems and health systems, on web browsers and payment systems. I enthusiastically voted for you three times: in the 2008 primary and in both presidential elections. When I wrote about my support for your campaign five years ago, I said:

In his campaign, Obama has proposed opening up to the public all bill debates and negotiations with lobbyists, via TV and the Internet. Why? Because he trusts that Americans, when given the tools to see and understand what their legislators are doing, will apply pressure to keep their government honest.

I gushed about how you supported transparency as broadly as possible, to enable better decision making, to empower individuals, and to build a better nation.

Now, I’m no stubborn idealist. I know that change is hard and slow. I know you cannot steer a ship as big as the United States as quickly as some would like. I know tough compromises are the inevitable path to progress.

I also imagine that, once you’re President, the enormity of the threat from those who would attack Americans must be overwhelming. The responsibility you feel, the level of detail you understand, must make prior principles sometimes feel quaint. I cannot imagine what it’s like to be in your shoes.

I also remember that you called on us, your supporters, to stay active, to call you and Congress to task. I want to believe that you asked for this because you knew that your perspective as Commander in Chief would inevitably become skewed. So this is what I’m doing here: I’m calling you to task.

You are failing hard on transparency and oversight when it comes to NSA surveillance. This failure is not the pragmatic compromise of Obamacare, which I strongly support. It is not the sheer difficulty of closing Guantanamo, which I understand. This failure is deep. If you fail to fix it, you will be the President principally responsible for the effective death of the Fourth Amendment and worse.

mass surveillance

The specific topic of concern, to be clear, is mass surveillance. I am not concerned with targeted data requests, based on probable cause and reviewed individually by publicly accountable judges. I can even live with secret data requests, provided they’re very limited, finely targeted, and protect the free-speech rights of service providers like Google and Facebook to release appropriately sanitized data about these requests as often as they’d like.

What I’m concerned about is the broad, dragnet NSA signals intelligence recently revealed by Edward Snowden. This kind of surveillance is a different beast, comparable to routine frisking of every individual simply for walking down the street. It is repulsive to me. It should be repulsive to you, too.

wrong in practice

If you’re a hypochondriac, you might be tempted to ask your doctor for a full body MRI or CT scan to catch health issues before detectable symptoms. Unfortunately, because of two simple probabilistic principles, you’re much worse off if you get the test.

First, it is relatively unlikely that a random person with no symptoms has a serious medical problem, ie the prior probability is low. Second, it is quite possible — not likely, but possible — that a completely benign thing appears potentially dangerous on imaging, ie there is a noticeable chance of false positive. Put those two things together, and you get this mind-bending outcome: if the full-body MRI says you have something to worry about, you actually don’t have anything to worry about. But try convincing yourself of that if you get a scary MRI result.

Mass surveillance to seek out terrorism is basically the same thing: very low prior probability that any given person is a terrorist, quite possible that normal behavior appears suspicious. Mass surveillance means wasting tremendous resources on dead ends. And because we’re human and we make mistakes when given bad data, mass surveillance sometimes means badly hurting innocent people, like Jean-Charles de Menezes.

So what happens when a massively funded effort has frustratingly poor outcomes? You get scope creep: the surveillance apparatus gets redirected to other purposes. The TSA starts overseeing sporting events. The DEA and IRS dip into the NSA dataset. Anti-terrorism laws with far-reaching powers are used to intimidate journalists and their loved ones.

Where does it stop? If we forgo due process for a certain category of investigation which, by design, will see its scope broaden to just about any type of investigation, is there any due process left?

wrong on principle

I can imagine some people, maybe some of your trusted advisors, will say that what I’ve just described is simply a “poor implementation” of surveillance, that the NSA does a much better job. So it’s worth asking: assuming we can perfect a surveillance system with zero false positives, is it then okay to live in a society that implements such surveillance and detects any illegal act?

This has always felt wrong to me, but I couldn’t express a simple, principled, ethical reason for this feeling, until I spoke with a colleague recently who said it better than I ever could:

For society to progress, individuals must be able to experiment very close to the limit of the law and sometimes cross into illegality. A society which perfectly enforces its laws is one that cannot make progress.

What would have become of the civil rights movement if all of its initial transgressions had been perfectly detected and punished? What about gay rights? Women’s rights? Is there even room for civil disobedience?

Though we want our laws to reflect morality, they are, at best, a very rough and sometimes completely broken approximation of morality. Our ability as citizens to occasionally transgress the law is the force that brings our society’s laws closer to our moral ideals. We should reject mass surveillance, even the theoretically perfect kind, with all the strength and fury of a people striving to form a more perfect union.

patriots

Mr. President, you have said that you do not consider Edward Snowden a patriot, and you have not commented on whether he is a whistleblower. I ask you to consider this: if you were an ordinary citizen, living your life as a Law Professor at the University of Chicago, and you found out, through Edward Snowden’s revelations, the scope of the NSA mass surveillance program and the misuse of the accumulated data by the DEA and the IRS, what would you think? Wouldn’t you, like many of us, be thankful that Mr. Snowden risked his life to give we the people this information, so that we may judge for ourselves whether this is the society we want?

And if there is even a possibility that you would feel this way, given that many thousands do, if government insiders believe Snowden to be a traitor while outsiders believe him to be a whisteblower, is that not all the information you need to realize the critical positive role he has played, and the need for the government to change?

the time to do something is now

I still believe that you are, at your core, a unique President who values a government by and for the people. As a continuing supporter of your Presidency, I implore you to look deeply at this issue, to bring in outside experts who are not involved in national security. This issue is critical to our future as a free nation.

Please do what is right so that your daughters and my sons can grow up with the privacy and dignity they deserve, free from surveillance, its inevitable abuses, and its paralyzing force. Our kids, too, will have civil rights battles to fight. They, too, will need the ability to challenge unjust laws. They, too, will need the space to make our country better still.

Please do not rob them of that opportunity.

Sincerely,

Ben Adida

The Onus is on Scientists – Shame on the AAAS

Posted on by

The American Association for the Advancement of Science (AAAS) has just come out against California’s Proposition 37, which would mandate the labeling of genetically-modified foods. In my opinion, the AAAS has failed its duty as promoters of Good Science.

The question is not whether genetically-modified foods are safe. I see the benefits, and I see the downsides (especially as a security guy, since food safety testing is, in my opinion, very poorly done), and the debate will rage on for a long time. But whether genetically-modified foods are safe is not the issue. The issue is whether consumers have a right to know what food they eat. There should be no debate here. Of course people have a right to know. And what better way to hear the people’s voice than to vote on this issue? The AAAS should be pro-labeling. If the AAAS believes that genetically-modified foods are, in fact, safer, as they claim in their statement, then they can make that point and rally the troops to explain to consumers that they should specifically seek out the GM-labeled foods. But withholding knowledge? Are you kidding me?

The world would be better off if people behaved according to scientific consensus. I wouldn’t have to worry about sending my kids to a school where up to 10% of kids might not be vaccinated, for example. But does that mean we should force parents to vaccinate their children? Of course not.

The onus is on scientists to make their case. Paternalism has no place in science. People have a right to know. The AAAS Board should be ashamed.

an ode to lessig’s optimism, taking on gigantic challenges… and a quibble

Posted on by

Last night, I went to see Lessig pitch his latest book, Republic, Lost. His latest spiel is fantastic, fine-tuned, gripping, thrilling, inspiring. I’ve been an avid fan of Lessigian story-telling for 13 years now. The way he sets up his argument, the way he goes far beyond the obvious, far beyond the quick fix, and the way he absolutely destroys any shred of doubt that may remain about his thesis. I saw him giving one of his first “Code” lectures at Harvard in 1998. In 2002, I waited in line at the Supreme Court and got to see the last five minutes of his argument. I saw him in the TV studio debating Jack Valenti. I was at the Creative Commons launch in 2003. I saw his first Corruption lecture at Stanford in 2008. It just doesn’t get old.

The central thing I deeply admire about Lessig is that he takes on gigantic battles with care and determination. He’s not deluded about his chances, but he fights anyways. He looks for, and finds, incredibly aggressive wins. Copyright reform against the Disneys of the world didn’t work, but Creative Commons is genuinely affecting how we share. The corruption of the political process is an impossible challenge, yet Lessig sees a path, and I believe his is the the most likely path to success. I don’t yet know how Lessig will find the equivalent of the Creative-Commons-win in this much larger battle. But I know he’s thinking about it, and I believe that, in time, he will move the needle, significantly.

That kind of “crazy” optimism is deeply inspiring, because it is, indeed, the only way to change the world. Time is too precious not to focus on the big, gigantic, mind-blowing battles. Lessig reminds me of that every time I attend one of his talks.

So, a quibble. Lessig brought up one argument I’ve seen him make before: because vaccine policy is influenced by experts who may have received compensation from the pharmaceutical industry, people may lose trust in vaccine policy. Now let’s be clear: Lessig is not saying that vaccines are unsafe. He’s saying that, because some vaccine experts do not appear to be fully unbiased, it is understandable that people lose trust in vaccine policy.

I disagree, and I think it weakens Lessig’s argument to make this connection. I’d like to see Paul Offit and his peers deciding our vaccine policy (in a public forum of course), even though he’s getting rich from his amazing Rotavirus vaccine. Checks and balances in areas that require deep expertise cannot be achieved by banning from advisory boards all experts with a potential conflict of interest. In fact, that’s a recipe for disaster by way of mediocrity. We have other checks and balances for this. We can require peer-reviewed publications. We can fund counter-studies. We can let the truth rise to the top via competition. This country’s national vaccine policy is something to be proud of.

There is, however, a subtle but serious corruption in the medical world that should make it into Lessig’s slideshow: pharmaceutical reps routinely treat physicians to dinners, trips, etc. They leave free drug samples, they leave pens and paper pads with drug logos prominently featured, they suggest that new drugs are better than old tried-and-true drugs, and sometimes they very subtly suggest off-label uses. Drug companies receive prescription records for individual physicians: they know where they’re having an impact and can calculate very clear Return On Investment. The result: Vioxx. Physicians aren’t evil, but they are human. The grey areas in medicine are large and common, providing fertile ground for skilled influencing.

That needs to stop: where vaccine policy is a mostly public forum with competing ideas, there isn’t any oversight or counter-balance to drug-rep influence. We can change this. Doctors could be required to provide to all patients, alongside the insane HIPAA disclosure form, a funding disclosure form of all compensation received from drug reps. That disclosure form alone might make doctors think twice before prescribing a drug, and drug reps before paying for dinner. And institutions should follow the path blazed by Mass General, banning their physicians from accepting gifts and banning pharmaceutical reps from physician offices.

with great power…

Posted on by

When Arvind writes something, I tend to wait until I have a quiet moment to read it, because it usually packs a particularly high signal to noise ratio. His latest post In Silicon Valley, Great Power but No Responsibility, is awesome:

We’re at a unique time in history in terms of technologists having so much direct power. There’s just something about the picture of an engineer in Silicon Valley pushing a feature live at the end of a week, and then heading out for some beer, while people halfway around the world wake up and start using the feature and trusting their lives to it. It gives you pause.

So true. I’ve been thinking about this issue a lot recently, especially as good technologists in the Valley are in exceptionally good financial / career health, while the rest of the country, and sometimes even the other half of our cities, are suffering through a long and deep recession.

Here’s one story that blew my mind a few months ago. Facebook (and I don’t mean to pick on Facebook, they just happen to have a lot of data) introduced a feature that shows you photos from your past you haven’t seen in a while. Except, that turned out to include a lot of photos of ex-boyfriends and ex-girlfriends, and people complained. But here’s the thing: Facebook photos often contain tags of people present in the photo. And you’ve told Facebook about your relationships over time (though it’s likely that, even if you didn’t, they can probably guess from your joint social network activity.) So what did Facebook do? They computed the graph of ex-relationships, and they ensured that you are no longer proactively shown photos of your exes. They did this in a matter of days. Think about that one again: in a matter of days, they figured out all the romantic relationships that ever occurred between their 600M+ users. The power of that knowledge is staggering, and if what I hear about Facebook is correct, that power is in just about every Facebook engineer’s hands.

Here’s another story. I used to lecture MIT Undergraduates about web security. My approach was basically: (a) hack a few of the student project web sites, then (b) hack a few public web sites to make the students understand how widespread the problems are. In late 2003, I showed students how to buy movie tickets for free (the price of the ticket was held in a hidden variable in a web form… duh). I ended my lecture with “but just because you can do this, doesn’t mean you should. Please don’t do this.” Over the years, I’ve received a few emails from former students to the tune of “hey Ben, you gave an awesome lecture, I still remember how a bunch of us went out to see Matrix 3 for free that weekend!”

I shudder to think about what happens when you put those two stories together. While the earliest hackers may have had a particularly well developed ethical sense, I get the sense that our profession’s average ethical sense doesn’t nearly measure up to the incredible power we have gained precipitously over the last 15 years.

And then there’s the additional point Arvind makes, which I’ve observed directly too:

I often hear a willful disdain for moral issues. Anything that’s technically feasible is seen as fair game and those who raise objections are seen as incompetent outsiders trying to rain on the parade of techno-utopia.

Yes! There’s this continued and surprisingly widespread delusion that technology is somehow neutral, that moral decisions are for other people to make. But that’s just not true. Lessig taught me (and a generation of other technologists) that Code is Law, or as I prefer to think about it, that Code defines the Laws of Physics on the Internet. Laws of Physics are only free of moral value if they are truly natural. When they are artificial, they become deeply intertwined with morals, because the technologists choose which artificial worlds to create, which defaults to set, which way gravity pulls you. Too often, artificial gravity tends to pull users in the direction that makes the providing company the most money.

A parting thought. In 2008, the world turned against bankers, because many profited by exploiting their expertise in a rapidly accelerating field (financial instruments) over others’ ignorance of even basic concepts (adjustable-rate mortgages). How long before we software engineers find our profession in a similar position? How long will we shield ourselves from the responsibility we have, as experts in the field much like experts in any other field, to guide others to make the best decision for them?

intelligently designing trust

Posted on by

For the past week, every security expert’s been talking about Comodo-Gate. I find it fascinating: Comodo-Gate goes to the core of how we handle trust and how web architecture evolves. And in the end, this crisis provides a rare opportunity.

warning signs

Last year, Chris Soghoian and Sid Stamm published a paper, Certified Lies [PDF], which identified the very issue that is at the center of this week’s crisis. Matt Blaze provided, as usual, a fantastic explanation:

A decade ago, I observed that commercial certificate authorities protect you from anyone from whom they are unwilling to take money. That turns out to be wrong; they don’t even do that much.

A Certificate Authority is a company that your web browser trusts to tell it who is who on the Internet. When you go to https://facebook.com, a Certificate Authority is vouching that, yes, this is indeed Facebook you’re talking to directly over a secure channel.

What Chris and Sid highlighted is an interesting detail of how web browsers have chosen to handle trust: any Certificate Authority can certify any web site. That design decision was reasonable in 1994, when there were only two Certificate Authorities and the world was in a rush to secure web transactions. But it’s not so great now, where a Certificate Authority in Italy can delegate its authority to a small reseller, who can then, in turn, certify any web site, including Facebook and Gmail, using more or less the level of assurance the small reseller sees fit.

what happened

It looks like someone from Iran hacked into one of the small resellers three degrees of delegation away from Comodo to issue to some unknown entity (the Iranian government?) certificates for major web sites, including Google and Microsoft. This gave that entity the power to impersonate those web sites, even over secure connections indicated by your browser padlock icon. It’s important to understand that this is not Google or Microsoft’s fault. They couldn’t do anything about it, nor could they detect this kind of attack. When Comodo discovered the situation, they revoked those certificates… but that didn’t do much good because the revocation protocol does not fail safely: if your web browser can’t contact the revocation server, it assumes the certificate is valid.

a detour via Dawkins, Evolution, and the Giraffe

Richard Dawkins, the world-famous evolutionary biologist, illustrates the truly contrived effects of evolution on a giraffe. The laryngeal nerve, which runs from the brain to the larynx, takes a detour around the heart. In the giraffe, it’s a ludicrous detour: down the animal’s enormous neck, around the heart, and back up the neck again to the larynx, right near where the nerve started to begin with!

If you haven’t seen this before, you really need to spend the 4 minutes to watch it:

In Dawkins’s words:

Over millions of generations, this nerve gradually lengthened, each small step simpler than a major rewiring to a more direct route.

and we’re back

This evolution is, in my opinion, exactly what happened with certificate authorities. At first, with only two certificate authorities, it made sense to keep certificate issuance as simple as possible. With each added certificate authority, it still made no sense to revamp the whole certification process; it made more sense each time to just add a certificate authority to the list. And now we have a giraffe-scale oddity: hundreds of certificate authorities and all of their delegates can certify anyone, and it makes for a very weak system.

This isn’t, in my mind, a failure of software design. It’s just the natural course of evolution, be it biology or software systems. We can and should try to predict how certain designs will evolve, so that we can steer clear of obvious problems. But it’s very unlikely we can predict even a reasonable fraction of these odd evolutions.

the opportunity

So now that we’ve had a crisis, we have an opportunity to do something that Nature simply cannot do: we can explore radically redesigned mechanisms. We can intelligently design trust. But let’s not be surprised, in 15 years, when the wonderful design we outline today has evolved once again into something barely viable.

taking further example from nature?

Nature deals with this problem of evolutionary dead-ends in an interesting way: there isn’t just one type of animal. There are thousands. All different, all evolving under slightly different selection pressures, all interacting with one another. Some go extinct, others take over.

Should we apply this approach to software system design? I think so. Having a rich ecosystem of different components is better. We shouldn’t all use the same web browser. We shouldn’t all use the same trust model. We should allow for niches of feature evolution in this grand ecosystem we call the Web, because we simply don’t know how the ecosystem will evolve. How do we design software systems and standards that way? Now that’s an interesting question…

i changed my mind on nuclear power

Posted on by

Until this recent catastrophe in Japan (it’s awful, please consider helping out), I was very pro nuclear-power. I’ve never been afraid of technology, and I was raised in France, where 80% of electricity comes from nuclear power and there has been no serious safety problem with it. Plus, nuclear power can be green. And with newer technology, it can be made passively safe, where even if everything fails, a meltdown cannot occur (unlike the Japanese reactors, unfortunately.)

So the recent crisis has changed my mind. I don’t think we can afford the risk of nuclear power. I’m not a nuclear power expert, and I would welcome counter-arguments. But I am fairly well versed in thinking about risk and risk mitigation. Three things now worry me greatly about nuclear power:

  • Dramatic outcomes: in case of dramatic failure, the outcome could be disastrous on a scale that’s difficult to comprehend. You think the oil spill in the Gulf of Mexico was bad (and it was)? Try decades or centuries of life-killing radioactivity. Imagine a meltdown that could contaminate large, heavily populated areas. The damage could be enormous. Yes, the probability is very, very low. But as we are seeing today in Japan, it’s far from zero, and if they had not reacted as well as they did, the result could be indeed as bad as I describe here. (To folks I work with on voting technology: isn’t this what we worry about regarding Internet voting for public office? That the outcome of an attack would be dramatically bad, not matter how low the likelihood?)

  • Storing nuclear waste: a friend on Facebook said “if Romans had used nuclear power, we would still be guarding their nuclear dump sites.” Think about that for a second. That’s just breathtaking. Are we ready to impose on our descendents 1000 years from now? We can barely figure out broad swaths of history from that long ago, let alone instructions on how to safeguard nuclear materials. Maybe it can be done. But it seems incredibly arrogant of us to assume that it’s okay to impose this burden on the next hundred generations.

  • Regulation (or lack thereof): this is my most pragmatic point, and it applies mostly to the US. We can’t even get our act together in this country to agree on requiring relief wells for deep-water oil drilling. Do we really think we can get our act together to regulate a nuclear industry to be truly safe? It looks like even Japan couldn’t quite do it, and they’re far more open to government safety regulation than we are.

So, I’m open to others’ arguments. But right now, I’m thinking nuclear power is not such a great idea.

Wikileaks — not ideal, but a force for good in the end

Posted on by

I’ve found myself quite conflicted over the latest Wikileaks “dump”, specifically the hundreds of thousands of US diplomatic cables.

On the one hand, there is no doubt that the mainstream press is failing miserably in its role of investigating and breaking stories about illegal secret activities. We’ve seen numerous high-profile publications delay stories for fear of impacting elections (e.g. the NY Times and Bush-era warrantless wiretapping). Where the War in Iraq is concerned, it seems fairly clear that the US government misled its people, and that, in my opinion, deserves complete whistleblower protection.

On the other hand, while Wikileaks claims to have information proving banking corruption during the financial crisis, BP corruption during the oil spill, and many others, they chose to release secret diplomatic cables first. The argument that the people have the right to know everything the government does in real time does not hold water: many lives have been saved by secret operations and negotiations. Secrecy has a role to play in a peaceful society. Of course, all information should eventually be made public, so the Freedom of Information Act is critical, and multi-partisan oversight of secret operations and negotiations is necessary while those are ongoing. So what is the justification for this particular leak? Does it reveal significant lies by the US government where the public is being deeply misled? I don’t quite see it, although it’s possible that I’m not looking closely enough.

All that said, in this fog of uncertainty, some (many) are arguing that Wikileaks is a terrorist organization and that Julian Assange should be arrested. Senators are pressuring tech companies to censor the information, and tech companies are buckling at record speed (ahem, Amazon, Paypal,…) This line of argument is deeply disturbing, and the speed with which the system is cracking down on Wikileaks through political pressure is surprisingly scary. Where is due process? Whatever happened to freedom of the press? Recently, some members of the State Department have implied that students vying for jobs with them should refrain from publicly discussing Wikileaks. Ummm, which country is this again? Home of the Brave, Land of the Free, right?

One note to the Wikileaks folks: why not focus on the areas that are clear no-brainers first? Tell us about the BP corruption. Tell us about how the banks abused the bailout funds. This is true, unadulterated whistle blowing. In the end, there may well be a case that releasing these diplomatic cables is proper whistleblowing. Unfortunately, it’s not nearly as clear-cut, and that is going to hurt the Wikileaks mission significantly in the long run.

All that said, Mr. Assange, you have balls of steel. I can’t quite believe that you are real, but I’m glad people like you exist to fight bravely for freedom of information, even if, in some cases, I’m not sure I agree with your judgment calls.

devices, payload data, and why Kim is (in part) right.

Posted on by

A few days ago, I wrote about privacy advocacy theater and lamented how some folks, including EPIC and Kim Cameron, are attacking Google in a needlessly harsh way for what was an accidental collection of data. Kim Cameron responded, and he is right to point out that my argument, in the Google case, missed an important issue.

Kim points out that two issues got confused in the flurry of press activity: the accidental collection of payload data, i.e. the URLs and web content you browsed on unsecured wifi at the moment the Google Street View car was driving by, and the intentional collection of device identifiers, i.e. the network hardware identifiers and network names of public wifi access points. Kim thinks the network identifiers are inherently more problematic than the payload, because they last for quite a bit of time, while payload data, collected for a few randomly chosen milliseconds, are quite ephemeral and unlikely to be problematic.

Kim’s right on both points. Discussion of device identifiers, which I missed in my first post, is necessary, because the data collection, in this case, was intentional, and apparently was not disclosed, as documented in EPIC’s letter to the FCC. If Google is collecting public wifi data, they should at least disclose it. In their blog post on this topic, Google does not clarify that issue.

So, Google, please tell us how long you’ve been collecting network identifiers, and how long you failed to disclose it. It may have been an oversight, but, given how much other data you’re collecting, it would really improve the public’s trust in you to be very precise here.

Now, two points:

  1. taking a second look at EPIC’s letter and Kim’s original post, it still seems to me that there’s some confusion of the device identifier and payload data issues: the uproar materialized after Google revealed they had mistakenly collected payload data, and EPIC’s letter and Kim’s original post seem to weave back and forth between both issues, never really mentioning intent. Is this because the payload data story is juicier in headlines, and so bundling the two issues helps make the more important point? Maybe, but still, I think we should be more precise and careful when we attack on privacy grounds.

  2. I agree that device privacy can be a big deal, especially when many people are walking around with RFIDs in their passports, pants, and with bluetooth headsets. But, in this particular case, is it a problem? If Google really only did collect the SSIDs of open, public networks that effectively invite anyone to connect to them and thus discover network name and device identifier, is that a violation of privacy, or of the Laws of Identity? I’m having trouble seeing the harm or the questionable act. Once again, these are public/open wifi networks. For the most part, these are static access points. Given Google’s stated interests in providing geolocation services, it would be detrimental to them if they catalogued roving access points. So, what’s the worst-case scenario here? Is it that, when I move to a new apartment, Google will know?

None of this excuses Google’s lack of disclosure. This was intentional data collection, it should be disclosed, period.

And it’s worth asking the questions that Kim asks, raising awareness of device privacy. I’m not sure I’m as worried as Kim is on this particular issue, but the questions are certainly legitimate.

So, in the end, the privacy advocacy theater is coming first and foremost from the EU privacy folks, who did get enraged about payload data more than anything else. There’s still some coming from EPIC and, to remain blunt, a little bit from Kim’s first post. But his second post brings up very legitimate questions, and Google should take some additional action here, at least to let us know what they were collecting, when, and whether they properly disclosed it.

Privacy Advocacy Theater

Posted on by

Ed Felten recently used the very nice term Privacy Theater in describing the insanity of 6,000-word privacy agreements that we pretend to understand. The term, inspired by Bruce Schneier’s “security theater” description of US airport security, may have been introduced by Rohit Khare in December 2009 on TechCrunch, where he described how “social networks only pretend to protect your privacy.” These are real issues, and I wholeheartedly agree that long privacy policies and generally consumer-directed fine-print are all theater.

I want to focus on a related problem that I’ll call privacy advocacy theater. This is a problem that my friends and colleagues are guilty of, and I’m sure I’m guilty of it at times, too. Privacy Advocacy Theater is the act of extreme criticism for an accidental data breach rather than a systemic privacy design flaw. Example: if you’re up in arms over the Google Street View privacy “fiasco” of the last few days, you’re guilty of Privacy Advocacy Theater. (If you’re generally worried about Google Street View, that’s a different problem, there are real concerns there, but I’m only talking about the collection of wifi network payload data Google performed by mistake.)

I’m looking at you, EU Privacy folks, who are investigating Google over accidental data collection. Where is your investigation of Opera, which provides Opera Mini, billed as “smarter web browsing”, smarter in the sense that it relays all data, including secure connections to your bank, through Opera’s servers? We should be much more concerned about designs that inherently create privacy risk. Oh sure, it’s easy political points to harp on accidental breaches for weeks, but it doesn’t help privacy much.

I also have to be harsh with people I respect deeply, like Kim Cameron who says that Google broke two of his very nicely crafted Laws of Identity. Come on, Kim, this was accidental data collection by code that the Google Street View folks didn’t even realize was running. (I’m giving them the benefit of the doubt. If they are lying, that’s a different problem, but no one’s claiming they’re lying, as far as I know.) The Laws of Identity apply predominantly to the systems that individuals choose to use to manage their data. If anyone is breaking the Laws of Identity, it’s the wifi access points that don’t actively nudge users towards encrypting their wifi network.

Another group I deeply admire and respect is EPIC. Here, they are also guilty of Privacy Advocacy Theater: they’re asking for an investigation into Google’s accidental wifi data collection. Now, I’m not a lawyer, and I certainly wouldn’t dare argue the law with Marc Rotenberg. But using common sense here, shouldn’t intent have something to do with this? Google did not intend to collect this data, didn’t even know they had it, and didn’t make any use of it. Shouldn’t we, instead of investigating them, help them define a process, maybe with third-party auditing from folks at EPIC, that helps them catalog what data they’re collecting, what data they’re using, etc…? At the very least, can we stop the press releases that make no distinction between intentional and unintentional data collection?

I’m getting worked up about this Privacy Advocacy Theater because, in the end, I believe it hurts privacy. Google is spending large amounts of time and money on this issue which is, as I’ve described previously, an inevitability in computer systems: accidental breaches happen all the time. We should be mostly commending them for revealing this flaw, and working with them to continue regular disclosure so that, with public oversight, these mistakes are discovered and addressed. Google has zero interest in making these mistakes. Slapping them on the wrist and having them feel some pain may be appropriate, but too much pain and too much focus on this non-issue is akin to a full-on criminal trial for driving 10 miles per hour over the speed limit: everyone’s doing it. Just fine them and move on. Then spend your time going after the folks who, by design, are endangering millions of users’ privacy.

There are plenty of real, systemic privacy issues: Facebook’s data sharing and privacy controls, Opera Mini’s design (tens of millions of users relaying all of their data to Opera, by design), Google’s intentional data retention practices, web-based ad networks, … We have enough real issues to deal with, who needs the advocacy theater?