managing photos and videos

Posted on by

This holiday, I finally spent time digging into how I manage photos and videos. With 2 young kids and some remote family and friends, this requires a good bit of thinking and planning. I know I’m not the only one, so I figured documenting where I landed might be useful to others.

I started with Dave Liggat’s Robust Photo Workflow, and found much of it resonates with my needs. Here’s where I landed:

  1. I take photos with a DSLR and two phones. My wife takes photos with her phone. We both take videos with our phones. We use Dropbox/Carousel auto-upload, which works just fine on both iOS and Android. For the DSLR, I manually load photos over USB.

  2. All photos and videos are now available on my desktop Mac (via USB or Dropbox). When I’m ready to review/edit photos, I drag and drop the batch into an all-photos/ directory I keep within my Dropbox.

  3. Hazel automatically categorizes photos and videos into subdirectories of the form 2015/01/. It’s really kind of awesome.

  4. all-photos and all-videos are thus simple date-classified folders of all of my photos and videos. They’re backed up locally using Time Machine. They’re backed up to the network using Dropbox. I can imagine eventually snapshotting this to Amazon S3/Glacier, but right now that doesn’t feel too urgent.

  5. I use Lightroom5 as an editor only, so if I blow away my Lightroom proprietary catalog info, it’s not that big a deal. To do this, I tell Lightroom to look at photos in all-photos without moving/copying them. After I’ve added a bunch of photos to the all-photos directory by drag-and-drop, I tell Lightroom to synchronize its catalog with the source folder, which takes a few seconds and gives me a screen with my latest imported photos and videos. I can then edit photos, reject them if they’re bad, and write back JPG/XMP data to each photo’s originating directory using Lightroom export. Dropbox backs those up automatically. To remove bad photos (blurry, etc.), I flag them as “rejected” in Lightroom using the X key, and when I’m done I command-delete, which gives me the option of removing the files from disk, too. I do this only for clear rejects, and it makes my mild OCD happy since I know I am not keeping totally useless files around, and the overhead of deleting photos is low. I could also delete photos easily using the Dropbox UI, which is pretty good, and then re-synchronize in Lightroom.

  6. I can then use Carousel (or Dropbox) on any mobile device to browse through all of my photos. It’s surprisingly good at dealing with large photo libraries (I have 20K) and large photos (I have a bunch of 13MP photos). As in, really, really good, even on a puny phone. Better than anything else I’ve seen.

  7. I’ve been using Flickr for years for private photo sharing, and Lightroom is really good at exporting to Flickr. That said, at this point I’m thinking of moving to Dropbox/Carousel based sharing. I can easily bundle photos & videos into albums on Dropbox, whereas videos are still limited on Flickr. Carousel conversations around a few photos are great with family. The only bummer is that Carousel and Dropbox have some mutually exclusive features: albums on Dropbox, conversations on Carousel. I suspect Dropbox will fix that in the next year.

  8. What I’d love to see:
    • unified photo features in Dropbox and Carousel
    • export Dropbox albums as directories of symlinks in my Dropbox folder, and export Carousel conversations in some other file-based way, too.
    • Lightroom export compatibility with Dropbox/Carousel albums.

I’m super happy with this new process: one funnel, easy, low overhead, and a very solid long-term photo storage solution. I’m only relying on RAW/JPG files and directories of said files to be readable for the long term, and that seems pretty safe. Lightroom is awesome, but I could replace it with a different tool if I needed to.

One more thing: if you’re going to use Dropbox to store all of your photos, make sure you pick a strong password and set up 2-factor authentication.

Power & Accountability

Posted on by

So there’s this hot new app called Secret. The app is really clever: it prompts you to share secrets, and it sends those secrets to your social circle. It doesn’t identify you directly to your friends. Instead, it tells readers that this secret was written by one of their friends without identifying which one. The popularity of the app appears to be off the charts, with significant venture-capital investment in a short period of time. There are amazing stories of people seeking out emotional support on Secret, and awful stories of bullying that have caused significant uproar. Secret has recently released features aimed at curbing bullying.

My sense is that the commentary to date is missing the mark. There’s talk of the danger of anonymous speech. Even the founders of Secret talk about their app like it’s anonymous speech:

“Anonymity is a really powerful thing, and with that power comes great responsibility. Figuring out these issues is the key to our long-term success, but it’s a hard, hard problem and we are doing the best we can.”

And this is certainly true: we’ve known for a while that anonymous speech can reveal the worst in people. But that’s not what we’re dealing with here. Posts on Secret are not anonymous. Posts on Secret are guaranteed to be authored by one of your friends. That guarantee is enabled and relayed by the Secret platform. That’s a very different beast than anonymity.

In general, if you seek good behavior, Power and Accountability need to be connected: the more Power you give someone, the more you hold them Accountable. Anonymity can be dangerous because it removes Accountability. That said, anonymity also removes some Power: if you’re not signing your name to your statement, it carries less weight. With Secret, Accountability is absent, just like with anonymous speech, but the power of identified speech remains in full force. That leads to amazing positive experiences: people can share thoughts of suicide with friends who can help, all under the cloak of group-anonymity that is both protecting and empowering. And it leads to disastrous power granted to bullies attacking their victims with the full force of speaking with authority – the bully is one of their friends! – while carrying zero accountability. That kind of power is likely to produce more bullies, too.

This is so much more potent that anonymity. And if this fascinating experiment is to do more good than harm, it will need to seriously push the envelope on systems for Accountability that are on par with the power Secret grants.

Here’s a free idea, straight out of crypto land. In cryptographic protocols that combine a need for good behavior with privacy/anonymity protections, there is often a trigger where bad behavior removes the anonymity shield. What if Secret revealed the identity of those users found to be in repeated violation of a code of good behavior? Would the threat of potential shame keep people in line, leaving the good uses intact while disincentivizing the destructive ones?

where the system eats itself

Posted on by

Larry Lessig just launched MayOne.us, the SuperPAC to end all SuperPACs. The idea is disarmingly simple: since SuperPACs funded by billionaires are corrupting politics, let’s crowd-source a SuperPAC funded by individuals, which will then work to put in power officials who answer to the people, maybe by undoing the whole SuperPAC insanity. Use a SuperPAC to kill all SuperPACs.

This is a fascinating pattern that we’ve actually seen before. And it makes me very, very happy, because it is the ultimate policy hack.

Take the GPL or the Apache License, two significant software licenses that make possible open-source and thus much of today’s software. These licenses, especially the GPL, enforce certain constraints on how source code can be used. If you take GPL source code, modify it, and redistribute it, you have to provide the source code to your modifications, too. If you don’t… you lose your license on the code to begin with, and now you’re guilty of copyright violation because you redistributed code without permission.

In other words, many open-source licenses work only because they leverage strong copyright law. The same goes for Creative Commons: you can freely license your work while requiring that people give you credit for it, or, if you prefer, only if they use it for non-commercial purposes. The only reason you can add those constraints is because Creative Commons is layered on top of Copyright. Some people believe Copyright Law has overreached. Those same people are using that Copyright overreach as a foundation for a stronger Commons.

The GPL, Creative Commons, and now MayOne basically use the overreach of the system against itself.

I find this idea – that an unnaturally strong system of rules can be counter-balanced by using the system against itself – fascinating and kind of awesome. Are there other areas where this can be applied?

But before I digress: go pledge to MayOne.us. Let’s make SuperPACs eat themselves.

Obama lets NTSB exploit widespread tire vulnerabilities

Posted on by

Stepping into a heated debate within the nation’s transportation safety agencies, President Obama has decided that when the National Transportation Safety Board (NTSB) discovers major flaws in transportation equipment, it should — in most circumstances — reveal them to assure that they will be fixed, rather than keep mum so that the flaws can be used in espionage, senior administration officials said Saturday.

But Mr. Obama carved a broad exception for “a clear national security or law enforcement need,” the officials said, a loophole that is likely to allow the NTSB to continue to exploit safety flaws to apprehend terrorist suspects as they go about their daily routines such as driving.

On Friday, the White House denied that it had any prior knowledge of the AirBleed defect, a newly discovered safety vulnerability in commonly used Michelin tires that often leads to spontaneous tire explosion when driving for prolonged periods of time at exactly 44mph. This flaw has led many manufacturers to urgently recall cars and many drivers to behave erratically in 45mph speed zones. The White House statement said that when such flaws are discovered, there is now a “bias” in the government to share that knowledge with car and tire manufacturers, so a remedy can be created and distributed to industry and consumers.

Sources indicate that some senior officials had urged the NTSB to get out of the business of weakening commercial transportation systems or trying to build in “trapdoor failures” that would make it far easier for the agency to intercept suspected terrorists. These officials concluded that the practice would undercut trust in the American auto industry. In recent months, Detroit has urged the United States to abandon such practices, while Germany and Japan, among other nations, have said they were considering pulling all auto production facilities back to their own countries.

Not surprisingly, officials at the NTSB and at its military partner, the United States Special Operations Command, warned that giving up the capability to exploit undisclosed safety flaws in widespread commercial equipment would amount to “unilateral disarmament” — a phrase taken from the battles over whether and how far to cut America’s nuclear arsenal.

When interviewed at his home in Maryland, John Smith, head of the NTSB in the 1980s, appeared incredulous: “Do you mean to tell me that the American Government knew about widespread life-threatening safety issues in our cars and chose not to disclose those findings to its citizens, just in case this weakness could be used to go after a presumed terrorist? I don’t believe it. The US government serves its people first, and never in a conspiracy-theorist’s wildest dreams would they engage in such despicable behavior.”

[a light parody of this New York Times Article.]

to Brendan and Mozilla

Posted on by

I was in the middle of writing a blog post about the controversy surrounding Mozilla when my Twitter feed exploded with the news that Brendan Eich stepped down from his new appointment as Mozilla CEO. So this is a different post. Also, this is not a post about Prop8 (which I abhored) or gay marriage (which I consider a basic civil right.)

to Brendan

There is little love lost between me and Brendan. We have different styles, and I butted heads with him in tech discussions on a regular basis while I was at Mozilla. He was, at times, infuriating. To be honest, he drove me up the wall.

Yet through all of these disagreements and conflicts, one thing was never in doubt: Brendan has the Mozilla mission of inclusiveness, openness, and freedom engrained in his heart. This is a man who has tirelessly worked in the open for close to 20 years, giving up riches at other companies so he could create an open playground for the world to use, a counter-balance to corporate interests on the Web, and opportunity for billions, even if they never know his name.

So Brendan, thank you, for everything you’ve done for the Open Web and for Humanity. The world is in your debt. You can take the man out of Mozilla, but you can never take the Mozilla mission out of the man.

to my friends at Mozilla

Some people love Mozilla for its mission, but much of the world doesn’t get it. People use Firefox because they like it; others use a different browser because they like that one better. I don’t think that will ever really change. It’s easy to become very depressed about this, to think “how can people be so harsh on Mozilla when we’ve been such good citizens?” or “can’t people see that Firefox is the only browser built by a non-profit?” People don’t care as much as they should. One bad PR cycle might be enough for them to switch browsers.

I want to suggest that this apathy can be empowering: don’t count on others groking the mission. Follow the Mozilla mission on your own terms, because you know it’s the right thing to do. Do the right thing because it is the right thing.

Keep doing the right thing, friends, and be excellent to each other.

when selfish acts become altruistic

Posted on by

My first open-source contribution was in 1998, when a ragtag bunch of web hackers and I published the first version of one of the first web application toolkits. In 2000, after I’d left the original project, a few other hackers and I “forked” that codebase to make it work on an open-source database, meaning we took the code, copied it to a different repository, and took it down a different path than that envisioned by its maintainers.

That has always been the beauty of open-source: if you don’t like the direction of the software, you can always fork it. The dirty little secret of open-source at the time was that this was much more an abstract threat than a common occurrence. Forking was almost always a bad word, a huge undertaking, and it happened very rarely. One central reason for this was that forking was almost always a one-way street: once you forked, it became very difficult to share improvements between forks.

So, in theory, thanks to open-source licenses, forking was always possible. In practice, it was a huge undertaking, one with deep consequences, and usually a sign of something gone awry.

Then in 2008 came Github. Github made hosting source code super easy. Github made forking other people’s code super easy, too. And thanks to the underlying Git system, Github made merging just as easy as forking. Within a few weeks, more than 6200 projects on Github had been forked at least once.

Forking became the optimistic way to contribute to a project: fork it, modify it for your own use, and suggest the change back to the original author. If the original author accepts your change, great, you can forget about your fork and go back to using the original code. If the author rejects your change, you can keep using your forked version, occasionally merging in upstream changes from the original author.

So forking became a good thing, a sign of interest in your project. People wore “Fork Me” t-shirts. And it was all done for years with little attention paid to the specifics of the licenses underlying it all. It was just assumed that, if you made a Github project public, you allowed open-source style forking.

In many ways, Github made real what open-source licenses mostly theorized. Standing on the shoulders of giants, contributing just a little tweak very easily, taking a different direction when you need to, etc. All the beauty of a vast open repository of code you can pick from and contribute to exists in Github.

And somehow, this amazing sharing ecosystem is based on purely selfish incentives. I need to host my code. I don’t like paying for things unless I need to, so I’ll make it public, because Github makes me pay for private repositories. I sometimes need to change other people’s code, and there’s a button for that. If someone changes my code I’d like to benefit from it, and there’s a button for that, too.

Like the Back-to-the-Future deLorean that runs on garbage, Github produces sharing behavior by consuming selfish acts.

I’d like to see many other Githubs. And I know startups are pitching Github-like projects to VCs daily. But it’s not just about a place to host and remix stuff. The magic of Github is that it generated a sharing ecosystem out of selfish incentives. Not sharing and selfishness side by side. Not questionable sharing of private content for the sake of virality. Sharing as a true side-effect of selfish behavior.

That’s not easy. And if it can be done in fields other than source code… I really like what that could do to benefit human knowledge and progress.

the French like their strikes like Americans like their guns

Posted on by

This week, French taxis went on strike because the government passed a law that made Uber and other modern chauffeur equivalents artificially less competitive… but apparently not sufficiently less competitive, and that was a tragedy that only a massive strike could rectify. Then when people jumped into Uber cars because, hey, there were no cabs, those cars were attacked, leaving some passengers bleeding and stranded on the side of the road.

If you go read the French press, these assaults on completely innocent people are footnotes. “Incidents.” “Scuffles.” It’s enough to make your blood boil, really, that no one other than Uber executives seems to be particularly offended.

And this is typical, really. Strikes in France are often launched over ludicrous demands, and they’re incredibly disruptive if not downright dangerous. Many people in France will tell you how much they hate the incredibly powerful unions and the strikes they engender. But that’s just how it is. Because strikes are, to many, the essence of French rights, the core of what made French society, at least in the past, an exemplar of workers’ rights against the oppressive corporations.

Meanwhile, in the same week, a man got shot in a Florida movie theater, apparently because he was texting and that got someone really annoyed. The press wrote “man killed over texting in a movie theater,” and the discussion was often about how annoying texting can be. Because guns don’t kill people. Texting in a movie theater… now that kills!

Never mind that in the year since the Sandy Hook school shooting, where more than a dozen 6-year-olds were shot (6 year-olds! come on!), we’ve done exactly nothing as a country to contain gun violence. Stupid fights escalate into shootings. Because Second Amendment! I’m sure that’s what the Founders had in mind when they wanted a “well-regulated militia”: people in movie theaters with guns to settle fights.

Guns are such a deep part of America’s identity that their inherent goodness cannot be challenged. Even if many Americans wish they could change things. It doesn’t happen. It’s too engrained in American culture.

Yes, yes, I know, these two things are not quite the same.

But in a really critical way, they are. We humans make stupid decisions, and I mean really stupid, because some things feel, on principle, like deep parts of our identity. Because at one point in the past, in theory, that thing was really, really important. It’s the insane thing you hold on to because, if you give it up, it feels like you’re giving up a piece of yourself, like you’re renouncing who you really are.

So. What’s your stupid cause you feel you must stick to lest you betray yourself? How is it stopping you from seeing the obvious mistake you’re making? Can you let go of it and accept that yes, you are still the same person? I ask myself that, every now and then.

Because we primates sure are irrational.

there are 3 kinds of crypto

Posted on by

When we use terminology that is too broad, too coarse-grained, we make discussion more difficult. That sounds obvious, but it’s easy to miss in practice. We’ve made this mistake in spades with crypto. Discussing the field as one broad topic is counter-productive and leads to needless bickering.

I see 3 major kinds of crypto: b2c crypto, b2b crypto, and p2p crypto. I suggest that we use this terminology consistently to help guide the discussion. We’ll spend less time talking about differences in our assumptions, and more time building better solutions.

b2c crypto

Business-to-Customer Crypto (b2c) is used to secure the relationship between an organization and a typical user. The user roughly trusts the organization, and the goal of b2c crypto is to enable that trust by keeping attackers out of that relationship. Both the organization and the user want to know that they’re talking to each other and not to an impostor. The organization is usually acting like an honest-but-curious party: they’ll mostly do what they promise. The b2c-crypto relationship is common between Internet service providers (in the broad sense, including Google, Amazon, etc.) and typical Internet users, as well as between employees and their employer’s IT department.

Web-browser SSL is a great example of b2c crypto. Users start with a computer that has at least one web browser with a set of root certs. Users can continue using that browser or download, over SSL secured by those initial root certs, another browser they trust more. Users then trust their preferred browser’s security indicators when they shop on Amazon or read their Gmail.

A critical feature of b2c crypto is that users don’t ever manage crypto keys. At best they manage a password, and even then they’re generally able to reset it. Users make trust decisions based on brands and hopefully clear security indicators: I want a Mac, I want to use Firefox, and I want to shop on Amazon but only when I see the green lock icon.

b2b crypto

Business-to-Business (b2b) crypto is used to secure the relationship between organizations, two or more at a time. There are two defining characteristics of b2b crypto:

  • all participants are expected to manage crypto keys
  • end-users are generally not involved or burdened

DKIM is a good example of b2b crypto. Organizations sign their outgoing emails and verify signatures on incoming emails. Spam and phishing are reduced, and end-users see only the positive result without being involved in the process. Organizations must maintain secret cryptographic keys for signing those emails and know how to publish their public keys (usually in DNS) to inform other organizations.

OAuth qualifies as b2b crypto. Consumers and Producers of Web APIs establish shared secret credentials and use them to secure API calls between organizations.

Another good example is SSL certificate issuance. Both the web site seeking a certificate and the Certificate Authority have to generate and secure secret keys. The complexity of the certification process is mostly hidden from end-users.

p2p crypto

Peer-to-Peer (p2p) crypto is used to secure communication between two or more crypto-savvy individuals. The defining characteristic of p2p crypto is that the crypto-savvy individuals trust no one by default. They tend to run code locally, manage crypto keys, and assume all intermediaries are active attackers.

PGP is a great example of p2p crypto. Everyone generates a keypair, and by default no one trusts anyone else. Emails are encrypted and signed, and if you lose your secret key, you’re out of luck.

so how does this help?

This naming scheme provides a clear shorthand for delineating crypto solutions. Is your wonderful crypto solution targeted at the general public? Then it’s probably a combination of b2c crypto for users and b2b crypto for organizations that support them. Are you building a specialized communications platform for journalists in war zones? Then it’s probably p2p crypto.

The implementation techniques we use for various kinds of crypto differ. So when some folks write that Javascript Crypto is considered harmful, I can easily respond “yes, dynamically-loaded Javascript is a poor approach for p2p crypto, but it’s great for b2c crypto.” In fact, when you look closely at a similar criticism of Javascript crypto from Tony Arcieri, you see this same differentiation, only with much more verbiage because we don’t have clear terminology:

Before I keep talking about where in-browser cryptography is inappropriate, let me talk about where I think it might work: I think it has great potential uses for encrypting messages sent between a user and the web site they are accessing. For example, my former employer LivingSocial used in-browser crypto to encrypt credit card numbers in-browser with their payment processor’s public key before sending them over the wire (via an HTTPS connection which effectively double-encrypted them). This provided end-to-end encryption between a user’s browser and the LivingSocial’s upstream payment gateway, even after HTTPS has been terminated by LivingSocial (i.e. all cardholder data seen by LivingSocial was encrypted).

This is a very good thing. It’s the kind of defense that can prevent the likes of the attack against Target’s 40M customers last month. And that’s exactly the point of b2c crypto.

most users can’t manage crypto keys

I use the term p2p crypto because I like to think of it as “Pro-to-Pro.” Expecting typical Internet users to engage in p2p crypto is, in my opinion, a pipe dream: typical users can’t manage secret crypto keys, so typical users must rely on organizations to do that for them. That’s why successful general-public crypto is a combination of b2c crypto between individuals and the organizations they choose to trust, and b2b crypto across organizations. More expertise and care is expected of the organizations, little is expected of individual users, and some trust is assumed between a user and the organizations they choose.

You don’t have to agree with me on this point to agree with the nomenclature. If you’re interested in protocols where individuals manage their own secret keys and don’t trust intermediaries, you’re interested in p2p crypto. I happen to think that p2p crypto is applicable only to some users and some specific situations.

on cooking turkey and solving problems

Posted on by

On Thursday, my wife and I hosted our 10th Thanksgiving. We both enjoy cooking and baking, though we remain clearly amateurs and tend to make it up as we go along. There was that one time we realized, the night before Thanksgiving, that a frozen 15-pound turkey requires 3 days to defrost in the fridge. I stayed up most of the night, soaking the bird in the bathtub.

We’ve gotten better over time: she focuses on stuffing and cranberry sauce, me on turkey and dessert, and we collaborate on some kind of sweet potato dish. The stress almost always comes around how long to roast the turkey and whether it’s fully cooked. For the last 4 years, we’ve had the added (but wonderful) complexity of little kids eager to eat. We had great luck once with a high-heat plus start-breast-side-down combination, but we were never able to recreate that success.

This year, I reached out on twitter:

I received this recommendation from a former student and fellow web hacker:

“What in the world is spatchcocking,” I thought. I was ready to try it after this video:

(The New York Times also has a nice take on the technique.)

Roughly, spatchcocking involves cutting out the turkey’s backbone, then breaking open the bird and laying it out flat. One layer of meat, all on the same plane, with the dark meat slightly protecting the white meat, which is what you want since white meat cooks faster. The technique promises shorter cook times, more even cooking, and juicier meat.

Turns out, it’s all absolutely true. Preparation was easy and eminently repeatable, with little risk of screwing things up. The bird cooked in about 2 hours, where typically it would have required 4. The whole turkey cooked at the same speed. The result: amazing fully cooked dark meat, juicy white meat, perfectly crispy skin, and plenty of oven time left for an apple pie and stuffing. Everyone at the table agreed: this was the best turkey I’ve ever cooked by far. Even the little kids ate triple portions.

So what’s the downside? Well, people claim there are two: (a) you can’t stuff the turkey and (b) you can’t present a typical, whole roasted turkey. Instead you’ve got a weird flat thing that indicates you got really angry in the kitchen. Neither of these matters to me, and I’ll go out on a limb and say they should matter little to most people: stuffing a raw turkey significantly increases the risk of food poisoning, and, as it turns out, being forced to carve the turkey before presenting it made serving the meal much easier.

So, first lesson: I will only cook spatchcock turkeys from now on.

And second lesson: even after 10 years of doing something, it’s still possible to find a solution that is faster, simpler, and better, with no real downsides. What’s crazy is that the solution is already out there, used by some, just not widely. Crazier still is that many people know about it, they just refuse to try it because there are “downsides” or the solution is unusual.

But what if the downsides are rhetorical at best? What if it’s really all upside?

I can’t help but link this to software engineering and problem-solving more broadly. There are so many technical solutions we simply accept as necessary and necessarily hard. We fail to search for simpler solutions, even when they already exist. Or if we know about them, we choose to ignore them because they seem too simple, too good to be true. We make up excuses, we make up theoretical downsides.

Why not stick to simple? There’s not necessarily a real tradeoff. Sometimes, even often, you can do faster, simpler, and better. I’m going to work to keep that in mind in everything I do. Kindergarden selection for my kids. Financial planning. And especially software.

Before going complicated, have you tried spatchcocking? The result might just be delicious.

Letter to President Obama on Surveillance and Freedom

Posted on by

Dear President Obama,

My name is Ben Adida. I am 36, married, two kids, working in Silicon Valley as a software engineer with a strong background in security. I’ve worked on the security of voting systems and health systems, on web browsers and payment systems. I enthusiastically voted for you three times: in the 2008 primary and in both presidential elections. When I wrote about my support for your campaign five years ago, I said:

In his campaign, Obama has proposed opening up to the public all bill debates and negotiations with lobbyists, via TV and the Internet. Why? Because he trusts that Americans, when given the tools to see and understand what their legislators are doing, will apply pressure to keep their government honest.

I gushed about how you supported transparency as broadly as possible, to enable better decision making, to empower individuals, and to build a better nation.

Now, I’m no stubborn idealist. I know that change is hard and slow. I know you cannot steer a ship as big as the United States as quickly as some would like. I know tough compromises are the inevitable path to progress.

I also imagine that, once you’re President, the enormity of the threat from those who would attack Americans must be overwhelming. The responsibility you feel, the level of detail you understand, must make prior principles sometimes feel quaint. I cannot imagine what it’s like to be in your shoes.

I also remember that you called on us, your supporters, to stay active, to call you and Congress to task. I want to believe that you asked for this because you knew that your perspective as Commander in Chief would inevitably become skewed. So this is what I’m doing here: I’m calling you to task.

You are failing hard on transparency and oversight when it comes to NSA surveillance. This failure is not the pragmatic compromise of Obamacare, which I strongly support. It is not the sheer difficulty of closing Guantanamo, which I understand. This failure is deep. If you fail to fix it, you will be the President principally responsible for the effective death of the Fourth Amendment and worse.

mass surveillance

The specific topic of concern, to be clear, is mass surveillance. I am not concerned with targeted data requests, based on probable cause and reviewed individually by publicly accountable judges. I can even live with secret data requests, provided they’re very limited, finely targeted, and protect the free-speech rights of service providers like Google and Facebook to release appropriately sanitized data about these requests as often as they’d like.

What I’m concerned about is the broad, dragnet NSA signals intelligence recently revealed by Edward Snowden. This kind of surveillance is a different beast, comparable to routine frisking of every individual simply for walking down the street. It is repulsive to me. It should be repulsive to you, too.

wrong in practice

If you’re a hypochondriac, you might be tempted to ask your doctor for a full body MRI or CT scan to catch health issues before detectable symptoms. Unfortunately, because of two simple probabilistic principles, you’re much worse off if you get the test.

First, it is relatively unlikely that a random person with no symptoms has a serious medical problem, ie the prior probability is low. Second, it is quite possible — not likely, but possible — that a completely benign thing appears potentially dangerous on imaging, ie there is a noticeable chance of false positive. Put those two things together, and you get this mind-bending outcome: if the full-body MRI says you have something to worry about, you actually don’t have anything to worry about. But try convincing yourself of that if you get a scary MRI result.

Mass surveillance to seek out terrorism is basically the same thing: very low prior probability that any given person is a terrorist, quite possible that normal behavior appears suspicious. Mass surveillance means wasting tremendous resources on dead ends. And because we’re human and we make mistakes when given bad data, mass surveillance sometimes means badly hurting innocent people, like Jean-Charles de Menezes.

So what happens when a massively funded effort has frustratingly poor outcomes? You get scope creep: the surveillance apparatus gets redirected to other purposes. The TSA starts overseeing sporting events. The DEA and IRS dip into the NSA dataset. Anti-terrorism laws with far-reaching powers are used to intimidate journalists and their loved ones.

Where does it stop? If we forgo due process for a certain category of investigation which, by design, will see its scope broaden to just about any type of investigation, is there any due process left?

wrong on principle

I can imagine some people, maybe some of your trusted advisors, will say that what I’ve just described is simply a “poor implementation” of surveillance, that the NSA does a much better job. So it’s worth asking: assuming we can perfect a surveillance system with zero false positives, is it then okay to live in a society that implements such surveillance and detects any illegal act?

This has always felt wrong to me, but I couldn’t express a simple, principled, ethical reason for this feeling, until I spoke with a colleague recently who said it better than I ever could:

For society to progress, individuals must be able to experiment very close to the limit of the law and sometimes cross into illegality. A society which perfectly enforces its laws is one that cannot make progress.

What would have become of the civil rights movement if all of its initial transgressions had been perfectly detected and punished? What about gay rights? Women’s rights? Is there even room for civil disobedience?

Though we want our laws to reflect morality, they are, at best, a very rough and sometimes completely broken approximation of morality. Our ability as citizens to occasionally transgress the law is the force that brings our society’s laws closer to our moral ideals. We should reject mass surveillance, even the theoretically perfect kind, with all the strength and fury of a people striving to form a more perfect union.

patriots

Mr. President, you have said that you do not consider Edward Snowden a patriot, and you have not commented on whether he is a whistleblower. I ask you to consider this: if you were an ordinary citizen, living your life as a Law Professor at the University of Chicago, and you found out, through Edward Snowden’s revelations, the scope of the NSA mass surveillance program and the misuse of the accumulated data by the DEA and the IRS, what would you think? Wouldn’t you, like many of us, be thankful that Mr. Snowden risked his life to give we the people this information, so that we may judge for ourselves whether this is the society we want?

And if there is even a possibility that you would feel this way, given that many thousands do, if government insiders believe Snowden to be a traitor while outsiders believe him to be a whisteblower, is that not all the information you need to realize the critical positive role he has played, and the need for the government to change?

the time to do something is now

I still believe that you are, at your core, a unique President who values a government by and for the people. As a continuing supporter of your Presidency, I implore you to look deeply at this issue, to bring in outside experts who are not involved in national security. This issue is critical to our future as a free nation.

Please do what is right so that your daughters and my sons can grow up with the privacy and dignity they deserve, free from surveillance, its inevitable abuses, and its paralyzing force. Our kids, too, will have civil rights battles to fight. They, too, will need the ability to challenge unjust laws. They, too, will need the space to make our country better still.

Please do not rob them of that opportunity.

Sincerely,

Ben Adida