the responsibility we have as software engineers

I had the chance to chat this week with the very awesome Kate Heddleston who mentioned that she’s been thinking a lot about the ethics of being a software engineer, something she just spoke about at PyCon Sweden. It brought me back to a post I wrote a few years ago, where I said:

There’s this continued and surprisingly widespread delusion that technology is somehow neutral, that moral decisions are for other people to make. But that’s just not true. Lessig taught me (and a generation of other technologists) that Code is Law


In 2008, the world turned against bankers, because many profited by exploiting their expertise in a rapidly accelerating field (financial instruments) over others’ ignorance of even basic concepts (adjustable-rate mortgages). How long before we software engineers find our profession in a similar position? How long will we shield ourselves from the responsibility we have, as experts in the field much like experts in any other field, to guide others to make the best decision for them?

Well, I think that time has come.

Everyone uses software, very few people understand it. What seems obvious to a small elite group is completely opaque to the majority of the world. This gap is incredibly hard for us, the software engineering elite, to see. A few examples:

  • The Radiolab Podcast did a wonderful piece – Trust Engineers – where they explored the case of Facebook running experiments on its newsfeed. For non-engineers, there’s an incredible feeling of breached trust upon realizing that a set of flesh-and-blood humans have that much control over the algorithm that feeds them daily information. (And, for that matter, to most researchers used to interacting with an IRB, there’s complete shock at what Facebook did.) For most engineers, including a number of very good and ethical people at Facebook, it’s surprising that this is even an issue.
  • A couple of years ago, a friend of a friend – who happens to be a world-renowned physician and research scientist – asked me: “Ben, can the system administrators at work read my email? Even if they don’t have my password?” The answer is yes and yes. This is obvious to us engineers, so much so that we don’t even think twice about it. To a non-engineer, even an incredibly smart person, this is absolutely non-obvious.
  • A close friend, another very smart person, was discussing something with his young child recently, and I overheard “if you don’t know, ask the computer, the computer knows and it’s always right.” Where do I begin?

We, software engineers, have superpowers most people don’t remotely understand. The trust society places in us is growing so rapidly that the only thing that looks even remotely similar is the trust placed in doctors. Except, most people have a pretty good idea of the trust they’re placing in their doctor, while they have almost no idea that every time they install an app, enter some personal data, or share a private thought in a private electronic conversation, they’re trusting a set of software engineers who have very little in the form of ethical guidelines.

Where’s our Hippocratic Oath, our “First, Do No Harm?”

I try very hard to think about this in my own work, and I try to share this sense of duty with every engineer I mentor and interact with. Still, I don’t have a good answer to the core question. Yet it feels increasingly urgent and important for us to figure this out.


This week, I joined Clever as VP Engineering. Clever makes K-12 education software vastly more efficient and effective by simplifying how students and teachers log in. It’s this simple: imagine if you could give teachers and students 10-15 minutes back in every single class. That’s 30-40% more time for actual teaching and learning. That’s what Clever does today, with much more in the works.

I’m incredibly excited about this new adventure, and I want to gush a bit.


My priorities in work are:

  1. people
  2. mission
  3. product

People – strong contributors who know how to work in teams that accomplish more than the sum of their parts – are my top priority. A mission and a product, no matter how good, survive contact with the real world only if backed by strong, honest team players.

A Mission – a clear goal to make a positive, socially beneficial dent in the universe – is my close second priority. Products come and go, pivots happen, but a strong mission gives an organization an invariant, a true north when the storm hits.

And finally, a strong Product. Because once you have great people, and once you have a clear and stable mission, you still need a compelling product to deliver that mission to the market. The product is the last mile of your impact on the world.


Clever meets all three of my priorities in spades.

I am blown away by the quality of people at Clever, starting with co-founders Tyler, Dan, and Rafael, and including every engineer, business-development partner, school experience advocate, recruiter, etc. Clever team members have oodles of education experience, with ex-teachers, ex-DOE, and ex-school-technologists joining hands to build the next-generation education platform. And it’s not just individual quality and skills, as Clever has also built a strong team culture, one that mirrors the values of the education products we want to see. Every team member is always a student, and Clever is a group effort.

The Clever mission is clear and deeply impactful: to save teacher and students time, all the while protecting student privacy and preserving data access controls enforced by schools.

And finally, the Clever product is catching on like wildfire. There remains a mountain of work because there’s a mountain of opportunity to make education software better. But the market is already speaking, and Clever has struck a a very clear chord.

Join Us

We’re a growing team of 20’ish engineers, passionate about applying technology to making K-12 education far more effective. We’re committing to building a diverse team, because teams with a variety of life experiences build better products, and because darnit it’s the right thing to do.

Do you like working on code that makes the world a better place? Do you want to learn from and teach your teammates every day? Do you want to work on hard technical problems not just because they’re hard, but because they’re hard and impactful?

Clever’s a unique place and a unique opportunity. Use your powers for good. Send me a note.

(your) information wants to be free – obamacare edition

My friends over at EFF just revealed that is sending personal data to dozens of tracking sites:

It’s especially troubling that the U.S. government is sending personal information to commercial companies on a website that’s touted as the place for people to obtain health care coverage. Even more troubling is the potential for companies like Doubleclick, Google, Twitter, Yahoo, and others to associate this data with a person’s actual identity.

The referenced AP story uses even more damning language:

The government’s health insurance website is quietly sending consumers’ personal data to private companies that specialize in advertising and analyzing Internet data for performance and marketing, The Associated Press has learned.

Sounds pretty bad, right? Except it’s almost certainly not what it sounds like. It’s almost certainly a simple mistake.

How could this be a mistake, you ask? Here’s what almost certainly happened:

  1. Someone at wanted to analyze patterns of usage of the site. This is often done to optimize sites for better usage. So they added a tracker to their page for MixPanel, for Optimizely, for Google Analytics, and a couple of other sites that help you understand how people use your site. In all likelihood, different departments added different trackers, each for their own purposes, almost certainly with good intentions of making the web site more usable.
  2. Meanwhile, someone else responsible for social media of added a “Tweet This” button, and someone else added a YouTube video. Once again, these come in the form of widgets, often snippets of JavaScript code, that load resources from their respective home base.
  3. Separately, someone built the web form that lets you enter basic information about yourself so you can find a health plan. That information is, in large part, fairly personal: your age, your zip code, whether or not you smoke, etc. And for some reason, almost certainly completely random, they used a web form with an action type of GET.
  4. Here’s the first mildly technical point. When you submit a GET form, the data in the form is appended to the URL, like so:

    Not a big deal, since that data is going to anyways.

  5. And now for the second mildly technical point. For tracking purposes, trackers often blindly copy the current URL and send it to their homebase, so that the trackers can tell you users spent 5s on this page, then 10s on that page, etc. In addition, when your browser requests an embedded YouTube video, or an embedded tracker, it sends the current URL as part of the request in a so-called Referrer field.
  6. Put those two technical points together, and boom: a web site that collects personal information with GET forms and uses third-party tracking widgets tends to send form data to those third parties.

This is extremely common. Many web sites with sufficiently large engineering teams have no idea how many trackers they’ve embedded. It’s typical for a web site to move from one site analysis tool to another and to forget to remove the first tracking widget in the process. When the Wall Street Journal reported on these issues a couple of years ago with their fantastic What They Know series, they forgot to mention that their own page has a half-dozen trackers embedded.

I’ve said it before, and I’ll say it again: unfortunately, your information wants to be free. My favorite analogy remains:

when building a skyscraper, workers are constantly fighting gravity. One moment of inattention, and a steel beam can fall from the 50th floor, turning a small oversight into a tragedy. The same goes for software systems and data breaches. The natural state of data is to be copied, logged, transmitted, stored, and stored again. It takes constant fighting and vigilance to prevent that breach. It takes privacy and security engineering.

So, am I letting off the hook? Not at all, they should have done their due diligence and done a more thorough privacy audit. And using GET forms is particularly sloppy, since it leads to data sprayed all over the place in logs, referrers, etc.

But was this a deliberate attempt at sharing private data with private companies? Not a chance. The press should do a better job of reporting this stuff. And, to my wonderful friends at EFF, this is a gentle nudge to say: so should you. It’s important to differentiate between negligence and malice, to not spread fear, uncertainty, and doubt, even when it’s issues we care about.

The good news is that has already responded by (a) reducing their number of trackers significantly and (b) submitting form data using XMLHttpRequest or POST. The bad news is how many people now actually believe that this was intentional, conspiratorial data selling. If that was’s intentions, there are much sneakier ways of doing that without getting caught so easily.

Oh, and if you want to understand more about trackers and block them as you surf the web, try the very excellent Ghostery extension for your browser.

managing photos and videos

This holiday, I finally spent time digging into how I manage photos and videos. With 2 young kids and some remote family and friends, this requires a good bit of thinking and planning. I know I’m not the only one, so I figured documenting where I landed might be useful to others.

I started with Dave Liggat’s Robust Photo Workflow, and found much of it resonates with my needs. Here’s where I landed:

  1. I take photos with a DSLR and two phones. My wife takes photos with her phone. We both take videos with our phones. We use Dropbox/Carousel auto-upload, which works just fine on both iOS and Android. For the DSLR, I manually load photos over USB.
  2. All photos and videos are now available on my desktop Mac (via USB or Dropbox). When I’m ready to review/edit photos, I drag and drop the batch into an all-photos/ directory I keep within my Dropbox.
  3. Hazel automatically categorizes photos and videos into subdirectories of the form 2015/01/. It’s really kind of awesome.
  4. all-photos and all-videos are thus simple date-classified folders of all of my photos and videos. They’re backed up locally using Time Machine. They’re backed up to the network using Dropbox. I can imagine eventually snapshotting this to Amazon S3/Glacier, but right now that doesn’t feel too urgent.
  5. I use Lightroom5 as an editor only, so if I blow away my Lightroom proprietary catalog info, it’s not that big a deal. To do this, I tell Lightroom to look at photos in all-photos without moving/copying them. After I’ve added a bunch of photos to the all-photos directory by drag-and-drop, I tell Lightroom to synchronize its catalog with the source folder, which takes a few seconds and gives me a screen with my latest imported photos and videos. I can then edit photos, reject them if they’re bad, and write back JPG/XMP data to each photo’s originating directory using Lightroom export. Dropbox backs those up automatically. To remove bad photos (blurry, etc.), I flag them as “rejected” in Lightroom using the X key, and when I’m done I command-delete, which gives me the option of removing the files from disk, too. I do this only for clear rejects, and it makes my mild OCD happy since I know I am not keeping totally useless files around, and the overhead of deleting photos is low. I could also delete photos easily using the Dropbox UI, which is pretty good, and then re-synchronize in Lightroom.
  6. I can then use Carousel (or Dropbox) on any mobile device to browse through all of my photos. It’s surprisingly good at dealing with large photo libraries (I have 20K) and large photos (I have a bunch of 13MP photos). As in, really, really good, even on a puny phone. Better than anything else I’ve seen.
  7. I’ve been using Flickr for years for private photo sharing, and Lightroom is really good at exporting to Flickr. That said, at this point I’m thinking of moving to Dropbox/Carousel based sharing. I can easily bundle photos & videos into albums on Dropbox, whereas videos are still limited on Flickr. Carousel conversations around a few photos are great with family. The only bummer is that Carousel and Dropbox have some mutually exclusive features: albums on Dropbox, conversations on Carousel. I suspect Dropbox will fix that in the next year.
  8. What I’d love to see:
    • unified photo features in Dropbox and Carousel
    • export Dropbox albums as directories of symlinks in my Dropbox folder, and export Carousel conversations in some other file-based way, too.
    • Lightroom export compatibility with Dropbox/Carousel albums.

I’m super happy with this new process: one funnel, easy, low overhead, and a very solid long-term photo storage solution. I’m only relying on RAW/JPG files and directories of said files to be readable for the long term, and that seems pretty safe. Lightroom is awesome, but I could replace it with a different tool if I needed to.

One more thing: if you’re going to use Dropbox to store all of your photos, make sure you pick a strong password and set up 2-factor authentication.

Power & Accountability

So there’s this hot new app called Secret. The app is really clever: it prompts you to share secrets, and it sends those secrets to your social circle. It doesn’t identify you directly to your friends. Instead, it tells readers that this secret was written by one of their friends without identifying which one. The popularity of the app appears to be off the charts, with significant venture-capital investment in a short period of time. There are amazing stories of people seeking out emotional support on Secret, and awful stories of bullying that have caused significant uproar. Secret has recently released features aimed at curbing bullying.

My sense is that the commentary to date is missing the mark. There’s talk of the danger of anonymous speech. Even the founders of Secret talk about their app like it’s anonymous speech:

“Anonymity is a really powerful thing, and with that power comes great responsibility. Figuring out these issues is the key to our long-term success, but it’s a hard, hard problem and we are doing the best we can.”

And this is certainly true: we’ve known for a while that anonymous speech can reveal the worst in people. But that’s not what we’re dealing with here. Posts on Secret are not anonymous. Posts on Secret are guaranteed to be authored by one of your friends. That guarantee is enabled and relayed by the Secret platform. That’s a very different beast than anonymity.

In general, if you seek good behavior, Power and Accountability need to be connected: the more Power you give someone, the more you hold them Accountable. Anonymity can be dangerous because it removes Accountability. That said, anonymity also removes some Power: if you’re not signing your name to your statement, it carries less weight. With Secret, Accountability is absent, just like with anonymous speech, but the power of identified speech remains in full force. That leads to amazing positive experiences: people can share thoughts of suicide with friends who can help, all under the cloak of group-anonymity that is both protecting and empowering. And it leads to disastrous power granted to bullies attacking their victims with the full force of speaking with authority – the bully is one of their friends! – while carrying zero accountability. That kind of power is likely to produce more bullies, too.

This is so much more potent that anonymity. And if this fascinating experiment is to do more good than harm, it will need to seriously push the envelope on systems for Accountability that are on par with the power Secret grants.

Here’s a free idea, straight out of crypto land. In cryptographic protocols that combine a need for good behavior with privacy/anonymity protections, there is often a trigger where bad behavior removes the anonymity shield. What if Secret revealed the identity of those users found to be in repeated violation of a code of good behavior? Would the threat of potential shame keep people in line, leaving the good uses intact while disincentivizing the destructive ones?

where the system eats itself

Larry Lessig just launched, the SuperPAC to end all SuperPACs. The idea is disarmingly simple: since SuperPACs funded by billionaires are corrupting politics, let’s crowd-source a SuperPAC funded by individuals, which will then work to put in power officials who answer to the people, maybe by undoing the whole SuperPAC insanity. Use a SuperPAC to kill all SuperPACs.

This is a fascinating pattern that we’ve actually seen before. And it makes me very, very happy, because it is the ultimate policy hack.

Take the GPL or the Apache License, two significant software licenses that make possible open-source and thus much of today’s software. These licenses, especially the GPL, enforce certain constraints on how source code can be used. If you take GPL source code, modify it, and redistribute it, you have to provide the source code to your modifications, too. If you don’t… you lose your license on the code to begin with, and now you’re guilty of copyright violation because you redistributed code without permission.

In other words, many open-source licenses work only because they leverage strong copyright law. The same goes for Creative Commons: you can freely license your work while requiring that people give you credit for it, or, if you prefer, only if they use it for non-commercial purposes. The only reason you can add those constraints is because Creative Commons is layered on top of Copyright. Some people believe Copyright Law has overreached. Those same people are using that Copyright overreach as a foundation for a stronger Commons.

The GPL, Creative Commons, and now MayOne basically use the overreach of the system against itself.

I find this idea – that an unnaturally strong system of rules can be counter-balanced by using the system against itself – fascinating and kind of awesome. Are there other areas where this can be applied?

But before I digress: go pledge to Let’s make SuperPACs eat themselves.

Obama lets NTSB exploit widespread tire vulnerabilities

Stepping into a heated debate within the nation’s transportation safety agencies, President Obama has decided that when the National Transportation Safety Board (NTSB) discovers major flaws in transportation equipment, it should — in most circumstances — reveal them to assure that they will be fixed, rather than keep mum so that the flaws can be used in espionage, senior administration officials said Saturday.

But Mr. Obama carved a broad exception for “a clear national security or law enforcement need,” the officials said, a loophole that is likely to allow the NTSB to continue to exploit safety flaws to apprehend terrorist suspects as they go about their daily routines such as driving.

On Friday, the White House denied that it had any prior knowledge of the AirBleed defect, a newly discovered safety vulnerability in commonly used Michelin tires that often leads to spontaneous tire explosion when driving for prolonged periods of time at exactly 44mph. This flaw has led many manufacturers to urgently recall cars and many drivers to behave erratically in 45mph speed zones. The White House statement said that when such flaws are discovered, there is now a “bias” in the government to share that knowledge with car and tire manufacturers, so a remedy can be created and distributed to industry and consumers.

Sources indicate that some senior officials had urged the NTSB to get out of the business of weakening commercial transportation systems or trying to build in “trapdoor failures” that would make it far easier for the agency to intercept suspected terrorists. These officials concluded that the practice would undercut trust in the American auto industry. In recent months, Detroit has urged the United States to abandon such practices, while Germany and Japan, among other nations, have said they were considering pulling all auto production facilities back to their own countries.

Not surprisingly, officials at the NTSB and at its military partner, the United States Special Operations Command, warned that giving up the capability to exploit undisclosed safety flaws in widespread commercial equipment would amount to “unilateral disarmament” — a phrase taken from the battles over whether and how far to cut America’s nuclear arsenal.

When interviewed at his home in Maryland, John Smith, head of the NTSB in the 1980s, appeared incredulous: “Do you mean to tell me that the American Government knew about widespread life-threatening safety issues in our cars and chose not to disclose those findings to its citizens, just in case this weakness could be used to go after a presumed terrorist? I don’t believe it. The US government serves its people first, and never in a conspiracy-theorist’s wildest dreams would they engage in such despicable behavior.”

[a light parody of this New York Times Article.]

to Brendan and Mozilla

I was in the middle of writing a blog post about the controversy surrounding Mozilla when my Twitter feed exploded with the news that Brendan Eich stepped down from his new appointment as Mozilla CEO. So this is a different post. Also, this is not a post about Prop8 (which I abhored) or gay marriage (which I consider a basic civil right.)

to Brendan

There is little love lost between me and Brendan. We have different styles, and I butted heads with him in tech discussions on a regular basis while I was at Mozilla. He was, at times, infuriating. To be honest, he drove me up the wall.

Yet through all of these disagreements and conflicts, one thing was never in doubt: Brendan has the Mozilla mission of inclusiveness, openness, and freedom engrained in his heart. This is a man who has tirelessly worked in the open for close to 20 years, giving up riches at other companies so he could create an open playground for the world to use, a counter-balance to corporate interests on the Web, and opportunity for billions, even if they never know his name.

So Brendan, thank you, for everything you’ve done for the Open Web and for Humanity. The world is in your debt. You can take the man out of Mozilla, but you can never take the Mozilla mission out of the man.

to my friends at Mozilla

Some people love Mozilla for its mission, but much of the world doesn’t get it. People use Firefox because they like it; others use a different browser because they like that one better. I don’t think that will ever really change. It’s easy to become very depressed about this, to think “how can people be so harsh on Mozilla when we’ve been such good citizens?” or “can’t people see that Firefox is the only browser built by a non-profit?” People don’t care as much as they should. One bad PR cycle might be enough for them to switch browsers.

I want to suggest that this apathy can be empowering: don’t count on others groking the mission. Follow the Mozilla mission on your own terms, because you know it’s the right thing to do. Do the right thing because it is the right thing.

Keep doing the right thing, friends, and be excellent to each other.

when selfish acts become altruistic

My first open-source contribution was in 1998, when a ragtag bunch of web hackers and I published the first version of one of the first web application toolkits. In 2000, after I’d left the original project, a few other hackers and I “forked” that codebase to make it work on an open-source database, meaning we took the code, copied it to a different repository, and took it down a different path than that envisioned by its maintainers.

That has always been the beauty of open-source: if you don’t like the direction of the software, you can always fork it. The dirty little secret of open-source at the time was that this was much more an abstract threat than a common occurrence. Forking was almost always a bad word, a huge undertaking, and it happened very rarely. One central reason for this was that forking was almost always a one-way street: once you forked, it became very difficult to share improvements between forks.

So, in theory, thanks to open-source licenses, forking was always possible. In practice, it was a huge undertaking, one with deep consequences, and usually a sign of something gone awry.

Then in 2008 came Github. Github made hosting source code super easy. Github made forking other people’s code super easy, too. And thanks to the underlying Git system, Github made merging just as easy as forking. Within a few weeks, more than 6200 projects on Github had been forked at least once.

Forking became the optimistic way to contribute to a project: fork it, modify it for your own use, and suggest the change back to the original author. If the original author accepts your change, great, you can forget about your fork and go back to using the original code. If the author rejects your change, you can keep using your forked version, occasionally merging in upstream changes from the original author.

So forking became a good thing, a sign of interest in your project. People wore “Fork Me” t-shirts. And it was all done for years with little attention paid to the specifics of the licenses underlying it all. It was just assumed that, if you made a Github project public, you allowed open-source style forking.

In many ways, Github made real what open-source licenses mostly theorized. Standing on the shoulders of giants, contributing just a little tweak very easily, taking a different direction when you need to, etc. All the beauty of a vast open repository of code you can pick from and contribute to exists in Github.

And somehow, this amazing sharing ecosystem is based on purely selfish incentives. I need to host my code. I don’t like paying for things unless I need to, so I’ll make it public, because Github makes me pay for private repositories. I sometimes need to change other people’s code, and there’s a button for that. If someone changes my code I’d like to benefit from it, and there’s a button for that, too.

Like the Back-to-the-Future deLorean that runs on garbage, Github produces sharing behavior by consuming selfish acts.

I’d like to see many other Githubs. And I know startups are pitching Github-like projects to VCs daily. But it’s not just about a place to host and remix stuff. The magic of Github is that it generated a sharing ecosystem out of selfish incentives. Not sharing and selfishness side by side. Not questionable sharing of private content for the sake of virality. Sharing as a true side-effect of selfish behavior.

That’s not easy. And if it can be done in fields other than source code… I really like what that could do to benefit human knowledge and progress.

the French like their strikes like Americans like their guns

This week, French taxis went on strike because the government passed a law that made Uber and other modern chauffeur equivalents artificially less competitive… but apparently not sufficiently less competitive, and that was a tragedy that only a massive strike could rectify. Then when people jumped into Uber cars because, hey, there were no cabs, those cars were attacked, leaving some passengers bleeding and stranded on the side of the road.

If you go read the French press, these assaults on completely innocent people are footnotes. “Incidents.” “Scuffles.” It’s enough to make your blood boil, really, that no one other than Uber executives seems to be particularly offended.

And this is typical, really. Strikes in France are often launched over ludicrous demands, and they’re incredibly disruptive if not downright dangerous. Many people in France will tell you how much they hate the incredibly powerful unions and the strikes they engender. But that’s just how it is. Because strikes are, to many, the essence of French rights, the core of what made French society, at least in the past, an exemplar of workers’ rights against the oppressive corporations.

Meanwhile, in the same week, a man got shot in a Florida movie theater, apparently because he was texting and that got someone really annoyed. The press wrote “man killed over texting in a movie theater,” and the discussion was often about how annoying texting can be. Because guns don’t kill people. Texting in a movie theater… now that kills!

Never mind that in the year since the Sandy Hook school shooting, where more than a dozen 6-year-olds were shot (6 year-olds! come on!), we’ve done exactly nothing as a country to contain gun violence. Stupid fights escalate into shootings. Because Second Amendment! I’m sure that’s what the Founders had in mind when they wanted a “well-regulated militia”: people in movie theaters with guns to settle fights.

Guns are such a deep part of America’s identity that their inherent goodness cannot be challenged. Even if many Americans wish they could change things. It doesn’t happen. It’s too engrained in American culture.

Yes, yes, I know, these two things are not quite the same.

But in a really critical way, they are. We humans make stupid decisions, and I mean really stupid, because some things feel, on principle, like deep parts of our identity. Because at one point in the past, in theory, that thing was really, really important. It’s the insane thing you hold on to because, if you give it up, it feels like you’re giving up a piece of yourself, like you’re renouncing who you really are.

So. What’s your stupid cause you feel you must stick to lest you betray yourself? How is it stopping you from seeing the obvious mistake you’re making? Can you let go of it and accept that yes, you are still the same person? I ask myself that, every now and then.

Because we primates sure are irrational.