I had the chance to chat this week with the very awesome Kate Heddleston who mentioned that she’s been thinking a lot about the ethics of being a software engineer, something she just spoke about at PyCon Sweden. It brought me back to a post I wrote a few years ago, where I said:
There’s this continued and surprisingly widespread delusion that technology is somehow neutral, that moral decisions are for other people to make. But that’s just not true. Lessig taught me (and a generation of other technologists) that Code is Law
[…]
In 2008, the world turned against bankers, because many profited by exploiting their expertise in a rapidly accelerating field (financial instruments) over others’ ignorance of even basic concepts (adjustable-rate mortgages). How long before we software engineers find our profession in a similar position? How long will we shield ourselves from the responsibility we have, as experts in the field much like experts in any other field, to guide others to make the best decision for them?
Well, I think that time has come.
Everyone uses software, very few people understand it. What seems obvious to a small elite group is completely opaque to the majority of the world. This gap is incredibly hard for us, the software engineering elite, to see. A few examples:
- The Radiolab Podcast did a wonderful piece – Trust Engineers – where they explored the case of Facebook running experiments on its newsfeed. For non-engineers, there’s an incredible feeling of breached trust upon realizing that a set of flesh-and-blood humans have that much control over the algorithm that feeds them daily information. (And, for that matter, to most researchers used to interacting with an IRB, there’s complete shock at what Facebook did.) For most engineers, including a number of very good and ethical people at Facebook, it’s surprising that this is even an issue.
- A couple of years ago, a friend of a friend – who happens to be a world-renowned physician and research scientist – asked me: “Ben, can the system administrators at work read my email? Even if they don’t have my password?” The answer is yes and yes. This is obvious to us engineers, so much so that we don’t even think twice about it. To a non-engineer, even an incredibly smart person, this is absolutely non-obvious.
- A close friend, another very smart person, was discussing something with his young child recently, and I overheard “if you don’t know, ask the computer, the computer knows and it’s always right.” Where do I begin?
We, software engineers, have superpowers most people don’t remotely understand. The trust society places in us is growing so rapidly that the only thing that looks even remotely similar is the trust placed in doctors. Except, most people have a pretty good idea of the trust they’re placing in their doctor, while they have almost no idea that every time they install an app, enter some personal data, or share a private thought in a private electronic conversation, they’re trusting a set of software engineers who have very little in the form of ethical guidelines.
Where’s our Hippocratic Oath, our “First, Do No Harm?”
I try very hard to think about this in my own work, and I try to share this sense of duty with every engineer I mentor and interact with. Still, I don’t have a good answer to the core question. Yet it feels increasingly urgent and important for us to figure this out.
Benlog 
Comments
16 responses to “the responsibility we have as software engineers”
[…] Ben Adida (via Ole Begemann and Eryn Wells): […]
Beautifully written.
Ethics is an underestimated issue in computer science. For too long it has been developed only as a motivation for the free software movement and we have neglected it.
Thinking about ethics is a sign of a maturing field.
[…] Source: the responsibility we have as software engineers | Benlog […]
What are your suggestions on which and how to be ethical? That’s the hard part no?
[…] https://benlog.com/2015/05/23/the-responsibility-we-have-as-software-engineers/ […]
In the medical context, even with their Hippocratic oath, there’s great concern among certain groups about how subtle ways in which professionals and the industry in general can harm their patients. They even published a book about this: How do we harm”. It describes the challenges that ordinary Americans experience while trying to get quality health care.
I’ve been concerned about our social responsibility not only for not harming anyone, but for contributing in a positive way to society, although I think that’s way too much to ask by now, since I happen to experience a similar situation as you do: almost nobody wants to listen.
Ethics is for everybody. Everyone makes decisions that can have an impact on other people’s lifes, we just tend to downplay the effects of our actions. And who knows who will be the next in position to press a button that will change everything for everybody? Ethics is significant a part of our life and we should strive to make everyone more aware of this reality.
[…] 原文地址（original source）：https://benlog.com/2015/05/23/the-responsibility-we-have-as-software-engineers/ […]
Are you familiar with http://en.m.wikipedia.org/wiki/Ritual_of_the_Calling_of_an_Engineer?
The german Gesellschaft für Informatik (https://en.wikipedia.org/wiki/Gesellschaft_f%C3%BCr_Informatik) has a well rounded set of guidlines of professional ethics for computer science out: https://www.gi.de/?id=120
For teaching / discussing professional ethics I can recommend “Gewissensbisse”, a collection of very real, very current cases to discuss with students http://www.amazon.de/Gewissensbisse-Informatik-Biometrie-Datenschutz-geistiges/dp/383761221X/
New cases are also published on http://gewissensbits.gi.de/
And a US source: http://plato.stanford.edu/entries/ethics-computer/#ProRes
There is a code of conduct for members of the BCS and probably for similar bodies around the world, doesn’t that outline the ethics by which any member should practice?
I’ve heard people suggest professional licensure for software engineers, but I immediately pointed out that licensure would likely just push development work to lower cost locales with no licensure. People need to be aware of how dependent they are on the goodwill and integrity of knowledgeable IT professionals, but what recourse do they have to protect themselves?
The problem for software engineers is that they are overwhelmingly cogs in large, corporate and intrinsically unethical machines. Doctors used to have a greater degree of direct interaction and control over their practices, so the Hippocratic Oath meant something. Today, between HMOs and group insurance and in-/out-of-network and for-profit hospitals and consolidation in the healthcare industry, doctors find their hands tied more and more and their Oath challenged.
That challenged place is where software engineers virtually always have been. Unless the ways that software is financed, marketed and sold or rented are radically changed, I doubt that they will ever have the leverage to make their ethics count.
Reblogged this on ARHAM and commented:
“We, software engineers, have superpowers most people don’t remotely understand. The trust society places in us is growing so rapidly that the only thing that looks even remotely similar is the trust placed in doctors. Except, most people have a pretty good idea of the trust they’re placing in their doctor, while they have almost no idea that every time they install an app, enter some personal data, or share a private thought in a private electronic conversation, they’re trusting a set of software engineers who have very little in the form of ethical guidelines.”
[…] La seconda lettura è un articolo che spsota le responsabilità su chi deve averle (ovvero su chi progetta gli strumenti, non su chi li usa). E che -giustamente, a mio parere- ipotizza un codice ippocratico per gli sviluppatori: The Responsibility We Have as Software Engineers — Where’s our Hippocratic Oath, our “Firs… […]