The Thorny Ethics of Computer Programming

A surprisingly wide-ranging discussion broke out last month among computer programmers to which ethical lines they just won’t cross. Coders have been sharing their own horror stories in highly-trafficked discussions on Medium, Hacker News, and Reddit. Collectively all the stories offered a glimpse into the unexpected challenges lurking in the developer world today.

This high-minded viral discussion started in mid-November when Bill Sourour shared an essay on Medium called “The Code I’m Still Ashamed Of.” Sourour is a teacher (and consultant), and he’s preparing to launch a website called DevMastery.com. “I’ve made a great living developing and architecting enterprise-class systems for nearly 20 years and I would like to find a way to give back,” he writes on his site.”  But maybe that’s what prompted him to reflect on one ethical question he’s faced.

Sixteen years ago Sourour built a website at his client’s request promoting a prescription drug while masquerading as an online quiz — despite Canada’s strict laws on medical marketing. No matter how people answered, the site always recommended the same drug. Right before the happy client took the team out for a steak dinner, a coworker sent him a news story about a teenaged girl who’d committed suicide after taking their drug — Its side effects had included severe depression and suicidal thoughts.

Litigation over its side effects continue to this day, Sourour writes, noting he’s still haunted by the experience. “It’s easy to make an argument that I had no part in it at all. Still, I’ve never felt okay about writing that code.”

Now he’s urging other programmers to consider the effects of their own code. “As developers, we are often one of the last lines of defense against potentially dangerous and unethical practices.” And he sees this as something worth considering as we look towards a future with self-driving cars and AIs that can diagnose diseases. “The more software continues to take over every aspect of our lives, the more important it will be for us to take a stand and ensure that our ethics are ever-present in our code.”

But what’s been fascinating is to see an online discussion break out about just how high the stakes can be — and all of the ethics-related resources that were shared along the way.

Several of the commenters cited Uncle Bob — Robert Cecil Martin, co-author of the Agile Manifesto — and Sourour says his article was inspired by one of Bob’s recent talks last December in Denmark. “There are no ethics,” Bob tells his audience. “We have never defined ethics to our job… And this is a problem because we rule the world.

“We didn’t know this, we didn’t want it, but it has become true… Other people think they rule the world, but then they write the rules down, and they give them to us…”

Another commenter shared a video by Alan Cooper, “the father of Visual Basic” to make the point that developers and UX designer “have more power than you think.” Someone also shared a video of a 2014 presentation by designer Mike Monteiro with the ominous title “How Designers Destroyed the World.” And another commenter posted a link to a YouTube video describing”the impact of Great Design” a talk from the NYC Digital Product Design meetup.

From Merry Prankster to Infrastructure Gnome

The shared resources came in many forms. Someone from an IT management consulting firm shared a linked to the new site SoftwareEthics.org, as well as a recent blog post highlighting its importance. Another commenter even shared the link to a 2012 essay by open source advocate Eric Raymond describing the transitioning role of programmers “from merry prankster to infrastructure gnome.”

“[H]aving created the future, we have to maintain it. And, as the sinews of civilization become ever more dependent on the Internet and software-intensive communications devices, that responsibility gets more serious every year,” Raymond wrote.

Some commenters also shared their own personal experiences. J. H. Stephenson, a college instructor and IT/Information Marketing Consultant said that when he had written his doctoral dissertation on ethics and morality in software development, he discovered something disturbing. He’d surveyed developers, several of whom reported they’d never encountered ethical issues, while later sharing examples of it.

“It does raise a lot of questions about just how much developers understand about the impact that ethical choices have on the applications they develop and the people who use them, or are used by them,” Stephenson wrote.

The article also gave coders a chance to share their own horror stories. For example, one Medium reader remembered working on a Medicare billing system in the 1990s, and discovering that it was designed to automatically (and illegally) pad all of its bills. Instead of looking away, they’d assembled (and anonymously submitted) evidence to the Medicare fraud division. The end result? “Client A got audited, audited, and audited some more, then fined. Action on the second site took over a year to commence. Client B got fined and shut down. So that was cool.”

And Rob Carr — who’s been running his software company, MyProgrammer, for 20 years — remembered a project for a client with an investment firm promising returns up to 15 percent — mostly to clients in their 70s. “Ultimately the guy fled the country, dying in a prison in France. We helped the authorities try to recover as much money as possible… I believe this scam cost investors over $500 million.”

Developer Scott Powell also had his own story to share. “I felt a similar twinge of guilt writing some time-tracking software for a legal firm in England, and after demoing it was told that starting a timer should *not* stop any currently running one. (this was single user software, back in DOS days, btw) Being able to bill many clients at a time struck me as being pretty dodgy.”

And the article also struck a nerve with iOS developer Jose Juan Qm, who’s now working on a project that involves medical patients and their exam results. He felt the requirements document included vague definitions of “healthy” results which could lead to risky behaviors, and “Reading your article made me realize I should not give up on that matter.”

Another commenter shared some history. “Engineers used to wear a special engineers’ ring on the right pinkie. The surface of it was stippled or ridged so that it would drag on the drafting paper when the engineer did his drawings. The idea of it was that it as there to remind the engineer of the grave responsibility he had. That every decision he made could possibly kill someone.”

Hugues Talbot sees ethics becoming an even bigger issue with machine learning-based artificial intelligence. “They produce rules based on input and annotated data, and the rules are not human-readable…” he wrote in a comment. “Soon we will have seemingly very high performing software that no one can tell [what it] will do faced with unforeseen circumstances.”

Julile Bort, an enterprise computing editor for Business Insider, took note of the burgeoning conversation, in an article which reminded readers about the ACM’s Software Engineering Code of Ethics and Professional Practice, as well as the IEEE’s own Code of Ethics.

Some of the comments on Reddit were distinctly more sardonic. One developer drew 2,135 upvotes for comment on Reddit describing how they wrote badly-functioning code that only technically met the design specs. But the code’s purpose was to distribute copyright violations from big movie and music studios, so they were actually glad that it wasn’t functioning as intended.

Another commenter recommended everyone read the ethics textbook A Gift of Fire: Social, Legal, and Ethical Issues for Computing Technology (now in its 4th edition). This drew a funny response when someone complained about the book’s price tag, and asked if there was a file-sharing site where they could download a copy.

“Let me get this straight, you want to pirate a book on ethics?”

Another commenter attracted 2,802 upvotes simply by posting a quote from Nathaniel S. Borenstein, one of the original designers of the MIME email protocol. “It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure.

“Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter.”

There seemed to be some question as to how much power developers really have. “What I’ve discovered over the years is simply if you refuse to build it they’ll get someone else to do it, and someone else certainly will,” wrote one commenter. “That’s not to say you should, just that you as a developer have very little power to actually stop it.”

Another added a wider perspective. “I personally think this is not just a developer issue, but affects every person living in a society… the sad thing about our industry to me is that sometimes it seems we give up before putting up any fight. ”

But one geek still felt compelled to point out that it’s also possible to be ashamed of code for other reasons. “To be honest, I was kind of hoping to see some beautifully ugly piece of code when I clicked the title…” wrote another commenter on Medium.

“But yeah, ethics matter and far too many developers/engineers fail to realize that.”