The ironies of automation explored



In 1983, Lisanne Bainbridge’s paper “Ironies of Automation” was published in Automatica. It’s since been widely recognised as one of the most influential papers in the discussions surrounding automation.

Automation has come a long way since the 80s. But how have the ironies outlined by Bainbridge manifested or evolved in today’s technology?

Here, we explore Lisanne Bainbridge’s classic paper and how it stands up next to modern automation.


What is meant by ‘ironies’?

Before exploring the ironies of automation identified by Bainbridge, it’s worth touching on the terminology. Specifically, what’s meant by ‘ironies’.

In her paper, Bainbridge defines irony as follows:

Irony: combination of circumstances, the result of which is the direct opposite of what might be expected.

In this sense, an irony of automation is not a good thing or a bad thing, but an unexpected result or situation springing from use of the technology.


Automation and the operator

The classic view of automation is to replace human manual efforts. Indeed, this is a view taken by Bainbridge in her paper. And this leads us to the first of the ironies of automation.

Automation takes on manual tasks. The goal of automating is to replace human toil. Yet, even high-powered automation systems need human supervisors. They need monitoring and maintenance. Automation, ironically, creates a need for humans.

Plus, as Bainbridge points out, any designer that tries to eliminate the human operator stands to make more problems, not fewer. In fact, this is the approach that results in the other ironies of automation.

When you disregard the need for an operator — a mindset that automation lends itself to — you leave them mismatched tasks and little support. The tasks that Bainbridge outlines, then, involve monitoring automation and taking over when it fails.


The irony of ‘de-skilling’

For all the importance of the human in the use of automation, Bainbridge argues that automation ‘de-skills’ human supervisors.

If automation takes over the bulk of the work, that means that human team members are less frequently using their skills and experience. They stop doing the task in question, and instead monitor the automation tool and watch the task get done. But skills and experience can deteriorate over time if they aren’t used. (Consider how much — or little — you can remember from your school classes, for example.)

There are two sides to the solution we’re seeing for this de-skilling irony, however. First, there’s the fact that automation still only handles rote – even mindless – tasks. This leaves plenty of opportunities for humans to apply and refine their skills.

Second, as automation (and now AI) continues to grow in ability, there’s been a shift in the skills companies want. Namely, the skills that automation and other technologies cannot replicate. So, as well as strategic thinking and creativity, there’s also a renewed focus on ‘soft’ interpersonal skills.


The ironies of monitoring

A large part of Bainbridge’s ironies of automation revolves around the need for someone to monitor automated processes. There are three ironies to this.

•       It (ironically) makes very little sense

The point of installing automation, Bainbridge argues, is that it can do the job better than the operator. Yet the operator is the one left to make sure that the automation is working effectively. This is another of the ironies of automation.

However, this irony seems to view automation almost as a separate entity, taking over a job, rather than acting as a tool.

Automation isn’t ‘better’ than the human. It offers a different set of abilities, but it can only work with the help of a human — feeding it the rules it will follow, for instance. The human tells automation what to do. And, in turn, it gives the human a way to get the job done more efficiently, consistently, and quickly.

•       The better the automation performs, the harder it is to spot errors

Automation is powerful because it can do more, faster. Computers can process more data and consider more parameters than a human. And it can do so on a much faster scale. How, then, can a human operator of automation (or an artificially intelligent system, for that matter) monitor and check results?

There’s no way to check in real-time that an automated system is following the rules correctly. Bainbridge points out that, as such, operators of automation can only monitor output on a meta-level. That is, deciding if it’s acceptable, without knowing exactly how the system got there.

This irony prevails today, and is especially pertinent to the issues surrounding ethical AI and the ‘black box’ problem. If there’s no way to know how an AI reaches an automated decision, how then, can we trust it?

•       Human attention and automated alarms

Finally, Bainbridge discusses how, when it comes to monitoring, human ability (or indeed, inability) gets in the way.

“[I]t is impossible for even a highly motivated human being to maintain effective visual attention towards a source of information in which very little happens, for more than about half an hour.”

– Lisanne Bainbridge, “Ironies of Automation”, Automatica

In other words, when an automated system performs well, it’s harder for the operator to stay focused on it. One of the solutions to this monitoring issue is to send an automated alarm when a process doesn’t go right or gets stuck. This is yet another of the ironies of automation: automation must help operators monitor the automation.

But rather than view this as a problem, it provides an example of how automation can (and should) help your team. When you think about how the automation can support your team, you answer the problems that arise from poor man-machine integration.


The ironies of automation

The ironies of automation focus on what happens when you forget to consider the humans that work with automation.

While they cover a series of problems that can befall automation and its users, they are avoidable. All you need to do is remember that automation software is a tool. It’s there to help and lift your team, not take their jobs. It’s there for the rote tasks, not to eat anything and everything it can.

You need a man-machine balance. When you have that, the ironies of automation are less of an issue.


Useful links

Automation and the resurgence of soft skills

Are AI ethics impossible?

The awkward interplay of man vs machine


Download