Vision and Automation in a post-Covid World (Journal of Visual Culture & HaFI, 42)
This is the forty-second instalment of a collaborative effort by the Journal of Visual Culture and the Harun Farocki Institut, initiated by the COVID-19 crisis. The call sent to JVC’s editorial board, and a wide selection of previous contributors and members of its extended communities, described the task as follows: „There is a lot of spontaneous, ad hoc opinion-making and premature commentary around, as to be expected. However, the ethics and politics of artistic and theoretical practice to be pursued in this situation should oblige us to stay cautious and to intervene with care in the discussion. As one of JVC’s editors, Brooke Belisle, explains: ‚We are not looking for sensationalism, but rather, moments of reflection that: make connections between what’s happening now and the larger intellectual contexts that our readership shares; offer small ways to be reflective and to draw on tools we have and things we know instead of just feeling numb and overwhelmed; help serve as intellectual community for one another while we are isolated; support the work of being thoughtful and trying to find/make meaning…which is always a collective endeavour, even if we are forced to be apart.'“ TH
Image from Common Objects in Context (COCO) image dataset used to train computer vision systems
Vision and Automation in a post-Covid World
By Joel McKim
The Guardian headline reading “Robots to be used in UK care homes to help reduce loneliness” is accompanied by a photo of a gleeful senior citizen interacting with a wheeled semi-humanoid robot named Pepper. The article describes a research trial studying the use of “culturally competent” robots to assist in caring for the elderly in the UK and Japan and although the project actually began several years ago, it’s only now, post-onset of Covid, that the story has found journalistic attention. The image of AI-powered robots offering a possible solution to the financial difficulties, health concerns and staff burnout facing the care industry, is one seemingly designed to elicit equal parts anxiety and condemnation.
But the role automation will play in our post-Covid world is likely to be less spectacular, yet more disruptive than such contentious visions of robotic health facilities. And while we may balk at AI replacing the forms of care and intimacy we view as innately human, there are already indications that the current crisis is making many of us increasingly amenable to other forms of more mundane automation. The move towards cashier-less retailers and automated warehouses that perhaps caused some unease in the past, may now appear as a sensible and even ethical solution. A debate about mass unemployment and increased corporate profits is quickly reframed into one about responsible health decisions and employee safety. As the human-computer interaction researcher Richard Pak asserts in a New York Times article on this very subject, “Pre-pandemic, people might have thought we were automating too much. This event is going to push people to think what more should be automated.”[i] For those with much to gain from greater levels of work-place automation, this would seem an opportunity to seize upon a moment of increased public receptivity.
If we’re to trust the investment reports, business magazines and policy think tank whitepapers on the topic, post-pandemic interest in automation is already surging. To take one circulating, but not-entirely disinterested example, the Honeywell Intelligrated Automation Investment Study found that since the onset of Covid over half of US businesses are increasingly open to investment in automation with e-commerce; grocery, food and beverage; and logistics companies chief among them. But on the topic of labour impact, these sources tend to offer either platitudes – “The vulnerable will be the most vulnerable” (Brookings Institute)[ii], or hollow reassurances – “People empowered by automation will bring us out of this crisis” (Forbes).[iii]
A flurry of more sustained academic considerations of the potential impact of AI and automation have emerged over recent years, all cautionary but with various levels of optimism or pessimism regarding how humans will fare in the automated world to come – books like Erik Brynjolfsson and Andrew McAfee’s The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies and Carl Benedikt Frey’s The Technology Trap: Capital, Labor, and Power in the Age of Automation. Frey’s 2013 research with Michael Osborne at the Oxford Martin School, published as “The Future of Employment: How Susceptible Are Jobs to Computerization,”[iv] was an early and influential warning of just how cataclysmically disruptive processes of automation may be for the current labour force. Analyzing O∗NET, an online detailed description of occupations developed for the US Department of Labor, Frey and Osborne determined that 47 percent of US employment falls in the high-risk category of being automated within the next decade or two. Frey has also acknowledged the potential of the Covid-crisis to acerbate an already troubling situation, writing in the Financial Times, “automation anxiety looks set to witness a revival – and with good reason. Coronavirus is likely to accelerate automation.”[v]
Why does this admittedly disheartening discussion of automation and work precarity belong in a series reflecting on Covid visual culture? Recent advances in computer vision technology (enabled by machine learning and deep neural networks) are one of the primary factors driving forms of automation that importantly span both the manufacturing and service economy, from driverless vehicles, to assembly line robotics, to checkout free shops. One indication of this development, image recognition error rates for neural networks have fallen precipitously in recent years, with computer vision algorithms now able to classify images in large datasets like ImageNet with as much as 98% accuracy.[vi] Our powers of perception – once, like caring, a seemingly innately human capacity resistant to machine replication – are apparently no longer the insurmountable barrier to automation we might have believed. Much of this technology has moved beyond mere speculation and into a prototyping phase, and of course some forms of workplace automation, like warehouse robotics, are already with us. The drone delivery systems and autonomous shipping vehicles, envisioned by companies like Amazon are still unproven, but the underlying computer vision systems they rely upon are developing quickly and a Covid investment push may well bring forward their timetable of deployment. In an economy increasingly reliant on now vulnerable service sector jobs, like transportation and retail, this could have a devastating impact.
These concerns regarding the relationship between human and computational vison were of course central to Harun Farocki’s own prescient investigations of machine vision and operational images, from military tracking systems to computer simulations. Today, artists like Heather Dewey-Hagborg and Adam Harvey, are producing work that compels us to think critically about our contemporary technologies of artificial perception, exposing the technical and social infrastructures that underpin facial recognition systems, for example. These artists help shed light on an already unfolding regime of vision and automation, one potentially accelerated in our post-Covid world.