While working on a series of projects aimed at improving the experience of pedestrians and bicyclists, I was reading Andy Clark’s book Surfing Uncertainty: Prediction, Action and the Embodied Mind. This proved an unusually fruitful combination, even though the book is about perception, not about design. But in the context of these design problems, Clark’s explanations became interesting points of departure which often reshaped my perspective.
His phrase: ‘a web of humans and machines, each of which are now busily anticipating the other’ seems to me a perfect description of what our busiest urban public spaces are becoming. As ‘smart’ systems become more prevalent, physical touchpoints are being minimised or disappearing altogether, from whole cashier-operated checkout counters to familiar everyday features like taps, handles and buttons. Displays of information are also becoming more fluid and dynamic, as printed posters and signs are replaced by screens with moving images, some of which invite direct interaction through touch. Augmented reality seems poised to become more practical, which means it will soon play a bigger role in the digital layer we use to make things findable with our mobile devices. All of this is driven by algorithms which attempt to both predict and steer our behaviour. And some algorithms are now becoming more than smart assistants – they’re becoming agents that act independently on our behalf, without much intervention on our part.
This raises interesting questions…
As physical touchpoints vanish or minimise, they take affordances with them. How might we add affordances and make the most of the remaining ones? The bicycle parking facility entrance we recently piloted has no doors – sensors identify the type of bicycle and user and initiate interactions. We realised that the sense of a boundary had largely vanished with the doors, so we paid special attention to the sculptural and visual qualities of the remaining components, so that approaching bicyclists would clearly perceive that they were approaching a boundary where some kind of interaction might take place.
What is the right way to make use of the virtual and physical realms for wayfinding? Serial mono-tasking – switching between the real and virtual worlds – is typical of mobile device use. This means less attention for the real world and its affordances, as people’s attention bounces back and forth between their mobile devices and surroundings. Clark mentions experiments that reveal that while doing a relatively simple task, people make more continual and intensive use of the world as an external buffer, than we might imagine. Will augmented reality stitch these two realms back together?
How smart should a touchpoint be? When should it work more like a tool, an assistant, or an agent?
I’ve often thought of this while observing the transformation of route information signage in public transportation. I rather miss the old ‘dumb’ printed overviews of whole bus routes. These have been replaced by screens that show only small parts of the route as the bus progresses, mixed with extra information and advertising. The result is that if I don’t know the route, I can’t pinpoint my present location and determine the number of stops to come so I know when to prepare to get off, and must rely on apps, announcements and asking fellow passengers to orient myself. I now also have to wait for ads to finish to see whether a stop is coming up, so I’m actually forced to spend more time watching the screen. What are the best practices for these kind of screen-based systems?
A place where a ‘smarter’ approach might work well is the subway map. Real-time, data-driven interactive subway maps can be smart assistants, using location info and predictive text to make finding a path easier. But they can also act as agents. For example, they might use real-time traffic data to change the visualisation of routes and nudge people to use less crowded alternatives, distributing traffic more evenly and avoiding blockages. Or the maps can offer these as defaults during peak times. (One of the advantages of Mr. Beck’s original schematic London subway map is that it distorted real-world topography, making the far-flung stations look closer by.) The maps could also offer more detailed information for first-time travellers than to experienced ones.
But public spaces must work for everyone, not only those with the most sophisticated technologies. How might we keep options open for old-fashioned, ‘dumb’ touchpoints where they provide more clarity to users?
Clark’s book is a tough read for the non-neuroscientist. The basic idea of predictive process – that perception is prediction minus a kind of ongoing correction based on back-flowing error signals, weighted for accuracy – and its ramifications, can be difficult to grasp.
The main thing I took away from reading it is that the see-think-act paradigm we’re used to is a good model for interaction, but shouldn’t be mistaken for a model of perception itself. Understanding the central role of prediction in perception and the way the brain, body and world form temporary ‘coalitions’ to solve problems, might help us to design the external buffers and supports our embodied, moving brains need to create optimal behaviours on the fly. This can help us create, in Clark’s words, ‘a world worth acting in’.
More about the ideas…
Clark presents a new theory of perception different from the basic sense-think-act model most of us are used to. The core idea is ‘predictive process’: that instead of passively sensing and interpreting signals, the brain pro-actively sorts out the incoming sensory barrage in advance and checks the accuracy of its prediction on the fly. So what we perceive is what our brains predict, minus a kind of constant correction based on incoming signals that don’t seem to match the prediction. If you’d like to know more, his interview with Ginger Campbell on The Brain Science Podcast is a fun and accessible introduction. For those of us working with robotics, the book also contains interesting examples of the theory’s application in that field.