Flatland is a collaboration between Extant, the UK’s leading performing arts company of visually impaired artists, robotics engineer and research scientist Dr. Ad Spiers, and Janet van der Linden from Open University’ Pervasive Interaction Lab. Led by blind Artistic Director Maria Oshodi, Flatland sprang from the partners’ shared desire to explore the potential of haptic technology – the science of communicating by touch – to deliver performance, an idea the three partners had begun to investigate in a 2010 scratch collaboration, The Question.
Flatland was designed as a proof of concept piece to show it is possible to create an immersive theatrical experience using technology, that is accessible to both sighted and visually impaired audience members. It aimed to challenge the status quo in arts and heritage, where digital expansion or enhancement of cultural opportunities focuses on the visual, and the use of screens as an interface. Instead, Flatland aimed to move theatre away from ‘spectacle’ to a more embodied experience.
The project was a collaboration between a multidisciplinary team of artists, engineers based in the UK, Holland and the US. Together we explored a range of innovative technologies to create an immersive, pitch-black installation inspired by 19th century satirical novella, ‘Flatland’. Specifically we developed a handheld haptic device and tracking system which guided audience members through the dark, created interactive set pieces that provided sensory experiences throughout the installation, including through the use of eTextiles, and by using sound effects and live actors sought to integrate all of these elements within a dramatic narrative. Each performance enabled both independent navigation through a set of four multisensory scenes, and group moments when audience members were brought together to test whether the technology could support a collective experience.
In creating the robotic haptic device, a range of prototypes were tested to identify best haptic communication capability, physical aesthetics, robustness and ease of fabrication. The final design, a cube which twists and protrudes its upper half to denote direction and distance to cover, was created using 3D printing. An extensive survey of available localisation systems, including Bluetooth, radio frequencies and magnetic field generation, identified an off-the-peg system called Ubisense as providing the best trade-off between size of area covered and accuracy in tracking people. This was combined with additional hardware (magnetometers and X-OSC microcontroller), and bespoke software, to create a system that could identify the position and orientation of an audience member, and through the haptic device, guide them to a specific target.
Over 100 people, sighted, visually impaired and wheelchair users, from a range of games, tech, arts and heritage sectors, were invited to the Flatland installation to explore this first hand, and to reflect with us on the success of this endeavour. Four performances per day took place over six days, with groups of four people being guided through and interacting with various haptic technologies in a multi-sensory dramatic installation – a world of sensation and sound. All audience wore distinctive ‘3D’ suits to protect them in Flatland, which housed the wiring connecting them to their handheld robotic devices. Different locations were created, inspired by descriptions in the novella of a domestic scene, church, hospital, university and trial, as well as other dimensional spaces.
The performances involved live actors as well as a recorded soundscape. As well as immersive sound played though external speakers, the audience also experienced individual sounds played through bone-conducting headphones (that leave the ear uncovered). These were triggered by each audience member’s proximity to various target areas in the installation.
By swapping the usual visual cues for multi-sensory ones, the drama of Flatland was experienced through the whole body. It is unique in exploring pitch-black, immersive territories from a visually impaired perspective, and creating a shared experience for visually impaired and sighted audiences – the performance was experienced to its full whatever the level of sight of those attending. Perhaps most importantly the Flatland installation highlighted how accessible technologies can be designed into an experience from the beginning, rather than as an afterthought.
The project showed the handheld navigation device and its localisation/navigation system, was successful in providing guidance, and represents a huge improvement on our work in ‘The Question’. In fact, our data suggests this type of shape-changing interface has potential to support general navigation tasks in real-world settings.
The project also enabled us to explore how new technologies can play with the audience’s engagement with their own senses, the environment and storytelling. We were able to develop ideas for this revolutionary new theatrical convention, although we are only just beginning to understand the rules of engagement.
What we learned:
- the selection of the best available localisation system, and its integration into a system that could co-ordinate multiple users, devices and zones in a dramatic production, was key.
- teams working at a distance need regular blocks of time in the same room when they can creatively explore together, to avoid precribed thinking.
- to further enable the creative process, we would in future locate these ‘play days’ within the actual venue to create a holistic build, rehearsal and performance outcome.
- in a multisensory environment, fine-tuning is required to to manage ‘cognitive load’ – ie to balance the attention required of audience between listening and feeling. Provide too much of either stimuli and you risk one of them being ignored.
- interdisciplinary working allows you to pull together the results of both precise, lab-based testing, and informal, in-the-wild research, and gain a valuable form of triangulation.
- the distinction between creative team, technology and research partner is unhelpful. All are engaged in a deeply creative process. Tech, in this context, is not something that can be pulled form a shelf, but is groundbreaking research that requires time. Similarly, research is an ongoing process of reflection in which all partners share.
The project aimed to influence and inspire research, tech, arts and heritage sectors, as well as to increase the public profile of our aims and methodology, as a way to ensure the legacy of our work. Flatland has been successful in generating a wave of interest and engagement via press coverage and ongoing presentation opportunities at events and conferences. We have created a ‘suitcase demo’ – a portable, mini version of the experience – to help us service this while Extant work towards the creation of a full-scale, public version of Flatland after 2018.
Meanwhile, Ad Spiers aims to undertake further research around the potential of shape-changing interfaces versus vibrotactile interfaces, and test outdoor/urban navigation with such devices. Janet van der Linden and team will pursue the more emotional connotations of tangible interfaces, and further exploration of an interactive, accessible evaluation device designed for to support Flatland’s post-show audience discussions.
For any enquiries about Flatland, contact firstname.lastname@example.org or 020 7820 3737.