Alison Humphrey: Performance Capture in Live Theatre

I spent last night in an alternate future. The RSC's Tempest had its press night at the Barbican Theatre, and I finally had the chance to see the show in the flesh after following its development online for ages.

This is the kind of mainstream breakthrough for performance-capture theatre that I've been waiting for for half a decade, from our 2013 mocap Midsummer Night's Dream at Theatre@York, through subsequent live-animated collaborations Faster than Night and The Augmentalist with innovator Pascal Langdale, and now the forthcoming Shadowpox. I was mesmerized.

I don't regret being out of town for the Tempest's cinema broadcast in March. It's an odd irony that as much as theatre almost always loses something indefinable by being filmed, theatre with projected effects (screen-on-stage-on-screen) loses double:

Ariel avatar character design concept art

Ariel's avatar projected onstage (Mark Quartley as Ariel and Simon Russell Beale as Prospero)

But I do wish I'd seen it at the Royal Shakespeare Theatre in Stratford, where all the challenges of making motion-tracking work in real time in the Barbican's proscenium arch would have been compounded by the the multiplicity of audience sightlines on the RST's thrust stage.

Apparently the move from Stratford to London allowed them to cut down the number of projectors from 27 to 14, though they still have some crazy-powerful wattage painting the space, in especially gorgeous evidence in the Iris/Ceres/Juno masque:

Above and below, Juno's voluminous skirt unfurls as she elevates from a stage trap. Other surfaces include the Lidar-scanned and projection-mapped ribs of the wrecked-ship set; "the Vortex", a flyable cylindrical scrim made of mosquito netting that made Ariel's projections all the more ethereal; and hand-held shields motion-tracked for precise projection during a chase by rabid phantom-hounds.

I owe the visionary Sarah Ellis, RSC Director of Digital Development, a great debt for the invitation to last night's show, as well as for the joy of participating in one of her first epic digital experiments for the RSC and Google+, the social-media-soaked, real-runtime Midsummer Night's Dreaming.

The RSC ran a series of Google Hangouts before the big weekend, one of which was "Radical Dreaming? – Reinventing Shakespeare's plays," streamed live on June 20, 2013. Creative producer / digital wonder-worker Vanessa Shaver and I had just finished our own mocap production of A Midsummer Night's Dream, and were invited to join Nicky Cox, Geraldine Collinge and Erin Sullivan to discuss using on-stage facial capture to live-animate a 3D computer-generated donkey head for Bottom, and other bleeding-edge fairy magic.

The video clips we shared during the hangout got a little choppy on the transatlantic live-stream (four years later I still can't quite believe we can do this, from our desktops, for free), so I include the originals below for the curious.

For The Tempest, Animation World News explains, "The Imaginarium Studios and the RSC used an Xsens MVN system to track the actor’s performance. The data is run through Autodesk’s MotionBuilder software, and from there into Epic’s Unreal Engine 4. The video output is then sent to d3 servers powered by Intel’s Xeon processor connected to the RSC lighting desk, which in turn controls 27 projectors located around the stage."

Our Dream didn't have an nth the budget of this gorgeous-looking production, but we had a wealth of youthful exuberance, an encouraging Theatre@York, generous tech sponsors, and the courage of not knowing what the hell we were letting ourselves in for.

And even if Moore's law has halved the price of digital horsepower since then, what really matters is what our dream team, and now Gregory Doran's company, conjure up with it.

Tech costs. "Thought is free."

For further insight:

Creative Review: Tech comes to The Tempest in new RSC production

In the video above, Stephen Brimson Lewis, director of design at the RSC, goes into some detail on the bits-and-atoms design of the show:

“We went back to the kind of source material of Shakespeare’s play in the first place… and we started to discuss how you might create magic onstage… to enhance the story, not necessarily to devise a piece that was entirely about the technology….

"We’re using some very old theatre techniques. We’re using transformation gauzes, which would have been seen probably in the Georgian theatre, in Theatre Royal Drury Lane when it was first built. We’re using trapdoors that you might see in a Victorian theatre….

"We can also 3D-map into the set, so that we can wrap some projection into the structure of the set itself, which is not a sort of 90-degree, flat, angled surface like a screen is… We’re projection mapping onto one of the actors’ skirts at one point. She has a huge skirt that fills the stage and we projection-map onto that. And we actually projection-map back onto actors themselves, as well. And also facial capture. Real-time facial capture is the holy grail at the minute for everybody, because the processing power to deliver it, completely in sync and in time, is something that everybody’s chasing.

"If there’s any legacy from this production, I hope it’ll take some of the fear out of using technology in the theatre. There’s something in us as humans that we’re sort of suspicious of technology. And particularly in the arts world. I think the arts world can, at times, feel like it wants to hold onto the past too much, because the past is a wonderful, comforting space, knowing that we’re going to move forward with theatre. And sometimes the science world is the opposite. The science world is too excited and thrilled about the future, without always bothering to look behind you and check back. So I’m hoping that it will just be a kind of tiny little first footprint in the sand of a new generation of how to tell stories in live theatre."