Pop phenomenon Katy Perry recently released the first single off of her upcoming album and performed it “Live” on American Idol’s finale, using some impressive technology.
For anyone who also lives under a rock, Katy Perry is expecting a child with The Lord of the Rings star Orlando Bloom. The combination of being pregnant and working on an album amidst a pandemic understandably limited the star’s options for putting on a live performance. Instead of performing for a crowd, she opted to use augmented reality. Augmented reality is the process of layering digital elements over a physical environment so the viewer sees a real place through a modified (augmented) view. For anyone who has ever played Pokemon Go and caught a Pokemon that sat on their coffee table, Perry’s performance was a much more expensive version of that.
Prior to performing “Daisies” on the American Idol season finale, Katy Perry did an interview with BBC Radio’s Zoe Ball last week. Variety, quotes her saying in the interview, “I just filmed [the performance] the other day on this incredible new technology; it’s augmented reality and virtual reality on a green screen. Someone set it up for me, I went in there, I filmed it, and then I left.” Given what we know about how augmented and virtual reality work, it’s possible to extrapolate how this innovative performance was created.
The Logistics of an Augmented Reality Video Shoot
Augmented reality is typically viewed through the lens of a smartphone camera. We can point our cameras at a physical space, and then the AR algorithms will overlay an image. This tech is how Amazon lets people plan furniture arrangements through its mobile app, for example. In the case of this “Daisies” performance, we’re likely watching through a crane camera pointed at Katy Perry. There’s a real-world lighting rig (perhaps a few) on the set, as well as the obvious microphone. There’s also a chair. Everything else in the room is probably a massive green screen.
The AR “lens” that we’re watching through is just the camera lens that recorded the performance. To relate it back to our phones overlaying a Pikachu from Pokemon Go onto a real-life park bench, the camera in Katy Perry’s performance is a camera pointed at her in a room, in front of a green screen. The green screen (and the chair, which is likely wrapped in green screen material) makes it impossible for us to see the background, so some producer at American Idol probably has footage of this same performance, but with the singer going through the routine in an empty void. The AR comes in when the digital artists replace that empty space with the effects we see in the final product.
This differs from the green screen scenes we’re used to seeing in movies and shows because the digital images aren’t simply making up the background and floor. In this case, there are sensors in the camera that recognize the parameters of the space and a digital artist working with them to control what we’re seeing around Katy Perry. It differs from traditional virtual reality as well, because VR makes the entire image into a virtual space, leaving no real-world objects visible.
