In our 'Notes from the Field' series, we speak with artists, filmmakers, distributors, exhibitors, programmers, writers, and thinkers working across the spectrum of art and film, hearing about the ways in which they are working towards creating new ecosystems for the creation, circulation, and consumption of moving images. In this first conversation, we speak with Alexander Porter, an artist working in immersive media and the creator of the virtual worlds that can be seen in our new release Truth or Consequences. Through Scatter, a company he co-founded, he has been recognised for defining the discipline of 'volumetric filmmaking', a practice that involves storytelling through the lens of three-dimensionally scanned or computer generated images. We talk about this discipline and its relationship to VR, as part of a wider discussion that touches upon the imaginative potentials (and pitfalls) of working with VR and other forms of immersive media.
Sentient.Art.Film: How do you use VR and other forms of immersive media in your practice? How did you first encounter VR and when did you begin using it?
Alexander Porter: I’ve been working in this space for about 10 years. Initially, I started collaborating with a dear friend of mine who is an artist and also has a background in computer science and computer vision. Out of that collaboration we created a series of tools - tools for using cameras, and cinematic tools that were applicable for a kind of virtual cinematography, through which we would be using cameras combined with three dimensional sensors or scanners in order to produce a kind of hybrid form of cinema. At the time, this sort of sat adjacent to "machinima", which involves using virtual spaces in video-game engines to tell real stories. For us, what was critically important was that we were actually sampling and capturing from the real world and bringing that information into virtual spaces, rather than the kind of the conventional mode of creation in visual effects or in video-games which is often all about creating synthetic spaces from nothing, giving digital artefacts life.
For us it was about contemplating the ways in which we can use advanced technologies that exist to capture from the world, then use it as source materials for storytelling. We created a tool which is now called "Depthkit",a storytelling tool for volumetric filmmaking. When we made it, it was a piece of software that would allow you to film with a 3D sensor or with the Kinect, a video device that was originally made to use with an Xbox. We used it for our creative practice and for client work, making documentaries and experiences with it, but at that time we were also very inspired by speculative fiction. We were reading people like Bruce Sterling and thinking about the ways in which he used fiction as a way to look critically at technologies, creating subtle futuristic fictions in order to talk about the issues and inequities we face in the present day. We kind of jokingly started referring to our software tools as a form of speculative fiction. Despite this, a whole community of people started using the software to make visual effects and stories that were dealing with this intersection between the digital world and the real world, creating surreal effects at this particular juncture.
We would talk about a theoretical future where it was almost like the audience members would play the role of the cinematographer and they would be re-photographing the material they encountered, or seeing things from different vantage points. The user would become a participant in the creation of the imagery and have control over the nature of the narrative. Then, in around 2014, Virtual Reality crested with the release of the development kits from the Oculus Kickstarter. At the same time, I was involved in a project that was directed by two friends of mine, James George and Jonathan Menard, called CLOUDS, which was basically designed as something between a game experience and a virtual documentary, which was accepted at the Sundance Film Festival. A few months before the festival, we merged the pre-existing, screen-based experience project with the Oculus SDK, and it felt very seamless because this methodology of thinking and working was already alive for us. Since we were creating dimensional spaces and stories that were cinematic, you could just pop the viewers head in there via Oculus and let them move around, more or less. That was our first touch-point with VR as a format, but it felt surreal as we'd already begun working working on the theoretical project years prior to its readiness as a technology.
Had you worked with video-games before, seeing as you mentioned machinima?
Most Virtual Reality experiences are built in video game engines now, right? I just used machinima as a reference point for stories told in game engines. I don't particularly like avatars as representations of people, at least not in that typical cartoony, game-like style. I bounced off of machinima as a discipline for that reason. Game avatars are like phantasms, whereas if you're recording footage, there is more of a relationship to a real person, and suddenly it gets more i intense, political and human. Though these issues are present in the politics of representation in video games, it all springs to the forefront when there's a real person directly behind the representation.
I love that because it brings intimacy to what you're creating. When it is a real person rather than an avatar, it takes you out of your head.
Yeah exactly, there is something specific about sensing that an image, or a piece of digital material, at its core was actually real and captured from the world at one point. It has this evidential quality which is easy to subvert or fake but remains a language that is familiar to people. Whereas I think oftentimes when you create an avatar, it feels more like a doll: clearly a representation of a person but still two stages of representation away.
With my company, when we make software tools, we try to think critically about what we want to create. For us, it's not just about basic enablement, which is of course important, but about trying to create a discipline that's larger than just the application of a technology. With volumetric filmmaking, this largeness comes from the fundamental idea of gathering real touch points from the world and using them for a new kind of storytelling.
You touched on this slightly earlier, but I wanted to ask you in what ways do you feel like VR or other forms of immersive media can be used to imagine new futures or new worlds, rather than recreating or building from things that already exist? What advantage is there from being unshackled from reality?
It's such a complicated subject, because there's this utopian sentiment - which I love - that when you're creating an entirely new environment where you don't need to be bound by the physical world, suddenly a whole new world is possible. Practically, what is more commonplace is that the same issues that we have in our current world just manifest and mirror in these new spaces. What I love about Virtual Reality is you can create really surreal, special, and unusual spaces that are pragmatically very difficult to create in the real world, often for cost reasons. You can make grand buildings and spaces that would be very difficult to create or work with in filmmaking.
So there's the creative dimension, which is about making new representations possible, but there is also the political side which is a more speculative notion that involves trying to create the world and society that would like to live in - which involves reflecting what we want through our images. The biggest barrier for both, I think, has to do with access to devices and to skills, both of which are currently quite rarefied. My company focuses on the creation side of things, and I think that this is a meaningful area. Knowing that the brutal and inevitable march of technological progress will always move forward, a meaningful place to intervene could be through making sure that the tools available are dynamic, flexible, and inclusive.
What you would like to see changing about the medium going forward? Is it purely a question of access then?
Oh my god, that’s a complicated question for me because I work in this space but also want to remain critical of it and remain concerned about its future. One thing that I think about a lot is the experience of trying to communicate profound or complex ideas with words. I find this really challenging in my daily life. Trying to convey a lived experience to someone else feels like a life's work. I think that a lot of the challenge in that is to not just transmit a cerebral experience, but conveying felt experience—not exactly lived experience but something a little bit more vivid or sensorial, even bodily in form. Immersive and interactive media are quite good at that because they can function not only at the level of content, but also in form and format, which means that they can give you bodily experiences that are difficult to convey in other forms of language: simple things like feeling very small in relation to something large, or feeling things moving quickly towards you. These experiences are very fundamental and sensorial and the ways that we react to them are really deep and physiological. I think it is these bodily experiences that really colours life, more than the kind of cerebral layer of content and ideas, so that capacity is something that excites me about VR, and I look forward to seeing it evolve.
I think there's a theme present within immersive media which is about evading the world and avoiding real life - checking out or tuning out, pure entertainment. VR can make it seem possible to avoid the world as it really is; it’s the perfect symbol for tuning out of the world and choosing a different reality. I find that really complicated. it's a steep hill to make sure that what is available inside of this new space is as complex as it needs to be, but also true and honest to what the world is like. I think about that a lot, but personally, I love using VR for creation, more than for consumption.
Do you think more people would think about creation, were they given a route in? I had always thought of VR as more of a passive medium, something where you are given a headset and you engage with an experience for a fixed duration, so it is interesting to hear you speak of it more like a tool for creation.
What I think is in the background of our conversation is the shortcomings of making analogies to prior technologies and prior media. A lot of people think of VR either as a natural progression from video-games or as some kind of evolution of cinema. I think it's better thought of as a weird confluence of a lot of technologies and a lot of really good design work to make it useable. This enables a strange hybrid which allows you to have a perceptually huge screen on your face, but also to have an extraordinary dexterity with your physical body and your hands. That capacity is not really possible with other technologies. You have access to your natural dexterity, and your capacities to sense space and velocity too. I love modelling, drawing and creating with these tools, and I think that a lot of the lasting impacts of VR will be on this generative, creative side as opposed to consumption.
I think something that was really interesting was how you said that when you started collaborating with your friend and creating tools, you were thinking “oh, this thing doesn’t exist for what we want to do, so we should build a tool to enable us to create what we are imagining”. I suppose that might be a good way to think about VR as this thing that people have a set of expectations and pre-conceptions about. This means that, when creating, they may replicate what has already been done, rather than thinking “what do I want to make, and what sort of tools help me to make that?”
I really agree with that. One of the things we thought a lot about at the time we started this project is the fact that any tool that you pick up has its use embedded in it. Tools tell you what to do with them; they guide you, often by narrowing what you're permitted to do with them based on choices made by the creator. So, in a very important way, politics, and what is permitted, are baked into the tools that we use. In practice, it's very it's hard to imagine all of the things that you're being prevented from doing based on the way that a tool is presented and narrowed. So, when you think about wanting to create in truly new ways, you have to contemplate the tools and the range of things they permit you to do, as carefully as you think about what it is that you are actually creating. For us, that was a very intentional practice: imagining the new things that we would want to create and then reverse engineering the tools that would need to exist in order to create them, if that makes sense?
Yeah, just about.
You encounter this a lot in interviews with science-fiction or speculative fiction writers speaking about their work. A lot of the work is about imagining futures or worlds and then drawing vignettes about the way that world is. Dystopian fiction is often about the opposite, predicting bad outcomes that are an extrapolation of what exists today. When you draw a little image about that world, you then have to explore how it got to be that way: what the background is and what all the backstories are for all the objects and people in that world. I think it’s a wonderful creative practice.