The Heat-Seeking Camera: João Rosa on AGGR0 DR1FT

Harmony Korine’s hallucinogenic AGGR0 DR1FT follows a homesick assassin on his rounds, amid visions of demonic enemies and gangster revelry. Like Korine’s SPRING BREAKERS (2012) or TRASH HUMPERS (2009), it’s shot with a wild aesthetic, probably his most out-there yet: the swirling, infra-red colors of thermal cameras, further ensorcelled with VFX and AI. To understand how the look was created, I spoke with João Rosa, the youthful creative director of Korine’s new company EDGLRD and his close collaborator on AGGR0 DR1FT. When we talked, he was working on a project involving The Weeknd: he had scanned the singer volumetrically, and then Korine did a photo shoot without The Weeknd present, using the singer’s full-body volumetric data and digital backgrounds. All in a day’s work in manifesting novel Korine visions just beyond our ken.

How did AGGR0 DR1FT start? Was thermal imaging part of the initial concept?

I still have the piece of paper with some things Harmony wrote on it, like “assassin.” It was a very mathematical thing—a lot of times Harmony approaches filmmaking and editing in a mathematical way, like, “I need a certain amount of time for this, a certain amount of time for that.” AGGR0 DR1FT is a name that came very late in the process. We were calling this BBX, and BBX was a formula of multiple scenes of this concept that he was creating about this assassin, and it was really not meant to be a film. It was this visual experimentation, maybe already thinking in terms of the exhibition that he [also] ended up doing.

That [process] lends itself to doing things in experimental and unpredictable ways because I couldn’t approach it from the standpoint of traditional VFX. I got involved because I’m very good friends with Harmony’s editor, Leo Scott, the person who edited TRASH HUMPERS,among other projects, and who was involved in AGGR0 DR1FT from the beginning. The idea to use thermal cameras happened very early in the process, but probably not before he had this little piece of paper with a formula.

Can you explain what kind of cameras capture thermal imaging?

These are extremely expensive thermal cameras. They’re not for use in entertainment at all. They are technical cameras for use with large construction or aerospace things. Their purpose is to detect things like a part that’s overheating in a rocket, or a leak of gas in a power plant. First of all, the format of the film is almost square, not cinematic like 1.85. And the sensor has a lot of damage and weirdness. We were using two cameras, and I can tell when they were using which, because the sensors literally have different damages in different parts of it. And compared to a professional cinema camera like an ARRI, these are in a completely different league when it comes to price!

When the images come back from a shoot, what do they look like and how do you work with them?

What the camera sees is a black-and-white image—where white is the hottest part of the image and black is the coldest. It was then all crafted together with Harmony, because you map colors into this black-and-white gradient. So it’s like, “Oh in this scene I want white to be pink, and as it goes toward cold, I want it to get greener and greener.” But the thermal wasn’t the only way we captured those scenes, because there’s a lot of detail that the thermal would never see. For example, thermal cannot see reflections, thermal cannot see like specific things like a detail in the eye, because it only sees the temperature of an object. So there are more layers. That includes the effects layers and the AI layers that we created to achieve the final look.

AGGR0 DRIFT looks different from “infra-red” sequences in other movies, which tend to look the same throughout. These color palettes are shifting from scene to scene, evolving, sprouting details. How were you working with color?

It’s 100% part of Harmony’s process to see what’s there [on screen] and make decisions. So the team was experimenting with several different color palettes and deciding what would be the most interesting one for each scene—always with the goal of being the most hypnotic possible. And sometimes the shifts introduce something dynamic to the film: just by shifting that color palette, you refresh the attention that you’re paying to the scene. And the camera is really interesting. It recalibrates sometimes. So if someone in the scene lights up a cigar, that causes the camera to shift the black-and-white a bit, and then we could decide: you can use just the raw output of the camera where it doesn’t shift, or we can embrace the almost auto exposure of the camera, which brings us closer to something like a lo-fi video camera. So it was all about deciding whether we wanted to embrace certain things or not. But the beautiful thing of working with Harmony is that it’s always possible to embrace the real nature of all the elements that we’re using.

Still from AGGR0 DR1FT. Courtesy of EDGLRD.

I think you can see that visual experimentation in Korine’s films like GUMMO and TRASH HUMPERS. Were there specific reference points for the palettes?

There wasn’t a thing that Harmony put in front of us as a reference. But there was this complete different vocabulary when it came to talking about color: it was really about paint. He loves color. Working with him day to day, I don’t dare show him things without color, like de-saturated things, because that’s immediately like a “no.” He approached this with a painter mentality, as if he was painting a canvas. We never spoke about color with a vocabulary of saturation, hue—it was never technical, it was always an artistic conversation.

How did you apply VFX to these images? Some additions are evident, like the demon wings on his rival, or Travis Scott’s lizard tongue, but I’m sure there are other things too.

Once we started getting some footage, we very quickly locked the edit so that I could really start this experimentation knowing that whatever I was working toward was actually in the film. Some things I conceptualized and showed to Harmony, like the wings, the demon, the mask—some of this work that is more traditional. Then I knew that if Harmony was down for those ideas, I could start tracking the footage, or rotoscoping things if I needed rotoscope. So I started with those larger and very unpredictable parts of the pipeline, because I didn’t know if I would be able to accurately track thermal [images when integrating VFX].

Would you sketch out concept images yourself?

Those concepts I did by hand. Probably nowadays I would have used some kind of AI thing, but at the time I used mostly like Photoshop and did some collages and some photobashing just to have an idea. And I had a very strong inner logic for why the guy has wings, why the guy has a mask, why the demon appeared. Because otherwise, it’s hard to approach a feature film and do effects.

Tell me about the logic!

First of all, Harmony never asked me what my logic was for any of that. Never! It was purely a visual reaction to the things I was showing. But what I have to say is that my favorite films are films that I don’t understand, because I love how my mind really works then and becomes occupied with interpreting it. Like Kubrick’s films. And one of the things that I did not like was this YouTube video where Kubrick is explaining some of 2001! [With AGGR0 DR1FT] I don’t think it’s worth describing the whole thing—just to keep it interesting. But my internal logic was more the dynamic between who the good guy was and [who] the bad guy was, and which symbols they could each have. One is more demonic, one, with the wings, is closer to something like an angel. It guided me because I had to be very resourceful with a small team. Which is why back then, like 2022, I did embrace some very, very new AI things that were starting to appear as a way of creating visuals that were very fresh.

How did you use AI to help generate imagery or effects?

There’s all the post work when it comes to what we just talked about the color, etc. But I wanted to touch as many scenes as possible with VFX. So, I was like, I’m going to use AI in a very visual way, just purely visually, aesthetically. It was early on—there was nothing commercially out. Nowadays there’s Midjourney and DALL-E or Sora. Back then, it was really uncharted territory. I assembled a team of AI engineers in Brazil. They were putting together scripts, protocols, like a security camera protocol that identifies faces together with one that estimates poses. So we would find a face and put something—what some people describe as tattoos, or robotic parts.

For example, the snake tongue. Harmony is never like, "Oh, please put a snake tongue on Travis Scott.” But he’s just like, “Oh, it would be funny, a snake tongue”—like very casually. Then I can experiment and show it to him. Or in this stripper scene, there are those sparklers coming from them. And by now I understand that’s part of his directing style is to inspire people with wild ideas but not force them to do it.

And the thermal thing just feels different. That kind of footage gets stored in your memory differently from normal footage. Leo was cutting this film and we would get to the next stage of the edit and almost forget what we’re looking at. And I think that affects the cut, that affects the way things come after each other. It just hits your brain differently.

The violent moments seem to hit differently, too. Were you adding blood imagery?

The blood was mostly added [in post], but there is quite an insane scene with a lot of blood that has a lot of practical [effects]. All the fights, I would say part of the reference and the intention was to be closer to video games. I think violence feels different in every medium. And in film there is a very traditional way that it’s done and the way it feels, and we were trying to move closer to video games.

GTA [GRAND THEFT AUTO] created a whole language—it’s a game, but it’s kind of cinematic. Playing GTA feels like watching a Harmony film because it’s this non-narrative thing. You’re like a wanderer, you know, just existing.

You’re partly in control, partly not.

Yeah. And we are working on a new film now called BABY INVASION that is even more connected to the language of video games. It’s shot all from first-person perspective. But I think AGGR0 DR1FT is more like the GTAs of the world—open-world and kind of world-building. And when it comes to anime, I mean, I have my absolute favorites. I love SAMURAI X and COWBOY BEBOP and things like that. My childhood is very connected to DRAGON BALL: it’s probably the ultimate thing that inspired me.

After AGGR0 DR1FT, EDGLRD went from this concept in Harmony’s mind to a company. Even though we’re always making Harmony Korine films, video games are really what we’re pushing towards, you know? We believe in it as a storytelling device that will push the language of film even further than film is now. And AGGR0 DR1FT and BABY INVASION are experimentations in this process of finding the language of entertainment from now on.

What game are you playing at the moment?

I was playing a lot of HELLDIVERS. It’s basically like a video game version of STARSHIP TROOPERS. A lot of us in the office were playing it. It’s a multiplayer game which is great. So we were even having very interesting work conversations discussing projects as we played HELLDIVERS. Not during work hours! And Harmony’s kid is here constantly playing FORTNITE. This connection with the youth is really important for him to understand what the youth is doing and seeing how they interact with streaming in general, like the fact that they watch Twitch streamers and not traditional series or films. We really try to pay attention to that when choosing our path.

Still from AGGR0 DR1FT. Courtesy of EDGLRD.

AGGR0 DR1FT also has a hip-hop vibe—the dancers on the boat feel like they’re from ’90s music videos.

Harmony follows everything that is happening in hip hop, and in horror films. He really pays attention because he thinks those are the spaces where creativity still exists. They are mainstream, but they are also allowed to be experimental. I think that’s why we get approached by a lot of hip hop artists to do music videos and campaigns. And I think hip hop and video game language actually have him as an inspiration in a lot of ways and then vice versa. I watch his films and I see the seed of so much of YouTube culture, and TikTok culture is in TRASH HUMPERS and SPRING BREAKERS and even GUMMO.

Ultimately AGGR0 DR1FT all comes back to this assassin character who’s ruminating about his longing and frustration. How do you work on character and keep that soul in it?

To me, character is everything. I think characters are what make things iconic, relevant, interesting. So with AGGR0 DR1FT, and I think with everything Harmony does, but with AGGR0 DR1FT particularly—I won’t speak for him—[but I think] there’s probably a biographical element. This idea of this character that is really good at something, but this thing can be disrupting to his life and to his family. And this idea that you’re really good at something, but you almost reject it. So my goal was to contribute not just to the film visually, but also to the crafting of this character.

I’ve been to the screenings where people react, people laugh, because the character feels video game-y, I think. He says things that are funny, like, “I’m the best assassin in the world. I’m the humble assassin.” And it’s these contradictory things that I fell in love with, because [Harmony] allows him to contradict himself in a way that I think film writers don’t. And he comes across as very human—he has a real human conflict in his life and a wife and kids. There’s complete internal chaos that manifests visually in the world and then within him and in the things he says.

More from Sloan Science and Film: