Weekly Topic Let’s Discuss Cinematics and Storytelling in VR

Sundance’s New Frontier this year featured a number of VR film and experiences that were Made with Unity. There were a few articles that popped up around the VR experiences there, if you’re interested in reading about them.

1 | 2 | 3

So, what do you think? Can VR films be a successful alternative media comparable to traditional cinematics? Are you working on any VR experiences/narratives/films yourself? Where do you think this technology is heading? We’d love to discuss this with you!


| Topic Index |

 

First link doesn't work for me.

Regarding the discussion - what are VR films anyway ? are they films where you can look around in 360? or something more interactive ?

 

Fixed!

Great questions! With such a new medium it's hard to find a really standard way of defining VR films, which is part of the fun of it I support.

You have animation studios such as Baobab Studios who have a very driven narrative that allows the player to have some form of interaction with the story. In their newest animated short you get to be a robot that helps with some very small tasks while observing the overall story.

On the other side you have people like Chris Milk who is working on 'Life of Us' which is a much more experience driven narrative that you get to share with someone else in a VR space.

What type of VR narrative do you think you would be more interested in @liortal ? Are you working on any VR projects yourself?

 

I think it's too early myself to say that VR could go on an equal level with how we currently view media but there are certainly some fantastic things out there which give us hope!

Within (who also worked on 'Life of Us' which was on at Sundance) worked on a VR experience for USA's Mr Robot, which was a short based around the protagonists past. I found the whole thing breathtaking, despite it being a live action short.

I think my view overall is that I'm really interested to see how VR as a medium can grow. I'd be more interested in it if things like the above became more common as short stories on existing shows, or a cool way to experience trailers. I haven't had a chance to experience a full movie with a VR headset myself, but I'd love to!

 

I saw that the SyFy channel show Halcyon had some short companion VR "episodes" with it, too. I wonder if (when) we'll see other shows following suit? Haven't tried those myself so don't know what it's like, but it's really interesting to imagine people switching back and forth between "watching" and "interacting" with TV or movies, especially if we can get to the point where it can happen fairly seemlessly with the hardware (which won't be any time soon I guess). I got to try Asteroids! and it was nice how they clearly show you when you're just an observer and when you are able to interact. The way they built that into the story was very clever.

 

Although I have very little to add to the conversation about VR in 'movies' or film, or VR in general because it's still new tech to me, I was drawn to the discussion by the thread title - cinematics and storytelling which is right up my ally as an animator. I think VR in general is intriguing for these specific aspects, and less so for the novelty and new shiny object draw of the tech.
Although I've only had two separate extended 'play' sessions within VR, these moments have driven me to write proposal papers set in the restricted confines within the world of VR. Note: Proposal papers are part of my own personal vetting process for greenlighting game concepts - because I'm weird and I like to write designs in addition to development.
And even though I describe this world as restricted - I see potential in enhanced narrative, cinematics and storytelling within VR which can never be attained without this type of immersive tech.

So thank you for starting this thread about this topic.

I'd also like to mention - this weekly Topic idea is a really great addition to this community, which I find to always be enlightening with points of view and knowledge I never consider without reading through threads I have no business having access to - because I have no insight into the topics. And secondly this community is inspiring and humbling with its openness to give and listen to opinions, share ideas and creations, and continues to grow together even if we are not always agreeable.

I believe the lack (low number) of comments in this thread are a result of the thought provoking topic and do not reflect a lack of interest in the weekly topic concept.

end of peroration -

 

I think is not comparable to traditional cinematics. Because is something more.
In a film, you are a passive spectator and you do not think that much and absorbs a solution. In a game, you are the active protagonist that give the solution.

I think that VR generates a new hybrid : "Interactive movie" that is not a standard film and is not a traditional game but both. Something in the middle: a new third thing that has a bit of both "Game-Movie" or "GameFilm". Must be explored. Some will tend more to a game and some will be more close to a film where you can make decisions in the storylining. For example: The story line can be inside a house where all the action is happening and you can look to the 360 movies been in different parts of the house. A movie that instead of having one film, can have multiple parallel strings of films, one for each interactive room with actors going in and out.

Like the books where you make decisions, and tells you "go to page..."
Here in Argentina, there is an old theatre play that you as viewer spectator con walk around a house where actors are making a roll... And you finish eating with them and discover that there was a... This kind of 3d show is what I'm expecting from Interactive movies made with vr.

 

"VR Film" Lets state first that I haven't seen a real VR film.

The term VR is overused in the wrong way. "just a 360 movie" isn't going to fit the VR terms.
I've worked for couple of VR companies in the past, and go to a lot of meetups with VR content creators and enthusiasts. But even they say its not really VR.

To be honest we will still use the term VR for these 360 movies and its a great way to advertise your film with the VR mark on it, but its really not in my opinion. To be a valid VRFilm it needs to be actual 3D, not just a plain 2D monoscopic 360 video.


but yes, I see 360movies take over the regular cinema's screenplays. It gives a new perspective to film-making.

Last meetup there was a presentation about "Ashes to Ashes", a 360 video created by WeMakeVR. And I loved the fact you can look around from the perspective of a kid. 360video has a lot of emersion, which makes you feel a part of the film.

 

If it's for cinematics in a game, I'd say keep it as short as possibly to progress the story in the game. I'm one of the many people who can only use VR in limited bursts before motion sickness kicks in, so having to sit through cutscenes while I'm progressively feeling more ill would get in the way of my actual VR gaming.

 

Now THAT is something I'd love to experience.

 

Interesting take, to be quite honest when we initially thought of this topic we weren't thinking of the 360 videos you see on a lot of right now, but instead thinking of something more along the lines of Asteroids mentioned above where you as a viewer are quite literally a character in the film.

Looking at how you define 360 vs VR films, the difference you seem to be making is that VR films need to have a space for the viewer thats inside the scene? Versus a 360 video where there is no viewer awareness? The later being more similar to current film making.

 

No i define 360 as monoscopic, zero depth video creation vs VRFilms which are 360 stereoscopic 3D videos. Its just that the term VRFilm is quite a topic within the VR Industry, we usually refer to such videos as 360videos and not Virtual Reality.

Obviously both have viewer awareness, its just that 360VRFilms aren't really VR unless they are stereo3D.

Anyhow could @aliceingameland send some reference to "Asteroids" ? Did some research on it but couldn't find one

 

"Asteroids" was a production by Baobab Studios, I don't believe they have the full project released to the public yet.

 

I'm a bit on the fence with VR movies. I mean, the stuff by Baobab is great - but a lot of movie making is about framing shots the right way and setting up scenes out of frame. Giving a doofus like me control over the camera is probably not the best idea there is. However, giving the user more control - making it slight more interactive - now there's an interesting idea. Being caught in desolate cabin somewhere in a forest, which weird noises outside - leaving it up to me when to open the door and actually seeing the horror that is waiting for me outside... that could work. rather well. The medium has a lot of very interesting possibilities, for sure, but I think I'd want to see it move more towards games and interactive experiences, but with a similar narrative.

 

I dont think VR movies will be great. One of the best things about VR is that you can be in your living room and have the experience of being across the world. Going to the movies is something that most people can do regularly or simply turn their tv on. I dont see the "VR" experience in movies. However I think museums and things like that will be a lot more interesting as not everyone can visit DC and go through a museum etc.

Horror video games are going to be very exciting I feel like with VR. Now adays movies are not that scary, and since I have gotten older I dont find them scary but they are still enjoyable. But with VR they will come to life and I will certainly get the thrill that I did when I was younger

 

Wanted to pipe in to say that weekly topic is a great initiative, I just really don't care about VR.

 

I'm in a similar position. I do care about VR but don't feel that I have a lot of useful stuff to talk about on the topic.

To me, film and cinematics are about storytelling or conveying a message. Right now VR is in a very early pioneering stage where the conventions aren't even established for those things, and I'm not spending time researching them myself... so it's interesting, but I've nothing to contribute.

One thing I guess I could say is that I'm not convinced that "VR film" is going to work long term. In traditional film the director has very explicit control over everything, including where the camera is pointed what what can and can not be seen. VR takes that control away, and things are going to have to change pretty fundamentally to account for that. There's a reason that video games still have cutscenes - it's because sometimes it's not desireable for the audience to be in control. So, in short, I suspect that storytelling in VR is going to have more in common with storytelling in video games than it will storytelling in film. (In fact, personally, I see it more as a new IO method for that than I do a new medium in its own right.)

 

I actually worked on 2 really small projects that involve 360 VR Video & Unity.

First one is a bit more "interactive" than the second one that just plays the 360 video. Both work pretty well.
The first one will also be filmed IRL (meaning no 3D modeling, etc). It has some interesting aspects where the video goes into a seamless loop and the next bit of the movie triggers when you look at a particular thing.

Like @Jacob_Unity said: "but a lot of movie making is about framing shots the right way and setting up scenes out of frame". This still holds true in VR to some degree. Don't think like a VR movie ... just as a 360 image... where you basically "see everything". Without going into to much details about what I worked on ... think about this. 2 persons are having a conversation. You are in the perspective of one of those persons. Of course, any VR n00b will just look around like "WHOOOOW VR... WHOOOOOOW 360..> OMG I CAN SEE EVERYTHING", ... but someone who went over that stage will actually be able to get into the role of that character, you become that character. Now, the way this was made ... as I said before, some events trigger only when you look at "something". If you don't look the video will just go into that loop. Now this might seem a bit strange... so here is where that "Framing shots the right way, setting up the scenes" comes into play... and also here is where the "game design" parts come into play. You will need to hint to the viewer that he should look towards a point.

And when the player does that ... he becomes imersed even more into that actual story. You can even pull him out of the character he just was.... the player becoming just an observer... then maybe again becoming one of the characters. There ware a lot of ways you can tell stories now. One small example ... you are having a conversation with someone else... video will go into a loop when the conversation will end... but if you turn on slightly ... you will be able to see a white line separating you from something.... if you turn... and turn... and turning completly 180 ... you will become the person you were just talking to... you being able to see yourself... but you are not yourself anymore, you are that other person.

Anyway... lot's of rambling here. I would love to show you the project... but everything in time I guess

P.S. The whole app was made with Unity, Playmaker & AVPro Video.

 

I only saw this topic today.. where is the new one already... stay on topic time Unity

 

I think this will be a great point of interest for anyone making narratives in VR, guiding the viewer to look where they want them to be looking without forcing it like a standard film.

Feel free to hop into our new topic! Link

 

This sounds a lot like "vamping" in concert/stage performance, except you're waiting for the user to do something instead of another performer.

Of course when a performer needs a moment that's still something that will take a finite, somewhat controlled period of time. With a player that may not be the case... are there any strategies you can share to overcome this?

 

Well it depends on the scenario I guess. You can do hits trough dialog, or the character can throw something in a direction making the player look for that direction... you can maybe play a 3D sound, etc.

 

I've seen developers play with both the vibration as well as audio queues while trying to get a user to look at a specific scenario or interact with something in the scene.

 

I like the idea of segmenting the story and allowing the viewer to choose the pace. Imagine scenes taking place in front of the viewer, after each scene a choice is given to click ahead, go on to the next stage or allow the viewer to linger in the setting and look around. Maybe even explore some new rooms or clues to better understand the plot of the story.

That way, people that just want a straight ahead movie-type experience can get on with the story. Or, someone that wants to look around a cool setting can also do so. Sort of a customized cinema experience.

If budget and time allows, it could be nice to have options for the viewer to make a decision that influences the story. This is where it gets interesting, but also more game-like, think choose your own adventure. The advantage is an immersive experience where the user is involved in the story outcome, but it also means much more work to develop these alternative scenarios.

 

We had a little post-Sundance press event & panel at the SF offices a week or two ago, and co-founder of Baobab Larry Cutler mentioned that they've been taking cues from other art forms to help with that challenge of directing a viewer's gaze. An example he used was that magicians have all these various techniques for directing your gaze -- if you want someone to look at you, you look directly at them. And if you look directly at a thing, people will naturally follow your gaze to see what you're looking at. I thought that was pretty interesting!

It's cool how you can play with people's already learned behaviors when it comes to reacting to even just audio cues. I'm sure if I heard the sound of a bicycle bell ringing behind me while in VR, my body would react by instinctually moving to the side and turning my head to look behind me. That's something you can't really get in a traditional game or film.

That sounds intriguing! Can you share the name of the project so we can keep an eye out for it?

 

Not yet. Soon hopefully

 

VR films will be a successful alternative when we use "Octane rendering within the Unity timeline that can render with photorealistic quality. I wish that all other rendering software can be available in the asset store.List of rendering software: https://en.wikipedia.org/wiki/List_of_3D_rendering_software

 

I really find this a interesting topic, and would love to see more search around this subject. Me and some classmates did a paper on VR and storytelling a year ago. Where we did some interviews with people at Fido (Swedish VFX company) and some other more VR specific companies. asked the questions:


1. What are some difficulties you could be facing when producing VR movies for a feature format?

2. In which interactive way do filmmakers use VR for feature film?

3. How would you tell a story and direct the viewer's attention when there is a 360° field of view according to the experts?

Im gonna check with my teacher if I can post the paper here, or link to it. The paper isn't finished (a section in it wasn't developed enough. And also English isn't our main language, but the paper is written in English.

 

Also with VR you has more capacity. You can make multi films that run at the same time all together in the same scene telling different story telling ("MultiStoryTelling" or "MultiStory") at the same time. As for example, Alien and Predator that in the last scene merge together in Alien vs Predator...

...and where they want to be without forcing them.

 

You are still trying to tell a story, so if you don´t "force" the viewer to make them look where you want them to look, they will miss important moments in the story that could destroy the meaning that you trying to say. With sound and movement you can make the viewer look where you want them to look, without them realizing it the first time.

I would like to know more what you mean with telling different stories, like multiStory telling, how would that work inside a 360 movie?

 

You are correct and I agree: They can suffer from vertigo or suffer by looking exolife alien. They can move back can hit something or fall down. VR is a strong tool must be used in a careful way. So one solution for this is to put the spectator viewer (after miss important moments) in a safe place with other spectators (in the film) where can talk and exchange, explain experiences about what is happening before moving to the next step of the story. Transforming the spectators in participants [using multiplayer]. It can be a car, a plane, a restaurant, a jail. VR can be so immersive that you need to give the option to escape or, if the feeling is to intense.... ! And new solutions to this problems will pop up long the way.

You are looking 360 movie as a universe bubble (with one bubble). You can have more than one 360 bubble that separates and merge in the story line.

 

Wouldn't a step like that take away the immersion for the viewer from the film, if you "take a break" to talk to other viewers. If that is what you mean with multiplayer? I do like the idea tho for being there as a participant inside a movie, but the characters then need to involve me as a viewer. Or it would feel like im a ghost talking a spot inside a car with no one talking or looking at me haha.

Oh right didn't think of that.You could say that you have one giant bubble with smaller bubbles inside, each characters its own little bubble with their own "story".

Here is a link to the google doc, paper we did for school.
https://docs.google.com/document/d/1kHfSrtEr8T-sIgVxZO4PgFoX78g4-n5hM8UkJDCg_Cw/edit?usp=sharing

 

Yes, exactly; Remember that each participant is not looking to a main screen but has it's own VR. Each VR is one reality. Since GPU and CPU will render photo realistic in the future, allows you to improvise new bubble solutions for the film in real-time. Multiplayer let you be in the car with other viewers, talk to them and exchange data: Transforming the spectators in participants [using multiplayer] can talk and exchange, (what you miss before) "take a break" while superman (or main character) flies around or the robot pilot marks warnings without braking the story. In this way participants give you the feedback that you are missing since in 360 degrees there is a lot to look around. The main scene can happen mostly in front in relationship of how is seating or standing. All or parts of the film is rendered in real-time and depend if is a 360 bubble or GameObjects. Because a combination works well for including the mesh of the participants characters that are included in the film. (And will be all finish scanned 3d.)

 

That's really good way to think about it, also very exhausting for my brain haha. Imagine going to the cinema with 50 other people, during a "loading" time you sitt and share experiences in a bus, the people who forget their phones on during a movie will become the people talking S*** during the break haha. Have you done any VR stuff?

 

This is a really interesting topic and something I've been myself wondering for a while. I've been working in Feature Films for a long time (visual effects) and I haven't experienced a VR movie yet that really exploits what VR really has to offers imho. All the VR shorts I've watched so far still use a language laid down my movies, which doesn't necessarily work. Let me explain what I mean. When you make a traditional movie, you know that your audience will be looking at a screen, and you work within that space to tell your story. And within that frame, you carefully design your movie so you always make sure the audience is focusing at what you want them to focus on (using common tricks like composition and depth of field). VR is a totally different medium and, I think, much better because it is a one-on-one experience - so instead of creating a movie that will be seen by hundreds at the time (in the case of movie theatre), you have this intimate relationship with your viewer. You have his/her attention on a personal level.
I personally think that VR movies will be interesting when they start exploiting that aspect of the medium. Additionally, the strength of the medium is also that you can generate it in real-time. Instead of being passive your movie can suddenly become active - what about a story that adapts to your audience on a personal level?
I could talk about this topic for hours as it is something I'm passionate about. In the last game I published, I tried to explore ways to analyze the viewer's gaze and to unfold events based on that information. It is however a game and not really a movie or short, but I'd love to see VR shorts using similar techniques.