Apple Vision Pro Stagnation

Apple Vision Pro Stagnation

Byrichard

People want us to see the Apple Vision Pro VR kit as revolutionary but it isn’t, for a simple reason. Several years ago I was going to the World VR conference and loved playing with various VR kits but they almost all had the same problem. They cost an arm and a leg to buy.

The HTC Vive was an alluring device because of how well it worked and how good it felt but it was made unreachable by its price. The same was true of the Oculus devices, except for the Oculus Quest 3. This had the best price of all. It came with a cost. The Facebook environment. If Facebook can mind duck us(intentional spelling) in two dimensions then imagine how much worse it will be in AR.

The Apple Vision Pro has the same problem as all the others. It will cost 3600 USD or more. This is too expensive. It’s yet another VR device that is pricing itself out of the market for normal people.

VR is a fantastic medium and I love using it, but it’s far too expensive. In some cases you need to spend 800 CHF on a phone, and then yet more on the headset adaptor to wear the phone within the VR headset.

For VR, AR and XR to be interesting they need to be made affordable. They need to cost 200-300 francs, like the Oculus Quest 3, but without being within the Facebook ecosystem. I would have bought the Oculus Quest 3 a few years ago, if it was independent, rather than owned by Facebook.

In a few months, when Apple Vision Pro start selling we will hear the evangelists sing praise for the device but they’re singing praise because they’re among the privilieged few, either to have access through their job, or with the financial situation to spend thousands of CHF on a headset.

And Finally

Some people hate VR and AR because they feel that it isolates people. Others hate AR and VR because it makes them feel sick. Yet more people think that wearing a headset will cut people off from their environment. I see the appeal in AR/VR and XR. What bothers me is that every company makes it unaffordable.

For VR content and games to become more popular AR/VR and XR need to be affordable. Apple, HTC Vive and Occulus aim for the unaffordable portion of the market. We need something affordable. Apple would be revolutionary if it made things affordable, rather than for the wealthy. VR has been around for decades, and it is still priced out of peoples’ reach.

Sea of Tranquility – Snorkeling VR by Pierre Friquet
| | |

Sea of Tranquility – Snorkeling VR by Pierre Friquet

Byrichard

During the World XR Forum this year in Crans Montana I helped Pierre Friquet with his Sea of Tranquility VR Experience. This VR experience was unique in that it required you to be either in your swimming clothes, your underwear or other.



This was a VR experience where you went from being outside where the temperature was descending to winter temperatures to what felt like a warm room when dressed but comfortable when wet.


The experience is simple. You change from your conference clothes into your swimsuit and step into the pool. You stood on a ledge with a depth of just 75 centimetres and received a short briefing. “Are you familiar with snorkelling?”, “Are you familiar with VR?”. They then stepped towards the rope and a tethered floatation belt was brought to them. They placed the belts around their hips and when ready the VR headset was placed on their head, the snorkel was placed, and then a headlight. “When you’re ready to let me know and I will start the video”.



As the video started to play those experiencing The Sea of Tranquility VR experience leaned forward and assumed a horizontal position and floated towards the deep part of the pool. From here they could look down. They went from being in a room if they looked down and started their trip towards the moon.


The audio they could hear was mission audio from Appollo 11. They could turn their head if they quickly wanted to look around but they could also turn by thrusting with their arms either left or right. Instead of a swivel chair or a wheelchair, their support was water.


I personally tried the experience once when tethered and then again when free. During this experience I wanted to swim down and so did others. Two others and I tried the experience untethered as well. The desire to dive down was strong so two of them did.


In one case I saw that a person was uncomfortable with snorkelling so I brought her back to the shallow part of the pool and she enjoyed it. Usually, this VR experience is in 1m20 of water and here people were in 2 meters of water. That’s why I brought the person who looked uncomfortable to the shallow end of the pool.



The torch that people wore on their head added to the experience. As they watched the video and moved in the water so their head turned, and as their head turn you could see that they were looking around. When two people were in the water at once it was quite entertaining.


When some people finished the experience they were so absorbed by the video and the VR experience that they needed to re-acclimatise to reality. In one case it took several minutes.


In several cases, people tried the VR experience and then went for a swim. In one case I did several laps with another person and that was one highlight of the event. It’s the third time I go as a volunteer to this event and I think that between sleeping in a bomb shelter, helping people experience VR in water and by having regular meals this is the best World XR event I have experienced to date.


As a bonus, some of us got to try two other VR experiences. One of them was floating through the ISS but the one I really liked was the scuba diving demo video. In this VR video, you are descending a slope and as you look right and left you to see fish, sharks, a wreck and more. At one point you get to the end of the slope and get to a wall. You float over the wall and then you’re in the water with three whales. As a diver, this sensation was familiar, as was looking around like this.


I really like snorkelling VR experiences and I would love to experience one with scuba equipment. Imagine being underwater, rather than floating on the surface. Imagine being able to dive down and experience the depth, as well as weightlessness.


Zwift events, a social ride and a race

Zwift events, a social ride and a race

Byrichard

Today I participated in two Zwift events, a social ride and a race. The social ride was one hour of pedalling at a comfortable pace trying to keep to the same speed as the group. Rather than trying to be as fast as the group I was trying to pace myself to be within the peloton rather than riding off. 


Social rides are amusing because you start on the peer and wait for the counter to get down to zero and when it does you go from being on a home trainer in the real world cycling on a home trainer in the virtual world to cycling on the roads of Watopia. Social rides are interesting because you have people from Australia, England, Denmark and many other countries, so long as they are in a timezone where it would be reasonable to cycle at such a time of day. 


It’s the first time that I get out of bed, look at the schedule for Zwift events and feel excited about doing something virtual like riding a bike. Keep in mind that today I could have burned diesel by heading to St Triphon to climb with people I know. I felt no motivation to drive that far so I was happy to stay home. 


Don’t be fooled into thinking that riding a bike on Zwift is riding an apartment bike because it isn’t. It’s similar to riding a bike in the real world, with real hills, virtual drafting, virtual fellow drivers, no cars and more. What is real is the conviviality, the desire to share a few phrases, the cheering and the desire to ride as a group. 



During the ride we did sprint twice and on the first sprint I beat my personal record and on the second go I had less energy remaining. During the first sprint I put out up to 934 watts in theory. That’s enough energy for me to rest on the handle bars for a little bit before catching my breath and continuing. 


When I ride in the real world it’s usually alone and I race against myself but when it’s on Zwift I have a choice between virtual social rides, virtual races, training, or just a “as the wind takes me” option. Of course there is no wind, I mean that you ride for the pleasure of riding. 


A Race up a Volcano



The second ride of the morning, about 15 minutes after the first ride finished was a 15km race up the side of the volcano on the Watopia map. I thought that with my abilities I should select group D. I pedalled during the countdown to keep warm and a few seconds before the race started I pedalled at a higher cadence to compensate for the lag that I experience with my current setup. I started well but I tired more quickly than most. While I was trying to maintain 200 or more watts they were racing off into the distance. I found myself in 110th or lower positions but I continued pedalling. I tried to maintain between 180 watts and 220 watts. I kept looking at the riders beside me and most were category C riders.


Keeping up


I worked hard to keep the power up and at moments when I felt that I needed to rest I changed gear, changed my cadence, but tried to keep the cadence up. Usually my cadence is about 85 when I’m riding alone but in this race, as I tried to keep up it reached an average of 105 strokes per minute. Over the duration of the race I was generating about 2.61 watts per kilo. 


I didn’t know what to expect from the other racers so my goal was simply not to be dropped into last position. After that and as I got closer to the end my goal was to be within the top one hundred so although my body wanted me to slow down I kept working at it. I was pushing at over 200 watts for 14:45 in this race. 


I spent hours and days training on Zwift before I felt ready to try a race. I trained for multiple sessions and those sessions got me used to pushing beyond my comfort levels for minutes at a time and it paid off in this race. If I continue training I can aim for better positioning. Today I was in 6th place in D category out of over 40 riders and 99th out of 180+ riders. So half of the racers have trained more than me. I can continue training and see where I come up next time. 


What I like about the race, that I don’t have except for sprints is information on how far it is to the end of the race. If I know for how many more kilometres I have to make an effort then psychologically I can push harder. This was just a short race at 15km and I would struggle on a longer race and I would struggle on the flat. 


I want to participate in more races and I need to choose which training course to attempt next. It’s nice to race virtual cyclists rather than real cars, e-bikes or people on lighter bikes than the one I ride in the physical world. In the virtual world today I was riding a trek emonda. This might be geeky but it’s pushing me to be fitter so fewer people can object. 

|

A 360° cooking Show would be interesting to watch.

Byrichard

For a few weeks now I have been thinking about how you could make a 360° cooking show. For this video I would like to be able to see the process from an angle where I see the person cooking. I would also like to see all of the ingredients and the preparation of various stages of the recipe. For this you would need an open plan kitchen where preparation takes place in the middle of the room.

If it was to be filmed with just one camera then the camera should be lower than eye level but not by much. I would want to look straight ahead in to the eyes of the person preparing the meal and talking. As the person speaks about ingredients I would like to be able to look down and see all of the ingredients. You would need enough room for chopping and marinating. I would think about having a camera above the cooking surfaces so that we can see how the textures change from the start of cooking to when the mix becomes “saucy”. That’s how they describe it in recipes.

I want the 360 approach to be justified. I want to make it so that the viewer has a reason to wear VR goggles. The video above justifies a 360° video whereas the video below does not. The camera needs to be placed as close to the action as possible. It should be designed so that the viewer has to turn his head to see what is happening.

This video does not justify the 360 approach because the action happens in front of the camera and although you can look around the background is not relevant. The action is too small on screen. You can’t see what they are doing.

The camera operator and the producer should learn the recipe and identify what the stages of preparation are. They should have a shot list of every thing that is relevant and place the camera accordingly. Is an oven or a grill used? is sauce prepared? Is something timelapse friendly?

A few years ago we were up in a swiss chalet with a friend and we decided to cook a pizza with a log fire rather than an oven. We started the log fire and then we placed the pizza next to the fire. As the fire was only on one side we rotated the pizza to get it to cook in full. In watching the playback you see the pizza cook as well as the logs go from being logs to embers and “melt” downwards as they burn. If the camera can safely be placed close enough to the fire then you could look down at the pizza as the ingredients change appearance and look up to see the logs decrease in size.

I believe that 360° cooking shows have a future. In theory you need just one well placed camera and editing is cut down to a minimum. Instead of editing and vision mixing from two or three cameras you need just one. I believe that the programme should not be more than 5-10 minutes. People, for now, are still getting used to VR headsets and may find extended periods of time more tiring to watch.

How long will it take before someone like Jamie Oliver uses VR cooking simulations to teach people to cook in the real world. With enough time they could emulate real cooking times at different heats so that when you transition from VR to reality you have the right intuition to successfully cook for yourself and others.

Climbing Virtual Reality, Uniform and Grip

Byrichard

Climbing Virtual Reality tuition is an interesting idea. Learning to climb is fun because it is a physical and intellectual challenge. The first step is to familiarise yourself with the sensations of climbing and learning to see where foot holds are and where hand holds are. When you begin the hand holds are easy to find and hold on to but as you progress and attempt more challenging routes you need to develop finger strength to hold on to those holds. Climbing Virtual Reality tuition is an interesting idea.

In some cases though finger strength is not the only challenge. You also need to learn to read the route and to know which hand to hold a hand hold with, which foot to use on a foot hold and how to reach the next hold. You can learn through trial and error which is fun or you can climb other routes and wait until someone who knows the route that has you flummoxed decides to climb that route.

Virtual reality Goggles are an alternative way of learning how to climb a route. In the example provided in the video above Grip by Uniform provides climbers with an alternative. In their example they speak about Shauna coxsey participating in the project and allowing people to film her as she is coached on how to climb a route. In other locations it could just as easily be the route setters.

It’s interesting to see such a project because the question we often hear people ask is “How did you climb that route” and the answer is often “I don’t remember”. With this technology it would be easy to provide people with guidance for the routes they find more challenging.

It is a new and immersive form of tuition which could help people progress and learn climbing skills more efficiently and within a shorter amount of time.

The Immersive Video market is coming of age
| | |

The Immersive Video market is coming of age

Byrichard

I see this as a good, not a bad thing. It means that 2016 is the year where we can experiment on what content and story types work best for VR. It gives us time to establish workflows and intuitions about what subjects will work in VR and which ones are better left to UHD HDR.

People did speak about how VR is more demanding, how short content currently works better than long form content. I think that adventure sports will fill the demand. Climbing videos are five to ten minutes long. They’re ideally suited for VR goggles because content is usually short and the content is dynamic. Base jumping, rock climbing, windsurfing, kayaking, snowboarding and other sports are well suited to the format.

Immersive videos are coming of age with dozens of headsets being made available. Samsung Gear VR costs 109 CHF, the Homido 79 CHF and the tepoinn around 19 USD. HTC Vive and the Oculus rift in contrast cost hundreds of CHF each and can be driven by one percent of computers currently on the market. (source)

The Gopro Omni, currently available on Pre-order is being launched at 5400 CHF. Contrast this to the Ricoh Theta S, available from about 390 CHF and you see why I settled for the cheaper solution. The Theta S is ideal to have with you at all times to experiment with whereas the Gopro Omni is ideal for facilities to rent out with an experienced camera operator. Software has been written specifically for this hardware solution which will simplify the workflow.

I look forward to when this technology comes of age, when people feel normal about watching certain types of content via VR goggles. I could easily see events where there is a VR goggles experience available. I also see content and goggles being made available at climbing gyms and sports shops to inspire people to make purchases based on the experiences they would like to experience themselves.

 

The Immersive Video Experience
| | |

The Immersive Video Experience

Byrichard

 

I was at the World Virtual Reality Forum in Crans Montana this weekend as a volunteer. During this time I was able to try many of the demonstrations and get a real feel for the potential of 360° and immersive videos. I was also able to listen to people comment on what they appreciated about the experiences and what they did not like.

Vulkane in 3D und 360

One of my favourite experiences was Vulkane in 3D und 360. I worked at this exhibit two out of the three days helping two people at a time every three minutes experience this. I had watched a number of videos before this one and when I watched this one I said Wow because the experience was so beautiful. The quality is excellent and there are some vistas that are spectacular. We are familiar with watching volcanic eruptions but with this experience we are watching the scene as if we are there. As the volcano erupts we can see the projectiles and follow their course through the sky. This is a novel experience for many of us.

Chernoby VR project – 360

Another experience that I got to try and received good feedback from was the Chernobyl 30th Anniversary experience. In this case you launch an app on the mobile phone and you can see a number of videos. You have interviews, videos of locations and interactive content. With this experience you can listen to an interview with a woman who still lives in Chernobyl. As she talks you can look around the room in which she lives. This provides you with a better understanding of whom she is.

Reframe Iran

Reframe Iran is another 360° immersive documentary experience. This experience was appreciated by some and disliked by others. If you study documentary making you are familiar with Cinéma Verité, direct cinema and the fly on the wall concept. In this documentary a 360° camera was put in between the journalist and camera crew and the artist whom is speaking. As the artist speaks you can look around the room. You can look at the journalist, at the artist’s work, the couch in one corner or the bookshelf. You are in the room with them, like a fly on the wall. If you appreciate direct cinema and Cinéma Verité then you will enjoy this documentary.

DEFROST – The Series

Defrost was filmed using the Nokia Ozo and puts you in the place of a woman who was cryogenically frozen and then reanimated. For this experience we wore Samsung VR headsets and sat in a wheel chair. We were the re-animated woman. The experience was interesting because it allows you to feel empathy for whom you, as the viewer are meant to be.

The Difficult People Project.

The difficult People project has as it’s aim to provide people with an immersive experience in to the world as perceived by people with a different perspective on the world. In the first film we saw the world as someone would with hyperactivity. A second video was produced for the World VR forum and showed the world as someone with OCD would see it. This is a project that I personally contributed to with sound recording. The aim of the video was to provide you, as the viewer, with an experience of what it is like to go shopping with the disorder. You see the coping mechanisms via her interior monologue. You hear her worry that people are watching her, that people will see that she buys three of everything and that she may steal if she is alone in the aisle.

On the first day of the conference I was helping people with this experience and as I received feedback I started to see 360° or immersive videos as literature because of the way in which it enables people to feel empathy for the subject of the video. Some people said that they were curious to see whether they suffered from the same problems.

Viens

Viens can be translated as either “come” or “follow me”. I saw this as art, as an experiment which took advantage of what 360° videos have to offer. The camera is hanging from the ceiling and you see a group of people standing around the camera. At first they are hidden behind plastic but as you get further in to the video you see nude people standing. As the film progresses the camera changes position. At one point the camera is lower and you are looking up at giants. At another moment you seeing the action from above.

In each sequence you have the choice of which action you most want to watch. In this experience you choose what you want to see, rather than the camera operator or director. You are in control. You are the camera operator. This video really demonstrates the power of 360° videos. It demonstrates how immersive film making is an opportunity to think about what works well as an immersive experience and what works well as conventional video content.

The new role of camera operators

Immersive video content requires the content maker to think as a camera operator, to think about what is in front of the camera and what is behind it. It requires a new way of lighting a scene and a new way of recording sound. Wireless technology makes this much easier. In hallway conversations I heard people speak about how important it is to think as much about what is going on in front of the camera as much as behind it. If there is nothing interesting to see behind the person then normal video is enough.

Content duration is also relevant. Virtual reality and immersive experiences are still new to people. The experience can be tiring so short content is best, for now. When people get used to immersive story telling program length can grow.

The challenge now is to find subjects that lend themselves well to immersive story telling.

 

| |

Virtual Reality Goggles and multicamera Production

Byrichard

I have worked with video cameras, from hi8 to MiniDV, Beta SP, SX, DVCAM, XDCAM, AVCHD and other formats. Cameras have grown and shrunk, controls have changed from manual to partially automated to fully automated. Television news and Studio camera productions have gone from three or four camera operators to needing a couple and then a single camera operator sitting in a side room with controls for all three cameras. Crane and jib moves are programmed so that the same action is performed at the start of each news program.

Virtual Reality technology and Virtual reality headsets are going down in price. Apps provide mobile phone users with 360° videos in normal vision and 3D. The technology we use to watch 360° content and immerse ourselves in the VR world could be adapted and made suitable for multi-camera production.

It would be nice for software to be written that moves the camera as we move our heads. This technology is already used by gunners flying in Apache helicopters. The point would be to adapt this technology to camera operating. I would manual controls for zoom and focus and a control  to lock off the camera once the desired shot is ready.

Imagine how much simpler controlling drone and crane cameras would be. Imagine also how much nicer it will be for conference attendees, concert goers and UN delegates if a smaller remote controlled camera could be used. Camera operators often obscure people’s view. This technology would be less intrusive. Camera operators could sit rather than stand for hours at a time, barely able to move.

VR goggles and the technology they contain should not be used just to consume a finished product but should instead be used as a creative/production tool. VR goggles and related tech could be used to simplify people’s work, to make it more intuitive. Multicamera production with VR goggles would reduce costs and make high-quality video coverage achievable even for modest budgets. The excuse for using a single webcam to Livestream an event will be gone making virtual attendance of events more enjoyable.

|

Video Editing in a virtual Reality environment

Byrichard

A few years ago I said that I would upgrade my mac book pro when apple came out with a dual display laptop where the keyboard is a touchscreen display that changes to suit the application in use.

Two days ago I had a change of heart. I do not expect Apple to come out with such a device. I have set my sites on a different idea. An edit suite which requires VR goggles to use. The edit suite could be used either by standing people or sitting people. Turn your eyes to the left and you see the rushes. You would have a choice. The rushes could be shown as keyframes on a board or as film strips. Motion feedback gloves would be worn that provide tactile feedback. Double click and the selected clip appears in the player window. With current editing systems keyboard shortcuts are learned and memorised by editors to avoid using the mouse. In this case specific finger positions would be short cuts. You could trim, splice, insert, overwrite, make multiclips and more. Sound and vision could be faded as if using physical controls.

Imagine multicamera editing in Final Cut VR. The setup would be like in an OB van except that you’re in an edit suite or in a park. You could even be sitting in the back seat of a car. The beauty of such a setup is that monitors and displays could eventually be removed from the desk to be replaced with goggles and a pair of gloves.

This means that the same edit suite could have a virtual 64 channel audio mixing board for sound technicians, colour correction wheels for colour graders, vision mixing console for vision mixers and standard video editing controls for video editors. In theory we could go back to the jug/shuttle controls from linear editing days.

I now look forward to seeing whether Avid, Apple or Adobe come out with the first virtual audiovisual creation suite. Imagine how immersive the experience would be. Enjoy the notion that this virtual environment will allow you to stand or sit down.