In my startup Immotionar, we are working since 2014 on adding full body of the user inside virtual reality, giving him/her the ability to see himself and to use all body to interact with the virtual environment. But… why full body virtual reality? How to obtain it? And what are the reactions of the user? Let me tell you my experience…
How to obtain full body virtual reality
First question I want to answer is HOW. How to give the user all his body inside virtual reality? Well, main methods are four:
- Suits: exoskeletons like AxonVR or professional motion tracking suits with lots of Vicon/OptiTrack markers are the way to obtain high-end body tracking, with high reliability and precision. Optitrack is the system used in high-end VR parks like The Void, so we’re talking about a solution widely tested and approved. What is the problem of these solutions? Well, first of all they require you to wear a suit made of sensors (and that’s bad and uncomfortable); then they cost a lot… and with a lot, I mean A LOT. Talking with a virtual reality park owner, he said to me that for $50K they made him barely see the suitcases containing the tracking sensors! 🙂
- Worn sensors / IMU-suits: PrioVR, Perception Neuron, Notch, Enflux, all these companies make some kind of wearable inertial sensors that you can wear on different parts of the body, to reconstruct its pose. The more sensors you put on your body, the more precise tracking you get. Price is moderate (PrioVR suit costs $1200) and performance is good. The bad side is that they require you to wear these sensors and that these suits usually require some kind of calibration to be used;
- Vive Trackers: holding Vive controllers in hands and putting 2 Vive Tracker on feet, plus one on the waist makes possible to have a good-enough emulation of full body for VR games. This is possible thanks to a magical stuff called Inverse Kinematics (IK), that given the position of just a few points of the body makes possible the reconstruction of the most probable pose of the whole body. Vive has recently open sourced the code to obtain such emulation and magazines like UploadVR are pushing a lot this solution. The advantage is that it is quite cheap (300$ for 3 trackers, if you already own the Vive) and it uses directly the SteamVR framework, so SteamVR games could employ it natively and people having Trackers at home could use it easily. Disadvantage is that it requires you to wear Trackers on your body and that the reconstructed position may not be the real one;
- External cameras: using one or more cameras around the play area, it is possible to reconstruct with some computer vision magic the pose of one or more users, without them wearing anything. So it is possible to obtain full body VR just configuring the system once and then capturing the users without them wearing nothing. This is the solution we have chosen at Immotionar, where we employ Kinects; and also by VicoVR, which uses Orbbec depth sensor and a proprietary body-tracking algorithm to offer full body VR for mobile headsets. The problem of this solution is that both hardware and computer vision algorighms are still not ready for this application (have not 90FPS and millimeter accuracy required by virtual reality). From the video below you can see the difference in accuracy between Oculus Touch and Kinect tracking.
We chose the last method because in our opinion that will be the future way of performing full body VR: wearing a suit (full or partial) every time you want to play with VR is awkward and boring. We all want to put on only a headset and then play. This is even more important for hygiene issues: if in exhibitions you make people to wear suits, you have to sanitize them between consecutive usages. The problem is that it is hard to perform a reliable body tracking only with external cameras (and furthermore the Kinect is almost dead!..)
Why Full Body VR?
Ok, now you know how to obtain full body VR, but you may still wonder “Why bother? With VR controller in hands I can already live fantastic VR!”.
Figure 1: Me while playing full body VR mixed with Vive Controllers… so much fun
I’ll start explaining that telling you something that has happened to me: first time I played with Oculus First Contact, I was having fun grabbing and throwing Coca cans using my brand new Touch Controllers. Suddenly a virtual can fell to the floor and I felt the instinct to kick it away. I did it, but then I suddenly realized that I had no feet in VR… I could not see them, I could not feel them, I could not use them: this moment has been a presence killer.
The first reason we need our body in VR is for presence. We have our full body in our every day life and we want to see and use it even in VR. But in VR, everytime you look down you see nothing and that reminds you that you’re just in a computer-generated world.
The second reason is for natural interactions: while it is true that we mostly use our hands for our everyday life (unless you’re in the fetish sector :D), it is also true that we naturally use our feet for walking, kicking stuff, performing exercise in the gym, etc… And let’s remember that “full body” does not mean only feet: we use our full body to dodge stuff, for example. Enabling full body means enabling in VR all natural full body interactions that we do each day (and yes, kicking a can is one of them).
The third is that it can enable lots of new kind of experiences that are impossible now. You can play kicking soccer penalties, for example.
You can see yourself with the body of a woman (we do that a lot in exhibitions and men laugh a lot) or with anyway a different identity. You can see a squirrel jumping on your feet and then climbing your body until your heart and make you die from VR cuteness. These can be experiences made for leisure, but can be important for other things that simple gaming: seeing your body different from the real one can be very important in psychology or in rehabilitation applications; seeing an animal climbing your clothes can be important for curing some phobias; having all joints of the body tracked can be useful for rehabilitation or for fitness apps (squatting in VR with Terry Crews teaching you would be awesome, I guess).
So, we want full body in VR because we have a body in real life and want to be able to do in VR the same things we do in real life.
Feedbacks of Full Body Virtual Reality
We made some local exhibitions and made hundreds of people to try our full body system. Despite the fact that it is far from perfect (imprecise body tracking due to Kinect errors and such), we’ve always obtained enthusiastic feedbacks.
First thing that people do in exhibitions, when they’re in VR with full body, is looking at their hands: when they see their hands, with all fingers and the arms, they remain all delighted and surprised! Someone also told us that our system is better than Oculus Touch… and even if I know that this is not true (see above video), well, the compliment really touched me.
Figure 2 This is exactly the reaction you get when you look at your hands and you can see them in VR
After they realize that they have the full body, they start to look at it and to touch it: they look at feet, legs and chest. In exhibitions we’ve showed people two kind of avatars: a female one and an abstract one made all of spheres (something like the Michelin mascot).
People compared their real body with the avatar’s one and remained surprised and delighted when it was different: usually men make jokes about having boobs, as my friend and VR-startupper Edgar Pironti is doing in this video (shot when he tried our system for the first time).
Seeing yourself as something different is strange but is also very interesting: virtual reality purpose is to make us dream about being something else, so seeing a different body is amazing on this sense.
People usually understand very easily how to use full body VR, since the only thing that is necessary to do is tell them “do as you would do in real life”: if you see a ball, just kick it with your feet or punch it with your hands, it’s ok this way.
We also made some surveys about all of this and data we get confirms all these impressions.
Figure 3: Our first exhibition (WTT of Turin) and the interest we got from people: someone has stayed in line for 2 hours to try our full body VR system!
Well, this is what I’ve learnt in 3 years of work in this field. When we started there was not a great interest towards full body VR (there was not even the Vive and room-scale), while now more and more startup are working in this field. I truly believe that in some years we’ll have wireless full body VR with no worn sensors and that we will be able to experience true virtual reality. As I love to say “VR without full body is not true VR”.
9 Best Samsung Gear VR New Controller Compatible Games & Experiences