Technical Documentation: [Link]
Post-Human Bodies
Post-Human Bodies was an experimental Virtual Reality (VR) project to explore the possibilities of alternative methods of control in VR and to make greater use of the underused functionality available to the human body in digital space. The goal of this project was to create a prototype of a control system in VR that requires the body to be used in an unorthodox way. E.G. controllers and trackers being used on shoulders, elbows, knees or hips instead of only in hands.
TASKS
VR Design & Programming, Unity
June 2019
DATE
Post-Human Bodies was a project focused on exploring the concept of homuncular flexibility and embodiment in VR as described in "Homuncular Flexibility in Virtual Reality" (Won et al., 2015). Homuncular flexibility is the brain's ability to adapt and control the body with precision under non-standard conditions. Homuncular flexibility has been documented in amputees or stroke patients, where in situations such as losing the use of a hand a person may become extraordinary dexterous with their feet. With this project I wanted to see if was possible to push a sense of human embodiment in a distinctly alien body by using VR to provide visual feedback and situating the player in a non-standard seating arrangement to provide physical feedback.
Early concept art for the proposed subject. In order to further reinforce the concept I physically demonstrated potential seating arrangements the project may hope to utilize in my initial pitch presentation. I wanted to utilize "hockey puck" VR trackers, which could be attached to the body of the player in a non-standard arrangement, such as on the knees, elbows or shoulder blades. My research had shown that this physical feedback may have a greater influence on feelings of embodiment (Steptoe et. al. 2013) as well as open up new kinesthetic uses for the human body in the digital age.
Post-Human bodies was an ambitious project and my first foray into developing Virtual Reality (VR). When work began I was using an HTC Vive 2.0 in order to access the Hockey Puck trackers, right as the then new SteamVR 2.0 SDK released. While this meant the technology was new and full of potential it also meant that there was little to no documentation available for a novice to read. At the time developers were very precious with their code as any new discovery was potentially marketable.
The 2.0 SDK also made a big assumptions which directly contradicted the core principle of Post-Human Bodies, which is that every player needs hands. While undoubtedly valuable for other developers who did not need to worry about developing accurate hand-tracking solutions it made my task more difficult, as I would have to first undo the hard work of the SDK in order to implement my intended interactions.
This meant I would have to reduce the scope of the project significantly. I decided to focus on implementing a simple but alternative movement system as a proof of concept given the new circumstances.
I settled upon using a Jellyfish design. The initial prototype design focused on a jellyfish body, as it was well received in my presentation. I decided to change the design from a squid to a jellyfish as it would be a simpler form of movement. The slower flapping motions would make it easier for the device to detect and register the movement more accurately, while reducing the margin for error.
The design of the jellyfish included large flaps, to make for an easy analogue to map to the controllers for the players hands. These flaps would also communicate how the player should move their hands to propel themself, relying on the players assumed knowledge of how flippers function
I modeled and fully rigged a 3D jellyfish in Maya to serve as the player's body. I created some simple movement animations then imported the jellyfish into Unity.
Please ignore my Unity scene asset names.
I created a simple test scene for the Jellyfish. It was here that I learned the rigging had failed to translate into unity, and even the placeholder colour of the jellyfish was not successfully exported. This project was not appropriately set up for VR development, and was merely a means for testing the jellyfish model and its animations, which had failed to make the jump from Maya. Undeterred I created a new VR project, and recycled the jellyfish flaps as a hand alternative.
This was my first attempt to repurpose one of the SteamVR sdk 2.0 velocity scripts. The Idea was that if I was unable to move the player, perhaps the entire environment should move instead. This would create a sense of relative movement. To implement this I made an ocean object and allowed the player to throw it around to replicate swimming. However the position of the ocean object was fixed while its rotation was not. This had unfortunate results.
This video shows the first truly functional prototype. The player hands have been substituted for fins and the player can "swim" through the small level. I placed a jellyfish in the scene parented to the ocean, to give myself a goal to try and use the movement system to reach the position of the other Jellyfish. As demonstrated here slow motions were required for accurate tracking, which in my excitement of a working prototype I seemed to forget. Regardless, it did somewhat provide the sensation of swimming/flying as I had hoped to achieve. The ocean being a large cube however did detract from this sensation with its hard edges, and my next iteration aimed to remedy this. This experiment also resulted in the creation of an accidental art house short film, about love and futility.
Latest prototype, the ocean object was changed from a cube to a sphere to remove the distracting edges. I used some particle effects to generate bubbles, as the player moved the bubbles would blur past, providing a sense of momentum as well as some visual interest while the player was not moving. This clip features a combined perspective of inside the game as well as the physical actions I was taking. Please ignore the mouse pointer.
What I learnt
My biggest takeaway was:
Overscoping / Underestimating the difficulty of the task. I did successfully discover a gap in knowledge but was unable to execute on it. VR is new and always changing. Unlike other game development, it did not have the level of support and documentation I would usually have expected when I became stuck. Even using popular technologies did not alleviate this issue, there was still so much unknown, or unoptimised in VR development. Regardless, I enjoyed the project and the ability to experiment, ultimately finding alternative solutions to my problems simply because I did not know the “correct” way to go about a task.
What I wish I could have done better:
I feel that while the project was successful, I forgot some of my own advice. I was unsuccessful in having the additional trackers be used as the main form of controls. The game ultimately still required the use of the standard vive controllers and for the player to be standing upright. This system could have been repurposed to be played while lying down instead of having the player required to stand. Additionally I had become impatient from long hours of testing, resulting in overly hasty movement. My own earlier design documentation shows I had planned for slow methodical movements to reduce margin for error. I feel that ironically I was too focused on the digital aspects of the project and neglected the kinesthetic aspects.
If I had more time to work on Post-Human Bodies:
I would not tackle this project alone, I believe a developer more dedicated towards specifically programming for VR is necessary to overcome the difficulties and challenges it presents. It was naive to believe I would with no prior VR programming experience develop a fully functional prototype which achieves novel methods of control new to the VR scene. Based on my conversations with other programmers it was suggested that the way I structured the project was reminiscent of normal 3d game development and not how a VR game should necessarily be structured, and as a result would require a complete overhaul in structure. It was a tough thing to hear that any progress would require starting over but I have come to accept it. I do not believe any additional time would have been of great use to the current iteration of the project, and it would be more wise to take lessons from this for any future attempts. Overall I still believe it was a valuable experiment and enjoyed exploring this new field.
References:
Steptoe, W., Steed, A., & Slater, M. (2013). Human Tails: Ownership and Control of Extended Humanoid Avatars. IEEE Transactions on Visualization and Computer Graphics, 19(4), 583-590.
Won, A. S., Bailenson, J., Lee, J., & Lanier, J. (2015). Homuncular Flexibility in Virtual Reality. Journal of Computer-Mediated Communication, 20(3), 241–259. https://doi.org/10.1111/jcc4.12107