New Scientist brings us a report of emerging virtual reality concepts that engage all five of the senses. The ability to create immersive audio and visual experiences is rapidly gaining maturity, but engaging smell, taste, and touch is significantly further off—perhaps a couple of decades or more. But it’s not too soon to begin considering the shape of those future experiences.

Immersive A/V experiences are already demonstrating that creating truly shared environments is exceptionally challenging. Conceptually, many envision these shared environments as variations on the Star Trek Holodeck theme. That is, you and your friends would enter a physical room which would somehow manifest the sights, sounds, and ultimately other senses in much the same way that they are manifested in the real world. However, this is an inherently problematic and difficult (and hence expensive) solution to the problem.

Consider that in order to render all of the sensual information in any virtual reality, the target experiential information must first be captured, digitized, and stored. Initially, this will be done by people trying to capture holistic experiences from the real world, but over time, I expect that sensory snippets will be stored in a vast library for mixing and matching of sights, sounds, smells, tastes, and touches. This snippets will be mashed-up into re-creations of reality or construction of novel virtualities. This simplifies the capture process as I only need to actually digitize the smell of dog poop once. The next time I just need to identify it. The early inklings of this model are already starting to show up related to audio-visual elements.

However, given that I have (or will have) these digitized sensual experiences, how will they be rendered? It is important to recognize that all human experience is ultimately a complex series of nerve impulses received by the brain. If those impulses could be perfectly simulated, there would be no way to distinguish reality from its virtual counterpart. In a sense, this is similar to the Matrix type model, but doesn’t require the context of being enslaved by machines.

It follows that the more transformations of sensory data that are required, the less real the experience will become and the more complex the rendering will become. This gets inordinately more complex when you start introducing multiple people into a shared experience. Consider simply viewing a scene in 3D. If you go to a 3D movie today and a hummingbird flies off the screen and hovers in front of your face, everyone experiences the bird as hovering before their own face. But that doesn’t cut it as a shared reality simulation. I should see the bird hovering in front of your face, and so should you. This means that each person’s perspective would need to be accurately projected into the holodeck in a way that didn’t corrupt anyone else’s view. That’s hard to do without actually animating matter, which is how Star Trek ostensibly does it, and which is a technology more than a few decades away. It’s far simpler if every individual has a contained environment (e.g. goggles) and the computer calculates and renders the perspective views specific to each individual. That same premise should hold for all the other senses as well.

This would mean that minimally, shared reality experiences would best be had by multiple people in individual pods or suits that provide close proximity sensory stimulation. But if we take that just a wee bit further… Futurist Ray Kurzweil talks at length about the coming singularity. A point in the not to distant future where we interface computers directly to our nervous system. In this world, the stimulation required for virtual reality immersion would be transmitted directly into your body.

Obviously this sort of technology brings with it some heavyweight ethical, moral, and sociological issues. It even opens many philosophical issues on the true nature of reality. All of which I’m sure will provide ample fodder for talk shows and Congressional sub-committees for years to come.

However, from a technology perspective, despite how much I’d prefer to live in the Star Trek universe, I believe we are destined for the capabilities of the Matrix universe. Of course that doesn’t mean we can’t still boldly go where no man has gone before. Just that we are likely to do it without leaving the house.