Whatever Happened to Virtual Reality?

If you weren't there, you probably wouldn't believe it. But way back at the start of the 90s, people at the edge of the emerging digital culture talked about Virtual Reality (VR) — the idea that we would soon interact in shared 3D worlds — as much as, if not more than, they talked about the internet. (Of course, we were talking about it on the internet, so I guess sometimes you just don't notice your immediate surroundings.)

These 3D worlds would be accessed through head-mounted displays. The idea was to put the user literally inside computer-created worlds, where she could move around and see and hear the goings on in a fully dimensional alternative reality and have the sensation of being in another world. The eyes were the primary organs of entrance into these other worlds, although touch, motion and sound were all also involved.

(Second Life is a timid sampling of what was then envisioned.)



There were dozens of conferences about VR and lots of national media coverage in every major outlet. There were movies and TV shows that revolved around VR and there was even one arcade game. But VR quickly disappeared from public consciousness.

Virtual Reality developer Jaron Lanier was generally accepted as the public face of VR during this heady period that lasted from about 1989-91. This interview was performed in 2002 and I have selected it as an excerpt from my new book, True Mutations: Conversations on the Edge of Technology, Science and Consciousness.


Imagine Jaron Lanier

I first met Jaron Lanier in the mid-1980s. His work in Virtual Reality was just getting noticed and it was clear to those who knew him that this bright and gentle young man was destined to do great things.

In the late 1980s Lanier's team at VPL (Virtual Programming Language) developed the first implementation of multi-person virtual worlds using head mounted displays. The work was applied to surgery and television production, among other things. He also led the team that developed the first widely used software platform architecture for immersive virtual reality applications. During the late 90s, Lanier served as the Lead Scientist of the National Tele-immersion Initiative, a coalition of research universities studying advanced applications for Internet 2. The Initiative demonstrated the first prototypes of tele-immersion in 2000 after a three year development period.

Lanier is also a musician. He has a remarkable collection of eclectic instruments from all over the world and has worked with Philip Glass, Ornette Coleman, Vernon Reid, George Clinton, and Sean Lennon among others. Additionally, his paintings and drawings have been exhibited in museums and galleries in the United States and Europe.

While most computer programmers and tech engineers display some degree of interest in art and aesthetics, Lanier is really an artist who happens to work in technology some of the time.

"Videobrain" and I sat with Lanier at a Mexican cafe in Marin and had a fragmented conversation that was frequently interrupted by waiters who wanted to please us, and Lanier's own incessantly ringing cell phone. Nevertheless, the conversation proved to be largely coherent.

RU SIRIUS: There was an extraordinary level of hype and excitement about Virtual Reality (VR) back in the late eighties and early nineties but it failed to live up to the expectations, presumably for technical reasons. I saw your The Top Eleven Reasons VR has not yet become commonplace. But why would you say there was that excited response to VR? What were people hoping for from it? What desires were raised?

JARON LANIER: Well, first of all, I personally think that a lot more could have happened with Virtual Reality than has happened. I feel that what went wrong with VR was that decent software standard platform didn't happen. The ones that were most in the forefront like VRML just didn't work well enough. So to get back to your question: what were people looking for? I still believe that what people really want from VR is to be able to touch upon the feeling of being able to share a dream with someone else — to take a little step away from the sense of isolation that people feel today. I think this is a universal and very healthy desire. (VR isn't the only way to address it obviously.)

But in VR, at some point, you would be able to be inside this place with other people where you were making it up as you went along. What people really wanted was a kind of intimacy where you're making up a dream together with other people. You're all experiencing it. I was calling it post-symbolic communication. The basic idea is that people thought that with VR they would be able to experience a kind of intense contact with imagination, some sort of fusion of the kind of extremes of aesthetics and emotional experience you might have when you open up the constraints of reality.



You can divide the requirements of the technology that will give you that into two pieces. You can call one piece the production quality or production standards — how detailed is the resolution? How realistic do surfaces look? That boils down to fast computers, high quality sensors and displays: the tech underpinnings of it all. But then there's this other side; the software side, which involves how you can get a virtual world to do things. My feeling is that even a low-res virtual world can get people the kind of experience that I was just describing. And I think we did have some great moments and great experiences in the '80s, even with very low-res systems that were available then. I think that the failure since then is that the software that's been developed is very rigid.

There are a couple of reasons for this. One was that there was a bizarre alliance between people doing military simulation and people doing recreational gaming. There are a lot of different kinds of games, so I don't want to put them all under one critical tent here. I think a lot of them are OK. But one of the dominant ideas is that a person who is playing is capable of being in the location, moving, shooting, or dying [laughs]. That's pretty much it. You might pick up an amulet or something, but it doesn't give you a lot to do.

RU: But in fairness, don't those kinds of simple applications come up because they're easier to code — so that they're steps along the way?

JL: Yeah, well it's precisely right that it's easier to code, especially when you get into a networked thing, but who said coding was supposed to be easy!? This brings up a little rant. I love to support the free software movement... I totally do. But just doing a piece of software in some political or economic context that's progressive like the Linux movement isn't enough. The software itself also has to be good [laughs]. That's sort of stating the obvious, but there are very large numbers of programmers in this newest, youngest generation of programmers who seem to feel that writing something that has existed for 20 or 30 years is somehow cool if you do it as part of a free software movement. And there is something to that, I don't want to say it's nothing but come on. So I think we have an epidemic of almost tautological coding. It's the same old stuff. There's no surprise. It's like ham radio or something. And that's been the worst problem for VR, because VR really needs a different attitude. Even today, you see people starting up a VR program and after some months they'll have a cube rotating or maybe a videogame where you're moving through a space and shooting at things. It's been done for decades! Do these people not know the meaning of boredom? How can people bear that?

RU: Well, you have the same thing in publishing. But the difference is that you can write something really dull or you can write something really amazing and it requires the same bandwidth. The tools are already available. But to do something really original in Virtual Reality — the steps are not so obvious. Do you visualize a huge project with lots of people working in parallel or how should it work?

JL: The case I want to make — and I can't prove it; it's speculation but my belief — is that even a really low-res system that's sort of manageable by a small group of people could be done that would be much more exciting and bring out more of this feeling of transcendence than what we're seeing now. Of course, anybody could ask me; "So, Mr. Snooty Oldtimer, if you feel that this can be done why don't you just go ahead and do it?" It's a reasonable question and I always think about it. I'm in this sort of bizarre quandary. The code that I really like the best for creativity is my old VPL code. I still use it for my own creative work but I'm not allowed to really work on it because it ended up being owned by Sun Microsystems. That would be a good resource for developing the open source treatment. And occasionally, when I give a talk at some university there will be some students who want to take it on as an open source challenge and make a new generation of something like that. Maybe it'll happen. It's definitely getting harder to use that code because it's old. There are a few things that we now know about how visible systems work that aren't really doable on it. It's hard to keep it running on the new generations of machines; it's really quite a challenge. How do you keep a twenty-year-old piece of software running without re-compiling?

RU: In your opinion, have there been fundamental changes in computer hardware that could make VR software more optimal in the intervening years?

JL: Not much. Just speed. More polygons.

RU The slowness in moving towards more creative forms of VR is a commercial problem also. If there was an obvious immediate market for it, a company with money would be working on it. You'd be working on it.

JL: Capitalism has proven really wonderful and optimal in encouraging certain kinds of improvements in technology but it seems to have these blind spots where it just hasn't been able to give support to other ones which are — at least by my value system — just as important. So for instance, the Moore's Law effect of processor speed going up — capitalism has been really good at that. It probably would not have happened under a command economy. So that's worked out pretty well. But with tools, if there's already an established market, like video editing or sound editing then you have production suites and there's a market. But if it's for something new and the market doesn't already exist, you get caught up in a sort of chicken-and-egg situation even though one can see that if all the pieces were in place there would be an incredible market. Capitalism can't serve as its own starter in a lot of places.

RU: Well, it seems that the contravening force to capitalism in the digital world is the gift economies of open source enthusiasts, which has the added charm of being non-coercive.

JL: Yeah. Well if I can find the personal focus for it I might try to start an open source movement for making VR tools. I should probably do that. It would be courageous.

RU: You've always seen VR in terms of play and sharing visions and so forth. What about the utilitarian aspect?

JL: Virtual Reality is already a success as an industrial technology. It just hasn't hit yet as a communications technology. But it's become absolutely essential. One of the stories I tell is the story of the oil supply. If we go back twenty or thirty years, most people thought that the oil would be running out about now. And the reason it's not is because computers allowed people to find and extract oil more efficiently...and from old fields. Ultimately, there's an illusion — created because of computers — that the oil supply is expanding instead of running out. The underlying reality is that the oil supply is running out, so, in a way, this is a dangerous situation. At any rate, VR was used to visualize oil fields and to visualize machinery to extract oil more efficiently from old fields. Similar things happened in medicine. We understand more about large molecules, we understand more about how the body heals from surgery through VR simulations.

RU: Is there a utilitarian aspect to the visionary ideal of VR?

JL: Whether one sees meaning as having utilitarian value is a matter of personal taste. I think the most important things can't be expressed in utilitarian terms because to be utilitarian you have to have a frame to refer to, and the most important things are the frames. You can't say that your values are utilitarian; you have to have smaller things that are utilitarian within your values. (Of course, some of our values are tautologically unavoidable like survival.)

One way of arguing that there is a utilitarian value is that people who are tinkerers ought to be able to find a fascination in tinkering with such things as aesthetics and communications, which are the most intense things to tinker with, after all. Because if we tinker with anything else, we'll destroy ourselves. My notion is that people are somewhat dangerous to their own survival because we're too creative. The metaphor I sometimes use is that people on planet earth are like a bunch of really technically bright teenagers without any supervision hanging out all summer in a chemistry lab [laughter].



I like to think of VR as an alternative way of thinking about a ramp of technological progress in the future where instead of making bigger and faster things, you make more intense experiences and more interesting forms of human connection. And if you think of that ramp, which is more of a McLuhanesque ramp than an Edward Teller ramp, that alternative ramp is the one that we can survive with. So in that sense, all this business about aesthetics and communications is a survival strategy. I really think it's the only imaginable future.

RU: You just said, "If we tinker with anything else we destroy ourselves." That's a pretty extreme statement! Aren't at least some kinds of tinkering convivial? In fact, don't we need to keep tinkering in order to evolve some of our current technologies into a more convivial state?

JL: I spoke poorly. I'm not against tinkering but against the idea of tinkering for the purpose of increasing human power as opposed to the purpose of increasing human connection and experience. If your only value is increasing technological power according to some extra-human definition, you will eventually hurt yourself. If that was the only possible form of tinkering I would have to be anti-tinkering, but I think I've articulated and practiced a different kind. I'm not even anti-power-oriented tinkering — it's fantastic for improving our lives; medicine and all that. I only become opposed to it when it is the only guiding value, which it ultimately is for a the totalists, as well as the goodness-of-the-immanent-singularity folks.

RU: The closest thing we have to shared dreaming right now is when people gather together in the movie theater.

JL: The movie theater is Stalin's version of a dream because somebody dictates a dream.

RU: You have a pretty strong critique of the transhumanist.

JL: Well, for instance, little nanomachines, little molecular machines can absolutely transform the world. We've seen that happen once. It's life on earth and it took billions of years. So the real question isn't whether there is the possibility that there could be another family of molecular machines in billions of years, the question is whether there is some alternate family of molecular machines that can do something interesting in a much shorter period of time that's relevant to any planning horizon for us. So in order for them to go faster than evolution did the first time, there has to be some other ingredient that evolution didn't have. And my illustrious colleagues, the "totalists" would say that evolution didn't have the benefits of their genius, and evolution is going to happen really fast this time.

RU: But you can make a distinction between say Ray Kurzweil's claim that somewhere in the foreseeable future we're going to infuse the lifeless parts of our universe with intelligence, and saying that we can build molecular machines adequately to particular purposes that can, for instance, go into the human body and maintain health, or can produce enough wealth to end material scarcity.

JL: This issue really comes down to the complexity ceiling problem. And that comes down to how well we can model really complicated things like the interactions of molecules so that we can design something. That's one of the big scientific unknowns right now. It may be that there is this complexity ceiling beyond which you cannot go, so that no matter how fast your computer is designed at the nano level, the level of complexity you need to calculate how to make that machine do certain things really does take billions of years to calculate, and evolution on earth was already optimized. Or it might turn out that we really can do something smart and come up with a way to do it in maybe 10 or 40 years because it turns out that evolution wasn't optimized and evolution wasted a lot of time on — oh, I don't know — snails [laughs]. So the single greatest question might be; how optimized was evolution from the point of view of wanting to design complicated things really quickly?

RU: We have this fairly ubiquitous view now that all the important systems in life come from basically very simple programs that iterate and accumulate complexity. We have currently the theory that the universe is just that; and then we have genes and memes. This works for me, but I also suspect that there's something else going on. I don't have the scientific language or knowledge to say what that is; it may be related to dimensionality; quantum physics... I'm not sure, but I do have that suspicion.

JL: It's tricky. I think there's something else too. This is a subtle difficult question. And there are alternative theories. There are a lot of different kinds of propagating programs we could be talking about like the Wolfram programs that are supposed to propagate a reality that is very different from the Darwinian set of programs. I have a feeling that the sense in which we find these theories to be true reaches a final point where they're tautological and therefore useless. Yes, of course you can conceive of it that way. But I think the question is whether this way of thinking about the world as a bunch of competing programs really gets you anywhere. Is it of any use? Does it make it easier and faster to think about anything? It's an open question but I'm a little skeptical. I think a lot of the meme-gene people are really drunk on their little metaphor but it's pretty short on substance. You end up with the "just so" problem. There wasn't any falsification potential, so all the stories were equally good.

The problem I worry about, which I expressed in my essay "One Half a Manifesto" is that you have this sort of trickle down effect. You have this big metaphor that starts to influence the way you think about little things. You start to think that things are made out of simpler algorithms than they really are and you sort of dumb yourself down. You start thinking of yourself and other people as simpler than they really are because you want it to fit nicely, potentially into a computer.

RU: Do you think big theories that try to explain everything are a waste of time?

JL: No, I don't reject explanation. I just think it's really hard to do.

This was an excerpt from True Mutations: Conversations on the Edge of Technology, Science and Consciousness by RU Sirius. The book includes interviews with Cory Doctorow, Robert Anton Wilson, DJ Spooky, John Markoff, David Pescovitz, Howard Rheingold, Steven Johnson, David Duncan, Genesis P. Orridge, Danel Pinchbeck, Howard Bloom and many others.

See also:
John Edward's Second Life Attackers Unmasked
Who Are Second Life's "Patriotic Nigras"?
Jimmy Wales Will Destroy Google
Neil Gaiman Has Lost His Clothes