Placeholder: Notes on Process

8 June 1994

Back in the early 70s, inspired by people like Robert Wilson and Richard Sheckner, I had been doing interactive theatre pieces in the midwest. This stuff had culminated in a series of environmental theatre pieces in which I collected narratives like the very early Robin Hood tales and staged them mansion style around Mirror Lake on the Ohio State campus, with children wandering in and out of the action in the landscape. My first interactive work was in the context of theatre and it was very much situated in natural landscapes. Twenty years later I found myself doing interactive work in landscapes again, but this time in the medium of virtual reality.

These are some musings about collaboration on an art/tech project among people who are, if not of different species, at least of different intelligences, different esthetics, different sensibilities, different skills - and what kinds of things go on when they undertake a passionate journey together. As a director in the theatre for many years, I thought I understood the fundamental bit: a director's job is to help people with many different skills and sensibilities see the same vision of the final production, and that seeing is what magnetizes each individual effort along its appropriate vector toward the emerging whole. I still believe in this principle. But I've discovered, as many of you have, that virtual reality involves a whole new set of questions and problems - the skill sets and sensibilities of the contributors seem vastly more divergent, and the traditional organizational paradigm of theatrical production is not there to establish a de facto framework for collaboration. Add the element of active participation by people who experience the piece as a potent factor in determining its form and structure and you have a much more complex problem. In other words, making something beautiful in VR is really fucking hard.

Looking at the process of designing and developing Placeholder is a way of getting to some of these issues. The project consisted of a set of three virtual environments overlaid with characters and narrative materials that was experienced by two live participants simulatneously [see Placeholder papers]. The project was designed primarily by myself and Rachel Strickland, an architect and videographer who learned cinema verite with Ricky Leacock in the Film and Video section at MIT, thinking about interactive media in collaboration with some of the Architecture Machine crowd in the 70s and early 80s. I met her at the Atari Research Laboratory in 1982, and we have worked together off and on ever since then.

My first piece of advice is: You have to fall in love. Not necessarily with each other, although that helps - but with a vision of what you are trying to do together. Rachel has always been interested in exploring how people interact with landscapes, with places - she has always been trying to understand the sense of place. I was always on the trail of narrative play - trying to understand how people, especially children, play together as a way to pursue a notion of interactive form that does not rely on traditional models of authorship for its content. Rachel and I fell in love over an essay by Barry Lopez called "Landscape and Narrative" that seemed to bring our two interests into luminous synergy. We have been arguing ever since - a sure sign of love.

Our work together in exploring narrative play began at Atari and continued in the Apple Vivarium project in the late 80s, where we worked on understanding narrative play and the materials and environments that support the most generative kinds of play. We spent a year with kindergarten children, devouring the ideas of Rudolf Steiner and Maria Montessori on the nature of play and its materials, fueling ourselves with long hikes and visits to petroglyph sites in Utah, Australia, and the Caribbean, and making deep descends into many flavors of aboriginal lore. We pondered the marking of places, the lush alternatives to the Cartesian idea of space, the exquisite ambiguity of clouds and rocks in which we could see faces. We told coyote stories to the kindergarteners and then watched them play with these unfamiliar materials in ways that revealed a "narrative intelligence" which had almost nothing to do with the traditional Western notion of story, but everything to do with ambiguity, manipulable objects, magical transformations, and - hovering over all - a synesthetic sense of place.

The PLAYGROUND Project at Apple Vivarium

The process of designing Placholder actually began in 1990 when Rachel and I were under contract to the Apple Vivarium Program to do research on what it would take to provide a more narrative-style interface for the PLAYGROUND programming language that was being developed there by Alan Kay et al. This work involved investigations into the nature of narrative play among children, including extensive readings and field work at Lakeside School in Los Gatos, California.

We exposed kindergarten children to Native American coyote stories throughout the winter months of 1990-91 in order to make them familiar with settings, characters, notions of place, and narrative structures that were different in some crucial ways from the "fairy tale" tradition of European children's literature. In the spring, we were able to observe the children working with these narrative materials to construct their own play activities at some distance from the influence of Western narrative traditions. By disrupting familiar conventions of causality, representations of space, and styles of characterization, we were able to glimpse something of children's "naive" or "constructive" narrative styles. Indeed, the children we worked with produced stories, enactments, and artifacts that were strikingly original - distinct from both Western and Native American narrative traditions.

The PLAYGROUND problem, however, turned out to be a nut that we couldn't crack. The language was not amenable to Western narrative constructions, due to such features as the use of multiple "agents" to construct characters (which are typically thought of as single sources of intention and action) and the difficulty of representing changes in the world due to characters' choices and actions. Likewise, PLAYGROUND was not well suited for creating alternative narrative styles, due to such obstacles as the difficulty of representing simultaneous events and action in non-contiguous spaces. We were forced to conclude that there was not a way to implement a story-making or narrative approach to the task of designing an interface for PLAYGROUND, because the structure of the language - especially its syntax - bore too little similarity to commonly held notions of story and character. We did resolve to continue our investigations of narrative play apart from the PLAYGROUND project, and our plan of action resulted in the original "Virtual Coyote" proposal that was submitted to the Banff Centre for the Arts in the summer of 1991.

Virtual Coyote Metamorphoses into Placeholder

The Banff Centre accepted our proposal to create a virtual-environment piece in which we would instantiate our hypotheses about narrative play through the use of camera-originated imagery and characters and story materials drawn from Native American and Native Canadian lore. The Banff Centre offered us a residency and the use of facilities, equipment, and technical staff to create the piece. Knowing that we needed additional funding for our own survival as well as for things like supplies, child care, and compensation for potential Native Canadian collaborators, we prepared a budget for corporate sponsorship and presented it to several Silicon Valley coporations. When I was hired by Interval Research Corporation in the summer of 1992, Interval president David Liddle pledged Interval's financial support. He later agreed to hire Rachel Strickland on a one-year artist-in-residence appointment principally to accomplish the project (Rachel's position at Interval has since become permanent).

Funding commitments in hand, Rachel and I made a preliminary visit to the Banff Centre in September of 1992. During that visit our contacts with Native people at the Banff Centre were limited to a few brief conversations with Native artists in residence. In an earlier phone contact, Buffy St. Marie (a professional musician who is a Cree person from the Calgary area) had expressed enthusiasm for our project and promised to connect us with some of her relatives in Canada, but we had not yet received further information from her. We had resolved to visit a local Powwow during that visit, but the night before the Powwow we viewed a video piece entitled "Travelogue" that had been produced at the Banff Centre by Stephan de Costa the previous year. The piece was powerfully critical of European colonialism and exploitation of native people and lands. In the emotional aftermath, we were unable to persuade ourselves to attend the Powwow. As it turned out, that might have been our only opportunity to make alliances with Native Canadians.

Political difficulties surfaced for us at the Banff Centre in the early spring of 1993. I got a call from Doug Macleod who explained to me somewhat apologetically that the First Nations Film Collective had complained to him about white women producing a piece that involved Native materials. As our studies of Aboriginal lore and materials had also included many other cultures from other times and places, I told Doug that we would be glad to refrain from focusing on Native Canadian or American materials and work instead with imagery and stories drawn from a variety of Paleolithic and Neolithic cultures. I mentioned in passing to Doug that I was personally a bit disappointed, having some Cherokee ancestors myself, that we would not be able to investigate Native materials further. At that point Doug expressed the opinion that my Indian blood would probably resolve the concerns of the people who had complained. I demurred, however, because I felt that my biological ancestry was no substitute for the kind of cultural training that would be necessary for me to claim that heritage. When Rachel and I dropped the focus on Coyote, we re-named the project Placeholder until we could think of a better name. Upon reflection, however, Placeholder seemed punnishly apt and the name stuck.

Design and Production in Banff

We arrived in Banff and began working in June of 1993, scouting locations and becoming acquainted with the technical staff and equipment. We also began fleshing out the conceptual design into precise specifications that could be scheduled and implemented. The usual production difficulties began early. Essential hardware had to be ordered or built or borrowed. Cameras and other equipment had to be shipped from Interval through Customs. The project that was scheduled to complete ahead of us was behind schedule and our lead programmer, John Harrison, became intensely involved in the swat team effort to finish it. We lost another week of John's time to SIGGRAPH. Two key technical people suffered health problems that kept them away from our project for several days.

In the meantime, we were inventing and building new interface hardware at Interval, writing device drivers, and attempting to modify existing code in Banff. The facilities for handling two simultaneous participants in the MRToolkit were being developed by the staff at the University of Alberta and were not completed until literally hours before the piece opened - over a month behind schedule. Technical difficulties of this nature and the vicissitudes of the development environment are well described in Rob Tow's section on implementation and I will spare the gory details here.

Setbacks like those metioned above were exacerbated by the scale of the project. Doug, Rachel and I were probably all guilty of commiting to an overly ambitious effort. We went through several painful cycles of down-sizing, but the production was still overwhelmingly complex. As project coordinator for Interval, I was responsible for Interval's goals being met (or as David Liddle put it, "your ass is on the griddle"). Interval's corporate structure gave me project coordination responsibility but not traditional managerial authority in the situation. Rachel, Michael, Rob, and I were peers in terms of our job titles at Interval (Member of the Research Staff), and Rachel, Michael, and I were peers in the eyes of the Banff Centre (Artists in Residence). Doug was in a position to manage the Centre's technical staff but not to manage the artistic side of design and development. John Harrison made day-to-day implementation decisions as best he could. Thirteen active contributors (and many more minor contributors) boiled around with only sporadic communication and no shared understanding of either the process or the outcome. Early on I took to publishing and distributing specifications, agendas for the weekly meetings, and meeting notes to everyone on the team, but these documents were rarely consulted, often superceded by conversations, and quickly outdated.

As matters grew steadily worse, I consulted with David Liddle and Doug Macleod and decided that the best course was to forthrightly assume a directorial stance. I felt that some tough decisions were necessary if the project was to be completed by the opening date with any hope of meeting our goals. Most difficult by far was the decision to stop work on the Troll Falls experiment and create a simpler model utilizing video from Johnston Canyon. This decision caused serious rifts between the four principal Interval people. Worst of all, my decision to "act like a director" came too late. I am convinced that if there had been one acknowledged "director" throughout the process, the project would have been better art, better research, and a better experience for everyone.

The project was never completed - Rachel quite appropriately refers to it as a "design sketch." We had to cancel one scheduled performance because of technical difficulties, and we continued feverishly to implement crucial features of the piece througout the performance run. Software was continually being rewritten, debugged, and recompiled. Such key elements as Crow flight and many aspects of the auditory environment were implemented only in time for the last day of performance. Other features, such as Placemarks, animal perception and locomotion, animation of character icons driven by sensor data, and various Goddess effects were designed but never implemented at all. The complexity of the hardware configuration, the use of borrowed equipment, the extreme fragility of the code, and the loss of several key technical people made it impossible to re-mount the project once it was torn down. What survives of the project, like a stage play, are only our photographs, videotapes, and the things we write and say about it.

Conclusions

1. Race, Gender, and Appropriation

I learned several things from these experiences. Doug's comments about my Cherokee ancestry raised the disturbing question of whether the issue for the Native people who had complained was more about culture or about race. Subsequent to Rachel's and my difficulties with the approriation issue, Michael Naimark fared considerably better than we did in his contacts with the local Native people regarding another Banff project. Through what he described as sharing tobacco and male bonding, Michael won approval for his work, which also involved the use of sites that were important to Native people. His experience added gender to the list of things that might be posing difficulties in communicating with the Native community. I continue to feel that it would have been better if the Native people who were upset with us had approached us directly to dialogue rather than asking the administration of the Banff Centre to represent their interests in the matter. Sometimes I wonder whether the original complaints were lodged by Native people at all, or whether they were in fact made by others on their "behalf," since we never learned who had actually spoken with Doug. In any case, cross-cultural dialogue was not achieved.

After the piece was produced, the only direct criticism on grounds of appropriation of Native materials was made by a French Canadian academic. Her position was that only Native people should use the Banff lanscape as material for art because they are its rightful guardians. She accused us of using primarily Native materials in the piece even though we did not, and when I assured her that all of our materials and images had been drawn from a variety of Paleolithic and Neolithic cultures, she responded that this made matters even worse, because the meaning of our piece "depended upon people's ignorance of the materials." Her observation was technically correct - we did not identify the sources of the various narrative motifs that we employed, but interwove them to create characters and places that we hoped would have archetypal significance. To the extent that the piece was evocative and powerful, it was precisely because of its Jungian qualities. Furthermore, the construction of a syncretic myth (with motifs drawn from diverse cultural sources) should not be treated as an obvious case of cultural criminality; for example, indigenous people from around the world collaborated to construct a syncretic origin myth and prophecy to represent their views at the Environmental Summit in Rio de Janeiro in 1992.

There are some contradictions in the battle over appropriation that surrounded this project which are too important to gloss over. By the purist theory of appropriation that we encountered at Banff, Native filmmakers should stop appropriating cameras, but this is clearly a ridiculous idea that no one would ever put forward, because Native people presumably have a righteous cause for their work, and the oppression that they have suffered justifies appropriation. But if it is true that European culture is spiritually bereft, where might artists of European descent look for inspiration in the reconstruction of our spirituality? If all the world's other cultural traditions and practices and philosophies are off limits, how can we participate in the cultural and spritual evolution of humankind? How will people ever come together if even the world's artists sit with their backs to eachother?

We need to have real conversations about these issues, not simply to shout slogans at each other. We need to talk to one another face to face rather than retreating to institutions to represent our interests. Native people need to speak for themselves and European cultural activists need to stop presuming to represent them in the discourse. We need to admit that values are at the heart of the matter, and that legislation, political maneuvering, and name-calling can express values but will never change them - only genuine communication can do that.

2. The Sociology of Production

Theirry Bardini and Michael Century, through their efforts to understand the sociology of production at the Banff Centre, have helped me to crystallize my thinking about the production process ("Virtual Art as a Socio-Technical Interface," presented at Cyberconf4, Banff Canada, May 1994). As Theirry charitably pointed out to me, VR production is a brand new field with no established roles or practices. Indeed, the sort of use that we made of live actors and field recording in the production of Placeholder had never been attempted. In the production of theatre and film, familiar roles exist with well-defined responsibilities - executive producer, producer, director, cinematographer, scenographer, production designer, playwright, actor, costumer, technical director, editor, etc. None of these roles can be assumed to mean the same thing in VR production. For any complex VR production to succeed, it seems clear that roles must be defined, and that a social contract regarding them must be agreed to by all the contributors. Those of us who worked on Placeholder should not be too harsh on ourselves for not having developed a perfect system on the fly. The fact that we confronted the issues head-on rather than letting go of our goals are reason for some pride. We would certainly do a better job of defining roles at the outset of a future project.

In Placeholder, few roles were straightforward. Doug Macleod and David Liddle shared the role of "executive producer" in that they were each the ultimate person to make decisions for the funding bodies. All the rest of the roles were shared in some way. Rachel, Russell Zeidner, and Michael Naimark all contributed to "production design" but no one was the lead. John Harrison and Doug Macleod carved up the role of "techical director." Michael and Rachel shared the role of "cinematographer." Many of these collaborations worked well, but they often suffered from the lack of a "point person" who could assume responsibility for an entire function. A point person is needed because the communication process simply becomes too diffuse when a large number of contributors are involved.

I would recommend to future VR artists that they put together teams with at least the following roles unambiguously defined and represented by one point person, with or without others operating in support. One person may perform more than one role as long as the roles are clearly defined.

VR Production Roles:

Executive Producer: This is the person who represents the interests of the funders.

Producer: This person is responsible for the logistical and fiscal aspects of production.

VR Designer: This person, equivalent to a playwright or screenplay writer, is the designer of the experience. The VR designer is responsible for the overall conceptual design of the piece, including virtual place, character, objects, dynamics, and possibilities.

Director: The director interprets the artistic vision of the VR designer (a case where it is desirable but not necessary that the same person fulfill both roles) and sees to it that that vision is shared by all the diverse contributors to the project. The director manages the efforts of all contributing designers and directors, soliciting and paying close attention to their inputs as the creative stuff of which the production will be made. A director is always synthesizing, honing, propagating, and amplifying the vision of the finished whole.

Visual Designer: This person designs all that is seen, in concert with the Director, and manages the production and postproduction of visual materials.

Auditory Designer: This person designs all that is heard, in concert with the Director, and manages audio production and postproduction.

Interaction Designer: This person designs the particulars of how humans will interact with the world, including the physical characteristics of the performance venue, and gives implementable specifications to the Technical Director.

Technical Director: This person manages the technical production effort. This includes the internal design and programming of tools, system behaviours, and participant-level interactions. It also includes the realization of the performance venue.

Realtime Director: Roughly equivalent to a stage manager in live theatre, this person is responsible for the realtime experience running smoothly. Also as in live theatre, this person may also be the Technical Director.

In VR, as in live theatre and film, the director is almost invariably the pivotal artist and orchestrator of talent. Doubtless there are many other feasible models of a production team, but I believe that this one will serve, at least until we see major changes in the nature of the tools and process of VR design and production.

3. Disagreements over Artistic Issues

An issue over which there was considerable disagreement was the artistic style of the piece. Often the disagreements were tacit, lurking beneath the surface of discussions about production methods or priorities. Sometimes they were more overt, as were the discussions surrounding the construction of the Troll Falls model. Rachel felt that Rob and I were privileging science over art by insisting that the representation accommodate human perception in certain ways [see "Troll Trials and Tribulations" in this section]. Rob questioned the accuracy of the visual capture techniques used and criticized the difficulties that some of those techniques introduced into the construction as well as the experience of imagery. I insisted that, since a virtual environment is by definition a three-dimensional, viewpoint-dependent spatial representation, it must perforce be physically navigable by human beings.

The whole point, it seemed to me, was to get our bodies into the representation in an intimate and natural way, and it seemed that this could not occur if people were unable to stand up or walk around without difficulty. This issues seemed doubly important to me since children would be experiencing the piece and we wanted to study their narrative activities rather than how they handled various forms of spatial disorientation. What Rachel saw as mere illusionism I saw as a necessary precondition for working in the medium. My opinion on the stylistic issues involved here is expressed in the essay entitled "Imagery and Evolution."

Although we held a substantially common vision of Placeholder and worked hard together to realize shared goals, substantial artistic disagreements between Rachel and myself predated the piece. From my perspective, the most fundamental of these is Rachel's profound distaste for Aristotelean theory and criticism. In interactive media, I see live participants as principal architects of a whole action which possesses form and structure. The relationship of participants to the authors of virtual worlds, characters, and dynamics is one of time-displaced collaboration. The overarching goal of my work in interactive media has been to discover how to orchestrate materials and environment so as to facilitate the emergence of interesting - even beautiful - forms in realtime. Aristotle's dramatic theory has been central to my understanding of how one might do such a thing, even though he himself did not envision either the purpose or the instrumentalities involved in contemporary interactive media. Even though our goals are often congruent, Rachel can't abide my Aristotelean structuralist framework. Agreeing to disagree - and disagreeing constructively - is easier when you are making theory than when you are making art.

Despite our differences, Rachel's predilection for the ineffable, her filmic eye and her mastery of cinema vérité, and her ability to comprehend postmodern criticism have contributed to a generative creative dynamic between us. She has often saved me from approaches that are too frontal, simplistic, or heavy-handed, and we have stimulated each other's thinking in many ways over the years. This project strained our collaboration to its limits. Despite the discomfort, however, I hope that we will both look back on Placeholder as more of a success than a failure, and as the catalyst for a reformulation of our creative relationship.