The Virtual World Data Server & The Virtual Los Angeles Project

After a decade of evolution of multimedia systems, spatial DBMS, visualization tools, virtual reality applications, switched and dedicated bandwidth networks and interactive problem solving environments, a new generation of information system is emerging. Advances in processor, display, network and storage technology now make it possible to combine aspects of these various systems into information managers that, for the first time, truly exploit human information processing abilities. A great deal rides on correctly designing new information systems to efficiently exploit the qualitatively different computing resources which are now becoming widely available.

Three-dimensional, interactive worlds are the next generation of multimedia systems. These worlds can be realistic representations of cities (see Figure 1), or imaginary worlds with terrains representing electric and magnetic fields invisible to the eye. Continuing technological achievements in processor power, storage technology and networking are enabling significant advances in such systems. Chip sets for 3D rendering will soon be available which will place today's high end graphics capability on the desktop for a few hundred dollars (e.g. Nintendo 64).

The impact of interactive graphics on the full range of computer applications can not be denied. As technology progresses, whole new application areas become feasible. We are at the verge of a qualitatively different world as high performance graphics makes it now possible to interactively render photorealistic 3D images in real-time. Such a system in use today at UCLA is aiding in the rebuilding of earthquake, fire, riot and flood damaged areas of Los Angeles (the UCLA Urban Simulator), by allowing community members from all walks of life to directly participate in the planning process. To allow this information to be directly accessed the "Virtual World Data Server." is being created. This system is designed to respond to real-world problems through the use of cutting edge and innovative technologies. The system is also being adapted (the Virtual World Data Server) to allow scientists and medical specialists to interactively navigate through massive amounts of data in ways never before imagined.

However, our ability to accurately create and interactively render these three dimensional worlds is rapidly exceeding our capability to efficiently store, manage and communicate this information in real-time. Current systems, in order to guarantee real-time performance, typically restrict database sizes to that which can fit within the main-memory of a particular target machine. For databases the size of the UCLA Virtual Los Angeles Project (see Figure 2), which is currently tens of gigabytes in size, within one year will total hundreds of gigabytes and within 3 to 5 years we estimate will exceed one terabyte, this requirement is clearly infeasible. Additionally, this requirement implies that the entire database needs to be loaded onto the local machine before the real-time simulation can begin, either requiring large amounts of bandwidth and local storage and/or lengthy delays due to the time required to transfer these massive amounts of information. An additional implication of this system architecture is that one is by necessity forced to design (or redesign) the database for a specific platform size, as the capacity to store these massive amounts of information is not necessarily available on every machine, thereby artifically limiting the size and veracity of the 3D environment to be navigated.

A second approach (as seen in the VRML specification) is to design a system which can load pieces of the database on demand. However, within this approach there is no guarantee of real-time delivery, quality of service and/or admittance control. Typically, with this more anarchistic system architecture long delays must be tolerated, while new data is loaded. Another issue with this approach is that there is no effective mechanism for look-ahead. Rather than a just-in-time system the VRML approach can be characterized as a take-your-time approach. These problems effectively eliminate an entire class of application which is dependent on guaranteed Quality of Service (QoS). For example, the UCLA Urban Simulation Team (UST) has been in discussions with the City of Los Angeles about the feasibility of using the Virtual Los Angeles Model in conjuction with Global Positioning System (GPS) transponders to accurately locate and remotely manage Emergency Response Vehicles in real-time. If this application is to be supported, a robust system for guaranteeing response times with bounded latencies is required.

The Virtual World Data Server (VWDS)

At UCLA, under an ongoing project (the Virtual World Data Server), sponsored by the National Science Foundation, we are designing and building a high performance real-time virtual world/synthetic environment database server (and clients). This real-time database system is designed to efficiently store and retrieve large amounts ( > 1 terabyte ) of spatially distributed heterogeneous data. The server is designed to provide guaranteed quality of service at pre-specified levels for multiple concurrent real-time 3D interactive sessions. This is achieved while simultaneously enabling a dramatic increase in the size of the problem that can be handled on a single workstation. In order to accomplish this task, we are combining existing domain expertise in designing and implementing disk based multi-media servers with existing expertise in real-time simulation and virtual reality.

Our approach is to incorporate a disk-based storage server and database system into the existing simulation, while retaining real-time service (see Figure 3). We expect to achieve (at least) an additional three orders of magnitude increase in the size of the problem that can be handled, while simultaneously supporting a multiuser environment. We are accomplishing this by extending the current system to a scalable, shared-nothing client-server configuration. In addition we are providing for a much richer level of functionality by incorporating two-way interaction with external spatially distributed databases. Because the system is predicated upon technology which supports multi-media and video and audio on demand applications, we can easily incorporate these functions into our real-time streams.

The system operates in a classical client/server configuration (see Figure 4). In the standard approach the rendering is performed on the client. At start up the client establishes a communication link with the server and requests a specific level of service. If sufficient "capacity" exists the server will admit the client to the system and the client will choose a database (e.g. Southern California) and a location (e.g. Westwood) from which to begin. The server then locates, retrieves and sends the required information for the requested start location to the visualization client. The client receives the startup data, merges it into its active "scene graph" (effectively a list of renderable objects) and begins the visual simulation. The user, via the use of an intuitive mouse driven interface, indicates direction and speed of travel for the simulation and the client begins to progressively move through (render) the 3D database. As the client walks, drives or flies through the database, it sends a constant stream of telemetry to the server describing its (the visualization client's) current state (e.g. position, heading, speed, etc.). Based upon this information the server decides what the client will need to see in the next few seconds and schedules the retrieval of the information. As the user moves away from areas, it visited earlier, the client purges the information from memory while notifying the server that this information is no longer being stored locally.

The server which consists of several processes, the most notable of which are the "mediator" and the real-time storage server (RTSS). The mediator is the process which accepts telemetry from the client and creates a prioritized list of "granules" for the real-time storage server to retrieve from disk. We define a granule as a conveniently sized module of data, which is indexed to specific spatial and/or temporal locations. The RTSS proceeds to schedule the retrieval of the information from disk, while attempting to satisfy the prioritization scheme created by the mediator. After retrieval, the data is passed directly to the client which merges the new information into its scene graph for rendering. The mediator maintains a record of the granules currently stored on each client and attempts to predict (based on the clients telemetry) which granules (not currently stored on the client) will need to be retrieved and sent over the next cycle. In cases where the requested (or predicted) amount of information exceeds the delivery capacity of the system, the server will elect (based on its prioritization) to send lower resolution or less articulated (and therefore smaller) versions (called Levels-of Detail or LOD's) of the database, until it can catch up (see Figure 5). The operations of the Virtual World Data Server are more fully described in [ ].

Because the system architecture we have designed is predicated upon multimedia technology, it will be able to support real-time data streams. This will facilitate the inclusion of virtual actors and avatars, as well as to allow for interaction and collaboration in an elegantly scalable and efficient manner. Since the server is required to track all movement within the system, it can also determine when two or more participants are near enough to each other to require notification and/or interaction. By acting as a mediator the server is able to avoid flooding the network and swamping the clients with updates on dynamic objects too far to be observed or to be of interest. It is our plan to have the server manage the dynamic creation and dissolution of multicast domains based on proximal location.

The system also incorporates the ability to dynamically access federated databases using a simple point and click interface. For example, a URL (Universal Resource Locator) or WorldWide Web address can be associated with any 3-dimensional graphic entity within the database. When "picked" with the mouse the system dynamically initiates a Netscape (or other Web browser) session using the URL for the address. This approach allows the use of what is now becoming a ubiquitous database/interface standard to associate a rich (and virtually infinite) assemblage of information with the 3D graphic entities located within the visual database.

For example, this interface is being used to annotate the Virtual Los Angeles database. The Urban Simulation Team is currently under contract from the City of Los Angeles to create virtual models of the Hollywood and MacArthur Park areas of the city. In these models, every parcel and storefront will have its own unique URL. Where feasible these addresses will point to a Web page maintained by the individual business (see Figure 6). If such a Web page does not exist the Urban Simulation team will create a place holder page which will be updated as warranted. Additionally, a reverse indexing scheme is being developed to allow attribute queries to locate graphic entities.

The Forum of Trajan

Another compelling example of the power of this design is the work we are doing with a project we call the Forum of Trajan. Working with two subject experts Jim Packer of the University of Northwestern and Diane Favro of the UCLA Department of Architecture and Urban Design with funding from the J. Paul Getty Foundation, the UCLA Urban Simulation Team is creating a model of the Forum of Trajan. This magnificent edifice was located in ancient Rome (see Figure 7). The building which was built circa 113 A.D. and destroyed in an earthquake around 850 A.D. was constructed using Prof. Packer's three volume book as the source materials. Because we have incorporated the concept of time (other than real-time) into the system, we can create a history of the Forum of Trajan or of ancient Rome (or Los Angeles for that matter), by adding attributes to the graphic objects specifying creation and dissolution dates. When loaded the system recognizes the time dependent nature of the database and adds a time slider to the simulation controls. The user can then interactively select (in addition to a starting location) a time or date (which can be changed at any point) to begin the simulation.

It is our intention to allow the subject experts to self document their reconstructions using the Web interface mentioned above. Since the system also allows multiple alternative graphic objects to be interactively selected from a context sensitive menu, subject experts will be able to create and document their own alternative reconstructions and add them to the master database. Ultimately both Jim and Diane will be able to conduct their office hours, not in their offices, but in ancient Rome at a time of their own choosing. The students will (from their own homes) join Diane and Jim in Rome on a pre-selected date in time and as avatars they will experience Rome as it was in the days of Augustus. During these sessions it will be possible for those proximally located to see and speak to each other, as well as see and hear the virtual actors (Augustus, Julius Caesar?) as they make their way through the virtual environment.

The Three Dimensional Information System

Ultimately we see this system not simply as a 3D graphics/database or real-time simulation system, but as an intuitive interface/index to the world of three-dimensional spatially distributed information. For example, because of the 1994 Northridge Earthquake, UCLA will need to rebuild most of its hospital complex. The UCLA hospital has been described as the second largest interconnected building in the U.S. (after the Pentagon). Our plan is to use the system not only to plan and design the new hospital, but to manage it throughout its useful life as well. We have begun a dialogue with various construction contractors (e.g. Dinwiddie) which would allow the association of the as-built drawings, which the contractors are required to provide, with each wall or surface in the building. Using this interface one could walk into any space, click on the wall or ceiling and pull up the appropriate as-built drawing(s). In addition we would have the contractor photograph each wall before the wall is closed up. These photos (locating the conduits, etc. within the walls) will be indexed to the appropriate wall/ceiling/floor object, so they can be retrieved using the same point and click interface described above. This would provide a much more accurate record of the actual placement of the objects (conduit, pipes, etc.) within the wall at a much lower cost than the standard approach used today.

Network Requirements

This system has been designed to exploit a level of network technology which is now becoming a practical reality. The client/server design is predicated on a switched/connection oriented network topology which supports both Quality of Service and bounded latencies.

We feel that Virtual World servers will become an important information infrastructure platform for many applications. These servers will provide a way of organizing the massive amounts of information and data that are rapidly becoming available online, by using the way we experience that information (i.e. as a 3-D metaphor) as the access paradigm.