Data model design
By thinking about the requirements of your data model in a conscious way early on in your development cycle, you can make Java 3D rendering an implementation of a view into your data model—rather than the data model itself.
Fundamentally you will have to decide upon an internal data model suitable for your application. This will be one of the following types:
The decisions you make in this chapter will probably have the greatest impact upon your application performance. From the outset you should be intimately aware of the performance objectives for your application and the features your data model will have to support.
When designing your 3D application the obvious place to start is to consider your user’s view of the application. Everything that is rendered (drawn) by the underlying Java 3D scenegraph renderer can be described as geometry. Discussions of geometry are generally independent of such issues as user interaction, view parameters and rotation, rendering attributes, and object lighting.
The geometry defined in your scene is rendered by the Java 3D renderer using the current view parameters and the geometry’s attributes, such as color, material, and lighting. The choice of geometry description will probably be the most influential factor in deciding eventual application performance. Deciding the geometry description will always entail making several compromises between contradictory factors such as rendering speed and quality, size of data models, asset managements and load time.
A simple 3D model viewer might simply load a 3D model file (perhaps in VRML format), use a Java 3D file loader to convert the model into Java 3D objects, then render it. For interactive applications however (such as games requiring complex character animation) or scientific visualization applications dealing with large data sets, such a simple approach will rarely suffice.
3D character animation may require that a data model support animation using inverse kinematics, that it implement skin-and-bones animation systems that can be scripted, or that it have data from motion capture devices played back through the characters. The Cosm team (http://www.cosm-game.com) is busy creating an online world built using Java and Java 3D that includes skin-and-bones character animation. You can also download some sample code for skin-and-bones animation from http://www.j3d.org.
Motion capture is becoming an increasingly important technology for the level of realism required by the latest games. Dedicated motion capture labs, such as EdVEC in Scotland (http://www.edvec.ed.ac.uk) , provide sophisticated camera equipment to capture 3D motion and data models.
Scientific visualization applications are generally required to visualize extremely complex data sets, which may not have an obvious representation as 3D objects. Key elements of these applications choose informative 3D data representations, process the raw data, and ensure that the processed data can be rendered efficiently. Visualization data sets may be N-dimensional (meteorology or MRI data for example) or may have to be created from thousands of 2D images as in the Visible Human Project (http://www.nlm.nih.gov/research/visible/visible_human.html).
Algorithms to construct surfaces from 3D data points, adaptive meshes, and dynamic levels of detail may all be required to generate compelling interactive worlds from the raw input data. It is hard to overstate the importance of understanding your feature and performance objectives as you design your data model. Java 3D provides little support for anything other than simple point, line, and triangle rendering, so you may need a convenient internal representation (model), which can then be converted into a representation for rendering (view). The rendering representation may need to make assumptions as to the current available hardware, or it may be adaptive and modify the representation based on a target frame-rate for the application.
7.1.1 Surface models
Typically, geometry is described as a collection of triangles, or faces, that define an approximation of the outer skin of the object. Importantly, most rendering hardware is optimized to render triangles. Unfortunately, defining objects using a triangular skin description loses many important properties of a 3D model.
A triangulation, by its nature, makes assumptions regarding the rendering hardware available to an application. The application developer may want to create an application that renders an object on a low-end system using 100 triangles, while rendering the object using 10,000 triangles on high-end systems. Such formulations can be accommodated by using a LOD description that allows an application to load several definitions of an object and choose the one most appropriate given the object’s position in the scene and system performance. It may also be possible to dynamically retriangulate the points that compose the surface of the model. This technique is commonly applied to terrain using variants of the ROAM algorithm. The Virtual Terrain site (http://www.vterrain.org) contains may good links to terrain rendering.
Any skin (surface triangulation) description cannot, by its nature, contain information for the internal characteristics of an object. Skin descriptions are merely concerned with surface appearance, and generally assume homogenous (or irrelevant) internal characteristics. Solid modeling operations—operations such as generating two new objects by slicing a plane through an object—are also difficult to implement using a triangular skin description.
Figures 7.1 and 7.2 show a 3D-facial model rendered as a simple skin model. A pair of carefully calibrated cameras was used to capture the model (including a photo realistic texture image). Surface construction was then performed on the input data to generate triangles, calculate normal vectors, and texture coordinates. Finally the processed data was saved as a VRML file and interactively rendered.
Figure 7.1 Skin description. Each triangular face is rendered using an applied texture image
Figure 7.2 Skin description. Each triangular face is rendered using three lines with a texture image applied
If your geometry is very dynamic, as in character animation, you should definitely consider using Java 3D geometry-by-reference support. Releases of Java 3D earlier than version 1.2 would always copy the arrays of data values you passed to it into internal data structures. Java 3D version 1.2 introduced a geometry-by-reference feature that eliminates the copying, and greatly improves performance through increased rendering speed and reduced heap allocation and garbage collection. See the GeometryArray class and GeometryUpdater interface for details of using geometry-by-reference.
7.1.2 Volumetric and mathematical models
Voxel description may be more appropriate if internal object characteristics are not homogenous or if complex object operations must be performed that will significantly alter surface geometry. Voxels are simple building blocks for an object (often cubes), and can possess individual material attributes, allowing nonuniform material descriptions. Thus a pyramid might be described by a sequence of stacked voxels (cubes). Slicing a pyramid with a plane can generate a complex series of geometric shapes; and developing robust algorithms for generating new triangular skin descriptions from an original skin description for a pyramid is a nontrivial problem. Using a voxel-level description, however, the problem is far simpler: if the centroid of a given voxel lies on one side of the slicing plane, assign it to object A, otherwise assign it to object B. Boolean operations, such as calculating the intersection of two arbitrary objects, are also far easier to implement. Applications must typically use a voxel description internally, and generate a skin surface description for rendering. This allows the application to optimize rendering performance (which is required for every frame) while not sacrificing the ability to perform solid modeling operations (which is required occasionally).
Voxel descriptions are often associated with medical visualization applications, particularly for magnetic resonance imaging (MRI) visualization. MRI equipment provides an array of coordinates along with an MRI reading for each coordinate. MRI data can be rendered as an array of cubes with the MRI reading for each cube assigned as a color or transparency for the cube. Finding surfaces across voxels (isosurfaces) may require using the Marching Cubes Algorithm or one of its variants. See W.E. Lorensen, et al., “Marching Cubes: a high resolution 3D surface reconstruction algorithm,” Computer Graphics, vol. 21, no. 4, pp 163–169 (Proc. of SIGGRAPH), 1987, and Alan Watt, et al., Advanced Animation and Rendering Techniques, Addison-Wesley, 1992.
Voxel descriptions also suffer from inherent assumptions regarding the performance of rendering hardware, as the object modeler must decide the voxel resolution required for an object (or part of an object). See figures 7.3, 7.4, and 7.5.
Figure 7.3 Transparent volume rendering of Lucky's (the virtual dog) abdomen. Courtesy Edinburgh Virtual Environment Centre (http://www.edvec.ed.ac.uk)
Figure 7.4 Voxel rendering of human skull from MRI data. Courtesy Ohio State University (http://www.cis.ohio-state.edu/volviz/)
Figure 7.5 Volume rendering of human skull with two sections removed to show internal detail. Courtesy State University of New York at Stony Brook (http://www.cs.sunysb.edu/~vislab/)
Mathematical object descriptions are useful in that they do not contain assumptions about rendering performance—if an object is described as a sphere, with radius 2.5 meters, the rendering system can render the object in a manner appropriate for the underlying hardware. Non-Uniform Relational B-Splines (NURBS) are commonly used in technical modeling applications as they describe an object’s surface mathematically and in a manner that can be rendered (with varying accuracy) compatibly with the hardware available.
It is important to reiterate, however, that current 3D video cards used to accelerate rendering deal exclusively in simple strips of triangles. Thus, no matter what level of description you choose for your application, your objects will need to be decomposed into strips of triangles. The low-level graphics language itself may handle this transformation; OpenGL can render NURBS, for example (figure 7.6). Java 3D does not have the ability to render NURBS directly, the application developer must write code to convert the NURBS into triangles prior to rendering.
Figure 7.6 Rendering of a tube and a deformed sphere, both described using NURBS. Image produced using Amapi 3D (http://www.eovia.com)
VOXELS OR SKIN DESCRIPTION?
You can use some of the items in this list to guide you in your choice of an internal data model:
7.1.3 Implementing in Java 3D
The Java 3D and OpenGL APIs are both focused around rendering skin-based models. The geometric primitives available for rendering points, lines, and triangles are all most suited to rendering surface geometry. Some OpenGL demos, however, implement volume rendering—converting, at runtime, an internal volume-based representation to a surface-based representation for rendering. Doug Gehringer, an engineer at Sun, has written a Java 3D volume rendering demo that is available at http://www.j3d.org.
Volume rendering (on Solaris, only) can also be approximated in Java 3D using the Solaris Texture3D support. Texture3D rendering allows a three-dimensional raster to be mapped onto three-dimensional geometry for rendering.
I have also successfully approximated volume rendering using Java 3D for meteorological visualization by rendering points and providing interactive controls to dynamically apply a rendering threshold to the points.
The JavaMet example, included with this book, uses a cloud of points to simulate simple volume rendering. Each point in the cloud has an associated frequency, and the end user can interactively control which points are rendered by adjusting the upper and lower frequency limits (figure 7.7).
Figure 7.7 The JavaMet application renders the red, green, and blue channels of the image on the left as a series of 3D voxels in the 3D view on the right. Each voxel has a frequency associated with it, and the voxel threshold can be interactively modified using the slider below the 3D view
One of the biggest issues you will face when developing 3D applications is performance, which usually translates to rendering time. Rendering time defines how long the Java 3D renderer will take to render a single frame that is a view of your 3D scene. This, in turn, defines the FPS that will be achievable for your application. For games applications and smooth animation, a rate of 20 or more FPS is desirable. Obviously, the FPS that your application will achieve is inherently linked to the hardware that it is running on. When designing your application it is therefore vital to specify your performance objectives on specific platforms. For example, you might want the application to achieve 30 FPS on a 500 MHz PC running Windows 98 with an nVidia GeForce MX video card. This typical system configuration (or target machine) must be the yardstick by which you judge your application and all the decisions you make while designing your application should be evaluated in the context of their impact on the performance of the application running on your target machine (or machines). During the initial design phase, you should run simulated tests on your target machine to evaluate your design decisions and assess whether the performance objectives are achievable.
A trap that developers sometimes fall into is to introduce features that look great on their development machines (which are typically high-end) but cause an overall degradation in performance on users’ machines (which are typically closer to the performance of the target machine).
Almost all 3D applications are interactive, that is, the user is able to influence the current point of view or camera and hence the rendered view. The required frame rate is often related to the mode of user interaction:
for 3D CAD visualization input is largely mouse or tablet driven and motion is usually confined to a small number of objects and is not continuous. Lower rates of five or more FPS may be acceptable. For interactive 3D games a frame rate is required that allows users to suspend belief and accept that they are moving within the scene. Lower frame rates provide a choppier motion and quickly destroy the impression of smooth movement, 20 or more FPS is typically required.
7.2.1 Rendering quality
Rendering quality depends on such factors as geometry description, low-level graphics language, rendering hardware, and display hardware. Typically the only factor under the developer’s direct control is geometry description. The following will influence the visual appearance of rendered geometry:
Rendering quality is a difficult balancing act, and a compromise must typically be struck between the complexity of model description plus texture images on the one hand and rendering time on the other. It may be necessary to dynamically change the model description based on the distance of the object from the viewer using a level of detail approach.
Unfortunately, not all software and hardware renderers are created equal, and noticeable differences will be perceived between DirectX and OpenGL rendering as well as between different makes (and versions) of 3D accelerator cards and drivers. To compound matters, certain makes of 3D cards will incorrectly render certain configurations of geometric primitives. The only advice that can be given is to ensure applications are tested on as wide a variety of hardware and software platforms as possible.
7.2.2 Load time
Load time is not typically of paramount importance, but loading large complex geometric models and texture images can become very time consuming, particularly if the models are loaded across the Internet. Like other objectives, load-time objectives should be set and adhered to throughout the development phase.
Before your application’s data can be loaded, the JVM must be loaded from disk and started. The time taken to load the JVM is heavily dependent upon available physical memory, and it can take minutes to load the JVM on machines with less than 64 MB. You must seriously consider using delayed loading techniques for application data to ensure that the user is not left waiting for several minutes for your application to complete startup.
7.2.3 Memory footprint
The total memory footprint (all allocated memory) of a Java 3D application is unfortunately quite large. Most of the memory allocated will probably be used by the JVM and the Java 3D subsystem. If models and textures are very large, however, they can come to play a significant part in determining runtime memory requirements. Determine the minimum amount of physical memory required on your target machine and endeavor to keep your application’s memory footprint within physical memory. Any paging of memory to or from disk (virtual memory) will probably cause your application to be so slow as to be unusable. For Java 3D applications, and certainly applications that also use Swing, the minimum memory requirements for reasonable performance is 64 MB. Running the simplest Java 3D example, HelloUniverse, requires upward of 20 MB.
7.2.4 Development time and asset management
Complex geometry for visualization will either be loaded into your application in a standard 3D graphics file format (such as VRML), a proprietary 3D graphics format, or as raw data that must be processed to produce geometry for rendering. For data in a standard 3D graphics format, one of many available Sun or third-party ObjectLoaders can be used. For data in a proprietary format a custom ObjectLoader can be written that adheres to the interfaces required for ObjectLoaders. Raw numerical data must be processed using application-specific methods. These could include triangulation, surface fitting, geometry assignment and generation, color assignment, and so on.
Many different data loaders (ObjectLoaders) have been developed and these are constantly being refined and supplemented. Some of the most popular are the VRML, DXF, and Lightwave data file loaders. These allow 3D graphics designers to work in parallel to define geometry, while Java 3D programmers can concentrate their efforts on UI, application logic, and behavior programming.
As you have probably gathered, designing the data model for your application is a complex undertaking. This chapter points out a few landmarks as you work through your design. The requirements of your data model are intimately linked to your application’s feature set—the data model for a 3D-combat game will be significantly different from the data model for a MRI visualization application. You should research algorithms that can provide some measure of hardware performance independence and separate your internal data model from that used by Java 3D for rendering. Although Java 3D provides adequate data structures for describing geometry, if you rely on these data structures for your own internal data model, you may be making invalid assumptions about available hardware and precluding the incorporation of your own application-specific optimizations in the future.