Object interaction—picking and collision detection
16.8 Using picking for collision detection
Two example applications of Java 3D’s support for picking are presented here. The first example, in section 16.7, loads a VRML scene and reports the name of the object from the VRML scene that was picked when a mouse click occurs. The second example uses Java 3D’s Bounds-based picking to implement simple collision detection. The example creates four Sphere objects within a cube, which ricochet off each other and the walls of the cube.
Picking is the act of identifying objects in the 3D scene, usually with a pointing device, such as the mouse. Java 3D’s support for behaviors and picking can also be used to implement simple collision detection and response within a 3D scene. Picking is central to many of the direct-manipulation UI paradigms. Using direct manipulation, for example, you would translate an object by clicking it and moving the mouse, as opposed to typing the object ID and its new position into an edit field. For precision work, the edit field will work best; however, it abstracts the user from the 3D scene as compared to direct manipulation.
Java 3D 1.2 includes some new classes that make picking relatively easy. These classes are a big improvement over the 1.1 picking classes, which have been widely deprecated.
Before going into the details of the examples, I will describe the Java 3D 1.2 picking classes. The Java documentation for the picking classes is excellent, probably the best in Java 3D, so it is useful to familiarize yourself with it.
java.lang.Object
|
+--javax.media.j3d.PickShape
PickShapes are used by the PickTool to provide information about the volume of space, segment, or infinite ray that the objects in the scene should be tested against. Java 3D supports the PickShapes listed in table 16.1:
Table 16.1 PickShapes supported by Java 3D
By using the appropriate PickShape you can find the objects in your scene that
java.lang.Object
|
+--com.sun.j3d.utils.picking.PickTool
PickTool is the base class for picking operations. The picking methods will return a PickResult object for each object picked, which can then be queried to obtain more detailed information about the specific objects that were picked. The pick mode specifies the detail level of picking before the PickResult is returned (see table 16.2).
Table 16.2 PickTools
The utility method PickTool.setCapabilities(Node, int) can be used before the scenegraph is made live to set the capabilities of Shape3D, Morph, or Geometry nodes to allow picking.
A PickResult from a lower level of detail pick can be used to inquire about more detailed information if the capability bits are set. This can be used to filter the PickResults before the more computationally intensive intersection processing. For example, the application can do a BOUNDS pick and then selectively query intersections on some of the PickResults. This will save the effort of doing intersection computation on the other PickResults. However, querying the intersections from a GEOMETRY pick will make the intersection computation happen twice, so use GEOMETRY_INTERSECT_INFO if you want to inquire the intersection information on all the PickResults.
When using pickAllSorted or pickClosest methods, the picks will be sorted by the distance from the start point of the pick shape to the intersection point.
Morph nodes cannot be picked using the displayed geometry in GEOMETRY_INTERSECT_INFO mode due to limitations in the current Java3D core API (the current geometry of the Morph cannot be queryed). Instead, they are picked by using the geometry at index 0 in the Morph. This limitation may be eliminated in a future release of Java3D.
If the pick shape is a PickBounds, the pick result will contain only the scenegraph path, even if the mode is GEOMETRY_INTERSECT_INFO.
java.lang.Object
|
+--com.sun.j3d.utils.picking.PickTool
|
+--com.sun.j3d.utils.picking.PickCanvas
PickCanvas, a subclass of PickTool, simplifies picking using mouse events from an AWT Canvas. This class allows picking using Canvas x,y locations by generating the appropriate pick shape.
The pick tolerance specifies the distance from the pick center to include in the pick shape. A tolerance of 0.0 may slightly speed up picking, but also makes it very difficult to pick points and lines. The pick canvas can be used to make a series of picks; for example, to initialize the pick canvas, do the following:
PickCanvas pickCanvas = new PickCanvas(canvas, scene);
pickCanvas.setMode(PickTool.GEOMETRY_INTERSECT_INFO);
pickCanvas.setTolerance(4.0f);
Then for each mouse event:
pickCanvas.setShapeLocation(mouseEvent);
PickResult[] results = pickCanvas.pickAll();
For the pickAllSorted or pickClosest methods, the picks will be sorted by the distance from the ViewPlatform to the intersection point.
java.lang.Object
|
+--com.sun.j3d.utils.picking.PickIntersection
PickIntersection holds information about an intersection of a PickShape with a Node as part of a PickResult. Information about the intersected geometry, intersected primitive, intersection point, and closest vertex can be queryed.
The intersected geometry is indicated by an index into the list of geometry arrays on the PickResult. It can also be queryed from this object.
The intersected primitive indicates which primitive out of the GeometryArray was intersected (where the primitive is a point, line, triangle, or quad, not a com.sun.j3d.utils.geometry.Primitive). For example, the intersection would indicate which triangle out of a triangle strip was intersected. The methods which return primitive data will have one value if the primitive is a point, two values if the primitive is a line, three values if the primitive is a triangle, and four values if the primitive is quad.
The primitive’s VWorld coordinates are saved when the intersection is calculated. The local coordinates, normal color, and texture coordinates for the primitive can also be queryed if they are present and readable.
The intersection point is the location on the primitive which intersects the pick shape closest to the center of the pick shape. The intersection point’s location in VWorld coordinates is saved when the intersection is calculated. The local coordinates, and the normal, color, and texture coordinates of the intersection can be interpolated if they are present and readable.
The closest vertex is the vertex of the primitive closest to the intersection point. The vertex index, VWorld coordinates, and local coordinates of the closest vertex can be queryed. The normal, color, and texture coordinate of the closest vertex can be queryed from the geometry array:
Vector3f getNormal(PickIntersection pi, int vertexIndex)
{
int index;
Vector3d normal = new Vector3f();
GeometryArray ga = pickIntersection.getGeometryArray();
if (pickIntersection.geometryIsIndexed())
{
index = ga.getNormalIndex(vertexIndex);
}
else
{
index = vertexIndex;
}
ga.getNormal(index, normal);
return normal;
}
The color, normal, and texture coordinate information for the intersected primitive and the intersection point can be queryed. The geometry includes them, and the corresponding READ capability bits are set. PickTool.setCapabilities(Node, int) can be used to set the capability bits to allow this data to be queryed.
java.lang.Object
|
+--com.sun.j3d.utils.picking.PickResult
PickResult stores information about a pick hit. Detailed information about the pick and each intersection of the PickShape with the picked Node can be inquired. The PickResult is constructed with basic information, and more detailed information is generated as needed. The additional information is only available if capability bits on the scenegraph Nodes are set properly; PickTool.setCapabilities(Node, int) can be used to ensure correct capabilities are set. Inquiring data that is not available due to capabilities not being set will generate a CapabilityNotSetException.
A PickResult can be used to calculate intersections on Node which is not part of a live scenegraph using the constructor which takes a local to VWorld transformation for the Node.
Pick hits on TriangleStrip primitives will store the triangle points in the PickIntersection with the vertices in counterclockwise order. For triangles that start with an odd numbered vertex, this will be the opposite of the order of the points in the TriangleStrip. This way the triangle in the PickIntersection will be displayed the same way as the triangle in the strip.
If the Shape3D being picked has multiple geometry arrays, the arrays are stored in the PickResult and referred to by a geometry index. If the Shape3D refers to a CompressedGeometry, the geometry is decompressed into an array of Shape3D nodes that can be queryed. The geometry NodeComponents for the Shape3D nodes are stored and used as if the Shape3D had multiple geometries. If there are multiple CompressedGeometries on the Shape3D, the decompressed Shape3Ds and GeometryArrays will be stored sequentially.
The intersection point for Morph nodes cannot be calculated using the displayed geometry due to limitations in the current Java3D core API (the current geometry of the Morph cannot be inquired). Instead, the geometry at index 0 in the Morph is used. This limitation may be eliminated in a future release of Java3D.
The VrmlPickingTest example illustrates how the PickCanvas and PickResult classes can be used. The example loads a VRML format data file and allows the user to rotate, translate, and scale the loaded model. When the mouse is clicked, a list is generated of the intersections of the model with a PickCylinderRay that passes perpendicularly through the clicked screen location into the 3D scene. All the intersections with the model are reported, as well as the closest intersection.
When a mouse click occurs, VrmlPickingTest produces the following output (which corresponds to clicking the mouse in the position shown in figure 16.1. The output is a list of the PickResult objects (sorted from nearest to farthest):
*** MouseClick ***
First, the path through the scenegraph to the intersected node is displayed. In this case, the path is the VirtualUniverse’s Locale, a BranchGroup, the Sphere primitive (user data is “Sphere”), then finally a Shape3D containing a TriangleStripArray.
Figure 16.1 A VRML scene loaded into the VrmlPickingTest example. Note the position of the cursor (the hand icon), which corresponds to the scene intersections in the code which follows
Sorted PickResult 0: PickResult:
sgp:javax.media.j3d.Locale@124fb8e :
javax.media.j3d.BranchGroup :
com.sun.j3d.utils.geometry.Sphere, Sphere :
javax.media.j3d.Shape3D,
Spherejavax.media.j3d.TriangleStripArray@12b486c
The transformation matrix required to convert the terminal node of the scenegraph path (Shape3D) to Virtual World coordinates is displayed. This is the transformation matrix that was in effect when the pick took place.
LocalToVworld Transform:
-0.5243562077058301, -0.8502316137753383, 0.04644104194946784,
0.35220520974733893
0.3928339572694004, -0.19315917400790997, 0.8990945531548112,
5.215472765774056
-0.7554680995624017, 0.4896894285499475, 0.4352840614012915,
0.5764203070064573
0.0, 0.0, 0.0, 1.0
Next, the intersection information for the ray is displayed. In this case, the ray intersected six Nodes in the model. For each intersection, the distance of the intersection from the ViewPlatform is calculated along with the point’s coordinates in Virtual World coordinates. As you can see from the Z-coordinates and distances, five of the intersections were with Nodes at the front of the Sphere while one was with a node at the rear of the Sphere.
node:javax.media.j3d.Shape3D@12b485a
PickIntersection: geomIndex = 0
dist:13.190385327169809
ptVW:(0.3082159939737674,5.101595194311164,-0.40544525181089597)
PickIntersection: geomIndex = 0
dist:11.285273011880047
ptVW:(0.2697997524391042, 4.782074528439611, 1.4723671948932975)
PickIntersection: geomIndex = 0
dist:11.28272787795884
ptVW:(0.2766647006417829, 4.784127302928557, 1.4754390646997604)
PickIntersection: geomIndex =
0 dist:11.282690605316592
ptVW:(0.26386760841671225, 4.797646503054273,1.4773578620510737)
PickIntersection: geomIndex = 0 dist:11.279971427880689
ptVW:(0.27735265885195876, 4.796380438058344, 1.4802262351804227)
PickIntersection: geomIndex = 0
dist:11.28272787795884
ptVW:(0.2766647006417829, 4.784127302928557, 1.4754390646997604)
Sorted Object 0: Sphere
Closest Object: Sphere
The second illustrated pick intersection is more complex (figure 16.2). As you can see, the sphere is still the closest intersection; however, the pick ray passes through the entire model. In this example (output following), the VRML part intersections are (sorted from nearest to farthest):
1. Sphere: 2 intersections
2. Cone: 2 intersections
3. Cone: 2 intersections
4. Box: 2 intersections
5. Cone: 7 intersections
*** MouseClick ***
Sorted PickResult 0: PickResult:
sgp:javax.media.j3d.Locale@124fb8e :
javax.media.j3d.BranchGroup :
Figure 16.2 A VRML scene loaded into the VrmlPickingTest example. Note the position of the cursor (the hand icon) which corresponds to the scene intersections in the code that follows
com.sun.j3d.utils.geometry.Sphere, Sphere :
javax.media.j3d.Shape3D, Sphere
javax.media.j3d.TriangleStripArray@12b486c
LocalToVworld Transform:
0.974680683424301, -0.19810018807842686, -0.10370092016385302,
-0.5185046008192652
0.2217376557236817, 0.9160774460188752, 0.3341175316108114,
2.8105876580540574
0.028809328241108344, -0.34865230298798766, 0.9368092624581957, 3.084046312290978
0.0, 0.0, 0.0, 1.0
node:javax.media.j3d.Shape3D@12b485a
PickIntersection: geomIndex = 0
dist:10.31109258625374
ptVW:(-0.503754844446497, 2.138046095717119, 2.3502490354035483)
PickIntersection: geomIndex =
0 dist:10.315735064224192
ptVW:(-0.48806121433886257, 2.1446076441445165, 2.3442903032651294)
PickIntersection: geomIndex = 0
dist:10.311507103034156
ptVW:(-0.46680214250863505, 2.1403178766932185, 2.3478813387527073)
PickIntersection: geomIndex =
0 dist:8.737141773923474
ptVW:(-0.41919205110931124, 2.265854783380931, 3.916754614302066)
PickIntersection: geomIndex = 0
dist:8.771580342395431
ptVW:(-0.41919205110931124, 2.265854783380931, 3.916754614302066)
PickIntersection: geomIndex = 0
dist:8.732273133281984
ptVW:(-0.41290180559136586, 2.275910225691348, 3.9205080490411017)
PickIntersection: geomIndex = 0
dist:8.73669779993455
ptVW:(-0.4106277895151771, 2.2691852339960756, 3.916514821486335)
Sorted Object 0: Sphere
Sorted PickResult 1: PickResult:
sgp:javax.media.j3d.Locale@124fb8e :
javax.media.j3d.BranchGroup :
javax.media.j3d.Shape3D, Cone
javax.media.j3d.TriangleFanArray@1262519
LocalToVworld Transform:
0.974680683424301, -0.19810018807842686, -0.10370092016385302,
-0.3111027604915591
0.2217376557236817, 0.9160774460188752, 0.3341175316108114,
2.1423525948324347
0.028809328241108344, -0.34865230298798766, 0.9368092624581957, 1.2104277873745866
0.0, 0.0, 0.0, 1.0
node:javax.media.j3d.Shape3D@1261cac
PickIntersection: geomIndex = 0
dist:10.943688351941072
ptVW:(-0.510896717862459, 2.1149716954978928, 1.7318035261186269)
PickIntersection: geomIndex = 0
dist:10.92767850911496
ptVW:(-0.5066210416916537, 2.112735307161519, 1.7330744444918968)
Sorted Object 1: Cone
Sorted PickResult 2: PickResult:
sgp:javax.media.j3d.Locale@124fb8e :
javax.media.j3d.BranchGroup :
javax.media.j3d.Shape3D, Cone
javax.media.j3d.TriangleFanArray@1262519
LocalToVworld Transform:
0.974680683424301, -0.19810018807842686, -0.10370092016385302,
-0.3111027604915591
0.2217376557236817, 0.9160774460188752, 0.3341175316108114,
2.1423525948324347
0.028809328241108344, -0.34865230298798766, 0.9368092624581957, 1.2104277873745866
0.0, 0.0, 0.0, 1.0
node:javax.media.j3d.Shape3D@1261cac
PickIntersection: geomIndex = 0
dist:10.943688351941072
ptVW:(-0.510896717862459, 2.1149716954978928, 1.7318035261186269)
PickIntersection: geomIndex = 0
dist:10.92767850911496
ptVW:(-0.5066210416916537, 2.112735307161519, 1.7330744444918968)
Sorted Object 2: Cone
Sorted PickResult 3: PickResult:
sgp:javax.media.j3d.Locale@124fb8e :
javax.media.j3d.BranchGroup :
javax.media.j3d.Shape3D, Box
javax.media.j3d.QuadArray@1264877
LocalToVworld Transform:
0.974680683424301, -0.10370092016385303, 0.19810018807842686,
-0.10370092016385303
0.2217376557236817, 0.3341175316108115, -0.9160774460188752, 1.4741175316108115
0.028809328241108344, 0.9368092624581957, 0.3486523029879877,
-0.6631907375418048
0.0, 0.0, 0.0, 1.0
node:javax.media.j3d.Shape3D@1264cfe
PickIntersection: geomIndex = 0
dist:12.494425040536017
ptVW:(-0.5914732681836042, 1.9639480320061125, 0.17556762285086336)
PickIntersection: geomIndex = 0
dist:14.587993543333791
ptVW:(-0.6908450104199546, 1.7903467955691152, -1.9084230065569017)
Sorted Object 3: Box
Sorted PickResult 4: PickResult:
sgp:javax.media.j3d.Locale@124fb8e :
javax.media.j3d.BranchGroup :
javax.media.j3d.Shape3D, Cone
javax.media.j3d.TriangleFanArray@124fa1a
LocalToVworld Transform:
0.974680683424301, -0.19810018807842686, -0.10370092016385302,
-0.2074018403277061
0.2217376557236817, 0.9160774460188752, 0.3341175316108114,
1.8082350632216233
0.028809328241108344, -0.34865230298798766, 0.9368092624581957, 0.2736185249163908
0.0, 0.0, 0.0, 1.0
node:javax.media.j3d.Shape3D@124fa08
PickIntersection: geomIndex = 0
dist:12.494425040536019
ptVW:(-0.5914732681836044, 1.9639480320061125, 0.17556762285086158)
PickIntersection: geomIndex = 0
dist:12.500884811804253
ptVW:(-0.5720301373107639, 1.9989535603646984, 0.16523500707364264)
Sorted Object 4: Cone
Closest PickResult: PickResult:
sgp:javax.media.j3d.Locale@124fb8e :
javax.media.j3d.BranchGroup :
com.sun.j3d.utils.geometry.Sphere, Sphere :
javax.media.j3d.Shape3D, Sphere
javax.media.j3d.TriangleStripArray@12b486c
LocalToVworld Transform:
0.974680683424301, -0.19810018807842686, -0.10370092016385302,
-0.5185046008192652