Re: [udig-devel] Java3D Rendering

[Jody Garnett <jody.garnett@xxxxxxxxx>]

Your ideas sound fine (and echo conversations I have had with others
interested in this work). We can use the hack space to work out any
design documents proposing changes to the application if needed.

A few more comments inline; reading over my comments now they are
short and to the point. Please rest assured that I am enthusiastic (I
just do not have time during work hours right now to give you a long
detailed answer).

On Mon, Mar 16, 2009 at 1:35 AM, Wellmann, Harald
<harald.wellmann@xxxxxxxxxx> wrote:
> In our uDig-based application, we need to support 3D terrain models and building models. So far, we have a couple of stand-alone Java3D viewers for rendering these models, and now we would like to somehow integrate them into uDig.
>
> This is my rough idea of a roadmap:
>
> 1) An Eclipse EditorPart that handles a 3D file format and renders it using Java3D.

It would be better if you could isolate this as a replacement
MapViewer. We have taken pains to present the uDig rendering
functionality as a MapViewer (ie similar workflow to a JFace viewer)
with the idea of making it easy to swap the widget used to work with a
map. This would enable your Java3D visualization to be used in both
editors and view.

> 2) A replacement for uDig's BasicFeatureRenderer that simply renders 2D content into a Canvas3D and ignores the third dimension.

Understood; there are two good ways to do this.
- Implement a graphics2d that generates out your opengl linework - and
pass it into the geotools streaming renderer. This is cheaper to
implement then reverse engineering the geotools streaming renderer
- Construction of a renderer as you describe; although you may wish to
introduce the concept of a DEM in order to drape the linework over the
surface. formally introducing a DEM to the map model is something I am
willing to do (since we have some TIN summer of code functionality
that would also benefit). I have written these a couple of times
before professionally just never for the uDig application.
- Aside: There was a summer of code project that rendered into a
buffer; and then texture mapped the buffer onto a polygon in order to
do flicker free panning etc. This is not a recommended solution for
linework and labels; but it is good for raster images.

Discussion:
- the labeler is pluggable so you can get all labels set to one object
for you to deal with in 3D as you see fit.
- The Rendering System is set up to allow you to choose render
implementaitons that are suitbale to your widget; so there should be
no api changes needed for you to experiment with alternate renderers
-

> 3) Mixing 2D and 3D content in one map with full support for 3D affine and projective transformations and multiple levels of detail.

Understood; as you take on this work we may need to expand what the
viewport model records to meet your needs. We already wish to expand
the viewport model to cover "time".

> Step 1) is finished and was almost trivial. I just had to osgify Java3D, wrap a Canvas3D in an SWT_AWT embedded frame and create an org.eclipse.ui.editors extension for a given 3D file format. This solution bypasses uDig completely, working directly with Eclipse RCP, but at least it allows us to open and view our 3D files.

Yep. The last commercial implementation I did used Java3D - I was
expecting to use JOGL this time around. The Summer of Code project
used some game/monkey/3d rendering widget I have never heard of
before.

> Step 2) is a kind of "easy exercise" and prerequisite for the final step 3), and it would have the pleasant side-effect of a drastic rendering speed-up for existing 2D feature layers.

I am mostly interested here as a replacement to the composition steps
we do in uDig; I would love to use the 3D card to minimize that costs
(especially with the tile renderer where we have even more images to
draw at each z order level).

> (By the way, I'm aware of that JOGL-based Geotools rendering pipeline, but I don't want to lose the scene graph and other high-level abstractions that Java3D offers on top of JOGL.)

Okay that makes sense; there is also a 2D scene graph that was being
mentioned by sun - not sure if that project is still alive.

I am afraid the scope of your work vastly outstrips the time I can
volunteer to support you; leaving you with a couple of options:
- talk to other PSC memebers
- join the project and get commit access etc
- commercial support

You should also talk to emilyg who would also be a useful person for
commercial support. She is not a PSC member but is far more active in
the rendering code due to some recent contract work.

> I've been browsing the uDig source code, trying to understand the interaction of Renderer, MapEditor and FeatureStore, and it would be great to get some feedback on the following ideas:

See above; you should ask for emilyg's feedback on your questions :-D

> - Create a ViewportPaneJava3D which implements ViewportPane on top of Java3D.

I think that is right; the MapViewer chooses which viewportPane to use
based on the constants supplied during construction

> - Create a new preference and arrange for MapEditor and MapViewer to use the ViewportPaneJava3D instead of ViewportPane[Tiled]SWT. (It would be nicer to have extension point for this purpose.)

Yes it would; now that there are two "customers" of an extension point
it would be time to define one - an API is usually only good if you
have three implementations to arrange a compromise between.

This is work I would be willing to do in an IRC breakout meeting; if
you can arrange a time when both you and emily would be available to
sort out what is needed.

> - Create a Java3DRenderer extending RendererImpl that somehow (?) gets access to the ViewportPaneJava3D by downcasting the IRenderContext (?) and updates the scene graph or moves the view platform when required, leaving all the painting to Java3D.

You should not need to extend RendererImpl; there are two interfaces
involved RendererCreator (that chooses which Renderers are needed
based on what you know); and RendererExecutor that would be
responsible for manipulating your scene graph.

> The biggest problem seems to be that the rendering models of Java3D and uDig are fundamentally different.

It should not be; we are simply gathering together layers that need to
be drawn. If you are worried about the "zorder" of the layers this is
a problem that occurs in 3D rendering as well; I remember writing code
that would offset roads so they were a small delta "higher" than a
landscape so they would be visible etc...

uDig is supplying a model of the map that wants to be displayed; you
can renderer that in any way you see fit.

> uDig gives you a BufferedImage to draw to, you have to handle all the world to screen transformations on your own and paint each feature to the image, whenever the viewport changes.

Nope - it gives you three things:
- render( Graphics 2d, progressmonitor)  to draw into - we call this
method when printing or generating PDFs - this is the non raster entry
point
- render( progressmonitor) to draw into the graphics context; this
usually creates a bufferedimage and calls the first method. Although
some renderers just draw onto a volitile image
- getContext() - this is a utility class that should be handling all
the world to screen transformations for you; it mostly delegates to
the viewport model here (and if you need to you may end up with a more
complicated viewport model that is aware of all your 3d goodness).

> In Java3D, you create a scene graph with shapes (= features) and appearances (= styles), you set up a view platform which can move around in your virtual universe, and all the rendering is handled by Java3D automatically. Supposing your map layer is small enough to fit into memory, you would just construct the scene graph once, and each render() request would simply move the view platform, depending on IRenderContext.getImageBounds(). No need to iterate over the feature collection for every frame, no need to do any low-level painting of individual geometries.

Right; so in this case you would make a graphics2d that renderer your
coordinates into 3d space - and set your world to screen
transformation to the identify matrix. This would produce your scene;
and you would only call this when your viewport model actually changed
the definition of the world ...

> And of course there are many more problems regarding layers, coordinate transformations and so on, but I would just like to get started somewhere...

I hope my email helped; I am familiar with Java3D; but really you will
need the input from a couple of players to make this a smooth ride.
Hopefully emily or jesse will also be able to get back to you.

Jody