| [udig-devel] Discussion about UDIG functionality and some proposals |
Discussion about UDIG functionality and some proposals.
In context of future development process of UDIG platform-based GIS application specific to forestry sector after some internal discussions in our company I want to present some ideas and specific functionality proposals we are interested in. The goal is to follow together with UDIG and GeoTools community to greatly improve UDIG functionality.
From user point of view, data workflow and sequence of user actions to perform their business actions with a success I go inside of application architecture and data models design: and from here new development issues and challenges appear.
Model for vector data. Is topology important thing?
It is obvious during vector data editing operations it is necessary to preserve topological consistence between different vector objects. Feature model is not appropriate at all because of there is no topological information keeping when each feature is supposed as a standalone feature. The case is when two features with polygonal geometries touch each other on some common segment of boundary and we change boundary points of one of them. Here this operation is hard to trace in respect to the boundary points of second feature. I want to move common segment of both features simultaneously. When we have simple model of features with geometry without topological information it is hard to maintain topological relations.
So the proposal is to build Topology Data Model for such operations from a list of simple features with geometry. TDM is well known term and a good description can be found in Oracle DB documentation (Oracle Topology Data Model). We can transfer this model from relational world to object world.
The main subjects of Topology Data Model are face, edge and a node. TDM is very important and useful when vector data is represented by polygonal coverage where all features touch each other along common segments of boundary. All editing operations deal with topology data model. There is a straightforward way to polygonize TDM into simple feature model with geometries by traversal of TDM graph.
Each feature with polygonal geometry is a face. Each face contains list of edges that represent common parts of boundary with neighbor features. Each edge has a right and left face (one side can be empty – without face). Ends of edges are nodes… etc.
Topologically consistent feature model (without intersections of features, just touches of boundaries) can be decomposed into topology data model and back.
Algorithmic questions are behind this discussion and not an issue at all (for me as an applied mathematician). The open issue is there implementation of such elements (node, edge, face) that can be used as a Topology Data Model together with GeoTools object models and in UDIG. Oracle’s topology data model java library is for relational world and is not appropriate but can be considered as a starting point to port. The stuff that is inside of JTS already deals partly with topology models, but it is more related to general graph representation and analysis.. What is the most appropriate package to start taking into account described problems: com.vividsolutions.jts.geomgraph or com.vividsolutions.jts.planargraph, or… something else.. that is a problem.
Seems JTS is overloaded by object instantiations in every simple operation while the question of large amount of spatial data to be processed (from features with geometry to TDM, for example) is very important from performance point of view. In such case I want to deal on the more low level for my specific tasks, probably only with simple geometries, coordinates or coordinate sequences, for constructing TDM for example. In this context JTS stuff seems having a lot of unnecessary additional information and functionality.. my model is more specific to polygonal coverage, for example again.. But also I would like to have such TDM, elements of which are appropriate for different analysis operations already existing in JTS, so it should not be standalone object model, or it should be adaptable to JTS object models as easy as possible ( probably extending any JTS object model).
It is possible to implement everything from scratch based on some prototype models for internal proprietary functionality, but I desire to follow community (OpenGIS, GeoTools, UDIG, whatever else) needs in this problem domain.
From the point of view of TDM the family of editing tools grows significantly and adds a lot of new magic and powerful functionality to manipulate with topologically related features with geometry from any data store inside of UDIG.
This is a first important issue.
Export to…
We have good framework for importing data into UDIG. Wizards, wizard pages and etc. are configured as extentions and the workflow is uniform that is good. Are there plans to create the same framework of UI components for exporting anything from UDIG to anything outside? Or is common Eclipse platform functionality enough for those purposes?
I want, for example, to merge any number of layers with features of the same type to one layer and export it as a shapefile.. The issue: I again want to follow to common concept of exporting from UDIG.. as it exists with importing approach. Any suggestions.
Vector data operations (spatially related operations)
I have different vector layers (actually features with geometry that constitute vector layer) with one geometry type – polygons, lines – I want to have an opportunity to merge those layers together. This task even can be wider – merging everything with everything, calculating all intersections and dividing polygons if necessary…
It is only example of spatial operation over some layers. I have a JTS and JCS.. are these libraries good enough to use them for construction of more complicate and specific for my needs spatial operations over set of non-uniform geometry objects, layers, whatever with geometry..?
So, probably I need an advise about what is ready and appropriate for using inside of UDIG just not having dozens of different frameworks, libraries etc.. GeoTools and JTS are examples of mainstream technologies for using in UDIG. It is based on them, that is good, is there something else for my specific problems or such startup functionality exist already in any plugin/ext/lib/.. folders ?
Feature type management
The situation when I load features from shapefile without attributes (only with geometries) is common and I would like to have the functionality to create my own feature type for the layer with all those features and after that probably export the layer to another shapefile, but with attribute scheme as a dbf file. So we should have UI, probably wizard-based, to compose arbitrary feature type and set it for all features in the layer.
Chain could be the following:
Shapefile with only geometries -> load to UDIG environment -> create feature type -> set feature type for all features with default values automatically -> probably set attrubute values to arbitrary features we are interested in -> export to shapefile with dbf-based attribute information.
So, we need functionality to manage feature type models of the whole feature layers.
Shapefile as a mainstream data store
Now shapefile serves as a source for reading at least in two important cases:
· quering to get features based on the specified conditions as bounding box
· reading for rendering when only geometries are needed
The problem is in concurrent access. Concurrent access is unsafe because of bad synchronization between rendering thread and committing transactions thread. The behavior is unpredictable , a lot of different exceptions because of concurrent access and as a consequence – locking in Lock object of shapefile plugin.
The conclusion: we need very stable support of shapefiles, because of it is the most appropriate, open and quite simple format for many cases. We should have support of memory representation of feature model for shapefiles: features are loaded into memory data store once, probably additional topology model is built, whatever else… processing data in memory (editing actions) and exporting to new shapefile, for example.
Now after intensive testing there are bugs and exceptions as a result in context of multithreading access to shp or dbf files from UDIG. Lets’s find the best solutions.
MIF support
In our project the support of MIF format is needed. As I know the situation with GeoTools MIF plugins seems to be OK. Isn’t it a time to add MIF geo resource into UDIG and start to play with it, test, etc..? Is anybody interested in MIF support inside of UDIG?
RDBMS support
The issue of RDBMS support is also open. In future I would like to have rich functionality from UI of UDIG to create and perform different kinds of requests to DB to get spatial data, analyze, process and update back into DB. Is anybody interested in creation of such framework extending existing basic functionality of org.geotools.data.jdbc.* for different databases, especially Oracle?
This proposal can be compared with different Eclipse plugins for SQL quering and processing data from IDE. We can perform arbitrary SQL query and the data will be displayed in different tables , etc.. Why can’t we do great database framework for getting spatial data from database with powerful support of WHERE clause (just to get the data of interest, not the whole table loading). So, what we have now in plugin for Oracle is straightforward loading everything from everything.
Does it sound reasonable for developers who are interested in database as a spatial data store with rich functionality?
Vector data validation
Progress is obvious, let’s wait until validation plugin (based on GeoTools and JTS) is ready to test and use.
Conclusion
As a conclusion I briefly count domains we are interested in and going to add/improve into/in UDIG:
· Topology Data Model as an internal optional “feature” to represent vector data inside of memory to perform different editing actions and preserve topology consistence between features.
· Rich functionality for vector data validation as for standalone features with geometry as for the whole layers with dozens of objects to be validated. Seems great job is being done based on GeoTools validation functionality and integration it into UDIG.
· MIF support for reading at least. Writing is optional.
· Rich functionality for direct communication with databases, especially Oracle. Creation various complex queries to get only piece of data we are interested in.
· Export functionality: export selected layers with vector data in one shapefile. Probably spatial operations (merging, etc) should be performed automatically. It concerns cases when attribute data is not important, only geometry is subject for editing inside of UDIG.
· Functionality to create and even edit feature type for the whole layer, setting default value automatically for example, with editing in the next. Solve all problematic issues that appear with this functionality.
· Do all this stuff in a such way that is would be acceptable for the existing plugins, architectures, libraries, data models, object models, etc…
So, I suggest to discuss these questions between all stakeholders to find the best solutions, not to invent the wheel and determine what we are really need J