Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [udig-devel] selection problems
I don't know if it is possible but if there were a way you could put the activity information in the feature as an attribute you could do much more efficient queries. FID filters are inherently dangerous just for the reason that we've been discussing. As for cancelling it is hard because it requires pushing a monitor all the way down into and through the Geotools code. You could certainly detect if there is a problem by checking the number of FIDS. But I think the final solution is to try to minimize the dependency on fids.
More ideas... I'll think about this and work on making the table view more efficient. 

Jesse

On Oct 30, 2007, at 7:23 AM, Vince Darley wrote:


Hi,
I certainly hope there's a way to improve things! Even if we know in advance the filter is going to fail and lock up uDig, then we'd prefer simply to stop the operation ("Sorry you can't select so much" is obviously better than "The application has locked up and you have to kill it" ;-).
Just to add some information on what we're doing in our code, it's roughly as follows:
List<Activity> selectedActivities = [get potentially huge list of things from our route editors] 
FilterFactory ff = FilterFactoryFinder.createFilterFactory();
FidFilter idFilter = ff.createFidFilter();
for (Activity a : selectedActivities) {
        idFilter .addFid(activity.getLocationId());
}
(it's a bit more complex in reality, but I think the above is equivalent), 

and then finally for the layer we want to select on, we do this:
UndoableMapCommand createSelectCommand = (UndoableMapCommand) SelectionCommandFactory.getInstance().createSelectCommand( 
layer, filter);
layer.getMap().sendCommandSync(createSelectCommand);
if any of the above seems a bad or unperformant way of doing things, please do suggest improvements (the 'Sync' above seems peculiar, for example). (But, as I say, I see the same problem if I select 30000 items in the TableView). 

cheers,

Vince.

At 04:30 30/10/2007, Jesse Eichar wrote:

Paul's right.

I talked to Kevin and he has some ideas how we can improve the worst
case scenario of many FIDS.  I'm going to play with that strategy a
bit.  Also I can improve the filter provided/created by the table
view so if the selection is created by an entry in the text box the
filter will be more intelligent.  But the worst case where the user
selects a huge range of features in the table...  I'll have to think
about that one because the fixes Kevin is talking about will work for
PostGIS and The huge FID queries work fine for Shapefiles but when we
get into Oracle, ArcSDE, WFS, we'll have problems.


Jesse


On Oct 29, 2007, at 7:54 PM, Paul Ramsey wrote:


That one looks pretty clear... you're generating your selection set
by creating a huge SQL statement with a list of FIDs.  And when you
slam 50000 elements into the SQL statement as "A or B or C or D
or ... or Z" you kill something along the way.

You're also using OID as a key, which is bad mojo, since OID is
deprecated in PgSQL, you should be using a real primary key.  Can
you express your filter as something other than a huge list of
FIDs?  Probably "yes" with a selection rectangle, "no" from the
table view, "yes" from a query builder.  Hard problem.

P


On 29-Oct-07, at 10:07 AM, Jesse Eichar wrote:


Sounds like a problem with how we interact with postgis.  I'll
talk to our resident Postgis expert about this issue.

Jesse

On Oct 29, 2007, at 10:05 AM, Vince Darley wrote:


The renderer seems to become very unhappy when the selection
contains large numbers of items on the map.  We typically have
one layer of locations containing 50000+ points.  If (either
through our own code or through the TableView) the majority of
these become selected, uDig locks up.

The symptoms seem to be:

1) Error log like this (notice how I have trimmed some v.v.v.
long lines with IDs on them):

!ENTRY net.refractions.udig.project 2 0 2007-10-29 14:18:56.812
!MESSAGE Problem rendering: Exception rendering layer
DefaultMapLayer[ Test, VISIBLE, style=StyleImpl<NO_PARENT> [ name=Default Styler], data=net.refractions.udig.project.internal.impl.UDIGFeatureStore@1 2f bd5b, query=Query: 
   feature type: accesspoint_layer_3
   filter: [ 90638, 75594, 75595, 90639, 75592, 75593, 75598,
90634, 75599, 90635, 90636, 75596, 75597, 90637, 86716, 86717,
86714, 86715.....
   [properties:  ALL ]]
!STACK 0
net.refractions.udig.project.render.RenderException: Problem
rendering: Exception rendering layer DefaultMapLayer[ Test,
VISIBLE, style=StyleImpl<NO_PARENT>[ name=Default Styler],
data=net.refractions.udig.project.internal.impl.UDIGFeatureStore@1 2f bd5b, query=Query: 
   feature type: accesspoint_layer_3
   filter: [ 90638, 75594, 75595, 90639, 75592, 75593, 75598,
90634, 75599, 90635, 90636, 75596,.....
   [properties:  ALL ]]
        at
net.refractions.udig.render.internal.feature.basic.BasicFeatureRen de rer.render(BasicFeatureRenderer.java:334) 
        at
net.refractions.udig.render.internal.feature.basic.BasicFeatureRen de rer.render(BasicFeatureRenderer.java:193) 
        at
net.refractions.udig.project.internal.render.impl.RenderJob.startR en dering(RenderJob.java:101) 
        at
net.refractions.udig.project.internal.render.impl.RenderJob.run (RenderJob.java:188) at org.eclipse.core.internal.jobs.Worker.run (Worker.java:55) 
Caused by: java.lang.Exception: Exception rendering layer
DefaultMapLayer[ Test, VISIBLE, style=StyleImpl<NO_PARENT> [ name=Default Styler], data=net.refractions.udig.project.internal.impl.UDIGFeatureStore@1 2f bd5b, query=Query: 
   feature type: accesspoint_layer_3
   filter: [ 90638, 75594, 75595, 90639, 75592, 75593, 75598,
90634, 75599, 90635, 90636, 75596, 75597, 90637,
   [properties:  ALL ]]
at org.geotools.renderer.lite.StreamingRenderer.paint (StreamingRenderer.java:467) at org.geotools.renderer.lite.StreamingRenderer.paint (StreamingRenderer.java:374) 
        at
net.refractions.udig.render.internal.feature.basic.BasicFeatureRen de rer.render(BasicFeatureRenderer.java:309) 
        ... 4 more
Caused by: org.geotools.data.DataSourceException: Error
Performing SQL query: SELECT "oid", AsText(force_2d("the_geom"))
FROM "public"."accesspoint_layer_3" WHERE (("oid" = '90638') OR
("oid" = '75594') OR ("oid" = '75595') OR ("oid" = '90639') OR
("oid" = '75592') OR ("oid" = '75593') OR ("oid" = '75598') OR
("oid" = '90634') OR ("oid" = ......
at org.geotools.data.jdbc.JDBC1DataStore.executeQuery (JDBC1DataStore.java:933) at org.geotools.data.jdbc.JDBC1DataStore.getFeatureReader (JDBC1DataStore.java:672) at org.geotools.data.jdbc.JDBCFeatureCollection.reader (JDBCFeatureCollection.java:78) at org.geotools.renderer.lite.StreamingRenderer.getReader (StreamingRenderer.java:1173) 
        at
org.geotools.renderer.lite.StreamingRenderer.processStylers (StreamingRenderer.java:1272) at org.geotools.renderer.lite.StreamingRenderer.paint (StreamingRenderer.java:464) 
        ... 6 more
Caused by: org.postgresql.util.PSQLException: ERROR: stack depth
limit exceeded
        at
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse (QueryExecutorImpl.java:1525) at org.postgresql.core.v3.QueryExecutorImpl.processResults (QueryExecutorImpl.java:1309) at org.postgresql.core.v3.QueryExecutorImpl.execute (QueryExecutorImpl.java:188) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute (AbstractJdbc2Statement.java:452) 
        at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags (AbstractJdbc2Statement.java:340) 
        at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery (AbstractJdbc2Statement.java:239) 
        at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown
Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (Unknown Source) 
        at java.lang.reflect.Method.invoke(Unknown Source)
at org.postgresql.ds.common.PooledConnectionImpl $StatementHandler.invoke(PooledConnectionImpl.java:474) 
        at $Proxy5.executeQuery(Unknown Source)
at org.geotools.data.jdbc.JDBC1DataStore.executeQuery (JDBC1DataStore.java:922) 
        ... 11 more


2) The above messages take a long time to appear, and are then
repeated every 30-60 seconds while uDig is locked up.

Is there anything we can do about this?  (This is with rc12)

----

On an entirely unrelated note, we've now coded up a first version
of a BIRT plugin which allow a uDig map to be a BIRT report item
-- in the report designer you can drag and drop the map to a
report, size it as desired, and control certain aspects of the
rendering (through an extension mechanism).  There are still a
few bugs and problems (BIRT's extension API is sadly poorly
documented and requires hundreds of lines of code for some
seemingly routine things).  When this finally works well we'll
release it as a community plugin.

regards,

Vince.


Vince Darley
CEO, Eurobios UK Ltd
Tel: +44 207 921 0445
Mobile: +44 794 154 8527
www.eurobios.com

_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel


_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel


_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel


_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel



--
No virus found in this incoming message.
Checked by AVG Free Edition. Version: 7.5.503 / Virus Database: 269.15.12/1097 - Release Date: 28/10/2007 13:58 

Vince Darley
CEO, Eurobios UK Ltd
Tel: +44 207 921 0445
Mobile: +44 794 154 8527
www.eurobios.com

_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel

Back to the top