Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [udig-devel] WMS-C Caching / MapInfo .TAB export

Hi John,

I don't have a lot of time at the moment to investigate the details of what you have said here but I have a few comments.

The tiled rendering system is "supposed to" eat up as much memory as it can. Basically it builds up the tiles and caches them so, if you come back to a tile and it is cached, then it reuses the tile. That said the tiles are kept in a soft hash map so they should be cleaned up and released before any out of memory exception occurs.

When just using the tiled rendering system are you getting out of memory errors? (not in associated with the preloading cache issue).

In terms of the amount of memory it uses by default; if there are 4 tiles on the screen than it basically will end up using about 2-4 times as much memory as the non-tiled rendering system. Most of the memory is used in storing the images. The individual tiles are smaller than the screen but since they go off the screen the combined image size of all the tiles is bigger than the screen. So how much more memory it actually uses depends a bit on your screen size and tile size etc.

I have not looked at or played much with the preloading cache issue. I would expect to get memory issues if I am trying to preload into a memory cache; but not with the disk cache. I guess it will take more investigation.

Emily



John Hudson wrote:
Thanks for that Emily, greatly appreciated.

I have fixed 4 issues related to the ones I listed last week:

* NPE on restart with WMS-C layers (2 issues there)
        ** Check for null on sequenceNumber, from capabilities doc, if sequenceNumber is null (where NPE was thrown) assume updated and update the capabilities ref
        ** Unique ID in catalogue now truly unique. (issue was org.opengis.referencing.crs.CoordinateReferenceSystem doesn't implement the hashcode method thus using the inherited one from Object)
* Disk Cache location now used instead of default "c:\tilecache" (in 2 spots calling incorrect persistence singleton 'CatalogPlugin', should be 'ProjectPlugin')

I'm not sure how to actually DO a patch from svn (I have been told that's how it's done), but I will have a look at the uDig wiki for the process and release these fixes before moving on.

I have had little success locating the memory leak which is killing the app when pre-loading (into disk cache & memory cache) the entire cache. This is eluding me completely, I have tried to do some profiling on the memory but will no success. One point to note is, I think the "Tiled Rendering System" has a big memory leak. With a few zooms it's easy to more than double the memory footprint of uDig and moving around for a few minutes the entire allocation of memory is used (I have mine set to 1024mb).

Notes:

At first I thought this maybe the desired functionality but after looking closely: starting memory is 122,836k (from max extent, on a restart of uDig, using the Tiled Rendering System) zooming in once (zoom-in button) memory footprint goes up-to 134,720k, that's a huge jump when there are no tiles actually retrieved from the tile server, this is only image processing on the client. Changing to NOT using the Tiled Rendering System sees memory down to (restart uDig & at max extend) 111,004k, and a single zoom is 114,324k. Zooming closer with the Tiled Rendering system off yields the same result, memory DOES increased but not as harshly. This is what pointed my initial thoughts to the Tiled Rendering System, I am probably far off :). Can anyone shed light on this? I wonder if this has anything to do with the preloading of the tiles?

SO... After releasing the above fixes, I will, for the time being put aside the pre-loading tiles issue and will start working on the addition of a plug-in that will allow users to specify an "area of interest" from a selected feature (WFS) then an operation that will allow the export of this bounded "area of interest".

Anyone have any suggestions how I should go about representing this in uDig?

I thought it would be nice to allow the User to right click on a feature (or multiple features) then an operation would be to "Add as area of interest" and possibly "Add to Area of interest..." (that would be a wizard, to select an existing AOI). Then a new view "Areas Of Interest" which would look similar to the Layer View but with the WMS-C node as a parent and the features as children of that layer(s).

Another possibility would be to extend the Layer view to show these Area Of Interest's. This would be the nice way to represent it, but might feel "cluttered" like there is too much going on in there already and might confuse the User?


Ok I'll stop now, if you're still reading, great and thanks.

Fyi, Cameron Shorter (Geospatial Architect here at LISAsoft) has a blog with some more information about our project (http://cameronshorter.blogspot.com/2009/01/loading-tilecache-images-onto-mapinfo_13.html), have a read.


Cheers and Happy Trails!


John


-----Original Message-----
From: udig-devel-bounces@xxxxxxxxxxxxxxxxxxxxx [mailto:udig-devel-bounces@xxxxxxxxxxxxxxxxxxxxx] On Behalf Of Emily Gouge
Sent: Saturday, January 10, 2009 2:57 AM
To: User-friendly Desktop Internet GIS
Subject: Re: [udig-devel] WMS-C Caching / MapInfo .TAB export

Hello John,

Have a look in the WMSTileSet.java class at the updateID().
Theoretically this method should return a unique identifier for each
layer that is supposed to be consistent across runs (at the moment it
doesn't look like it does that).

Emily



John Hudson wrote:
Hello, I have been working on a patch for the issues but have run into something that is a bit beyond me at the present:

The issue with restarting uDig and not having any wms-c layers re-rendered on the map with error: "Connection to Resource Failed..." seems to stem from the check in CatalogImpl when trying to resolve a handle (an IResolve) for the wms-c layer.

The upon checking each of the layers in a wms-c catalogue entry (in URLUtils.urlEquals ), fails. This is due to the stripRef which is  always different when the wms-c catalogue entry is re-loaded.

Does anyone know how this "ID" is created or saved in the memo, and if it can be changed from this seemingly random ID to something useful? So far I have had little luck tracking it down. Seems it could be a default thing in the SAX parser.

For some clarity the Map.umap is attached: the layersID is "http://wmsc1.terrapages.net/getpngmap?REQUEST=GetCapabilities&SERVICE=WMS?tiled=true#464378263";

But upon reloading uDig the ID for the catalogue entry will be something completely different, so it always returns the "Connection to Resource Failed..." error on that layer.

My forward apologies if this makes little sense...but does anyone have any ideas where I could go looking to solve this? Or any pointers?


Cheers, John


-----Original Message-----
From: udig-devel-bounces@xxxxxxxxxxxxxxxxxxxxx [mailto:udig-devel-bounces@xxxxxxxxxxxxxxxxxxxxx] On Behalf Of Graham Davis
Sent: Friday, January 09, 2009 3:10 AM
To: Jody Garnett
Cc: User-friendly Desktop Internet GIS
Subject: Re: [udig-devel] WMS-C Caching / MapInfo .TAB export

Hi Mr. Hudson,

We are not currently working on the WMS-C tile caching.  It is possible
it has some bugs as it was developed as a first step and version.  And
the tile preloading can definitely put a large strain on uDig and your
system since it is literally sending thousands of requests and writing
them to disk (in queued groups of course) but it doesn't surprise me
that it could crash.  If you see areas for improvements please feel free
do to so.

Graham


Jody Garnett wrote:
Hi Mr Hudson:

Sounds like you have your first couple of bugs to report; Graham Davis
was the developer working on this stuff and he may or may not be aware
of the problem. While you wait to hear from him perhaps you can look
into where the problem occurs and make a patch?

Jody

On Thu, Jan 8, 2009 at 7:36 AM, John Hudson <John.Hudson@xxxxxxxxxxxx
<mailto:John.Hudson@xxxxxxxxxxxx>> wrote:

    Hello everyone,



    I'm starting a custom build that will require features from the
    trunk, particularly WMS-C, the custom build involves mostly two
    fronts:



    1.       Disk caching TileCache

    2.       Exporting the disk cache MapInfo .TAB files.



    Is anyone currently working on WMS-C?  Jody mentioned in an email
    last month that we (LISAsoft) might have be getting some work on
    this, I guess that's my queue to say "we get the work, YAY!"... so
    looking at uDig, this is where I'm at:



    After building uDig from the trunk, there is a few issues I would
    like to fix:



                    testing with:


    http://wmsc1.terrapages.net/getpngmap?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetCapabilities&tiled=true
    <http://wmsc1.terrapages.net/getpngmap?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetCapabilities&tiled=true>


    http://labs.metacarta.com/wms-c/Basic.py?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetCapabilities&tiled=true
    <http://labs.metacarta.com/wms-c/Basic.py?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetCapabilities&tiled=true>



    ·         Upon restarting uDig, (most times) wms-c layers are not
    rendered and causes a NPE here:



    _java.lang.NullPointerException_

          at sun.misc.FloatingDecimal.readJavaFormatString(Unknown Source)

          at java.lang.Double.parseDouble(Unknown Source)

          at
    net.refractions.udig.catalog.wmsc.server.TiledWebMapServer.<init>(_TiledWebMapServer.java:93_)

          at
    net.refractions.udig.catalog.internal.wmsc.WMSCServiceImpl.getWMSC(_WMSCServiceImpl.java:187_)

          at
    net.refractions.udig.catalog.internal.wmsc.WMSCServiceImpl.members(_WMSCServiceImpl.java:105_)

          at
    net.refractions.udig.catalog.internal.CatalogImpl.getChildById(_CatalogImpl.java:537_)

    at
    net.refractions.udig.catalog.internal.CatalogImpl.find(_CatalogImpl.java:292_)



    ·         In Window à Preferences à WMS-C Tiles: Disk Cache
    Location is ignored when preloading cache (win32), it is instead
    just downloaded to "c:/tilecache"

    ·         uDig crash's when preloading WMS-C layers, (NPE in
    TileImageReadWriter.writeTile) (disk not full)



    Possible issues / functionality to be implemented:



    ·         On Disk Cache doesn't populate



    Once these are fixed I will need to add:



    ·         Cache expiry rules (as an extension to the current
    options in WMS-C Tiles)

    ·         Export plugin to export to MapInfo .TAB files







    Cheers,





    Johnathon Hudson
    Software Engineer

    LISAsoft Pty Ltd

    john.hudson@xxxxxxxxxxxx <mailto:john.hudson@xxxxxxxxxxxx>
    Level 1, 30 Currie Street
    Adelaide SA 5000 AUSTRALIA
    Ph: +61 8 8425 8050
    Fax: +61 8 8425 8099
    Mobile: +61 402 300 415
    Website: www.lisasoft.com <http://www.lisasoft.com/>




    ------------------------------------------------------------------------
    The contents of this email are confidential and may be subject to
    legal or professional privilege and copyright. No representation
    is made that this email is free of viruses or other defects. If
    you have received this communication in error, you may not copy or
    distribute any part of it or otherwise disclose its contents to
    anyone. Please advise the sender of your incorrect receipt of this
    correspondence.

    _______________________________________________
    User-friendly Desktop Internet GIS (uDig)
    http://udig.refractions.net
    http://lists.refractions.net/mailman/listinfo/udig-devel



--
Graham Davis
Refractions Research Inc.
gdavis@xxxxxxxxxxxxxxx

_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel

The contents of this email are confidential and may be subject to legal or professional privilege and copyright. No representation is made that this email is free of viruses or other defects. If you have received this communication in error, you may not copy or distribute any part of it or otherwise disclose its contents to anyone. Please advise the sender of your incorrect receipt of this correspondence.


------------------------------------------------------------------------

_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel
_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel

The contents of this email are confidential and may be subject to legal or professional privilege and copyright. No representation is made that this email is free of viruses or other defects. If you have received this communication in error, you may not copy or distribute any part of it or otherwise disclose its contents to anyone. Please advise the sender of your incorrect receipt of this correspondence.
_______________________________________________
User-friendly Desktop Internet GIS (uDig)
http://udig.refractions.net
http://lists.refractions.net/mailman/listinfo/udig-devel


Back to the top