[udig-devel] Re: Work required for (was:) Spatial Operation and Editing

[Victor Mauricio Pazos <mauricio.pazos@xxxxxxxx>]

Hi Adrian, we appreciate a lot your reply, I think this debate will contribute 
to reach an useful group of features for uDig.
 

Thanks a lot for your reply :).
Comments in line.


On Thursday 25 January 2007 22:47, Adrian Custer wrote:
> Hey all,
>
> Congratulations to the Axios team for keeping at this! You've got some
> really useful functionality coming along. It's good to see you thinking
> globally about the design and I'm glad things are coming along so well.
>
> My comments below are meant to encourage you to take a usable design and
> make it fantastic. They come out of ideas I struggled with last year
> when I tried to do a buffer operation---they are meant to get a
> world-class design in place so we then can readily code up all the
> different kind of operations we will want.
>
> Do you have an update site so we could try out your work? If not I can
> only comment on what I see in the images you put on the wiki.
>
>
> Overall comments:
>
> Based on the experience of the Gnome project, we came to think in terms
> of workflow and speed. For example in buffering, we can expect a bulk of
> users to want to make a simple buffer at a fixed distance to see it on
> the map, another bunch will want to try several distances and eventually
> settle on one, another bunch will want to tweak the process in infinite
> detail. The best free software makes it easy to be fast by hiding the
> details from most operations; because we can't make decisions for users,
> we also need to provide a way to slow down and tweak all the detail. I
> would encourage you to think in those terms: fast for most users with a
> way to tweak the details if the fast is not right. So a user wanting a
> new layer with a 100 meter buffer should be able to generate this in
> five clicks max. For users trying different distances, you might want
> feedback on the map showing a preview of the buffer of at least some of
> the selected features. For users with complex needs, we can give them a
> button to access that.
>
> I'm still not sure about the trim/operation split. There is an elegant
> split for in place vs into new layer so if that's the split, I can be
> happy with the distinction. Tools essentially become operations with
> known destinations. However, I don't see that split as necessitating a
> view / dialog split. The trim tool, for example, might be better as a
> view so that the user can first select the layer to trim (say a property
> boundary polygon layer) but then later select from another layer the
> river feature to trim the polygon layer (say everything on the left bank
> of the river).

This is a good option if the user already have the feature to do the wished 
trim. I think It is a good alternative interaction. That issue was defined by 
stakeholders, in user story, I will talk about this issue with they thinking 
on your alternative. 

I see, operation/tool can be thought of as the same things, but we need 
different interactions to provide parameters. 

>
>
> OPERATIONS
>
> The buffer operation:
>
> 1) uDig has a serious design constraint due to the RCP design in having
> the visual map extent dependent on the outer shape of the user window
> and on the particular shuffle of the views on display in the GUI. What
> happens when a user first opens the buffer view? Does the map resize to
> show all the previously visible extent in the now smaller window?
>
>         JESSE, can this become a uDig wide behaviour 'Adding a view will
>         rescale the map so that the entire extent previously visible is
>         still visible'?
>
> 2) The UI in the view should be more strongly grouped. Could you put in
> a border around 'Source' and 'Target' (perhaps call the latter 'result')
> so a user would see a clean split of Source/Result/Options and the
> overall 'Accept' button. (BTW, there is no need for the close button
> since there's one on the tab and until the user has done something,
> nothing has happened.)

OK

>
> 3) The 'source' layer should be a list of layers and features. Somehow
> we need to suggest to users that they can simply do those selections
> with the mouse---That is, we need to show that the Source is 'active'
> and will change if the selection changes. (Thinking in more general
> terms, where operations require two selections, we need to show when one
> area of the Operation view is receiving input as against another. Think
> of a 'clip' operation where a user needs to select the clipped features
> and then select the clipping features.) Perhaps there is a way to put an
> alpha mask on all the non-selected groups and have the currently
> selected input group visible without a mask.

I agree on setting the source layer as a list where to choose it from, to make 
that interaction consistent through all the operations.
Regarding suggesting users what the operation workflow is, we thought of two 
feedbacks: the first is showing the #of selected features in the source 
layer, which reacts to normal udig selections. The second is to keep the 
Perform button disabled until all the operation inputs are settled.

>
> 4) We might want to generalize the approach to operations by considering
> the common elements involved. For example, all operations will require
> several GUI element groups:
>
>         Title
>         Demonstration visual (e.g. Unbuffered pts -> Buffered pts)
>         Options
>         PerformButton
>
> ideally, the 'Options' provide a limited set by default and an
> 'advanced' button to expand the options to more details. For buffer, the
> simple options are 'output layer' and 'distance'. The advanced options
> include:
>
>         which unit - 'distance in layer unit rather than renderer
>         units', (note that CRS is not enough since your two axes could
>         be in different units)
>
>         what result - if the 'buffer' is one merged multi-polygon or a
>         polygon for each input feature,
>
>         ...
>
> 5) We might consider a generic "Operation" view instead of a view for
> each operation. I imagine the tab would read "Operations" and the top
> would be a drop down list of Operations. Then below that we would have
> the standard
>         'visual/options | PerformButton'
> combination which would change according to the selected operations. The
> Options would have a standard
>         Input / Output / Settings | Advanced Settings
> split.
>
>         See the attached image for a mock-up (a quick and dirty
>         example).
>
> 6) The Operation views have to function in either the column or
> landscape view. That is, if possible, a user should be able to choose if
> they have it next to or above/below the map.
>
Great!, It is a good layout. 
>
> TOOLS:
>
> * On the toolbar, maybe the spatial tools should have their own group
> and perhaps a different shading (e.g. the green tools are spatial
> tools).

Good, we're glad to follow udig directions on this regard.

>
>
> Merge Tool
>
> 1) Where do the new1 and new2 come from in the dialog? As I understand
> it, we will only have a single new layer at the end of the tool
> operation. It sounds like you duplicated the source features to start
> with. If so, the user should *never* know that you are doing this.
> That's an internal decision you made about the implementation. From a
> user standpoint, I'm merging the two original layers into a new layer.

The story here is that the user wants to merge two features (no layers) onto a 
new one, thus the original features get deleted. The features presented on 
tree view come from a previous selection. This transaction has the problem 
that the model state will  be updated (deletes two features and creates a new 
one). Then the state of editor is not sure until the transaction finished.  
Therefore , the user can not change the value of features, using the Feature 
editor, when the merge transaction is in progress.
This case outlines the difference between tool and operations quite well. It 
modifies the source features, is undoable, etc.

>
> 2) The Feature merge dialog will need work. uDig needs a standardized
> approach for putting attributes from existing layers into a new layer.
> This is actually a very hard problem. Things like densities will have to
> be re-calculated. Some attributes will only apply to part of the merged
> results. Some will probably have to be hand edited as a late step in the
> merge process (obviously we can't do this until we have table editing
> capabilities but you should consider that as a step to be added
> eventually into the workflow).

This transaction is not exactly what our users require, the interaction 
required produce the result on the same layer (delete and create). We could 
have many features source and only one target feature (merged feature). We 
need that the user take the decision about what data may be saved on merged 
feature.

Anyway, thinking to merge some features to produce a new feature without lost 
the source features could be "other" operation that produce new features. In 
this case I don't see any problem that the user resolve the data problems as 
a late step. But this is a different matter. 

>
> 3) Note that the generic result of merging several layers of polygons
> will be a multi-polygon. It will be fairly rare to have a single polygon
> as a result. So I'm not sure if we want to standardize all outputs from
> the merge operation on the most general result for consistency.
>

I think we need two transaction: merge and union. Union could take the 
features selected on source layer (or more) and produce a result on some 
target layer. Union could be standardize like buffer, it only would create 
new features and does not do any change on source. Moreover the feature's 
data could be edited later.  

> Trim tool:
>
> 1) The arrow doesn't work visually for me. It seems like you should
> indicate what will be trimmed with both the arrow and the polygon to its
> right. I imagine a red arrow and a grey alpha mask polygon extending
> infinitely to the right---not sure how this would work on a 3D globe...
>
I agree, this tool require more feedback, we present only the functional idea. 
I think there are two feedback problem:

1)Trim Line Presenting: it must be highlighted from feature's geometries, 
selecting a color could be a problem (color deficiency). I think the user 
could personalize this issue. It could select color and others line patterns 
to present the trim line (maybe other tools too).

2) Result preview: your idea could be interesting (grey alpha mask). I am 
thinking something similar but mask will be applied on the area of features 
affected only. The reason why I say that is    the tool only affect the right 
side of feature under the trim line.     


> Trim/Split tools:
>
> 1) you might consider allowing the user to draw a line (continuous set
> of line segements) and trim or split based on that rather than on a
> simple line.
>
Done.

> 2) Similarly, you might allow trim/split based on a feature in a
> different layer.
>

I think this is the key idea. We have similar low level transactions with some 
inputs then we need to display a group of allowable user interactions to get 
that inputs. When the user has a geometry of reference in other layer he 
could use it (first group of interaction), but if he doesn't have a reference 
he need a sketch tool (trim, clip, split).

>
> Hope these ideas help you. Sorry that doing this well takes so long. We
> all know that creating a simple buffer process is simple. However,
> making the tools fantastic will have a huge impact on user experience
> and on the eventual adoption of uDig so I think it's worth the extra
> work.
>
> all the best,
> adrian
>
> On Thu, 2007-01-25 at 12:33 +0100, Victor Mauricio Pazos wrote:
> > Hi Adrian, we are planning Spatial Operation and Editing Tools project
> > and we have taken into consideration your suggestion.
> >
> > We have planned to refactor buffer UI in 0.1.0-rc1 Iteration.
> > http://udig.refractions.net/confluence/pages/viewpage.action?pageId=9643
> > Is it more adjusted to your idea? Do you want suggest anymore?
> >
> > Additionally, we agree about the tools and operation definition problems,
> > I can not find definitions but I think, the distinction between tools and
> > operations could be important to take some implementation decisions and
> > to establish a user language in uDig. Then, we propose initial
> > definitions and classify the project requirements taking into account 
> > some Paul e-mails (union merge sujects).
> >
> > http://udig.refractions.net/confluence/display/COM/Spatial+Operations+and
> >+Editing+Tools
> >
> > Nowadays, we are working in 0.1.0-m3 - Intersect, Clip, Trim, Split.
> > http://udig.refractions.net/confluence/pages/viewpage.action?pageId=9560
> > Comments?
> >
> > Thanks a lots
> >
> > Best regards
> >
> > --
> > Mauricio Pazos
> > www.axios.es
> >
> > On Wednesday 06 December 2006 11:19, Adrian Custer wrote:
> > > Hey all,
> > >
> > > These are my responses to the Axios team, and the the work I imagine
> > > will be required to have uDig support geospatial operations.
> > >
> > > My ultimate suggestion is: let's implement a really good buffering
> > > operation, solving all the issues related to workflow, user interface,
> > > and file creation with just that operation. Adding more operations
> > > afterwards will be fast and easy.
> > >
> > > --adrian
> > >
> > > On Tue, 2006-12-05 at 20:00 +0100, Victor Mauricio Pazos wrote:
> > > > Hi list!, we have begun a project to develop Spatial Operations and
> > > > Editing Tools features. This new features will be LGPL products.
> > > >
> > > > Our first goal are Spatial Operation (buffer and merge). The
> > > > following link has the project details.
> > > >
> > > >  http://udig.refractions.net/confluence/display/COM/Axios
> > > >
> > > > We expect yours comments.
> > > >
> > > > Thanks in advance
> > >
> > > Great to hear you are getting into this issue.
> > >
> > > First, this is hard in many ways most especially the user interface.
> > > There are many solutions and we may have to poke around a little.
> > >
> > > Second, because this is the first attempt at an operation interface,
> > > difficult workflow and GUI issues have to be resolved.
> > >
> > >
> > >
> > >
> > >
> > > On workflow:
> > > -----------
> > > In [[http://www.axios.es/projects/udig/html/]] you show a particular
> > > workflow for buffering operations
> > >
> > >   Select  ->  Invoke Buffer  -> Configure Output -> Configure
> > > parameters
> > >
> > > Two issues arise with this. First, new users don't know this is the
> > > workflow before starting their operation so they don't know they have
> > > to start by selecting. Your strategy provides no way to recover from
> > >
> > >   Invoke Buffering
> > >
> > > When developing the Gnumeric spreadsheet's graphing interface, we faced
> > > a similar problem, having to develop a way for users to select after
> > > calling the graphing wizard. For Gnumeric, this involves entering a
> > > weird state where you can select and do nothing else. This workflow is
> > >
> > >   Invoke Buffering -> Select
> > >
> > > uDig must support this strategy.
> > >
> > >
> > > The rest of the workflow should also be consistent; I believe it should
> > > be organized as:
> > >
> > >   Inputs  -> Operation parameters -> Outputs
> > >
> > > which would reverse the order of your wizard.
> > >
> > >
> > > So ideally, we have a workflow that goes
> > >
> > >   |Invoke Buffer|
> > >   |-------------| -> Configure parameters -> Configure Output -> run
> > >   |  Selection  |
> > >
> > > in which, when the user invokes the buffering operation, uDig opens a
> > > UI to the 'Inputs' page which shows the user the
> > > features/layers/attributes on which the operation will proceed. If a
> > > user (for example an advanced user) has pre-selected data, then the
> > > various inputs fields are filled out with that information. If they are
> > > empty or if the user wants to change the inputs, that is possible.
> > >
> > >         IMPLICATION: uDig needs a standard set of GUI elements which
> > >         together show the current selection.
> > >
> > >         IMPLICATION: uDig a way to go from that GUI element set back
> > >         into some sort of 'selection mode'.
> > >
> > > During parameter configuration, a user may also want to go back into
> > > screen mode. For example, a user may want to buffer by a distance that
> > > they know as a visual separation on screen but not as a metric. We may
> > > want to support a way for users to go back to the screen to click on a
> > > start and end point to get their buffer distance. So now we have
> > >
> > >   |Invoke Buffer|    |Configure parameters|
> > >   |-------------| -> |--------------------| -> Configure Output -> run
> > >   |  Selection  |    | Select Distance    |
> > >
> > > which is getting more complex, showing that the user may need to go
> > > back and forth to the map.
> > >
> > >         IMPLICATION: uDig needs a way to go from the Operation GUI back
> > >         into a 'input via map mode'.
> > >
> > > During output, we need to distinguish the processes that create new
> > > layers from those that modify existing layers even if the rest of the
> > > process is identical. Someone brought this up on the mailing list as
> > > 'operations' versus 'tools'. They have a good point but it may not be
> > > that we want to make an initial terminological distinction as much as
> > > let users pick which way they want to go: into same layer, into a
> > > temporary layer or into new, saved layer? With initial data or without?
> > > With initial attributes or without? Note, that for now, if we go into a
> > > new layer, uDig will ask the users on exit if they want to save the
> > > layer to a file.
> > >
> > >
> > >
> > > On GUI:
> > > --------
> > >
> > > The GUI needs to be (1) efficient (2) easy to figure out (3) consistent
> > > across operations. As I have just shown above, the GUI also needs to
> > > allow users to go back and forth between the map and the inputs.
> > >
> > > You've chosen the 'druid' or 'wizard' multi dialog approach. This is an
> > > easy choice when there is only one operation but one which may make
> > > less sense when a user wants to do repeated analysis. For example, if a
> > > user is trying to find by trial and error a buffer distance that works,
> > > having a druid is painfully slow.
> > >
> > > In eclipse it would be possible to have a 'view' for all the user
> > > input. This might be divided into three areas or three tabs depending
> > > on the complexity of the input. That view could even have a standard
> > > location, say across the bottom, in an analytics perspective. I suspect
> > > that this will lead to much faster workflow for anyone doing serious
> > > analysis.
> > >
> > > Ideally, I'd like us to think about these two approaches, and think
> > > about them in the eventual state where uDig will have twenty to thirty
> > > core operations and a massive number of user created operations.
> > >
> > >
> > > On a different tack, the GUI should provide users with some graphical
> > > examples of each action, so, for example, users can see the semantics
> > > of 'merge' without having to figure out exactly whose terminology is
> > > being used. These are all set theoretic operations which have been
> > > talked about with lots of different vocabulary in different domains.
> > > For each operation, I can guess what it might do without being sure of
> > > the semantics (eg do I get 1 feature or many at the end? Do I modify
> > > the layer or am I going to get a new one? Are new nodes being crated?).
> > > A diagram, carefully and correctly constructed, could save users having
> > > to read the text and interpret it really carefully.
> > >
> > >
> > >
> > >
> > >
> > > On the Buffering Operation:
> > > --------------------------
> > >
> > > You have set things up so your 'buffer' layer has as many features as
> > > the original. In common situations, this will *not* be what is wanted.
> > > The general use of buffer is to end up with a single geometry, the
> > > 'merge' of what you currently create. So we need to let users pick
> > > between the two results.
> > >
> > > Consider also buffering a stream network, which is made up of a series
> > > of interconnected stream segment features. Buffering the whole thing
> > > into one final geometry is straight forward. Buffering each feature is
> > > also straightforward but gives a result that is not really what will be
> > > desired. If we want to create 'areas of responsibility' we need to
> > > buffer each feature and then divide overlaps by nearest neighbourhood.
> > > That means, on an intermediate segment of the network, a stream feature
> > > buffered as a separate feature will have the two terminal semi-circles;
> > > those will have to be cut off by the vornoi.
> > >
> > >
> > >
> > > Conclusion:
> > > -----------
> > > As you see, even this simplest spatial operation, for which all the
> > > computational infrastructure is in place in JTS, still leaves a lot of
> > > work to be done. I suggest we focus on this 'Buffer' as an example of
> > > all spatial operations and get it working. We will need to keep in mind
> > > how the operations which need several input layers, e.g. clip and
> > > merge, or result in several output layers, e.g. divide, alter the
> > > workflow and UI requirements.
> > >
> > > hope that's enough to ratchet up your thinking a notch,
> > > --adrian