Re: [ptp-dev] Updated Resource Manager Schema

["Albert L. Rossi" <arossi@xxxxxxxxxxxxx>]

Dave, my response in-lined below.
----- Dave Wootton <dwootton@xxxxxxxxxx> wrote:
> Al

> 1) In the list of elements in the control element, a bunch of them have 
> minOccurs=0 indicating they are optional. I noticed the cancel-job element 
> doesn't have minOccurs specified. Should there be a minOccurs 
> specification here?

Throughout I tried to reason through what must always be on a resource manager.  Certainly you will always want to cancel jobs.
The problem with the other commands is that any given one of them is optional, but at least one of the run commands are.  I probably need to group those into a choice which has min=1 max =3, so as to get the constraints correct.

> 2) Other elements like on-start-up have a minOccurs=0 specification, and 
> then the definition of on-start-up contains an element specification for 
> command elements that may also occur 0 or more times. To me, this means I 
> can omit the on-start-up element entirely, or I can specify one which can 
> itself have no command elements. Is this intended? If not, does this 
> complicate the generated classes?

OK, I should probably eliminate the 0 on the command elements.  That was an oversight (probably cut and paste hastiness).


> 3) Should the parsers element have a minOccurs specification? I'm thinking 
> I may not always want to process stdout or stderr with any parser logic.

Again, my reasoning was that you will always need at least one to capture the JobId from a run command.

> 4) All of the elements contained in the script element don't have a 
> minOccurs specification. To me this implies I must specify at least one of 
> each of these elements even though I may not actually care about them.

True.  These should be minOccurs=0.

> 5) The job-attribute element can contain a choice element, which I 
> understand is intended for cases where the user selects a value from a 
> predefined list. In some cases, PE accepts values that are in the 
> predefined list or an arbitrary value, where the distinction I make 
> between this and just a simple arbitary value is that my PE resource 
> manager resource tab displayes an editable combo-box with the predefined 
> values in the list and then the user can enter an arbitrary value in the 
> edit field in the combo box. I think that's a little more user friendly 
> than an editable text field where the user has to remember the predefined 
> values.

The choice is there to be used, at least as I think I set it up, in a read-only or non-read-only manner (see the attribute on widget).  If the widget says combo/read-only, then only those predefined values will be set; not read-only means the editable combo you mention above.

> 6) Does the job-attribute element need the notion of OS or mode (PE with 
> or without LoadLeveler as a back end scheduler) expressed as elements 
> within the job-attribute element? Is the expectation that each unique 
> permutation of OS, mode, and possibly other attributes is a separate 
> instance of a resource manager and therefore a unique configuration file?

If you think this better and won't clutter up the file, then certainly these could be added.

> 7) The job-attribute element has minValue and maxValue elements that I 
> think need a minOccurs specification. There are PE options which accept 
> numbers in the range 0..MAXINT, where no validation applies. I also have 
> validation where I could use a regular expression to validate, for 
> instance where a numeric field is expressed as something like 4K, 3M, etc. 
> Also, I do validation on pathnames, where depending on the specific field, 
> I need to validate that the specification is an accessible directory, a 
> writable directory, a readable file, a file that is accessible for read 
> and write.

As for attributes, leaving out use="required" is the same as minOccurs=0 for an element.  So neither of those are necessary, especially since they appear on all widgets (I did not try to do inheritance with this schema).

We can add a "validate" element.  How would you specify it?  Would these two uses 
<validate>
   <regex>...</regex>
</validate>

<validate>
   <acl>-rwxr-xr-x</acl>
</validate>

> 8) The command element contains one or two stream-parser elements. Should 
> this have minOccur=0 to make it optional? I'm thinking of cases where I 
> want stdout or stderr to go to a console viewer or be redirected to a 
> file.
Absolutely.  Another oversight.


> 9) How do I handle stdin, which may be an issue only for interactive and 
> debug runs?

Good question.  Do we currently?

> 
> It looks like what you have here will work for PE since it looks like PE 
> options can be specified by either environment variable name or the 
> corresponding command line option. I think I favor the environment 
> variables, since in the advanced mode case, I think a file consisting of a 
> set of name=value pairs is more readable than a set of command line option 
> and value pairs.
> 
> The LoadLeveler resource manager works a little differently. That resource 
> manager takes the values from the resources tab and uses them to update a 
> LoadLeveler job command file template, substituting actual values for 
> substitution keywords in the template file. 
This is the way PBS currently works too.

I'm thinking that the resource 
> manager needs to still generate the customized job command file then needs 
> to send that command file to the remote system where it can be specified 
> as the target of a llsubmit command.

This is what I contemplated by the <script> definition and the extra commands to be issued in connection with a run-command.  So you set up the configuration to tell the RM how to write out the script (if it sees a script definition) and your command for run would be
scp script somewhere
... then the (ssh) llsubmit ...

 Alternatively, the substitution 
> values get sent across as parameters on a command to generate a job 
> command file on the remote system and then issue a llsubmit command 
> specifying that file.
> 
> As a side note, I think any stdout/stderr redirection needs to be dealt 
> with on the remote (login/front end) node rather than in the Eclipse 
> client. If you handle the redirection on the remote node, then there's no 
> network traffic involved in sending what could be a large amount of data 
> to the Eclipse client.

I think we talked about this once, and roland has a few ideas about how to do this on the client side.  In any case, this config does not specify how it should be done.

Dave, thank you for the very careful examination of the config.  I will incorporate your corrections and suggestions.

> 
> Dave
> 
> 
> 
> From:
> "Albert L. Rossi" <arossi@xxxxxxxxxxxxx>
> To:
> Parallel Tools Platform general developers <ptp-dev@xxxxxxxxxxx>
> Date:
> 01/16/2011 06:09 PM
> Subject:
> Re: [ptp-dev] Updated Resource Manager Schema
> Sent by:
> ptp-dev-bounces@xxxxxxxxxxx
> 
> 
> 
> Greg, Randy, Dave,
> 
> I have posted a draft of the (control part) of the ResourceManager.xsd (
> http://wiki.eclipse.org/PTP/designs/resource_manager_xsd); please take a 
> look when you get a chance.
> 
> This is a valid XSD which will generate the necessary Java Classes using 
> the JAXB compiler.  I have not yet decided whether it will be simpler to 
> use JAXB runtime to build the actual instance objects from valid instances 
> of the XML, or whether we just might want to parse it in using a DOM 
> parser and then pull from it as necessary. 
> 
> Let me know about the holes here, or what you don't think is going to 
> work.  Undoubtedly we will need to iterate a few times.
> 
> Thanks, Al
> 
> _______________________________________________
> ptp-dev mailing list
> ptp-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/ptp-dev
> 
>