[ptp-dev] Handling stdout for large numbers of tasks in PTP

[Dave Wootton <dwootton@xxxxxxxxxx>]

Greg
I'm starting to look at some of the
large system scaling problems in PTP. It looks like a fairly simple place
to start working is the code which handles stdout processing for tasks
in the application. Currently, it looks like an output file is generated
for each application task when the corresponding PProcess object is created.
The result is that the directory specified by the PTP preferences output
directory setting gets filled with a lot of 0-byte files, unless each application
task actually writes to stdout. This results in several problems
1) An inode (or windows equivalent)
of disk space is consumed for each application task
2) System time is spent creating files
that may never be used. With a large number of tasks, the overhead is probably
substantial
3) PTP is holding file resources for
each task, which could consume all available file handles. I'm not sure
what the file handle limit in windows is, but suspect it's probably small.
4) The output directory has a large
number of files in it, which could result in poor performance when listing
(or browsing via file selector dialog), etc that directory

I think the place to start is to have
the PProcess constructor set up the filename to be used for the task, but
to not create a new OutputTextFile object. Then PProcess.addOutput() is
modified so that it checks for the existence of the OutputTextFile and
creates it if needed (on the first call to that method). Methods which
access outputFile for viewing, cleanup, etc) need to be modified so they
handle the case of a non existent file without raising exceptions or other
errors. I think that if the user double clicks an icon in the jobs view
for a task which has no stdout output, that he gets a popup stating there
is no stdout for that task. Alternatively, should the icon present some
state (maybe disabled) indicating there is no output (that could result
in additional state processing as the user scrolls thru the set of icons
in the job view, and therefore may not be a good idea)

Does this make sense?
Dave