|
Hi Mario,
It would be good to collaborate more with the Taverna
community. If only they were in the Eclipse Foundation, then it
would be much easier. One key difference between Taverna and
Ptolemy/Kepler/Triquetrum is that Taverna supports one model of
computation, whereas Ptolemy/Kepler/Triquetrum supports multiple
models of computation, see
http://dev.mygrid.org.uk/wiki/pages/viewpage.action?pageId=18974818
from May, 2014. This may have changed with recent Taverna work.
The Taverna community seems very active, one of their mailing
lists had 1748 messages in 2015. Triquetrum and Kepler could
learn much from Taverna.
Ptolemy does support RESTful interfaces as part of the Ptango
project (https://chess.eecs.berkeley.edu/ptango/). The Ptango
project has evolved in to Accessors, described below.
One goal I have for Triquetrum is to use it as a platform for
Accessors, which is a technology for composing heterogenous
devices and services in the Internet of Things. I see work that
involves sensing and actuating as a cyber-physical system (CPS).
IoT systems are a type of CPS. I see quite a bit of connection
between scientific computing involving computation and actuation
of large physical systems and CPS and thus IoT. An Accessor may be
a service such as a RESTful service that provides machine learning
about audio data. The Accessors work is under active development
by Professor Edward A. Lee and others. Details about Accessors may
be found at http://accessors.org.
_Christopher
What are Accessors?
Accessors are a technology,developed by the TerraSwarm Research Center,
for composing heterogeneous devices and services in the
Internet of Things (IoT).
An actor is a software component
that reacts to streaming input events and produces streaming
output events.
An accessor is an actor that wraps a (possibly remote)
device or
service in an actor interface.
Accessors embrace concurrency, atomicity, and
asynchrony.
The actor model, which governs interaction between accessors,
permits accessors to execute concurrently with segregated
private data and a message-passing interface for interaction.
Internally, many accessors use asynchronous atomic
callbacks (AAC) to invoke remote services and
handle responses asynchronously and atomically. See comparisons
with related technologies for insight into how accessors
work.
Accessors are defined in a _javascript_ file that includes a
specification of the
interface (inputs, outputs, and parameters) and an
implementation of the
functionality (reactions to inputs and/or production of
outputs).
Any _javascript_ file that conforms with the
accessor
specification
defines an accessor class.
The TerraSwarm
accessor library provides a collection of example
accessors.
This library is maintained via an SVN
repository
that permits many contributors to add accessors to the
library.
An instance of an accessor is created by a swarmlet
host
that evaluates the _javascript_ in the accessor definition.
At this time, there are at least three accessor hosts
compatible
with accessor specification 1.0:
- A browser
host, which allows inspection of the accessor,
and if the accessor is suitable for execution in a browser,
interactive invocation of the accessor.
- A Node.js
host, an interactive program that runs in
Node.js that allows instantiation and execution of
accessors.
- A Ptolemy
II host, which supports composition of accessors
with visual block diagrams and provides a large library of
actors that the accessors can interact with.
To experiment with accessors now, see Getting
Started with Accessors.
On 6/7/16 4:44 AM, Erwin de Ley wrote:
Hi Mario,
Thank you for the very interesting summary and points-of-view.
I must first of all apologize for the somewhat chaotic intro to
Triquetrum.
I was tired from traveling and was not well prepared for the kind
of "sales pitch" format ;-)
To address your remarks below, in the context of Triquetrum :
1. As I explained, a large part of the internal architecture is
inherited from a predecessor project called Passerelle.
This also uses Ptolemy II as its "engine", and mainly targets
server clusters as deployment platform for production usage.
Workflows get launched, hundreds of thousands a day, via remote
service invocations, scheduled jobs etc. (and only sometimes by
user triggers)
It has been developed as OSGi bundles since 10 years or so
allowing us to package it depending on the target environment
needs.
One incarnation is using a plain personal GUI in Swing (derived
from Ptolemy's Vergil). It can do workflows locally or connect to
a server to edit,configure models, run them on the server and get
progress info back.
The other extreme (which is not open source though) is a complete
web-based management GUI + web-based editor etc.
These approaches share most of their modules, but have different
interface/view components on top of that (a bit like the Sandia
packaging approach from the meeting yesterday).
The goal for Triquetrum in this respect would be to contain all
required service APIs and core implementations. But I don't think
currently that the web-based interface would be rebuilt in there
(unless we can get significant interest and collaboration to get
that done). I would try to reach a situation where the RCP
application could act as management tool and editor in a kind of
client/server setup.
2. About development languages : we're natively Java-based, but
integration with R, Python/Jython, _javascript_, Drools, ... is
available in different ways in Ptolemy, Passerelle and related
Science projects.
Python is on the list as the first "non-JVM" integration in
Triquetrum. The basis could be the work we've done in the past to
get that working on Passerelle workflow servers at ESRF.
That code has been prepared for reuse, this will be discussed this
week with Jonah Graham (Kichwa Coders) who has deep knowledge on
Java/Python integration and on the related Science projects.
FYI, the screenshots of the beamline experiment that you refer to,
were completely based on using a workflow definition as a
coordination of Python scripts that they already had.
And their custom GUI to control the beamlines is also Python and
Qt if I'm not mistaken, so we're surrounded by it! ;-)
3. The Triquetrum slides will be my talk at EclipseCon Toulouse
tomorrow. I'll make them available from our Wiki, and I guess
Eclipse will also upload them on the EclipseCon site.
Regards
erwin
Op 07/06/2016 om 09:23 schreef Mario Valle:
Dear group,
thanks for the yesterday meeting! It was really informative and
helpful to me.
One of my jobs here at CSCS is to find, if it exists, a workflow
manager that could be adopted for applications like the Material
Cloud, the NExT VRE European project, or in general for the kind
of computational experiments done at CSCS. The second goal is to
understand what these systems need to provide to become the
generic workflow management system that could be offered as a
service by CSCS.
That said, I should confess that I “kindly hate” workflow
managers that do everything locally behind a GUI. I found them
limited and not very user friendly. Maybe it is better to call
them, as KNIME does, “analytics platforms”. For these reasons, I
was pleasantly surprised to hear about the Sandia work and their
split desktop / HPC workflow “console”. Also the Triquetrum from
the slides (contrasted to the one downloaded from the Eclipse
project) seems heading in this direction.
The second thing I discovered yesterday is that integration with
HPC machines and workload managers is possible. Seems trivial,
but WF managers like FireWorks have a quite clumsy way to do
this. AiiDA is in a better shape, but it is inextricably linked
to the material simulation part. On a different path, Taverna
introduced the executable components as Web services and
Triquetrum will have REST services. In my opinion, this is the
right way to decouple the “console” (WF editor and management)
from the execution part.
Last consideration is about how to define and make available new
executable components. I grew up with massive doses of C, C++,
Shell, TCL/TK and Python so I tend to stay clear of every WF
tool that requires Java programming, but I welcome Python and R
scripting. From the presentations seems I'm not alone in this
request.
To make complete my learning I would like to ask one thing. Is
it possible to access the slides presented yesterday? And to
have pointers to the public tools mentioned? Forgive me if I
have misunderstood or forgot something from yesterday, but with
Socrates: “I know that I do not know”.
See you!
mario
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
--
Christopher Brooks, PMP University of California
Academic Program Manager & Software Engineer US Mail: 337 Cory Hall
CHESS/iCyPhy/Ptolemy/TerraSwarm Berkeley, CA 94720-1774
cxh@xxxxxxxxxxxxxxxxx, 707.332.0670 (Office: 545Q Cory)
|
