Hello Janos,
This is a great idea! Making Jython OSGi is something we have talked about often internally and occasionally with Jonah from Kichwa coders. It would solve many
problems with Jython.
However Jython is less important to science as a whole. DAWNSci have created a numpy-like API in Jython but the power of numpy/scipy is not there and cpython
will generally have more APIs which are more familiar to python programmers.
Therefore the AnalysisRPC project was created to make communication with cpython and Java easy. On the Java side we have IDataset which is a Java representation
of an ndarry (and can be used in Jython) on the cpython side this becomes a ndarray in numpy. Jonah made it fast, there is arguably an issue with copied memory but with 64-bit machines where this has been hit, we just get more memory.
Sincerely,
Matt
From: science-iwg-bounces@xxxxxxxxxxx
[mailto:science-iwg-bounces@xxxxxxxxxxx] On Behalf Of Janos Binder
Sent: 23 June 2015 10:17
To: Science Industry Working Group
Subject: [science-iwg] Integrating OSGi bundles into P/Jython
Dear all,
this is impressive what is happening at Science IWG recently. I am new to the list and we are working on OpenChrom with Philip Wenig. We have also trying to implement Jython scripting into our Eclipse RCP-based project, however we have not figured out how to
integrate OSGi bundles dynamically. When we look at Jython implementation in DAWN for example, I see that the jar files are added to the PYTHONPATH manually.
Would it be possible to get the OSGi bundles (via Activator.getContext().getBundles()) and add it to the Jython Intrepreter whenever it is initialized (e.g. via PySystemState or PythonInterpreter.initialize)?
Best wishes,
Janos
On 23.06.2015 08:58, Erwin de Ley wrote:
Tracy, all,
(I've cc-d the Science IWG, as I think this is becoming an interesting cross-project discussion, I hope this is OK... )
First of all it's great that you're presenting the Java/Python work at eclipsecon.
I wish I could have participated as well, I'll try to be free for the November conference.
The work done for DAWN is impressive, integrating Python scripting in many aspects of the RCP workbench.
And this may indeed be of value also for ICE and other integrated scientific workbenches.
(Jython doesn't seem to be sufficient in all cases. I learned from Matt and Olof that there's a need for an integration of "real" Python to ensure that all popular Python libraries can be used, hence the work on AnalysisRPC.)
Triquetrum hasn't got the same ambition, i.e. it's not the goal to become a complete analysis workbench.
The focus is on workflows, and I would imagine/hope that our plugins could then be integrated in larger applications, similarly to the approach with Passerelle and DAWN.
So I would think that we would mainly target the "AnalysisRPC" approach, i.e. the Python-related requirements from Triquetrum would focus on (if I understood the scope correctly of AnalysisRPC):
- being able to launch a Python runtime in a separate process
- load system scripts in there to set-up an "rpc server"
- load user scripts
- flattening/marshalling of Java data structures and sending them over to the Python side
- execute the user script
- return result data
- unmarshall/unflatten it to Java structures again
Some more advanced topics that we didn't tackle yet in "Passerelle with AnalysisRPC", would be :
- efficient handling of large concurrent loads : I would imagine that it's not optimal to start new python processes for each script execution.
Is there a way to reuse/share a pool of processes, or execute scripts concurrently in a single multi-threaded Python "server"?
(cfr evolution for web servers for CGI/FASTCGI/apache mods/...)
- debugging : if there would be a way to have integrated python debugging on scripts running on the remote python instance, that would be a tremendous feature (cfr remote Java debugging)
Of course, any inspiration is welcome for extra features that would be useful in the context of defining/running/debugging/monitoring workflows!
erwin.
Jay Jay Billings schreef op 22/06/2015 om 23:25:
Tracy,
I would really love to talk to you about this. We are just now building out our scripting stuff and I'm happy to share our requirements.
We are targeting Eclipse EASE and Jython at the moment.
Jay
On Jun 22, 2015 4:43 PM, "Jonah Graham" <jonah@xxxxxxxxxxxxxxxx> wrote:
At the conference we will also be talking to other members of the Science Working Group who have expressed interest in the technology so we can capture requirements, scope the project appropriately and determine
whether/how it can be resourced. Also, the project is part of DAWN today, but we would like to consider where it would be best to live (for example, Pydev or also alongside some of the newer Eclipse scripting projects).
So I will be in a better position to give you an indication after the conference. It will be good to have your input into the requirements capture as well.
Regards,
Tracy
On 22 June 2015 at 18:02, <Matt.Gerring@xxxxxxxxxxxxx> wrote:
Hello Erwin,
>>
Matt, do you have more details on if/how/when this would become a separate eclipse project or not? If that would not yet be available, I assume we could initially work with the forked version I created for Olof (ESRF), last year?
Tracy and Jonah from Kichwa coders will be project leads for the AnalysisRPC project I think. I have
taken the liberty of copying them in for their thoughts on the timescales. In the meantime using the python connection from AnalysisRPC which we abstracted for your work at the ESRF with Olof would be ok to use. It is a subset of all that can be done however!
Matt
From:
Erwin de Ley [mailto:erwin.de.ley@xxxxxxxxxx]
Sent: 22 June 2015 16:52
To: Jay Jay Billings
Cc: Gerring, Matt (DLSLtd,RAL,LSCI); Christopher Brooks
Subject: triquetrum dependencies overview
Hi Jay,
I have attached an overview of the initial set of dependencies for Triquetrum, i.e. for the initial code drop and the work in the months after that (as far as I can predict this and ignoring more detailed requirements coming from ICE, SAW, ...).
I've checked on eclipse ORBIT and most things seem to be available there.
There's mainly the ptolemy stuff that's new of course. Christopher already gave details on copyrights/licenses on the forum.
In the second phase, we would get REST services to manage workflows remotely, and this is currently based on JBoss RESTEasy.
Another important topic would be Python integration via the DAWN work.
Matt, do you have more details on if/how/when this would become a separate eclipse project or not?
If that would not yet be available, I assume we could initially work with the forked version I created for Olof (ESRF), last year?
Do you think it's worth getting this done via ICE, seeing the rather limited set of potential issues?
--
Met vriendelijke groeten - Bien à vous - Kind regards
Erwin De Ley
![http://thinkbeforeprinting.org/struct/signature-1.gif](pngs7K9js063O.png)
--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee
please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
--
~~~~~~~~~~~~~~~~~~~~~~~~
Dr. Janos Binder » Software developer » ph +49 (0) 40 428 38 - 3140 » janos.binder@xxxxxxxxxxxxx » Lablicate UG (haftungsbeschränkt) » Martin-Luther-King-Platz 6, 20146 Hamburg » Germany
Lablicate UG (haftungsbeschränkt), Amtsgericht Hamburg, HRB 129018, USt-IdNr.: DE291156854
Geschäftsführer: Dr. Philip Wenig, Dr. Andreas Klingberg
The information contained in the email may be private and confidential. If you are not the intended recipient(s) or have otherwise received this email in error, please delete the email and inform the sender as soon as possible. This email may not be disclosed, used or copied by anyone other than the intended recipient(s) Any opinions, statements or comments contained in this email are not necessarily those of Lablicate UG. If you wish clarification of any matter, please request confirmation in writing.
~~~~~~~~~~~~~~~~~~~~~~~~
--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
|