Part of my vision for OSGi is to be able to integrate services
running outside the virtual machine with an instance of OSGi
running inside the VM. This would not require having an external
registry, but rather provide a means for external services to
register with the registry and life-cycle manager in the OSGi
instance: a kind of remote services light.
I would also like to see a means of wrapping such services in a
bundle, so that versioning and OTA could also use the facilities
already available in OSGi. In other words, avoid duplication.
This is particularly useful for embedded systems.
Thus you approach is closer to what I propose than requiring
another Registry written in Python. The base Python runtime could
be a service as well as the actual application running on it. We
already use the Java ProcessBuilder to start and stop external
programs. POSIX or C Level Signals can be used to support full
life-cycle management. The integration with the service registry
is the interesting part. This is not just about finding services,
but also about how services communicate with their clients.
Python is an interesting use case. Python dominates application
domains where Java is weak and visa versa. It is not the only
interesting case, but it is a good place to start. I could see
programs written inC or C++, containers, GPU programs used for
applications other than graphics (GPGPU programs), and FPGA
applications as well. We have demonstrated how an FPGA bitstream
can be downloaded and run from our Realtime OSGi framework and
interface with it over javax.realtime.device APIs from the
Realtime and Embedded Specification for Java.
In any case, this looks like a fruitful area of cooperation.
Hi Jim,
First, thanks for expressing this sentiment. It's been my
feeling for some time that other languages/runtimes (like
Python) could absolutely benefit from the innovations from
osgi.
Just to be clear, whether or not [1] gets contributed to the
osgi-technology project, the repo at [1] is available and works
right now, and new contributors can and will be easily added.
Architectural Overview for the Repo at [1]
On the java side, the repo at [1] provides an ECF distribution
provider for OSGi Remote Services. The Remote Services
specification (chapter 100 and RSA 122) standardizes remote
service instance metadata (e.g. EndpointDescription and other
specified classes and service properties) for services in the
OSGi service registry.
The original assumption for remote services is that both
processes (service host and service consumer) would have a
service registry...since both were assumed to be Java/osgi
framework processes.
For the py4j distribution provider at [1] below, obviously the
java/osgi process has a service registry, and so can serve as
both a service host/impl, or a service consumer...or both...as
per the Remote Services and Remote Service Admin specifications.
For the python process, however, there is no service registry.
The purpose of the osgiservicebridge python component is to
provide a bridge to the Java-based service registry that uses
the standard Remote Services metadata to communicate between
the Java and Python processes (both directions). Note that
this approach (using standardized meta-data for inter-process
service interaction) can/could be done with any language or
runtime.
You might ask: but what about the osgi service registry,
remote services, (and e.g. declarative services) on non-Java
processes? That too can be implemented and that's where I've
chosen to reuse the excellent work of the ipopo project [A].
ipopo/pelix already has a python-based bundle layer, a service
registry, and many of the features of a full osgi framework.
Years ago, I contributed to ipopo an impl of RS/RSA (in
pelix/rsa module and sub-modules) and also contributed a
python-based distribution provider (along with a few other
distribution providers) that uses the osgiservicebridge to
communicate between the ipopo/python service registry and the
java service registry. With ipopo+osgiservicebridge running in
python, it's possible to have python-based service impls
register themselves in the ipopo service registry, be exported
by the python rsa impl, be imported by the java/osgi rsa impl,
and consumed by the java process (e.g. injected by declarative
services). A tutorial with an example ipopo service registry
<- RSA -> java service registry interaction is here [B].
What's the point? As the sole implementer of the repo at [1],
I already make this functionality available via [1]. I can also
contribute this codebase in some form to the osgi technology
group, and am proposing to do so. It could be contributed
under the ECF project (which I lead) or under a new project.
I've spoken with the creator and maintainer of ipopo (Thomas
Calmant) and he would consider contributing ipopo + the ipopo
RSA impl, but that would need to be a separate discussion from
the one I'm initiating here since I don't hold the ipopo
copyrights for the whole ipopo codebase.
There are 3 other features in the Python.Java distribution
provider that have not been completely documented yet. If
interested in another email describing those features please
just respond to this posting positively and I will describe
these features with example code references from [1].
Thanks,
Scott
[A] https://ipopo.readthedocs.io/en/1.0.1/
[B] https://ipopo.readthedocs.io/en/1.0.1/tutorials/rsa_pythonjava.html#
On 7/9/2024 4:34 AM, Dr. James J.
Hunt via osgi-technology-dev wrote:
Dear Colleagues,
I find this project interesting to have integrated into
OSGi. We are generally interested in extending the OSGi model
to code running outside a JVM and this goes in that
direction. If we can add contributors, even better.
Sincerely,
James
On 7/8/24 17:10, Scott Lewis via
osgi-technology-dev wrote:
On
7/8/2024 5:06 AM, Mark Hoffmann via osgi-technology-dev wrote:
Hi Scott,
thank you for this request. In general we are interested
having such a provider that allows communication between
Python and Java via Remote Services.
We already doing that as well in a custom component to
trigger ML applications, that are implemented in Python.
So, having a distribution provider would be nice.
I have seen the Py4J project made its last release in 2022.
Is there still something happening in that project?
I'm aware that the primary maintainer for many years
(Barthelemy Dagenais) stepped away from that role a couple of
years ago, but that others (Apache Spark committers, Eclipse
Foundation committers such as Jonah Graham, who is a committer
on several Eclipse projects) have taken up that maintenance
mantle. In my usage for implementing a distribution
provider, I've not found any major problems so far.
Just to get it right, the contribution would consist of
Python and Java components?
Yes. The repo I've provided has several osgi bundles, and
the primary python component (in that repo) is the
osgiservicebridge, which is available via pip.
Just to be clear, some years ago I also contributed a full
python-side RS/RSA implementation to the iPopo project. https://ipopo.readthedocs.io/en/1.0.1/
in rsa package. This python impl of the pelix/ipopo/RSA
implementation depends upon osgiservicebridge.
It would be great, if we could discuss
that topic, maybe in one of the next specification calls. I
would clarify that. Anyway, you are always invited to
participate these calls.
Please let me know when this is to be discussed and I'll see
if I can attend. I would prefer written questions and
answers, however...either here or some other appropriate
forum.
Scott
Regards,
Mark
Am 04.07.24 um 05:45 schrieb Scott Lewis via
osgi-technology-dev:
Howdy,
For many years, the ECF project has had an implementation
of the OSGi Remote Services (100) and Remote Service Admin
(122) specifications. I believe ECF's RS/RSA impl is
currently still used in the OSGi compatibility test suite.
One of our distribution providers enables remote services
between Java and Python[1]. It uses the Py4j protocol to
do this. Py4j is an open source project [2] that is used
for Apache Spark on the server-side (I believe), Eclipse
EASE project, and other projects and products/services.
As per the OSGi Remote Services specification, a service
registry exists in both the Java and Python processes, and
the distribution provider connects them (via
osgi-standardized meta-data aka the endpoint description)
to export and import services between the two processes.
[1] has all the Java and Python components to implement
this distribution provider, along with some utilities and
examples. This distribution provider can also use Google
protocol buffers as the serialization mechanism and so it
happens to work well with the both gRPC (based upon
protocol buffers) and bndtools-based grpc tooling [3].
If there is interest and community support I would be
willing to contribute this distribution provider to the
Eclipse community, perhaps as an effort to integrate java
and python-based services and apps at the service level
for a broader 'artificial intelligence' effort at Eclipse.
Scott
[1] https://github.com/ECF/Py4j-RemoteServicesProvider/
[2] https://www.py4j.org/
[3] https://github.com/ECF/grpc-RemoteServicesProvider
_______________________________________________
osgi-technology-dev mailing list
osgi-technology-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://accounts.eclipse.org
_______________________________________________
osgi-technology-dev mailing list
osgi-technology-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://accounts.eclipse.org
--
Dr. James J. Hunt
CEO & Geschäftsführer
aicas GmbH
Emmy-Noether-Straße 9 ● 76131 Karlsruhe ● Germany
https://www.aicas.com ● Tel: +49
721 663968 22
USt-Id: DE216375633, Handelsregister HRB 109481, AG Mannheim
Geschäftsführer: Dr. James J. Hunt
aicas incorporated
6 Landmark Sq., Ste 400 ● Stamford, CT 06901 ● USA
https://www.aicas.com ● Tel: +1
203 435 0521
aicas America limited
4023 Kennett Pike, Ste 810 ● Wilmington, DE 19807 ● USA
https://www.aicas.com ● Tel: +1
203 435 0521
_______________________________________________
osgi-technology-dev mailing list
osgi-technology-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://accounts.eclipse.org
_______________________________________________
osgi-technology-dev mailing list
osgi-technology-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://accounts.eclipse.org
--
Dr. James J. Hunt
CEO & Geschäftsführer
aicas GmbH
Emmy-Noether-Straße 9 ● 76131 Karlsruhe ● Germany
https://www.aicas.com
● Tel: +49 721 663968 22
USt-Id: DE216375633, Handelsregister HRB 109481, AG Mannheim
Geschäftsführer: Dr. James J. Hunt
aicas incorporated
6 Landmark Sq., Ste 400 ● Stamford, CT 06901 ● USA
https://www.aicas.com
● Tel: +1 203 435 0521
aicas America limited
4023 Kennett Pike, Ste 810 ● Wilmington, DE 19807 ● USA
https://www.aicas.com
● Tel: +1 203 435 0521