Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jakartabatch-dev] Remote start / submit of Java-defined jobs?

Thanks Cheng,

So I might stop and summarize and say "yes, there is some interest in having a registration mechanism for a Java-defined job", though JBeret itself doesn't already provide one.

So I did open a separate issue: https://github.com/eclipse-ee4j/batch-api/issues/177 for that, where we can discuss some of these ideas more (not that we can't use the ML too).

Thanks,
------------------------------------------------------
Scott Kurz
WebSphere / Open Liberty Batch and Developer Experience
skurz@xxxxxxxxxx
--------------------------------------------------------

Inactive hide details for Cheng Fang ---04/08/2021 08:55:12 AM---Just to share some experience when implementing Java-based jobCheng Fang ---04/08/2021 08:55:12 AM---Just to share some experience when implementing Java-based job definition in project JBeret. We (JBe

From: Cheng Fang <cfang@xxxxxxxxxx>
To: jakartabatch developer discussions <jakartabatch-dev@xxxxxxxxxxx>
Date: 04/08/2021 08:55 AM
Subject: [EXTERNAL] Re: [jakartabatch-dev] Remote start / submit of Java-defined jobs?
Sent by: "jakartabatch-dev" <jakartabatch-dev-bounces@xxxxxxxxxxx>




Just to share some experience when implementing Java-based job definition in project JBeret. We (JBeret team) strived to keep consistent between the classic JSL and Java-based job definition. So the definition of job is separate from the launch of a job: users can create a job via either JSL or api, and start the job execution via the same JobOperator interface. JBeret added a non-standard start method in job operator impl which takes a Job parameter instead of a string job name.

To support java-based job definition, JBeret leverages its internal job model and exposes some of it public to allow programmatic creation of job definition, while adding a thin layer of builder classes. For the spec, we need to consider how much of the job model we're willing to expose to support this.

The full set of JBeret's job model: https://github.com/jberet/jsr352/tree/master/jberet-core/src/main/java/org/jberet/job/model

This test case shows how JBeret supports programmatic job definition: https://github.com/jberet/jsr352/blob/master/test-apps/javaJSL/src/test/java/org/jberet/testapps/javajsl/JavaJSLIT.java

On Tue, Apr 6, 2021 at 12:09 PM Romain Manni-Bucau <rmannibucau@xxxxxxxxx> wrote:
    AFAIK JBeret ends by needing to call start(Job,...) on the operator which has the drawback to loose the job registry feature which is key as soon as you have an UI/API on top of job operator so the registration at startup is important for me - otherwise it kind of defeats the management API JBatch provides with JobOperator and the discovery.
    About the CDI producer vs startup event registration (likely with a fluent API as CDI configurators in both cases) it is mainly the startup and runtime impact it implies I think.
    On startup the event is clearly faster in general (1 vs N) and at runtime it is a bit faster too since it will not impact other parts/lookups+resolutions of the application for no reason (the beans are actually needed only once) so it makes lookups faster and use less memory. It also avoids some unexpected ambiguous error in some cases adding new jobs or composing libraries of jobs.
    Last point is the coding style but both cases are pretty equivalent and can rely on injection to ease the writing of repeatitive parts so I guess the coding style is not key between both options. The event makes the callback explicit whereas the producer will require the AfterDeploymentValidation event to init all beans implicitly - which can break surprisingly if a producer was relying on a bean not available at that time.

    Hope it helps a bit,
    Romain Manni-Bucau
    @rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book


    Le mar. 6 avr. 2021 à 17:52, Reza Rahman <reza_rahman@xxxxxxxxx> a écrit :
    It looks like what JBeret does is automatically register a job defined programmatically: https://jberet.gitbooks.io/jberet-user-guide/content/programmatic_job_definition_with_java/. That seems a fine approach. Is there a way to get those folks to chime in? Maybe they would say that a CDI producer based approach is actually better and more type-safe? What does Spring Batch do?

    Reza Rahman
    Jakarta EE Ambassador, Author, Blogger, Speaker

    Please note views expressed here are my own as an individual community member and do not reflect the views of my employer.

    On 4/6/2021 8:56 AM, Scott Kurz wrote:

        I'm not trying to take us too far into orchestration.

        But we need to decide if we're going to define "registration" of the Java-defined job such that some other Jakarta component: servlet, etc. can start the job         by name in some context (WAR, some other scope, etc., like the XML definition in WEB-INF/classes/META-INF/batch-jobs provided for us.

        So while one might argue this shouldn't be a spec concern, it seems Romain (who sketched out a piece of this) and Reza think this is worth discussing more.

        I don't see JBeret doing this but I'm not that familiar. Reza suggested maybe this can be done with CDI producers. If the "standard" way were tied to CDI only, I think that would be OK.

        ------------------------------------------------------
        Scott Kurz
        WebSphere / Open Liberty Batch and Developer Experience
        skurz@xxxxxxxxxx
        --------------------------------------------------------

        Inactive hide details for Michael Minella ---04/05/2021 12:22:32 PM---With regards to this issue, if I could provide some visibMichael Minella ---04/05/2021 12:22:32 PM---With regards to this issue, if I could provide some visibility to how Spring Batch views this. The l

        From:         Michael Minella <mminella@xxxxxxxxxx>
        To:         jakartabatch developer discussions <jakartabatch-dev@xxxxxxxxxxx>
        Date:         04/05/2021 12:22 PM
        Subject:         [EXTERNAL] Re: [jakartabatch-dev] Remote start / submit of Java-defined jobs?
        Sent by:         "jakartabatch-dev" <jakartabatch-dev-bounces@xxxxxxxxxxx>




        With regards to this issue, if I could provide some visibility to how Spring Batch views this. The launching of a job and the mechanisms involved in it are really an orchestration concern. Spring Batch, from the beginning, has taken no position on how jobs are orchestrated. We provide a single class that could help as a utility, but most of the time it wasn't used. Spring Boot provides abilities to launch batch jobs. Spring Cloud Data flow can launch batch jobs. Cron, Control-M, etc all can launch batch jobs. But Spring Batch itself defers from orchestration concerns. This is by design to allow Spring Batch to integrate with whatever orchestration tool an enterprise chooses to use. If you are familiar with batch systems in large enterprises, orchestration tools can be a bit of a religion and not a place that is easily changed. It's best to allow integration with whatever the enterprise has choosen than to be prescriptive about how to orchestrate batch jobs.

        Just my two cents.

        Thanks,
        Michael Minella         (He/Him)
        Sr. Manager - Spring Engineering
        mminella@xxxxxxxxxx
        3401 Hillview Avenue, Palo Alto, CA 94304



        From:         jakartabatch-dev <jakartabatch-dev-bounces@xxxxxxxxxxx> on behalf of Romain Manni-Bucau <rmannibucau@xxxxxxxxx>
        Sent:         Sunday, April 4, 2021 9:55 AM
        To:         jakartabatch developer discussions <jakartabatch-dev@xxxxxxxxxxx>
        Subject:         Re: [jakartabatch-dev] Remote start / submit of Java-defined jobs?

        Just to try to highlight one point of Reza's answer: it is exactly the goal of a java dsl: not have any "reference" which are actually indirections making it harder for end users without any real gain. With the java dsl you get the injection as in any CDI app (or spring) in the job defining bean and you reference the jbatch bean/backing bean with a .... reference not an alias in a lambda.
        Spring xml -> java movement was exactly this - even if technically a bit different - and today almost nobody write descriptors anymore (from JakartaEE which drops it slowly to spring).

        Ex:

        void defineBatches(@Observes JobRegistrationCallback cb, JobOperator op, MyService service) {
        cb.defineJob("my-job")
        .batchlet("exec-sql", () -> service.executeBulkQuery().then(service::flush));
        }


        Romain Manni-Bucau
        @rmannibucau | Blog | Old Blog | Github | LinkedIn | Book


        Le dim. 4 avr. 2021 à 16:33, Reza Rahman <reza_rahman@xxxxxxxxx> a écrit :
            If you look at the JBeret implementation, that looks about right to me. In general, job configuration would be rather static except for the parameters perhaps. I would expect to define it in a factory or CDI producer and reference it later, ideally in a type safe way or by name in the least. There is a case for dynamically building jobs, but it is fairly rare.

            Hopefully the JBeret and Spring Batch folks can chime in? Both should have some implementation experience on this already?

            Reza Rahman
            Jakarta EE Ambassador, Author, Blogger, Speaker

            Please note views expressed here are my own as an individual community member and do not reflect the views of my employer.
                    On Apr 1, 2021, at 8:49 AM, Scott Kurz <skurz@xxxxxxxxxx> wrote:


                    I wanted to call out a point on the ML here, on the Java-defined job:                     https://github.com/eclipse-ee4j/batch-api/issues/109

                    Are we just looking to provide an API to dynamically construct a job and then go and immediately execute it?

                    I'm sure that'd be useful. I mean to say, are we ALSO looking to provide an API to "register" the job by name so that it could be easily invoked later, by name and job params, for example?

                    Using the existing interface for starting an XML-defined job:
                    start(String jobXMLName, Properties jobParameters)
                    it's certainly easy to expose that remotely.

                    I haven't looked at the existing implementations and their approaches yet.

                    But wondering what people were thinking this would look like?

                    ------------------------------------------------------
                    Scott Kurz
                    WebSphere / Open Liberty Batch and Developer Experience
                    skurz@xxxxxxxxxx
                    --------------------------------------------------------
                    _______________________________________________
                    jakartabatch-dev mailing list
                    jakartabatch-dev@xxxxxxxxxxx
                    To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartabatch-dev
            _______________________________________________
            jakartabatch-dev mailing list
            jakartabatch-dev@xxxxxxxxxxx
            To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartabatch-dev_______________________________________________
            jakartabatch-dev mailing list
            jakartabatch-dev@xxxxxxxxxxx
            To unsubscribe from this list, visit             https://www.eclipse.org/mailman/listinfo/jakartabatch-dev



        _______________________________________________
        jakartabatch-dev mailing list
        jakartabatch-dev@xxxxxxxxxxx
        To unsubscribe from this list, visit         https://www.eclipse.org/mailman/listinfo/jakartabatch-dev
    _______________________________________________
    jakartabatch-dev mailing list
    jakartabatch-dev@xxxxxxxxxxx
    To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartabatch-dev
    _______________________________________________
    jakartabatch-dev mailing list
    jakartabatch-dev@xxxxxxxxxxx
    To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartabatch-dev_______________________________________________
    jakartabatch-dev mailing list
    jakartabatch-dev@xxxxxxxxxxx
    To unsubscribe from this list, visit     https://www.eclipse.org/mailman/listinfo/jakartabatch-dev



Back to the top