I think the best approach here is an actual working implementation that has some adoption, ideally by a major vendor in the batch processing space. Until then, it feels far too premature to be talking about putting any of this into a specification. With the limited bandwidth we all have, it’s best we focus on some relatively modest enhancements to Jakarta Batch for now.
Reza Rahman Jakarta EE Ambassador, Author, Blogger, Speaker
Please note views expressed here are my own as an individual community member and do not reflect the views of my employer. On Mar 24, 2021, at 12:01 PM, Scott Kurz <skurz@xxxxxxxxxx> wrote:
Hi Romain,
Your idea to make batch more "reactive" seemed worth breaking out into a separate thread.
Thanks for offering to explain more. It's hard for me to comment since this isn't very concrete to me.
If I had to paraphrase, it sounds like you're talking about "rebasing" the API to take advantage of patterns made possible by Java SE developments like Java 8 Streams (java.util.stream.*) and Concurrency APIs CompletionStage/CompletableFuture), and lambdas when appropriate, that would also facilitate
async NIO and more "reactive" usages.
Would it be possible for you to point to an example of a "job" (batch-like flow) constructed as you were describing, but without the batch container-provided persistence and "controller" (providing stop) ?
I don't think I have a great mental model here but I wonder.. could we imagine a series of options like:
1. static, XML job definition
2. programmatic, synchronous job definition
3. programmatic, async / reactive definition
The first we have today; the second provides a similar flow between container and batch artifacts, and the third more aligns with your description.
If so, perhaps these all could fit together in one spec. But maybe premature to speculate.
But I think we need some sample code snippets (if not prototype) to give us something to focus on and discuss further?
Thanks,
------------------------------------------------------
Scott Kurz
WebSphere / Open Liberty Batch and Developer Experience
skurz@xxxxxxxxxx
--------------------------------------------------------
"jakartabatch-dev" <jakartabatch-dev-bounces@xxxxxxxxxxx> wrote on 03/23/2021 11:07:54 AM:
> From: Romain Manni-Bucau <rmannibucau@xxxxxxxxx>
> To: jakartabatch developer discussions <jakartabatch-dev@xxxxxxxxxxx>
> Date: 03/23/2021 11:08 AM
> Subject: [EXTERNAL] Re: [jakartabatch-dev] Kick off conversation on
> next Jakarta Batch release - Jakarta EE 10 ? Batch + CDI integration?
> Sent by: "jakartabatch-dev" <jakartabatch-dev-bounces@xxxxxxxxxxx>
>
> Hi Scott,
>
> Maybe coming from nowhere but personally i'd like jbatch 2 to not
> inherit from jbatch 1 and be reactive driven.
> Overall idea is to be able to use more easily custom flows (by
> combining CompletionStages).
> This has a ton of advantages:
>
> 1. Makes the batch writer responsible of the flow (whereas a mix
> between the runtime and writer),
> 2. Keep the composability of existing batch "components",
> 3. Enables to work with any IoC without having to do any integration
> (batch writer injects what it needs),
> 4. No need of any batch/job repository (in code veritas est ;)),
> 5. Enables to be more efficient in more and more common "remoting
> first" batches we meet with microservices (ie leverage NIO more than
> in current synchronous API),
> 6. Makes the testing way easier:
> myBatch.userStart().toCompletionFuture().get();
>
> The main missing piece is the persistence of the steps+overall batch
> state(s) and some controller (to stop a running batch for example).
>
> Indeed JBatch 2 can be a BatchController { saveStepState(...);
> saveBatchState(...) } etc but thinking more in terms of events
> sounds more natural (we can start by CDI events since it is about
> jakarta but implementations can wire events on what they desire).
>
> To rephrase it a bit more clearly: JBatch wouldn't be a runtime with
> very low value as of today but the actual infrastructure API
> (persistence mainly and state "isBatch(key, STOPPED)").
>
> In terms of API the only small challenges are to enable to inject
> into a promise flow these persistent steps properly and to probably
> enable to orchestrate more easily chunking and so (even if Stream
> API kind of enables it but it misses some Iteration object to make
> it easier) but overally it does not sound crazy:
>
> final var batch = jbatch.startingBatch("my-batch");
> final var batchPromise = batch.wrapStep("step 1 name", fetchData())
> .thenApply(read -> batch.wrapStep("step 2 name", processData(read)));
>
> and so on.
_______________________________________________
jakartabatch-dev mailing list
jakartabatch-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartabatch-dev
|
