Re: [jakartabatch-dev] Proposal: give more control over jobcontroller qu

["Scott Kurz" <skurz@xxxxxxxxxx>]

Martijn,

Thanks for writing up those suggestions.

Let me just add a quick historical note that in the 1.0 spec we purposely avoided getting into queueing or orchestration of jobs, reasoning that there were plenty of job scheduling solutions available.

Also implementations can take different approaches to distributing executions across JVMs, not that that couldn't be incorporated into a spec-defined behavior.

Still, the fact that we didn't want to add this previously doesn't automatically rule out the idea, just noting some thoughts we've had until now.

Romain's suggestion allows you to write your own JobOperator and queue/throttle yourself, and I think this type of CDI integration is high on our future priority list. Opened issue: https://github.com/eclipse-ee4j/batch-api/issues/17

Thanks,
------------------------------------------------------
Scott Kurz
WebSphere Batch and Developer Experience
skurz@xxxxxxxxxx
--------------------------------------------------------

Romain Manni-Bucau ---03/11/2020 08:43:48 AM---Hi Martijn, As a side note, I'm not sure the qualifier need - it looks like a

From: Romain Manni-Bucau <rmannibucau@xxxxxxxxx>
To: jakartabatch developer discussions <jakartabatch-dev@xxxxxxxxxxx>
Date: 03/11/2020 08:43 AM
Subject: [EXTERNAL] Re: [jakartabatch-dev] Proposal: give more control over jobcontroller queue(s)
Sent by: jakartabatch-dev-bounces@xxxxxxxxxxx

---

Hi Martijn,

As a side note, I'm not sure the qualifier need - it looks like a configuration annotation and not a qualifier which is part of the bean type. That aside, I'd like to share an old though on this. Here are the steps I had in mind:

1. Ensure JBatch is fully CDI integrated, it means that JobOperator can be injected (or more concretely is a standard bean),
2. With 1 you can now decorate JobOperator (either with @Decorators or @Interceptors)

With 2 you can inject any logic on job startup (operator.start()) and control whatever policy you want (security/permissions, throttling, metrics etc...).

Concretely code would be (in pseudo code):

@ApplicationScoped
public class BatchLauncher { // whatever way to launch code
    @Inject
    private JobOperator operator;

    public void launch() {
        operator.start("...", ....);
    }
}

@Dependent
@Decorator
@Priority
public abstract class ThrottledOperator implements JobOperator {
  @Inject
  @Delegate
  private JobOperator delegate;

   @Override
   public long start(...) {
      doChecks();
      return delegate.start(...);
   }
}

The queueing requires some more cleverness and in particular to be able to send an id which is actually a future launch but using negative numbers + hashes with low collision level - xxhash?) it is very doable.

Last note: in standalone, this is also trivially doable with a plain old java proxy.

Romain Manni-Bucau
@rmannibucau |  Blog | Old Blog | Github | LinkedIn | Book


Le mer. 11 mars 2020 à 13:34, Martijn Dashorst <martijn.dashorst@xxxxxxxxx> a écrit :

In the project I'm working on, I'd like to have separate queues for my jobs. Some jobs should be limited to 1 concurrent, and others max 2, while even other jobs should not have a limit other than the available thread pool.

I'd love that this would be possible to configure in code, at least which jobs end up in which queue. E.G.

<job name="fooJob" queue="file-read">
    <...>
</job>

and in Java:

@Job(queue="file-read")
@Named
public class FooJob extends AbstractJob {
    ...
}

Currently there is no Java peer to the <job> JSL part.

As for configuring the JobController from your application, that is something I haven't thought about *that* hard, I have no experience with running multiple applications that all run jobs on one appserver. However, how the job controller works internally with multiple deployments, is something we all can figure out I guess.

For the configuration part inside a deployment, a CDI qualifier should do the trick:

@JobControllerConfiguration({@JobQueue(name="file-read", maxConcurrency=1, maxQueueLength=10), @JobQueue(name="banana")}
public class MyAppsJobControllerConfiguration {
}

or a specific JSL:

<jobcontroller>
    <queues>
        <queue name="file-read" maxConcurrency="1" maxQueueLength="1" />
        <...>
    </queues>
</jobcontroller>

Martijn

_______________________________________________
jakartabatch-dev mailing list
jakartabatch-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartabatch-dev_______________________________________________
jakartabatch-dev mailing list
jakartabatch-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jakartabatch-dev