Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [ee4j-build] Meeting today
The requirements are mainly around the ability to execute a large number of jobs in parallel.
What you've shared works.

We have an internal setup in place (Jenkins + k8s), where we will be doing some more experiments with the GlassFish builds.
Hopefully we'll have something to share at the next meeting.

Thanks,
Romain

On Thu, May 31, 2018 at 5:09 AM, Mikaël Barbero <mikael.barbero@xxxxxxxxxxxxxxxxxxxxxx> wrote:
Hi,

I've done some testing with declarative pipelines. I managed to get to something which should fit your needs. The pipeline itself is there 


and the workflow is as pictured below:



The build starts with stage 1. It is composed of 3 sub-stages that execute in parallel. Each sub-stage executes on a dedicated (dynamic/kubernetes) agent, and each agent uses a given Docker image (could be the same, or could be different).  The workflow itself and the stage 3 both run in the same agent (but it's not mandatory).

Stage 1.1 and 1.2 each write a specific string to a file that they both name "output.txt". There are then 2 files "output.txt": one in the workspace of the agent running the stage1.1 and one in the workspace running the stage 1.2. Then, the files are stashed, i.e. it will be possible to retrieve them in a later stage.The stash names need to be different in each sub-stage, otherwise, one would overwrite the other (and, to add insult to injury, the one being overwritten is unknown as both runs in parallel).

Once all substages are over, stage3 (should be named stage 2 ;)) is triggered. It retrieves ("unstash") results from sub-stage 1.1 and 1.2 and finally archives these files.

You will find attached the sanitized log of an execution.

You will probably find the pipeline script a bit verbose, but it can be improved via a shared library. I'll try that now.

Apart from the verbosity, does it fulfill your requirements?



--
Mikaël Barbero - Eclipse Foundation
IT Services - Release Engineering
📱 (+33) 642 028 039
🐦 @mikbarbero

Le 30 mai 2018 à 16:48, Denis Roy <denis.roy@eclipse-foundation.org> a écrit :

Thank you for hosting the meeting. I've noted two action items:

GF team: investigate Declarative Pipelines

Eclipse team: Confirm that pipelines offer the functionality needed, and provide a Jenkins sample implementation

Mikaël has confirmed just now with the CloudBees consultant on-site that Jenkins Pipelines are indeed the best solution for this, and together they will prepare a sample job to demonstrate the functionality and the implementation.

Many thanks,


Denis



On 2018-05-30 09:21 AM, Dmitry Kornilov wrote:
Hi,

Just reminding you that today's meeting will be hosted by me. Please use my Zoom: https://oracle.zoom.us/my/dmitry.kornilov

Thanks,
Dmitry


_______________________________________________
ee4j-build mailing list
ee4j-build@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ee4j-build


_______________________________________________
ee4j-build mailing list
ee4j-build@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/ee4j-build


Back to the top