| Re: [ee4j-build] Meeting today |
| Hi, I've done some testing with declarative pipelines. I managed to get to something which should fit your needs. The pipeline itself is there and the workflow is as pictured below: |
The build starts with stage 1. It is composed of 3 sub-stages that execute in parallel. Each sub-stage executes on a dedicated (dynamic/kubernetes) agent, and each agent uses a given Docker image (could be the same, or could be different). The workflow itself and the stage 3 both run in the same agent (but it's not mandatory). Stage 1.1 and 1.2 each write a specific string to a file that they both name "output.txt". There are then 2 files "output.txt": one in the workspace of the agent running the stage1.1 and one in the workspace running the stage 1.2. Then, the files are stashed, i.e. it will be possible to retrieve them in a later stage.The stash names need to be different in each sub-stage, otherwise, one would overwrite the other (and, to add insult to injury, the one being overwritten is unknown as both runs in parallel). Once all substages are over, stage3 (should be named stage 2 ;)) is triggered. It retrieves ("unstash") results from sub-stage 1.1 and 1.2 and finally archives these files. You will find attached the sanitized log of an execution. You will probably find the pipeline script a bit verbose, but it can be improved via a shared library. I'll try that now. Apart from the verbosity, does it fulfill your requirements? |
Started by user mikael.barbero@xxxxxxxxxxxxxxxxxxxxxx 11:30:21 Connecting to https://api.github.com using mikael.barbero@xxxxxxxxxxxxxxxxxxxxxx/****** (GitHub Access Token) Obtained Jenkinsfile from 01fd3d568a83db84541675d7c76d8e9d0e331402 Running on mypod-6dvrv-36zfv in /home/jenkins/workspace/s-pipeline-parrallel_master-4K3E5CVYOYAXECKL22ENW3K7F2AAFREDCLZDEFSJTNY5PXNFHZUA Cloning the remote Git repository [stage1.1] Running on mypod-a-8zmm5-bl1dh in /home/jenkins/workspace/s-pipeline-parrallel_master-4K3E5CVYOYAXECKL22ENW3K7F2AAFREDCLZDEFSJTNY5PXNFHZUA [Pipeline] [stage1.1] checkout [stage 1.2] Running on mypod-b-3j4ft-85gvn in /home/jenkins/workspace/s-pipeline-parrallel_master-4K3E5CVYOYAXECKL22ENW3K7F2AAFREDCLZDEFSJTNY5PXNFHZUA [Pipeline] [stage 1.2] checkout [stage 1.3] Running on mypod-c-fs78z-zxqld in /home/jenkins/workspace/s-pipeline-parrallel_master-4K3E5CVYOYAXECKL22ENW3K7F2AAFREDCLZDEFSJTNY5PXNFHZUA [Pipeline] [stage 1.3] checkout [stage1.1] Cloning the remote Git repository [stage 1.3] Cloning the remote Git repository [stage 1.2] Cloning the remote Git repository [Pipeline] [stage1.1] echo [stage1.1] from Stage1.1 [stage1.1] [s-pipeline-parrallel_master-4K3E5CVYOYAXECKL22ENW3K7F2AAFREDCLZDEFSJTNY5PXNFHZUA] Running shell script [stage 1.2] from Stage 1.2 [stage1.1] Stashed 1 file(s) [stage 1.2] [s-pipeline-parrallel_master-4K3E5CVYOYAXECKL22ENW3K7F2AAFREDCLZDEFSJTNY5PXNFHZUA] Running shell script [stage 1.3] from stage 1.3 [stage 1.2] Stashed 1 file(s) from stage3 Running in /home/jenkins/workspace/s-pipeline-parrallel_master-4K3E5CVYOYAXECKL22ENW3K7F2AAFREDCLZDEFSJTNY5PXNFHZUA/from-stage-1.1 [from-stage-1.1] Running shell script + pwd /home/jenkins/workspace/s-pipeline-parrallel_master-4K3E5CVYOYAXECKL22ENW3K7F2AAFREDCLZDEFSJTNY5PXNFHZUA/from-stage-1.1 + ls -al . total 4 drwxr-sr-x 2 10001000 10001000 24 May 31 11:34 . drwxr-sr-x 5 10001000 10001000 102 May 31 11:34 .. -rw-r--r-- 1 10001000 10001000 9 May 31 11:34 output.txt + ls -al ../ total 8 drwxr-sr-x 5 10001000 10001000 102 May 31 11:34 . drwxr-sr-x 4 10001000 10001000 186 May 31 11:33 .. drwxr-sr-x 8 10001000 10001000 179 May 31 11:33 .git -rw-r--r-- 1 10001000 10001000 2255 May 31 11:33 Jenkinsfile -rw-r--r-- 1 10001000 10001000 28 May 31 11:33 README.md drwxr-sr-x 2 10001000 10001000 24 May 31 11:34 from-stage-1.1 + cat output.txt stage1.1 Running in /home/jenkins/workspace/s-pipeline-parrallel_master-4K3E5CVYOYAXECKL22ENW3K7F2AAFREDCLZDEFSJTNY5PXNFHZUA/from-stage-1.2 [from-stage-1.2] Running shell script + pwd /home/jenkins/workspace/s-pipeline-parrallel_master-4K3E5CVYOYAXECKL22ENW3K7F2AAFREDCLZDEFSJTNY5PXNFHZUA/from-stage-1.2 + ls -al . total 4 drwxr-sr-x 2 10001000 10001000 24 May 31 11:34 . drwxr-sr-x 7 10001000 10001000 150 May 31 11:34 .. -rw-r--r-- 1 10001000 10001000 9 May 31 11:34 output.txt + ls -al ../ total 8 drwxr-sr-x 7 10001000 10001000 150 May 31 11:34 . drwxr-sr-x 4 10001000 10001000 186 May 31 11:33 .. drwxr-sr-x 8 10001000 10001000 179 May 31 11:33 .git -rw-r--r-- 1 10001000 10001000 2255 May 31 11:33 Jenkinsfile -rw-r--r-- 1 10001000 10001000 28 May 31 11:33 README.md drwxr-sr-x 2 10001000 10001000 24 May 31 11:34 from-stage-1.1 drwxr-sr-x 2 10001000 10001000 24 May 31 11:34 from-stage-1.2 + cat output.txt stage1.2 Archiving artifacts GitHub has been notified of this commitâs build result Finished: SUCCESS
-- Mikaël Barbero - Eclipse Foundation IT Services - Release Engineering 📱 (+33) 642 028 039 🐦 @mikbarbero
|
Attachment:
signature.asc
Description: Message signed with OpenPGP