Optimizing CI/CD Pipelines by Looping Jobs with Dependencies in YAML

This blog explains how to create multiple jobs using an array and define a final job that depends on the previous loop jobs.

The following is the first YAML template file ( deploystage.yaml) with stages. Under the jobs section, it includes two templates. The first template is called within a loop, creating multiple jobs. 

parameters:

  - name: stage

    type: string

  - name: dependsOn

    type: object

    default: []

  - name: productList

    type: object

stages:

  - stage: "${{parameters.stage}}"

    dependsOn: "${{parameters.dependsOn}}"

    condition: succeeded()

    jobs:

      - ${{each component in parameters.productList}}:

          - template: ../jobs/deployjob.yaml

            parameters:

              stage: "${{parameters.stage}}"

              productList: "${{component}}"

      - template: ../jobs/deployjob2.yaml

        parameters:

          stage: "${{parameters.stage}}"

          productList: "${{parameters.productList}}"

The second template (deployjob2.yaml) should run only after all the jobs created by the loop have started and completed. The following template demonstrates how to define the depends on condition for a job that depends on a loop of jobs.

parameters:

  - name: stage

    type: string

  - name: dependsOn

    type: object

    default: []

  - name: productList

    type: object

jobs:

  - job: "myJob_${{parameters.stage}}"

    timeoutInMinutes: 10

    dependsOn:

      - ${{each component in parameters.productList}}:

          - "A_${{component}}"

    steps:

      - bash: echo "Hello world"

After execution, the pipeline looks as follows: three jobs are created using a loop, and the final job depends on all the previously generated loop jobs

Find the complete YAML script sample here