Pipeline support

Pipeline support and enhancements

Jervis provides an easy means to build on Jenkins quickly. However, it also provides the ability to run Jenkinsfile syntax directly after YAML is processed.

Jervis runs by default within a pipeline environment. Jervis will automatically run your pipeline if the following are defined in a repository.

• .jervis.yml and Jenkinsfile exist in the root of the repository. Jervis will automatically run the Jenkinsfile after performing a build.
• .jervis.yml contains a YAML key with the following value. Jervis will build within a pipeline and then execute the Jenkinsfile defined in a custom location.

  jenkins:
    pipeline_jenkinsfile: 'custom/path/to/Jenkinsfile'

Extending native pipeline functionality

Jervis provides several user-friendly steps which can be used to provide additional logic and filtering for Jenkins pipelines.

Documentation by steps:

• getMatrixAxes()
• hasGlobalVar()
• isBuilding()
• withEnvSecretWrapper()

getMatrixAxes step

Source code for this step

This step is described in a Jenkins community blog post. This step allows you to define a list of matrix axes and generate a list of matrix combinations that you can choose to execute in serial or parallel. This step merely returns a list of the combinations so you can do with it what you want.

Usage Example: Executing matrix axes in parallel.

// you can add more axes and this will still work
Map matrix_axes = [
    PLATFORM: ['linux', 'windows', 'mac'],
    JAVA: ['openjdk8', 'openjdk10', 'openjdk11'],
    BROWSER: ['firefox', 'chrome', 'safari', 'edge']
]

List axes = getMatrixAxes(matrix_axes) { Map axis ->
    !(axis['BROWSER'] == 'safari' && axis['PLATFORM'] == 'linux') &&
    !(axis['BROWSER'] == 'edge' && axis['PLATFORM'] != 'windows')
}

for(int i = 0; i < axes.size(); i++) {
    // convert the Axis into valid values for withEnv step
    Map axis = axes[i]
    List axisEnv = axis.collect { k, v ->
        "${k}=${v}"
    }
    // let's say you have diverse agents among Windows, Mac and Linux all of
    // which have proper labels for their platform and what browsers are
    // available on those agents.
    String nodeLabel = "os:${axis['PLATFORM']} && browser:${axis['BROWSER']}"
    tasks[axisEnv.join(', ')] = { ->
        node(nodeLabel) {
            withEnv(axisEnv) {
                stage("Build") {
                    echo nodeLabel
                    sh 'echo Do Build for ${PLATFORM} - ${BROWSER}'
                }
                stage("Test") {
                    echo nodeLabel
                    sh 'echo Do Build for ${PLATFORM} - ${BROWSER}'
                }
            }
        }
    }
}

stage("Matrix builds") {
    parallel(tasks)
}

You can also optionally surface a prompt only when a build occurs manually so that the user can choose to customize the matrix axes. To do this, you'll rely on another step provided by Jervis called isBuilding().

List axes = getMatrixAxes(matrix_axes, user_prompt: isBuilding('manually')) { Map axis ->
    !(axis['BROWSER'] == 'safari' && axis['PLATFORM'] == 'linux') &&
    !(axis['BROWSER'] == 'edge' && axis['PLATFORM'] != 'windows')
}

hasGlobalVar step

Source code for this step

Allows you to check if a global variable is available provided by a shared library in Jenkins pipelines. This allows you gracefully handle a missing step and not use it if it isn't available. The default behavior in Jenkins is to fail a pipeline when you use a step that doesn't exist so this step allows you to program around this limitation.

Usage Example: check if a step exists before attempting to use it.

if(hasGlobalVar('someSharedLibraryStep')) {
    // the step exists so you can use it.
    someSharedLibraryStep()
} else {
    // you can fall back to doing something else if the step doesn't exist.
}

isBuilding step

Source code for this step

This is an extended enduser filter which provides logic capabilities which should be a part of Jenkins multibranch pipelines but are not currently in a convenient way. This allows you to filter logic based on the kind of multibranch pipeline build as well as how it was triggered.

Kinds of multibranch pipeline builds:

• Pull Request
• Branch
• Tag

How builds can be triggered:

• SCM event.
• A user or script manually builds.
• A periodic build occurs due to cron configuration.

For these events isBuilding() provides the following basic filters.

isBuilding('pr')
isBuilding('branch')
isBuilding('tag')
isBuilding('cron')
isBuilding('manually')

These statements will return true or false which can be used in conditionals within Jenkinsfile pipeline logic.

From the above filters, there are extended filtering capabilities.

Extended filters for branches and tags

A branch can be filtered in three ways: any branch, specific named branch, or regular expression matching a branch name.

Match branch or tag example:

isBuilding('branch')
isBuilding('tag')

Named branch example:

isBuilding(branch: 'master')
isBuilding(tag: '1.0')

Regular expression matching branch names example.

// match master or hotfix branches (e.g. 1.0.5-hotfix)
isBuilding(branch: '/^\\Qmaster\\E$|^[0-9.]+-hotfix$')
// the following only matches semantic version tags
isBuilding(tag: '/([0-9]+\\.){2}[0-9]+(-.*)?$/')

Extended filters for manually triggered builds.

Filtering for manually triggered builds has many options available.

If you want a simple boolean as to whether a build was triggered, then use the following syntax.

isBuilding('manually')

If a build is triggered manually and you want the username of who triggered the build, then you would use the following filter.

isBuilding(manually: true)

If you want only when a build is triggered by a single specific user, then use the following syntax including the desired Jenkins username.

isBuilding(manually: 'samrocketman')

If you want to get if a build was triggered by anything except manually by a user, then use the following filter.

isBuilding(manually: false, combined: true)

Combining multiple filters

The isBuilding() step supports combining multiple filters at once. The following filter will match a tag or pull request but not branch builds.

isBuilding(['pr', 'tag'])

Match a specific branch and a specific tag.

// match master branch, hotfix branch, and tags which match semantic versioning.
isBuilding(
    branch: '/^\\Qmaster\\E$|^[0-9.]+-hotfix$',
    tag: '/([0-9]+\\.){2}[0-9]+(-.*)?$/'
    )

This will return a hashmap of matched values or an empty map if there are no matches.

Combination filters can also be forced to be combined into a single boolean returned. For this there's the combined option which can be true to combine filters or false. The default behavior if not specified is false. The following behavior is with it enabled.

Only match manually triggered tags.

isBuilding(tag: '/.*/', manually: true, combined: true)
isBuilding(['combined', 'tag', 'manually'])

Match pull requests or tags, but not branches.

isBuilding(['pr', 'tag'])

Match a manually built pull request or tag. Not branches or any other build trigger event type.

isBuilding(['combined', ['pr', 'tag'], 'manually'])

withEnvSecretWrapper step

Source code for this step

The mask passwords plugin does not provide an easy way to both filter secrets from console output and have them available in the environment. To do this you must use the mask passwords wrapper and withEnv Jenkins step. In order to avoid complexity the withEnvSecretWrapper() step was developed to provide both hiding secrets from console output and making them available in shell environments similar to how secrets would be.

Let's say you have a secrets map.

Map secrets = [
    MY_SECRET_VAR: supersecret,
    SECRET_KEY: anothersecret
]

To make the secrets available as environment variables MY_SECRET_VAR and SECRET_KEY and hidden in console output you would call the wrapper in the following way.

node {
    withEnvSecretWrapper(secrets) {
        sh 'env'
    }
}

withEnvSecretWrapper() is similar to withEnv where it makes environment variables available. Where it differs is the value of the environment variables is hidden in the console output and replaced with asterisks (****) if it is echoed like with env.

Mixing plain text environment and secret environment is just as simple.

Map secrets = [
    AWS_SECRET_ACCESS_KEY: 'XXX'
]

List plain = ['AWS_ACCESS_KEY_ID=YYY', 'AWS_DEFAULT_REGION=us-east-1']

node {
    withEnv(plain) {
        withEnvSecretWrapper(secrets) {
            sh 'env'
        }
    }
}