Jenkins is a wonderful system for managing builds, and people love using its UI to configure jobs

Jenkins Job DSL Plugin

Introduction

Jenkins is a wonderful system for managing builds, and people love using its UI to configure jobs. Unfortunately, as the number of jobs grows, maintaining them becomes tedious, and the paradigm of using a UI falls apart. Additionally, the common pattern in this situation is to copy jobs to create new ones, these "children" have a habit of diverging from their original "template" and consequently it becomes difficult to maintain consistency between these jobs.

The Job DSL plugin attempts to solve this problem by allowing jobs to be defined in a programmatic form in a human readable file. Writing such a file is feasible without being a Jenkins expert as the configuration from the web UI translates intuitively into code.

configuration form

The job configuration above can be generated from the following code.

pipelineJob('job-dsl-plugin') {
  definition {
    cpsScm {
      scm {
        git {
          remote {
            url('https://github.com/jenkinsci/job-dsl-plugin.git')
          }
          branch('*/master')
        }
      }
      lightweight()
    }
  }
}

Job DSL was one of the first popular plugins for Jenkins which allows managing configuration as code and many other plugins dealing with this aspect have been created since then, most notably the Jenkins Pipeline and Configuration as Code plugins. It is important to understand the differences between these plugins and Job DSL for managing Jenkins configuration efficiently.

The Pipeline plugins support implementing and integrating continuous delivery pipelines via the Pipeline DSL. While it is possible to use Job DSL to create complex pipelines using freestyle jobs in combination with many plugins from the Jenkins ecosystem, creating and maintaining these pipeline, including generating jobs for individual SCM branches and possibly running steps in parallel to improve build performance, poses a significant challenge. Jenkins Pipeline is often the better choice for creating complex automated processes. Job DSL can be used to create Pipeline and Multibranch Pipeline jobs. Do not confuse Job DSL with Pipeline DSL, both have their own syntax and scope of application.

The Configuration as Code plugin can be used to manage the global system configuration of Jenkins. It comes with an integration for Job DSL to create an initial set of jobs.

Getting Started

First, start a Jenkins instance with the Job DSL plugin installed.

Then create a freestyle project named "seed".

configuration form

Add a "Process Job DSLs" build step and paste the script below into the "DSL Script" field.

job('example') {
  steps {
    shell('echo Hello World!')
  }
}

When running Jenkins on Windows, replace the shell step by a batchFile step.

job('example') {
  steps {
    batchFile('echo Hello World!')
  }
}

configuration form

Save the configuration, start a build and inspect the console output.

Started by user admin
Running as SYSTEM
Building in workspace /var/jenkins_home/workspace/seed
Processing provided DSL script
Added items:
    GeneratedJob{name='example'}
Finished: SUCCESS

The seed job has generated the "example" job. Verify the result on the job's configuration page.

generated job

Instead of creating a seed job manually, consider using the Configuration as Code plugin. See the Wiki for details.

Documentation

The complete DSL API reference is available in your Jenkins installation at https://your.jenkins.installation/plugin/job-dsl/api-viewer/index.html. You can find links to the API reference on the seed job page and the Job DSL build step.

A limited sub-set of the API reference is available online at https://jenkinsci.github.io/job-dsl-plugin/. But be aware that this does not show all API that is available in your Jenkins installation because a lot of the documentation is generated at runtime by introspecting the plugins that have been installed.

Jenkins saves the configuration of each job in a XML file. The Job DSL plugin is in principle a generator for these XML files, translating the DSL code into XML. If a configuration option is not available in the high-level DSL, it is possible to generate the XML directly using a Configure Block. Use the Job DSL Playground to create and test your configure blocks. Please note that the playground only supports the DSL API that is available in the online API Reference.

Find the complete documentation on the Wiki.

Release Notes

See the Wiki.

Community

Browse the collection of talks and blog posts about Job DSL. If you have a talk or blog post to share, please raise a hand, e.g. by posting to the mailing list.

Head over to Stack Overflow or the mailing list to get help.

Use the Issue Tracker for reporting bugs and making feature requests. Select the job-dsl-plugin component when searching or creating issues.

You can actively help to improve Job DSL by contributing code, documentation and tests or by reviewing and testing upcoming changes on GitHub. Start by reading the guidelines for contributors.

Please use the mailing list to provide feedback.

Owner
Jenkins
Jenkins is an open source automation server with an unparalleled plugin ecosystem to support practically every tool as part of your delivery pipelines
Jenkins
Comments
  • dsl api documentation

    dsl api documentation

    This PR adds the initial implementation of the auto-generating documentation. demo: http://sheehan.github.io/job-dsl-plugin/

    Steps for publishing the docs:

    1. Run ./gradlew generateApiDoc
      • This will create a JSON file in /job-dsl-plugin-viewer/data/dsl-<version>.json
    2. Add the version to the select at /job-dsl-api-viewer/index.html if not already there
    3. Run ./gradlew :job-dsl-api-viewer:publishGhPages

    The generator will use JavaDoc for the method documentation. For longer docs, you can create a markdown file in job-dsl-core/api-docs/ instead. I've moved a few docs there from the wiki so far.

  • Allow using classes not supported by `structs` for `DescribableContext`

    Allow using classes not supported by `structs` for `DescribableContext`

    This allows for types that are not directly supported by structs to be used in Job DSL. An example for this is hudson.util.Secret, that's used by by the ssh-credentials plugin. See JENKINS-57435 for details.

  • JENKINS-21750 DSL support for the promoted-builds plugin (take 2 for #157)

    JENKINS-21750 DSL support for the promoted-builds plugin (take 2 for #157)

    Hi @dickerpulli and @daspilker

    This is my humble attempt to help the project ship Thomas' awesome work in #157. I fully understand the motivation to create extension points in the DSL for other plugins in the future. Thomas' work is solid - I've been using a local .hpi build of his PR branch for many weeks now. It's been very reliable.

    I think the Jenkins community overall would benefit quite a bit by shipping #157 as-is, and refactor the promoted-builds support in a separate PR for an extension point. The work on the extension point seems to have stalled out (no criticism - it's a heroic effort to do both the promoted builds and abstract that support to something generic at the same time).

    In an effort to make help make you life easier as the maintainer of the project (and author of the original PR), I did a few things:

    1. rebased from the latest upstream master for v1.25 (this was no fun, thanks to @thommay for the first rebase back in July)
    2. squashed Thomas' work down to a single commit (maintaining Thomas as the author of course)
    3. added a small fix for parameterizedBuild actions in promotions

    I hope you find this useful. The promoted-builds plugin can be a workhorse for continuous delivery in Jenkins, and the job-dsl plugin makes continuous delivery way easier to implement in Jenkins compared to point and click manual job creation in the UI. Having proper support for promotions in the job-dsl is a great step forward.

    All the tests pass on my local build, however, the rebase was fairly complicated so I welcome some serious scrutiny on the changes.

    Cheers, Steve

  • Support seed job creation in Configuration as Code, updated

    Support seed job creation in Configuration as Code, updated

    It is using a released version and the changes that have happened to the configurator in the meantime. SeedJob configurator now has access to JCasC secrets.

  • Build parameters support

    Build parameters support

    We wrote tests and everyfink!

    (Someone might want to take a look at things, but it was really quite simple once Daniel tidied things up)

    NOTE: the password support is currently "in the clear".

  • Xunit support

    Xunit support

    Adds support for the xunit plugin (https://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin). If this looks like a sane way of dealing with it, I'll add some documentation to the appropriate sections.

  • Add support for Phabricator Plugin

    Add support for Phabricator Plugin

    I'm trying to add the support for the Phabricator-Jenkins plugin, however I'm not really sure why the build fails:

    javaposse.jobdsl.dsl.DslScriptLoaderSpec > use @Grab FAILED
        org.codehaus.groovy.control.MultipleCompilationErrorsException at DslScriptLoaderSpec.groovy:183
    

    My first guess is that it's due to the plugin not being officially hosted yet, as the registration is in progress (see https://github.com/uber/phabricator-jenkins-plugin/issues/8).

    Thoughts?

  • Renaming of Jobs by Pattern

    Renaming of Jobs by Pattern

    Implementation of the possibility to configure a job so that on rename the existing Job is also renamed.

    We give our job some index numbers to have a better ordering in the UI. These index number change when jobs are added or removed. In order not to loose the build history and not having to decide what to do with the old jobs this feature would be really handy.

    Please comment if I tried implementing it the right way. If it is OK, then I would add some documentation.

  • Matrixjob support

    Matrixjob support

    MatrixJob job type support replaces PR #195 and PR #151

    job(type: MatrixJob) {
       axis{
          text('axisName1', ['val1', 'val2'...])
          label('axisName2' [label-node','label-node'...])
          labelExpression('axisName3', ['label-node-expression'...])
          jdk('axisName4', ['jdk1','jdk2'...])
    
          configure { project ->
            def ax = project/'axes'
            ax << {
              'hudson.matrix.TextAxis' {
                delegate.createNode('name', 'axis1')
                values {
                  string 'z'
                  string 'y'
                  string 'x'
                }
              }
            }
          }
       }
       sequential(true)
       touchStoneFilter('axisName1=="val1"', true)
       combinationFilter('axisName1=="val1"||axisName2=='master')
    }
    
  • [JENKINS-16364] Add support for Build Dsl job type.

    [JENKINS-16364] Add support for Build Dsl job type.

    This will allow creating build flow jobs by passing in the dsl block.

    This change does not provide support for making the dsl block pretty; it must be escaped for the resulting job to be pretty printed.

    Tests included under a new Spec file.

    Usage:

    • buildDslBlock( <build dsl text, escaped or flat> )

    Example (simple):

    • buildDslBlock('build ("job1")')

    Example (escaped):

    • buildDslBlock("parallel( \n { \n \" job1 \" } \n) " )

    Example (passing in a variable's value, so the created job has the value in the text of the dsl block):

    • buildDslBlock('build("${BUILD_TAG}")')
    • End result in build DSL node: build("my-base-job-dsl-job-12")

    Example (passing in a variable, so the end-result is a created job that has the variable (and not its value)):

    • buildDslBlock('build("\${BUILD_TAG}")')
    • End result in build DSL node: build("${BUILD_TAG}")
  • [Jenkins-21947] m2release plugin support

    [Jenkins-21947] m2release plugin support

    This pull request adds support for the m2release plugin.

    I successfully tested the plugin on Jenkins 1.532.1 and m2release plugin 0.12.0 running on Windows 2008 R2 SP1.

    Also tested on Jenkins 1.480 m2release plugin 0.12.0 on Windows 7, but it seems that the m2release plugin does not support this old Jenkins version, couldn't get the m2release plugin to work.

    Please review and merge.

  • Gerrit Event Trigger - Add Private & WIP State changed

    Gerrit Event Trigger - Add Private & WIP State changed

    Since Gerrit 2.15, new events hooks have been introduced which can be used to trigger new Builds:

    • Private state changed
    • Work In Progress state changed
    • Draft Publication removed

    Ref: https://www.gerritcodereview.com/2.15.html#new-workflows

    In this PR, we add support for the new events and keep the removed one for compatibility with Gerrit < 2.15.

    • [x] Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
    • [x] Ensure that the pull request title represents the desired changelog entry
    • [x] Please describe what you did
    • [ ] Link to relevant issues in GitHub or Jira
    • [ ] Link to relevant pull requests, esp. upstream and downstream changes
    • [x] Ensure you have provided tests - that demonstrates feature works or fixes the issue
  • Added configId to NodeJS wrapper

    Added configId to NodeJS wrapper

    Added configId to nodejs-wrapper (supported by the nodejs-plugin: https://github.com/jenkinsci/nodejs-plugin/blob/master/src/main/java/jenkins/plugins/nodejs/NodeJSBuildWrapper.java#L102) to configure custom .npmrc files. A custom .npmrc file can be useful to configure custom registries etc. Setting the default to null should maintain backwards compatibility.

    • [x] Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
    • [x] Ensure that the pull request title represents the desired changelog entry
    • [x] Please describe what you did
    • [x] ~Link to relevant issues in GitHub or Jira~
    • [x] ~Link to relevant pull requests, esp. upstream and downstream changes~
    • [x] Ensure you have provided tests - that demonstrates feature works or fixes the issue
  • [JENKINS-69849] Add support for additional classpath with seed config

    [JENKINS-69849] Add support for additional classpath with seed config

    Given that CasC have no ways to configure classpath and jenkins bootstrap classpath is isolated from plugins there is no way to actually deliver a shared library to jobdsl beyond pipeline. As described in https://issues.jenkins.io/browse/JENKINS-69849 this approach may double amount of configurations needed to be maintained and complicate basic scenarios.

    Provided pull request introduces a new "additionalClasspath" option accepted by seed job configurator which is processed in similar fashion as providedEnv - it is accumulated and passed to lower layers which generate script request and in effect also compile it.

    • [x] Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
    • [x] Ensure that the pull request title represents the desired changelog entry
    • [x] Please describe what you did
    • [x] Link to relevant issues in GitHub or Jira
    • [ ] Link to relevant pull requests, esp. upstream and downstream changes
    • [x] Ensure you have provided tests - that demonstrates feature works or fixes the issue
  • Enable configuration for icons in folders

    Enable configuration for icons in folders

    This PR enables the icon configuration on folders, multi-branch projects and organization folders to be accessible. This way one can customize the folder icon via jenkinsci/custom-folder-icon-plugin. See https://github.com/jenkinsci/custom-folder-icon-plugin/issues/92 as well for details and use case.

    image

    Here the jenkinsci/custom-folder-icon-plugin is installed, by default there is only stockFolderIcon and metaDataActionFolderIcon available.

    Example configuration:

    folder('stock')
    
    userContent('customFolderIcons/custom.png', streamFileFromWorkspace('custom.png'))
    folder('custom') {
      icon {
        customFolderIcon {
          foldericon('custom.png')
        } 
      }
    }
    folder('ionicon') {
      icon {
        ioniconFolderIcon {
          ionicon('jenkins')
        } 
      }
    }
    folder('build-status') {
      icon {
          buildStatusFolderIcon()
      }
    }
    
  • Add support for Archive Artifacts' caseSensitive and followSymlinks settings

    Add support for Archive Artifacts' caseSensitive and followSymlinks settings

    Added two new fields to ArchiveArtifactsContext, reflecting two new settings in this wrapper:

    • caseSensitive
    • followSymlinks

    I've also updated the existing tests to work correctly with the new code.

    Relevant JIRA issue

    • [x] Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
    • [x] Ensure that the pull request title represents the desired changelog entry
    • [x] Please describe what you did
    • [x] Link to relevant issues in GitHub or Jira
    • [ ] Link to relevant pull requests, esp. upstream and downstream changes
    • [x] Ensure you have provided tests - that demonstrates feature works or fixes the issue
Run Jobs on a schedule, supports fixed interval, timely, and cron-expression timers; Instrument your processes and expose metrics for each job.

A simple process manager that allows you to specify a Schedule that execute a Job based on a Timer. Schedule manage the state of this job allowing you to start/stop/restart in concurrent safe way. Schedule also instrument this Job and gather metrics and optionally expose them via uber-go/tally scope.

Dec 8, 2022
A persistent and flexible background jobs library for go.

Jobs Development Status Jobs is no longer being actively developed. I will still try my best to respond to issues and pull requests, but in general yo

Nov 21, 2022
gron, Cron Jobs in Go.

gron Gron provides a clear syntax for writing and deploying cron jobs. Goals Minimalist APIs for scheduling jobs. Thread safety. Customizable Job Type

Dec 20, 2022
YTask is an asynchronous task queue for handling distributed jobs in golang
YTask is an asynchronous task queue for handling distributed jobs in golang

YTask is an asynchronous task queue for handling distributed jobs in golang

Dec 24, 2022
Tiny library to handle background jobs.

bgjob Tiny library to handle background jobs. Use PostgreSQL to organize job queues. Highly inspired by gue Features Durable job storage At-least-ones

Nov 16, 2021
A way of scheduling volcano jobs

JobFlow 背景 volcano Volcano是CNCF 下首个也是唯一的基于Kubernetes的容器批量计算平台,主要用于高性能计算场景。 它提供了Kubernetes目前缺 少的一套机制,这些机制通常是机器学习大数据应用、科学计算、 特效渲染等多种高性能工作负载所需的。 现状:当前vol

Oct 12, 2022
Executes jobs in separate GO routines. Provides Timeout, StartTime controls. Provides Cancel all running job before new job is run.

jobExecutor Library to execute jobs in GO routines. Provides for Job Timeout/Deadline (MaxDuration()) Job Start WallClock control (When()) Add a job b

Jan 10, 2022
Graceful shutdown with repeating "cron" jobs (running at a regular interval) in Go

Graceful shutdown with repeating "cron" jobs (running at a regular interval) in Go Illustrates how to implement the following in Go: run functions ("j

May 30, 2022
A programmable, observable and distributed job orchestration system.
A programmable, observable and distributed job orchestration system.

?? Overview Odin is a programmable, observable and distributed job orchestration system which allows for the scheduling, management and unattended bac

Dec 21, 2022
Distributed Task Scheduling System|分布式定时任务调度平台
Distributed Task Scheduling System|分布式定时任务调度平台

Crocodile Distributed Task Scheduling System English | 中文 Introduction A distributed task scheduling system based on Golang that supports http request

Jan 5, 2023
high performance distributed task scheduling system, Support multi protocol scheduling tasks
 high performance distributed task scheduling system, Support multi protocol scheduling tasks

high performance distributed task scheduling system, Support multi protocol scheduling tasks

Dec 2, 2022
goInterLock is golang job/task scheduler with distributed locking mechanism (by Using Redis🔒).
goInterLock is golang job/task scheduler with distributed locking mechanism (by Using Redis🔒).

goInterLock is golang job/task scheduler with distributed locking mechanism. In distributed system locking is preventing task been executed in every instant that has the scheduler,

Dec 5, 2022
RESTful-JSON-API - RESTful-JSON-API using Go

RESTful-JSON-API using Go This basic REST-API principle establishes a one-to-one

Feb 15, 2022
A simple Cron library for go that can execute closures or functions at varying intervals, from once a second to once a year on a specific date and time. Primarily for web applications and long running daemons.

Cron.go This is a simple library to handle scheduled tasks. Tasks can be run in a minimum delay of once a second--for which Cron isn't actually design

Dec 17, 2022
Lightweight, fast and dependency-free Cron expression parser (due checker) for Golang (tested on v1.13 and above)

adhocore/gronx gronx is Golang cron expression parser ported from adhocore/cron-expr. Zero dependency. Very fast because it bails early in case a segm

Dec 30, 2022
Marshmallow provides a flexible and performant JSON unmarshalling in Go. It specializes in dealing with unstructured struct - when some fields are known and some aren't, with zero performance overhead nor extra coding needed.
Marshmallow provides a flexible and performant JSON unmarshalling in Go. It specializes in dealing with unstructured struct - when some fields are known and some aren't, with zero performance overhead nor extra coding needed.

Marshmallow Marshmallow package provides a simple API to perform flexible and performant JSON unmarshalling in Go. Marshmallow specializes in dealing

Dec 26, 2022
clockwork - Simple and intuitive job scheduling library in Go.
clockwork - Simple and intuitive job scheduling library in Go.

clockwork A simple and intuitive scheduling library in Go. Inspired by python's schedule and ruby's clockwork libraries. Example use package main imp

Jul 27, 2022
Easy and fluent Go cron scheduling

goCron: A Golang Job Scheduling Package. goCron is a Golang job scheduling package which lets you run Go functions periodically at pre-determined inte

Jan 8, 2023
Efficient and reliable background processing for Go

CurlyQ CurlyQ provides a simple, easy-to-use interface for performing background processing in Go. It supports scheduled jobs, job deduplication, and

Nov 11, 2022