Jenkins is a wonderful system for managing builds, and people love using its UI to configure jobs

Jenkins Job DSL Plugin

Introduction

Jenkins is a wonderful system for managing builds, and people love using its UI to configure jobs. Unfortunately, as the number of jobs grows, maintaining them becomes tedious, and the paradigm of using a UI falls apart. Additionally, the common pattern in this situation is to copy jobs to create new ones, these "children" have a habit of diverging from their original "template" and consequently it becomes difficult to maintain consistency between these jobs.

The Job DSL plugin attempts to solve this problem by allowing jobs to be defined in a programmatic form in a human readable file. Writing such a file is feasible without being a Jenkins expert as the configuration from the web UI translates intuitively into code.

configuration form

The job configuration above can be generated from the following code.

pipelineJob('job-dsl-plugin') {
  definition {
    cpsScm {
      scm {
        git {
          remote {
            url('https://github.com/jenkinsci/job-dsl-plugin.git')
          }
          branch('*/master')
        }
      }
      lightweight()
    }
  }
}

Job DSL was one of the first popular plugins for Jenkins which allows managing configuration as code and many other plugins dealing with this aspect have been created since then, most notably the Jenkins Pipeline and Configuration as Code plugins. It is important to understand the differences between these plugins and Job DSL for managing Jenkins configuration efficiently.

The Pipeline plugins support implementing and integrating continuous delivery pipelines via the Pipeline DSL. While it is possible to use Job DSL to create complex pipelines using freestyle jobs in combination with many plugins from the Jenkins ecosystem, creating and maintaining these pipeline, including generating jobs for individual SCM branches and possibly running steps in parallel to improve build performance, poses a significant challenge. Jenkins Pipeline is often the better choice for creating complex automated processes. Job DSL can be used to create Pipeline and Multibranch Pipeline jobs. Do not confuse Job DSL with Pipeline DSL, both have their own syntax and scope of application.

The Configuration as Code plugin can be used to manage the global system configuration of Jenkins. It comes with an integration for Job DSL to create an initial set of jobs.

Getting Started

First, start a Jenkins instance with the Job DSL plugin installed.

Then create a freestyle project named "seed".

configuration form

Add a "Process Job DSLs" build step and paste the script below into the "DSL Script" field.

job('example') {
  steps {
    shell('echo Hello World!')
  }
}

When running Jenkins on Windows, replace the shell step by a batchFile step.

job('example') {
  steps {
    batchFile('echo Hello World!')
  }
}

configuration form

Save the configuration, start a build and inspect the console output.

Started by user admin
Running as SYSTEM
Building in workspace /var/jenkins_home/workspace/seed
Processing provided DSL script
Added items:
    GeneratedJob{name='example'}
Finished: SUCCESS

The seed job has generated the "example" job. Verify the result on the job's configuration page.

generated job

Instead of creating a seed job manually, consider using the Configuration as Code plugin. See the Wiki for details.

Documentation

The complete DSL API reference is available in your Jenkins installation at https://your.jenkins.installation/plugin/job-dsl/api-viewer/index.html. You can find links to the API reference on the seed job page and the Job DSL build step.

A limited sub-set of the API reference is available online at https://jenkinsci.github.io/job-dsl-plugin/. But be aware that this does not show all API that is available in your Jenkins installation because a lot of the documentation is generated at runtime by introspecting the plugins that have been installed.

Jenkins saves the configuration of each job in a XML file. The Job DSL plugin is in principle a generator for these XML files, translating the DSL code into XML. If a configuration option is not available in the high-level DSL, it is possible to generate the XML directly using a Configure Block. Use the Job DSL Playground to create and test your configure blocks. Please note that the playground only supports the DSL API that is available in the online API Reference.

Find the complete documentation on the Wiki.

Release Notes

See the Wiki.

Community

Browse the collection of talks and blog posts about Job DSL. If you have a talk or blog post to share, please raise a hand, e.g. by posting to the mailing list.

Head over to Stack Overflow or the mailing list to get help.

Use the Issue Tracker for reporting bugs and making feature requests. Select the job-dsl-plugin component when searching or creating issues.

You can actively help to improve Job DSL by contributing code, documentation and tests or by reviewing and testing upcoming changes on GitHub. Start by reading the guidelines for contributors.

Please use the mailing list to provide feedback.

Owner
Jenkins
Jenkins is an open source automation server with an unparalleled plugin ecosystem to support practically every tool as part of your delivery pipelines
Jenkins
Comments
  • dsl api documentation

    dsl api documentation

    This PR adds the initial implementation of the auto-generating documentation. demo: http://sheehan.github.io/job-dsl-plugin/

    Steps for publishing the docs:

    1. Run ./gradlew generateApiDoc
      • This will create a JSON file in /job-dsl-plugin-viewer/data/dsl-<version>.json
    2. Add the version to the select at /job-dsl-api-viewer/index.html if not already there
    3. Run ./gradlew :job-dsl-api-viewer:publishGhPages

    The generator will use JavaDoc for the method documentation. For longer docs, you can create a markdown file in job-dsl-core/api-docs/ instead. I've moved a few docs there from the wiki so far.

  • Allow using classes not supported by `structs` for `DescribableContext`

    Allow using classes not supported by `structs` for `DescribableContext`

    This allows for types that are not directly supported by structs to be used in Job DSL. An example for this is hudson.util.Secret, that's used by by the ssh-credentials plugin. See JENKINS-57435 for details.

  • JENKINS-21750 DSL support for the promoted-builds plugin (take 2 for #157)

    JENKINS-21750 DSL support for the promoted-builds plugin (take 2 for #157)

    Hi @dickerpulli and @daspilker

    This is my humble attempt to help the project ship Thomas' awesome work in #157. I fully understand the motivation to create extension points in the DSL for other plugins in the future. Thomas' work is solid - I've been using a local .hpi build of his PR branch for many weeks now. It's been very reliable.

    I think the Jenkins community overall would benefit quite a bit by shipping #157 as-is, and refactor the promoted-builds support in a separate PR for an extension point. The work on the extension point seems to have stalled out (no criticism - it's a heroic effort to do both the promoted builds and abstract that support to something generic at the same time).

    In an effort to make help make you life easier as the maintainer of the project (and author of the original PR), I did a few things:

    1. rebased from the latest upstream master for v1.25 (this was no fun, thanks to @thommay for the first rebase back in July)
    2. squashed Thomas' work down to a single commit (maintaining Thomas as the author of course)
    3. added a small fix for parameterizedBuild actions in promotions

    I hope you find this useful. The promoted-builds plugin can be a workhorse for continuous delivery in Jenkins, and the job-dsl plugin makes continuous delivery way easier to implement in Jenkins compared to point and click manual job creation in the UI. Having proper support for promotions in the job-dsl is a great step forward.

    All the tests pass on my local build, however, the rebase was fairly complicated so I welcome some serious scrutiny on the changes.

    Cheers, Steve

  • Support seed job creation in Configuration as Code, updated

    Support seed job creation in Configuration as Code, updated

    It is using a released version and the changes that have happened to the configurator in the meantime. SeedJob configurator now has access to JCasC secrets.

  • Build parameters support

    Build parameters support

    We wrote tests and everyfink!

    (Someone might want to take a look at things, but it was really quite simple once Daniel tidied things up)

    NOTE: the password support is currently "in the clear".

  • Xunit support

    Xunit support

    Adds support for the xunit plugin (https://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin). If this looks like a sane way of dealing with it, I'll add some documentation to the appropriate sections.

  • Add support for Phabricator Plugin

    Add support for Phabricator Plugin

    I'm trying to add the support for the Phabricator-Jenkins plugin, however I'm not really sure why the build fails:

    javaposse.jobdsl.dsl.DslScriptLoaderSpec > use @Grab FAILED
        org.codehaus.groovy.control.MultipleCompilationErrorsException at DslScriptLoaderSpec.groovy:183
    

    My first guess is that it's due to the plugin not being officially hosted yet, as the registration is in progress (see https://github.com/uber/phabricator-jenkins-plugin/issues/8).

    Thoughts?

  • Renaming of Jobs by Pattern

    Renaming of Jobs by Pattern

    Implementation of the possibility to configure a job so that on rename the existing Job is also renamed.

    We give our job some index numbers to have a better ordering in the UI. These index number change when jobs are added or removed. In order not to loose the build history and not having to decide what to do with the old jobs this feature would be really handy.

    Please comment if I tried implementing it the right way. If it is OK, then I would add some documentation.

  • Matrixjob support

    Matrixjob support

    MatrixJob job type support replaces PR #195 and PR #151

    job(type: MatrixJob) {
       axis{
          text('axisName1', ['val1', 'val2'...])
          label('axisName2' [label-node','label-node'...])
          labelExpression('axisName3', ['label-node-expression'...])
          jdk('axisName4', ['jdk1','jdk2'...])
    
          configure { project ->
            def ax = project/'axes'
            ax << {
              'hudson.matrix.TextAxis' {
                delegate.createNode('name', 'axis1')
                values {
                  string 'z'
                  string 'y'
                  string 'x'
                }
              }
            }
          }
       }
       sequential(true)
       touchStoneFilter('axisName1=="val1"', true)
       combinationFilter('axisName1=="val1"||axisName2=='master')
    }
    
  • [JENKINS-16364] Add support for Build Dsl job type.

    [JENKINS-16364] Add support for Build Dsl job type.

    This will allow creating build flow jobs by passing in the dsl block.

    This change does not provide support for making the dsl block pretty; it must be escaped for the resulting job to be pretty printed.

    Tests included under a new Spec file.

    Usage:

    • buildDslBlock( <build dsl text, escaped or flat> )

    Example (simple):

    • buildDslBlock('build ("job1")')

    Example (escaped):

    • buildDslBlock("parallel( \n { \n \" job1 \" } \n) " )

    Example (passing in a variable's value, so the created job has the value in the text of the dsl block):

    • buildDslBlock('build("${BUILD_TAG}")')
    • End result in build DSL node: build("my-base-job-dsl-job-12")

    Example (passing in a variable, so the end-result is a created job that has the variable (and not its value)):

    • buildDslBlock('build("\${BUILD_TAG}")')
    • End result in build DSL node: build("${BUILD_TAG}")
  • [Jenkins-21947] m2release plugin support

    [Jenkins-21947] m2release plugin support

    This pull request adds support for the m2release plugin.

    I successfully tested the plugin on Jenkins 1.532.1 and m2release plugin 0.12.0 running on Windows 2008 R2 SP1.

    Also tested on Jenkins 1.480 m2release plugin 0.12.0 on Windows 7, but it seems that the m2release plugin does not support this old Jenkins version, couldn't get the m2release plugin to work.

    Please review and merge.

  • Update permissions based on changes made in matrix auth plugin 3.0

    Update permissions based on changes made in matrix auth plugin 3.0

    • [ ] Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
    • [ ] Ensure that the pull request title represents the desired changelog entry
    • [ ] Please describe what you did
    • [ ] Link to relevant issues in GitHub or Jira
    • [ ] Link to relevant pull requests, esp. upstream and downstream changes
    • [ ] Ensure you have provided tests - that demonstrates feature works or fixes the issue
  • Fix: Don't destroy credentials when re-creating organization folder

    Fix: Don't destroy credentials when re-creating organization folder

    Extend the current code which does it for cloudbee Folder to Organization Folder.

    refs JENKINS-44681 Previous fix for cloudbee Folder: https://github.com/jenkinsci/job-dsl-plugin/pull/1232

  • JENKINS-67594: support for item generation next to parent directory

    JENKINS-67594: support for item generation next to parent directory

    If you group several/many seed jobs together in a folder, you may want to get the items generated not next to their seed job, but next to their folder.

    I.e. you would not clutter the project folders with all the seed jobs.

    /projectA/foo/...
    /projectA/bar/...
    /projectA/seed-jobs/foo-seed
    /projectA/seed-jobs/bar-seed
    /projectB/baz
    /projectB/seed-jobs/baz-seed
    

    instead of

    /projectA/foo/...
    /projectA/foo-seed
    /projectA/bar/...
    /projectA/bar-seed
    /projectB/baz
    /projectB/baz-seed
    
    • [x] Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
    • [x] Ensure that the pull request title represents the desired changelog entry
    • [x] Please describe what you did
    • [x] Link to relevant issues in GitHub or Jira
    • [x] Link to relevant pull requests, esp. upstream and downstream changes
    • [x] Ensure you have provided tests - that demonstrates feature works or fixes the issue
  • Add svn repository browser support

    Add svn repository browser support

    Added a high-level DSL option to configure a repository browser for svn SCM. To this day, a svn browser could only be configured modifying the underlying XML with a configure block.

    svn {
        location('https://svn.didiez.es/')
        configure { scmNode ->
            scmNode / browser(class: 'hudson.scm.browsers.WebSVN') {
                url('https://didiez.es/websvn/')
            }
        }
    }
    

    Usage example:

    New high-level DSL:

    svn {
        location('https://svn.didiez.es/')
        browser {
            webSVN('https://didiez.es/websvn/')
        }
    }
    

    or with Dynamic DSL:

    svn {
        location('https://svn.didiez.es/')
        browser {
            webSVN {
                url('https://didiez.es/websvn/')
            }
        }
    }
    
  • JENKINS-66189 Use dynamic DSL instead of outdated PostBuildScript syntax

    JENKINS-66189 Use dynamic DSL instead of outdated PostBuildScript syntax

    Hi @daspilker!

    A user complained in https://issues.jenkins.io/browse/JENKINS-66189 that his Job DSL generated PostBuildScript configuration fails. That's because I removed some deprecated configuration options in version 3.x and forgot that there is a non-dynamic DSL in the Job DSL plugin.

    To be sure, no one uses the old syntax any longer, please let us remove the entries from the Job DSL plugin. I removed them in this pull request.

    Best regards

    Daniel

    • [x] Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
    • [x] Ensure that the pull request title represents the desired changelog entry
    • [x] Please describe what you did
    • [x] Link to relevant issues in GitHub or Jira
    • [x] Link to relevant pull requests, esp. upstream and downstream changes
    • [x] Ensure you have provided tests - that demonstrates feature works or fixes the issue
  • Add post build task for multiply conditions

    Add post build task for multiply conditions

    https://issues.jenkins.io/browse/JENKINS-66055

    Problem statement: Currently, using the postBuildTask method is possible to create only separated check conditions.

    Solution: Create a method to support the creation of multiply chain conditions.

Run Jobs on a schedule, supports fixed interval, timely, and cron-expression timers; Instrument your processes and expose metrics for each job.

A simple process manager that allows you to specify a Schedule that execute a Job based on a Timer. Schedule manage the state of this job allowing you to start/stop/restart in concurrent safe way. Schedule also instrument this Job and gather metrics and optionally expose them via uber-go/tally scope.

Mar 28, 2022
A persistent and flexible background jobs library for go.

Jobs Development Status Jobs is no longer being actively developed. I will still try my best to respond to issues and pull requests, but in general yo

Apr 24, 2022
gron, Cron Jobs in Go.

gron Gron provides a clear syntax for writing and deploying cron jobs. Goals Minimalist APIs for scheduling jobs. Thread safety. Customizable Job Type

May 9, 2022
YTask is an asynchronous task queue for handling distributed jobs in golang
YTask is an asynchronous task queue for handling distributed jobs in golang

YTask is an asynchronous task queue for handling distributed jobs in golang

May 10, 2022
Tiny library to handle background jobs.

bgjob Tiny library to handle background jobs. Use PostgreSQL to organize job queues. Highly inspired by gue Features Durable job storage At-least-ones

Nov 16, 2021
A way of scheduling volcano jobs

JobFlow 背景 volcano Volcano是CNCF 下首个也是唯一的基于Kubernetes的容器批量计算平台,主要用于高性能计算场景。 它提供了Kubernetes目前缺 少的一套机制,这些机制通常是机器学习大数据应用、科学计算、 特效渲染等多种高性能工作负载所需的。 现状:当前vol

Apr 21, 2022
Executes jobs in separate GO routines. Provides Timeout, StartTime controls. Provides Cancel all running job before new job is run.

jobExecutor Library to execute jobs in GO routines. Provides for Job Timeout/Deadline (MaxDuration()) Job Start WallClock control (When()) Add a job b

Jan 10, 2022
Graceful shutdown with repeating "cron" jobs (running at a regular interval) in Go

Graceful shutdown with repeating "cron" jobs (running at a regular interval) in Go Illustrates how to implement the following in Go: run functions ("j

Jan 15, 2022
A programmable, observable and distributed job orchestration system.
A programmable, observable and distributed job orchestration system.

?? Overview Odin is a programmable, observable and distributed job orchestration system which allows for the scheduling, management and unattended bac

Apr 7, 2022
Distributed Task Scheduling System|分布式定时任务调度平台
Distributed Task Scheduling System|分布式定时任务调度平台

Crocodile Distributed Task Scheduling System English | 中文 Introduction A distributed task scheduling system based on Golang that supports http request

May 13, 2022
high performance distributed task scheduling system, Support multi protocol scheduling tasks
 high performance distributed task scheduling system, Support multi protocol scheduling tasks

high performance distributed task scheduling system, Support multi protocol scheduling tasks

May 4, 2022
goInterLock is golang job/task scheduler with distributed locking mechanism (by Using Redis🔒).
goInterLock is golang job/task scheduler with distributed locking mechanism (by Using Redis🔒).

goInterLock is golang job/task scheduler with distributed locking mechanism. In distributed system locking is preventing task been executed in every instant that has the scheduler,

Apr 20, 2022
RESTful-JSON-API - RESTful-JSON-API using Go

RESTful-JSON-API using Go This basic REST-API principle establishes a one-to-one

Feb 15, 2022
A simple Cron library for go that can execute closures or functions at varying intervals, from once a second to once a year on a specific date and time. Primarily for web applications and long running daemons.

Cron.go This is a simple library to handle scheduled tasks. Tasks can be run in a minimum delay of once a second--for which Cron isn't actually design

May 4, 2022
Lightweight, fast and dependency-free Cron expression parser (due checker) for Golang (tested on v1.13 and above)

adhocore/gronx gronx is Golang cron expression parser ported from adhocore/cron-expr. Zero dependency. Very fast because it bails early in case a segm

May 8, 2022
clockwork - Simple and intuitive job scheduling library in Go.
clockwork - Simple and intuitive job scheduling library in Go.

clockwork A simple and intuitive scheduling library in Go. Inspired by python's schedule and ruby's clockwork libraries. Example use package main imp

Mar 3, 2022
Easy and fluent Go cron scheduling

goCron: A Golang Job Scheduling Package. goCron is a Golang job scheduling package which lets you run Go functions periodically at pre-determined inte

May 15, 2022
Efficient and reliable background processing for Go

CurlyQ CurlyQ provides a simple, easy-to-use interface for performing background processing in Go. It supports scheduled jobs, job deduplication, and

Mar 3, 2022
A lightweight job scheduler based on priority queue with timeout, retry, replica, context cancellation and easy semantics for job chaining. Build for golang web apps.

Table of Contents Introduction What is RIO? Concern An asynchronous job processor Easy management of these goroutines and chaining them Introduction W

May 3, 2022