Configuration as code is a well-established practice for CI servers. The benefits of this approach include versioning support via VCS repository, a simplified audit of the configuration changes, and improved portability of the configurations. Some users may also prefer code to configuring the builds with point-and-click via UI. In TeamCity, we can use Kotlin DSL to author build configurations.

The possibility to use Kotlin for defining build configurations was added in TeamCity 10. In TeamCity 2018.x Kotlin support was greatly improved for a more pleasant user experience.

In this series of posts, we are going to explain how to use Kotlin to define build configurations for TeamCity. We will start with the basics on how to get started with configuration-as-code in TeamCity. We will then dive into the practicalities of using Kotlin DSL for build configurations. Finally, we will take a look at advanced topics such as extending the DSL.

1. Getting started with Kotlin DSL
2. Working with configuration scripts
3. Creating build configurations dynamically
4. Extending Kotlin DSL
5. Using libraries
6. Testing configuration scripts

The demo application

In this tutorial, we are going to use the famous spring-petclinic project for demonstration. Spring Petclinic is a Java project that uses Maven for the build. The goal of this post is to demonstrate an approach for building an existing application with Kotlin DSL.

It is best to add a Kotlin DSL configuration to an existing project in TeamCity. First, create a new project in TeamCity by pointing to a repository URL. For our demo project, TeamCity will detect the presence of the Maven pom.xml and propose the required build steps. As a result, the new project will include one VCS root and a build configuration with a Maven build step and a VCS trigger.

The next step for this project is transitioning to configuration-as-code with Kotlin DSL. To do that, we have to enable Versioned Settings in our project.

WARNING! Once you enable Versioned Settings for the project, TeamCity will generate the corresponding files and immediately commit/push them into the selected repository.

Enable Versioned Settings

To start using Kotlin build scripts in TeamCity the Versioned Settings have to be enabled. Regardless of whether you are starting from scratch or you have an existing project. In the project configuration, under Versioned Settings, select the Synchronization enabled option, select a VCS root for the settings, and choose Kotlin format.

The VCS root selected to store the settings can be the same as the source code of the application you want to build. For our demo, however, we are going to use a specific VCS root dedicated to only the settings. Hence, there will be 2 VCS roots in the project: one for source code of the application and the other for build settings.

In addition to this, it is possible to define which settings to take when the build starts. When we use Kotlin DSL and make changes to the scripts in our favorite IDE, the “source of truth” is located in the VCS repository. Hence, it is advised to enable the use settings from VCS option. The changes to the settings are then always reflected on the server.

Once the Versioned Settings are enabled, TeamCity will generate the corresponding script and commit to the selected VCS root. Now we can pull the changes from the settings repository and start editing.

Opening the configuration in IntelliJ IDEA

The generated settings layout is a .teamcity directory with two files: settings.kts and pom.xml. For instance, the following are the files checked in to version control for the spring-petclinic application.

Clone the demo repository:

git clone https://github.com/antonarhipov/spring-petclinic-teamcity-dsl.git

Open the folder in IntelliJ IDEA, you will see the following layout:

Essentially, the .teamcity folder is a Maven module. Right-click on the pom.xml file and select Add as Maven Project – the IDE will import the Maven module and download the required dependencies.

pom.xml

The TeamCity specific classes that we use in the Kotlin script are coming from the dependencies that we declare in the pom.xml as part of the Maven module. This is a standard pom.xml file, besides that, we have to pay attention to a few things.

The DSL comes in a series of packages with the main one being the actual DSL which is contained in the configs-dsl-kotlin-{version}.jar file. Examining it, we can see that it has a series of classes that describe pretty much the entire TeamCity user interface.

The other dependencies that we see in the list are the Kotlin DSL extensions contributed by TeamCity plugins.

Some of the dependencies are downloaded from JetBrains’ own Maven repository. However, the plugins you have installed on your TeamCity instance may provide the extensions for the DSL, and therefore our pom.xml needs to pull the dependencies from the corresponding server. This is why you can find an additional repository URL in the pom.xml file.

The source directory is also redefined in this pom.xml since in many cases there will be just one settings.kts file. There is not much use for the standard Maven project layout here.

settings.kts

kts is a Kotlin Script file, different from a Kotlin file (.kt) in that it can be run as a script. All the code relevant to the TeamCity configuration can be stored in this file, but it can also be divided into several files to provide a better separation of concerns. Imports omitted, this is how the settings.kts for our demo project looks:

version indicates the TeamCity version, and project() is the main entry point to the configuration script. It is a function call, which takes as a parameter a block that represents the entire TeamCity project. In that block, we compose the structure of the project.

The vcsRoot(...) function call registers a predefined VCS root to the project. The buildType(...) function registers a build configuration. As a result, there is one project, with one VCS root, and one build configuration declared in our settings.kts file.

The corresponding objects for VCS root and the build configuration are declared in the same script.

Build configuration

The Build object inherits from the BuildType class that represents the build configuration in TeamCity. The object is registered in the project using a buildType(...) function call.

The declarations in the object are self-explanatory: you can see the name of the build configuration, artifact rules, the related VCS root, build steps, and triggers. There are more possibilities that we can use, which we will cover in further posts.

VCS root

The VCS root object, PetclinicVcs, is a very simple one in our example. It has just two attributes: the name, and the URL of the repository.

The parent type of the object, GitVcsRoot, indicates that this is a git repository that we’re going to connect to.

There are more attributes that we can specify for the VCS root object, like branches specification, and authentication type if needed.

Import project with the existing Kotlin script

It is possible to import existing Kotlin settings. When creating the project from a repository URL, TeamCity will scan the sources. If existing Kotlin settings are detected, the wizard will suggest importing them.

You can then decide if you want to just import the project from the settings, import and enable the synchronization with the VCS repository, or proceed without the import.

Summary

In this part of the series, we’ve looked at how to get started configuring the builds in TeamCity with Kotlin DSL. We explored the main components of a TeamCity configuration script and its dependencies. In part two, we’ll dive a little deeper into the DSL, modify the script, and see some of the benefits that Kotlin and IntelliJ IDEA already start providing us with in terms of guidance via code assistants.

This is part two of the six-part series on working with Kotlin to create build configurations for TeamCity.

1. Getting started with Kotlin DSL
2. Working with configuration scripts
3. Creating build configurations dynamically
4. Extending Kotlin DSL
5. Using libraries
6. Testing configuration scripts

In the first part of the series, we have seen how to get started with Kotlin DSL for TeamCity. Now we’ll dive a little deeper into the DSL and see what it provides us with in terms of building configuration scripts.

An important thing to note is that TeamCity 2018.x uses Kotlin version 1.2.50.

Configuration script

Because the configuration is actually a valid Kotlin program, we get to use all the assistance from the IDE – code completion, refactoring, and navigation.

As we saw in the previous post, the entry point to the configuration is the project {...} function call defined in settings.kts. Let’s examine the individual blocks of the script.

Project

The top-level project element in the settings.kts file represents a top-level context for all the build configurations and subprojects that are declared within the scope.

project {
}

A top-level project does not need to have an id or a name. These attributes are defined when we register a new project in TeamCity.

A project in TeamCity may include sub-projects and build configurations. A sub-project should be registered in the main context using the subProject function:

project {
   subProject(MyProject)
}

We also can register a few other entities, like one or more VCS roots and build configurations.

project {
   vcsRoot(PetclinicVcs)
   buildType(Build)
}

The above exactly matches our demo project configuration script. PetclinicVcs and Build objects represent a VCS root and build configuration – this is where the build happens!

Build configuration

The build configuration is represented by a BuildType class in TeamCity’s Kotlin DSL. To define a new build configuration we define an object that derives from the BuildType.

object Build: BuildType({
   
})

The constructor of BuildType receives a block of code, enclosed within the curly braces {}. This is where we define all the required attributes for the build configuration.

object Build: BuildType({
   id(“Build”)
   name = “Build”

   vcs {
     root(PetclinicVcs)
   }
   
   steps {
     maven {
       goals = “clean package”
     }
   }
 
   triggers {
      vcs {}
   }
})

Let’s examine the individual configuration blocks in the example above.

The id and name

The first lines in the Build object denote its id and name. The id, if not specified explicitly, will be derived from the object name.

Version control settings

The vcs{} block is used to define the version control settings, including the list of VCS roots and other attributes.

Build steps

After VCS settings block you will find the most important block, steps{}, where we define all the required build steps.

In our example, for a simple Maven project, we only define one Maven build step with clean and package goals.

steps {
     maven {
       goals = “clean package”

       //Other options
       dockerImage = “maven:3.6.0-jdk-8”
       jvmArgs = “-Xmx512”
       //etc
     }
}

Besides Maven, there are plenty of other build steps to choose from in TeamCity. Here are a few examples:

Gradle build step

gradle {
    tasks = “clean build”
}

Command line runner

script {
    scriptContent = “echo %build.number%”
}

Ant task with inline build script

ant {
    mode = antScript {
        content = """
            <?xml version="1.0" encoding="UTF-8">
            <project name="echoMessage">
            <target name="sayHello">
              <echo message="Hello, world!"/>
            </target>
            </project>  
        """.trimIndent()
    }
    targets = "sayHello"
}

Docker command

dockerCommand {
    commandType = build {
        source = path {
            path = "Dockerfile"
        }
        namesAndTags = "antonarhipov/myimage:%build.number%"
        commandArgs = "--pull"
    }
}

Finding a DSL snippet for a build step

Despite the IDE support for Kotlin it might still be a bit challenging for new users to configure something in the code. “How do I know how to configure the desired build step and what are the configuration options for that?” Have no fear – TeamCity can help with that! For a build configuration, find the View DSL toggle on the left-hand side of the screen:

The toggle will provide a preview of the given build configuration, here you can locate the build step that you want to configure. Say, we’d like to add a new build step for building a Maven module.

Add a new build step, choose Maven, fill in the attributes, and without saving the build step – click on the toggle to preview the build configuration. The new build step will be highlighted as follows:

You can now copy the code snippet and paste it to the configuration script opened in your IDE.

Please note that If you save the new build step for a project that is already configured via Kotlin DSL scripts — this is allowed — TeamCity will generate a patch and commit it to the settings VCS root. It is then the user’s responsibility to merge the patch into the main Kotlin script.

The VCS root

Besides the build configuration, our demo project also includes a VCS root definition that the build configuration depends on.

object PetclinicVcs : GitVcsRoot({
   name = "PetclinicVcs"
   url = "https://github.com/spring-projects/spring-petclinic.git"
})

This is a minimal definition of the VCS root for a Git repository. The id attribute is not explicitly specified here, hence it is calculated automatically from the object’s name. The name attribute is required for displaying in the UI, and the url defines the location of the sources.

Depending on the kind of VCS repository we have, we may specify other attributes as well.

Modifying the build process

Let’s extend this build a little bit. TeamCity provides a feature called Build Files Cleaner, also known as Swabra. Swabra makes sure that files left by the previous build are removed before running new builds.

We can add it using the features function. As we start to type, we can see that the IDE provides us with completion:

The features function takes a series of feature functions, each of which adds a particular feature. In our case, the code we’re looking for is

features {
   swabra {
   }
}

In UI, you will find the result in the Build Features view:

We have now modified the build configuration, and it works well. The problem is, if we want to have this feature for every build configuration, we’re going to end up repeating the code. Let’s refactor it to a better solution.

Refactoring the DSL

What we’d ideally like is to have every build configuration automatically have the Build Files Cleaner feature, without having to manually add it. In order to do this, we could introduce a function that wraps every instance of BuildType with this feature. In essence, instead of having the Project call

buildType(Build)
buildType(AnotherBuild)
buildType(OneMoreBuild)

we would have it call

buildType(cleanFiles(Build))
buildType(cleanFiles(AnotherBuild))
buildType(cleanFiles(OneMoreBuild))

For this to work, we’d need to create the following function

fun cleanFiles(buildType: BuildType): BuildType {
   buildType.features {
       swabra {}
   }
   return buildType
}

The new function essentially takes a BuildType, adds a feature to it, and then returns the BuildType. Given that Kotlin allows top-level functions (i.e. no objects or classes are required to host a function), we can put it anywhere in the code or create a specific file to hold it.

We can improve the code a little so that it only adds the feature if it doesn’t already exist:

fun cleanFiles(buildType: BuildType): BuildType {
   if (buildType.features.items.find { it.type == "swabra" } == null) {
       buildType.features {
           swabra {
           }
       }
   }
   return buildType
}

Generalizing feature wrappers

The above function is great in that it allows us to add a specific feature to all the build configurations. What if we wanted to generalize this so that we could define the feature ourselves? We can do so by passing a block of code to our cleanFiles function, which we’ll also rename to something more generic.

What we’re doing here is creating what’s known as a higher-order function, a function that takes another function as a function. In fact, this is exactly what features, feature, and many of the other TeamCity DSL’s are.

fun wrapWithFeature(buildType: BuildType, featureBlock: BuildFeatures.() -> Unit): BuildType {
   buildType.features {
       featureBlock()
   }
   return buildType
}

One particular thing about this function, however, is that it’s taking a special function as a parameter, which is an extension function in Kotlin. When passing in this type of parameter, we refer to it as Lambdas with Receivers (i.e. there is a receiver object that the function is applied on).

buildType(wrapWithFeature(Build){
   swabra {}
})

This then allows us to make calls to this function in a nice way, referencing feature directly.

Summary

In this post, we’ve seen how we can modify TeamCity configuration scripts using the extensive Kotlin-based DSL. What we have in our hands is a full programming language along with all the features and power that it provides. We can encapsulate functionality in functions to re-use, we can use higher-order functions as well as other things that open up many possibilities.

In the next post, we’ll see how to use some of this to dynamically create scripts.

This is part three of the six-part series on working with Kotlin to create build configurations for TeamCity.

1. Getting started with Kotlin DSL
2. Working with configuration scripts
3. Creating build configurations dynamically
4. Extending Kotlin DSL
5. Using libraries
6. Testing configuration scripts

We have seen in the previous post how we can leverage some of Kotlin’s language features to reuse code. In this part, we’re going to take advantage of the fact that we are dealing with a full programming language and not just a limited DSL, to create a dynamic build configuration.

Generating build configurations

The scenario is the following: we have a Maven project that we need to test on different operating systems and different JDK versions. This potentially generates a lot of different build configurations that we’d need to create and maintain.

Here’s an example configuration for building a Maven project:

version = "2018.2"

project {
    buildType(BuildForMacOSX)
}

object BuildForMacOSX : BuildType({
   name = "Build for Mac OS X"

   vcs {
       root(DslContext.settingsRoot)
   }

   steps {
       maven {
           goals = "clean package"
           mavenVersion = defaultProvidedVersion()
           jdkHome = "%env.JDK_18%"
       }
   }

   requirements {
       equals("teamcity.agent.jvm.os.name", "Mac OS X")
   }
})

If we try to create each individual configuration for all the combinations of OS types and JDK versions we will end up with a lot of code to maintain. Instead of creating each build configuration manually, what we can do is write some code to generate all the different build configurations for us.

A very simple approach we could take here is to have two lists with the versions of OS types and JDK versions, and then iterate over them to generate the build configurations:

val operatingSystems = listOf("Mac OS X", "Windows", "Linux")
val jdkVersions = listOf("JDK_18", "JDK_11")

project {
   for (os in operatingSystems) {
       for (jdk in jdkVersions) {
           buildType(Build(os, jdk))
       }
   }
}

We need to adjust our build configuration a little to use the parameters. Instead of an object, we will declare a class with a constructor that will accept the parameters for the OS type and JDK version.

class Build(val os: String, val jdk: String) : BuildType({
   id("Build_${os}_${jdk}".toExtId())
   name = "Build ($os, $jdk)"

   vcs {
       root(DslContext.settingsRoot)
   }

   steps {
       maven {
           goals = "clean package"
           mavenVersion = defaultProvidedVersion()
           jdkHome = "%env.${jdk}%"
       }
   }

   requirements {
       equals("teamcity.agent.jvm.os.name", os)
   }
})

An important thing to notice here is that we are now setting the id of the build configuration explicitly using the id(...) function call, e.g. id("Build_${os}_${jdk}".toExtId())
Since the id shouldn’t contain any other characters the DSL library provides a toExtId() function that can be used to sanitize the value that we want to assign.

The result of this is that we will see 6 build configurations created:

Summary

The above is just a sample of what can be done when creating dynamic build scripts. In this case, we created multiple build configurations, but we could have just as easily created multiple steps, certain VCS triggers, or whatever else that might come in useful. The important thing to understand is that at the end of the day, Kotlin Configuration Script isn’t just merely a DSL but a fully fledged programming language.

1. Getting started with Kotlin DSL
2. Working with configuration scripts
3. Creating build configurations dynamically
4. Extending Kotlin DSL
5. Using libraries
6. Testing configuration scripts

TeamCity allows us to create build configurations that are dependent on one another, with the dependency being either snapshots or artifacts. The configuration for defining dependencies is done at the build configuration level. For instance, assuming that we have a build type Publish, that has a snapshot and artifact dependencies on Package, we would define this in the build type Publish in the following way:

object Package : BuildType({
   name = "Package"

   artifactRules = “application.zip”

   steps {
       // define the steps needed to produce the application.zip
   }
})

object Publish: BuildType({
   name="Publish"

   steps {
       // define the steps needed to publish the artifacts
   }

   dependencies {
       snapshot(Package){}
       artifacts(Package) {
           artifactRules = "application.zip"
       }
   }
})

and in turn, if Package had dependencies on previous build configurations, we’d define these in the dependencies segment of its build configuration.

TeamCity then allows us to visually see this using the Build Chains tab in the user interface:

The canonical approach to defining build chains in TeamCity is when we declare the individual dependencies in the build configuration. The approach is simple but as the number of build configurations in the build chain grows it becomes harder to maintain the configurations.

Imagine there’s a large number of build configurations in the chain, and we want to add one more somewhere in the middle of the workflow. For this to work, we have to configure the correct dependencies in the new build configuration. But we also need to update the dependencies in the existing build configurations to point at the new one. This approach does not seem to scale well.

But we can work around this problem by introducing our own abstractions in TeamCity’s DSL.

Defining the pipeline in code

What if we had a way to describe the pipeline on top of the build configurations that we define separately in the project? The pipeline abstraction is something we need to create ourselves. The goal of this abstraction is to allow us to omit specifying the snapshot dependencies in the build configurations that we want to combine into a build chain.

Assume that we have a few build configurations: Compile, Test, Package, and Publish. Test needs a snapshot dependency on Compile, Package depends on Test, Publish depends on Package, and so on. So these build configurations compose a build chain.

Let’s define, how the new abstraction would look. We think of the build chain described above as of a “sequence of builds”. So why not to describe it as follows:

project {
    sequence {
        build(Compile)
        build(Test)
        build(Package)
        build(Publish)
    }
}

Almost immediately, we could think of a case where we need to be able to run some builds in parallel.

project {
    sequence {
        build(Compile)
        parallel {
            build(Test1)
            build(Test2)
        } 
        build(Package)
        build(Publish)
    }
}

In the example above, Test1 and Test2 are defined in the parallel block, both depend on Compile. Package depends on both, Test1 and Test2. This can handle simple but common kinds of build chains where a build produces an artifact, several builds test it in parallel and the final build deploys the result if all its dependencies are successful.

For our new abstraction, we need to define, what sequence, parallel, and build are. Currently, the TeamCity DSL does not provide this functionality. But that’s where Kotlin’s extensibility proves quite valuable, as we’ll now see.

Creating our own DSL definitions

Kotlin allows us to create extension functions and properties, which are the means to extend a specific type with new functionality, without having to inherit from them. When passing extension functions as arguments to other functions (i.e. higher-order functions), we get what we call in Kotlin Lambdas with Receivers, something we’ve seen already in this series when Generalising feature wrappers in the second part of this series. We will apply the same concept here to create our DSL.

class Sequence {
   val buildTypes = arrayListOf<BuildType>()

   fun build(buildType: BuildType) {
       buildTypes.add(buildType)
   }
}

fun Project.sequence(block: Sequence.() -> Unit){
   val sequence = Sequence().apply(block)

   var previous: BuildType? = null

   // create snapshot dependencies
   for (current in sequence.buildTypes) {
       if (previous != null) {
           current.dependencies.snapshot(previous){}
       }
       previous = current
   }

   //call buildType function on each build type
   //to include it into the current Project
   sequence.buildTypes.forEach(this::buildType)
}

The code above adds an extension function to the Project class and allow us to declare the sequence. Using the aforementioned Lambda with Receivers feature we declare that the block used as a parameter to the sequence function will provide the context of the Sequence class. Hence, we will be able to call the build function directly within that block:

project {
    sequence {
         build(BuildA)
         build(BuildB) // BuildB has a snapshot dependency on BuildA
    }
}

Adding parallel blocks

To support the parallel block we need to extend our abstraction a little bit. There will be a serial stage that consists of a single build type and a parallel stage that may include many build types.

interface Stage

class Single(val buildType: BuildType) : Stage

class Parallel : Stage {
   val buildTypes = arrayListOf<BuildType>()

   fun build(buildType: BuildType) {
       buildTypes.add(buildType)
   }
}

class Sequence {
   val stages = arrayListOf<Stage>()

   fun build(buildType: BuildType) {
       stages.add(Single(buildType))
   }

   fun parallel(block: Parallel.() -> Unit) {
       val parallel = Parallel().apply(block)
       stages.add(parallel)
   }
}

To support the parallel blocks we will need to write slightly more code. Every build type defined in the parallel block will have a dependency on the build type which was declared before the parallel block. And the build type declared after the parallel block will depend on all the build types declared in the block. We’ll make the assumption that a parallel block cannot follow a parallel block, though it’s not a big problem to support this feature.

fun Project.sequence(block: Sequence.() -> Unit) {
   val sequence = Sequence().apply(block)

   var previous: Stage? = null

   for (current in sequence.stages) {
       if (previous != null) {
           createSnapshotDependency(current, previous)
       }
       previous = current
   }

   sequence.stages.forEach {
       if (it is Single) {
           buildType(it.buildType)
       }
       if (it is Parallel) {
           it.buildTypes.forEach(this::buildType)
       }
   }
}

fun createSnapshotDependency(stage: Stage, dependency: Stage){
   if (dependency is Single) {
       stageDependsOnSingle(stage, dependency)
   }
   if (dependency is Parallel) {
       stageDependsOnParallel(stage, dependency)
   }
}

fun stageDependsOnSingle(stage: Stage, dependency: Single) {
   if (stage is Single) {
       singleDependsOnSingle(stage, dependency)
   }
   if (stage is Parallel) {
       parallelDependsOnSingle(stage, dependency)
   }
}

fun stageDependsOnParallel(stage: Stage, dependency: Parallel) {
   if (stage is Single) {
       singleDependsOnParallel(stage, dependency)
   }
   if (stage is Parallel) {
       throw IllegalStateException("Parallel cannot snapshot-depend on parallel")
   }
}

fun parallelDependsOnSingle(stage: Parallel, dependency: Single) {
   stage.buildTypes.forEach { buildType ->
       singleDependsOnSingle(Single(buildType), dependency)
   }
}

fun singleDependsOnParallel(stage: Single, dependency: Parallel) {
   dependency.buildTypes.forEach { buildType ->
       singleDependsOnSingle(stage, Single(buildType))
   }
}

fun singleDependsOnSingle(stage: Single, dependency: Single) {
   stage.buildType.dependencies.snapshot(dependency.buildType) {}
}

The DSL now supports parallel blocks in the sequence:

parallel {
  sequence {
    build(Compile)
    parallel {
       build(Test1)
       build(Test2)
    }
    build(Package)
    build(Publish)
  }
}

We could extend the DSL even further to support nesting of the blocks by allowing defining the sequence inside the parallel blocks.

project {
   sequence {
       build(Compile) 
       parallel {
           build(Test1) 
           sequence {
              build(Test2) 
              build(Test3)
           } 
       }
       build(Package) 
       build(Publish) 
   }
}

Nesting the blocks allows us to create build chains of almost any complexity. However, our example only covers snapshot dependencies. We haven’t covered artifact dependencies here yet and these would be nice to see in the sequence definition as well.

Adding artifact dependencies

For passing an artifact dependency from Compile to Test, simply specify that Compile produces the artifact and Test requires the same artifact.

sequence {
   build(Compile) {
      produces("application.jar")
   }
   build(Test) {
      requires(Compile, "application.jar")
   }
}

produces and requires are the new extension functions for the BuildType:

fun BuildType.produces(artifacts: String) {
   artifactRules = artifacts
}

fun BuildType.requires(bt: BuildType, artifacts: String) {
   dependencies.artifacts(bt) {
       artifactRules = artifacts
   }
}

We also need to provide a way to execute these new functions in the context of BuildType. For this, we can override the build() function of the Sequence and Parallel classes to accept the corresponding block by using Lambda with Receivers declaration:

fun Sequence.build(bt: BuildType, block: BuildType.() -> Unit = {}){
   bt.apply(block)
   stages.add(Single(bt))
}

fun Parallel.build(bt: BuildType, block: BuildType.() -> Unit = {}){
   bt.apply(block)
   stages.add(Single(bt))
}

As a result, we can define a more complex sequence with our brand new DSL:

sequence {
   build(Compile) {
       produces("application.jar")
   }
   parallel {
       build(Test1) {
           requires(Compile, "application.jar")
           produces("test.reports.zip")
       }
       sequence {
           build(Test2) {
               requires(Compile, "application.jar")
               produces("test.reports.zip")
           }
           build(Test3) {
               requires(Compile, "application.jar")
               produces("test.reports.zip")
           }
       }
   }
   build(Package) {
       requires(Compile, "application.jar")
       produces("application.zip")
   }
   build(Publish) {
       requires(Package, "application.zip")
   }
}

Summary

It’s important to understand that this is just one of many ways in which we can define pipelines. We’ve used the terms sequence, parallel and build. We could just as well have used the term buildchain to align it better with the UI. We also added the convenience methods to the BuildType to work with the artifacts.

The ability to easily extend the TeamCity DSL with our own constructs, provides us with flexibility. We can create custom abstractions on top of the existing DSL to better reflect how we reason about our build workflow.

In the next post, we’ll see how to extract our DSL extensions into a library for further re-use.

1. Getting started with Kotlin DSL
2. Working with configuration scripts
3. Creating build configurations dynamically
4. Extending Kotlin DSL
5. Using libraries
6. Testing configuration scripts

In the previous post, we have seen how to extend TeamCity’s Kotlin DSL by adding new abstractions. If the new abstraction is generic enough, it would make sense to reuse it in different projects. In this post, we are going to look at how to extract the common code into a library. We will then use this library as a dependency in a TeamCity project.

Maven project and dependencies

In the first post of this series, we started out by creating an empty project in TeamCity. We then instructed the server to generate the configuration settings in Kotlin format.

The generated pom.xml file pointed at two repositories and a few dependencies. This pom.xml is a little excessive for our next goal, but we can use it as a base, and remove the parts that we don’t need for the DSL library.

The two repositories in the pom.xml file are jetbrains-all, the public JetBrains repository, and teamcity-server that points to the TeamCity server where we generated the settings. The reason why the TeamCity server is used as a repository for the Maven project is that there may be some plugins installed that extend the TeamCity Kotlin DSL. And we may want to use those extensions for configuring the builds.

However, for a library, it makes sense to rely on a minimal set of dependencies to ensure portability. Hence, we keep only those dependencies that are downloaded from the public JetBrains Maven repository and remove all the others. The resulting pom.xml lists only 3 libraries: configs-dsl-kotlin-{version}.jar, kotlin-stdlib-jdk8-{version}.jar, and kotlin-script-runtime-{version}.jar.

The code

It’s time to write some code! In fact, it’s already written. In the previous post, we have introduced the new abstraction, the sequence, to automatically configure the snapshot dependencies for the build configurations. We only need to put this code into a *.kt file in our new Maven project.

We have published the example project on GitHub. Pipelines.kt lists all the extensions to the TeamCity DSL. That’s it! We now can build the library, publish it, and use it as a dependency in any TeamCity project with Kotlin DSL.

Using the library

The new library project is on GitHub, but we haven’t published it to any Maven repository yet. To add it as a dependency to any other Maven project we can use the awesome jitpack.io. The demo project demonstrates how the DSL library is applied.

Here’s how we can use the library:

1. Add the JitPack repository to the pom.xml file:

<repositories>
  <repository>
    <id>jitpack.io</id>
    <url>https://jitpack.io</url>
  </repository>
</repositories>

2. Add the dependency to the dependent DSL project’s pom.xml:

<dependencies>
  <dependency>
    <groupId>com.github.JetBrains</groupId>
    <artifactId>teamcity-pipelines-dsl</artifactId>
    <version>0.8</version>
  </dependency>
</dependencies>

The version is equal to a tag in the GitHub repository:

Once the IDE has downloaded the dependencies, we are able to use the DSL extensions provided by the library. See settings.kts of the demo project for an example.

Summary

TeamCity allows adding 3rd-party libraries as Maven dependencies. In this post, we have demonstrated how to add a dependency on the library that adds extensions to the TeamCity Kotlin DSL.

In this blog post, we are going to look at how to test TeamCity configuration scripts.

1. Getting started with Kotlin DSL
2. Working with configuration scripts
3. Creating build configurations dynamically
4. Extending Kotlin DSL
5. Using libraries
6. Testing configuration scripts

Given that the script is implemented with Kotlin, we can simply add a dependency to a testing framework of our choice, set a few parameters and start writing tests for different aspects of our builds.

In our case, we’re going to use JUnit. For this, we need to add the JUnit dependency to the pom.xml file

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
</dependency>

We also need to define the test directory.

<testSourceDirectory>tests</testSourceDirectory>
<sourceDirectory>settings</sourceDirectory>

In this example, we have redefined the source directory as well, so it corresponds with the following directory layout.

Once we have this in place, we can write unit tests as we would in any other Kotlin or Java project, accessing the different components of our project, build types, etc.

However, before we can start writing any code we need to make a few adjustments to the script. The reason is that our code for the configuration resides in settings.kts file. The objects that we declared in the kts file are not visible in the other files. Hence, to make these objects visible, we have to extract them into a file (or multiple files) with a kt file extension.

First, instead of declaring the project definition as a block of code in the settings.kts file, we can extract it into an object:

version = "2018.2"

project(SpringPetclinic)

object SpringPetclinic : Project ({
   …
})

The SpringPetclinic object then refers to the build types, VCS roots, etc.

Next, to make this new object visible to the test code, we need to move this declaration into a file with a kt extension:

settings.kts now serves as an entry point for the configuration where the project { } function is called. Everything else can be declared in the other *.kt files and referred to from the main script.

After the adjustments, we can add some tests. For instance, we could validate if all the build types start with a clean checkout:

import org.junit.Assert.assertTrue
import org.junit.Test

class StringTests {

   @Test
   fun buildsHaveCleanCheckOut() {
       val project = SpringPetclinic

       project.buildTypes.forEach { bt ->
           assertTrue("BuildType '${bt.id}' doesn't use clean checkout",
               bt.vcs.cleanCheckout)
       }
   }
}

Configuration checks as part of the CI pipeline

Running the tests locally is just one part of the story. Wouldn’t it be nice to run validation before the build starts?

When we make changes to the Kotlin configuration and check it into the source control, TeamCity synchronizes the changes and it will report any errors it encounters. The ability to now add tests allows us to add another extra layer of checks to make sure that our build script doesn’t contain any scripting errors and that certain things are validated such as the correct VCS checkout, as we’ve seen above, and the appropriate number of build steps are being defined, etc.

We can define a build configuration in TeamCity that will execute the tests for our Kotlin scripts prior to the actual build. Since it is a Maven project, we can apply Maven build step – we just need to specify the correct path to pom.xml, i.e. .teamcity/pom.xml.

The successful run of the new build configuration is a prerequisite for the rest of the build chain. Meaning, if there are any JUnit test failures, then the rest of the chain will not be able to start.