This post will describe how to compile a small application written in Kotlin using Bazel, tests, as well as how to use static analyzers.

Phosphorus

Phosphorus is the application that this post will cover. It’s a small utility that I wrote to check if an image matches a reference. If it doesn’t, Phosphorus generates an image highlighting the differences. The goal is to be able to check that something generates an image in a given way, and doesn’t change - at least if it’s not expected. The actual usage will be covered later in this series. While it’s not open-source yet, it’s something I intend to do at some point.

It’s written in Kotlin, as a couple external dependencies ( Clikt and Dagger), as well as a few tests. This is the structure:

The differ module contains the core logic - comparing two images, and generating a DiffResult. This DiffResult contains both the straightforward result of the comparison (are the two images identical?) and an image highlighting the differences, if any. The loader package is responsible for loading and writing images. Finally, the Phosphorus class orchestrates all that, in addition to processing command line arguments with Clikt.

Dependencies

Phosphorus has two dependencies: Clikt, and Dagger. Both of them are available as Maven artifacts. In order to pull Maven artifacts, the Bazel team provides a set of rules called rules_jvm_external. The idea is the following: you list a bunch of Maven coordinates and repositories, the rule will fetch all of them (and their transitive dependencies) during the loading phase, and generate Bazel targets corresponding to those Maven artifacts, on which you can depend. Let’s see how we can use them. The first step is to load the rules, in the WORKSPACE:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_jvm_external",
    sha256 = "62133c125bf4109dfd9d2af64830208356ce4ef8b165a6ef15bbff7460b35c3a",
    strip_prefix = "rules_jvm_external-3.0",
    url = "https://github.com/bazelbuild/rules_jvm_external/archive/3.0.zip",
)

Then, we can load and call maven_install with the list of Maven coordinates we want, in the WORKSPACE too:

load("@rules_jvm_external//:defs.bzl", "maven_install")

maven_install(
    artifacts = [
        "com.github.ajalt:clikt:2.2.0",
        "com.google.dagger:dagger:2.25.2",
        "com.google.dagger:dagger-compiler:2.25.2",
        "com.google.truth:truth:1.0",
        "javax.inject:javax.inject:1",
        "junit:junit:4.12",
    ],
    fetch_sources = True,
    repositories = [
        "https://maven.google.com",
        "https://repo1.maven.org/maven2",
        "https://jcenter.bintray.com/",
    ],
    strict_visibility = True,
)

A couple of things to note:

• We’re also downloading JUnit and Truth, that we’re going to use in tests
• maven_install can try to download the sources, if they’re available on Maven, to be able to see them directly from the IDE

At this point, Clikt, JUnit and Truth are ready to be used. They are exposed respectively as @maven//:com_github_ajalt_clikt, @maven//:junit_junit and @maven//:com_google_truth_truth.

Dagger, on the other hand, comes with an annotation processor and, as such, needs some more work: it needs to be exposed as a Java Plugin. Because it’s a third party dependency, this will be defined in //third_party/dagger/BUILD:

java_plugin(
    name = "dagger_plugin",
    processor_class = "dagger.internal.codegen.ComponentProcessor",
    deps = [
        "@maven//:com_google_dagger_dagger_compiler",
    ],
)

java_library(
    name = "dagger",
    exported_plugins = [":dagger_plugin"],
    visibility = ["//visibility:public"],
    exports = [
        "@maven//:com_google_dagger_dagger",
        "@maven//:com_google_dagger_dagger_compiler",
        "@maven//:javax_inject_javax_inject",
    ],
)

It can now be used as //third_party/dagger.

Compilation

Bazel doesn’t support Kotlin out of the box (the few languages natively supported, Java and C++, are currently getting extracted from Bazel’s core, so all languages will soon share a similar integration). In order to compile some Kotlin code, we’ll have to use some Starlark rules describing how to use kotlinc. A set of rules is available here. While they don’t support Kotlin/Native, they do support targeting both the JVM (including Android) and JavaScript.

In order to use those rules, we need to declare them in the WORKSPACE:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "io_bazel_rules_kotlin",
    sha256 = "54678552125753d9fc0a37736d140f1d2e69778d3e52cf454df41a913b964ede",
    strip_prefix = "rules_kotlin-legacy-1.3.0-rc3",
    url = "https://github.com/bazelbuild/rules_kotlin/archive/legacy-1.3.0-rc3.zip",
)

load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kotlin_repositories", "kt_register_toolchains")

kotlin_repositories()

kt_register_toolchains()

Once that’s done, we have access to a few rules:

• kt_js_library
• kt_js_import
• kt_jvm_binary
• kt_jvm_import
• kt_jvm_library
• kt_jvm_test
• kt_android_library

We’re going to use kt_jvm_binary, kt_jvm_library as well as kt_jvm_test.

As JVM-based languages have a strong correlation between packages and folder structure, we need to be careful about where we store our source code. Bazel handles a few names as potential Java “roots”: java, javatests and src. Anything inside a directory named like this needs to follow the package/folder correlation. For example, a class fr.enoent.phosphorus.client.matcher.Phosphorus can be stored at those locations:

• //java/fr/enoent/phosphorus/Phosphorus.kt
• //tools/images/java/fr/enoent/phosphorus/Phosphorus.kt
• //java/tools/images/src/fr/enoent/phosphorus/Phosphorus.kt

In my repo, everything Java-related is stored under //java, and the corresponding tests are in //javatests (following the same structure). Phosphorus will hence be in //java/fr/enoent/phosphorus.

Let’s see how we can define a simple Kotlin library, with the data module. In //java/fr/enoent/phosphorus/data/BUILD:

load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_jvm_library")

kt_jvm_library(
    name = "data",
    srcs = [
        "DiffResult.kt",
        "Image.kt",
    ],
    visibility = [
        "//java/fr/enoent/phosphorus:__subpackages__",
        "//javatests/fr/enoent/phosphorus:__subpackages__",
    ],
)

And that’s it, we have our first library ready to be compiled! I won’t describe all the modules as it’s pretty repetitive and there’s not a lot of value into doing that, but let’s see what the main binary looks like. Defined in //java/fr/enoent/phosphorus/BUILD, we have:

load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_jvm_binary")

kt_jvm_binary(
    name = "phosphorus",
    srcs = [
        "Phosphorus.kt",
    ],
    main_class = "fr.enoent.phosphorus.PhosphorusKt",
    visibility = ["//visibility:public"],
    deps = [
        "//java/fr/enoent/phosphorus/differ",
        "//java/fr/enoent/phosphorus/differ/impl:module",
        "//java/fr/enoent/phosphorus/loader",
        "//java/fr/enoent/phosphorus/loader/io_impl:module",
        "//third_party/dagger",
        "@maven//:com_github_ajalt_clikt",
    ],
)

Note the name of the main_class: because it’s a Kotlin class, the compiler will append Kt at the end of its name. Once this is defined, we can run Phosphorus with this command:

bazel run //java/fr/enoent/phosphorus -- arguments passed to Phosphorus directly

Tests

As mentioned previously, the test root will be //javatests. Because we need to follow the packages structure, the tests themselves will be under //javatests/fr/enoent/phosphorus. They are regular JUnit 4 tests, using Truth for the assertions.

Defining unit tests is really straightforward, and follows really closely the pattern we saw with libraries and binaries. For example, the ImageTest test is defined like this, in //javatests/fr/enoent/phosphorus/data/BUILD:

load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_jvm_test")

kt_jvm_test(
    name = "ImageTest",
    srcs = ["ImageTest.kt"],
    deps = [
        "//java/fr/enoent/phosphorus/data",
        "@maven//:com_google_truth_truth",
        "@maven//:junit_junit",
    ],
)

This will define a Bazel target that we can invoke like this:

bazel test //javatests/fr/enoent/phosphorus/data:ImageTest

Hopefully, the output should look like this:

//javatests/fr/enoent/phosphorus/data:ImageTest                          PASSED in 0.3s

Once this is done, it’s possible to run ibazel test //javatests/fr/enoent/phosphorus/... - Bazel will then monitor all the test targets defined under that path, as well as their dependencies, and re-run all the affected tests as soon as something is edited. Because Bazel encourages small build targets, has some great caching, and the Kotlin compiler uses a persistent worker, the feedback loop is really quick.

Static analysis

For Kotlin, two tools are quite useful: Detekt, and Ktlint. The idea to run them will be really similar: having two supporting test targets for each actual Kotlin target, running Detekt and Ktlint on its sources. In order to do that easily, we’ll define some wrappers around the kt_jvm_* set of rules. Those wrappers will be responsible for generating the two supporting test targets, as well as calling the original kt_jvm_* rule. The resulting macro will be entirely transparent to use, the only difference being the load call.

Let’s see what those macros could look like. In //java/rules/defs.bzl:

load(
    "@io_bazel_rules_kotlin//kotlin:kotlin.bzl",
    upstream_kt_jvm_binary = "kt_jvm_binary",
    upstream_kt_jvm_library = "kt_jvm_library",
    upstream_kt_jvm_test = "kt_jvm_test",
)
def kt_jvm_binary(name, srcs, **kwargs):
    upstream_kt_jvm_binary(
        name = name,
        srcs = srcs,
        **kwargs
    )

    _common_tests(name = name, srcs = srcs)

def kt_jvm_library(name, srcs, **kwargs):
    upstream_kt_jvm_library(
        name = name,
        srcs = srcs,
        **kwargs
    )

    _common_tests(name = name, srcs = srcs)

def kt_jvm_test(name, srcs, size = "small", **kwargs):
    upstream_kt_jvm_test(
        name = name,
        srcs = srcs,
        size = size,
        **kwargs
    )

    _common_tests(name = name, srcs = srcs)

def _common_tests(name, srcs):
    # This will come soon, no-op for now

With those wrappers defined, we need to actually call them. Because we’re following the same signature and name as the upstream rules, we just need to update our load calls in the different BUILD files. load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_jvm_test") will become load("//java/rules:defs.bzl", "kt_jvm_test"), and so on. _common_tests will be responsible for calling Detekt and Ktlint, let’s see how.

Detekt

Artem Zinnatullin published a set of rules to run Detekt a week before I started writing this, making things way easier. As usual, let’s start by loading this in the WORKSPACE:

http_file(
    name = "detekt_cli_jar",
    sha256 = "e9710fb9260c0824b3a9ae7d8326294ab7a01af68cfa510cab66de964da80862",
    urls = ["https://jcenter.bintray.com/io/gitlab/arturbosch/detekt/detekt-cli/1.2.0/detekt-cli-1.2.0-all.jar"],
)

http_archive(
    name = "rules_detekt",
    sha256 = "f1632c2492291f5144a5e0f5e360a094005e20987518d228709516cc935ad1a1",
    strip_prefix = "bazel_rules_detekt-0.2.0",
    url = "https://github.com/buildfoundation/bazel_rules_detekt/archive/v0.2.0.zip",
)

This exposes a rule named detekt, which defines a build target, generating the Detekt report. While there are a few options, we’ll keep things simple. This is what a basic invocation looks like, in any BUILD file:

detekt(
    name = "detekt_report",
    srcs = glob(["**/*.kt"]),
)

We can integrate that in our _common_tests macro, to generate a Detekt target automatically for every Kotlin target:

def _common_tests(name, srcs):
    detekt(
        name = "%s_detekt_report" % name,
        srcs = srcs,
        config = "//java/rules/internal:detekt-config.yml",
    )

All our Kotlin targets now have a $name_detekt_report target generated automatically, using a common Detekt configuration.

The way this detekt rule work is by creating a build target, that generates the report. Which means that it’s not actually a test - which is what we were trying to achieve. In order to do this, we can use Bazel Skylib’s build_test. This test rule generates a test target that just has a dependency on other targets - if any of those dependencies fails to build, then the test fails. Otherwise, it passes. Our macro becomes:

def _common_tests(name, srcs):
    detekt(
        name = "%s_detekt_report" % name,
        srcs = srcs,
        config = "//java/rules/internal:detekt-config.yml",
    )

    build_test(
        name = "%s_detekt_test" % name,
        targets = [":%s_detekt_report" % name],
    )

And there we have it - a $name_detekt_test that is actually a test, and will fail if Detekt raises errors.

Ktlint

Ktlint doesn’t have any existing open-source rules. Let’s see how we can write our own minimal one. It will take as inputs the list of files to check, as well as an optional editorconfig configuration, that Ktlint supports natively.

The definition of the rules will be split in three files: two internal files defining respectively the action (how to invoke Ktlint) and the rule interface (what’s its name, its arguments…), as well as a third, public file, meant to be consumed by users.

Let’s start by downloading Ktlint itself. In the WORKSPACE, as usual:

http_file(
    name = "com_github_pinterest_ktlint",
    executable = True,
    sha256 = "a656342cfce5c1fa14f13353b84b1505581af246638eb970c919fb053e695d5e",
    urls = ["https://github.com/pinterest/ktlint/releases/download/0.36.0/ktlint"],
)

Let’s move onto the action definition. It’s a simple macro returning a string, which defines how to invoke Ktlint, given some arguments. In //tools/ktlint/internal/actions.bzl:

def ktlint(ctx, srcs, editorconfig):
    """Generates a test action linting the input files.

    Args:
      ctx: analysis context.
      srcs: list of source files to be checked.
      editorconfig: editorconfig file to use (optional)

    Returns:
      A script running ktlint on the input files.
    """

    args = []

    if editorconfig:
        args.append("--editorconfig={file}".format(file = editorconfig.short_path))


    for f in srcs:
        args.append(f.path)

    return "{linter} {args}".format(
        linter = ctx.executable._ktlint_tool.short_path,
        args = " ".join(args),
    )

Pretty straightforward - we combine both Ktlint’s executable path, the editorconfig file if it’s provided, and the list of source files.

Now for the rule interface, we will define a rule named ktlint_test. Building a ktlint_test target will mean generating a shell script to invoke Ktlint with the given set of argument, and running it will invoke that script - hence running Ktlint as well. In //tools/ktlint/internal/rules.bzl:

load(":actions.bzl", "ktlint")

def _ktlint_test_impl(ctx):
    script = ktlint(
        ctx,
        srcs = ctx.files.srcs,
        editorconfig = ctx.file.editorconfig,
    )

    ctx.actions.write(
        output = ctx.outputs.executable,
        content = script,
    )

    files = [ctx.executable._ktlint_tool] + ctx.files.srcs

    if ctx.file.editorconfig:
        files.append(ctx.file.editorconfig)

    return [
        DefaultInfo(
            runfiles = ctx.runfiles(
                files = files,
            ).merge(ctx.attr._ktlint_tool[DefaultInfo].default_runfiles),
            executable = ctx.outputs.executable,
        ),
    ]

ktlint_test = rule(
    _ktlint_test_impl,
    attrs = {
        "srcs": attr.label_list(
            allow_files = [".kt", ".kts"],
            doc = "Source files to lint",
            mandatory = True,
            allow_empty = False,
        ),
        "editorconfig": attr.label(
            doc = "Editor config file to use",
            mandatory = False,
            allow_single_file = True,
        ),
        "_ktlint_tool": attr.label(
            default = "@com_github_pinterest_ktlint//file",
            executable = True,
            cfg = "target",
        ),
    },
    doc = "Lint Kotlin files, and fail if the linter raises errors.",
    test = True,
)

We have two different parts here - the definition of the interface, with the call to rule, and the implementation of that rule, defined as _ktlint_test_impl.

The call to rule define how this rule can be invoked. We define that it requires a list of .kt and/or .kts files named srcs, an optional file named editorconfig, as well as a hidden argument named _ktlint_tool, which is just a helper for us to reference the Ktlint binary - to which we pass the file we defined in the WORKSPACE earlier.

The actual implementation is working in multiple steps:

1. It invokes the ktlint action we defined earlier, to generate the script that will be invoked.
2. It generates an action to write that script, in a file referred as ctx.outputs.executable (which Bazel knows how to handle and what to do with it, we don’t need to worry about where it is or anything, it won’t be in the source tree anyway).
3. It computes a list of files that are needed to run this target. This is what allows Bazel to ensure hermeticity - it will know that this rule needs to be re-run if any of those files are changed. If the target runs in a sandboxed environment (which is the default on most platforms, as far as I’m aware), only those files will be available.
4. It returns a Provider, responsible for holding a description of what this target needs.

Finally, we write a file that only exposes the bits users should care about. It’s not mandatory, but makes a clear delimitation between what is an implementation detail and what users can actually rely on. In //tools/ktlint/defs.bzl:

load(
    "//tools/ktlint/internal:rules.bzl",
    _ktlint_test = "ktlint_test",
)

ktlint_test = _ktlint_test

We just expose the rule we wrote in rules.bzl as ktlint_test.

Once this is done, we can use this ktlint_test rule where we needed it, in our _common_tests macro for Kotlin targets:

def _common_tests(name, srcs):
    ktlint_test(
        name = "%s_ktlint_test" % name,
        srcs = srcs,
        editorconfig = "//:.editorconfig",
    )

    detekt(
        name = "%s_detekt_report" % name,
        srcs = srcs,
        config = "//java/rules/internal:detekt-config.yml",
    )

    build_test(
        name = "%s_detekt_test" % name,
        targets = [":%s_detekt_report" % name],
    )

And there we have it - all our Kotlin targets have both Detekt and Ktlint test targets. Because we’re exposing those as Bazel targets, we automatically benefit from its caching and remote execution capabilities - those linters won’t re-run if the inputs didn’t change, and can run remotely, with Bazel being aware of which files are needed on the remote machine.

Closing thoughts

But what’s the link between generating a blog with Bazel and compiling a Kotlin application? Well, almost none, but there is one. The class diagram included earlier in this article is generated with a tool called PlantUML, which generates images from a text representation of a graph. The next article in this series will talk about integrating this tool into Bazel (in a similar way as we did with Ktlint), but also how to test the Bazel rule. And to have some integration tests, Phosphorus will come in handy!