k2script - Having fun with Kotlin scripting

Enhanced scripting support for Kotlin on *nix-based and Windows systems.

Kotlin has some built-in support for scripting already, but it is not yet feature-rich enough to be a viable alternative in the shell.

this is a great kotlin feature that has been updated for kotlin2 and the scripting api and has to be a little more clever and bring options to mitigate hosting and vm challanges and kotlin adveristies.

In particular this wrapper around kotlinc adds

• Compiled script caching (using md5 checksums)

• Dependency declarations using gradle-style resource locators and automatic dependency resolution

• More options to provide scripts including interpreter mode, reading from stdin, local files or URLs

• Embedded configuration for Kotlin runtime options

• Support library to ease the writing of Kotlin scriptlets

• Deploy scripts as stand-alone binaries

Taken all these features together, kscript provides an easy-to-use, very flexible, and almost zero-overhead solution to write self-contained mini-applications with Kotlin.

Kotlin’s native scripting capabilities have continually improved, providing a solid foundation. kscript builds upon this by offering a rich set of features to enhance the scripting experience, including simplified dependency management, compiled script caching, flexible script input modes, and deployment options. While Kotlin’s own scripting support is powerful, kscript aims to provide additional conveniences and power tools for scripters.

---

• Installation

• Script Input Modes

• Script Configuration

• Text Processing Mode

• Treat yourself a REPL with --interactive

• Boostrap IDEA from a scriptlet

• Deploy scripts as standalone binaries

• Embed kscript installer within your script

• kscript configuration file

• FAQ

• Support

• How to contribute?

• Acknowledgements

Installation

To use kscript just Kotlin is required. To install Kotlin we recommend sdkman:

curl -s "https://get.sdkman.io" | bash     # install sdkman
source "$HOME/.sdkman/bin/sdkman-init.sh"  # add sdkman to PATH

sdk install kotlin                         # install Kotlin

Once Kotlin is ready, you can install kscript with

sdk install kscript

Package managers like SDKMAN, Homebrew, and Scoop provide convenient ways to install kscript. Note that the version they provide might not always be the absolute latest. For the most recent updates or to use kscript with very new Kotlin/Java versions, consider the Build it yourself section.

To test your installation simply run

kscript --help

This will check and inform about updates. To update kscript simply install it again as described above.

Run with docker

We provide an executable docker container to run kscript

# using the latest version of kscript
docker run -i kscripting/kscript 'println("Hello, world!")'

# or using versioned container
docker run -i kscripting/kscript:4.2.0 'println("Hello, world!")'

To use a script file outside the container as input, you could do

docker run -i kscripting/kscript - < script.kts

This will make kscript read the code from stdin while piping the file. Beware that the -i flag is needed to have stdout redirected outside the container.

Please note, that currently @Import are not supported when using a dockerized kscript. Also, any resource outside the container context may not be resolved correctly. To overcome this limitation, you could use for instance bind mounts.

Installation without sdkman

If you have Kotlin already, and you would like to install the latest kscript release without using sdkman you can do so by unzipping the latest binary release. Don’t forget to update your $PATH accordingly.

Installation with Homebrew

On MacOS you can install kscript also with Homebrew

brew install kscripting/tap/kscript

To upgrade to latest version

brew update
brew upgrade kscripting/tap/kscript

Installation on Arch Linux

On Arch Linux, kscript is available through the Arch Linux User repository (AUR). Use your favorite AUR helper to install, e.g. yay:

yay -S kscript

There is an uncommon directory layout of Kotlin package for Arch Linux, which causes problems when using kscript with default Kotlin package. Two workarounds for ArchLinux exists, which can be used to make 'kscript' working with ArchLinux:

1. Manually create symlinks in the system…

   sudo mkdir /usr/share/kotlin/bin
   sudo ln -s /usr/bin/kotlin /usr/share/kotlin/bin/kotlin
   sudo ln -s /usr/bin/kotlinc /usr/share/kotlin/bin/kotlinc

2. …or install Kotlin using SdkMan:

   Installation using SdkMan

The problem should be fixed in the Kotlin package for ArchLinux. See more in the Github issue:
kscripting#371

Installation on Windows

On Windows, kscript is available through the Scoop Extras bucket. Use the following commands to install:

scoop bucket add extras
scoop install kscript

To install scoop use the official guide.

Build it yourself

To build kscript yourself, which can be useful for accessing the very latest features or using it with specific modern Kotlin/Java versions (like Kotlin 2.2.0+ and Java 21+ which have been tested):

Ensure you have a modern JDK installed (e.g., JDK 17 or newer, JDK 21 recommended for recent Kotlin versions). The build uses the Gradle wrapper (gradlew), which will download the appropriate Gradle version.

Clone the repository and then run:

./gradlew assemble

## Run kscript from output dir
./build/kscript/bin/kscript

Script Input Modes

The main mode of operation is kscript <script>.

The <script> can be a Kotlin .kts script file , a script URL, - for stdin, a process substitution file handle, a .kt source file with a main method, or some kotlin code.

Interpreter Usage

To use kscript as interpreter for a script just point to it in the shebang line of your Kotlin scripts:

#!/usr/bin/env kscript

println("Hello from Kotlin!")
for (arg in args) {
    println("arg: $arg")
}

Inlined Usage

To use kscript in a workflow without creating an additional script file, you can also use one of its supported modes for inlined usage. The following modes are supported:

• Directly provide a Kotlin scriptlet as argument

kscript 'println("hello world")'

• Pipe a Kotlin snippet into kscript and instruct it to read from stdin by using - as script argument

echo 'println("Hello Kotlin.")' |  kscript -

• Using heredoc (preferred solution for inlining) which gives you some more flexibility to also use single quotes in your script:

kscript - <<"EOF"
println("It's a beautiful day!")
EOF

• Since the piped content is considered as a regular script it can also have dependencies

kscript - <<"EOF"
@file:DependsOn("com.offbytwo:docopt:0.6.0.20150202", "log4j:log4j:1.2.14")

import org.docopt.Docopt
val docopt = Docopt("Usage: jl <command> [options] [<joblist_file>]")

println("hello again")
EOF

• Finally, (for sake of completeness), it also works with process substitution and for sure you can always provide additional arguments (exposed as args : Array<String> within the script)

kscript <(echo 'println("k-onliner")') arg1 arg2 arg3

Inlined kscripts are also cached based on md5 checksum, so running the same snippet again will use a cached jar ( sitting in ~/.kscript).

URL usage

To support remote scriplet repositories, kscript can also work with URLs. Consider the following hello-world-gist-scriptlet which is hosted on github (but any URL would work). To run it locally as a tool simply refer to it (here using the shortened raw-URL of the script for better readability)

kscript https://git.io/v1cG6 my argu ments

To streamline the usage, the first part could be even aliased:

alias hello_kscript="kscript https://git.io/v1cG6"
hello_kscript my argu ments

Via this mechanism, kscript allows for easy integration of remotely hosted (mini) programs into data workflows.

URL-scripts are cached locally to speed up processing, and kscript --clear-cache can be used to wipe the cache if needed.

See this blogpost for a more extensive overview about URL support in kscript.

Script Configuration

The following directives supported by kscript to configure scripts:

• @file:DependsOn to declare dependencies with gradle-style locators

• @file:Import to source kotlin files into the script

• @file:EntryPoint to declare the application entrypoint for kotlin *.kt applications

• @file:CompilerOptions to configure the compilation options

• @file:KotlinOptions to configure the kotlin/java runtime environment

Declare dependencies with @file:DependsOn

To specify dependencies simply use gradle-style locators. Here’s an example using docopt and log4j

#!/usr/bin/env kscript
@file:DependsOn("com.offbytwo:docopt:0.6.0.20150202", "log4j:log4j:1.2.14")

import org.docopt.Docopt
import java.util.*

val usage = """
Use this cool tool to do cool stuff
Usage: cooltool.kts [options] <igenome> <fastq_files>...

Options:
 --gtf <gtfFile>     Custom gtf file instead of igenome bundled copy
 --pc-only           Use protein coding genes only for mapping and quantification
"""

val doArgs = Docopt(usage).parse(args.toList())

println("Hello from Kotlin!")
println("Parsed script arguments are: \n$doArgs")

kscript will read dependencies from all lines in a script that start with @file:DependsOn (if any). Multiple dependencies can be split by comma, space or semicolon.

Configure the runtime with @file:KotlinOptions

kscript allows to provide a @file:KotlinOptions directive followed by parameters passed on to kotlin similar to how dependencies are defined:

#!/usr/bin/env kscript
@file:KotlinOptions("-J-Xmx5g", "-J-server")

println("Hello from Kotlin with 5g of heap memory running in server mode!")

Note: Similar to the runtime you can also tweak the compile step by providing @file:CompilerOptions.

Ease prototyping with @file:Import

kscript supports an @file:Import directive to directly include other source files without prior compilation. Absolute and relative paths, as well as URLs are supported. Example:

//utils.kt
fun Array<Double>.median(): Double {
    val (lower, upper) = sorted().let { take(size / 2) to takeLast(size / 2) }
    return if (size % 2 == 0) (lower.last() + upper.first()) / 2.0 else upper.first()
}

Which can be now used using the @file:Import directive with

#!/usr/bin/env kscript

@file:Import("utils.kt")

val robustMean = listOf(1.3, 42.3, 7.0).median()
println(robustMean)

The argument can be an URL, absolute or relative file path. Note that URLs used in include directives are cached locally to speed up processing, that is kscript won’t fetch URLs again unless the user actively clears the cache with kscript --clear-cache.

For more examples see here.

Use @file:EntryPoint to run applications with main method

kscript also supports running regular Kotlin kt files.

Example: ./examples/Foo.kt:

package examples

@file:EntryPoint("examples.Bar")

class Bar {
    companion object {
        @JvmStatic
        fun main(args: Array<String>) {
            println("Foo was called")
        }
    }
}

fun main(args: Array<String>) = println("main was called")

To run top-level main instead we would use @file:EntryPoint("examples.FooKt")

The latter is the default for kt files and could be omitted

Examples of annotation driven configuration

#!/usr/bin/env kscript

// Declare dependencies
@file:DependsOn("com.github.holgerbrandl:kutils:0.12")
@file:DependsOn("com.beust:klaxon:0.24", "com.github.kittinunf.fuel:fuel:2.3.1")

// To use a custom maven repository you can declare it with
@file:Repository("http://maven.imagej.net/content/repositories/releases")

// For compatibility with https://github.com/ligee/kotlin-jupyter kscript supports also
@file:DependsOnMaven("net.clearvolume:cleargl:2.0.1")
// Note that for compatibility reasons, only one locator argument is allowed for @DependsOnMaven

// also protected artifact repositories are supported, see <https://github.com/kscripting/kscript/blob/master/test/TestsReadme.md#manual-testing>
// @file:Repository("my-art", "http://localhost:8081/artifactory/authenticated_repo", user="auth_user", password="password")
// You can use environment variables for user and password when string surrounded by double {} brackets
// @file:Repository("my-art", "http://localhost:8081/artifactory/authenticated_repo", user="{{ARTIFACTORY_USER}}", password="{{ARTIFACTORY_PASSWORD}}")
// will be use 'ARTIFACTORY_USER' and 'ARTIFACTORY_PASSWORD' environment variables
// if the value doesn't found in the script environment  will fail

// Include helper scripts without deployment or prior compilation
@file:Import("util.kt")

// Define kotlin options
@file:KotlinOptions("-J-Xmx5g")
@file:KotlinOptions("-J-server")
@file:CompilerOptions("-jvm-target 1.8")

// declare application entry point (applies on for kt-files)
@file:EntryPoint("Foo.bar")

print("1+1")

To enable the use of these annotations in Intellij, the user must add the following artifact to the project dependencies:

io.github.kscripting:kscript-annotations:1.5

kscript will automatically detect an annotation-driven script, and if so will declare a dependency on io.github.kscripting:kscript-annotations (historically version 1.5) internally.

Note, that if a script is located in a package other than the root package, you need to import the annotations with ( e.g. import DependsOn).

Text Processing Mode

kscript can be used as a speedier and more flexible substitute for built-in terminal text tools such as awk or sed . Its text processing mode can be enabled with -t or --text. If so, kscript will

• Declare com.github.kscripting:kscript-support-api:1.2.5 as dependency for the script. This support library eases the writing of Kotlin scriptlets for text-processing. It includes solutions to common use-cases like argument parsing, data streaming, IO utilities, and various iterators to streamline the writing of scriptlets for text processing.

• Import the kscript.* namespace

• Define variable val lines = kscript.text.resolveArgFile(args) which returns an iterator over the lines in the first input argument of the script, or the standard input if no file arguments are provided to the script

This allows to replace awk`ward constructs (or `sed or`perl`) with kotlinesque solutions such as

cat some_file | kscript -t 'lines
    .filter { "^de0[-0]*".toRegex().matches(it) }
    .map { it + "foo:" }
    .print()
'

In this example, the extension method Iterable<String>.print() to print the lines to stdout comes from the support API. The rest is stdlib Kotlin.

For more examples using the support library see this blog post.

Treat yourself a REPL with --interactive

To create an interactive kotlin shell ( aka REPL) with all script dependencies added to the classpath you can use --interactive.

For example, let’s assume the following short script, named CountRecords.kts

#!/usr/bin/env kscript
@file:DependsOn("com.github.holgerbrandl:kutils:0.12")

import de.mpicbg.scicomp.bioinfo.openFasta

if (args.size != 1) {
    System.err.println("Usage: CountRecords <fasta>")
    kotlin.system.exitProcess(-1)
}

val records = openFasta(java.io.File(args[0]))
println(records.count())

To build a REPL that has the declared artifact in its classpath, we can just do

kscript --interactive CountRecords.kts

which will bring up the classpath-enhanced REPL:

Creating REPL from CountRecords.kts
Welcome to Kotlin version 1.1.51 (JRE 1.8.0_151-b12)
>>> import de.mpicbg.scicomp.bioinfo.openFasta
>>>

Boostrap IDEA from a scriptlet

Artifacts and versions will differ between scripts, so it is hard to maintain them all in a single project. To nevertheless provide optimal tooling when scripting with Kotlin kscript allows to create temporary projects for <script> arguments.

kscript --idea CountRecords.kts

If you have available gradle in the path project will be automatically built and if there is idea in the path the project will be opened in IntelliJ IDEA with a minimalistic project containing just your (1) <script> and (2) a generated build.gradle.kts file:

The idea command line launcher can can be created in IntelliJ with Create Command-line Launcher command, or you can set the command used to launch your IntelliJ as KSCRIPT_COMMAND_IDEA env property. Similarly, you can set gradle command using KSCRIPT_COMMAND_GRADLE env property.

Deploy scripts as standalone binaries

To deploy a script simply do

kscript --package some_script.kts
./some_script --arg u ments

The created binary will contain a compiled copy of the script, as well as all declared dependencies (fatjar). Also runtime jvm parameters declared via @file:KotlinOptions are used to spin up the JVM.

Just java is required to run these binaries.

Embed kscript installer within your script

To make a script automatically install kscript and its dependencies on first run if necessary, run:

kscript --add-bootstrap-header some_script.kts

Now some_script.kts can be shared and run directly on any other machine that has bash, without having to go through the Installation steps first.

Note that unlike the --package option this doesn’t produce a separate file, allowing the distributed script to be read and modified(including with kscript --idea) similar to what you might expect with bash/python/ruby scripts. On the other hand this doesn’t embed dependencies within the script("fat jar"), so internet connection may be required on its first run.

Python and NPM Packaging

Starting with version 4.2.3, the main kscript binary distribution ZIP file (e.g., kscript-4.2.3-bin.zip) now includes helper files to allow users to easily build and install kscript as a Python package or an NPM package. This provides a convenient way to integrate kscript into Python or Node.js project environments and makes kscript available as a command-line tool through pip or npm.

The necessary files (setup.py for Python, package.json for Node.js, and various wrapper scripts) are located in the extracted distribution archive. When you extract the main kscript zip, these files will be in the root directory, and the wrappers along with kscript.jar will be in the wrappers/ subdirectory.

Python (pip)

To build and install kscript as a Python package:

1. Download and extract the kscript-4.2.3-bin.zip (or the appropriate version) distribution.

2. Navigate to the root of the extracted directory in your terminal.

3. The setup.py script expects kscript.jar to be in the wrappers/ subdirectory, where it should be placed automatically by the build process.

4. Build the wheel package:

   python setup.py bdist_wheel

   Alternatively, you can create a source distribution:

   python setup.py sdist

5. Install the generated package (the exact filename will depend on the version and build tags):

   pip install dist/kscript-*.whl

6. After installation, kscript should be available as a command-line tool, using the Python wrapper to execute kscript.jar.

Node.js (npm)

To build and install kscript as an NPM package:

1. Download and extract the kscript-4.2.3-bin.zip (or the appropriate version) distribution.

2. Navigate to the root of the extracted directory in your terminal.

3. The package.json file expects kscript.jar to be in the wrappers/ subdirectory, where it should be by default.

4. Create the NPM package:

   npm pack

   This will create a kscript-4.2.3.tgz file (the version comes from package.json).

5. Install the package. For global installation:

   npm install -g kscript-4.2.3.tgz

   Or, to install it as a project dependency, navigate to your project directory and run (adjust path as necessary):

   npm install /path/to/extracted_kscript_dist/kscript-4.2.3.tgz

6. After installation (globally, or locally if node_modules/.bin is in your PATH), kscript should be available as a command-line tool, using the Node.js wrapper.

Direct Wrapper Usage

Advanced users can also utilize the wrapper scripts directly if they prefer to manage their environment and kscript.jar location manually: * Python wrapper: wrappers/kscript_py_wrapper.py * Node.js wrapper: wrappers/kscript_js_wrapper.js (make it executable or run with node)

These wrappers expect kscript.jar to be in the same directory (wrappers/) by default. This approach requires java to be available in the system PATH.

kscript configuration file

To keep some options stored permanently in configuration you can create kscript configuration file.

KScript follows XDG directory standard, so the file should be created in (paths are resolved in provided order; first existing path is used):

OS	PATH
Windows	%LOCALAPPDATA%\kscript\kscript.properties; %USERPROFILE%.config\kscript\kscript.properties
MacOs	~/Library/Application Support/kscript/kscript.properties;
Posix	\${XDG_CONFIG_DIR}/kscript/kscript.properties; \${user.home}/.config/kscript/kscript.properties

If the environment variable: KSCRIPT_DIRECTORY is defined the configuration file 'kscript.properties' will be placed directly inside it.

Content of kscript.properties file is a standard Java format, with following properties available:

scripting.preamble=
scripting.kotlin.opts=
scripting.repository.url=
scripting.repository.user=
scripting.repository.password=
scripting.directory.artifacts=

Example configuration file:

scripting.preamble=// declare dependencies\n\
@file:DependsOn("com.github.holgerbrandl:kutils:0.12")\n\
\n\
// make sure to also support includes in here\n\
// @file:Import("util.kt")\n\
@file:Import("https://raw.githubusercontent.com/kscripting/kscript/master/test/resources/custom_dsl/test_dsl_include.kt")\n\
\n\
\n\
// define some important variables to be used throughout the dsl\n\
val foo = "bar"

scripting.kotlin.opts=-J-Xmx4g
scripting.repository.url=https://repository.example
scripting.repository.user=user
scripting.repository.password=password

FAQ

How to edit kscript in VS Code?

See https://magnusgunnarsson.se/offentlig/kscript-in-visual-studio-code-vsc/ for a walkthrough and the required editor configuration.

Why is kscript not calling the main method in my .kts script?

There is no need for a main method in a Kotlin script. Kotlin .kts scripts can be more simplistic compared to more common kotlin .kt source files. The former work without a main method by directly running the provided code from top to bottom. E.g.

print("hello kotlin!")

is a valid Kotlin kts script. Plain and simple, no main, no companion, just a few bits of code.

Does kscript also work for regular kotlin .kt source files with a main as entry point?

Yes, (since kscript v1.6) you can run kotlin source files through kscript. By default, it will assume a top-level main method as entry-point.

However, in case you’re using a companion object to declare the entry point, you need to indicate this via the @file:Entry.

What are performance and resource usage difference between scripting with kotlin and python?

Kotlin scripts involve a JVM startup and, for the first run of a script, a compilation step. While the JVM offers excellent peak performance for longer-running or complex tasks, the initial overhead might be noticeable for very short-lived, simple scripts when compared to languages like Python that have minimal startup time. The best choice often depends on the specific use case, script complexity, access to Java/Kotlin libraries, and developer familiarity with the ecosystems.

Does kscript work with java?

The only language supported by kscript is kotlin. For a similar approach centering around Java scripting see jbang.

Can I use custom artifact repositories?

Yes, via the @Repository annotation. See annotations section or custom_mvn_repo_annot for a complete example

Support

Feel welcome to post ideas and suggestions to our tracker.

More advanced use-cases are documented in the complementary user guide

How to contribute?

We always welcome pull requests and trouble tickets. :-)

Help to spread the word. Great community articles about kscript include

• Type-safety in the shell with kscript

• Using Kotlin kscript for Preprocessing Data

• kscript as substitute for awk

• A complete backend with Kotlin and your favourite text-editor -using kscript

You could also show your support by upvoting kscript here on github, or by voting for issues in Intellij IDEA which impact `kscript`ing. For specific kscript issues or feature proposals, please use the kscript GitHub issue tracker. For broader Kotlin language or IDE-related issues, the official JetBrains YouTrack is the appropriate place.

Acknowledgements

The initial version of kscript was kindly contributed by Oscar Gonzalez.

Special thanks to Ilan Pillemer, Andrey Mischenko , Stephen Byrne, Eugene Susla , Eli Hart, Hwijae Lee and oshai for contributing PRs to this repo.

Thanks also to the Scionics Computer Innovation GmbH and the MPI-CBG for supporting this project.

Version 3.1 to 4.0 rewrite and current maintenance by Marcin Kuszczak

kscript was inspired by kotlin-script which is another great way (now deprecated) to do scripting in Kotlin.