Designing and Testing a CLI tool with BATS

Command line tool is a very common category of software. They are usually lightweighted, though there are certainly many extremely powerful command line tools, such as git and homebrew.

This tutorial will walk you through the typical design process of a CLI tool, and how to write integration tests with BATS.

“Time to find your Axis”

So what are we going to build?

For this tutorial, we are going to build a time management tool called Axis. You can access the source code for this tutorial here.

The idea of Axis is simple: listing, managing, and prioritizing your daily tasks. We are all familiar with the use of To-Do lists, where we list out the tasks we plan to do and crossing them off one by one. In real life, however, we often need to multitask and prioritize:

• People might have multiple ongoing projects they have to tend to. Mixing all tasks in one list makes the list poorly organized in that case.
• People often want to prioritize their work. More importantly, the priority of tasks may change over time.

Axis is designed to be a flexible time management tool that supports concurrent timelines and tasks reproritizing. More specifically, the primary use cases of Axis include:

• Creating and deleting tasks in a specific timeline
• Listing tasks in a specific timeline ordered by priority
• Updating the details or priority of a certain task

CLI Design

Before we delve into coding, making a thorough and comprehensive interface design is always a good way to lay a solid foundation for your project. A very helpful command line interface design guide can be found here. Just to highlight a few basic concepts of command line interface design:

• Arguments, or args, are positional parameters to a command.
• Flags are named parameters, denoted with either a hyphen and a single-letter name (-r) or a double hyphen and a multiple-letter name (–recursive).

The interface of Axis-CLI can be grouped into two subcommands: axis universe and axis item.

axis universe

$ axis universe create
 usage: axis universe create [options] universe
	options:
		-d, —description text

$ axis universe list
 usage: axis universe list [options]
	options:

$ axis universe show
 usage: axis universe show  [options] name
	options:

$ axis universe delete
 usage: axis universe delete [options] name
	options:
		-f, —force

$ axis universe update
usage: axis universe update [options] name
	options:
		—name text
		-d, —description	text

axis item

$ axis item create
usage: axis item create [options] title
	options:
		-p, —priority value		default: 1
		-u, —universe name	(required)
		—due timestamp
		-d, —description	text

$ axis item list
 usage: axis item list [options]
	options:
		-n, —limit N	default: 20	max: 100
		—skip M	
		-f, —filter json
		-p, —priority value	
		-u, —universe name
		-s, —status string

$ axis item search
 usage: axis item search [options] regex
	options:
		-n, —limit N	default: 20	max: 100
		—skip M
		-d, —search-description	default: False

$ axis item show
 usage: axis item show  [options] id
	options:

$ axis item delete
 usage: axis item delete [options] id
	options:

$ axis item update
usage: axis item update [options] id
	options:
		-t, —title text
		-p, —priority value
		—due timestamp
		-d, —description text
		-s, —status string

$ axis item move
usage: axis item move [options] id
	options:
		-u, —universe name

Data Design

Collection	Fields
AxisItem	id title description timeCreated timeDue priority universe status
AxisUniverse	name description

“Who are you? I’m BATS!”

Now that we have the design of our command line interface, what’s a better way to express those specifications than writing automated tests for each interface? Test-driven Development principles help us gain confidence in the correctness of our code and make the development process incremental.

BATS (Bash Automated Testing System) is an automated testing framework based on bash scripts. BATS is highly declarative, platform- and language-independent, and it comes with powerful features including setup, cleanup, and filtering. For our command line tool, BATS is the perfect language-independent testing framework.

Installation

In the project root directory, run the following to install BATS:

$ git submodule add https://github.com/bats-core/bats-core.git test/bats
$ git submodule add https://github.com/bats-core/bats-support.git test/test_helper/bats-support
$ git submodule add https://github.com/bats-core/bats-assert.git test/test_helper/bats-assert

Setup Actions

In the BATS world, test cases are organized into test suites. Each test script is typically an independent test suite.

BATS supports both suite-wide and per-case setup actions. For our project, we will be using both.

In test/setup_suite.bash, we define a setup_suite() function to define some environment variables that will come handy when we write tests. This function will be called before the execution of each test suite.

Source code of this tutorial can be found at https://github.com/axis-project/axis-cli/tree/main/test

setup_suite() {
    DIR="$( cd "$( dirname "$AXIS_BINARY_PATH" )" >/dev/null 2>&1 && pwd )"
    PATH="$DIR:$PATH"
    AXIS_BINARY_NAME="axis-cli"
    export AXIS_HOME="$BATS_TEST_DIRNAME/.tmp"
    export AXIS_COMMAND=$AXIS_BINARY_NAME
}

In each test script, we define another setup() function to define the preperation actions to be executed before each test case. Here, we will cleanup the DB file generated by Axis-CLI and load some helper functions that come with BATS.

Source code of this tutorial can be found at https://github.com/axis-project/axis-cli/tree/main/test

setup() {
    rm $AXIS_HOME/*.db || true
    load 'test_helper/bats-support/load'
    load 'test_helper/bats-assert/load'
}

Writing BATS Tests

BATS is highly declarative. To declare a test case, simply write a code block starting with a @test annotation, followed by the name of the test case and the test body.

Source code of this tutorial can be found at https://github.com/axis-project/axis-cli/tree/main/test

@test "axis universe" {
    run $AXIS_COMMAND universe
}

To write more complex test cases, use assert_success, assert_failure, assert_output, and refute_output to verify the return code and output of our command line tool.

Source code of this tutorial can be found at https://github.com/axis-project/axis-cli/tree/main/test

@test "axis universe create duplicate" {
    run $AXIS_COMMAND universe create "Main Universe"
    assert_success
    run $AXIS_COMMAND universe create "Main Universe"
    assert_failure
}

@test "axis universe show" {
    run $AXIS_COMMAND universe create "Main Universe"
    assert_success
    run $AXIS_COMMAND universe create "Alternate Universe"
    assert_success
    run $AXIS_COMMAND universe show "Main Universe"
    assert_success
    assert_output --partial "Main Universe"
    refute_output --partial "Alternate Universe"
}

For more advanced use of BATS, refer to the official doc of BATS.

Running Tests

To run tests with a specific implementation of Axis-CLI, simply invoke the BATS binary with intended test suites.

env AXIS_BINARY_PATH=$(pwd)/axis-cli $(pwd)/../../../test/bats/bin/bats $(pwd)/../../../test/*.bats

See the golang implementation for more details.

Summary

In this tutorial, we discussed the product idea of Axis – a command-line time management tool. We practiced some of the principles for command line interface design by designing the interface of Axis-CLI. We also learned how to write integration tests for command line tools using BATS. In future tutorials, we will explore how we can write command line tools with different languages and frameworks.

References

• Command Line Interface Guidelines: https://clig.dev
• Bats-core: https://bats-core.readthedocs.io/en/stable/
• Axis-CLI: https://github.com/axis-project/axis-cli