Generating Command Line Interfaces in Python

I like to use a Python for simple scripting and glue code. Usually, a significant part of the effort comes down to defining an argparse command line parser. Now, argparse is nice and all, but for a simple and personal-use-only script, it is quite a lot of effort. Of course, there are many libraries that improve upon argparse, or libraries for turning Python functions (dicts, classes, et. al) directly into command line interfaces. One such library is Fire.

This sounds great, but would introduce an external dependency deeply into my personal scripts, which I don’t like. So I wondered whether there was some easy way of getting a good enough approximation of Fire that I could just put into my personal utility library and be done with it. I came up with the following.

We will be working with an example script called example.py. I decided to stick to making functions callable from the command line, where the main function would be called by default (e.g., when running ./example.py) and any other function in the module could be called in a submode-like fashion, e.g., ./example.py doit would call a function def doit() in example.py. Anything more complex than that deserves argparse.

We can quite easily achieve this by passing globals() to the following function:

import sys

def cli(gb):
    function = sys.argv[1] if len(argv) > 1 else 'main'
    gb[function]()

The function cli() is assumed to be defined in some module pscript. We can use it as follows:

import psutil

def main():
    print("hello, world")

if __name__ == '__main__':
   pscript.cli(globals())

And, when we run the script:

$ ./example.py
hello, world

Cool, hello, world with extra steps. Next, we need to pass some arguments from the command line to the function we intend to call. Here, I decided to again keep it simple: Both positional arguments and keyword arguments use Python syntax, but are separated by whitespace (instead of commata). Thus, we need to first differentiate the two and then split keyword arguments on the equals sign. Finally, we pass the arguments to the function we looked up in the globals.

import sys

def partition(predicate, iterable):
    """Partition iterable into two lists, using predicate."""
    ...

def cli(gb):
    # Partition all arguments: Keyword arguments contain "=", positionals do not.
    kwargs_raw, posargs_raw = partition(lambda a: "=" in a, sys.argv[1:])

    # Process keyword arguments, generating a dictionary.
    kwargs = {
        str(k): v for k, v in (kw.split('=') for kw in kwargs_raw)
    }

    # Get the top level function to call or 'main' as a default.
    if len(posargs_raw) > 0 and posargs_raw[0] in gb:
        function = posargs_raw[0]
        posargs_raw = posargs_raw[1:]
    elif "main" in gb:
        function = "main"
    else:
        print("No function to call.")
        sys.exit(0)

    # Finally, call 'function' with positional and keyword arguments.
    try:
        gb[function](*posargs, **kwargs)
    except TypeError as e:
        # On TypeError (wrong number of arguments), print help and args.
        print(gb[function].__doc__)
        print(e)

Again, great! We can now pass positional and keyword arguments to any function in a module! Also, instead of hoping for the best, we even catch TypeError: If the wrong number of arguments is supplied on the command line, a helpful message about what arguments are missing, as well as the functions docstring are printed. This is almost, approximately like a decent CLI would behave.

Let’s try this out. First, we define:

import psutil

def main(a, b):
    print(a + b)

if __name__ == '__main__':
   pscript.cli(globals())

and then we call

$ ./example.py 40 2
402

Oh. Yeah, nah, that won’t do. Sure, we can work with and expect strings here; but would it not be much nicer to get actual Python values? Lukily, there is a module ast in the standard library that has a function literal_eval that we can use to do just that for some value:

import ast

try:
    return ast.literal_eval(value)
except Exception:
    return str(value)

This gives us either a Python value (e.g., 42 for "42" or [1, 2, 3] for "[1, 2, 3]"), or just a string, if parsing fails. Note, that literal_eval only returns strings for "\"this\"" but we also want strings for "this", so we return the value in case parsing fails. Applying this function to all arguments in cli(), we get:

$ ./example.py 40 2
42

In the final version, we’ll add a flag so one can decide to enable or diable this parsing of strings behaviour. While we’re at it, we’ll also add a flag to go even further and try to load data from files, if the string argument is indeed a valid, existing path and has one of a few given extensions; otherwise, we return the Path.

import os

path = os.path.join(os.getcwd(), value)
if load_files and os.path.exists(path):
    if path.endswith(".json"):
        import json
        with open(path, encoding='utf8') as f:
            return json.load(f)
    elif path.endswith(".csv"):
        import csv
        with open(path, encoding='utf8') as f:
            return csv.reader(f)
    elif path.endswith(".txt"):
        with open(path, encoding='utf8') as f:
            return [line.rstrip() for line in f]
    else:
        return path

Overall, this scraps loads of boilerplate from Python scripts, such as argparse parsers, loading of files, and conversion of strings to some useful value, by making Python functions directly callable from the command line. Of course, this is not suitable for any actual user-facing applications: The syntax deviates from common CLI standards, help messages are raw docstrings, malicious use, or miss-use, are certainly on the table. But for personal scripts, I think this is great.

Here is the full (current) source of the cli function, which also adds a special --help flag to print docstrings.