OpenResty OpsLang™ User Manual

The Ops Language User Manual

Table of Contents

Description

This manual documents the Ops language from the user’s perspective.

Some of the features documented in this manual may not have been implemented in the current Ops language implementation. The Ops language development team is quickly catching up though. We try marking all not-yet-implemented features in the documentation. When in doubt, please contact the OpenResty Inc. company directly.

This document is still a draft. Many details are still subject to change.

Convention

We use the word opslang for the Ops language throughout this document for convenience.

Example code with problems will get a question mark, ?, at the beginning of each of its lines. For example:

? $ echo hello,
? stream {
? }

Program Layout

On the highest level, an Ops program consists of one main program file and an optional set of module files.

For each .ops file, it consists of various declaration statements of the following kinds:

  1. module declaration statements,
  2. goal and default goal declarations,
  3. top-level variable declarations,
  4. exception declarations, and
  5. user action declarations.

The first one must appear at the beginning of every Ops module file while non-module files must not have it.

The main Ops program file must contain at least one goal declaration.

Other things are optional.

To follow the old tradition in the programming languages’ world, a “hello world” program in the Ops language looks like this:

goal hello {
    run {
        say("Hello, world!");
    }
}

Assuming this program is in the file hello.ops, We can run it with a single command like this:

$ opslang --run hello.ops
Hello, world!

Or use opslang as a compiler to generate the Lua module file named hello.lua:

opslang -o hello.lua hello.ops

And then run the Lua module generated with the resty command-line utility provided by OpenResty:

$ resty -e 'require "hello".main(arg)'
Hello, world!

Back to TOC

Bits and Pieces

Identifiers

Identifiers in opslang is one or more words connected by dashes. A word is a sequence of alphanumeric characters and underscores. An underscore character cannot appear at the beginning of a word, however. Below are some examples of valid opslang identifiers:

foo
hisName
uri-prefix
Your_Name1234
a-b_1-c

The Ops language is a case-sensitive language. So identifiers like foo and Foo do mean completely different things.

Identifiers cannot be one of the language keywords, except for variable names.

Back to TOC

Keywords

The Ops language has the following keywords:

for     while      if      while     func      action
ge      gt         le      lt        eq        ne
contains           contains-word     suffix    prefix
my      our        macro   use       INIT      END
as      rx         wc      qw        phase     goal
exception          return  label     for       async
lua

Back to TOC

Qualified names

Qualified names can be used to referencing symbols in other Ops modules or referencing a particular action phasers in a goal.

For example, std.say is to explicitly reference the function say() in the std module or namespace. And all.run is referencing the run action phaser in the goal all.

It is also possible to combine these two cases. For instance, foo.all.run is the run action phaser of the goal all defined in the module foo.

Back to TOC

Variables

Variable names consist of two parts: a leading special character called a sigil and a following identifier. The sigil is used to denote the type of the variable. The following three different sigils are supported:

  • $ is for scalar variables
  • @ is for array variables
  • % is for hash variables

Scalar variables hold simple values like numbers, strings, booleans, and quantities.

Array variables are a ordered list container for simple values.

Hash variables are an unordered list for key-value pairs.

Variables are usually declared by the my keyword as follows:

my Int $count;
my Str $name;
my Str @domains;
my Bool %map{Str};

Variable declarations may take an initial value too, as in:

my Int $count = 0;
my Str @domains = qw/ foo.com bar.blah.org /;

Each scalar variable declared in any scopes can only have one data type throughout its lifetime. And each variable’s data type must be able to be determined at compile time. There are 4 data types for scalar variables:

  • Str
  • Num
  • Int
  • Bool

The Ops language does not support other data types in other contexts, however, like Exception and Nil.

The user must explicitly specify a data type for any scalar variables defined by my or defined as user action parameters.

For convenience, multiple variables of the same type can be declared by a single my statement, as in

my Str ($foo, $bar, $baz);

or even mixing scalar, array, and hash variables:

my Str ($foo, @bar, %baz{Int});

This form of variable declarations cannot take any initializer expressions, however.

Back to TOC

Array Variables

Array variables hold a linear list of simple values called an array.

Below is an example:

goal all {
    run {
        my Int @ints;

        push(@ints, 32),
        push(@ints, -3),
        push(@ints, 0),
        say("index 0: ", @ints[0]),
        say("index 2: ", @ints[2]),
        say("array: [@ints]");
    }
}

As we can see, the element type of arrays is specified between the my keyword and the @ints variable name in the declaration.

The push standard function appends new elements to the end of the array. Multiple elements can be appended at the same time in a single push() call.

Individual elements can be later accessed by indexes like @arr[index]. This notation can also be used to set a new value for a particular element, as in

@ints[2] = 100

The full array can also be interpolated in string literals just like scalar variables.

Running this int-array.ops example program yields

$ opslang --run int-array.ops
index 0: 32
index 2: 0
array: [32 -3 0]

There are other standard functions which operate on arrays:

  • the pop() function removes and returns the last element of an array;
  • the shift() function removes and returns the first element of an array;
  • the unshift function prepends one or more elements to the beginning of an array;
  • the elems function returns the length of an array.

When being used in boolean contexts, array variables are evaluated to true when they are non-empty, false otherwise. For instance:

goal all {
    run {
        my Num @nums;

        {
            @nums =>
                say("array is NOT empty"),
                done;

            true =>
                say("array is empty");
        };
    }
}

Running this array-cond.ops example gives

$ opslang --run array-cond.ops
array is empty

We can initialize an array variable with a literal array, as in

goal all {
    run {
        my Num @nums = (3.14, 0);

        {
            @nums =>
                say("array is NOT empty"),
                done;

            true =>
                say("array is empty");
        };
    }
}

Running this example now gives the output array is NOT empty since the array has 2 elements now.

Array variables can be iterated through via the for statement.

Back to TOC

Hash Variables

Hash variables hold dictionary values mapping from keys to simple values.

Below is a simple example:

goal all {
    run {
        my Num %ages{Str} = (dog: 1.5, cat: 5.4, bird: 0.3);

        say("bird: ", %ages<bird>),
        say("dog: ", %ages{'dog'}),
        say("cat: ", %ages{"cat"}),
        say("tiger: ", %ages<tiger>);
    }
}

Running this ages.ops example program gives

$ opslang --run ages.ops
bird: 0.3
dog: 1.5
cat: 5.4
tiger: nil

Here we first declare a hash variable named %ages (note the % sigil). Its key type is Str while its value type is Num. At the same time we initialize the hash variable with a [literal hash value)(#literal-hash-values) which specifies the age numbers for 3 animals, dog, cat, and bird. And then we output the values of this hash variable for various keys. Note that the tiger key does not exit in the hash value, so we get a nil value for it.

As we can see from this example, when the string key is a word, we could use the %hash<word> syntax as a shorthand for the more general forms %hash{'word'} and %hash{"word"}. In general, to read the value of a specified key, we can write %hash{key}; and to newly set a value or overwrite the existing value of a specified key, we can write %hash{key} = value.

One can also use scalar variables as the key, as in

my $key = 'dog';

say("$key: ", %ages{$key});

Keys of other data types are also supported, for instance,

goal all {
    run {
        my Str %weekdays{Int} = (1: "Monday", 2: "Tuesday", 3: "Wednesday",
                                 4: "Thursday", 5: "Friday");

        say(%weekdays{3}),
        say(%weekdays{5});
    }
}

Running this weekdays.ops example program gives

$ opslang --run weekdays.ops
Wednesday
Friday

When being used in boolean contexts, hash variables are evaluated to true when they are non-empty, false otherwise. For example:

goal all {
    run {
        my Int %ages{Str};

        {
            %ages =>
                say("hash NOT empty"),
                done;

            true =>
                say("hash empty");
        };
    }
}

Running this hash-empty.ops example program gives

$ opslang --run hash-empty.ops
hash empty

Hash variables cannot be interpolated in double-quoted strings or shell-quoted strings. The % character is always interpreted literally in those contexts.

Hash variables can be iterated through via the for statement.

Back to TOC

Capture Variables

Special capture variables are provided to hold sub-match captures of the previous regex matching (if any). For example, variable $1 holds the matched substring for the first numbered sub-match capture group, while $2 holds the second capture group, $3 for the 3rd, and etc. The special variable $0 holds the fully matched substring for the whole regex, regardless of the sub-match capturing groups.

Consider the following example:

goal all {
    run {
        my Str $s = "hello, world";

        {
            $s contains /(\w+), (\w+)/ =>
                say("0: [$0], 1: [$1], 2: [$2]"),
                done;

            true =>
                say("not matched!");
        };
    }
}

Running this Ops program yields the output

0: [hello, world], 1: [hello], 2: [world]

Back to TOC

Function Names and Function Calls

The language extensively use functions for predicates and actions in rules. The user can also define their own functions (as user actions) if they want to.

Standard functions and user-defined actions can be used in the exactly same way. In fact, many standard functions are indeed implemented as “user actions” in the std module shipped with the opslang compiler.

Function names are represented by identifiers directly, no sigils involved.

Function calls are denoted by a function name followed by a pair of parentheses enclosing any arguments, as in:

say("hello, ", "world")

Function calls without any arguments can omit the parentheses. For example:

redo()

can be simplified to:

redo

Arguments can be passed either by positions or by names. The say() builtin action function, for example, accepts positional arguments as the response body message pieces, as the previous example demonstrates. Named arguments are passed with the argument name and a colon character as the prefix, as in:

goto(my-label, tries: 3);

This goto() call takes a named argument called tries with the value 3.

Builtin functions may require certain arguments to be passed by names, and the others passed by positions. Please consult the documentation of specific builtin functions for the actual usage.

There are also some syntactic constructs which are equivalent to function calls. For example, shell quoted strings used directly as an action and dollar actions.

Back to TOC

Literal Strings

Single-Quoted Strings

Single-quoted strings are literal string values enclosed by single quotes (''), as in

'foo 1234'

The characters $ and @ are always their literal meaning. Scalar and array variables can never be interpolated.

Only the following escaping sequences are supported in single-quoted strings:

\'
\\

Any other appearances of the \ character in a single-quoted string literal will be interpreted as a literal \.

Back to TOC

Double-Quoted Strings

Double-quoted strings are literal string values enclosed by double quotes (""), as in

"hello, world!\n"

The following escaping sequences are supported in double-quoted strings:

\a
\b
\f
\n
\r
\t
\v
\\
\0
\$
\@
\%
\'
\"

Scalar and array variables can be interpolated into double-quoted string literals, as in:

"Hello, $name!"

"Names: @names"

If there are ambiguous characters coming after the interpolated variable names, then we can use curly braces to disambiguate, as in

"Hello, ${name}ism"

"Hello, @{names}ya"

For convenience, an alternative form of variable interpolation syntax (similar to the shell-quoted strings) is also supported:

"Hello, $.name!"

"Hello, @.names!"

Or with the curly braces form:

"Hello, $.{name}!"

"Hello, @.{names}!"

So literal $ and ‘@’ characters in a double-quoted string must be escaped by \, to avoid unwanted interpolations, as in:

"Hello, \$name!"

Back to TOC

Long-Bracketed Strings

Long-bracketed strings are string literals enclosed by Lua-style long brackets. Examples of long brackets are [[...]], [=[...]=], [==[...]==], and etc.

All of the characters inside a long brackets always take their literal values and there are no special characters at all. For example, the literal

[=[\n"hello, 'world'!"\]=]

is equivalent to the single-quoted string '\n"hello, \'world\'!"\\'.

For convenience, if the first character inside the long brackets is a newline character, then this newline character will be removed from the final string value. So the following long-bracketed string

[=[
hello world
]=]

is equivalent to the double-quoted string "hello world\n".

Back to TOC

Shell-Quoted Strings

Shell-quoted strings provide a convenient notation for embedding shell command strings into Ops programs. It takes various forms:

sh/.../

sh{...}

sh[...]

sh(...)

sh'...'

sh"..."

sh!...!

sh#...#

Similar to long-bracketed strings, all the characters inside the shell-quotes are interpreted literally. It is possible to escape the quote characters with double-quotes, but the escaping sequence itself will still be part of the final interpreted string value.

The Ops compiler always parse the parsed string values in the shell quotes as Bash commands. So never use arbitrary strings in this syntactic construct.

Variable interpolation using the form $foo and @foo are not supported in shell-quotes since the Bash language itself uses this notation. So we support another “dot form” of variable interpolation in shell quotes, as in

sh/echo "$.name!"/

or use curly braces to avoid ambiguity with any following characters:

sh/echo "$.{name}ism"/

For convenience, this “dot form” of interpolation syntax is also supported in double-quoted strings and regex literals.

Unlike other contexts doing variable interpolation, the shell quotes also perform proper shell string quoting and escaping for interpolated Ops variables according to the shell context where the variables are used. For example, variables used inside and outside Bash’s double quotes are escaped and quoted differently according to the Bash language’s syntax. Consider this variable to be interpolated:

$name = "hello\n\"boy\"!";

It takes the literal value

hello
"boy"!

And when we interpolate it into shell’s double quotes,

sh/echo "$.name"/

we will get this final shell command string value to be sent to the terminal emulator:

echo "hello
\"boy\"!"

(Note that ! does not require escaping since the standard function setup-sh() will always disable the history feature of Bash.)

On the other hand, if we interpolate the $name variable outside any shell double quotes, then it is escaped differently:

sh/echo $.name/

yields the final shell command string

echo hello\
\"boy\"\!

One important caveat about using interpolated variables inside or outside shell double quotes is that the latter would not escape space characters and thus the scalar variable value may end up being multiple shell values, as in

my Str $files = 'a.txt b.txt';

sh{ls $.files},
stream {
    found-prompt => break;
}

This example is equivalent to

sh{ls a.txt b.txt},
stream {
    found-prompt => break;
}

Basically the $files variable value is interpreted as two separate arguments for the shell command ls. For this reason, if the file path may contain spaces, it is important to always use double quotes to enclose the interpolated Ops variable, as in

my Str $file = '/home/agentzh/My Documents/a.txt';

sh{ls "$.file"},
stream {
    found-prompt => break;
}

Without the double quotes in this example, the ls command will try to find two separate files, /home/agentzh/My and Documents/a.txt.

The shell quotes also support other shell value contexts like $(...), back-quotes (`...`), $((...)), $"...", $'...', and etc.

Variable interpolation is disabled in shell’s single quotes, just like the shell language itself. For example,

sh/echo '$.foo'/

will result in the final shell command string echo '$.foo'.

One can always output the final value of the shell-quoted string but feeding it as an argument to the print() or say() function calls, as in

say(sh/echo $.foo/)

One should avoid using control characters like tabs and those non-printable characters (like the ESC character) in shell quotes.

When shell quoted strings are used directly as actions, as in this rule:

true =>
    sh/echo hi/,
    done;

then it would be automatically converted to a std.send-cmd() function call with this shell-quoted string as its argument, as in

true =>
    std.send-cmd(sh/echo hi/),
    done;

Array variables can also be interpolated in this context just as with double-quoted strings. For example:

my Str @foo = qw(hello world);

goal all {
    run {
        say(sh/echo @.foo "@.foo"/);
    }
}

Running this example yields the output

echo hello world "hello world"

Back to TOC

Dollar Actions

Dollar actions serve as a kind of convenient syntactic sugar to save the Ops programmers’ typing. A dollar action $ CMD is a shorthand for the following Ops code snippet:

std.send-cmd(sh/CMD/),  # the actual quote character does not matter here
stream {
    found-prompt => break;
}

The / quoting character used above is arbitrary. The user is free to use the slashes in the dollar action.

The CMD after the dollar sign and the space is a user shell command string as specified in the shell-quoted strings. The command string extends all the way to the end of the line, excluding any line-trailing comma or semicolon characters.

Thanks to the implicit stream { found-prompt => break; } block, a dollar action usually won’t complete until the shell command finishes running (i.e., a new shell prompt appears).

When using dollar actions in an action chain (like in a rule’s consequent part), the comma separator is still required, as in

true =>
    $ cd /tmp/,
    $ mkdir foo,
    done;

Make sure you do not add any whitespace characters between the comma and the newline character. It is also sometimes required to add a semicolon right before the newline character if the dollar action is at the end of the action chain, as in

true =>
    $ cd /tmp;

true =>
    say("...");

It is important to note that it is still possible to override the default stream {} block introduced implicitly by the dollar action itself. Just put an explicit stream {} block right after the dollar action, as in

$ echo hi,
stream {
    out contains-word /hi/ =>
        say("hit!");

    found-prompt =>
        break;
}

Always remember to add the found-prompt rule at the end of your own stream {} block, otherwise your dollar action’s shell command won’t wait for the shell prompt at all, leading to timeout errors and etc.

Back to TOC

Numeric Constants

A numeric constant can be written as one of the following forms:

1527
3.14159
-32
-3.14
78e-3      (with a decimal exponent)
0xBEFF     (hexadecimal)
0157       (octal)

Back to TOC

Regex Literals

A regex literal is for specifying a Perl-compatible regular expression value. It is denoted by the keyword rx with a quoting structure. Below are some examples:

rx{ hello, \w+}
rx/ [0-9]+ /
rx( [a-zA-Z]* )
rx[ [a-zA-Z]* ]
rx" \d+ - \d+ "
rx' ([^a-z][a-z]+) '
rx! ([^a-z][a-z]+) !
rx# ([^a-z][a-z]+) #

The user is free to use curly braces, slashes, parentheses, double quotes, or single quotes in the regex literals. They are all equivalent, except the requirement on what specific quoting characters need to be escaped inside the regex string. For example, in the rx(...) form, use of the slash character (/) inside the regex string does not require any escaping.

One or more options can be specified on the regex, for example:

rx:i/hello/

dictates a case-insensitive match of the pattern hello. Similarly:

rx:x/ hello, world /

makes whitespace characters used in the pattern string become insignificant, and thus the regex above matches string "hello,world" but not "hello, world ". Whitespace characters used in character classes, like [a b], however, are still significant even when the :x option is specified.

Multiple options can be specified at the same time by stacking them together, as in

rx:i:s/hello, world/

When no options are to be specified, we can also save the rx prefix and just use slashes to indicate a regex literal, as in

/\w+/

/hello world/

In Ops’s regex literals, the meta character . always match any character, including the newline character ("\n") and the special pattern \s always match any whitespace characters including the newline.

Back to TOC

Wildcard Literals

A wildcard literal is for specifying a string matching pattern using the UNIX-style wildcard syntax. It is denoted by the keyword wc with a quoting structure, for instance:

wc{foo???}
wc/*.foo.com/;
wc(/country/??/);
wc"[a-z].bar.org"
wc'[a-z].*.gov'

As with regex literals, wildcard literals can also take flexible quoting characters.

Three wildcard meta patterns are supported: * for matching any sub-string, ? for matching any one single character, and [...] for character classes.

One or more options can be specified on the wildcard, for example:

wc:i/hello/

dictates a case-insensitive match of the pattern hello.

Back to TOC

Literal Arrays

Literal arrays provide an convenient way to specify constant array values. The general syntax is as follows:

(elem1, elem2, elem3, ...)

Below is an example:

action print-int-array(Int @arr) {
    print("[@arr]");
}

goal all {
    run {
        print-int-array((3, 79, -51));
    }
}

Running this print-int-array.ops example gives

$ opslang --run print-int-array.ops
[3 79 -51]

Literal arrays can also be used to initialize an array variable or set the default value of an array-typed user action parameter.

For convenience, a comma after the last element of the value list is allowed, as in

(3,14, 5, 0,)

Back to TOC

Quoted Words

Quoted words provide a convenient way to specify a constant literal string array without typing too many string enclosing quotes.

It is denoted by the keyword qw and a subsequent flexible quoting construct. For example:

qw/ foo bar baz /

is equivalent to:

("foo", "bar", "baz")

Like regex and wildcard literals, the user can choose from various quoting characters for the quoting construct used in quoted words, for instance:

qw{...}
qw[...]
qw(...)
qw'...'
qw"..."
qw!...!
qw#...#

Back to TOC

Literal Hash Values

Literal hash values provide a convenient way to specify constant hash values. The general syntax is like this:

(key1: value1, key2: value2, key3: value3, ...)

Below is an example:

action print-dog-age (Int %ages{Str}) {
    say("dog age: ", %ages<dog>);
}

goal all {
    run {
        print-dog-age((dog: 3, cat: 4));
    }
}

Running this print-dog-age.ops example gives the output

$ opslang --run print-dog-age.ops
dog age: 3

Literal hash values can also be used to initialize a hash variable or set the default value of a hash-typed user action parameter.

For convenience, a comma after the last key-value pair of the list is allowed, as in

(dog: 3, cat: 5,)

Back to TOC

Booleans

Boolean values are presented by the values of the builtin function calls true() and false(), respectively. All the relational expressions evaluate to boolean values as well.

The following values are considered “conditional false”:

  • number 0
  • string “0”
  • the value of false()
  • an empty string
  • an empty list or array
  • an empty hash table

All other values are considered “conditional true”.

Function calls true() and false() are often abbreviated to just true and false.

Back to TOC

Comments

A comment starts with the character #, and continues to the end of the current line. For example:

# this is a comment

Block comments are also supported, as in

#`(
This is a block
comment...
)

Note the backtick character and left parenthesis character directly following the # character. Parentheses can still be used inside block comments as long as they are properly paired. Even nested parentheses are allowed too:

#`(
3 * (2 - (5 - 3))
)

Back to TOC

Exceptions

Ops supports an efficient and powerful exception model kinda similar to those found in other programming languages like C++ and Java.

There are two kinds of exceptions, standard ones and user-defined ones. Standard exceptions are as follows (with brief description for each exception):

  • timeout

    Reading from or writing to the terminal emulator (tty) timed out.

  • failed

    The previous (shell) command failed with a nonzero exit code.

  • found-prompt

    The previous (shell) command completed (regardless of the exit code). Note that this exception never throws since it is nonfatal.

  • error

    Some weird errors happen when reading from or writing to the terminal emulator (tty), excluding the timeout and closed exception cases.

  • closed

    The connections to the terminal emulator (tty) are closed prematurely.

  • too-much-out

    Bufffering too much data read from the terminal emulator (tty). This limit can be tuned via the --max-out-buf-size SIZE command-line option of the opslang command-line utility.

  • too-many-tries

    The goto, redo or some other retrying calls have attempted too many times.

To reference an exception, just do a function call with the exception name as the function name.

When an exception is referenced in a rule condition, then it is the Ops language’s way to catch an exception generated in an earlier action. Here is an example:

goal all {
    run {
        $ ( exit 3 ),
        {
            failed =>
                say("cmd failed with exit code: ", exit-code);
        },
        say("done");
    }
}

Running this Ops program yields the following output (assuming the program file is failed-cmd.ops):

cmd failed with exit code: 3
done

The dollar action $ ( exit 3 ) in this example intentionally returns the exit code 3 for the current Bash command, thus leading to the failed standard exception being thrown out. In the subsequent action block, the rule with failed as the (sole) condition effectively catches this failed exception, and print out a message with the exit code of the previous shell command. Because the failed exception is properly caught and handled, the execution flow continues normally to the final say("done") action ad generates the output.

If we remove the action block for catching the exception from the preceding example, as in

goal all {
    run {
        $ ( exit 3 ),
        say("done");
    }
}

Then the Ops program will get aborted by the uncaught failed exception and prints out an error message immediately after executing the dollar action, as in

ERROR: failed-cmd-uncaught.ops:3: failed: process returned 3
  in rule/action at failed-cmd-uncaught.ops line 3

The failed exception is also able to capture the error message automatically if there is indeed any, for instance,

goal all {
    run {
        $ ( echo "Bad bad bad!" > /dev/stderr && exit 3 ),
        say("done");
    }
}

yields

ERROR: failed-msg.ops:3: failed: process returned 3: Bad bad bad!
  in rule/action at failed-msg.ops line 3

When an exception is used as an argument to the standard function throw, then the throw() function call is throwing out an exception to the upper layers of the current Ops program. If none of the upper layer catches it, then the Ops program will abort with an error message.

Back to TOC

User Exceptions

The Ops programmers can reuse any of the standard exceptions mentioned above, or define their own exceptions. To declare new exceptions, just use the exception statement as follows in the top level scope of any .ops source file:

exception too-large;

Here we define a new exception named too-large.

Multiple exceptions can be declared in the same exception statement, as in

exception foo, bar, baz;

We can use the throw standard function to throw such exceptions in our Ops programs wherever desired. And user exceptions can be caught in the exactly same way as standard ones by simply referencing the exceptions as function calls in rule conditions. For instance,

exception too-large;

action foo (Int $n) {
    {
        $n > 100 =>
            throw(too-large);
    };
}

goal all {
    run {
        foo(101),
        {
            too-large =>
                say("caught too large"),
                done;
        },
        say("done");
    }
}

Running this program yields

caught too large
done

Similarly, uncaught user exceptions lead to Ops program aborts, just like standard exceptions. Consider

exception too-large;

action foo (Int $n) {
    {
        $n > 100 =>
            throw(too-large);
    };
}

goal all {
    run {
        foo(101),
        say("done");
    }
}

which yields the output

ERROR: too-large.ops:6: too-large
  in action 'foo' at too-large.ops line 6
  in rule/action at too-large.ops line 12
  in rule/action at too-large.ops line 12

A text message can also be specified in the throw() call, as in

exception too-large;

action foo (Int $n) {
    {
        $n > 100 =>
            throw(too-large, msg: "$n is larger than 100");
    };
}

goal all {
    run {
        foo(101),
        say("done");
    }
}

which yields

ERROR: throw-msg.ops:6: too-large: 101 is larger than 100
  in action 'foo' at throw-msg.ops line 6
  in rule/action at throw-msg.ops line 12
  in rule/action at throw-msg.ops line 12

We can see there is now a more detailed error message on the first line.

Exceptions defined in modules can always be referenced by fully-qualified names in the form <module-name>.<exception-name>.

Back to TOC

Labels & Goto

The Ops language provided the goto function, which is similar to the “goto” statement in the C language but much safer.

The goto() ops function can be used to jump directly to an action with a name label either within the current scope or an outer scope.

One cannot jump across the boundaries of the current user action or the current goal action phaser.

The action which can be jumped to must carry a user-defined label, as in:

goal all {
    run {
        say("before"),

    again:

        say("middle"),
        goto(again),
        say("after");
    }
}

Here we add a label named “again” to the action say("middle"), and then call the goto function with the label as the only argument. Note that we pass the again label as a function call. The call goto(again) is equivalent to goto(again()). The again() function call evaluates to a Label typed value corresponding to the label again defined earlier.

We can run this Ops example and see what happens (assuming the program is in the file goto.ops):

$ opslang --run goto.ops
before
middle
middle
middle
middle
ERROR: goto.ops:8: too-many-tries: attempted to do goto for 4 times
  in rule/action at goto.ops line 8

So this program terminates abnormally due to an exception thrown:

ERROR: goto.ops:8: too-many-tries: attempted to do goto for 4 times

This is because that goto() call will only allow jumping for 3 times by default. This limit is per scope. So entering and leaving the current scope will reset the jumping counter to zero. Also, each action labels have separate jumping counters.

The exception thrown has the tag too-many-tries and it can be caught by the Ops code like this:

goal all {
    run {
        say("before"),

    again:

        say("middle"),
        goto(again),
        {
            too-many-tries =>
                say("caught the too-many-tries exception");
        },
        say("after");
    }
}

This program (assuming to be goto2.ops) can completes successfully:

$ opslang --run goto2.ops
before
middle
middle
middle
middle
caught the too-many-tries exception
after

Back to TOC

Jumping Count Limits

We already see the default jumping count limit is 3 times and the 4th goto call for the same label will yield the too-many-tries exception. We can change this limit by passing the tries: N named argument to the goto function call, as in

goto(again, tries: 10);

Now we have 10 jumps at maximum instead of 3.

Instead of limiting the count of jumps, we can also add varying delays before each goto() jump. For example:

goto(again, init-delay: 0.001, delay-factor: 2, max-delay: 1,
     max-total-delay: 10);

Here we start adding larger and larger delays (sleeping) before each goto() retry for the again label, starting from 0.001 seconds, topped at 1 seconds, increasing 2 times for each new retry. Once the total delays accumulated from all preceding retries go beyond 10 seconds, the too-many-tries exception will be thrown.

Please note that the very first goto() call will never have any delays or sleeps at all. The init-delay setting only takes effect on the 2nd try (or the first retry).

Please see the documentation for the goto function for more details.

Back to TOC

Label Declaration

The preceding examples both use the goto() function to jump backward (the labeled action being jumped to is before the goto call). It is also possible to jump forward. But in this case the label must be declared by the label statement, otherwise the label is used before it is declared. For instance,

goal all {
    run {
        label done;

        say("before"),
        goto(done),
        say("middle"),

    done:

        say("after");
    }
}

Here the done label is declared at the beginning of the run action phaser of the goal all.

Running this example (assuming to be goto-fwd.ops) yields

$ opslang --run goto-fwd.ops
before
after

Note how the action say("middle") is effectively skipped by the preceding goto() function call while the labeled action say("after") still runs.

Back to TOC

Jumping from Check Phaser to Action Phaser

It is possible to use goto() to jump to labels defined in an outer scope as long as it does not leave the current goal action phaser or the current user action declaration body.

Thus it is naturally possible to jump from within a check phaser to a labeled action in the surrounding action phaser, as the following example demonstrated:

goal all {
    run {
        label do-work;

        check {
            ! file-exists("/tmp/a") =>
                goto(do-work);

            true => ok;
        }

    do-work:

        say("hi from all.run");
    }
}

When the file /tmp/a does not exist, then this program will yield the output

hi from all.run

Otherwise no output will be generated.

Sometimes it is desired to always conditionally execute all the goals’ action phasers regardless of the ok() calls in their check phasers. It can be conveniently achieved by passing the -B option to the opslang command-line utility, as in

opslang -B --run foo.ops

Or as a compiler:

opslang -B -o foo.lua foo.ops

Under the hood, the -B option would simply make the ok function behave exactly like the nok function. Among other things, it is very handy when debugging or developing Ops programs.

Back to TOC

Rule-Scoped Labels

Rules have their own scopes, so if you want to use label forward declarations and the goto function inside a single rule’s consequent part, then you should use the label declaration action syntax, as in

goal all {
    run {
        label done;

        check {
            true =>
                label done,
                say("check pre"),
                goto(done),
                say("NOT THIS"),
            done:
                say("check post");
        }

        say("run pre"),
        goto(done),
        say("NOT THIS"),
    done:
        say("run post");
    }
}

Running this example gives the output

check pre
check post
run pre
run post

Note that we can have labels of the same name in different scopes.

Back to TOC

Operators

The following operators are supported, in the order of their precedence:

Precedence          Operators
0                   post-circumfix [], {}, <>
1                   **
2                   unary +/-/~, as
3                   * / % x
4                   + - ~
5                   << >>
6                   &
7                   | ^
8                   unary !, > < <= >= == !=
                    contains contains-word prefix suffix
                    !contains !contains-word !prefix !suffix
                    eq ne lt le gt ge
9                   ..
10                  ?:

The user may use parentheses, (), to explicitly change the relative precedence or associativity of the operators in a single expression.

Back to TOC

Arithmetic Operators

The language supports the following binary arithmetic operators:

**      power
*       multiplication
/       division
%       modulo
+       addition
-       subtraction

For example:

2 ** (3 * 2)        # evaluates to 64
(7 - 2) * 5         # evaluates to 25

The unary prefix operator - is also supported for arithmetic negation, as in:

-(3.15 * 2)         # evaluates to -6.3

Back to TOC

String Operators

The language supports the following binary string operators:

x       repeat a string for several times and concatenate them together
~       string concatenation

For instance:

"abc" x 3           # evaluates to "abcabcabc"
"hello" ~ "world"   # evaluates to "helloworld"

Back to TOC

Bit Operators

The following binary bit operators are supported:

<<          shift left
>>          shift right
&           bit AND
|           bit OR
^           bit XOR

The unary prefix operator ~ is for the bit NOT operation. Do not confuse it with the binary operator ~ for string concatenation.

The bit operators have not been implemented yet.

Back to TOC

Relational Operators

Use of all relational operators lead to a boolean value for the current expression. Expressions using a relational operators are relational expressions.

The following binary operators compare the two operands numerically:

>           greater than
<           less than
<=          less than or equal to
>=          great than or equal to
==          equal to
!=          not equal to

The following binary operators are for comparing string values alphabetically:

gt          greater than
lt          less than
le          less than or equal to
ge          great than or equal to
eq          equal to
ne          not equal to

There are also 3 special string binary operators for pattern matching in a string value:

contains            holds when the right hand side operator is "contained" in
                    the left hand side operator

contains-word       holds when the right hand side operator is "contained" as
                    a word in the left hand side operator

prefix              holds when the right hand side operator is a "prefix" of
                    the left hand side operator

suffix              holds when the right hand operator is a "suffix" of the
                    left hand side operator

The unary prefix operator ! negates the (boolean) value of the operand.

When the right hand side of the string comparison operators is a pattern like a wildcard or a regex value, then matching anchors are assumed in the pattern. For example:

uri eq rx{ /foo } =>
    say("hit");

is equivalent to:

uri contains rx{ \A /foo \z } =>
    say("hit");

where the regex pattern \A only matches the beginning of the string while \z only matches the end. The contains operator, on the other hand, assumes no implicit matching anchors.

Similarly, the contains-word operator assumes the surrounding \b regex anchor on both sides of the user regex.

Back to TOC

Range Operator

The binary operator .. can be used to form a range expression, as in:

1 .. 5          # equivalent to 1, 2, 3, 4, 5
'a'..'d'        # equivalent to 'a', 'b', 'c', 'd'

The value of a range expression is a flattened list of all the individual values in that range.

This operator is not implemented yet.

Back to TOC

Ternary Operator

The ternary relational operator ?: can be used to conditionally choose between two user expressions according to a user condition.

For example:

$a < 3 ? $a + 1 : $a

this expression evaluates to the value of $a + 1 when the expression $a < 3 is true, or evaluates to $a otherwise.

Back to TOC

Subscript Operators

The post-circumfix operator [] can be used as subscript of an array. For example:

my @names = ('Tom', 'Bob', 'John');

true =>
    say(@names[0]),  # output Tom
    say(@names[1]),  # output Bob
    say(@names[2]);  # output John

Negative indexes are used to access elements from the end of the array, for instance, -1 is for the last element, -2 is for the second last one, and etc.

Similarly, the post-circumfix operator {} is used to index a hash table, as in:

my %scores = (Tom: 78, Bob: 100, John: 91);

true =>
    say(%scores{'Bob'});    # output 100

The post-circumfix operator <> is used to index a hash table via literal string keys, for example, %scores<John> is equivalent to %scores{'John'}.

This operator is not implemented yet.

Back to TOC

Rules

Rules provide the basic control flow language structure in the Ops language. They replace the if/else statements in traditional programming languages and make the code for complicated branching control flow much easier to read and write.

Rules can be used in the following language contexts in Ops:

  1. action blocks, and
  2. stream blocks.

Back to TOC

Basic Rule Layout

The Ops language rules come with two basic parts, a condition, and a consequent. The condition and the consequent are connected by =>, and the whole rule is terminated by a semicolon character. The basic form of rule is like this:

<condition> => <consequent>;

The condition part of the rule can take one or more relational expressions, like resp-status == 200. All the relational expressions are connected by the comma character (,), which make all the relational expressions AND’d together, that is, all the relational expressions must hold true for the whole condition to be true. The conditions should have no side effects and this property is enforced by the opslang compiler. For this reason, the order of valuation of the relational expressions in the same condition do not change the result of the whole condition.

The consequent part usually contains one ore more actions. Each action can have side effects like changing some request aspects, performing a 302 redirect, or changing the route the current request will go in the backend. It is also possible to specify a full block of rules as a single action (see the Action Blocks section).

Below is a simple opslang rule:

file-exists("/tmp/a.txt") =>
    $ rm /tmp/a.txt;

In the condition part, file-exists("/tmp/a.txt") is a relational expression. The file-exists() function checks the existence of the file path specified by its argument.

This rule says that if the file /tmp/a.txt exits, then remove it. The consequent part is a dollar action which executes the shell command rm /tmp/a.txt.

It is worth noting that the file-exists() function takes a positional argument and no named arguments.

The Ops language is a free format language, so you can use whitespace characters freely. The indentation used before the consequent part in the example above is not significant but just for aesthetic considerations. It is totally valid to write the whole rule in a single line, for example:

file-exists("/tmp/a.txt") => $ rm /tmp/a.txt;

Back to TOC

Multiple Relational Expressions

The user can also specify multiple relational expressions in a single condition, for instance:

file-exists("/tmp/a.txt"), hostname eq 'tiger' =>
    say("hit!");

Here we have one more relational expression in the condition, i.e., hostname eq 'tiger', which tests if the current machine’s “hostname” is exactly equal to 'tiger', via the hostname() standard function. We use a different action, say("hit!"), in this example, which sends a “hit!” line to the stdout stream of the current running opslang program when it is executed.

Note the comma between the two relational expressions of the condition, it means AND, and the relational expressions on both sides of the comma must be true at the same time for the whole condition to be true.

The user can specify even more relational expressions in the same condition, as in:

file-exists("/tmp/a.txt"), hostname eq 'tiger', prompt-user eq 'agentzh' =>
    say("hit!");

We have a 3rd relational expression, that tests whether the value of the current user name shown in the last shell prompt string (usually automatically configured by the standard function setup-sh()). Note that is is also fine to replace the hostname function call in the preceding examples with the prompt-host standard function. The prompt-host function call is usually faster than hostname since it does not require running a new shell command (hostname() requires executing the hostname shell command).

Back to TOC

Multiple Conditions

Ops rules can actually take multiple parallel conditions, connected by the semicolon operator. These conditions are logically OR’d together for the current rule.

For example:

file-exists("/foo"), prompt-host eq 'glass';
file-exists("/bar"), prompt-host eq 'nuc'
=>
    say("hit!");

When either of these 2 conditions matches, the rule matches. When both of the conditions match, the rule also matches, of course.

Back to TOC

Multiple Actions

It is possible to specify multiple actions in the same rule consequent. Consider the following example:

file-exits("/tmp/foo") =>
    chdir("/tmp/"),
    $ rm -f foo,
    say("done!");

This example has 3 actions in the consequent. The first action calls the chdir() builtin function and changes the current working directory to /tmp/. The second action is a dollar action which executes the shell command rm -f foo synchronously (but still nonblocking from the perspective of IO). Finally, the call of the say() function output the line done! to the stdout stream of the current opslang program.

Back to TOC

Unconditional Rules

Some rules may choose to run their actions unconditionally. An opslang rule, however, always does require a condition part. To achieve the effect of unconditional rule triggering, the user can use the always-true predicate true() as the sole relational expression in the condition, as in:

true() =>
    say("hello world");

In this rule, the action say() always runs regardless.

Because opslang function calls without any arguments can omit their parentheses, it is preferable to write true instead of true(), as in:

true =>
    say("hello world");

Back to TOC

Multiple Rules

Multiple rules specified in the same block are executed in series. The rule written first will be executed first.

Consider the following example:

file-exists("/tmp/foo") =>
    say("file exists!");

file-not-empty("/tmp/foo") =>
    say("file not empty!");

When there is a /tmp/foo file with some data in it, we should get the following output when running this opslang code snippet:

file exists!
file not empty!

The conditions of multiple rules may get optimized by the opslang compiler, however, to be matched at the same time, and may be evaluated even before any rules are actually executed. This happens when the opslang compiler finds it safe in doing so.

Back to TOC

Action Blocks

Action blocks are formed by a pair of curly braces ({}), which also forming a new scope for variables. Action blocks can be used wherever an action can be used.

In the following example, we have two different $a variables since they each belongs to a different block (or scope):

my Int $a = 3;

{
    my Str $a = "hello";
},
say("a = $a");   # output `3` instead of `hello`

Rules are also lexical to the containing block, just like variables. Action blocks can be used to group closely related rules together, as a whole. In such a setting, some early-executed rules may use the special action done to skip all the subsequent rules in the same block. The following example demonstrates this:

{
    file-exists("/tmp/a.txt") =>
        print("hello"),
        done;

    true =>
        print("howdy");
},
say(", outside!");

If file /tmp/a.txt exists in the current system, then the output of this Ops code snippet would be hello, outside!. Note how the done action in the 1st rule skips the execution of the 2nd rule. On the other hand, in the absence of the file /tmp/a.txt, it would yield the output howdy, outside!, since the 1st rule does not match.

Action blocks can be nested to an arbitrary depth, as in:

file-exists("a") =>
    say("outer-most"),
    {
        true =>
            say("2nd level");
            {
                uri("b") =>
                    say("3rd level");
            };
    };
};

Back to TOC

Async Blocks

The Ops language supports multiple terminal sessions running in parallel. This is achieved by the use of async blocks. The actions in an async block would run asynchronously in a new lightweight thread with a brand new terminal screen.

Consider this example:

goal all {
    run {
        my Int $c = 0;

        $ FOO=32,
        async {
            $ echo in async,
            print(cout);
        },
        $ echo in main,
        print(cout);
    }
}

A typical run yields

$ opslang --run async.ops
in main
in async

Sometimes we may also get the output lines swapped:

$ opslang --run async.ops
in async
in main

This is expected due to the asynchronous nature of the async {} block execution.

Async blocks automatically create a new terminal screen which is separate from the current one. In fact, it invokes the new-screen standard function under the hood. For this reason, the total number of async blocks that can be created is also subject to the terminal screen count limit which can be tuned by the --max-screens N command-line option of the opslang utility.

It is prohibited to switch to a terminal screen which is not owned by the current lightweight thread. An error will be raised when such attempts are being made, like this:

test.ops:11: ERROR: screen 'foo' not owned by the current thread

Async blocks can be nested or be used in user actions. And async blocks can also create its own terminal screens.

Async blocks can also take a name, as in

async foo {
    ...
}

This way, the terminal screen corresponding to the async block also takes the same name. Otherwise they get serial number names like 1, 2, 3, and etc, depending on the order of their appearance in the ops source files.

The script log file for the screens follow the same naming convention used by those screens explicitly created by the new-screen function. For anonymous screens, they have special screen names like 1, 2, 3, and etc.

The Ops program won’t terminate until all the async {} blocks and the main thread are terminated. There are several exceptions to this rule, however:

  1. when a thread of execution calls the exit() standard function,
  2. when a thread of execution calls the die() standard function, or
  3. when a thread of execution has an uncaught exception like timeout,

Back to TOC

Variable Sharing Among Threads

All Ops programs have one lightweight thread, which is the main thread created upon startup. The user may create more lightweight threads via async {} blocks. All these threads share the same top-level variables declared by my. Consider the following example:

my Int $c = 0;

goal all {
    run {
        setup-sh,
        $ FOO=32,
        async {
            setup-sh,
            $c++,
            say("c=", $c);
        },
        $c++,
        say("c=", $c);
    }
}

Running this example always gives the output

c=1
c=2

For local variables declared inside an action phaser or a user action, if they are visible to the async {} block, they are immediately cloned to a copy when the thread for the async {} block is spawned. In the new thread, those local variables still have their initial values when entering the async block, but writing new values to them will no longer affect other threads, nor will any changes to them in any other threads affect this new thread. For instance,

goal all {
    run {
        my Int $c = 0;

        setup-sh,
        $ FOO=32,
        async {
            setup-sh,
            $c++,
            say("async: c=", $c);
        },
        sleep(0.1),
        $c++,
        say("main: c=", $c);
    }
}

Running this example always yields the output

async: c=1
main: c=1

So essentially each of these 2 threads has its own copy of this local variable $c.

Back to TOC

Assignment Actions

The assignment operator = is used to specify an action that assigns a value to a variable or an expression that can be an lvalue. For example:

my $a;

$a = 3;

Like all the other actions, an assignment expression has no value for itself. So it is not allowed to embed an assignment expression in other expressions. For example, the following example will yield a compile-time error:

? my $a;
?
? say($a = 3);

This is because the assignment $a = 3 returns no value and it can only be used as a standalone action.

The assignment:

$a = $a + 3

can be simplified using the operator +=:

$a += 3

Similarly, *=, /=, %=, x=, ~= are provided for the binary operators *, /, %, x, and ~, respectively.

Furthermore, the postfix operator ++ can be used to simplify the += 1 case. For example:

$a++

is equivalent to $a += 1 or $a = $a + 1. Similarly, the postfix operator -- is provided as a shorthand for -= 1.

Like the standard = operator, all these assignment variations do not take any values themselves and can only be used as standalone actions.

Back to TOC

Lua Blocks

Inline Lua code snippets in opslang programs is supported to make hard things possible.

Lua code snippets are enclosed in lua blocks which can be used on the global scope, as an action, or as an ops expression.

Back to TOC

Global Lua Blocks

Global lua blocks are used on the top-level scope of the current .ops source file (either an Ops module file or not). Such Lua code snippets would expand in a separate Lua variable scope on the top-level scope of the resulting Lua source file. Any Lua local variables defined directly in the Lua code block would not be visible outside this lua block. If you want to define Lua variables visible to the entire file’s Lua code, use _M.NAME instead where NAME is your variable name.

Below is a simple example:

lua {
    print("init!")
}

goal all {
    run {
        say("done");
    }
}

Running this global-lua-block.lua example gives

$ opslang --run global-lua-block.lua
init!
done

Back to TOC

Lua Block Actions

Lua blocks can also be used directly as actions. In such case, no return values are expected from within the Lua block. Below is an example:

goal all {
    run {
        print("hello, "),
        lua {
            print("world!")
        };
    }
}

Running this lua-action.ops example gives:

$ opslang --run lua-action.ops
hello, world!

Back to TOC

Lua Block Expressions

Lua blocks can be used as Ops expressions but in this case a return value type must be specified and the inlined Lua source must return a Lua value of the correspond data type, as in

goal all {
    run {
        say("value: ", lua ret Int { return 3 + 2 })
    }
}

Running this lua-expr.ops example gives

$ opslang --run lua-expr.ops
value: 5

Back to TOC

Variable interpolation in Lua blocks

It is possible to interpolate opslang variables in all three kinds of Lua blocks just introduced above.

All the following forms of Ops variables are supported in the inlined Lua source code:

  1. $.NAME
  2. $.{NAME}
  3. @.NAME
  4. @.{NAME}
  5. @{NAME}
  6. %.{NAME}
  7. %{NAME}

When Ops variables are used in Lua’s single or double quoted strings, they would retain their original string values in the final Lua values, even if their string values contain special characters like ', ", and \. On the other hand, use of Ops variables inside Lua’s long bracketed strings does not enable Ops variable interpolation at all.

Interpolated Ops variables can also be used where a Lua expression is expected in the Lua code.

Below is an example for interpolating Ops array variables inside a Lua single-quoted string literal:

goal all {
    run {
        my Str @foo = ("hello", 'world');

        say("lua: ", lua ret Str {
            return '[@.foo]'
        })
    }
}

Running it gives the output

lua: [hello world]

Below is an example for interpolating Ops scalar variables:

goal all {
    run {
        my Str $foo = "hello";
        my Int $bar = 32;

        say("lua: ", lua ret Str {
            return $.foo .. $.bar + 1
        })
    }
}

Running this example gives the output

lua: hello33

Special characters like $, @, %, and \ can be escaped by prefixing it with a \ character.

Back to TOC

For Statements

The for statement is a special kind of actions for iteratoring through array or hash containers. The general syntax is

for CONTAINER -> VAR1 [, VAR2] {
    SYMBOL-DECLARATION;
    SYMBOL-DECLARATION;
    ...

    ACTION,
    ACTION,
    ...;
}

Below is a simple example for iterating an Int array:

goal all {
    run {
        my Int @arr = (56, -3, 0);

        say("begin"),
        for @arr -> $elem {
            say("elem: $elem");
        },
        say("end");
    }
}

Running this yields the output

begin
elem: 56
elem: -3
elem: 0
end

Note that we do not need to declare the $elem iterator variable via the my keyword since the -> operator of the for statement already implicitly defines this local variable in the scope of the for statement itself.

Also note that we do not specify the type for the iterator variable $elem since its type is always determined by the element type of the array container expression value right before the -> operator.

We can also declare two iterator variables in this example to get the array index value for the current loop iteration:

goal all {
    run {
        my Int @arr = (56, -3, 0);

        say("begin"),
        for @arr -> $i, $elem {
            say("elem $i: $elem");
        },
        say("end");
    }
}

which gives

begin
elem 0: 56
elem 1: -3
elem 2: 0
end

Again, we do not explicitly specify the type of the $i variable after -> because its type is always Int.

We can iterator through hash table values in a similar way where 2 iterator variables must be specified for the keys and values of the hash respectively. For instance,

goal all {
    run {
        my Int %a{Str} = (dog: 3, cat: -51);

        say("begin"),
        for %a -> $k, $v {
            say("key: $k, value: ", $v);
        },
        say("end");
    }
}

yields the output

begin
key: dog, value: 3
key: cat, value: -51
end

Back to TOC

Terminal Emulator Operations

The biggest strength of the Ops language is its native support for automating any terminal emulators. It also has good support for transparently handling control sequences of *NIX style terminals. Upon startup, each Ops program will always create a pseudo terminator emulator instance (aka pseudo tty) for the rest of the operations. This way, the Ops program can interact with the terminal interface of the operating system in the exactly way as a human operator (usually being a developer or a dev ops). By default, this terminal is bound to an interactive bash session, which can be changed through the --shell SHELL option of the opslang command-line utility.

We shall use the terms terminal and terminal emulator interchangeably in this text. Both of them mean the pseudo terminal emulator instance just mentioned above.

To some extend, the terminal interaction capabilities provided by the Ops language and runtime are very similar to those provided by the classic Tcl/Expect tool chain, but much more powerful and much more flexible in many ways.

The full terminal interaction details are recorded in a log file generated by the script command-line utility (usually coming from the util-linux project). It is often very useful to check out the full terminal logs by checking the ./ops.script.log file after running the Ops program on the command line. This log file path can be changed through the --script-log-file PATH option of the opslang command-line utility.

Back to TOC

Writing to the Terminal

The Ops program can send text data, like echo hello, to the terminal, via the send-text standard function. It can also send special control characters (like the enter key or the Ctrl-C key combination) to the terminal via the send-key standard function.

Back to TOC

Echoed-Back Typing

Please keep in mind, for many cases when writing to a terminal (though not always), the terminal would echo back what is typed to the terminal to give the feedback (for human users). What’s worse is that some times, when the typing is too fast, the echoed back output might even repeat itself completely or partially, depending on the actual contexts. The Ops program should always be prepared for such “echoed-back” output from the terminal. Fortunately, the Ops language does provide some built-in features and functions to help dealing with such bloody details so the Ops programmers usually do not have to worry about these unless they want to handle every little detail themselves on a very low level.

Back to TOC

Reading from the Terminal

Reading data from the terminal is inherently more difficult than writing to it. This is because the size and content of the data stream output by the terminal may not be predictable at all. The Ops language provides two ways of reading from the terminal: the streaming way and the fully-buffered way. The Ops programmer is free to choose either way according to the concrete use cases.

Back to TOC

Streaming Reading

The Ops language provided the stream blocks to do streaming reading from the terminal emulator.

Inside the stream blocks, the standard function out represents the output stream of the terminal in an abstract way. This out function call can be used in binary relational expressions to do regex and string pattern matching (involved with the operators like contains, prefix, eq, and suffix). Never try using the out value like normal string values; you will get a compile time error like this:

ERROR: out(): this function can only be used as the lvalue of relational expression operators at out.ops line 5

The out stream automatically filters out any console control sequences for showing colors or doing some other fancy things in a real console. If you want to match against raw terminal output with those control sequences intact, then you should use the raw-out standard function instead.

There is another echoed-out variant which is used for matching echoed-back typing in the output stream.

Back to TOC

Stream Blocks

A stream block has the following format:

stream {
    rule-1;
    rule-2;
    ...
}

Basically a stream block starts with the keyword stream, and then a block enclosed by a pair of curly braces, in which one or more rules can be specified.

Consider the following example:

goal all {
    run {
        stream {
            out suffix /.*?For details type `warranty'\.\s*\z/ =>
                print($0),
                break;
        }
    }
}

We run this Ops program with the ubiquitous UNIX bc program like this (assuming the Ops program is in the file bc-banner.ops):

$ opslang --shell bc --run bc-banner.ops
bc 1.06
Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.

So this Ops program just reads the initial banner of bc (printed out by the bc program itself to the terminal) and output it to its own stdout device. The actual text in the output may differ if your system’s bc program is of a different version (or even of a different origin).

Note the rule (and also the only rule) in the stream {} block of the preceding example. We use the text For details type `warranty`. as the indicator for the end of banner. Once this regex pattern matches, we print out everything matched by this regex by printing out the value of the special capturing variable $0.

Note the break action in the rule above. This function call immediately terminates the reading process of the current stream {} block, otherwise the stream block will keep looping and waiting for more rules to fire (until the timeout exception is thrown).

The stream blocks can have multiple rules and those rules will be matched independently against the terminal’s output data stream in a streaming fashion. Consider another example dealing with the bc program:

goal all {
    run {
        stream {
            out contains /\w+/ =>
                say("rule 1: $0");

            out contains /\w+/ =>
                say("rule 2: $0");

            out suffix /For details type `warranty'\.\s*/ =>
                say("quitting..."),
                break;
        };
    }
}

Running this program yields the output

rule 1: bc
rule 2: bc
quitting...

So all the 3 rules are fired in turn. The order of the rule firing is important here. The conditions in the first 2 rules both match the exactly the same substring and locations in the out data stream, so their firing order is now depending on the order of their appearance in the Ops program source file. Consequently, the first rule would always fire before the second rule since it appears before the second one. On the hand, the 3rd rule matches the last sentence of the out data stream, so it will always run after the first 2 rules. Let’s try moving the 3rd rule before the first 2 rules and see what happens:

goal all {
    run {
        stream {
            out suffix /For details type `warranty'\.\s*/ =>
                say("quitting..."),
                break;

            out contains /[a-zA-Z]+/ =>
                say("rule 1: $0");

            out contains /[a-zA-Z]+/ =>
                say("rule 2: $0");
        };
    }
}

Running it yields the exactly same output:

rule 1: bc
rule 2: bc
quitting...

The location of the matched text of the first rule is after the matched text of the latter 2 rules, so even the 1st rule appears first in the Ops source code, it runs after the latter 2 rules. This is the essence of streaming processing. One never waits for the full output before doing any pattern matching. One does the matching as soon as seeing some more output data.

Also note that in the preceding example, the latter 2 rules do not carry the break action. This is important since break would break the execution flow immediately out of the current stream {} block while we really want as many rules to match and fire in this very example. We can try adding the break action to all 3 rules in the stream block:

goal all {
    run {
        stream {
            out suffix /For details type `warranty'\.\s*/ =>
                say("quitting..."),
                break;

            out contains /[a-zA-Z]+/ =>
                say("rule 1: $0"),
                break;

            out contains /[a-zA-Z]+/ =>
                say("rule 2: $0"),
                break;
        };
    }
}

Now naturally, only the first rule will have a chance to fire (assuming this code example is put into file all-break.ops):

$ opslang --shell bc --run all-break.ops
rule 1: bc

Another very important observation from the preceding examples is that all rules in the stream block will fire no more than once. Apparently the out contains /[a-zA-Z]+/ condition, for example, would match many times in the bc program banner text. Whereas, we only get the first match in the output. This is the expected behavior according to the Ops language’s design. To make a rule fire multiple times, one should call the redo action at the end of the corresponding rule’s consequent part, as in

goal all {
    run {
        stream {
            out suffix /For details type `warranty'\.\s*/ =>
                say("quitting..."),
                break;

            out contains /[a-zA-Z]+/ =>
                say("word: $0"),
                redo;
        };
    }
}

Running this redo.ops program gives the output

$ opslang --shell bc --run redo.ops
word: bc
word: Copyright
word: Free
word: Software
ERROR: redo.ops:10: too-many-tries
  in stream block rule at redo.ops line 10
  in rule/action at redo.ops line 4

This time, we matched more “words” through the redo action: bc, Copyright, Free, and Software. Interestingly, the Ops program aborts with an uncaught exception too-many-tries after outputting 4 words. The too-many-tries exception is thrown because the redo action in that rule are to be fired more than 3 times. The standard function redo does have a default counting limit of 3, just like the standard function goto. To match and output more words in our preceding example, we can specify the tries named argument for the redo() call, as in

goal all {
    run {
        stream {
            out suffix /For details type `warranty'\.\s*/ =>
                say("quitting..."),
                break;

            out contains /[a-zA-Z]+/ =>
                say("word: $0"),
                redo(tries: 20);
        };
    }
}

This time we can output all the words in the bc banner without getting aborted by the too-many-tries exception:

word: bc
word: Copyright
word: Free
word: Software
word: Foundation
word: Inc
word: This
word: is
word: free
word: software
word: with
word: ABSOLUTELY
word: NO
word: WARRANTY
word: For
word: details
word: type
word: warranty
quitting...

Now let’s actually use bc to do some useful work, i.e., to do some simple arithmetic calculations like 3 + 5 * 2:

goal calc {
    run {
        my Str $expr;

        stream {
            out suffix /For details type `warranty'\.\s*/ =>
                break;
        },

        # enter the expression:
        $expr = "3 + 5 * 2",
        send-text($expr),
        stream {
            echoed-out suffix $expr =>
                break;
        },

        # submit the entered expression by pressing Enter:
        send-key("\n"),
        stream {
            echoed-out prefix /.*?\n/ =>
                break;
        },

        # collect the result in the new output:
        stream {
            out suffix /^(\d+)\n/ =>
                say("res = $1"),
                break;
        };
    }
}

This example is a bit lengthy since it deals with the raw terminal interaction directly. First of all, we use a stream block to wait the bc to output the full banner text. Then, we enter the expression 3 + 5 * 2 via the standard function send-text. After that, we use another stream block to wait the terminal to echo back the expression text we just “typed”. Once we are sure our expression is indeed entered, we emulate the action of pressing the Enter key on the keyboard by calling the send-key function (just like a human user would do). Again, we use a new stream block to make sure this Enter key press is well received and echoed back to us. Finally, we use a final stream block to receive the integer number output by bc for the calculation result.

It is important to note that we use the standard function echoed-out to match our echoed-back typing. The main difference between ehoed-out and out is that the former will also automatically filter out special character sequences introduced by the terminal’s automatic line wrapping (every terminal has a width and when the typed line exceeds that width limit, line wrapping occurs).

You may wonder why we bother waiting for the echoed back text for our own input. This is important because all our communications with bc must go through the terminal emulator, and if we type too fast, our typing may get lost, just like fast human users may sometimes lose typing in a real terminal which is slow to respond.

Running this calc.ops program yields

$ opslang --shell bc --run calc.ops
res = 13

Let’s check out the ./ops.script.log file for the raw terminal session history:

$ cat ops.script.log
Script started on 2019-06-12 17:10:52-07:00
bc 1.07.1
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
3 + 5 * 2
13

That’s exactly what we’d expect.

Back to TOC

Fully Buffered Reading

Fully buffered terminal output reading is achieved by defining a proper prompt and reading the string value returned by the standard function cout (or cmd-out for the long form).

The string value returned by cout is defined to be any data between the out stream location left by the previous stream {} block and the starting location of the currently matched prompt string through the current stream {} block.

See the GDB Prompts section for examples.

Back to TOC

Prompt String Handling

In earlier examples with bc, we already see how to deal with terminal interactions directly in Ops. For much more complex interactive programs like bash and gdb, we will need to deal with “prompt strings” (or just “prompts”) like bash-4.4$ and (gdb). One can imagine that manually handling the prompt string in every Ops program can be a really tedious and daunting task. Fortunately, Ops provides built-in support for prompt string handling, through various standard functions like push-prompt and pop-prompt, as well as the standard exception found-prompt.

Back to TOC

GDB Prompts

Consider this example which types some gdb command and output its result:

goal all {
    run {
        my Str $cmd;

        push-prompt("(gdb) "),

        # wait for the prompt to show up:
        stream {
            found-prompt => break;
        },

        # send our gdb command:
        $cmd = "p 3 + 2 * 5",
        send-text($cmd),

        # wait for our typing to echo back:
        stream {
            echoed-out suffix $cmd =>
                break;
        },

        # press Enter to submit the command:
        send-key("\n"),
        stream {
            echoed-out prefix /.*?\n/ =>
                break;
        },

        # collect and output the result:
        stream {
            found-prompt =>
                print("res: ", cout),
                break;
        };
    }
}

In this example, we first use the push-prompt standard function to register a new prompt string pattern, "(gdb) ", to the Ops runtime. Then in the following stream {} block, we can use a rule with the condition found-prompt to wait for the first appearance of such a prompt string in the terminal output stream. Once we get the gdb prompt, we just send the gdb command, p 3 + 2 * 5, wait for the echoed-back command typing, press an Enter key, receive its echoed-back character too, and finally wait for the next gdb prompt. Once we get the new gdb prompt, we use the standard function cout to extract the output for the previously executed command (which is defined to be the data before the currently matched prompt string, and after the output stream byte position left by the previous stream block (which is for the echoed-back output of the Enter key press). The use of the cout function is actually the fully buffered reading mode. Apparently it is built upon the detection of prompt strings and the default streaming reading mode.

We need to pay special attention to the space character right after the (gdb) string in our prompt string pattern. That space is necessary since prompt string matching is always a suffix matching operation and GDB always put a space immediately after the (gdb) string. It is part of the real GDB prompt string.

Let’s run this gdb.ops program with gdb as our “shell”:

$ opslang --shell gdb --run gdb.ops
res: $1 = 13

Exactly we want.

Actually we could use the standard function send-cmd to simplify our example above. This function combines the operations of sending a command string, waiting for its echoed-back output, pressing the Enter key, and waiting for the echoed-back newline. The simplified version of the preceding example is as follows:

goal all {
    run {
        push-prompt("(gdb) "),

        # wait for the prompt to show up:
        stream {
            found-prompt => break;
        },

        # send our gdb command:
        send-cmd("p 3 + 2 * 5"),

        # collect and output the result:
        stream {
            found-prompt =>
                print("res: ", cout),
                break;
        };
    }
}

Gladly the code is now much shorter and cleaner. The result is exactly the same:

$ opslang --shell gdb --run gdb2.ops
res: $1 = 13

Back to TOC

Bash Prompts

Bash’s prompt strings are more complex than GDB’s since it is not a fixed string at all. It can be configured by the user (by setting a prompt template string to the PS1 bash variable, for example) and may contain dynamic information due to the use of shell variables in the prompt string template.

Below is an example trying to match a generic bash prompt string:

goal all {
    run {
        stream {
            out suffix rx:x/^ [^\r\n]+ [\$%\#>)\]\}] \s*/
            =>
                push-prompt($0),
                break;
        },
        send-cmd("echo hello"),
        stream {
            found-prompt =>
                break;
        },
        say("out: [", cout, "]");
    }
}

Here we first use a stream block and a regex pattern match to find a string which looks like a shell prompt string, and then we push the matched string to the prompt pattern stack via the push-prompt standard function. One feature of the push-prompt function is that it automatically canonicalize all the numbers in the prompt string pattern to 0, so that even the shell prompt template contains variables like $?, our prompt detection will still succeed.

Let’s run this sh-prompt.ops program with the --shell bash option:

$ opslang --shell bash --run sh-prompt.ops
out: [hello
]

Ops provides several additional standard functions to make Bash sessions’ handling even easier. For instance, the preceding example could be simplified by using shell-quoted strings:

goal all {
    run {
        stream {
            out suffix rx:x/^ [^\r\n]+ [\$%\#>)\]\}] \s*/ =>
                push-prompt($0),
                break;
        },
        sh/echo hello/,
        stream {
            found-prompt =>
                break;
        },
        say("out: [", cout, "]");
    }
}

And we can further simplify this with the dollar action feature:

goal all {
    run {
        stream {
            out suffix rx:x/^ [^\r\n]+ [\$%\#>)\]\}] \s*/ =>
                push-prompt($0),
                break;
        },
        $ echo hello,
        say("out: [", cout, "]");
    }
}

Even the initial shell prompt detection can be simplified by calling the standard function setup-sh:

goal all {
    run {
        setup-sh,
        $ echo hello,
        say("out: [", cout, "]");
    }
}

Of course, the setup-sh function can do a lot more than our poor man’s shell prompt detection shown above.

Indeed, setup-sh is almost always the first call for any Ops programs interacting with shell sessions, so the opslang utility would automatically call it if the user does not specify the --shell SHELL command -line option at all. Make sure you do not make duplicate setup-sh function call at the beginning of your goal action phasers if the opslang utility automatically calls it already. Duplicate calls will result in reading timeout errors since there won’t be any duplicate shell prompt strings for us to match at shell session startup anyway.

Therefore, if we run the opslang command line without the --shell bash option like previous examples, then we must remove the setup-sh call from our Ops program, as in:

goal all {
    run {
        $ echo hello,
        say("out: [", cout, "]");
    }
}

And run this sh-prompt2.ops program like this:

$ opslang --run sh-prompt2.ops
out: [hello
]

This example now finally becomes as concise as it should have been.

Back to TOC

Nested Bash Sessions

One unique strength of Ops is the simplicity of handling nested shell sessions. Below is an example:

goal all {
    run {
        sh/bash/,
        setup-sh,
        $ echo $(( 3 + 1 )),
        say("out: [", cout, "]"),
        exit-sh;  # to quit the nested shell session
    }
}

Running this nested-sh.ops program gives the output

$ opslang --run nested-sh.ops
out: [4
]

Here we must call setup-sh explicitly for the nested shell session ourselves and also call the exit-sh standard function to properly quite it. The exit-sh function essentially does the following two actions under the hood:

pop-prompt,
$ exit 0;

The pop-prompt standard function call removes the current nested shell session’s prompt pattern and restores the parent shell session’s prompt pattern setting. And then it performs the shell command exit 0, after which the Ops runtime can successfully detect the parent bash session’s prompt string without problem.

It is important to note that we intentionally avoid using a dollar action for the first bash shell command. Recall that the shell-quoted string action sh/bash/ above is just a shorthand notation for the full form, send-cmd(sh/bash/). We must not use dollar actions here for the bash command because dollar actions would automatically wait for the prompt string with an implicit stream block. Here we definitely need a setup-sh() call to automatically detect a new prompt string for the new shell session (not the old session’s prompt!).

Following the pattern in the previous example, we could nest as many shell sessions in the same Ops program session as we’d like. The push-prompt and pop-prompt function calls simply work on an internal prompt pattern stack which can grow as deeply as necessary.

Nested bash sessions may also be created by running other shell commands like su, instead of calling bash directly.

Back to TOC

Nested SSH Sessions

Dealing with nested SSH sessions is just as easy as nested bash sessions. Below is an example:

goal all {
    run {
        sh/ssh -o StrictHostKeyChecking=no 127.0.0.1/,
        setup-sh,
        $ echo $(( 3 + 1 )),
        say("out: [", cout, "]"),
        exit-sh;  # to quit the nested ssh session
    }
}

Running this nested-ssh.ops program yields

$ opslang --run nested-ssh.ops
out: [4
]

Of course, connecting to the local host is not very useful in the real world. One can choose to connect to any remote servers as desired.

As with nested bash sessions, nested SSH sessions can also nest as deeply as needed, for instance,

goal all {
    run {
        sh/ssh -o StrictHostKeyChecking=no 127.0.0.1/,
        setup-sh,
        $ echo $(( 3 + 1 )),
        say("1: out: [", cout, "]"),

        sh/ssh -o StrictHostKeyChecking=no 127.0.0.1/,
        setup-sh,

        $ echo $(( 7 + 8 )),
        say("2: out: [", cout, "]"),

        exit-sh,  # to quit the second ssh session
        exit-sh;  # to quit the first ssh session
    }
}

Running this program gives the output

1: out: [4
]
2: out: [15
]

This is especially useful for logging into machines which require first logging onto an intermediate “jump machine”.

Back to TOC

Multiple Terminal Screens

The Ops language supports creating multiple terminal screens in the same program, similar to the popular GNU screen and tmux command-line utilities.

All the output handling and prompt string handling are separated among different screens. The Ops variables are shared, however.

One can create a new terminal screen via the new-screen() standard function and switch among multiple screens via the switch-screen function.

Indeed Ops always creates a default screen called “main”.

Screen names must be valid identifiers. Use of bad name strings will lead to runtime or compile-time errors.

Consider the following example:

goal all {
    run {
        # in the default 'main' screen
        $ Foo=,

        new-screen('foo'),

        # in our new 'foo' screen
        $ FOO=32,

        switch-screen('main'),

        say("main: FOO: [", sh-var('FOO'), "]"),

        switch-screen('foo'),

        say("foo: FOO: [", sh-var('FOO'), "]");
    }
}

Here we create a second screen named foo, and we set the shell variable FOO to an empty value in the default main screen and to 32 in our own foo screen. This example outputs the value of this FOO shell variable in these 2 different screens. We use the sh-var standard function to retrieve the value of the specified shell variable conveniently.

Running this screens.ops program yields

$ opslang --run screens.ops
main: FOO: []
foo: FOO: [32]

Naturally, different terminal screens have separate shell environments.

Note that the new-screen() function call also automatically switches to the newly created screen. And when the screen name already exists, the new-screen() call is equivalent to a switch-screen() call.

One important behavior of the switch-screen() function call is that its effect only lasts to the end of the current user action body or the current user goal’s action phaser. The effect of switch-screen() will last beyond the end of the check {} phaser and until the end of the surrounding action phaser block.

There is a limit for how many parallel screens that can be created for a single Ops program run. By default, the limit is 20. It can be tuned via the command line option --max-screens, as in

opslang --run --max-screens 100 foo.ops

The generated script log files are also separate for each screen. The main screen’s log file is still ./ops.script.log by default and the log file for the foo screen is ./ops.script.log.foo. User-specified script log file paths would follow the same naming scheme (the non-main screen adds the screen name to the end of the main file name).

Back to TOC

User Actions

The user can define their own parametric actions by grouping some other actions together. The general syntax for defining custom actions is like below:

action <name> (<arg>...) [ret TYPE] [is export] {
    [<label-declaration> | <variable-declaration>]...

    <action1>,
    <action2>,
    ...
    <actionN>;
}

For example:

action say-hi(Str $who) {
    say("hi, $who!"),
    say("bye");
}

Calling this say-hi action as say-hi("Tom") will yield the output

hi, Tom!
bye

Multiple parameters can also be specified.

User-defined actions are a great way to introduce your own vocabulary to the actions that you can use in rules and other contexts.

Recursive actions are prohibited.

Parameters must be declared with the type information, just like local variables declared by the my keyword. Optional parameters can be specified with a default value, as in

action say-hi(Str $who = "Bob") {
    say("hi, $who");
}

Then the say-hi action is allowed to be called without any arguments, like say-hi(), in which case, the parameter $who will get the default value "Bob".

It is possible to specify a nil default value for such optional parameters, as in

action foo(Int $a = nil) {
    {
        defined($a) =>
            say("a has a value!"),
            done;

        true =>
            say("a has no value :(");
    }
}

One can use the standard function defined to test whether a variable is nil.

Back to TOC

User Actions With A Return Value

User actions can return a value of a user-specified type. It is achieved by specifying the ret TYPE trait right after the parameter list, as in

action world() ret Str {
    return "world";
}

Note how we use the return statement to return the string value "world" to the caller of this world user action.

The return value type can be

  1. a scalar type (Str, Num, Int, or Bool),
  2. an array type (Str[], Num[], Int[], or Bool[]), or
  3. a hash type (Num{Str}, Int{Str}, Str{Num} and etc).

When the ret TYPE trait is omitted, then it is assumed to be returning nothing or a nil value.

Back to TOC

Goals

Goals in opslang are similar to the goals in Makefiles though they never directly correspond to any files as Makefiles’ goals. In a sense, they are more similar to GNU Makefiles’ “phony goals” or “phony targets”. Ops’s goals are named tasks to be fulfilled for each run of any Ops programs. They are the entry points of Ops programs (just like the C language’s main function).

Each Ops program must define at least one goal and they may choose to define multiple goals. For Ops programs with multiple goals defined, they also have more than one entry points. The default entry point is called the default goal. By default, the default goal is the first goal defined in the Ops program, which can be overridden by the special default goal statement.

A simplest goal definition in Ops is as follows:

goal hello {
    run {
        say("hello, world!");
    }
}

This defines a new goal named hello which when runs prints out the line hello, world! to the stdout stream of the current Ops program. This is also the classical “hello world” program for the Ops language. Assuming this Ops program is in the file hello.ops, and we can run it on a terminal:

$ opslang --run hello.ops
hello, world!

Exactly what we’d expect.

There are some things we should note in this example:

  1. We define a run {} block in the hello goal, which is called an action phaser. A goal can take several different kinds of action phasers, like run, build, install, prep, and package. The run action phaser is one of the most commonly used action phasers of goals. When other kinds of action phasers do not make sense, just use the run one.
  2. The run action phaser in this example contains a single action, which is a function call say("hello, world!").
  3. The hello.ops program file contains only this one goal, hello, so it is also the default goal being executed when no target goals are specified on the command line when running this Ops program via the opslang command-line utility.

We can explicitly specify the entry point goal on the opslang command line, as follows:

opslang --run hello.ops hello

The extra command-line argument hello specify the target goal to run.

It is possible to specify multiple target goals as even more command-line arguments, as in

opslang --run example.ops foo bar baz

In this case, the goals foo, bar, and baz will be executed in turn.

Back to TOC

The Default Goal

Every Ops program must have one and only one default goal, which is executed when the Ops program is run without any target goals specified either on the command line or in the Lua API call. The default goal can be explicitly specified by the default goal statement as follows:

default goal foo;

This statement must be on the top-level scope of an Ops program, usually near the beginning of the whole file. The goal foo can be defined after the default goal statement. There is no requirement for foo to be already defined before this default goal statement.

When there is no default goal statement in an Ops program, the first goal defined will become the default goal.

Back to TOC

Action Phasers

A goal can define one or more action phasers. An action phaser is a named group of actions to be executed when the goal is run. The following names are predefined for phaser actions, in the order of their usual execution:

  • prep: do some preparation work.
  • build: do software building (usually code compiling and code generation).
  • install: install the software.
  • run: run the software or other actions which cannot easily be categorized here.
  • package: do package the built and installed software (as RPM or Deb packages, for instance).

A goal can define one or multiple action phasers, or even none at all.

When a goal defines no action phasers, it is said to be an abstract action phaser, which must have some goal dependencies.

To explicitly invoke a goal foo’s action phaser xxx, we can just reference the phaser using the fully qualified name, foo.xxx. For example, to reference goal foo’s action phaser run, we just write foo.run(). The parentheses may be omitted just like the normal Ops function calls.

When a goal defines multiple action phasers, then the action phasers will form a chain. If a later action phaser is about to run, then any defined action phasers earlier than this action phaser will be run in order. For example, if a goal foo has prep, build, and run phasers defined, and the caller runs the foo.run phaser. Before the foo.run phaser runs, the goal will automatically run foo.prep, and then foo.build. The foo.install phaser won’t run since it is not defined in goal foo in this example.

A goal can also be invoked without specifying the action phaser name, just treat the goal itself as a callable function. For example, to invoke goal foo, we write foo(). This will result in calling the default action phaser of that goal.

Back to TOC

Local Symbols of Action Phaser

Action phasers may define their own local symbols, like variables and labels. Such symbols are also visible to the check phaser block if it is defined inside the current action phaser.

Below is an example:

goal hello {
    run {
        my Str $name = "Yichun";

        say("Hi, $name!");
    }
}

Running this goal yields the output

Hi, Yichun!

Back to TOC

Default Action Phaser

When a goal is invoked without specifying an action phaser, the default action phaser will be called if there is any.

The default action phaser is defined to be 1) the latest running phaser except the package and prep phasers, or 2) the package phaser when it is the only phaser defined.

For example, if a goal foo has action phasers build, install, run, and package phasers defined, then the default action phaser will be the run phaser. For another example, if the goal bar only defines the package phaser, then the default one will be it.

Abstract goals without any action phaser defined will only execute its dependency goals.

Back to TOC

Check Phasers

Each action phaser can take an optional check phaser, which runs a set of rules to determine whether the current action phaser’s main actions should be run at all. The check phaser is usually used to verify if the current goal has already been built or fulfilled.

Two standard functions can be used in the check phaser context to signal success or failures immediately. These are ok and nok functions. A call to the ok() function will cause the current check phaser immediately succeed and the current action phaser will be skipped running. On the other hand, the nok() function causes the current check phaser to immediately fail, and start executing the surrounding action phaser’s main actions immediately.

If neither of these two functions are ever called in a check phaser, then the check phaser fails by default (i.e., need to run the action phaser’s main actions).

Consider the following trivial example:

goal hello {
    run {
        check {
            file-exists("/tmp/a.txt") =>
                ok();
        }
        say("hello, world!");
    }
}

Let’s run this Ops program on the terminal:

$ rm -f /tmp/a.txt

$ opslang --run hello.ops
hello, world!

$ touch /tmp/a.txt

$ opslang --run hello.ops

$

So the first run fails the check {} phaser due to the nonexistent /tmp/a.txt file, and thus the main action say("hello, world!") of the run {} action phaser is executed and generates a line of output. And for the second run, because the file /tmp/a.txt now exists, the check phaser now succeeds, and thus the main action is no longer executed this time.

Back to TOC

Goal Dependencies

A goal can mention some other goals as its dependencies. These dependency goals’ default action phasers will automatically get checked and run before the current goal.

For instance,

goal foo: bar, baz {
    run { say("hi from foo") }
}

goal bar {
    run { say("hi from bar") }
}

goal baz {
    run { say("hi from baz") }
}

This goal foo defines two dependencies, bar and baz. These two goals will run before foo runs. Let’s just try this example out on the terminal (assuming this example is in the file deps.ops):

$ opslang --run deps.ops
hi from bar
hi from baz
hi from foo

Back to TOC

Goal Parameters

Just like user actions, goals can define their own parameters. The parameter syntax is very similar to that of user actions. For example,

default goal all;

goal foo(Int $n, Str $s) {
    run {
        say("n = $n, s = $s");
    }
}

goal all {
    run {
        foo(3, "hello"),
        foo(-17, "world");
    }
}

Running this Ops program yields the output

n = 3, s = hello
n = -17, s = world

It is also possible to invoke parameterized goals directly from the command-line interface. Consider the following Ops program:

goal foo(Int $n, Str $s) {
    run {
        say("n = $n, s = $s");
    }
}

goal bar(Int $m) {
    run {
        say("m = $m");
    }
}

And assuming this file is named param-goal.ops, and we can run it like this:

$ opslang --run param-goal.ops 'foo(32, "hello")'
n = 32, s = hello

or passing goal arguments by names, like this:

$ opslang --run param-goal.ops foo n=32 s=hello
n = 32, s = hello

It is also possible to invoke multi-goals from the command-line interface. and we can run it like this:

$ opslang --run param-goal.ops 'foo(32, "hello")' 'foo(34, "world")' 'bar(66)'
n = 32, s = hello
n = 34, s = world
m = 66

or passing goal arguments by names, like this:

$ opslang --run param-goal.ops foo n=32 s=hello foo n=34 s=world bar m=66
n = 32, s = hello
n = 34, s = world
m = 66

The double-quoted strings and single-quoted strings used in the goal specifiers follow the same semantics of the Ops language’s own. It is just that variable interpolation is always disabled in this context. Consider the following examples:

$ opslang --run param-goal.ops 'foo(0, "hello\nworld!")'
n = 32, s = hello
world!

$ opslang --run param-goal.ops "foo(0, 'hello\nworld')"
n = 32, s = hello\nworld

For example, using the passing-arguments-by-names way:

$ opslang --run param-goal.ops foo n=0 's="hello\nworld!"'
n = 32, s = hello
world!

$ opslang --run param-goal.ops foo n=0 "s='hello\nworld'"
n = 32, s = hello\nworld

Boolean literals true and false are also supported here.

The value types must exactly match the parameter types since no implicit type conversion would ever happen in this context.

In the compiler mode, we can do the following:

$ opslang -o param-goal.lua param-goal.ops

$ resty -e 'require "param-goal".main(arg)' /dev/null 'foo(56, "world")'
n = 56, s = world

$ resty -e 'require "param-goal".main(arg)' /dev/null foo n=56 s=world
n = 56, s = world

Back to TOC

Goal Memoization

For any single Ops program run, every goal will be run at most once. The Ops runtime keeps track of goals successfully executed (or checked) so that no goals will be run more than once.

For goals taking arguments, each combination of the actual argument values for each goal call will be recorded instead. So foo(32) and foo(56) will both run without problems.

This is also one of the key difference between goals and user actions.

Consider the following example:

goal foo {
    run {
        say("hi from foo");
    }
}

goal all {
    run {
        foo,
        foo,
        foo;
    }
}

Running this example will only yield a single line of output:

hi from foo

This is because the latter two foo goal calls get skipped due to the goal result memoization.

Back to TOC

Returning From Within Goals

It is possible to use the return statement to return from the goal’s main action chain. It is just that goals do not support returning any values (yet). Use top-level variables to store computed results from goal executions.

Back to TOC

Modules

The Ops language supports modular programming, which means it can be used to construct large-scale programs with ease.

Each Ops module must be in its own .ops file. For example, module foo should be in file foo.ops alone, while module foo.bar should be in the file foo/bar.ops.

Each Ops module file should start with a module statement specifying the module name, as in

module foo;

or

module foo.bar;

The module name must be the same as indicated by the file path name.

To load an Ops module from another Ops module or an Ops main program file, just use the use statement as below:

use foo;
use foo.bar;

The first statement loads the foo module by looking for the foo.ops file in the module search paths while the latter looks for the foo/bar.ops file instead.

Each module can only be loaded once. Duplicate use statement for the same module is a “no-op”.

Actions, exceptions, and goals defined in a module foo can always be referenced in other .ops files with full-qualified names like foo.bar where foo is the module name while the bar is the symbol name defined in the foo module.

Back to TOC

Module Search Paths

Ops programs search Ops module files in a list of directories. This list of directories are called the module search paths. The default search paths consist of two directories:

  1. the current working directory, ..
  2. the <bin-dir>/../opslib directory where <bin-dir> is the directory holding the opslang command-line utility’s executable file itself. This default search path is for loading the standard Ops module std, which defines many of the standard functions.

The user can add its own search paths by specifying the -I PATH option for one or more times to the opslang command-line utility, as in

opslang -I /foo/bar -I /blah/ ...

New paths added by the -I option will be appended to the existing search paths. Note the order of the paths is important. The Ops runtime will simply use the first matched path and skip the remaining ones.

Back to TOC

The Std Module

The Ops compiler ships with a standard Ops module called std. This std module defines many of the standard functions and is automatically loaded by default for any Ops programs unless the opslang command-line option --no-std-lib is specified.

Back to TOC

Exporting Module Symbols

Symbols like goals, user actions, and user-exceptions defined in a module can be exported to the loader of the module. This is achieved by adding the is export trait to the corresponding user action declarations, user exception declarations, or goal declarations, as in

module foo;

action do-this () is export {
    say("do this!");
}

Then in the main program file main.ops, we can load this module and start using the exported do-this() user action right away:

use foo;

goal all {
    run {
        do-this();
    }
}

Running this program yields the output

do this!

It always works to reference symbols defined in a loaded module via the fully-qualified names no matter whether those symbols are exported or not. For the preceding example:

use foo;

goal all {
    run {
        foo.do-this();
    }
}

Exporting goals is similar:

module foo;

goal foo (Str $name): bar, baz is export {
    run {
        say("running foo: $name!");
    }
}

goal bar {
    run {}
}

goal baz {
    run {}
}

Note that dependencies of an exported goal are not exported automatically. If you want to export those dependencies, then the is export trait must be added to those goals as well.

To export a user exception, we can write

exception foo is export;

Be careful not to export too many symbols since it may pollute the module loaders’ namespace.

Back to TOC

Command-Line Interface

TODO

Back to TOC

Running in Special Environments

When running in the OpenResty or NGINX server environment, it is required to specify the TERM=xterm-256color environment explicitly. For example, we can put the following line to the nginx.conf configuration file:

env TERM=xterm-256color;

It is also a good idea to explicitly specify the PATH env as well:

env PATH=/usr/local/bin:/usr/bin:/usr/sbin:/bin:/sbin;

Or in the shell commands embedded in the Lua code.

It can also be desired to use the stdbuf tool to make line-buffered output possible in such environment (like using OpenResty’s ngx.pipe API), as in

local shell = require "resty.shell"

local ok, stdout, stderr, reason, status =
    shell.run([[stdbuf -oL opslang --run my_tool.ops]], nil)

The same applies to other non-login shell environment like cronjobs (configured by the crontab command).

Back to TOC

Lua API

TODO

Back to TOC

Standard Functions

The Ops language provides a rich set of standard functions. Some of them are implemented as built-in directly by the compiler while the others are implemented in the std module.

The user can override any of these built-in functions with their own user actions using the same name, though it is generally not recommended due to the risk of breaking Ops’s core system.

Below is the documentation for all our standard functions in the alphabetic order.

Back to TOC

abs

amazon-major-ver

append-to-path

assert

assert-host

assert-user

backspace

bad-cmd-out

basename

break

ceil

centos-ver

chdir

check-git-uncommitted-changes

chomp

clear-prompt-host

clear-prompt-user

closed

cmd-out

cout

ctrl-c

ctrl-d

ctrl-l

ctrl-v

ctrl-z

cur-screen

cwd

deb-codename

default-read-timeout

default-send-timeout

defined

del-key

die

dir-exists

do-ctrl-c

done

echoed-out

elems

error

esc-key

exit

exit-code

exit-sh

failed

false

file-exists

file-mod-time

file-not-empty

file-size

floor

found-prompt

goto

home-dir

hostname

is-regular-file

join

split

new-screen

nil

nok

nop

now

ok

os-id

os-is-amazon

os-is-deb

os-is-debian

os-is-fedora

os-is-opensuse

os-is-redhat

os-is-sles

os-is-ubuntu

out

pop

pop-prompt

prepend-to-path

print

prompt

prompt-host

prompt-user

push

push-prompt

raw-out

read-timeout

redhat-major-ver

redo

return

say

send-cmd

send-key

send-text

send-timeout

setup-sh

sh-var

shift

sleep

strlen

substr

encode-json

switch-screen

wait-all-async

Wait until all async blocks terminal.

Back to TOC

throw

timeout

to-bool

to-int

to-num

too-many-tries

too-much-out

trim

true

typing-delay

unshift

warn

whoami

with-std-prompt

Back to TOC

Author

Yichun Zhang <yichun@openresty.com>, OpenResty Inc.

Back to TOC

Copyright (C) 2019-2020 by OpenResty Inc. All rights reserved.

This document is proprietary and contains confidential information. Redistribution of this document without written permission from the copyright holders is prohibited at all times.

Back to TOC