OpenResty OpsLang™ User Manual
The Ops Language User Manual
Table of Contents
Description
This manual documents the Ops language from the user’s perspective.
Some of the features documented in this manual may not have been implemented in the current Ops language implementation. The Ops language development team is quickly catching up though. We try marking all not-yet-implemented features in the documentation. When in doubt, please contact the OpenResty Inc. company directly.
This document is still a draft. Many details are still subject to change.
Convention
We use the word opslang for the Ops language throughout this document
for convenience.
Example code with problems will get a question mark, ?, at the beginning
of each of its lines. For example:
? $ echo hello,
? stream {
? }
Program Layout
On the highest level, an Ops program consists of one main program file and an optional set of module files.
For each .ops file, it consists of various declaration statements of the following
kinds:
- module declaration statements,
- goal and default goal declarations,
- top-level variable declarations,
- exception declarations, and
- user action declarations.
The first one must appear at the beginning of every Ops module file while non-module files must not have it.
The main Ops program file must contain at least one goal declaration.
Other things are optional.
To follow the old tradition in the programming languages’ world, a “hello world” program in the Ops language looks like this:
goal hello {
run {
say("Hello, world!");
}
}
Assuming this program is in the file hello.ops, We can run it with a single
command like this:
$ opslang --run hello.ops
Hello, world!
Or use opslang as a compiler to generate the Lua module file named
hello.lua:
opslang -o hello.lua hello.ops
And then run the Lua module generated with the resty command-line utility
provided by OpenResty:
$ resty -e 'require "hello".main(arg)'
Hello, world!
Bits and Pieces
Identifiers
Identifiers in opslang is one or more words connected by dashes. A word is a sequence of alphanumeric characters and underscores. An underscore character cannot appear at the beginning of a word, however. Below are some examples of valid opslang identifiers:
foo
hisName
uri-prefix
Your_Name1234
a-b_1-c
The Ops language is a case-sensitive language. So identifiers like foo and
Foo do mean completely different things.
Identifiers cannot be one of the language keywords, except for variable names.
Keywords
The Ops language has the following keywords:
for while if while func action
ge gt le lt eq ne
contains contains-word suffix prefix
my our macro use INIT END
as rx wc qw phase goal
exception return label for async
lua
Qualified names
Qualified names can be used to referencing symbols in other Ops modules or referencing a particular action phasers in a goal.
For example, std.say is to explicitly reference the function say() in
the std module or namespace. And all.run is referencing the run action
phaser in the goal all.
It is also possible to combine these two cases. For instance, foo.all.run
is the run action phaser of the goal all defined in the module foo.
Variables
Variable names consist of two parts: a leading special character called a sigil and a following identifier. The sigil is used to denote the type of the variable. The following three different sigils are supported:
$is for scalar variables@is for array variables%is for hash variables
Scalar variables hold simple values like numbers, strings, booleans, and quantities.
Array variables are a ordered list container for simple values.
Hash variables are an unordered list for key-value pairs.
Variables are usually declared by the my keyword as follows:
my Int $count;
my Str $name;
my Str @domains;
my Bool %map{Str};
Variable declarations may take an initial value too, as in:
my Int $count = 0;
my Str @domains = qw/ foo.com bar.blah.org /;
Each scalar variable declared in any scopes can only have one data type throughout its lifetime. And each variable’s data type must be able to be determined at compile time. There are 4 data types for scalar variables:
StrNumIntBool
The Ops language does not support other data types in other contexts, however,
like Exception and Nil.
The user must explicitly specify a data type for any scalar variables defined
by my or defined as user action parameters.
For convenience, multiple variables of the same type can be declared by a single
my statement, as in
my Str ($foo, $bar, $baz);
or even mixing scalar, array, and hash variables:
my Str ($foo, @bar, %baz{Int});
This form of variable declarations cannot take any initializer expressions, however.
Array Variables
Array variables hold a linear list of simple values called an array.
Below is an example:
goal all {
run {
my Int @ints;
push(@ints, 32),
push(@ints, -3),
push(@ints, 0),
say("index 0: ", @ints[0]),
say("index 2: ", @ints[2]),
say("array: [@ints]");
}
}
As we can see, the element type of arrays is specified between the my keyword
and the @ints variable name in the declaration.
The push standard function appends new elements to the end of the
array. Multiple elements can be appended at the same time in a single push()
call.
Individual elements can be later accessed by indexes like @arr[index]. This
notation can also be used to set a new value for a particular element, as in
@ints[2] = 100
The full array can also be interpolated in string literals just like scalar variables.
Running this int-array.ops example program yields
$ opslang --run int-array.ops
index 0: 32
index 2: 0
array: [32 -3 0]
There are other standard functions which operate on arrays:
- the pop() function removes and returns the last element of an array;
- the shift() function removes and returns the first element of an array;
- the unshift function prepends one or more elements to the beginning of an array;
- the elems function returns the length of an array.
When being used in boolean contexts, array variables are evaluated to true when they are non-empty, false otherwise. For instance:
goal all {
run {
my Num @nums;
{
@nums =>
say("array is NOT empty"),
done;
true =>
say("array is empty");
};
}
}
Running this array-cond.ops example gives
$ opslang --run array-cond.ops
array is empty
We can initialize an array variable with a literal array, as in
goal all {
run {
my Num @nums = (3.14, 0);
{
@nums =>
say("array is NOT empty"),
done;
true =>
say("array is empty");
};
}
}
Running this example now gives the output array is NOT empty since the array
has 2 elements now.
Array variables can be iterated through via the for statement.
Hash Variables
Hash variables hold dictionary values mapping from keys to simple values.
Below is a simple example:
goal all {
run {
my Num %ages{Str} = (dog: 1.5, cat: 5.4, bird: 0.3);
say("bird: ", %ages<bird>),
say("dog: ", %ages{'dog'}),
say("cat: ", %ages{"cat"}),
say("tiger: ", %ages<tiger>);
}
}
Running this ages.ops example program gives
$ opslang --run ages.ops
bird: 0.3
dog: 1.5
cat: 5.4
tiger: nil
Here we first declare a hash variable named %ages (note the % sigil). Its
key type is Str while its value type is Num. At the same time we initialize
the hash variable with a [literal hash value)(#literal-hash-values) which
specifies the age numbers for 3 animals, dog, cat, and bird. And then we
output the values of this hash variable for various keys. Note that the tiger
key does not exit in the hash value, so we get a nil value for it.
As we can see from this example, when the string key is a word, we could use
the %hash<word> syntax as a shorthand for the more general forms %hash{'word'}
and %hash{"word"}. In general, to read the value of a specified key, we can
write %hash{key}; and to newly set a value or overwrite the existing value
of a specified key, we can write %hash{key} = value.
One can also use scalar variables as the key, as in
my $key = 'dog';
say("$key: ", %ages{$key});
Keys of other data types are also supported, for instance,
goal all {
run {
my Str %weekdays{Int} = (1: "Monday", 2: "Tuesday", 3: "Wednesday",
4: "Thursday", 5: "Friday");
say(%weekdays{3}),
say(%weekdays{5});
}
}
Running this weekdays.ops example program gives
$ opslang --run weekdays.ops
Wednesday
Friday
When being used in boolean contexts, hash variables are evaluated to true when they are non-empty, false otherwise. For example:
goal all {
run {
my Int %ages{Str};
{
%ages =>
say("hash NOT empty"),
done;
true =>
say("hash empty");
};
}
}
Running this hash-empty.ops example program gives
$ opslang --run hash-empty.ops
hash empty
Hash variables cannot be interpolated in
double-quoted strings or
shell-quoted strings. The % character is always
interpreted literally in those contexts.
Hash variables can be iterated through via the for statement.
Capture Variables
Special capture variables are provided to hold sub-match captures of the previous
regex matching (if any). For example, variable $1 holds the matched substring for
the first numbered sub-match capture group, while $2 holds the second capture group,
$3 for the 3rd, and etc. The special variable $0 holds the fully matched substring
for the whole regex, regardless of the sub-match capturing groups.
Consider the following example:
goal all {
run {
my Str $s = "hello, world";
{
$s contains /(\w+), (\w+)/ =>
say("0: [$0], 1: [$1], 2: [$2]"),
done;
true =>
say("not matched!");
};
}
}
Running this Ops program yields the output
0: [hello, world], 1: [hello], 2: [world]
Function Names and Function Calls
The language extensively use functions for predicates and actions in rules. The user can also define their own functions (as user actions) if they want to.
Standard functions and user-defined actions can be used in the exactly same way.
In fact, many standard functions are indeed implemented as “user actions” in
the std module shipped with the opslang compiler.
Function names are represented by identifiers directly, no sigils involved.
Function calls are denoted by a function name followed by a pair of parentheses enclosing any arguments, as in:
say("hello, ", "world")
Function calls without any arguments can omit the parentheses. For example:
redo()
can be simplified to:
redo
Arguments can be passed either by positions or by names. The say() builtin
action function, for example, accepts positional arguments as the response
body message pieces, as the previous example demonstrates. Named arguments
are passed with the argument name and a colon character as the prefix,
as in:
goto(my-label, tries: 3);
This goto() call takes a named argument called tries with the value 3.
Builtin functions may require certain arguments to be passed by names, and the others passed by positions. Please consult the documentation of specific builtin functions for the actual usage.
There are also some syntactic constructs which are equivalent to function calls. For example, shell quoted strings used directly as an action and dollar actions.
Literal Strings
Single-Quoted Strings
Single-quoted strings are literal string values enclosed by single quotes
(''), as in
'foo 1234'
The characters $ and @ are always their literal meaning. Scalar and array
variables can never be interpolated.
Only the following escaping sequences are supported in single-quoted strings:
\'
\\
Any other appearances of the \ character in a single-quoted string literal
will be interpreted as a literal \.
Double-Quoted Strings
Double-quoted strings are literal string values enclosed by double quotes
(""), as in
"hello, world!\n"
The following escaping sequences are supported in double-quoted strings:
\a
\b
\f
\n
\r
\t
\v
\\
\0
\$
\@
\%
\'
\"
Scalar and array variables can be interpolated into double-quoted string literals, as in:
"Hello, $name!"
"Names: @names"
If there are ambiguous characters coming after the interpolated variable names, then we can use curly braces to disambiguate, as in
"Hello, ${name}ism"
"Hello, @{names}ya"
For convenience, an alternative form of variable interpolation syntax (similar to the shell-quoted strings) is also supported:
"Hello, $.name!"
"Hello, @.names!"
Or with the curly braces form:
"Hello, $.{name}!"
"Hello, @.{names}!"
So literal $ and ‘@’ characters in a double-quoted string must be escaped
by \, to avoid unwanted interpolations, as in:
"Hello, \$name!"
Long-Bracketed Strings
Long-bracketed strings are string literals enclosed by Lua-style long brackets.
Examples of long brackets are [[...]], [=[...]=], [==[...]==], and etc.
All of the characters inside a long brackets always take their literal values and there are no special characters at all. For example, the literal
[=[\n"hello, 'world'!"\]=]
is equivalent to the single-quoted string '\n"hello, \'world\'!"\\'.
For convenience, if the first character inside the long brackets is a newline character, then this newline character will be removed from the final string value. So the following long-bracketed string
[=[
hello world
]=]
is equivalent to the double-quoted string
"hello world\n".
Shell-Quoted Strings
Shell-quoted strings provide a convenient notation for embedding shell command strings into Ops programs. It takes various forms:
sh/.../
sh{...}
sh[...]
sh(...)
sh'...'
sh"..."
sh!...!
sh#...#
Similar to long-bracketed strings, all the characters inside the shell-quotes are interpreted literally. It is possible to escape the quote characters with double-quotes, but the escaping sequence itself will still be part of the final interpreted string value.
The Ops compiler always parse the parsed string values in the shell quotes as Bash commands. So never use arbitrary strings in this syntactic construct.
Variable interpolation using the form $foo and @foo are not supported
in shell-quotes since the Bash language itself uses this notation. So we
support another “dot form” of variable interpolation in shell quotes, as in
sh/echo "$.name!"/
or use curly braces to avoid ambiguity with any following characters:
sh/echo "$.{name}ism"/
For convenience, this “dot form” of interpolation syntax is also supported in double-quoted strings and regex literals.
Unlike other contexts doing variable interpolation, the shell quotes also perform proper shell string quoting and escaping for interpolated Ops variables according to the shell context where the variables are used. For example, variables used inside and outside Bash’s double quotes are escaped and quoted differently according to the Bash language’s syntax. Consider this variable to be interpolated:
$name = "hello\n\"boy\"!";
It takes the literal value
hello
"boy"!
And when we interpolate it into shell’s double quotes,
sh/echo "$.name"/
we will get this final shell command string value to be sent to the terminal emulator:
echo "hello
\"boy\"!"
(Note that ! does not require escaping since the standard function
setup-sh() will always disable the history feature of Bash.)
On the other hand, if we interpolate the $name variable outside any shell
double quotes, then it is escaped differently:
sh/echo $.name/
yields the final shell command string
echo hello\
\"boy\"\!
One important caveat about using interpolated variables inside or outside shell double quotes is that the latter would not escape space characters and thus the scalar variable value may end up being multiple shell values, as in
my Str $files = 'a.txt b.txt';
sh{ls $.files},
stream {
found-prompt => break;
}
This example is equivalent to
sh{ls a.txt b.txt},
stream {
found-prompt => break;
}
Basically the $files variable value is interpreted as two separate arguments
for the shell command ls. For this reason, if the file path may contain
spaces, it is important to always use double quotes to enclose the interpolated
Ops variable, as in
my Str $file = '/home/agentzh/My Documents/a.txt';
sh{ls "$.file"},
stream {
found-prompt => break;
}
Without the double quotes in this example, the ls command will try to find
two separate files, /home/agentzh/My and Documents/a.txt.
The shell quotes also support other shell value contexts like $(...),
back-quotes (`...`), $((...)), $"...", $'...', and etc.
Variable interpolation is disabled in shell’s single quotes, just like the shell language itself. For example,
sh/echo '$.foo'/
will result in the final shell command string echo '$.foo'.
One can always output the final value of the shell-quoted string but feeding
it as an argument to the print() or say() function calls, as in
say(sh/echo $.foo/)
One should avoid using control characters like tabs and those non-printable
characters (like the ESC character) in shell quotes.
When shell quoted strings are used directly as actions, as in this rule:
true =>
sh/echo hi/,
done;
then it would be automatically converted to a std.send-cmd() function call
with this shell-quoted string as its argument, as in
true =>
std.send-cmd(sh/echo hi/),
done;
Array variables can also be interpolated in this context just as with double-quoted strings. For example:
my Str @foo = qw(hello world);
goal all {
run {
say(sh/echo @.foo "@.foo"/);
}
}
Running this example yields the output
echo hello world "hello world"
Dollar Actions
Dollar actions serve as a kind of convenient syntactic sugar to save the
Ops programmers’ typing. A dollar action $ CMD is a shorthand for the
following Ops code snippet:
std.send-cmd(sh/CMD/), # the actual quote character does not matter here
stream {
found-prompt => break;
}
The / quoting character used above is arbitrary. The user is
free to use the slashes in the dollar action.
The CMD after the dollar sign and the space is a user shell command
string as specified in the shell-quoted strings.
The command string extends all the way to the end of the line, excluding
any line-trailing comma or semicolon characters.
Thanks to the implicit stream { found-prompt => break; } block, a
dollar action usually won’t complete until the shell command finishes
running (i.e., a new shell prompt appears).
When using dollar actions in an action chain (like in a rule’s consequent part), the comma separator is still required, as in
true =>
$ cd /tmp/,
$ mkdir foo,
done;
Make sure you do not add any whitespace characters between the comma and the newline character. It is also sometimes required to add a semicolon right before the newline character if the dollar action is at the end of the action chain, as in
true =>
$ cd /tmp;
true =>
say("...");
It is important to note that it is still possible to override the
default stream {} block introduced implicitly by the dollar action
itself. Just put an explicit stream {} block right after the dollar
action, as in
$ echo hi,
stream {
out contains-word /hi/ =>
say("hit!");
found-prompt =>
break;
}
Always remember to add the found-prompt rule at the end of your own
stream {} block, otherwise your dollar action’s shell command won’t
wait for the shell prompt at all, leading to timeout errors and etc.
Numeric Constants
A numeric constant can be written as one of the following forms:
1527
3.14159
-32
-3.14
78e-3 (with a decimal exponent)
0xBEFF (hexadecimal)
0157 (octal)
Regex Literals
A regex literal is for specifying a Perl-compatible regular expression value.
It is denoted by the keyword rx with a quoting structure. Below are some
examples:
rx{ hello, \w+}
rx/ [0-9]+ /
rx( [a-zA-Z]* )
rx[ [a-zA-Z]* ]
rx" \d+ - \d+ "
rx' ([^a-z][a-z]+) '
rx! ([^a-z][a-z]+) !
rx# ([^a-z][a-z]+) #
The user is free to use curly braces, slashes, parentheses, double quotes,
or single quotes in the regex literals. They are all equivalent, except
the requirement on what specific quoting characters need to be escaped
inside the regex string. For example, in the rx(...) form, use of the
slash character (/) inside the regex string does not require any escaping.
One or more options can be specified on the regex, for example:
rx:i/hello/
dictates a case-insensitive match of the pattern hello. Similarly:
rx:x/ hello, world /
makes whitespace characters used in the pattern string become insignificant,
and thus the regex above matches string "hello,world" but not
"hello, world ". Whitespace characters used in character classes, like
[a b], however, are still significant even when the :x option is specified.
Multiple options can be specified at the same time by stacking them together, as in
rx:i:s/hello, world/
When no options are to be specified, we can also save the rx prefix and just
use slashes to indicate a regex literal, as in
/\w+/
/hello world/
In Ops’s regex literals, the meta character . always match any character, including
the newline character ("\n") and the special pattern \s always match any whitespace
characters including the newline.
Wildcard Literals
A wildcard literal is for specifying a string matching pattern using
the UNIX-style wildcard syntax. It is denoted by the keyword wc with
a quoting structure, for instance:
wc{foo???}
wc/*.foo.com/;
wc(/country/??/);
wc"[a-z].bar.org"
wc'[a-z].*.gov'
As with regex literals, wildcard literals can also take flexible quoting characters.
Three wildcard meta patterns are supported: * for matching any sub-string,
? for matching any one single character, and [...] for character classes.
One or more options can be specified on the wildcard, for example:
wc:i/hello/
dictates a case-insensitive match of the pattern hello.
Literal Arrays
Literal arrays provide an convenient way to specify constant array values. The general syntax is as follows:
(elem1, elem2, elem3, ...)
Below is an example:
action print-int-array(Int @arr) {
print("[@arr]");
}
goal all {
run {
print-int-array((3, 79, -51));
}
}
Running this print-int-array.ops example gives
$ opslang --run print-int-array.ops
[3 79 -51]
Literal arrays can also be used to initialize an array variable or set the default value of an array-typed user action parameter.
For convenience, a comma after the last element of the value list is allowed, as in
(3,14, 5, 0,)
Quoted Words
Quoted words provide a convenient way to specify a constant literal string array without typing too many string enclosing quotes.
It is denoted by the keyword qw and a subsequent flexible quoting construct.
For example:
qw/ foo bar baz /
is equivalent to:
("foo", "bar", "baz")
Like regex and wildcard literals, the user can choose from various quoting characters for the quoting construct used in quoted words, for instance:
qw{...}
qw[...]
qw(...)
qw'...'
qw"..."
qw!...!
qw#...#
Literal Hash Values
Literal hash values provide a convenient way to specify constant hash values. The general syntax is like this:
(key1: value1, key2: value2, key3: value3, ...)
Below is an example:
action print-dog-age (Int %ages{Str}) {
say("dog age: ", %ages<dog>);
}
goal all {
run {
print-dog-age((dog: 3, cat: 4));
}
}
Running this print-dog-age.ops example gives the output
$ opslang --run print-dog-age.ops
dog age: 3
Literal hash values can also be used to initialize a hash variable or set the default value of a hash-typed user action parameter.
For convenience, a comma after the last key-value pair of the list is allowed, as in
(dog: 3, cat: 5,)
Booleans
Boolean values are presented by the values of the builtin function calls
true() and false(), respectively. All the relational expressions evaluate
to boolean values as well.
The following values are considered “conditional false”:
- number 0
- string “0”
- the value of
false() - an empty string
- an empty list or array
- an empty hash table
All other values are considered “conditional true”.
Function calls true() and false() are often abbreviated to just true
and false.
Comments
A comment starts with the character #, and continues to the end of
the current line. For example:
# this is a comment
Block comments are also supported, as in
#`(
This is a block
comment...
)
Note the backtick character and left parenthesis character directly following
the # character. Parentheses can still be used inside block comments as
long as they are properly paired. Even nested parentheses are allowed too:
#`(
3 * (2 - (5 - 3))
)
Exceptions
Ops supports an efficient and powerful exception model kinda similar to those found in other programming languages like C++ and Java.
There are two kinds of exceptions, standard ones and user-defined ones. Standard exceptions are as follows (with brief description for each exception):
timeoutReading from or writing to the terminal emulator (tty) timed out.
failedThe previous (shell) command failed with a nonzero exit code.
found-promptThe previous (shell) command completed (regardless of the exit code). Note that this exception never throws since it is nonfatal.
errorSome weird errors happen when reading from or writing to the terminal emulator (tty), excluding the
timeoutandclosedexception cases.closedThe connections to the terminal emulator (tty) are closed prematurely.
too-much-outBufffering too much data read from the terminal emulator (tty). This limit can be tuned via the
--max-out-buf-size SIZEcommand-line option of theopslangcommand-line utility.too-many-triesThe goto, redo or some other retrying calls have attempted too many times.
To reference an exception, just do a function call with the exception name as the function name.
When an exception is referenced in a rule condition, then it is the Ops language’s way to catch an exception generated in an earlier action. Here is an example:
goal all {
run {
$ ( exit 3 ),
{
failed =>
say("cmd failed with exit code: ", exit-code);
},
say("done");
}
}
Running this Ops program yields the following output (assuming the program
file is failed-cmd.ops):
cmd failed with exit code: 3
done
The dollar action $ ( exit 3 ) in this example intentionally returns the
exit code 3 for the current Bash command, thus leading to the failed standard
exception being thrown out. In the subsequent action block, the rule with
failed as the (sole) condition effectively catches this failed exception,
and print out a message with the exit code of the previous shell command.
Because the failed exception is properly caught and handled, the execution
flow continues normally to the final say("done") action ad generates the
output.
If we remove the action block for catching the exception from the preceding example, as in
goal all {
run {
$ ( exit 3 ),
say("done");
}
}
Then the Ops program will get aborted by the uncaught failed exception and
prints out an error message immediately after executing the dollar action, as
in
ERROR: failed-cmd-uncaught.ops:3: failed: process returned 3
in rule/action at failed-cmd-uncaught.ops line 3
The failed exception is also able to capture the error message automatically
if there is indeed any, for instance,
goal all {
run {
$ ( echo "Bad bad bad!" > /dev/stderr && exit 3 ),
say("done");
}
}
yields
ERROR: failed-msg.ops:3: failed: process returned 3: Bad bad bad!
in rule/action at failed-msg.ops line 3
When an exception is used as an argument to the standard function throw,
then the throw() function call is throwing out an exception to the upper layers
of the current Ops program. If none of the upper layer catches it, then the
Ops program will abort with an error message.
User Exceptions
The Ops programmers can reuse any of the standard exceptions mentioned above,
or define their own exceptions. To declare new exceptions, just use the exception
statement as follows in the top level scope of any .ops source file:
exception too-large;
Here we define a new exception named too-large.
Multiple exceptions can be declared in the same exception statement, as in
exception foo, bar, baz;
We can use the throw standard function to throw such exceptions in our Ops programs wherever desired. And user exceptions can be caught in the exactly same way as standard ones by simply referencing the exceptions as function calls in rule conditions. For instance,
exception too-large;
action foo (Int $n) {
{
$n > 100 =>
throw(too-large);
};
}
goal all {
run {
foo(101),
{
too-large =>
say("caught too large"),
done;
},
say("done");
}
}
Running this program yields
caught too large
done
Similarly, uncaught user exceptions lead to Ops program aborts, just like standard exceptions. Consider
exception too-large;
action foo (Int $n) {
{
$n > 100 =>
throw(too-large);
};
}
goal all {
run {
foo(101),
say("done");
}
}
which yields the output
ERROR: too-large.ops:6: too-large
in action 'foo' at too-large.ops line 6
in rule/action at too-large.ops line 12
in rule/action at too-large.ops line 12
A text message can also be specified in the throw() call, as in
exception too-large;
action foo (Int $n) {
{
$n > 100 =>
throw(too-large, msg: "$n is larger than 100");
};
}
goal all {
run {
foo(101),
say("done");
}
}
which yields
ERROR: throw-msg.ops:6: too-large: 101 is larger than 100
in action 'foo' at throw-msg.ops line 6
in rule/action at throw-msg.ops line 12
in rule/action at throw-msg.ops line 12
We can see there is now a more detailed error message on the first line.
Exceptions defined in modules can always be referenced by fully-qualified
names in the form <module-name>.<exception-name>.
Labels & Goto
The Ops language provided the goto function, which is similar to the “goto” statement in the C language but much safer.
The goto() ops function can be used to jump directly to an action with
a name label either within the current scope or an outer scope.
One cannot jump across the boundaries of the current user action or the current goal action phaser.
The action which can be jumped to must carry a user-defined label, as in:
goal all {
run {
say("before"),
again:
say("middle"),
goto(again),
say("after");
}
}
Here we add a label named “again” to the action say("middle"), and then
call the goto function with the label as the only argument. Note that
we pass the again label as a function call. The call goto(again) is
equivalent to goto(again()). The again() function call evaluates to
a Label typed value corresponding to the label again defined earlier.
We can run this Ops example and see what happens (assuming the program is
in the file goto.ops):
$ opslang --run goto.ops
before
middle
middle
middle
middle
ERROR: goto.ops:8: too-many-tries: attempted to do goto for 4 times
in rule/action at goto.ops line 8
So this program terminates abnormally due to an exception thrown:
ERROR: goto.ops:8: too-many-tries: attempted to do goto for 4 times
This is because that goto() call will only allow jumping for 3 times by
default. This limit is per scope. So entering and leaving the current scope
will reset the jumping counter to zero. Also, each action labels have separate
jumping counters.
The exception thrown has the tag too-many-tries and it can
be caught by the Ops code like this:
goal all {
run {
say("before"),
again:
say("middle"),
goto(again),
{
too-many-tries =>
say("caught the too-many-tries exception");
},
say("after");
}
}
This program (assuming to be goto2.ops) can completes successfully:
$ opslang --run goto2.ops
before
middle
middle
middle
middle
caught the too-many-tries exception
after
Jumping Count Limits
We already see the default jumping count limit is 3 times and the 4th goto
call for the same label will yield the too-many-tries exception. We
can change this limit by passing the tries: N named argument to the goto
function call, as in
goto(again, tries: 10);
Now we have 10 jumps at maximum instead of 3.
Instead of limiting the count of jumps, we can also add varying delays before
each goto() jump. For example:
goto(again, init-delay: 0.001, delay-factor: 2, max-delay: 1,
max-total-delay: 10);
Here we start adding larger and larger delays (sleeping) before each goto()
retry for the again label, starting from 0.001 seconds, topped at 1 seconds,
increasing 2 times for each new retry. Once the total delays accumulated from
all preceding retries go beyond 10 seconds, the too-many-tries exception will
be thrown.
Please note that the very first goto() call will never have any delays or
sleeps at all. The init-delay setting only takes effect on the 2nd try (or
the first retry).
Please see the documentation for the goto function for more details.
Label Declaration
The preceding examples both use the goto() function to jump backward (the labeled
action being jumped to is before the goto call). It is also possible to jump
forward. But in this case the label must be declared by the label statement,
otherwise the label is used before it is declared. For instance,
goal all {
run {
label done;
say("before"),
goto(done),
say("middle"),
done:
say("after");
}
}
Here the done label is declared at the beginning of the run action phaser
of the goal all.
Running this example (assuming to be goto-fwd.ops) yields
$ opslang --run goto-fwd.ops
before
after
Note how the action say("middle") is effectively skipped by the preceding
goto() function call while the labeled action say("after") still runs.
Jumping from Check Phaser to Action Phaser
It is possible to use goto() to jump to labels defined in an outer scope
as long as it does not leave the current goal action phaser or the current
user action declaration body.
Thus it is naturally possible to jump from within a check phaser to a labeled action in the surrounding action phaser, as the following example demonstrated:
goal all {
run {
label do-work;
check {
! file-exists("/tmp/a") =>
goto(do-work);
true => ok;
}
do-work:
say("hi from all.run");
}
}
When the file /tmp/a does not exist, then this program will yield the output
hi from all.run
Otherwise no output will be generated.
Sometimes it is desired to always conditionally execute all the goals’
action phasers regardless of the ok() calls in their check phasers. It can
be conveniently achieved by passing the -B option to the opslang
command-line utility, as in
opslang -B --run foo.ops
Or as a compiler:
opslang -B -o foo.lua foo.ops
Under the hood, the -B option would simply make the ok function behave
exactly like the nok function. Among other things, it is very handy when
debugging or developing Ops programs.
Rule-Scoped Labels
Rules have their own scopes, so if you want to use label forward declarations and the goto function inside a single rule’s consequent part, then you should use the label declaration action syntax, as in
goal all {
run {
label done;
check {
true =>
label done,
say("check pre"),
goto(done),
say("NOT THIS"),
done:
say("check post");
}
say("run pre"),
goto(done),
say("NOT THIS"),
done:
say("run post");
}
}
Running this example gives the output
check pre
check post
run pre
run post
Note that we can have labels of the same name in different scopes.
Operators
The following operators are supported, in the order of their precedence:
Precedence Operators
0 post-circumfix [], {}, <>
1 **
2 unary +/-/~, as
3 * / % x
4 + - ~
5 << >>
6 &
7 | ^
8 unary !, > < <= >= == !=
contains contains-word prefix suffix
!contains !contains-word !prefix !suffix
eq ne lt le gt ge
9 ..
10 ?:
The user may use parentheses, (), to explicitly change the relative precedence
or associativity of the operators in a single expression.
Arithmetic Operators
The language supports the following binary arithmetic operators:
** power
* multiplication
/ division
% modulo
+ addition
- subtraction
For example:
2 ** (3 * 2) # evaluates to 64
(7 - 2) * 5 # evaluates to 25
The unary prefix operator - is also supported for arithmetic negation, as in:
-(3.15 * 2) # evaluates to -6.3
String Operators
The language supports the following binary string operators:
x repeat a string for several times and concatenate them together
~ string concatenation
For instance:
"abc" x 3 # evaluates to "abcabcabc"
"hello" ~ "world" # evaluates to "helloworld"
Bit Operators
The following binary bit operators are supported:
<< shift left
>> shift right
& bit AND
| bit OR
^ bit XOR
The unary prefix operator ~ is for the bit NOT operation. Do not confuse
it with the binary operator ~ for string concatenation.
The bit operators have not been implemented yet.
Relational Operators
Use of all relational operators lead to a boolean value for the current expression. Expressions using a relational operators are relational expressions.
The following binary operators compare the two operands numerically:
> greater than
< less than
<= less than or equal to
>= great than or equal to
== equal to
!= not equal to
The following binary operators are for comparing string values alphabetically:
gt greater than
lt less than
le less than or equal to
ge great than or equal to
eq equal to
ne not equal to
There are also 3 special string binary operators for pattern matching in a string value:
contains holds when the right hand side operator is "contained" in
the left hand side operator
contains-word holds when the right hand side operator is "contained" as
a word in the left hand side operator
prefix holds when the right hand side operator is a "prefix" of
the left hand side operator
suffix holds when the right hand operator is a "suffix" of the
left hand side operator
The unary prefix operator ! negates the (boolean) value of the operand.
When the right hand side of the string comparison operators is a pattern like a wildcard or a regex value, then matching anchors are assumed in the pattern. For example:
uri eq rx{ /foo } =>
say("hit");
is equivalent to:
uri contains rx{ \A /foo \z } =>
say("hit");
where the regex pattern \A only matches the beginning of the string while
\z only matches the end. The contains operator, on the other hand,
assumes no implicit matching anchors.
Similarly, the contains-word operator assumes the surrounding \b regex
anchor on both sides of the user regex.
Range Operator
The binary operator .. can be used to form a range expression, as in:
1 .. 5 # equivalent to 1, 2, 3, 4, 5
'a'..'d' # equivalent to 'a', 'b', 'c', 'd'
The value of a range expression is a flattened list of all the individual values in that range.
This operator is not implemented yet.
Ternary Operator
The ternary relational operator ?: can be used to conditionally choose
between two user expressions according to a user condition.
For example:
$a < 3 ? $a + 1 : $a
this expression evaluates to the value of $a + 1 when the expression
$a < 3 is true, or evaluates to $a otherwise.
Subscript Operators
The post-circumfix operator [] can be used as subscript of an array.
For example:
my @names = ('Tom', 'Bob', 'John');
true =>
say(@names[0]), # output Tom
say(@names[1]), # output Bob
say(@names[2]); # output John
Negative indexes are used to access elements from the end of the array,
for instance, -1 is for the last element, -2 is for the second last
one, and etc.
Similarly, the post-circumfix operator {} is used to index a hash table,
as in:
my %scores = (Tom: 78, Bob: 100, John: 91);
true =>
say(%scores{'Bob'}); # output 100
The post-circumfix operator <> is used to index a hash table via literal
string keys, for example, %scores<John> is equivalent to %scores{'John'}.
This operator is not implemented yet.
Rules
Rules provide the basic control flow language structure in the Ops language.
They replace the if/else statements in traditional programming languages
and make the code for complicated branching control flow much easier to
read and write.
Rules can be used in the following language contexts in Ops:
Basic Rule Layout
The Ops language rules come with two basic parts, a condition, and a
consequent. The condition and the consequent are connected by =>, and the
whole rule is terminated by a semicolon character. The basic form of rule is
like this:
<condition> => <consequent>;
The condition part of the rule can take one or more relational expressions,
like resp-status == 200. All the relational expressions are connected
by the comma character (,), which make all the relational expressions
AND’d together, that is, all the relational expressions must hold true
for the whole condition to be true. The conditions should have no side
effects and this property is enforced by the opslang compiler. For this
reason, the order of valuation of the relational expressions in the same
condition do not change the result of the whole condition.
The consequent part usually contains one ore more actions. Each action can have side effects like changing some request aspects, performing a 302 redirect, or changing the route the current request will go in the backend. It is also possible to specify a full block of rules as a single action (see the Action Blocks section).
Below is a simple opslang rule:
file-exists("/tmp/a.txt") =>
$ rm /tmp/a.txt;
In the condition part, file-exists("/tmp/a.txt") is a relational expression.
The file-exists() function checks the existence of the file path specified
by its argument.
This rule says that if the file /tmp/a.txt exits, then remove it. The
consequent part is a dollar action which executes the
shell command rm /tmp/a.txt.
It is worth noting that the file-exists() function takes a positional argument and no named arguments.
The Ops language is a free format language, so you can use whitespace characters freely. The indentation used before the consequent part in the example above is not significant but just for aesthetic considerations. It is totally valid to write the whole rule in a single line, for example:
file-exists("/tmp/a.txt") => $ rm /tmp/a.txt;
Multiple Relational Expressions
The user can also specify multiple relational expressions in a single condition, for instance:
file-exists("/tmp/a.txt"), hostname eq 'tiger' =>
say("hit!");
Here we have one more relational expression in the condition, i.e.,
hostname eq 'tiger', which tests if the current machine’s “hostname” is
exactly equal to 'tiger', via the hostname() standard function.
We use a different action, say("hit!"), in this
example, which sends a “hit!” line to the stdout stream of the current
running opslang program when it is executed.
Note the comma between the two relational expressions of the condition, it means AND, and the relational expressions on both sides of the comma must be true at the same time for the whole condition to be true.
The user can specify even more relational expressions in the same condition, as in:
file-exists("/tmp/a.txt"), hostname eq 'tiger', prompt-user eq 'agentzh' =>
say("hit!");
We have a 3rd relational expression, that tests whether the value of the
current user name shown in the last shell prompt string (usually automatically
configured by the standard function setup-sh()). Note that
is is also fine to replace the hostname function
call in the preceding examples with the prompt-host standard
function. The prompt-host function call is usually faster than hostname
since it does not require running a new shell command (hostname() requires
executing the hostname shell command).
Multiple Conditions
Ops rules can actually take multiple parallel conditions, connected by the semicolon operator. These conditions are logically OR’d together for the current rule.
For example:
file-exists("/foo"), prompt-host eq 'glass';
file-exists("/bar"), prompt-host eq 'nuc'
=>
say("hit!");
When either of these 2 conditions matches, the rule matches. When both of the conditions match, the rule also matches, of course.
Multiple Actions
It is possible to specify multiple actions in the same rule consequent. Consider the following example:
file-exits("/tmp/foo") =>
chdir("/tmp/"),
$ rm -f foo,
say("done!");
This example has 3 actions in the consequent. The first action calls the
chdir() builtin function and changes the current working
directory to /tmp/. The second action is a dollar action
which executes the shell command rm -f foo synchronously (but still
nonblocking from the perspective of IO). Finally, the call of the say()
function output the line done! to the stdout stream of the current
opslang program.
Unconditional Rules
Some rules may choose to run their actions unconditionally. An opslang rule,
however, always does require a condition part. To achieve the effect of
unconditional rule triggering, the user can use the always-true predicate
true() as the sole relational expression in the condition, as in:
true() =>
say("hello world");
In this rule, the action say() always runs regardless.
Because opslang function calls without any arguments can omit their
parentheses, it is preferable to write true instead of true(), as in:
true =>
say("hello world");
Multiple Rules
Multiple rules specified in the same block are executed in series. The rule written first will be executed first.
Consider the following example:
file-exists("/tmp/foo") =>
say("file exists!");
file-not-empty("/tmp/foo") =>
say("file not empty!");
When there is a /tmp/foo file with some data in it, we should get the
following output when running this opslang code snippet:
file exists!
file not empty!
The conditions of multiple rules may get optimized by the opslang compiler, however, to be matched at the same time, and may be evaluated even before any rules are actually executed. This happens when the opslang compiler finds it safe in doing so.
Action Blocks
Action blocks are formed by a pair of curly braces ({}), which also
forming a new scope for variables. Action blocks can be used wherever
an action can be used.
In the following example, we have two different $a variables since they
each belongs to a different block (or scope):
my Int $a = 3;
{
my Str $a = "hello";
},
say("a = $a"); # output `3` instead of `hello`
Rules are also lexical to the containing block, just like variables. Action blocks can be used to group closely related rules together, as a whole. In such a setting, some early-executed rules may use the special action done to skip all the subsequent rules in the same block. The following example demonstrates this:
{
file-exists("/tmp/a.txt") =>
print("hello"),
done;
true =>
print("howdy");
},
say(", outside!");
If file /tmp/a.txt exists in the current system, then the output of
this Ops code snippet would be hello, outside!.
Note how the done action in the 1st rule skips the execution
of the 2nd rule. On the other hand, in the absence of the file
/tmp/a.txt, it would yield the output howdy, outside!, since the
1st rule does not match.
Action blocks can be nested to an arbitrary depth, as in:
file-exists("a") =>
say("outer-most"),
{
true =>
say("2nd level");
{
uri("b") =>
say("3rd level");
};
};
};
Async Blocks
The Ops language supports multiple terminal sessions running in parallel. This
is achieved by the use of async blocks. The actions in an async block would
run asynchronously in a new lightweight thread with a brand new
terminal screen.
Consider this example:
goal all {
run {
my Int $c = 0;
$ FOO=32,
async {
$ echo in async,
print(cout);
},
$ echo in main,
print(cout);
}
}
A typical run yields
$ opslang --run async.ops
in main
in async
Sometimes we may also get the output lines swapped:
$ opslang --run async.ops
in async
in main
This is expected due to the asynchronous nature of the async {} block
execution.
Async blocks automatically create a new terminal screen which is separate from
the current one. In fact, it invokes the new-screen standard
function under the hood. For this reason, the total number of async blocks that
can be created is also subject to the terminal screen count limit which can
be tuned by the --max-screens N command-line option of the opslang utility.
It is prohibited to switch to a terminal screen which is not owned by the current lightweight thread. An error will be raised when such attempts are being made, like this:
test.ops:11: ERROR: screen 'foo' not owned by the current thread
Async blocks can be nested or be used in user actions. And async blocks can also create its own terminal screens.
Async blocks can also take a name, as in
async foo {
...
}
This way, the terminal screen corresponding to the async block also takes the
same name. Otherwise they get serial number names like 1, 2, 3, and etc,
depending on the order of their appearance in the ops source files.
The script log file for the screens follow the same naming convention used
by those screens explicitly created by the new-screen function.
For anonymous screens, they have special screen names like 1, 2, 3, and
etc.
The Ops program won’t terminate until all the async {} blocks and the main
thread are terminated. There are several exceptions to this rule, however:
- when a thread of execution calls the exit() standard function,
- when a thread of execution calls the die() standard function, or
- when a thread of execution has an uncaught exception like timeout,
Variable Sharing Among Threads
All Ops programs have one lightweight thread, which is the main thread created
upon startup. The user may create more lightweight threads via async {}
blocks. All these threads share the same top-level variables declared by
my. Consider the following example:
my Int $c = 0;
goal all {
run {
setup-sh,
$ FOO=32,
async {
setup-sh,
$c++,
say("c=", $c);
},
$c++,
say("c=", $c);
}
}
Running this example always gives the output
c=1
c=2
For local variables declared inside an action phaser or a
user action, if they are visible to the async {} block,
they are immediately cloned to a copy when the thread for the async {} block
is spawned. In the new thread, those local variables still have their initial
values when entering the async block, but writing new values to them will
no longer affect other threads, nor will any changes to them in any other
threads affect this new thread. For instance,
goal all {
run {
my Int $c = 0;
setup-sh,
$ FOO=32,
async {
setup-sh,
$c++,
say("async: c=", $c);
},
sleep(0.1),
$c++,
say("main: c=", $c);
}
}
Running this example always yields the output
async: c=1
main: c=1
So essentially each of these 2 threads has its own copy of this local variable
$c.
Assignment Actions
The assignment operator = is used to specify an action that assigns a
value to a variable or an expression that can be an lvalue. For example:
my $a;
$a = 3;
Like all the other actions, an assignment expression has no value for itself. So it is not allowed to embed an assignment expression in other expressions. For example, the following example will yield a compile-time error:
? my $a;
?
? say($a = 3);
This is because the assignment $a = 3 returns no value and it can only
be used as a standalone action.
The assignment:
$a = $a + 3
can be simplified using the operator +=:
$a += 3
Similarly, *=, /=, %=, x=, ~= are provided for the binary operators
*, /, %, x, and ~, respectively.
Furthermore, the postfix operator ++ can be used to simplify the += 1 case.
For example:
$a++
is equivalent to $a += 1 or $a = $a + 1. Similarly, the postfix operator
-- is provided as a shorthand for -= 1.
Like the standard = operator, all these assignment variations do not
take any values themselves and can only be used as standalone actions.
Lua Blocks
Inline Lua code snippets in opslang programs is supported to make hard things possible.
Lua code snippets are enclosed in lua blocks which can be used on the global scope, as an action, or as an ops expression.
Global Lua Blocks
Global lua blocks are used on the top-level scope of the current .ops source
file (either an Ops module file or not). Such Lua code snippets would expand
in a separate Lua variable scope on the top-level scope of the resulting Lua source
file. Any Lua local
variables defined directly in the Lua code block would not be visible outside
this lua block. If you want to define Lua variables visible to the entire
file’s Lua code, use _M.NAME instead where NAME is your variable name.
Below is a simple example:
lua {
print("init!")
}
goal all {
run {
say("done");
}
}
Running this global-lua-block.lua example gives
$ opslang --run global-lua-block.lua
init!
done
Lua Block Actions
Lua blocks can also be used directly as actions. In such case, no return values are expected from within the Lua block. Below is an example:
goal all {
run {
print("hello, "),
lua {
print("world!")
};
}
}
Running this lua-action.ops example gives:
$ opslang --run lua-action.ops
hello, world!
Lua Block Expressions
Lua blocks can be used as Ops expressions but in this case a return value type must be specified and the inlined Lua source must return a Lua value of the correspond data type, as in
goal all {
run {
say("value: ", lua ret Int { return 3 + 2 })
}
}
Running this lua-expr.ops example gives
$ opslang --run lua-expr.ops
value: 5
Variable interpolation in Lua blocks
It is possible to interpolate opslang variables in all three kinds of Lua blocks just introduced above.
All the following forms of Ops variables are supported in the inlined Lua source code:
$.NAME$.{NAME}@.NAME@.{NAME}@{NAME}%.{NAME}%{NAME}
When Ops variables are used in Lua’s single or double quoted strings, they
would retain their original string values in the final Lua values, even if
their string values contain special characters like ', ", and \.
On the other hand, use of Ops variables inside Lua’s long bracketed strings
does not enable Ops variable interpolation at all.
Interpolated Ops variables can also be used where a Lua expression is expected in the Lua code.
Below is an example for interpolating Ops array variables inside a Lua single-quoted string literal:
goal all {
run {
my Str @foo = ("hello", 'world');
say("lua: ", lua ret Str {
return '[@.foo]'
})
}
}
Running it gives the output
lua: [hello world]
Below is an example for interpolating Ops scalar variables:
goal all {
run {
my Str $foo = "hello";
my Int $bar = 32;
say("lua: ", lua ret Str {
return $.foo .. $.bar + 1
})
}
}
Running this example gives the output
lua: hello33
Special characters like $, @, %, and \ can be escaped by prefixing it
with a \ character.
For Statements
The for statement is a special kind of actions for iteratoring through
array or hash containers. The general syntax is
for CONTAINER -> VAR1 [, VAR2] {
SYMBOL-DECLARATION;
SYMBOL-DECLARATION;
...
ACTION,
ACTION,
...;
}
Below is a simple example for iterating an Int array:
goal all {
run {
my Int @arr = (56, -3, 0);
say("begin"),
for @arr -> $elem {
say("elem: $elem");
},
say("end");
}
}
Running this yields the output
begin
elem: 56
elem: -3
elem: 0
end
Note that we do not need to declare the $elem iterator variable via the my
keyword since the -> operator of the for statement already implicitly defines
this local variable in the scope of the for statement itself.
Also note that we do not specify the type for the iterator variable $elem
since its type is always determined by the element type of the array container
expression value right before the -> operator.
We can also declare two iterator variables in this example to get the array index value for the current loop iteration:
goal all {
run {
my Int @arr = (56, -3, 0);
say("begin"),
for @arr -> $i, $elem {
say("elem $i: $elem");
},
say("end");
}
}
which gives
begin
elem 0: 56
elem 1: -3
elem 2: 0
end
Again, we do not explicitly specify the type of the $i variable after ->
because its type is always Int.
We can iterator through hash table values in a similar way where 2 iterator variables must be specified for the keys and values of the hash respectively. For instance,
goal all {
run {
my Int %a{Str} = (dog: 3, cat: -51);
say("begin"),
for %a -> $k, $v {
say("key: $k, value: ", $v);
},
say("end");
}
}
yields the output
begin
key: dog, value: 3
key: cat, value: -51
end
Terminal Emulator Operations
The biggest strength of the Ops language is its native support for automating any
terminal emulators. It also has good support for transparently handling control
sequences of *NIX style terminals. Upon startup, each Ops program will always create
a pseudo terminator emulator instance (aka pseudo tty) for the rest of the
operations. This way, the Ops program can interact with the terminal interface of the
operating system in the exactly way as a human operator (usually being a developer or
a dev ops). By default, this terminal is bound to an interactive bash session,
which can be changed through the --shell SHELL option of the opslang
command-line utility.
We shall use the terms terminal and terminal emulator interchangeably in this
text. Both of them mean the pseudo terminal emulator instance just mentioned above.
To some extend, the terminal interaction capabilities provided by the Ops language and runtime are very similar to those provided by the classic Tcl/Expect tool chain, but much more powerful and much more flexible in many ways.
The full terminal interaction details are recorded in a log
file generated by the script command-line utility (usually
coming from the util-linux project). It is often very useful
to check out the full terminal logs by checking the
./ops.script.log file after running the Ops program
on the command line. This log file path can be changed
through the --script-log-file PATH option of the opslang
command-line utility.
Writing to the Terminal
The Ops program can send text data, like echo hello, to the terminal, via the
send-text standard function. It can also send special control characters
(like the enter key or the Ctrl-C key combination) to the terminal via the
send-key standard function.
Echoed-Back Typing
Please keep in mind, for many cases when writing to a terminal (though not always), the terminal would echo back what is typed to the terminal to give the feedback (for human users). What’s worse is that some times, when the typing is too fast, the echoed back output might even repeat itself completely or partially, depending on the actual contexts. The Ops program should always be prepared for such “echoed-back” output from the terminal. Fortunately, the Ops language does provide some built-in features and functions to help dealing with such bloody details so the Ops programmers usually do not have to worry about these unless they want to handle every little detail themselves on a very low level.
Reading from the Terminal
Reading data from the terminal is inherently more difficult than writing to it. This is because the size and content of the data stream output by the terminal may not be predictable at all. The Ops language provides two ways of reading from the terminal: the streaming way and the fully-buffered way. The Ops programmer is free to choose either way according to the concrete use cases.
Streaming Reading
The Ops language provided the stream blocks to do streaming reading from the terminal emulator.
Inside the stream blocks, the standard function out represents the output stream
of the terminal in an abstract way. This out function call
can be used in binary relational expressions to do
regex and string pattern matching (involved with the operators like contains, prefix, eq, and suffix). Never try using the out value like normal string values; you will get a compile time error like this:
ERROR: out(): this function can only be used as the lvalue of relational expression operators at out.ops line 5
The out stream automatically filters out any console control
sequences for showing colors or doing some other fancy things
in a real console. If you want to match against raw terminal
output with those control sequences intact, then you should
use the raw-out standard function instead.
There is another echoed-out variant which is used for matching echoed-back typing in the output stream.
Stream Blocks
A stream block has the following format:
stream {
rule-1;
rule-2;
...
}
Basically a stream block starts with the keyword stream, and then a block enclosed by
a pair of curly braces, in which one or more rules can be specified.
Consider the following example:
goal all {
run {
stream {
out suffix /.*?For details type `warranty'\.\s*\z/ =>
print($0),
break;
}
}
}
We run this Ops program with the ubiquitous UNIX bc program
like this (assuming the Ops program is in the file bc-banner.ops):
$ opslang --shell bc --run bc-banner.ops
bc 1.06
Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
So this Ops program just reads the initial banner of bc
(printed out by the bc program itself to the terminal)
and output it to its own stdout device. The actual text in
the output may differ if your system’s bc program is of
a different version (or even of a different origin).
Note the rule (and also the only rule) in the stream {}
block of the preceding example. We use the text
For details type `warranty`. as the indicator for the
end of banner. Once this regex pattern matches, we print
out everything matched by this regex by printing out the
value of the special capturing variable $0.
Note the break action in the rule above. This function call
immediately terminates the reading process of the current
stream {} block, otherwise the stream block will keep
looping and waiting for more rules to fire (until the
timeout exception is thrown).
The stream blocks can have multiple rules and those rules will be matched independently against the terminal’s output data stream in a streaming fashion. Consider another example dealing with the bc program:
goal all {
run {
stream {
out contains /\w+/ =>
say("rule 1: $0");
out contains /\w+/ =>
say("rule 2: $0");
out suffix /For details type `warranty'\.\s*/ =>
say("quitting..."),
break;
};
}
}
Running this program yields the output
rule 1: bc
rule 2: bc
quitting...
So all the 3 rules are fired in turn. The order of the rule
firing is important here. The conditions in the first 2 rules
both match the exactly the same substring and locations in
the out data stream, so their firing order is now depending
on the order of their appearance in the Ops program source file.
Consequently, the first rule would always fire before the
second rule since it appears before the second one. On the
hand, the 3rd rule matches the last sentence of the out
data stream, so it will always run after the first 2 rules.
Let’s try moving the 3rd rule before the first 2 rules and
see what happens:
goal all {
run {
stream {
out suffix /For details type `warranty'\.\s*/ =>
say("quitting..."),
break;
out contains /[a-zA-Z]+/ =>
say("rule 1: $0");
out contains /[a-zA-Z]+/ =>
say("rule 2: $0");
};
}
}
Running it yields the exactly same output:
rule 1: bc
rule 2: bc
quitting...
The location of the matched text of the first rule is after the matched text of the latter 2 rules, so even the 1st rule appears first in the Ops source code, it runs after the latter 2 rules. This is the essence of streaming processing. One never waits for the full output before doing any pattern matching. One does the matching as soon as seeing some more output data.
Also note that in the preceding example, the latter 2 rules
do not carry the break action. This is important since break
would break the execution flow immediately out of the current
stream {} block while we really want as many rules to match
and fire in this very example. We can try adding the break action
to all 3 rules in the stream block:
goal all {
run {
stream {
out suffix /For details type `warranty'\.\s*/ =>
say("quitting..."),
break;
out contains /[a-zA-Z]+/ =>
say("rule 1: $0"),
break;
out contains /[a-zA-Z]+/ =>
say("rule 2: $0"),
break;
};
}
}
Now naturally, only the first rule will have a chance to fire (assuming this code example is put into file all-break.ops):
$ opslang --shell bc --run all-break.ops
rule 1: bc
Another very important observation from the preceding examples
is that all rules in the stream block will fire no more than once. Apparently
the out contains /[a-zA-Z]+/ condition, for example, would match
many times in the bc program banner text. Whereas, we only
get the first match in the output. This is the expected behavior
according to the Ops language’s design. To make a rule fire
multiple times, one should call the redo action
at the end of the corresponding rule’s consequent part, as in
goal all {
run {
stream {
out suffix /For details type `warranty'\.\s*/ =>
say("quitting..."),
break;
out contains /[a-zA-Z]+/ =>
say("word: $0"),
redo;
};
}
}
Running this redo.ops program gives the output
$ opslang --shell bc --run redo.ops
word: bc
word: Copyright
word: Free
word: Software
ERROR: redo.ops:10: too-many-tries
in stream block rule at redo.ops line 10
in rule/action at redo.ops line 4
This time, we matched more “words” through the redo action: bc, Copyright, Free, and Software. Interestingly, the Ops
program aborts with an uncaught exception
too-many-tries after outputting 4 words. The too-many-tries
exception is thrown because the redo action in that rule
are to be fired more than 3 times. The standard function
redo does have a default counting limit of 3, just
like the standard function goto. To match and output
more words in our preceding example, we can specify the tries named argument
for the redo() call, as in
goal all {
run {
stream {
out suffix /For details type `warranty'\.\s*/ =>
say("quitting..."),
break;
out contains /[a-zA-Z]+/ =>
say("word: $0"),
redo(tries: 20);
};
}
}
This time we can output all the words in the bc banner without
getting aborted by the too-many-tries exception:
word: bc
word: Copyright
word: Free
word: Software
word: Foundation
word: Inc
word: This
word: is
word: free
word: software
word: with
word: ABSOLUTELY
word: NO
word: WARRANTY
word: For
word: details
word: type
word: warranty
quitting...
Now let’s actually use bc to do some useful work, i.e., to
do some simple arithmetic calculations like 3 + 5 * 2:
goal calc {
run {
my Str $expr;
stream {
out suffix /For details type `warranty'\.\s*/ =>
break;
},
# enter the expression:
$expr = "3 + 5 * 2",
send-text($expr),
stream {
echoed-out suffix $expr =>
break;
},
# submit the entered expression by pressing Enter:
send-key("\n"),
stream {
echoed-out prefix /.*?\n/ =>
break;
},
# collect the result in the new output:
stream {
out suffix /^(\d+)\n/ =>
say("res = $1"),
break;
};
}
}
This example is a bit lengthy since it deals with the raw
terminal interaction directly. First of all, we use a stream
block
to wait the bc to output the full banner text. Then, we
enter the expression 3 + 5 * 2 via the standard function
send-text. After that, we use another stream block
to wait the terminal to echo back the expression text we
just “typed”. Once we are sure our expression is indeed
entered, we emulate the action of pressing the Enter key
on the keyboard by calling the send-key function
(just like a human user would
do). Again, we use a new stream block to make sure this Enter
key press is well received and echoed back to us. Finally,
we use a final stream block to receive the integer number output by
bc for the calculation result.
It is important to note that we use the standard function
echoed-out to match our echoed-back typing.
The main difference between ehoed-out and out is that
the former will also automatically filter out special character
sequences introduced by the terminal’s automatic line wrapping (every terminal has a width and when the typed line exceeds that width limit, line wrapping occurs).
You may wonder why we bother waiting for the echoed back text for our own input. This is important because all our communications with bc must go through the terminal emulator, and if we type too fast, our typing may get lost, just like fast human users may sometimes lose typing in a real terminal which is slow to respond.
Running this calc.ops program yields
$ opslang --shell bc --run calc.ops
res = 13
Let’s check out the ./ops.script.log file for the raw
terminal session history:
$ cat ops.script.log
Script started on 2019-06-12 17:10:52-07:00
bc 1.07.1
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
3 + 5 * 2
13
That’s exactly what we’d expect.
Fully Buffered Reading
Fully buffered terminal output reading is achieved by defining a proper prompt and reading the string value returned by the standard function cout (or cmd-out for the long form).
The string value returned by cout is defined to be
any data between the out stream location left by the previous stream {} block and the starting location of the
currently matched prompt string through the current stream {} block.
See the GDB Prompts section for examples.
Prompt String Handling
In earlier examples with bc, we already see how to
deal with terminal interactions directly in Ops. For much
more complex interactive programs like bash and gdb, we
will need to deal with “prompt strings” (or just “prompts”) like bash-4.4$ and (gdb). One can imagine that
manually handling the prompt string in every Ops program can be
a really tedious and daunting task. Fortunately, Ops provides
built-in support for prompt string handling, through various standard functions
like push-prompt and pop-prompt, as well as the standard exception found-prompt.
GDB Prompts
Consider this example which types some gdb command and output its result:
goal all {
run {
my Str $cmd;
push-prompt("(gdb) "),
# wait for the prompt to show up:
stream {
found-prompt => break;
},
# send our gdb command:
$cmd = "p 3 + 2 * 5",
send-text($cmd),
# wait for our typing to echo back:
stream {
echoed-out suffix $cmd =>
break;
},
# press Enter to submit the command:
send-key("\n"),
stream {
echoed-out prefix /.*?\n/ =>
break;
},
# collect and output the result:
stream {
found-prompt =>
print("res: ", cout),
break;
};
}
}
In this example, we first use the push-prompt standard function
to register a new prompt string pattern, "(gdb) ", to the Ops runtime. Then in
the following stream {} block, we can use a rule with the condition found-prompt
to wait for the first appearance of such a prompt string in the terminal output stream. Once we get the gdb prompt, we
just send the gdb command, p 3 + 2 * 5, wait for the echoed-back
command typing, press an Enter key, receive its echoed-back
character too, and finally wait for the next gdb prompt. Once we get
the new gdb prompt, we use the standard function cout
to extract the output for the previously executed command (which is defined
to be the data before the currently matched prompt string,
and after the output stream byte position left by the previous
stream block (which is for the echoed-back output of the Enter key press). The use of the cout function is actually
the fully buffered reading mode.
Apparently it is built upon the detection of prompt strings
and the default streaming reading mode.
We need to pay special attention to the space character
right after the (gdb) string in our prompt string pattern. That space is necessary since prompt string matching is
always a suffix matching operation and GDB always put a
space immediately after the (gdb) string. It is part of the
real GDB prompt string.
Let’s run this gdb.ops program with gdb as our “shell”:
$ opslang --shell gdb --run gdb.ops
res: $1 = 13
Exactly we want.
Actually we could use the standard function send-cmd to simplify our example above. This function combines the operations of sending a command string, waiting for its echoed-back output, pressing the Enter key, and waiting for the echoed-back newline. The simplified version of the preceding example is as follows:
goal all {
run {
push-prompt("(gdb) "),
# wait for the prompt to show up:
stream {
found-prompt => break;
},
# send our gdb command:
send-cmd("p 3 + 2 * 5"),
# collect and output the result:
stream {
found-prompt =>
print("res: ", cout),
break;
};
}
}
Gladly the code is now much shorter and cleaner. The result is exactly the same:
$ opslang --shell gdb --run gdb2.ops
res: $1 = 13
Bash Prompts
Bash’s prompt strings are more complex than GDB’s since it is not a fixed string at all. It can be configured by the user
(by setting a prompt template string to the PS1 bash variable, for example) and may contain dynamic information due to the
use of shell variables in the prompt string template.
Below is an example trying to match a generic bash prompt string:
goal all {
run {
stream {
out suffix rx:x/^ [^\r\n]+ [\$%\#>)\]\}] \s*/
=>
push-prompt($0),
break;
},
send-cmd("echo hello"),
stream {
found-prompt =>
break;
},
say("out: [", cout, "]");
}
}
Here we first use a stream block and a regex pattern match
to find a string which looks like a shell prompt string,
and then we push the matched string to the prompt pattern
stack via the push-prompt standard function. One feature of the push-prompt function is that it
automatically canonicalize all the numbers in the prompt
string pattern to 0, so that even the shell prompt
template contains variables like $?, our prompt detection
will still succeed.
Let’s run this sh-prompt.ops program with the --shell bash option:
$ opslang --shell bash --run sh-prompt.ops
out: [hello
]
Ops provides several additional standard functions to make Bash sessions’ handling even easier. For instance, the preceding example could be simplified by using shell-quoted strings:
goal all {
run {
stream {
out suffix rx:x/^ [^\r\n]+ [\$%\#>)\]\}] \s*/ =>
push-prompt($0),
break;
},
sh/echo hello/,
stream {
found-prompt =>
break;
},
say("out: [", cout, "]");
}
}
And we can further simplify this with the dollar action feature:
goal all {
run {
stream {
out suffix rx:x/^ [^\r\n]+ [\$%\#>)\]\}] \s*/ =>
push-prompt($0),
break;
},
$ echo hello,
say("out: [", cout, "]");
}
}
Even the initial shell prompt detection can be simplified by calling the standard function setup-sh:
goal all {
run {
setup-sh,
$ echo hello,
say("out: [", cout, "]");
}
}
Of course, the setup-sh function can do a lot more than our
poor man’s shell prompt detection shown above.
Indeed, setup-sh is almost always the first call for
any Ops programs interacting with shell sessions, so the
opslang utility would automatically call it if the user
does not specify the --shell SHELL command -line option at all. Make sure
you do not make duplicate setup-sh function call at the beginning
of your goal action phasers if the opslang utility automatically
calls it already. Duplicate calls will result in reading timeout
errors since there won’t be any duplicate shell prompt
strings for us to match at shell session startup anyway.
Therefore, if we run the opslang command line without
the --shell bash option like previous examples, then
we must remove the setup-sh call from our Ops program,
as in:
goal all {
run {
$ echo hello,
say("out: [", cout, "]");
}
}
And run this sh-prompt2.ops program like this:
$ opslang --run sh-prompt2.ops
out: [hello
]
This example now finally becomes as concise as it should have been.
Nested Bash Sessions
One unique strength of Ops is the simplicity of handling nested shell sessions. Below is an example:
goal all {
run {
sh/bash/,
setup-sh,
$ echo $(( 3 + 1 )),
say("out: [", cout, "]"),
exit-sh; # to quit the nested shell session
}
}
Running this nested-sh.ops program gives the output
$ opslang --run nested-sh.ops
out: [4
]
Here we must call setup-sh explicitly for the nested
shell session ourselves and also call the exit-sh
standard function to properly quite it. The exit-sh function
essentially does the following two actions under the hood:
pop-prompt,
$ exit 0;
The pop-prompt standard function call removes
the current nested shell session’s prompt pattern and restores
the parent shell session’s prompt pattern setting. And then
it performs the shell command exit 0, after which the Ops
runtime can successfully detect the parent bash session’s prompt string
without problem.
It is important to note that we intentionally avoid using a dollar action for the first bash shell command.
Recall that the shell-quoted string action sh/bash/ above is just a shorthand notation for
the full form, send-cmd(sh/bash/). We must not use dollar
actions here for the bash command because dollar
actions would automatically wait for the prompt string with
an implicit stream block. Here we definitely need a setup-sh() call to
automatically detect a new prompt string for the new shell
session (not the old session’s prompt!).
Following the pattern in the previous example, we could nest as many
shell sessions in the same Ops program session as we’d like. The push-prompt and pop-prompt function calls simply work
on an internal prompt pattern stack which can grow as deeply as necessary.
Nested bash sessions may also be created by running other shell commands
like su, instead of calling bash directly.
Nested SSH Sessions
Dealing with nested SSH sessions is just as easy as nested bash sessions. Below is an example:
goal all {
run {
sh/ssh -o StrictHostKeyChecking=no 127.0.0.1/,
setup-sh,
$ echo $(( 3 + 1 )),
say("out: [", cout, "]"),
exit-sh; # to quit the nested ssh session
}
}
Running this nested-ssh.ops program yields
$ opslang --run nested-ssh.ops
out: [4
]
Of course, connecting to the local host is not very useful in the real world. One can choose to connect to any remote servers as desired.
As with nested bash sessions, nested SSH sessions can also nest as deeply as needed, for instance,
goal all {
run {
sh/ssh -o StrictHostKeyChecking=no 127.0.0.1/,
setup-sh,
$ echo $(( 3 + 1 )),
say("1: out: [", cout, "]"),
sh/ssh -o StrictHostKeyChecking=no 127.0.0.1/,
setup-sh,
$ echo $(( 7 + 8 )),
say("2: out: [", cout, "]"),
exit-sh, # to quit the second ssh session
exit-sh; # to quit the first ssh session
}
}
Running this program gives the output
1: out: [4
]
2: out: [15
]
This is especially useful for logging into machines which require first logging onto an intermediate “jump machine”.
Multiple Terminal Screens
The Ops language supports creating multiple terminal screens in the same program, similar to the popular GNU screen and tmux command-line utilities.
All the output handling and prompt string handling are separated among different screens. The Ops variables are shared, however.
One can create a new terminal screen via the new-screen() standard function and switch among multiple screens via the switch-screen function.
Indeed Ops always creates a default screen called “main”.
Screen names must be valid identifiers. Use of bad name strings will lead to runtime or compile-time errors.
Consider the following example:
goal all {
run {
# in the default 'main' screen
$ Foo=,
new-screen('foo'),
# in our new 'foo' screen
$ FOO=32,
switch-screen('main'),
say("main: FOO: [", sh-var('FOO'), "]"),
switch-screen('foo'),
say("foo: FOO: [", sh-var('FOO'), "]");
}
}
Here we create a second screen named foo, and we set the shell variable FOO
to an empty value in the default main screen and to 32 in our own foo
screen. This example outputs the value of this FOO shell variable in
these 2 different screens. We use the sh-var standard function to
retrieve the value of the specified shell variable conveniently.
Running this screens.ops program yields
$ opslang --run screens.ops
main: FOO: []
foo: FOO: [32]
Naturally, different terminal screens have separate shell environments.
Note that the new-screen() function call also automatically switches to the
newly created screen. And when the screen name already exists, the
new-screen() call is equivalent to a switch-screen() call.
One important behavior of the switch-screen() function call is that its effect
only lasts to the end of the current user action body or the current
user goal’s action phaser. The effect of switch-screen() will last beyond
the end of the check {} phaser and until the end of the surrounding action
phaser block.
There is a limit for how many parallel screens that can be created for a single
Ops program run. By default, the limit is 20. It can be tuned via the command
line option --max-screens, as in
opslang --run --max-screens 100 foo.ops
The generated script log files are also separate for each screen. The main
screen’s log file is still ./ops.script.log by default and the log file for
the foo screen is ./ops.script.log.foo. User-specified script log file
paths would follow the same naming scheme (the non-main screen adds the screen
name to the end of the main file name).
User Actions
The user can define their own parametric actions by grouping some other actions together. The general syntax for defining custom actions is like below:
action <name> (<arg>...) [ret TYPE] [is export] {
[<label-declaration> | <variable-declaration>]...
<action1>,
<action2>,
...
<actionN>;
}
For example:
action say-hi(Str $who) {
say("hi, $who!"),
say("bye");
}
Calling this say-hi action as say-hi("Tom") will yield the output
hi, Tom!
bye
Multiple parameters can also be specified.
User-defined actions are a great way to introduce your own vocabulary to the actions that you can use in rules and other contexts.
Recursive actions are prohibited.
Parameters must be declared with the type information, just like local
variables declared by the my keyword. Optional parameters can be specified
with a default value, as in
action say-hi(Str $who = "Bob") {
say("hi, $who");
}
Then the say-hi action is allowed to be called without any arguments,
like say-hi(), in which case, the parameter $who will get the default
value "Bob".
It is possible to specify a nil default value for such optional parameters,
as in
action foo(Int $a = nil) {
{
defined($a) =>
say("a has a value!"),
done;
true =>
say("a has no value :(");
}
}
One can use the standard function defined to test whether a variable is nil.
User Actions With A Return Value
User actions can return a value of a user-specified type. It is achieved
by specifying the ret TYPE trait right after the parameter list, as in
action world() ret Str {
return "world";
}
Note how we use the return statement to return the string value "world"
to the caller of this world user action.
The return value type can be
- a scalar type (
Str,Num,Int, orBool), - an array type (
Str[],Num[],Int[], orBool[]), or - a hash type (
Num{Str},Int{Str},Str{Num}and etc).
When the ret TYPE trait is omitted, then it is assumed to be returning
nothing or a nil value.
Goals
Goals in opslang are similar to the goals in Makefiles though they never directly
correspond to any files as Makefiles’ goals. In a sense, they are more similar to
GNU Makefiles’ “phony goals” or “phony targets”. Ops’s goals are named tasks to be
fulfilled for each run of any Ops programs. They are the entry points of Ops programs
(just like the C language’s main function).
Each Ops program must define at least one goal and they may choose to define multiple goals.
For Ops programs with multiple goals defined, they also have more than one entry points.
The default entry point is called the default goal. By default, the default goal is
the first goal defined in the Ops program, which can be overridden by the special
default goal statement.
A simplest goal definition in Ops is as follows:
goal hello {
run {
say("hello, world!");
}
}
This defines a new goal named hello which when runs prints out the line
hello, world! to the stdout stream of the current Ops program. This is also
the classical “hello world” program for the Ops language. Assuming this Ops program
is in the file hello.ops, and we can run it on a terminal:
$ opslang --run hello.ops
hello, world!
Exactly what we’d expect.
There are some things we should note in this example:
- We define a
run {}block in thehellogoal, which is called an action phaser. A goal can take several different kinds of action phasers, likerun,build,install,prep, andpackage. Therunaction phaser is one of the most commonly used action phasers of goals. When other kinds of action phasers do not make sense, just use therunone. - The
runaction phaser in this example contains a single action, which is a function callsay("hello, world!"). - The
hello.opsprogram file contains only this one goal,hello, so it is also the default goal being executed when no target goals are specified on the command line when running this Ops program via theopslangcommand-line utility.
We can explicitly specify the entry point goal on the opslang command line, as follows:
opslang --run hello.ops hello
The extra command-line argument hello specify the target
goal to run.
It is possible to specify multiple target goals as even more command-line arguments, as in
opslang --run example.ops foo bar baz
In this case, the goals foo, bar, and baz will be
executed in turn.
The Default Goal
Every Ops program must have one and only one default goal,
which is executed when the Ops program is run without any
target goals specified either on the command line or in the Lua
API call. The default goal can be
explicitly specified by the default goal statement as
follows:
default goal foo;
This statement must be on the top-level scope of an Ops program, usually near the beginning of the whole file. The
goal foo can be defined after the default goal statement.
There is no requirement for foo to be already defined before
this default goal statement.
When there is no default goal statement in an Ops program,
the first goal defined will become the default goal.
Action Phasers
A goal can define one or more action phasers. An action phaser is a named group of actions to be executed when the goal is run. The following names are predefined for phaser actions, in the order of their usual execution:
prep: do some preparation work.build: do software building (usually code compiling and code generation).install: install the software.run: run the software or other actions which cannot easily be categorized here.package: do package the built and installed software (as RPM or Deb packages, for instance).
A goal can define one or multiple action phasers, or even none at all.
When a goal defines no action phasers, it is said to be an abstract action phaser, which must have some goal dependencies.
To explicitly invoke a goal foo’s action phaser xxx, we can just
reference the phaser using the fully qualified name, foo.xxx.
For example, to reference goal foo’s action phaser run,
we just write foo.run(). The parentheses may be omitted
just like the normal Ops function calls.
When a goal defines multiple action phasers, then the action phasers will form a chain. If a later action phaser is
about to run, then any defined action phasers earlier than
this action phaser will be run in order. For example, if
a goal foo has prep, build, and run phasers defined, and
the caller runs the foo.run phaser. Before the foo.run
phaser runs, the goal will automatically run foo.prep,
and then foo.build. The foo.install phaser won’t run
since it is not defined in goal foo in this example.
A goal can also be invoked without specifying the action
phaser name, just treat the goal itself as a callable
function. For example, to invoke goal foo, we write foo().
This will result in calling the
default action phaser of that goal.
Local Symbols of Action Phaser
Action phasers may define their own local symbols, like variables and labels. Such symbols are also visible to the check phaser block if it is defined inside the current action phaser.
Below is an example:
goal hello {
run {
my Str $name = "Yichun";
say("Hi, $name!");
}
}
Running this goal yields the output
Hi, Yichun!
Default Action Phaser
When a goal is invoked without specifying an action phaser, the default action phaser will be called if there is any.
The default action phaser is defined to be 1) the latest running
phaser except the package and prep phasers,
or 2) the package phaser when it is the only phaser
defined.
For example, if a goal
foo has action phasers build, install, run, and
package phasers defined, then the default action phaser
will be the run phaser. For another example, if the goal
bar only defines the package phaser, then the default one
will be it.
Abstract goals without any action phaser defined will only execute its dependency goals.
Check Phasers
Each action phaser can take an optional check phaser, which runs a set of rules to determine whether the current action phaser’s main actions should be run at all. The check phaser is usually used to verify if the current goal has already been built or fulfilled.
Two standard functions can be used in the check phaser context
to signal success or failures immediately. These are ok
and nok functions. A call to the ok() function will
cause the current check phaser immediately succeed and the
current action phaser will be skipped running. On the
other hand, the nok() function causes the current check
phaser to immediately fail, and start executing the surrounding
action phaser’s main actions immediately.
If neither of these two functions are ever called in a check phaser, then the check phaser fails by default (i.e., need to run the action phaser’s main actions).
Consider the following trivial example:
goal hello {
run {
check {
file-exists("/tmp/a.txt") =>
ok();
}
say("hello, world!");
}
}
Let’s run this Ops program on the terminal:
$ rm -f /tmp/a.txt
$ opslang --run hello.ops
hello, world!
$ touch /tmp/a.txt
$ opslang --run hello.ops
$
So the first run fails the check {} phaser due to the nonexistent
/tmp/a.txt file, and thus the main action say("hello, world!") of
the run {} action phaser is executed and
generates a line of output. And for the second run, because
the file /tmp/a.txt now exists, the check phaser now
succeeds, and thus the main action is no longer executed this time.
Goal Dependencies
A goal can mention some other goals as its dependencies. These dependency goals’ default action phasers will automatically get checked and run before the current goal.
For instance,
goal foo: bar, baz {
run { say("hi from foo") }
}
goal bar {
run { say("hi from bar") }
}
goal baz {
run { say("hi from baz") }
}
This goal foo defines two dependencies, bar and baz.
These two goals will run before foo runs. Let’s just
try this example out on the terminal (assuming this example
is in the file deps.ops):
$ opslang --run deps.ops
hi from bar
hi from baz
hi from foo
Goal Parameters
Just like user actions, goals can define their own parameters. The parameter syntax is very similar to that of user actions. For example,
default goal all;
goal foo(Int $n, Str $s) {
run {
say("n = $n, s = $s");
}
}
goal all {
run {
foo(3, "hello"),
foo(-17, "world");
}
}
Running this Ops program yields the output
n = 3, s = hello
n = -17, s = world
It is also possible to invoke parameterized goals directly from the command-line interface. Consider the following Ops program:
goal foo(Int $n, Str $s) {
run {
say("n = $n, s = $s");
}
}
goal bar(Int $m) {
run {
say("m = $m");
}
}
And assuming this file is named param-goal.ops, and we can run it like this:
$ opslang --run param-goal.ops 'foo(32, "hello")'
n = 32, s = hello
or passing goal arguments by names, like this:
$ opslang --run param-goal.ops foo n=32 s=hello
n = 32, s = hello
It is also possible to invoke multi-goals from the command-line interface. and we can run it like this:
$ opslang --run param-goal.ops 'foo(32, "hello")' 'foo(34, "world")' 'bar(66)'
n = 32, s = hello
n = 34, s = world
m = 66
or passing goal arguments by names, like this:
$ opslang --run param-goal.ops foo n=32 s=hello foo n=34 s=world bar m=66
n = 32, s = hello
n = 34, s = world
m = 66
The double-quoted strings and single-quoted strings used in the goal specifiers follow the same semantics of the Ops language’s own. It is just that variable interpolation is always disabled in this context. Consider the following examples:
$ opslang --run param-goal.ops 'foo(0, "hello\nworld!")'
n = 32, s = hello
world!
$ opslang --run param-goal.ops "foo(0, 'hello\nworld')"
n = 32, s = hello\nworld
For example, using the passing-arguments-by-names way:
$ opslang --run param-goal.ops foo n=0 's="hello\nworld!"'
n = 32, s = hello
world!
$ opslang --run param-goal.ops foo n=0 "s='hello\nworld'"
n = 32, s = hello\nworld
Boolean literals true and false are also supported here.
The value types must exactly match the parameter types since no implicit type conversion would ever happen in this context.
In the compiler mode, we can do the following:
$ opslang -o param-goal.lua param-goal.ops
$ resty -e 'require "param-goal".main(arg)' /dev/null 'foo(56, "world")'
n = 56, s = world
$ resty -e 'require "param-goal".main(arg)' /dev/null foo n=56 s=world
n = 56, s = world
Goal Memoization
For any single Ops program run, every goal will be run at most once. The Ops runtime keeps track of goals successfully executed (or checked) so that no goals will be run more than once.
For goals taking arguments, each combination of the actual
argument values for each goal call will be recorded instead. So foo(32) and foo(56) will both run without problems.
This is also one of the key difference between goals and user actions.
Consider the following example:
goal foo {
run {
say("hi from foo");
}
}
goal all {
run {
foo,
foo,
foo;
}
}
Running this example will only yield a single line of output:
hi from foo
This is because the latter two foo goal calls get skipped
due to the goal result memoization.
Returning From Within Goals
It is possible to use the return statement to return from the goal’s main action chain. It is just that goals do not support returning any values (yet). Use top-level variables to store computed results from goal executions.
Modules
The Ops language supports modular programming, which means it can be used to construct large-scale programs with ease.
Each Ops module must be in its own .ops file. For example,
module foo should be in file foo.ops alone, while
module foo.bar should be in the file foo/bar.ops.
Each Ops module file should start with a module statement
specifying the module name, as in
module foo;
or
module foo.bar;
The module name must be the same as indicated by the file path name.
To load an Ops module from another Ops module or an Ops main program file,
just use the use statement as below:
use foo;
use foo.bar;
The first statement loads the foo module by looking for the foo.ops
file in the module search paths while the
latter looks for the foo/bar.ops file instead.
Each module can only be loaded once. Duplicate use statement for the same
module is a “no-op”.
Actions, exceptions, and goals defined in a module foo can always
be referenced in other .ops files with full-qualified names like
foo.bar where foo is the module name while the bar is the symbol
name defined in the foo module.
Module Search Paths
Ops programs search Ops module files in a list of directories. This list of directories are called the module search paths. The default search paths consist of two directories:
- the current working directory,
.. - the
<bin-dir>/../opslibdirectory where<bin-dir>is the directory holding theopslangcommand-line utility’s executable file itself. This default search path is for loading the standard Ops module std, which defines many of the standard functions.
The user can add its own search paths by specifying the -I PATH
option for one or more times to the opslang command-line utility, as in
opslang -I /foo/bar -I /blah/ ...
New paths added by the -I option will be appended to the existing search paths. Note the order of the paths is important. The Ops runtime will simply
use the first matched path and skip the remaining ones.
The Std Module
The Ops compiler ships with a standard Ops module called std. This std
module defines many of the standard functions
and is automatically loaded by default for any Ops programs unless the
opslang command-line option --no-std-lib is specified.
Exporting Module Symbols
Symbols like goals, user actions,
and user-exceptions defined in
a module can be exported to the loader of the module. This is achieved by
adding the is export trait to the corresponding user action declarations,
user exception declarations, or goal declarations, as in
module foo;
action do-this () is export {
say("do this!");
}
Then in the main program file main.ops, we can load this module and start
using the exported do-this() user action right away:
use foo;
goal all {
run {
do-this();
}
}
Running this program yields the output
do this!
It always works to reference symbols defined in a loaded module via the fully-qualified names no matter whether those symbols are exported or not. For the preceding example:
use foo;
goal all {
run {
foo.do-this();
}
}
Exporting goals is similar:
module foo;
goal foo (Str $name): bar, baz is export {
run {
say("running foo: $name!");
}
}
goal bar {
run {}
}
goal baz {
run {}
}
Note that dependencies of an exported goal are not exported automatically.
If you want to export those dependencies, then the is export trait must
be added to those goals as well.
To export a user exception, we can write
exception foo is export;
Be careful not to export too many symbols since it may pollute the module loaders’ namespace.
Command-Line Interface
TODO
Running in Special Environments
When running in the OpenResty or NGINX server environment, it is required to
specify the TERM=xterm-256color environment explicitly. For example, we can
put the following line to the nginx.conf configuration file:
env TERM=xterm-256color;
It is also a good idea to explicitly specify the PATH env as well:
env PATH=/usr/local/bin:/usr/bin:/usr/sbin:/bin:/sbin;
Or in the shell commands embedded in the Lua code.
It can also be desired to use the stdbuf tool to make line-buffered output
possible in such environment (like using OpenResty’s ngx.pipe API), as in
local shell = require "resty.shell"
local ok, stdout, stderr, reason, status =
shell.run([[stdbuf -oL opslang --run my_tool.ops]], nil)
The same applies to other non-login shell environment like cronjobs (configured
by the crontab command).
Lua API
TODO
Standard Functions
The Ops language provides a rich set of standard functions. Some of them are implemented as built-in directly by the compiler while the others are implemented in the std module.
The user can override any of these built-in functions with their own user actions using the same name, though it is generally not recommended due to the risk of breaking Ops’s core system.
Below is the documentation for all our standard functions in the alphabetic order.
abs
amazon-major-ver
append-to-path
assert
assert-host
assert-user
backspace
bad-cmd-out
basename
break
ceil
centos-ver
chdir
check-git-uncommitted-changes
chomp
clear-prompt-host
clear-prompt-user
closed
cmd-out
cout
ctrl-c
ctrl-d
ctrl-l
ctrl-v
ctrl-z
cur-screen
cwd
deb-codename
default-read-timeout
default-send-timeout
defined
del-key
die
dir-exists
do-ctrl-c
done
echoed-out
elems
error
esc-key
exit
exit-code
exit-sh
failed
false
file-exists
file-mod-time
file-not-empty
file-size
floor
found-prompt
goto
home-dir
hostname
is-regular-file
join
split
new-screen
nil
nok
nop
now
ok
os-id
os-is-amazon
os-is-deb
os-is-debian
os-is-fedora
os-is-opensuse
os-is-redhat
os-is-sles
os-is-ubuntu
out
pop
pop-prompt
prepend-to-path
prompt
prompt-host
prompt-user
push
push-prompt
raw-out
read-timeout
redhat-major-ver
redo
return
say
send-cmd
send-key
send-text
send-timeout
setup-sh
sh-var
shift
sleep
strlen
substr
encode-json
switch-screen
wait-all-async
Wait until all async blocks terminal.
throw
timeout
to-bool
to-int
to-num
too-many-tries
too-much-out
trim
true
typing-delay
unshift
warn
whoami
with-std-prompt
Author
Yichun Zhang <yichun@openresty.com>, OpenResty Inc.
Copyright & License
Copyright (C) 2019-2020 by OpenResty Inc. All rights reserved.
This document is proprietary and contains confidential information. Redistribution of this document without written permission from the copyright holders is prohibited at all times.