Flexible patterns via optionality

In this article, we discuss how to use streams with optional variables in TypeDB. If a variable is optional in a stream of concept maps, this means that some of the stream’s maps contain the variable but others may not. The introduction of optionality drastically enhances the expressivity and flexibility of patterns.

At a glance: optionality syntax essentials

There are three places in which optional variables may be introduced.

• The or keyword is used to specify several cases of a pattern, that are to-be-searched separately. Importantly, cases may introduce new variables individually! All variables used exclusively inside an or branch could be optional (i.e. not appear in all query results).
• The try keyword indicates that a subpattern should only be included in a super-pattern if there are results for it—otherwise it can be ignored. All variables used exclusively inside the try could be optional.
• Variable assign to an optional function return are non-optional unless marked with a ?. For example, a function with signature fun my_fun($a: A) -> B, C? will return either just B or a tuple B, C. When calling that function, we may write $b, $c = my_fun($a); or $b, $c? = my_fun($a);. In the former case, $c is non-optionally used, i.e. it must exist in every result of the enclosing pattern. In the latter case, we mark the return of $c as optional—i.e., not every result may contain data for $c.

Remark. Importantly, optional variables can be turned into non-optional variables by matching them as a non-optional variables (i.e. outside an or or a try or an optional function assignment). The opposite is not true!

Optionality from or statements

An or statement is of the form { P1 } or { P2 } or ... ; where the P‘s are patterns called the case blocks. When an or statement is used in a pattern, then each case block is searched individually. For example, consider the match clause:

match
  $stakeholder isa customer;
  { $stakeholder has contributor_rank $rank; $rank > 5; }
    or { $stakeholder has stake $stake; $stake > 0.1; };

When matching results to this pattern, we consider the cases of the or branch separately, i.e., we look for results to the two pattern

# match this pattern (case 1)
match
  $stakeholder isa customer;
  $stakeholder has contributor_rank $rank; $rank > 5; 
# or match this pattern (case 2)
match
  $stakeholder isa customer;
  $stakeholder has stake $stake; $stake > 0.1;

As a result of matching two different patterns, the output stream of the original match clause may look something like this:

{
  ($stakeholder -> <cust8>, $rank -> 7),
  ($stakeholder -> <cust3>, $stake -> 0.13),
  ($stakeholder -> <cust3>, $rank -> 9),
  ($stakeholder -> <cust5>, $stake -> 0.15)
}

Both $rank are $stake are optional variables here: they do not appear consistently in all of the maps.

Optionality from try statements

A try statement is of the form try { P } where the P is a pattern called the optional block. When an try statement is used in a pattern then matched results are first searched for with the optional pattern included. Only if no such results are found, we then search for results

For example, consider the try clause

match
  $stakeholder isa customer;
  try { $stakeholder has birthday $bday; };

When matching results for this pattern, we would proceed as follows:

# match this pattern first
match
  $stakeholder isa customer;
  $stakeholder has birthday $bday;
# or, if that didn't return anything, match this pattern instead
match
  $stakeholder isa customer;

As a result of this processs, the output stream of the original match clause could look something like this:

{
  ($stakeholder -> <cust8>, $bday -> 1997-30-05),
  ($stakeholder -> <cust3>),
  ($stakeholder -> <cust5>, $bday -> 1975-30-11)
}

In this case, $bday is an optional variable.

Optional function returns

A function may have optional return types:

1. A stream-return function could have return type my_stream($x:T) -> { A, B? }.
2. A single-return function could have return type my_tuple($y:S) -> C?, D.

In the first case, we may assign variables using the keyword in as follows:

$a, $b? in my_stream(x=$var);

In the second case, we may assign variables using the operator = as follows:

$c?, $d = my_tuple(y=$var);

In both case, if the variables $b and $c are used non-optionally in the rest of the pattern, they will implicitly turn into non-optional variables. For example, consider the pattern:

$c?, $d = my_tuple(y=$var);
$p isa person, has name $c, has age $d;

Every result to this pattern will contain, in particular, a person $p with a (non-optional) name $c. If instead we would write:

$c?, $d = my_tuple(y=$var);
$p isa person, has age $d;
try { $p has name $c; }

then $c would not need to appear in all query results, i.e. it would remain optional.

Optional variables in insert, delete, and put

Statements with optional variables in insert, delete and put clauses must be wrapped in try clauses. Unlike in the pattern of a match clause, it makes no sense to nest try (or or) block—any statement can either be executed or not. However, it is possible to group multiple statements in a single try block which means they will either all be executed or none will be executed.

To see how this works, let’s extend our previous example a bit:

match
  $stakeholder isa customer;
  { $stakeholder has contributor_rank $rank; $rank > 5; }
    or { $stakeholder has stake $stake; $stake > 0.1; };
insert
  try { $stakeholder has contributor_rank $rank + 1 @replace; };
  try { $stakeholder has stake $stake + 0.01 @replace; };

The above query will increase the rank of a $stakeholder by 1 if a $rank is present, and similarly increase the stake of the $stakeholder by 0.01 if a $stake is present.

We remark that it is easy to check “potential” optionality of variables statically, simply by checking for variables which aren’t bound outside of or or try blocks. If potentially optional variables are found that are not wrapped in a try block then then a type-checking error will be thrown.

Optional variables for stream modifiers

The stream operator select and sort work with optional variables just as one would expect: if an optional variable is selected (or sorted on) then it will still be optional in the resulting stream:

• Selecting $x and $y in the stream { ($x -> a, $y -> 6, $z -> b), ($x -> c, $z -> d) } (i.e. $y is optional) will yield { ($x -> a, $y -> 6), ($x -> c) }
• Sorting on $y ascendingly in the stream { ($x -> a, $y -> 6), ($x -> a, $y -> 4), ($x -> c) } will yield { ($x -> a, $y -> 4), ($x -> a, $y -> 6), ($x -> c) }, i.e. answer in which $y does not appear are return at the end of the stream.

Similarly, limit and offset are clearly unaffected by optional variables.

Optional variables in fetch

When using fetch on streams with optional variables, the following simple principles is applied: if the variable $x in "key": $x (or "key": $x.att) is missing from the stream then the value is set to null!

Summary

Optionality is a key part of most functional programming models, and this extends to TypeDB functional database programming model. Our examples here illustrate well that optionality plays very nicely with TypeQL’s declarative patterns and near-natural syntax, which enables users to intuitively work with optional variables in their queries.