Flexible patterns via optionality

In this article, we discuss how to use streams with optional variables in TypeDB. If a variable is optional in a stream of concept maps, this means that some of the stream’s maps contain the variable but others may not. The introduction of optionality drastically enhances the expressivity and flexibility of patterns.

At a glance: optionality syntax essentials

The TypeQL keywords used to work with optional variables in patterns are or and try.

• The or keyword is used to specify several cases of a pattern, that are to-be-searched separately. Importantly, cases may introduce new variables individually!
• The try keyword indicates that a subpattern should only be included in a super-pattern if there are results for it—otherwise it can be ignored.

We remark that, in pipelines, optional variables can be turned into non-optional variables by matching them as a non-optional variables (i.e. outside an or or a try). The opposite is not true!

Optionality from or statements

An or statement is of the form { P1 } or { P2 } or ... ; where the P‘s are patterns called the case blocks. When an or statement is used in a pattern, then each case block is searched individually. For example, consider the match clause:

match
  $stakeholder isa customer;
  { $stakeholder has contributor_rank $rank; $rank > 5; }
    or { $stakeholder has stake $stake; $stake > 0.1; };

When matching results to this pattern, we consider the cases of the or branch separately, i.e., we look for results to the two pattern

# match this pattern (case 1)
match
  $stakeholder isa customer;
  $stakeholder has contributor_rank $rank; $rank > 5; 
# or match this pattern (case 2)
match
  $stakeholder isa customer;
  $stakeholder has stake $stake; $stake > 0.1;

As a result of matching two different patterns, the output stream of the original match clause may look something like this:

{
  ($stakeholder -> <cust8>, $rank -> 7),
  ($stakeholder -> <cust3>, $stake -> 0.13),
  ($stakeholder -> <cust3>, $rank -> 9),
  ($stakeholder -> <cust5>, $stake -> 0.15)
}

Both $rank are $stake are optional variables here: they do not appear consistently in all of the maps.

Optionality from try statements

A try statement is of the form try { P } where the P is a pattern called the optional block. When an try statement is used in a pattern then matched results are first searched for with the optional pattern included. Only if no such results are found, we then search for results

For example, consider the try clause

match
  $stakeholder isa customer;
  try { $stakeholder has birthday $bday; };

When matching results for this pattern, we would proceed as follows:

# match this pattern first
match
  $stakeholder isa customer;
  $stakeholder has birthday $bday;
# or, if that didn't return anything, match this pattern instead
match
  $stakeholder isa customer;

As a result of this processs, the output stream of the original match clause could look something like this:

{
  ($stakeholder -> <cust8>, $bday -> 1997-30-05),
  ($stakeholder -> <cust3>),
  ($stakeholder -> <cust5>, $bday -> 1975-30-11)
}

In this case, $bday is an optional variable.

Optional variables in insert, delete, and put

Statements with optional variables in insert, delete and put clauses must be wrapped in try clauses. Unlike in the pattern of a match clause, it makes no sense to nest try (or or) block—any statement can either be executed or not. However, it is possible to group multiple statements in a single try block which means they will either all be executed or none will be executed.

To see how this works, let’s extend our previous example a bit:

match
  $stakeholder isa customer;
  { $stakeholder has contributor_rank $rank; $rank > 5; }
    or { $stakeholder has stake $stake; $stake > 0.1; };
insert
  try { $stakeholder has contributor_rank $rank + 1 @replace; };
  try { $stakeholder has stake $stake + 0.01 @replace; };

The above query will increase the rank of a $stakeholder by 1 if a $rank is present, and similarly increase the stake of the $stakeholder by 0.01 if a $stake is present.

We remark that it is easy to check “potential” optionality of variables statically, simply by checking for variables which aren’t bound outside of or or try blocks. If potentially optional variables are found that are not wrapped in a try block then then a type-checking error will be thrown.

Optional variables for stream modifiers

We have already discussed that filter and sort work with optional variables just as expect. Similarly, limit and offset are clearly unaffected by optional variables.

Optional variables in fetch

When using fetch on streams with optional variables, the following two principles are applied:

1. If in any key: value pair the variable used for the key is missing then this field is omitted from the JSON.
2. If in any key: value pair one or more variables used for the value expression is missing (this expression could be a subquery or function call) then this value will become null.

Summary

Optionality is a key part of most functional programming models, and this extends to TypeDB functional database programming model. Our examples here illustrate well that optionality plays very nicely with TypeQL’s declarative patterns and near-natural syntax, which enables users to intuitively work with optional variables in their queries.