# TypeQL 3.0 Syntactic and Semantic Summary

## Overview
TypeQL 3.0 is a strongly-typed, polymorphic query language for TypeDB that enables schema definition, data manipulation, and complex queries through a type-theoretic approach. This summary covers the key syntactic and semantic differences from TypeDB 2.0 and provides comprehensive guidance for TypeQL 3.0.

## Major Changes compared to TypeQL 2.0

### Value Type Renaming
- `long` → `integer` (all integer values now use `integer` type)

### Variables
- **All variables**: Must start with `$` (no more `?` for value variables)
- **Computed values**: Use `let` keyword for assignments for expressions or function calls

### Patterns
- **Relation role player shorthand has changed**: We NO LONGER use `$relation (role: $player) isa relation-type;`. Instead, we use `$relation isa relation-type (role: $player)`. Any usage of `$var (role: $player)` forces `$var` to be a _type_, instead of an _instance_ in TypeQL 3.0

### Schema Definition: Type Kinds
- **No more predefined root types** - all types must be explicitly declared with their kind
- **Syntax**: `entity person`, `relation friendship`, `attribute name`
- **Subtyping**: Still supported with separate definition: `entity person, sub animal;`

### Updated Query Syntax

#### Undefine and Delete Operations
- **Type removal**: `undefine person;` (for entities, relations, attributes)
- **Trait removal**:
  - `undefine owns name from person;`
  - `undefine relates best-friend from friendship;`
  - `undefine plays friendship:best-friend from person;`
- **Instance deletion**: `delete $x;`
- **Connection deletion**:
  - `delete has $name of $x`
  - `delete links (friend: $p) of $rel`;

#### Query Clauses
- **Variable filtering**: 'get' operation has been renamed 'select'

#### New Fetch Syntax
- **JSON-aligned formatting** with nested structures. Syntax describes the structure of JSON output.
- **Subqueries**: Fetch subqueries with many answers are embedded using `[]`, while single-returning subqueries use `()`
- **All attributes**: Getting all attributes can be done with `fetch { $person.* };`, or `fetch { "person-attributes": { $person.* } };`, since the .* operation expands into a dictionary, it must be wrapped in {}

### 4. Annotations Consolidation
- **Abstract and regex**: Now grouped under `@` annotations
- **Syntax**: `define entity abstract-person @abstract, owns email @regex(".*@.*");`

### 5. Cardinality Defaults
- **Plays**: `@card(0..)` (unlimited) - default
- **Owns/Relates**: `@card(0..1)` (up to one) - default
- **Explicit cardinality**: `@card(0..)` required for unbounded cardinality

## Core Syntax Elements

Comments always start from `#` in typeql to the end of the line, and there are no block comments.

These are reserved identifiers. **Never use them as user-defined identifier, in any capitalization**:
with, match, fetch, update, define, undefine, redefine, insert, put, delete, end, entity, relation, attribute, role, asc, desc, struct, fun, return, alias, sub, owns, as, plays, relates, iid, isa, links, has, is, or, not, try, in, true, false, of, from, first, last

### Schema Definition
* use `define` to idempotently add to the schema
* use `undefine` to remove elements from the schema
* use `redefine` to replace parts of the schema

### Query Pipeline Structure
TypeQL 3.0 uses pipeline-based queries with multiple stages:

Pipeline stages can be composed in any order, except that `fetch` can only be the final stage.

Stages can be chained: a `reduce` stage produces new bindings that can feed into another `match` stage (see "Chained Reduce" below).

#### Match Stage
- **Purpose**: Find data matching patterns
- **Syntax**: `match <pattern>;`
- **Example**:
```typeql
match
  $user isa user, has username "user_0";
  friendship (friend: $user, friend: $friend);
```
#### Insert Stage
- **Purpose**: Add new data
- **Syntax**: `insert <pattern>;`
- **Example**: `insert $user has status "VIP";`

#### Delete Stage
- **Purpose**: Remove data
- **Syntax**: `delete <pattern>;`

#### Update Stage
- **Purpose**: Replace existing data
- **Syntax**: `update <pattern>;`

#### Put Stage
- **Purpose**: Insert or update data conditionally (upsert operation)
- **Syntax**: `put <pattern>;`
- **Behavior**: Creates new data if **the entire pattern** doesn't exist, otherwise returns the existing matching data
- **Example**:
```typeql
match
  $p isa person, has full-name "John Doe";
put
  $p has age 30;
```

#### Fetch Stage
- **Purpose**: Retrieve and format data
- **Syntax**: `fetch { <structure> };`
- **Features**:
  - JSON-like nested structures
  - Subqueries with `match` and either terminated with `fetch` or `return`
  - Attribute access with `$var.*` to get all (wrapped in {}), `$var.<attribute type>` for a 0 or 1 cardinality attribute ownership, and `[ $var.<attribute type> ]` for higher cardinality attribute ownership

#### Reduce Stage
- **Purpose**: Aggregate and compute values
- **Syntax**: `reduce <variable> = <function> [groupby <variables>];`
- **Example**: `reduce $VIP-friend-count = count groupby $friend;`

##### Select Operator
- **Purpose**: Specify which variables to return in query results
- **Syntax**: `select $var1, $var2;`
- **Behavior**: Filters output to only include specified variables
- **Example**:
```typeql
match
  $p isa person, has full-name $n, has age $a;
select $n, $a;
```

##### Sort Operator
- **Purpose**: Order query results based on specified variables
- **Syntax**: `sort $var [asc/desc] (, $var2 [asc/desc], ...);`
- **Default**: Ascending order if not specified
- **Example**:
```typeql
match
  $p isa person, has full-name $n, has age $a;
sort $a desc;
```

##### Limit Operator
- **Purpose**: Restrict the number of results returned
- **Syntax**: `limit <number>;`
- **Example**:
```typeql
match
  $p isa person, has full-name $n;
limit 2;
```

##### Offset Operator
- **Purpose**: Skip a specified number of results before returning remaining ones
- **Syntax**: `offset <number>;`
- **Example**:
```typeql
match
  $p isa person, has full-name $n;
offset 2;
```

### Variable System
- **Concept Variables**: `$variable-name` (represent instances/values)
- **Value Variables**: `$variable-name` (all variables now use `$`)
- **Computed Variables**: `let $computed = <expression>;`

### Pattern Construction

#### Basic Data Statements
- **Type assertion**: `$x isa person;`
- **Attribute ownership**: `$x has $name;` or `$x has name "John";`
- **Anonymous Relation participation**: `friendship (friend: $x, friend: $y);`
- **Variablized relation participation**: `$rel isa friendship, links (friend: $x, friend: $y);`
  - Shorthand form: `$rel isa friendship (friend: $x, friend: $y);`
- **Value comparison operation**: `$x > 25;`. Comparing two variables will ignore attribute type if the variables hold attributes and not primitive values.
- **Concept comparison**: use `is` to do exact equality between concepts, eg `$x is $y;`. This will _not_ ignore attribute type if the variables hold attributes.

Be careful: when a relation type **relates** a role, you can query data instances with links player syntax as the linking variable: `$relation-instance links (role: $player)`.
However, when a relation type (or any other type) **plays** a role, then the instances are queried with links player syntax with the variable **inside** the parenthesis: `$other-relation-instance links (role: $relation-instance)`.

#### Basic schema statements
- **Entity, relation or attribute type**: `entity $entity-type`; `relation $relation-type`; `attribute $attribute-type`
- **Label**: `$type-variable label <label>`
- **Value type**: `$attribute-type value <value type>;`
- **Owns**: `$owner-type owns <attribute-type>`. Each attribute ownership declaration requires its own 'owns' declaration.
- **Plays**: `$player-type plays <scoped role-type>`. Each role playing declaration requires its own 'plays'.
- **Relates**: `$relation-type relates <unscoped role type>`

Note that you can use a type label instead of variables in most cases.

Warning! Using any of these statements means the variables or types on both sides are **schema types**, not **instances**.

These are the available annotations in TypeQL:
- @abstract : used directly on a type declaration
- @key : only used on `owns` declarations
- @unique : only used on `owns` declarations
- @card(start..end) : used on owns, relates, or plays declarations
- @range(start..end) : only used on `value <value-type>` declarations
- @regex("regex") : only used on `value <value-type>` declarations
- @values(value-1, value-2, ...) : only used on `value <value-type>` declarations


#### Data vs schema statements

Schema statements have **corresponding** statements when looking up data:

- `relates` & `plays` corresponds to `links`:
  `define entity person plays friendship:friend; relation friendship relates friend;`
  corresponds to data patterns like `match $p isa person; $friendship links (friend: $p);`
- `owns` corresponds to `has`:
  `define entity person owns name; attribute name value string;`
  corresponds to data patterns like `match $p isa person, has $name; $name isa name;`

#### Logical Pattern Operators
- **Conjunction**: `and` (implicit in sequences separated by `;` or `,`)
- **Disjunction**: `{pattern} or { pattern } [ or { pattern } or ... ]`
- **Negation**: `not { pattern }`
- **Optionality**: `try { pattern }`

Use these to build complex patterns.

#### Comparison Operators
- **Equality**: `=`, `!=`
- **Ordering**: `<`, `<=`, `>`, `>=`
- **Pattern matching**: `like` (for regexes using standard regex syntax: `.*` not SQL `%`), `contains` (for substring matching)

**IMPORTANT**: `like` uses regex syntax, NOT SQL LIKE wildcards. Use `$x like ".*Smith.*"` (not `%Smith%`). Use `$x like "^John.*"` for starts-with. Use `$x like ".*son$"` for ends-with.

### Functions and Aggregations

#### Built-in Functions
- **Count**: `count`, `count groupby $var`
- **Mathematical**: `sum`, `mean`, `max`, `min`
- **Arithmetic operators in `let`**: `+`, `-`, `*`, `/`, `abs()`
- **String**: `concat`, `substring`
- **Date/Time**: `datetime`, `duration`

#### Custom Functions
- **Query-scoped**: `with fun friend_count($user: user) -> integer: ...`
- **Schema-defined**: `define fun friend_count($user: user) -> integer: ...`

### Value Types
- **Implemented**: `string`, `integer`, `double`, `boolean`, `datetime`, `datetime-tz`, `date`, `duration`, `decimal` (for fixed-decimal operations such as currency).

Struct value types are not supported yet.

## Advanced Query Patterns

### Chained Reduce (HAVING Equivalent)

Filter on aggregation results by chaining a `reduce` stage into another `match` stage:

```typeql
# Find tweets retweeted more than 100 times
match
  $tweet isa tweet;
  retweets (original_tweet: $tweet, retweeting_tweet: $retweet);
reduce $count = count groupby $tweet;
match $count > 100;
fetch { "tweet": $tweet.text, "retweets": $count };
```

This replaces SQL/Cypher `HAVING` clauses. The second `match` stage receives bindings from `reduce` and can filter on them.

### Multiple Chained Reduce Stages

Reduce stages can be chained multiple times for multi-step aggregation:

```typeql
# Find top 3 hashtags in tweets retweeted more than 100 times
match
  $tweet isa tweet;
  retweets (original_tweet: $tweet);
reduce $retweet_count = count groupby $tweet;
match
  $retweet_count > 100;
  tags (tagged_tweet: $tweet, tag: $hashtag);
reduce $hashtag_count = count groupby $hashtag;
sort $hashtag_count desc;
limit 3;
fetch { "hashtag": $hashtag.hashtag_name, "count": $hashtag_count };
```

### Let Expressions (Computed Values)

Use `let` for arithmetic, ratios, differences, and function calls:

```typeql
# Sort by computed ratio
match
  $user isa user, has following_count $follows, has follower_count $followers;
  $followers > 0;
let $ratio = $follows / $followers;
sort $ratio desc;
limit 10;
fetch { "user": $user.screen_name, "ratio": $ratio };

# Absolute difference for similarity ranking
match
  $ref isa user, has name "TypeDB", has betweenness $ref_b;
  $other isa user, has betweenness $other_b;
  not { $ref is $other; };
let $diff = abs($other_b - $ref_b);
sort $diff asc;
limit 5;
fetch { "user": $other.name };
```

### Custom Functions (Reusable Query Logic)

Define reusable query logic with `with fun` (query-scoped) or `define fun` (schema-scoped):

```typeql
with fun follower_count($user: user) -> integer:
  match follows (followed: $user);
  return count;
with fun follows_count($user: user) -> integer:
  match follows (follower: $user);
  return count;
match
  $user isa user;
let $followers = follower_count($user);
let $following = follows_count($user);
$followers > 0;
let $ratio = $following / $followers;
sort $ratio desc;
limit 10;
fetch { "user": $user.name, "ratio": $ratio };
```

Functions can use `select`, `distinct`, and `return count` internally:

```typeql
# Count distinct users who retweeted a tweet
with fun retweeting_users($tweet: tweet) -> integer:
  match
    retweets (original_tweet: $tweet, retweeting_tweet: $retweet);
    posts (author: $user, content: $retweet);
  select $user;
  distinct;
  return count;
match
  $tweet isa tweet;
let $rt_users = retweeting_users($tweet);
$rt_users > 5;
fetch { "tweet": $tweet.text, "unique_retweeters": $rt_users };
```

### Polymorphic Queries (Disjunctions Across Relation Types)

Match across multiple relation types using disjunctions:

```typeql
# Count all interactions (mentions + retweets + replies) for each tweet
match
  $tweet isa tweet;
  { $rel links (source_tweet: $tweet); } or { $rel links (original_tweet: $tweet); };
reduce $count = count groupby $tweet;
sort $count desc;
limit 5;
fetch { "tweet": $tweet.text, "interactions": $count };
```

### Variable Scoping in Disjunctions

Variables introduced inside disjunction branches are accessible outside **only if they are bound in every branch**. Variables bound in only some branches are not returned.

```typeql
# WRONG - $rel only bound in one branch
{ $rel isa interacts (character1: $c); } or { interacts (character2: $c); };
reduce $count = count($rel) groupby $c;  # Error: $rel not in every branch

# RIGHT - $rel bound in every branch
{ $rel isa interacts (character1: $c); } or { $rel isa interacts (character2: $c); };
reduce $count = count($rel) groupby $c;  # Works: $rel is in all branches

# RIGHT - also works across different relation types
{ $rel links (source_tweet: $tweet); } or { $rel links (original_tweet: $tweet); };
reduce $count = count($rel) groupby $tweet;  # Works: $rel is in all branches
```

**Rule**: A variable from inside a disjunction is available outside only if it appears in **every** branch of the `{ } or { }` block.

### Role Inference (Omitting Roles)

Omit roles to match a player in ANY role position:

```typeql
# Match $c in ANY role of the interacts relation (character1 OR character2)
$rel isa interacts ($c);

# Bidirectional/symmetric matching - omit roles for BOTH players
subsidiary_of ($o1, $o2);
# Matches: (parent: $o1, subsidiary: $o2) OR (parent: $o2, subsidiary: $o1)
# Much simpler than writing out both disjunction branches explicitly
```

### Explicit Role Type Checking

When you need to check which role a player fills:

```typeql
$rel isa interacts ($role: $c);
{ $role sub interacts:character1; } or { $role sub interacts:character2; };
```

### Select + Distinct (COUNT DISTINCT)

```typeql
# Count distinct cities per company
match
  $c isa company;
  located_in (organization: $c, location: $city);
select $c, $city;
distinct;
reduce $city_count = count groupby $c;
```

### Tuple Groupby

Group by multiple variables:

```typeql
# Count co-occurrences of pairs
match
  acted_in (actor: $a, film: $m);
  acted_in (actor: $b, film: $m);
  not { $a is $b; };
reduce $count = count groupby $a, $b;
sort $count desc;
limit 10;
```

### Optionality (OPTIONAL MATCH equivalent)

Use `try {}` for left-join semantics where a pattern may or may not match:

```typeql
match
  $p isa person;
  try { $p has nickname $nick; };
fetch { "name": $p.name };
```

The query returns all persons regardless of whether they have a nickname. Without `try`, only persons WITH a nickname would match.

### Negation

```typeql
# Users who have NOT acted in any film
match
  $p isa person;
  not { acted_in (actor: $p, film: $m); };
fetch { "name": $p.name };
```

### Disjunction for IN-list Filters

```typeql
# Match specific values (IN equivalent)
match
  $m isa movie, has genre $g;
  { $g == "Action"; } or { $g == "Comedy"; } or { $g == "Drama"; };
fetch { "title": $m.title };
```

## Best Practices for LLM Query Generation

### 1. Schema-First Approach
- Always define schema before data operations
- Use explicit type kinds (`entity`, `relation`, `attribute`)
- Set appropriate cardinalities for performance and to constrain data correctly

### 2. Variable Naming
- Use descriptive variable names (`$user`, `$friend`, `$company`)
- All variables must start with `$`
- Use underscores in variable names, not hyphens (`$typedb_score`, not `$typedb-score`)
- Use `let` for computed values or function assignments

### 3. Relation Syntax
- **Preferred (anonymous)**: `acted_in (actor: $p, film: $m);`
- **With variable**: `$rel isa acted_in (actor: $p, film: $m);`
- **NEVER write**: `$rel (actor: $p, film: $m) isa acted_in;` (this makes `$rel` a type, not an instance)
- Use exact role names from the schema
- TypeQL relations are undirected - role names convey directionality

### 4. Attribute Access
- **Direct fetch (preferred)**: `fetch { "name": $p.name };` (no need to bind)
- **Bind only when filtering/sorting**: `$p has age $a; $a > 25; sort $a;`
- **Multi-cardinality**: `fetch { "emails": [ $p.email ] };`

### 5. Pipeline Design
- Use `match` to find data
- Use `insert`/`delete`/`update`/`put` for modifications
- Use `reduce` for aggregations
- Use `select`/`sort`/`limit`/`offset` for pipeline stream control
- Use `fetch` for formatted output and projection
- **Order**: Pipeline stages can be composed in any order, except that `fetch` can only be the final stage

### 6. String Matching
- **Substring**: `$x contains "graph";` (preferred for simple substring)
- **Regex**: `$x like ".*pattern.*";` (use regex syntax, NOT SQL `%` wildcards)
- **Starts with**: `$x like "^prefix.*";`
- **Ends with**: `$x like ".*suffix$";`
- **Case-insensitive**: `$x like "(?i)pattern";`

### 7. NULL Handling
- TypeQL has no NULL concept. Missing attributes simply don't match.
- `WHERE x IS NOT NULL` in SQL/Cypher translates to simply including the attribute in the match pattern.
- For OPTIONAL MATCH / left-join semantics, use `try { pattern; };`

## Common Patterns

### Entity Creation
```typeql
insert $person isa person, has name "Alice", has age 30;
```

### Relation Creation
```typeql
insert
  $alice isa person, has name "Alice";
  $bob isa person, has name "Bob";
  friendship (friend: $alice, friend: $bob);
```

If you need to have a variable for the relation, use `links` to move the role players to their own statement:
```typeql
insert
  $alice isa person, has name "Alice";
  $bob isa person, has name "Bob";
  $f isa friendship;
  $f links (friend: $alice, friend: $bob);
```

### Common Read Patterns

```typeql
# Simple entity lookup
match $p isa person, has name "Alice";
fetch { "age": $p.age };

# Relation traversal
match
  $p isa person, has name "Tom Hanks";
  acted_in (actor: $p, film: $m);
fetch { "movie": $m.title };

# Filtered with sorting
match
  $m isa movie, has released $year;
  $year > 2000;
sort $year desc;
limit 10;
fetch { "title": $m.title, "year": $year };

# Grouped count with sort
match
  $p isa person;
  acted_in (actor: $p, film: $m);
reduce $count = count($m) groupby $p;
sort $count desc;
limit 5;
fetch { "actor": $p.name, "movies": $count };

# Aggregation with HAVING filter (chained reduce)
match
  $h isa hashtag;
  tags (tagged_tweet: $tweet, tag: $h);
reduce $count = count groupby $h;
match $count > 5;
sort $count desc;
fetch { "hashtag": $h.name, "tweets": $count };

# Negation (NOT EXISTS)
match
  $u isa user;
  not { follows (follower: $u, followed: $other); };
fetch { "loner": $u.screen_name };

# Optional pattern (OPTIONAL MATCH / LEFT JOIN)
match
  $u isa user;
  try { $u has bio $b; };
fetch { "user": $u.screen_name };

# Disjunction (OR)
match
  $m isa movie;
  { $m has genre "Action"; } or { $m has genre "Thriller"; };
fetch { "title": $m.title };

# Computed values
match
  $m isa movie, has revenue $r, has budget $b;
  $b > 0;
let $profit = $r - $b;
sort $profit desc;
limit 5;
fetch { "title": $m.title, "profit": $profit };
```

This summary provides the essential syntactic and semantic knowledge needed for generating accurate and effective TypeQL 3.0 queries.

## Examples

## Schema
Here's a complete example of a small TypeDB 3.0 schema modeling a social network with pages, which can be either user pages or organization pages. Organization pages can themselves be specialized to be company or university pages.
```typeql
define
  entity page,
    owns username @key;
  entity user,
    sub page,
    owns email @card(0..) @unique,
    owns age,
    plays following:follower;
  entity organization,
    sub page,
    owns running,
    plays following:target;
  entity university,
    sub organization,
    owns non-profit-status;
  entity company,
    sub organization;
  relation following,
    relates follower @card(1),
    relates target @card(1),
    owns start-date;
  attribute username, value string;
  attribute email, value string @regex(".*@.*");
  attribute age, value integer;
  attribute running, value string @values("running", "defunct");
  attribute start-date, value datetime;
  attribute non-profit-status, value string;
```

## Inserting data

Inserting some initial data
```typeql
insert
  $john isa user,
    has username "john",
    has email "john@email.com",
    has age 30;

  $stanford-page isa university,
    has username "stanford",
    has non-profit-status "501(c)(3)";

  following (follower: $john, target: $stanford-page);

  $typedb-page isa company,
    has username "typedb";

  $typedb-following isa following,
    links (follower: $john, target: $typedb-page),
    has start-date 2025-01-01T00:00:00.000000;
```

Extending existing data:
```typeql
match
  $john isa user, has username "john";
insert
  $berkeley-page isa university,
    has username "berkeley",
    has non-profit-status "501(c)(3)";
  $alice isa user, has username "alice";
  following (follower: $john, target: $berkeley-page);
  following (follower: $alice, target: $berkeley-page);
```

## Reading data

Look up the most followed organization:
```typeql
match
  $page isa organization;
  following (follower: $follower, target: $page);
reduce $count = count groupby $page;
sort $count desc;
limit 1;
fetch {
  "page": $page.username,
  "count": $count,
};
```

Find any followings from a specific date range:
```typeql
match
  $following isa following,
    links (target: $page),
    has start-date $date;
  $date >= 2025-01-01T00:00:00;
  $date < 2025-02-01T00:00:00;
fetch {
  "page": $page.username,
  "following attributes": { $following.* },
};
```

Fetch query with nested structure using a demonstrator function:
```typeql
with fun silly_function() -> { integer }:
  match
    let $value = 1;
  return { $value };
match
  $user isa user;
fetch {
  "all attributes": { $user.* },
  "username": $user.username,
  "emails": [ $user.email ],
  "emails-nested": {
    "double nested": {
      "emails": [$user.email ],
    }
  },
  "direct function call": [silly_function()],
  "multi valued subquery": [
    match
      following (follower: $user, target: $org);
    fetch {
      "org information": { $org.* },
    };
  ],
  "single value subquery": (
    match
      following (follower: $user);
    return count;
  )
};
```

## Advanced Examples

### Custom function with chained reduce
```typeql
# Tweets by a user that have been retweeted by more than 5 distinct users
with fun retweeting_users($tweet: tweet) -> integer:
  match
    retweets (original_tweet: $tweet, retweeting_tweet: $retweet);
    posts (author: $user, content: $retweet);
  select $user;
  distinct;
  return count;
match
  $me isa me, has screen_name "typedb";
  posts (author: $me, content: $tweet);
let $rt_users = retweeting_users($tweet);
$rt_users > 5;
fetch { "tweet": $tweet.text };
```

### Two custom functions with ratio computation
```typeql
# Users with highest following-to-follower ratio
with fun follower_count($user: user) -> integer:
  match follows (followed: $user);
  return count;
with fun follows_count($user: user) -> integer:
  match follows (follower: $user);
  return count;
match
  $user isa user;
let $followers = follower_count($user);
let $following = follows_count($user);
$followers > 0;
$following > 0;
let $ratio = $following / $followers;
sort $ratio desc;
limit 10;
fetch { "user": $user.name, "ratio": $ratio };
```