Lesson 6.5: Processing results

The structure of results

In this lesson, we’ll be looking at how we can process the results of queries to produce the data structures we need. We’ll start with the following query.

match $book isa hardback;
fetch { "title": $book.title, "genre": [$book.genre], "page-count": $book.page-count };
from typedb.driver import TypeDB, Credentials, DriverOptions

DB = "bookstore"
address = "localhost:1729"
credentials = Credentials("admin", "password")
options = DriverOptions(is_tls_enabled=True, tls_root_ca_path=None)

with TypeDB.driver(address, credentials, options) as driver:
    with driver.transaction(DB, TransactionType.READ) as tx:
        docs = tx.query(
            """
            match $book isa hardback;
            fetch { "title": $book.title, "genre": [$book.genre], "page-count": $book.page-count };
            """
        ).resolve().as_concept_documents()
        for doc in docs:
            print(doc)

Notice that there’s a .resolve() operation and a as_concept_documents() operation. .resolve() is required since tx.query() returns a promise that needs to be awaited (if using Rust with the async-compatible driver, you can just use await). This will result in either an error, or an QueryAnswer.

QueryAnswer can either be a simple acknowledgement for define or other schema queries, as they do not return anything. QueryAnswer could also be an iterator or ConceptRow or of ConceptDocument. Queries ending in fetch {} clauses always return documents, so we downcast the answer into a ConceptDocumentIterator with .as_concept_documents().

Answer iterators (in GRPC) are lazy, so: - they must be consumed them inside the transaction scope - they will prefetch a certain number of answers to hide latency, but must be consumed to trigger further computation on the server side

Note that since they’re standard Python iterators, you can do the usual operations like using all_answers = list(tx.query(my_query).resolve().as_answer_documents()).

If we run this code snippet, then we get the following output.

{
  "genre": [
    "history",
    "nonfiction"
  ],
  "title": "Under the Black Flag: The Romance and the Reality of Life Among the Pirates",
  "page-count": 296
}
{

  "genre": [
    "nonfiction",
    "technology"
  ]
  "title": "Electron Backscatter Diffraction in Materials Science",
  "page-count": 425,
}
{
  "genre": [
    "comics",
    "fiction"
  ],
  "title": "The Complete Calvin and Hobbes"
  "page-count": 1451,
}

Each output line is a concept document (JSON). Keys correspond to the fetch body.

For each result, the value of result["book"] is another dictionary. The keys of these nested dictionaries are the names of the attributes that we fetched of book, in addition to a type field as we saw in Lesson 3.2.

for result in results:
    print(result["book"].keys())
dict_keys(['type', 'genre', 'page-count', 'title'])
dict_keys(['type', 'genre', 'page-count', 'title'])
dict_keys(['title', 'genre', 'page-count', 'type'])

In the example above, title and page-count are scalars. genre is a list as requested by the list syntax in the fetch for the genres: [$book.genre].

for doc in docs:
    print(f"Title: {doc['title']}")
    for g in doc["genre"]:
        print(f"Genre: {g}")
    print(f"Page count: {doc['page-count']}")
    print()
Title: The Complete Calvin and Hobbes
Genre: comics
Genre: fiction
Page count: 1451

Title: Under the Black Flag: The Romance and the Reality of Life Among the Pirates
Genre: history
Genre: nonfiction
Page count: 296

Title: Electron Backscatter Diffraction in Materials Science
Genre: nonfiction
Genre: technology
Page count: 425

We can see that each book in the result set has one title and page count (as would be expected), in addition to two genres.

Exercise

Write a function that prints the title and ISBNs of every hardback. For each ISBN printed, make sure to specify if it is an ISBN-13 or an ISBN-10. The function should have the following signature.

print_hardback_isbns(tx) -> None
Hint

The type of ISBN can be accessed via the label field of the type field of the returned dictionary representing the ISBN, for example:

for result in results:
    for isbn in result["book"]["isbn"]:
        print(isbn["type"]["label"])
Sample solution 
def print_hardback_isbns(tx) -> None:
    docs = tx.query(
        """
        match $book isa hardback;
        fetch { "title": $book.title, "isbn": [$book.isbn] };
        """
    ).resolve().as_concept_documents()
    for doc in docs:
        print(f"Title: {doc['title']}")
        for v in doc["isbn"]:
            print(v)
        print()

Yielding results

By accessing the result dictionaries in this way, we can structure the results of a query as needed for further processing. The following code snippet shows a function that retrieves the IDs of orders that include a book, specified by ISBN, and the quantity ordered. The order ID and quantity are yielded as a tuple that we can further operate on elsewhere in our application.

def get_orders_of_book(tx, isbn: str) -> Iterator[tuple[str, int]]:
    docs = tx.query(
        f"""
        match $book isa book, has isbn \"{isbn}\";
              $line (order: $order, item: $book) isa order-line;
        fetch {{ "order": $order.id, "quantity": $line.quantity }};
        """
    ).resolve().as_concept_documents()
    for doc in docs:
        yield doc["order"], doc["quantity"]

We make sure to return an iterator by using the yield keyword rather than the return keyword. This will allow us to begin processing individual results as soon as they are each returned by the server rather than having to wait for the last one to be found. The following code snippet shows how we might call this function, and we can see it prints the results in the desired format.

with TypeDB.driver(address, credentials, options) as driver:
    with driver.transaction(DB, TransactionType.READ) as tx:
        for order in get_orders_of_book(tx, "9780446310789"):
            print(order)
('o0016', 1)
('o0032', 1)
('o0036', 2)
Exercise

Write a function that yields the ISBN-13s and titles of books in a specified genre as tuples. You can assume that each book has exactly one ISBN-13 and one title. The function should have the following signature.

get_books_in_genre(tx, genre: str) -> Iterator[tuple[str, str]]

You should ensure the following test case produces the result given for the sample database. Additional results may be produced if you have inserted data.

with TypeDB.driver(address, credentials, options) as driver:
    with driver.transaction(DB, TransactionType.READ) as tx:
        for book in get_books_in_genre(tx, "science fiction"):
            print(book)
('9780575104419', 'Dune')
('9780671461492', "The Hitchhiker's Guide to the Galaxy")
Sample solution 
def get_books_in_genre(tx, genre: str) -> Iterator[tuple[str, str]]:
    docs = tx.query(
        f"""
        match $book isa book, has genre \"{genre}\";
        fetch {{ "isbn-13": $book.isbn-13, "title": $book.title }};
        """
    ).resolve().as_concept_documents()
    for doc in docs:
        yield doc["isbn-13"], doc["title"]

Warning: because your query iterator is a stateful object tied to the transaction in the server, you must finish using the query iterator before the transaction closes. In other words, since TypeDB’s GRPC drivers are reactively streaming over the network as you consume answers, you must keep the transaction open while consuming answers.

Exercise

Write a sample app that allows an employee of the bookstore to query books via a CLI. The app should allow employees to look up books by specific ISBNs, and to search for books based on one or more of the following fields in combination:

  • Format.

  • Genres.

  • Contributor names.

  • Publisher name.

  • Publication year.

Whether looked up directly by ISBN or as part of a search, the following details should be displayed for each book returned:

  • All attributes of the book.

  • The names of any contributors to the book.

  • The name of the book’s publisher.

  • The book’s year of publication.

You may find it useful to refer to the bookstore’s schema.

Schema 
define

entity book @abstract,
    owns isbn @card(0..2),
    owns isbn-13 @key,
    owns isbn-10 @unique,
    owns title,
    owns page-count,
    owns genre @card(0..),
    owns price,
    plays contribution:work,
    plays publishing:published,
    plays promotion-inclusion:item,
    plays order-line:item,
    plays rating:rated,
    plays recommendation:recommended;

entity hardback sub book,
    owns stock;

entity paperback sub book,
    owns stock;

entity ebook sub book;

entity contributor,
    owns name,
    plays contribution:contributor,
    plays authoring:author,
    plays editing:editor,
    plays illustrating:illustrator;

entity company @abstract,
    owns name;

entity publisher sub company,
    plays publishing:publisher;

entity courier sub company,
    plays delivery:deliverer;

entity publication,
    owns year,
    plays publishing:publication,
    plays locating:located;

entity user,
    owns id @key,
    owns name,
    owns birth-date,
    plays action-execution:executor,
    plays locating:located,
    plays recommendation:recipient;

entity order,
    owns id @key,
    owns status,
    plays order-line:order,
    plays action-execution:action,
    plays delivery:delivered;

entity promotion,
    owns code @key,
    owns name,
    owns start-timestamp,
    owns end-timestamp,
    plays promotion-inclusion:promotion;

entity review,
    owns id @key,
    owns score,
    owns verified,
    plays rating:review,
    plays action-execution:action;

entity login,
    owns success,
    plays action-execution:action;

entity address,
    owns street,
    plays delivery:destination,
    plays locating:located;

entity place @abstract,
    owns name,
    plays locating:located,
    plays locating:location;

entity city sub place;

entity state sub place;

entity country sub place;

relation contribution,
    relates contributor,
    relates work;

relation authoring sub contribution,
    relates author as contributor;

relation editing sub contribution,
    relates editor as contributor;

relation illustrating sub contribution,
    relates illustrator as contributor;

relation publishing,
    relates publisher,
    relates published,
    relates publication;

relation promotion-inclusion,
    relates promotion,
    relates item,
    owns discount;

relation order-line,
    relates order,
    relates item,
    owns quantity,
    owns price;

relation rating,
    relates review,
    relates rated;

relation action-execution,
    relates action,
    relates executor,
    owns timestamp;

relation delivery,
    relates deliverer,
    relates delivered,
    relates destination;

relation locating,
    relates located,
    relates location;

relation recommendation,
    relates recommended,
    relates recipient;

attribute isbn @abstract, value string;
attribute isbn-13 sub isbn;
attribute isbn-10 sub isbn;
attribute title, value string;
attribute page-count, value integer;
attribute genre, value string;
attribute stock, value integer;
attribute price, value double;
attribute discount, value double;
attribute id, value string;
attribute code, value string;
attribute name, value string;
attribute birth-date, value datetime;
attribute street, value string;
attribute year, value integer;
attribute quantity, value integer;
attribute score, value integer;
attribute verified, value boolean;
attribute timestamp, value datetime;
attribute start-timestamp, value datetime;
attribute end-timestamp, value datetime;
attribute status, value string @regex("^(paid|dispatched|delivered|returned|canceled)$");
attribute success, value boolean;

# TODO: Change to check
fun is_review_verified_by_purchase($review: review) -> { order }:
  match
    ($review, $product) isa rating;
    ($order, $product) isa order-line;
    ($user, $review) isa action-execution, has timestamp $review-time;
    ($user, $order) isa action-execution, has timestamp $order-time;
    $review-time > $order-time;
  return { $order };

fun book_recommendations_for($user: user) -> {book}:
  match
    $new-book isa book;
    {
        let $new-book in book_recommendations_by_author($user);
    } or {
        let $new-book in book_recommendations_by_genre($user);
    };
  return { $new-book };

fun book_recommendations_by_genre($user: user) -> { book }:
match
    $user isa user;
    $liked-book isa book;
    {
        ($user, $order-for-liked) isa action-execution;
        ($order-for-liked, $liked-book) isa order-line;
    } or {
        ($user, $review-for-liked) isa action-execution;
        ($review-for-liked, $liked-book) isa rating;
        $review-for-liked has score >= 7;
    };
    $new-book isa book;
    not { {
        ($user, $order-for-new) isa action-execution;
        ($order-for-new, $new-book) isa order-line;
    } or {
        ($user, $review-for-new) isa action-execution;
        ($review-for-new, $new-book) isa rating;
    }; };
    $liked-book has genre $shared-genre;
    $new-book has genre $shared-genre;
    not { {
        $shared-genre == "fiction";
    } or {
        $shared-genre == "nonfiction";
    }; };
  return { $new-book };

fun book_recommendations_by_author($user: user) -> { book }:
  match
    $user isa user;
    $liked-book isa book;
    {
        ($user, $order-for-liked) isa action-execution;
        ($order-for-liked, $liked-book) isa order-line;
    } or {
        ($user, $review-for-liked) isa action-execution;
        ($review-for-liked, $liked-book) isa rating;
        $review-for-liked has score >= 7;
    };
    $new-book isa book;
    not { {
        ($user, $order-for-new) isa action-execution;
        ($order-for-new, $new-book) isa order-line;
    } or {
        ($user, $review-for-new) isa action-execution;
        ($review-for-new, $new-book) isa rating;
    }; };
    ($liked-book, $shared-author) isa authoring;
    ($new-book, $shared-author) isa authoring;
  return { $new-book };

fun order_line_best_price($line: order-line) -> { double }:
  match
    ($order) isa action-execution, has timestamp $order-time;
    $line isa order-line, links ($order, $item);
    $item has price $retail-price;
    let $time_value = $order-time;
    let $best-discount = best_discount_for_item($item, $time_value);
    let $discounted-price = round(100 * $retail-price * (1 - $best-discount)) / 100;
    $line has quantity $quantity;
    let $line-total = $quantity * $discounted-price;
  return { $line-total };

fun best_discount_for_item($item: book, $order-time: datetime) -> double:
  match
    {
        $inclusion isa promotion-inclusion,
            links ($promotion, $item),
            has discount $discount-attr;
        $promotion has start-timestamp <= $order-time,
            has end-timestamp >= $order-time;
        let $discount = $discount-attr;
    } or {
        let $discount = 0.0; # default
    };
return max($discount);

fun transitive_places($place: place) -> { place }:
  match
    {
      locating (located: $place, location: $parent);
    } or {
      locating (located: $place, location: $middle);
      let $parent in transitive_places($middle);
    };
  return { $parent };
Sample solution 
from typing import Iterator, Optional
from typedb.driver import TypeDB, TransactionType, Credentials, DriverOptions

ADDRESS = "localhost:1729"
DATABASE = "bookstore"


class BookFormat(Enum):
    PAPERBACK = "paperback"
    HARDBACK = "hardback"
    EBOOK = "ebook"


def filter_books(
    query_prefix: str,
    format: Optional[BookFormat],
    genres: Optional[list[str]],
    contributor_names: Optional[list[str]],
    publisher_name: Optional[str],
    publication_year: Optional[int],
) -> str:
    if format is None:
        query = f"{query_prefix} $book isa book;"
    else:
        query = f"{query_prefix} $book isa {format.value};"

    if genres is not None:
        for genre in genres:
            query += f" $book has genre \"{genre}\";"

    if contributor_names is not None:
        for i, name in enumerate(contributor_names):
            query += (
                f" $contributor_{i} isa contributor, has name \"{name}\";"
                f" ($book, $contributor_{i}) isa contribution;"
            )

    if publisher_name is not None:
        query += (
            f" $publisher isa publisher, has name \"{publisher_name}\";"
            f" ($book, $publisher) isa publishing;"
        )

    if publication_year is not None:
        query += (
            f" $publication isa publication, has year {publication_year};"
            f" ($book, $publication) isa publishing;"
        )

    query += " fetch { \"isbn-13\": $book.isbn-13 };"
    return query


def get_book_details_query(isbn: str) -> list[str]:
    return [
        f"""
        match $book isa! $book_type, has isbn \"{isbn}\";
        fetch {{ "format": $book_type, "attributes": {{ $book.* }} }};
        """,
        f"""
        match $book isa book, has isbn \"{isbn}\";
              $contributor isa contributor;
              ($book, $contributor_role: $contributor) isa! $contribution_type;
              $contribution_type relates $contributor_role;
        fetch {{ "name": $contributor.name, "role": $contributor_role }};
        """,
        f"""
        match $book isa book, has isbn \"{isbn}\";
              $publisher isa publisher;
              $publication isa publication;
              ($book, $publisher, $publication) isa publishing;
        fetch {{ "publisher": $publisher.name, "year": $publication.year }};
        """,
    ]


def print_book_details(driver) -> None:
    isbn = input("Enter ISBN-13 or ISBN-10: ").strip()
    with driver.transaction(DATABASE, TransactionType.READ) as tx:
        answers = [tx.query(q).resolve() for q in get_book_details_query(isbn)]

        # format
        docs = answers[0].as_concept_documents()
        try:
            first = next(docs)
        except StopIteration:
            print(f"No book found with ISBN: \"{isbn}\"")
            return

        print(f"\nFormat: {first['format']['label']}")
        for (key, attr) in first["attributes"].items():
            print(f"{key.replace('-', ' ').title().replace('Isbn ', 'ISBN-')}: {attr}")

        # contributors
        for doc in answers[1].as_concept_documents():
            print(f"{doc['role']['label'].split(':')[1]}: {doc['name']}")

        # publishing
        pub = next(answers[2].as_concept_documents())
        print(f"Publisher: {pub['publisher']}")
        print(f"Year: {pub['year']}")


def search_books(driver) -> None:
    print("Available filters:")
    print(" - Format.")
    print(" - Genres.")
    print(" - Contributor names.")
    print(" - Publisher name.")
    print(" - Publication year.")

    if input("Filter on format? (Y/N): ").strip().upper() == "Y":
        try:
            format_ = BookFormat(input("Enter format: ").strip())
        except ValueError:
            print("Not a valid book format.")
            return
    else:
        format_ = None

    if input("Filter on genres? (Y/N): ").strip().upper() == "Y":
        genres = []
        while True:
            genres.append(input("Enter genre: ").strip())
            if input("Filter on another genre? (Y/N): ").strip().upper() != "Y":
                break
    else:
        genres = None

    if input("Filter on contributors? (Y/N): ").strip().upper() == "Y":
        contributor_names = []
        while True:
            contributor_names.append(input("Enter contributor name: ").strip())
            if input("Filter on another contributor? (Y/N): ").strip().upper() != "Y":
                break
    else:
        contributor_names = None

    if input("Filter on publisher? (Y/N): ").strip().upper() == "Y":
        publisher_name = input("Enter publisher name: ")
    else:
        publisher_name = None

    if input("Filter on publication year? (Y/N): ").strip().upper() == "Y":
        publication_year = int(input("Enter year: "))
    else:
        publication_year = None

    query = filter_books("match", format_, genres, contributor_names, publisher_name, publication_year)
    with driver.transaction(DATABASE, TransactionType.READ) as tx:
        docs = tx.query(query).resolve().as_concept_documents()
        found = False
        for doc in docs:
            print(f"\nISBN-13: {doc['isbn-13']}")
            found = True
        if not found:
            print("\nNo results found.")


if __name__ == "__main__":
    # Update credentials/options as needed for your deployment
    credentials = Credentials("admin", "password")
    options = DriverOptions(is_tls_enabled=False, tls_root_ca_path=None)
    with TypeDB.driver(ADDRESS, credentials, options) as driver:
        while True:
            print("Available commands:")
            print(" - R: Retrieve book details by ISBN.")
            print(" - S: Search for books.")
            print(" - Q: Quit.")

            command = input("Enter command: ").strip().upper()

            if command == "R":
                print_book_details(driver)
            elif command == "S":
                search_books(driver)
            elif command == "Q":
                break
            else:
                print(f"Unrecognised command: \"{command}\"")

            print()