TypeDB Enterprise

Run TypeDB Enterprise on your own infrastructure, a self-hosted deployment of TypeDB, fully under your control. TypeDB Enterprise servers can be clustered for high availability.

TypeDB Enterprise is distributed and installed separately from other editions of TypeDB and requires a license.

TypeDB is also available as a fully managed cloud service with no installation required at cloud.typedb.com.

License

To access TypeDB Enterprise, contact our sales team via e-mail or contact form.

System Requirements

TypeDB Enterprise runs on macOS, Linux, and Windows.

Download and Install

Install your TypeDB Enterprise distribution with Docker or via manual download and installation.

You can deploy a cluster of TypeDB Enterprise servers manually or via Kubernetes.

Docker

The TypeDB Enterprise image is hosted in our private Docker repository. Make sure to use docker login first to authenticate.

To pull the TypeDB Enterprise Docker image, run:

docker pull typedb/typedb-cluster:latest

Use docker create to create a new container with the downloaded image. The container’s entrypoint is typedb server, so server options are passed as arguments. A TypeDB Enterprise server listens on port 1729 for gRPC, 8000 for HTTP, and on its clustering address (port 1730) for replication between servers. Because the server binds the clustering address itself, run the container on the host network so that the address other servers use is one it can bind. To ensure that data is preserved even when the container is killed or recreated, mount Docker volumes for both the data directory and the clustering directory:

docker create --name typedb --network host \
    -v $(pwd)/db/data:/opt/typedb-cluster-all-linux-x86_64/server/data \
    -v $(pwd)/db/clustering:/opt/typedb-cluster-all-linux-x86_64/server/clustering \
    typedb/typedb-cluster:latest \
    --diagnostics.deployment-id=my-cluster \
    --server.clustering.id=1 \
    --server.clustering.address=<host-ip>:1730 \
    --server.advertise-address=<host-ip>:1729 \
    --server.http.advertise-address=http://<host-ip>:8000 \
    --server.admin.enabled=true

Replace <host-ip> with the host’s address at which clients and other servers reach this server, and give every server its own --server.clustering.id. The image is published for x86_64 and arm64; on arm64 hosts, the paths inside the container start with /opt/typedb-cluster-all-linux-arm64.

The first server of a new cluster (replica id 1) must start once with --initialize.create-cluster=true. The arguments given to docker create are passed again on every docker start, so do not add the flag to the container. Instead, run the first start in a temporary container that uses the same volumes:

docker run --rm --network host --volumes-from typedb \
    typedb/typedb-cluster:latest \
    --diagnostics.deployment-id=my-cluster \
    --server.clustering.id=1 \
    --server.clustering.address=<host-ip>:1730 \
    --server.advertise-address=<host-ip>:1729 \
    --server.http.advertise-address=http://<host-ip>:8000 \
    --initialize.create-cluster=true

Once it prints Ready to process requests, stop it with kbd:[Ctrl+C] and start the typedb container as shown below. The other servers need no such step: they wait to be registered, which you do with the admin tool inside the primary’s container, for example:

docker exec typedb ./typedb admin --socket-path ./server/data/admin.sock \
    --command "servers register 2 <server-2-ip>:1730"

See Deploying manually for the full procedure.

The port numbers are configurable. For example, you could configure TypeDB Enterprise to listen on port 80 instead of the default port 1729.

Start and Stop TypeDB Enterprise in Docker

To start the Docker container, or restart a stopped container:

docker start typedb

To check the containers running:

docker ps

To stop the Docker container:

docker stop typedb

Manual installation

Download the latest TypeDB Enterprise release from the private release repository (you need access to it), unzip it in a location on your machine that is easily accessible via terminal.

If TypeDB doesn’t have a distribution you need, please open an issue on GitHub.

Start and Stop TypeDB Enterprise manually

To start TypeDB Enterprise manually:

  1. Navigate into the directory with the unpacked files of TypeDB Enterprise.

  2. Run:

    ./typedb server

Now TypeDB Enterprise should show a welcome screen with its version and connection addresses.

To stop TypeDB Enterprise:

Close the terminal or press kbd:[Ctrl+C].

Connecting

To check whether TypeDB Enterprise is working and interact with it, connect with TypeDB Studio or TypeDB Console. Note that TypeDB Console is also available from your TypeDB Enterprise directory:

./typedb console --address 127.0.0.1:1729 --username admin --tls-disabled

To run TypeDB Console from a Docker container, run:

docker exec -ti typedb bash -c '/opt/typedb-cluster-all-linux-x86_64/typedb console --address 127.0.0.1:1729 --username admin --tls-disabled'

Console prompts for the password; the default password of the admin user is password, change it on first use. Drop --tls-disabled once you have enabled TLS on the server.

Deploying manually

Multi-server setup

TypeDB Enterprise can run as a single server, but a highly available, fault-tolerant production setup is a cluster of several servers that replicate the data between them. At any point in time, one server is the primary and the others are secondaries; see Clustered TypeDB for how a cluster works. To tolerate N servers failing, a cluster needs 2N + 1 servers. This section sets up a 3-server cluster, which tolerates the failure of one server.

Each server in a cluster binds to three network ports: a gRPC port for TypeDB drivers and Console (1729 by default), an HTTP port for the HTTP API and Studio (8000), and a clustering port for server-to-server replication (1730). It also serves its monitoring endpoint on port 4104.

For this tutorial, all three servers are on the same private network with these ports open, and have the IP addresses 10.0.0.1, 10.0.0.2, and 10.0.0.3.

To try a cluster on a single machine, use 127.0.0.1 for every address and prefix each port with the server number:

  • 11729, 18000, 11730, 14104 for server 1;

  • 21729, 28000, 21730, 24104 for server 2;

  • 31729, 38000, 31730, 34104 for server 3.

Set these ports in each server’s listen addresses (--server.listen-address, --server.http.listen-address), advertise addresses, clustering address, and --diagnostics.monitoring.port. Run each server from its own copy of the distribution, or also give each one its own --storage.data-directory, --storage.clustering-directory, --logging.directory, and --server.admin.socket-path.

Starting the servers

Every server needs a unique replica id (server.clustering.id), its clustering address (server.clustering.address, which it also listens on for replication), the cluster’s deployment id (diagnostics.deployment-id), and the addresses that clients should use to reach it (server.advertise-address and server.http.advertise-address). The shipped server/config.yml holds localhost placeholders for these; set them in the file or pass them as command-line arguments, which take priority over the config file. See Cluster configuration.

Example 1. Server #1

On the machine with the IP address 10.0.0.1, start the first server with --initialize.create-cluster=true. This flag lets replica 1 create a new cluster on its first start. Leave it out of every later start, including startup scripts and service definitions: a server started with it on empty storage (for example, after its disk was replaced) creates a new, separate cluster instead of waiting to be registered into the existing one.

./typedb server \
    --diagnostics.deployment-id=my-cluster \
    --server.clustering.id=1 \
    --server.clustering.address=10.0.0.1:1730 \
    --server.advertise-address=10.0.0.1:1729 \
    --server.http.advertise-address=http://10.0.0.1:8000 \
    --server.admin.enabled=true \
    --initialize.create-cluster=true

The server prints Ready to process requests and already serves clients as a cluster of one.

Example 2. Servers #2 and #3

On 10.0.0.2, start the second server without the --initialize.create-cluster flag:

./typedb server \
    --diagnostics.deployment-id=my-cluster \
    --server.clustering.id=2 \
    --server.clustering.address=10.0.0.2:1730 \
    --server.advertise-address=10.0.0.2:1729 \
    --server.http.advertise-address=http://10.0.0.2:8000 \
    --server.admin.enabled=true

Start the third server on 10.0.0.3 the same way with --server.clustering.id=3 and its own addresses. Both servers wait to be registered.

Registering the servers

On the machine of server #1 (the primary), register the two new servers with the admin tool, giving each one’s replica id and clustering address:

./typedb admin --socket-path ./server/data/admin.sock \
    --command "servers register 2 10.0.0.2:1730" \
    --command "servers register 3 10.0.0.3:1730"

Each new server receives the data from the primary and becomes a voting secondary once it has caught up. Once they have caught up, check the result:

./typedb admin --socket-path ./server/data/admin.sock --command "servers status"
id | address       | role      | term | status
-------------------------------------------------
1  | 10.0.0.1:1729 | primary   | 1    | available
2  | 10.0.0.2:1729 | secondary | 1    | available
3  | 10.0.0.3:1729 | secondary | 1    | available

The cluster is ready. Connect with TypeDB Console or a TypeDB driver to any of the three addresses; requests are routed to the primary automatically. See Administering a cluster for adding and removing servers later, failover, and migrating an existing TypeDB CE server into a cluster.

Example 3. Clients on a different network

Clients connect to every server at its advertise address, not only at the address they were given. When clients reach the servers through addresses other than the ones the servers use among themselves (for example, public hostnames in front of a private network), set the advertise addresses to the client-facing ones and keep the clustering addresses private. For example, the first start of server #1 becomes:

./typedb server \
    --diagnostics.deployment-id=my-cluster \
    --server.clustering.id=1 \
    --server.clustering.address=10.0.0.1:1730 \
    --server.advertise-address=typedb-1.example.com:1729 \
    --server.http.advertise-address=http://typedb-1.example.com:8000 \
    --server.admin.enabled=true \
    --initialize.create-cluster=true

In this case, ports 1729 and 8000 need to be reachable by clients at the advertised hostnames, while port 1730 only needs to be open between the servers. If no single address per server works for all clients, keep the advertise addresses private and let clients use address translation instead; see How clients reach the replicas. With TLS enabled, the HTTP advertise address uses https://.

Stopping servers

Stopping a clustered server is done the same way as a single server: close the terminal or press kbd:[Ctrl+C]. Each server is shut down independently. As long as a majority of the servers is running, the cluster keeps serving.

Migrating from TypeDB Community Edition

An existing TypeDB CE data directory can be migrated in place to become the first server of a TypeDB Enterprise cluster. See Migrating a TypeDB CE server into a cluster.

Deploying with Kubernetes

To deploy a TypeDB Enterprise cluster with Kubernetes, we can use the Helm package manager.

The Helm chart described below is the TypeDB Enterprise 2.x chart. Kubernetes instructions for TypeDB Enterprise 3.x clusters are being prepared; until then, deploy the 3.x servers with the manual procedure above.

Requirements

  • kubectl installed and configured for your Kubernetes deployment. You can use minikube for local testing.

  • helm installed.

Initial Setup

First, create a secret to access TypeDB Enterprise image on Docker Hub:

kubectl create secret docker-registry private-docker-hub --docker-server=https://index.docker.io/v2/ \
--docker-username=USERNAME --docker-password='PASSWORD' --docker-email=EMAIL

You can use an access token instead of password.

Next, add the TypeDB Helm repo:

helm repo add typedb https://repo.typedb.com/repository/helm/

Encryption setup (optional)

This step is necessary if you wish to deploy TypeDB Enterprise with in-flight encryption support. There are two certificates that need to be configured: external certificate (TLS) and internal certificate (Curve). The certificates need to be generated and then added to Kubernetes Secrets.

An external certificate can either be obtained from trusted third party providers such as CloudFlare or letsencrypt.org. Alternatively, it is also possible to generate it manually with a tool we provide with TypeDB Enterprise in the tool directory:

java -jar encryption-gen.jar --ca-common-name=<x500-common-name> --hostname=<external-hostname>,<internal-hostname>

Please note that an external certificate is always bound to URL address, not IP address.

Ensure the external certificate (<external-hostname> in the command above) is bound to *.<helm-release-name>. For example, for a Helm release named typedb-cloud, the certificate needs to be bound to *.typedb-cloud.

The encryption-gen.jar generates three directories:

  • internal-ca and external-ca — The CA keypairs stored only in case you want to sign more certificates in the future.

  • <external-hostname> — A directory with certificates to be stored on all servers in the cluster.

All files from the directory named after external domain shall be copied to the proper directory on every server in a cluster. By default, they are stored in /server/conf/encryption inside the TypeDB Enterprise main directory. For example, typedb-cloud-all-mac-x86_64-2.25.12/server/conf/encryption. The path to each file is configured in the encryption section of the TypeDB Enterprise config file.

Once the external and internal certificates are all generated, we can upload it to Kubernetes Secrets. Navigate into the directory with cluster certificates and run:

kubectl create secret generic ext-grpc \
  --from-file ext-grpc-certificate.pem \
  --from-file ext-grpc-private-key.pem \
  --from-file ext-grpc-root-ca.pem
kubectl create secret generic int-grpc \
    --from-file int-grpc-certificate.pem \
    --from-file int-grpc-private-key.pem \
    --from-file int-grpc-root-ca.pem
kubectl create secret generic int-zmq \
  --from-file int-zmq-private-key \
  --from-file int-zmq-public-key

Deploying a cluster via K8s

There are three alternative deployment modes that you can choose from:

  • Private Cluster — For applications (clients) that are located within the same Kubernetes network as the cluster.

  • Public Cluster — To access the cluster from outside the Kubernetes network.

  • Public Cluster (Minikube) — To deploy a development cluster on your local machine.

Deploying a Private Cluster

This deployment mode is preferred if your application is located within the same Kubernetes network as the cluster. In order to deploy in this mode, ensure that the exposed flag is set to false.

In-flight encryption
  • Without encryption

  • With encryption

Deploy:

helm install typedb-cloud typedb/typedb-cloud --set "exposed=false,encrypted=false"

To enable in-flight encryption for your private cluster, make sure the encrypted flag is set to true:

helm install typedb-cloud typedb/typedb-cloud --set "exposed=false,encrypted=true" \
--set servers=3,cpu=1,storage.persistent=false,storage.size=1Gi,exposed=true,domain=localhost-ext --set encryption.enable=true --set encryption.enable=true,encryption.externalGRPC.secretName=ext-grpc,encryption.externalGRPC.content.privateKeyName=ext-grpc-private-key.pem,encryption.externalGRPC.content.certificateName=ext-grpc-certificate.pem,encryption.externalGRPC.content.rootCAName=ext-grpc-root-ca.pem \
--set encryption.internalGRPC.secretName=int-grpc,encryption.internalGRPC.content.privateKeyName=int-grpc-private-key.pem,encryption.internalGRPC.content.certificateName=int-grpc-certificate.pem,encryption.internalGRPC.content.rootCAName=int-grpc-root-ca.pem \
--set encryption.internalZMQ.secretName=int-zmq,encryption.internalZMQ.content.privateKeyName=int-zmq-private-key,encryption.internalZMQ.content.publicKeyName=int-zmq-public-key

The servers will be accessible via the internal hostname within the Kubernetes network, i.e., typedb-cloud-0.typedb-cloud, typedb-cloud-1.typedb-cloud, and typedb-cloud-2.typedb-cloud.

Deploying a Public Cluster

This deployment mode is preferred if you need to access the cluster from outside the Kubernetes network. For example, if you need to access the cluster from TypeDB Studio or TypeDB Console running on your local machine.

Deploying a public cluster can be done by setting the exposed flag to true.

Technically, the servers are made public by binding each one to a LoadBalancer instance which is assigned a public IP/hostname. The IP/hostname assignments are done automatically by the cloud provider that the Kubernetes platform is running on.

In-flight encryption
  • Without encryption

  • With encryption

Deploy:

helm install typedb-cloud typedb/typedb-cloud --set "exposed=true"

Once the deployment has been completed, the servers can be accessible via public IPs/hostnames assigned to the Kubernetes LoadBalancer services. The addresses can be obtained with this command:

kubectl get svc -l external-ip-for=typedb-cloud \
-o='custom-columns=NAME:.metadata.name,IP OR HOSTNAME:.status.loadBalancer.ingress[0].*'

To enable in-flight encryption, the servers must be assigned URL addresses. This restriction comes from the fact that external certificates must be bound to a domain name, and not an IP address.

Given a "domain name" and a "Helm release name", the address structure of the servers will follow the specified format:

<helm-release-name>-{0..n}.<domain-name>

The format must be taken into account when generating the external certificate of all servers such that they’re properly bound to the address. For example, you can generate an external certificate using wildcard, i.e., *.<helm-release-name>.<domain-name>, that can be shared by all servers.

Once the domain name and external certificate have been configured accordingly, we can proceed to perform the deployment. Ensure that the encrypted flag is set to true and the domain flag set accordingly.

helm install typedb-cloud typedb/typedb-cloud --set "exposed=true,encrypted=true,domain=<domain-name>" \
--set servers=3,cpu=1,storage.persistent=false,storage.size=1Gi,exposed=true,domain=localhost-ext --set encryption.enable=true --set encryption.enable=true,encryption.externalGRPC.secretName=ext-grpc,encryption.externalGRPC.content.privateKeyName=ext-grpc-private-key.pem,encryption.externalGRPC.content.certificateName=ext-grpc-certificate.pem,encryption.externalGRPC.content.rootCAName=ext-grpc-root-ca.pem \
--set encryption.internalGRPC.secretName=int-grpc,encryption.internalGRPC.content.privateKeyName=int-grpc-private-key.pem,encryption.internalGRPC.content.certificateName=int-grpc-certificate.pem,encryption.internalGRPC.content.rootCAName=int-grpc-root-ca.pem \
--set encryption.internalZMQ.secretName=int-zmq,encryption.internalZMQ.content.privateKeyName=int-zmq-private-key,encryption.internalZMQ.content.publicKeyName=int-zmq-public-key

After the deployment has been completed, we need to configure these URL addresses to correctly point to the servers. This can be done by configuring the A record (for IPs) or CNAME record (for hostnames) of all the servers in your trusted DNS provider:

typedb-cloud-0.typedb-cloud.example.com => <public IP/hostname of typedb-cloud-0 service>
typedb-cloud-1.typedb-cloud.example.com => <public IP/hostname of typedb-cloud-1 service>
typedb-cloud-2.typedb-cloud.example.com => <public IP/hostname of typedb-cloud-2 service>

Deploying a Public Cluster with Minikube

Please note that in-flight encryption cannot be enabled in this configuration.

This deployment mode is primarily intended for development purposes as it runs a K8s cluster locally.

Ensure to have Minikube installed and running.

Deploy, adjusting the parameters for CPU and storage to run on a local machine:

helm install typedb-cloud typedb/typedb-cloud --set image.pullPolicy=Always,servers=3,singlePodPerNode=false,cpu=1,storage.persistent=false,storage.size=1Gi,exposed=true,javaopts=-Xmx4G --set encryption.enable=false

Once the deployment has been completed, enable tunneling from another terminal:

minikube tunnel

K8s cluster status check

To check the status of a cluster:

kubectl describe sts typedb-cloud

It should show Pods Status field as Running for all the nodes after a few minutes after deploying a TypeDB Enterprise cluster.

You can connect to a pod:

kubectl exec --stdin --tty typedb-cloud-0 -- /bin/bash

K8s cluster removal

To stop and remove a K8s cluster from Kubernetes, use the helm uninstall with the helm release name:

helm uninstall typedb-cloud

K8s troubleshooting

To see pod details for the typedb-cloud-0 pod:

kubectl describe pod typedb-cloud-0

The following are the common error scenarios and how to troubleshoot them.

All pods are stuck in ErrImagePull or ImagePullBackOff state

This means the secret to pull the image from Docker Hub has not been created. Make sure you’ve followed Initial Setup instructions and verify that the pull secret is present by executing kubectl get secret/private-docker-hub. Correct state looks like this:

 kubectl get secret/private-docker-hub
 NAME                 TYPE                             DATA   AGE
 private-docker-hub   kubernetes.io/dockerconfigjson   1      11d

One or more pods of TypeDB Enterprise are stuck in Pending state

This might mean pods requested more resources than available. To check if that’s the case, run on a stuck pod (e.g. typedb-cloud-0):

kubectl describe pod/typedb-cloud-0

Error message similar to 0/1 nodes are available: 1 Insufficient cpu. or 0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims. indicates that cpu or storage.size settings need to be decreased.

One or more pods of TypeDB Enterprise are stuck in CrashLoopBackOff state

This might indicate a misconfiguration of TypeDB Enterprise. Please check the logs:

kubectl logs pod/typedb-cloud-0

Helm configuration reference

Configurable settings for Helm package include:

Key Default value Description

name

null

Used for naming deployed objects. When not provided, the Helm release name will be used instead.

image.repository

vaticle/typedb-cloud

The docker hub organization and repository from which to pull an appropriate image.

image.tag

2.25.9

The version of TypeDB Enterprise to use.

image.pullPolicy

IfNotPresent

Image pulling policy.
For more information, see the image pull policy in Kubernetes documentation.

image.pullSecret

-

The name of a secret containing a container image registry key used to authenticate against the image repository.

exposed

false

Whether TypeDB Enterprise supports connections via public IP/hostname (outside of Kubernetes network).

serviceAnnotations

null

Kubernetes annotations to be added to the Kubernetes services responsible for directing traffic to the TypeDB Enterprise pods.

tolerations

[]

Kubernetes tolerations of taints on nodes.
For more information, see the tolerations in Kubernetes documentation.

Example
[key: "typedb-cloud-only"
    operator: "Exists"
    effect: "NoSchedule"]`

nodeAffinities

{}

Kubernetes node affinities.
For more information, see the node affinities in Kubernetes documentation.

podAffinities

{}

Kubernetes pod affinities.
For more information, see the pod affinities in Kubernetes documentation.

podAntiAffinities

{}

Kubernetes pod anti-affinities.
For more information, see the pod affinities in Kubernetes documentation.

singlePodPerNode

true

Whether a pod should share nodes with other TypeDB Enterprise instances from the same Helm installation.

Warning: changing this to false and making no anti-affinities of your own will allow Kubernetes to place multiple cluster servers on the same node, negating the high-availability guarantees of TypeDB Enterprise.

podLabels

{}

Kubernetes pod labels.
For more information, see the pod labels in Kubernetes documentation.

servers

3

Number of TypeDB Enterprise servers to run.

resources

{}

Kubernetes resources specification.
For more information, see the resource requests and limits in Kubernetes documentation.

storage.size

100Gi

How much disk space should be allocated for each TypeDB Enterprise server.

storage.persistent

true

Whether TypeDB Enterprise should use a persistent volume to store data.

encryption.enabled

false

Whether TypeDB Enterprise uses an in-flight encryption.

encryption.externalGRPC

Encryption settings for client-server communications.

encryption.internalGRPC

Encryption settings for cluster management, e.g., creating a database on all replicas.

encryption.internalZMQ

Encryption settings for data replication.

authentication.password.​disallowDefault

false

Check whether the admin account has the default password.

logstash.enabled

false

Whether TypeDB Enterprise pushes logs into Logstash

logstash.uri

""

Hostname and port of a Logstash daemon accepting log records

Current Limitations

TypeDB Enterprise doesn’t support dynamic reconfiguration of server count without restarting all the servers.