TypeDB Enterprise
Run TypeDB Enterprise on your own infrastructure, a self-hosted deployment of TypeDB, fully under your control. TypeDB Enterprise servers can be clustered for high availability.
TypeDB Enterprise is distributed and installed separately from other editions of TypeDB and requires a license.
|
TypeDB is also available as a fully managed cloud service with no installation required at cloud.typedb.com. |
License
To access TypeDB Enterprise, contact our sales team via e-mail or contact form.
Download and Install
Install your TypeDB Enterprise distribution with Docker or via manual download and installation.
You can deploy a cluster of TypeDB Enterprise servers manually or via Kubernetes.
Docker
The TypeDB Enterprise image is hosted in our private Docker repository.
Make sure to use docker login first to authenticate.
To pull the TypeDB Enterprise Docker image, run:
docker pull typedb/typedb-cluster:latest
Use docker create to create a new container with the downloaded image.
The container’s entrypoint is typedb server, so server options are passed as arguments.
A TypeDB Enterprise server listens on port 1729 for gRPC, 8000 for HTTP, and on its clustering address (port 1730) for replication between servers.
Because the server binds the clustering address itself, run the container on the host network so that the address other servers use is one it can bind.
To ensure that data is preserved even when the container is killed or recreated, mount Docker volumes for both the data directory and the clustering directory:
docker create --name typedb --network host \
-v $(pwd)/db/data:/opt/typedb-cluster-all-linux-x86_64/server/data \
-v $(pwd)/db/clustering:/opt/typedb-cluster-all-linux-x86_64/server/clustering \
typedb/typedb-cluster:latest \
--diagnostics.deployment-id=my-cluster \
--server.clustering.id=1 \
--server.clustering.address=<host-ip>:1730 \
--server.advertise-address=<host-ip>:1729 \
--server.http.advertise-address=http://<host-ip>:8000 \
--server.admin.enabled=true
Replace <host-ip> with the host’s address at which clients and other servers reach this server, and give every server its own --server.clustering.id.
The image is published for x86_64 and arm64; on arm64 hosts, the paths inside the container start with /opt/typedb-cluster-all-linux-arm64.
The first server of a new cluster (replica id 1) must start once with --initialize.create-cluster=true.
The arguments given to docker create are passed again on every docker start, so do not add the flag to the container.
Instead, run the first start in a temporary container that uses the same volumes:
docker run --rm --network host --volumes-from typedb \
typedb/typedb-cluster:latest \
--diagnostics.deployment-id=my-cluster \
--server.clustering.id=1 \
--server.clustering.address=<host-ip>:1730 \
--server.advertise-address=<host-ip>:1729 \
--server.http.advertise-address=http://<host-ip>:8000 \
--initialize.create-cluster=true
Once it prints Ready to process requests, stop it with kbd:[Ctrl+C] and start the typedb container as shown below.
The other servers need no such step: they wait to be registered, which you do with the admin tool inside the primary’s container, for example:
docker exec typedb ./typedb admin --socket-path ./server/data/admin.sock \
--command "servers register 2 <server-2-ip>:1730"
See Deploying manually for the full procedure.
|
The port numbers are configurable. For example, you could configure TypeDB Enterprise to listen on port 80 instead of the default port 1729. |
Manual installation
Download the latest TypeDB Enterprise release from the private release repository (you need access to it), unzip it in a location on your machine that is easily accessible via terminal.
If TypeDB doesn’t have a distribution you need, please open an issue on GitHub.
Start and Stop TypeDB Enterprise manually
To start TypeDB Enterprise manually:
-
Navigate into the directory with the unpacked files of TypeDB Enterprise.
-
Run:
./typedb server
Now TypeDB Enterprise should show a welcome screen with its version and connection addresses.
To stop TypeDB Enterprise:
Close the terminal or press kbd:[Ctrl+C].
Connecting
To check whether TypeDB Enterprise is working and interact with it, connect with TypeDB Studio or TypeDB Console. Note that TypeDB Console is also available from your TypeDB Enterprise directory:
./typedb console --address 127.0.0.1:1729 --username admin --tls-disabled
To run TypeDB Console from a Docker container, run:
docker exec -ti typedb bash -c '/opt/typedb-cluster-all-linux-x86_64/typedb console --address 127.0.0.1:1729 --username admin --tls-disabled'
Console prompts for the password; the default password of the admin user is password, change it on first use.
Drop --tls-disabled once you have enabled TLS on the server.
Deploying manually
Multi-server setup
TypeDB Enterprise can run as a single server, but a highly available, fault-tolerant production setup is a cluster of several servers that replicate the data between them.
At any point in time, one server is the primary and the others are secondaries; see Clustered TypeDB for how a cluster works.
To tolerate N servers failing, a cluster needs 2N + 1 servers.
This section sets up a 3-server cluster, which tolerates the failure of one server.
Each server in a cluster binds to three network ports:
a gRPC port for TypeDB drivers and Console (1729 by default), an HTTP port for the HTTP API and Studio (8000), and a clustering port for server-to-server replication (1730).
It also serves its monitoring endpoint on port 4104.
For this tutorial, all three servers are on the same private network with these ports open, and have the IP addresses 10.0.0.1, 10.0.0.2, and 10.0.0.3.
|
To try a cluster on a single machine, use
Set these ports in each server’s listen addresses ( |
Starting the servers
Every server needs a unique replica id (server.clustering.id), its clustering address (server.clustering.address, which it also listens on for replication), the cluster’s deployment id (diagnostics.deployment-id), and the addresses that clients should use to reach it (server.advertise-address and server.http.advertise-address).
The shipped server/config.yml holds localhost placeholders for these; set them in the file or pass them as command-line arguments, which take priority over the config file. See Cluster configuration.
On the machine with the IP address 10.0.0.1, start the first server with --initialize.create-cluster=true.
This flag lets replica 1 create a new cluster on its first start.
Leave it out of every later start, including startup scripts and service definitions: a server started with it on empty storage (for example, after its disk was replaced) creates a new, separate cluster instead of waiting to be registered into the existing one.
./typedb server \
--diagnostics.deployment-id=my-cluster \
--server.clustering.id=1 \
--server.clustering.address=10.0.0.1:1730 \
--server.advertise-address=10.0.0.1:1729 \
--server.http.advertise-address=http://10.0.0.1:8000 \
--server.admin.enabled=true \
--initialize.create-cluster=true
The server prints Ready to process requests and already serves clients as a cluster of one.
On 10.0.0.2, start the second server without the --initialize.create-cluster flag:
./typedb server \
--diagnostics.deployment-id=my-cluster \
--server.clustering.id=2 \
--server.clustering.address=10.0.0.2:1730 \
--server.advertise-address=10.0.0.2:1729 \
--server.http.advertise-address=http://10.0.0.2:8000 \
--server.admin.enabled=true
Start the third server on 10.0.0.3 the same way with --server.clustering.id=3 and its own addresses.
Both servers wait to be registered.
Registering the servers
On the machine of server #1 (the primary), register the two new servers with the admin tool, giving each one’s replica id and clustering address:
./typedb admin --socket-path ./server/data/admin.sock \
--command "servers register 2 10.0.0.2:1730" \
--command "servers register 3 10.0.0.3:1730"
Each new server receives the data from the primary and becomes a voting secondary once it has caught up. Once they have caught up, check the result:
./typedb admin --socket-path ./server/data/admin.sock --command "servers status"
id | address | role | term | status
-------------------------------------------------
1 | 10.0.0.1:1729 | primary | 1 | available
2 | 10.0.0.2:1729 | secondary | 1 | available
3 | 10.0.0.3:1729 | secondary | 1 | available
The cluster is ready. Connect with TypeDB Console or a TypeDB driver to any of the three addresses; requests are routed to the primary automatically. See Administering a cluster for adding and removing servers later, failover, and migrating an existing TypeDB CE server into a cluster.
Clients connect to every server at its advertise address, not only at the address they were given. When clients reach the servers through addresses other than the ones the servers use among themselves (for example, public hostnames in front of a private network), set the advertise addresses to the client-facing ones and keep the clustering addresses private. For example, the first start of server #1 becomes:
./typedb server \
--diagnostics.deployment-id=my-cluster \
--server.clustering.id=1 \
--server.clustering.address=10.0.0.1:1730 \
--server.advertise-address=typedb-1.example.com:1729 \
--server.http.advertise-address=http://typedb-1.example.com:8000 \
--server.admin.enabled=true \
--initialize.create-cluster=true
In this case, ports 1729 and 8000 need to be reachable by clients at the advertised hostnames, while port 1730 only needs to be open between the servers.
If no single address per server works for all clients, keep the advertise addresses private and let clients use address translation instead; see How clients reach the replicas.
With TLS enabled, the HTTP advertise address uses https://.
Migrating from TypeDB Community Edition
An existing TypeDB CE data directory can be migrated in place to become the first server of a TypeDB Enterprise cluster. See Migrating a TypeDB CE server into a cluster.
Deploying with Kubernetes
To deploy a TypeDB Enterprise cluster with Kubernetes, we can use the Helm package manager.
|
The Helm chart described below is the TypeDB Enterprise 2.x chart. Kubernetes instructions for TypeDB Enterprise 3.x clusters are being prepared; until then, deploy the 3.x servers with the manual procedure above. |
Requirements
-
kubectlinstalled and configured for your Kubernetes deployment. You can use minikube for local testing. -
helminstalled.
Initial Setup
First, create a secret to access TypeDB Enterprise image on Docker Hub:
kubectl create secret docker-registry private-docker-hub --docker-server=https://index.docker.io/v2/ \
--docker-username=USERNAME --docker-password='PASSWORD' --docker-email=EMAIL
You can use an access token instead of password.
Next, add the TypeDB Helm repo:
helm repo add typedb https://repo.typedb.com/repository/helm/
Encryption setup (optional)
This step is necessary if you wish to deploy TypeDB Enterprise with in-flight encryption support. There are two certificates that need to be configured: external certificate (TLS) and internal certificate (Curve). The certificates need to be generated and then added to Kubernetes Secrets.
An external certificate can either be obtained from trusted third party providers such as
CloudFlare or letsencrypt.org.
Alternatively, it is also possible to generate it manually with a tool we provide with TypeDB Enterprise
in the tool directory:
java -jar encryption-gen.jar --ca-common-name=<x500-common-name> --hostname=<external-hostname>,<internal-hostname>
Please note that an external certificate is always bound to URL address, not IP address.
Ensure the external certificate (<external-hostname> in the command above) is bound to *.<helm-release-name>.
For example, for a Helm release named typedb-cloud, the certificate needs to be bound to *.typedb-cloud.
The encryption-gen.jar generates three directories:
-
internal-caandexternal-ca— The CA keypairs stored only in case you want to sign more certificates in the future. -
<external-hostname> — A directory with certificates to be stored on all servers in the cluster.
All files from the directory
named after external domain shall be copied to the proper directory on every server in a cluster.
By default, they are stored in /server/conf/encryption inside the TypeDB Enterprise main directory.
For example, typedb-cloud-all-mac-x86_64-2.25.12/server/conf/encryption.
The path to each file is configured in the encryption section of the TypeDB Enterprise config file.
Once the external and internal certificates are all generated, we can upload it to Kubernetes Secrets. Navigate into the directory with cluster certificates and run:
kubectl create secret generic ext-grpc \
--from-file ext-grpc-certificate.pem \
--from-file ext-grpc-private-key.pem \
--from-file ext-grpc-root-ca.pem
kubectl create secret generic int-grpc \
--from-file int-grpc-certificate.pem \
--from-file int-grpc-private-key.pem \
--from-file int-grpc-root-ca.pem
kubectl create secret generic int-zmq \
--from-file int-zmq-private-key \
--from-file int-zmq-public-key
Deploying a cluster via K8s
There are three alternative deployment modes that you can choose from:
-
Private Cluster — For applications (clients) that are located within the same Kubernetes network as the cluster.
-
Public Cluster — To access the cluster from outside the Kubernetes network.
-
Public Cluster (Minikube) — To deploy a development cluster on your local machine.
Deploying a Private Cluster
This deployment mode is preferred if your application is located within the same Kubernetes network as the cluster.
In order to deploy in this mode, ensure that the exposed flag is set to false.
-
Without encryption
-
With encryption
Deploy:
helm install typedb-cloud typedb/typedb-cloud --set "exposed=false,encrypted=false"
To enable in-flight encryption for your private cluster, make sure the encrypted flag is set to true:
helm install typedb-cloud typedb/typedb-cloud --set "exposed=false,encrypted=true" \
--set servers=3,cpu=1,storage.persistent=false,storage.size=1Gi,exposed=true,domain=localhost-ext --set encryption.enable=true --set encryption.enable=true,encryption.externalGRPC.secretName=ext-grpc,encryption.externalGRPC.content.privateKeyName=ext-grpc-private-key.pem,encryption.externalGRPC.content.certificateName=ext-grpc-certificate.pem,encryption.externalGRPC.content.rootCAName=ext-grpc-root-ca.pem \
--set encryption.internalGRPC.secretName=int-grpc,encryption.internalGRPC.content.privateKeyName=int-grpc-private-key.pem,encryption.internalGRPC.content.certificateName=int-grpc-certificate.pem,encryption.internalGRPC.content.rootCAName=int-grpc-root-ca.pem \
--set encryption.internalZMQ.secretName=int-zmq,encryption.internalZMQ.content.privateKeyName=int-zmq-private-key,encryption.internalZMQ.content.publicKeyName=int-zmq-public-key
The servers will be accessible via the internal hostname within the Kubernetes network,
i.e.,
typedb-cloud-0.typedb-cloud,
typedb-cloud-1.typedb-cloud, and
typedb-cloud-2.typedb-cloud.
Deploying a Public Cluster
This deployment mode is preferred if you need to access the cluster from outside the Kubernetes network. For example, if you need to access the cluster from TypeDB Studio or TypeDB Console running on your local machine.
Deploying a public cluster can be done by setting the exposed flag to true.
Technically, the servers are made public by binding each one to a LoadBalancer instance which is assigned a public
IP/hostname. The IP/hostname assignments are done automatically by the cloud provider that the Kubernetes platform is
running on.
-
Without encryption
-
With encryption
Deploy:
helm install typedb-cloud typedb/typedb-cloud --set "exposed=true"
Once the deployment has been completed,
the servers can be accessible via public IPs/hostnames assigned to the Kubernetes LoadBalancer services.
The addresses can be obtained with this command:
kubectl get svc -l external-ip-for=typedb-cloud \
-o='custom-columns=NAME:.metadata.name,IP OR HOSTNAME:.status.loadBalancer.ingress[0].*'
To enable in-flight encryption, the servers must be assigned URL addresses. This restriction comes from the fact that external certificates must be bound to a domain name, and not an IP address.
Given a "domain name" and a "Helm release name", the address structure of the servers will follow the specified format:
<helm-release-name>-{0..n}.<domain-name>
The format must be taken into account when generating the external certificate of all servers such that they’re properly
bound to the address.
For example, you can generate an external certificate using wildcard, i.e.,
*.<helm-release-name>.<domain-name>, that can be shared by all servers.
Once the domain name and external certificate have been configured accordingly,
we can proceed to perform the deployment.
Ensure that the encrypted flag is set to true and the domain flag set accordingly.
helm install typedb-cloud typedb/typedb-cloud --set "exposed=true,encrypted=true,domain=<domain-name>" \
--set servers=3,cpu=1,storage.persistent=false,storage.size=1Gi,exposed=true,domain=localhost-ext --set encryption.enable=true --set encryption.enable=true,encryption.externalGRPC.secretName=ext-grpc,encryption.externalGRPC.content.privateKeyName=ext-grpc-private-key.pem,encryption.externalGRPC.content.certificateName=ext-grpc-certificate.pem,encryption.externalGRPC.content.rootCAName=ext-grpc-root-ca.pem \
--set encryption.internalGRPC.secretName=int-grpc,encryption.internalGRPC.content.privateKeyName=int-grpc-private-key.pem,encryption.internalGRPC.content.certificateName=int-grpc-certificate.pem,encryption.internalGRPC.content.rootCAName=int-grpc-root-ca.pem \
--set encryption.internalZMQ.secretName=int-zmq,encryption.internalZMQ.content.privateKeyName=int-zmq-private-key,encryption.internalZMQ.content.publicKeyName=int-zmq-public-key
After the deployment has been completed, we need to configure these URL addresses to correctly point to the servers.
This can be done by configuring the A record (for IPs) or CNAME record (for hostnames) of all the servers in your
trusted DNS provider:
typedb-cloud-0.typedb-cloud.example.com => <public IP/hostname of typedb-cloud-0 service>
typedb-cloud-1.typedb-cloud.example.com => <public IP/hostname of typedb-cloud-1 service>
typedb-cloud-2.typedb-cloud.example.com => <public IP/hostname of typedb-cloud-2 service>
Deploying a Public Cluster with Minikube
Please note that in-flight encryption cannot be enabled in this configuration.
This deployment mode is primarily intended for development purposes as it runs a K8s cluster locally.
Ensure to have Minikube installed and running.
Deploy, adjusting the parameters for CPU and storage to run on a local machine:
helm install typedb-cloud typedb/typedb-cloud --set image.pullPolicy=Always,servers=3,singlePodPerNode=false,cpu=1,storage.persistent=false,storage.size=1Gi,exposed=true,javaopts=-Xmx4G --set encryption.enable=false
Once the deployment has been completed, enable tunneling from another terminal:
minikube tunnel
K8s cluster status check
To check the status of a cluster:
kubectl describe sts typedb-cloud
It should show Pods Status field as Running for all the nodes after a few minutes
after deploying a TypeDB Enterprise cluster.
You can connect to a pod:
kubectl exec --stdin --tty typedb-cloud-0 -- /bin/bash
K8s cluster removal
To stop and remove a K8s cluster from Kubernetes, use the helm uninstall with the helm release name:
helm uninstall typedb-cloud
K8s troubleshooting
To see pod details for the typedb-cloud-0 pod:
kubectl describe pod typedb-cloud-0
The following are the common error scenarios and how to troubleshoot them.
All pods are stuck in ErrImagePull or ImagePullBackOff state
This means the secret to pull the image from Docker Hub has not been created.
Make sure you’ve followed Initial Setup instructions and verify that the pull secret is present by
executing kubectl get secret/private-docker-hub. Correct state looks like this:
kubectl get secret/private-docker-hub
NAME TYPE DATA AGE
private-docker-hub kubernetes.io/dockerconfigjson 1 11d
One or more pods of TypeDB Enterprise are stuck in Pending state
This might mean pods requested more resources than available.
To check if that’s the case, run on a stuck pod (e.g. typedb-cloud-0):
kubectl describe pod/typedb-cloud-0
Error message similar to
0/1 nodes are available: 1 Insufficient cpu. or
0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims.
indicates that cpu or storage.size settings need to be decreased.
Helm configuration reference
Configurable settings for Helm package include:
| Key | Default value | Description |
|---|---|---|
|
|
Used for naming deployed objects. When not provided, the Helm release name will be used instead. |
|
|
The docker hub organization and repository from which to pull an appropriate image. |
|
|
The version of TypeDB Enterprise to use. |
|
|
Image pulling policy. |
|
- |
The name of a secret containing a container image registry key used to authenticate against the image repository. |
|
|
Whether TypeDB Enterprise supports connections via public IP/hostname (outside of Kubernetes network). |
|
|
Kubernetes annotations to be added to the Kubernetes services responsible for directing traffic to the TypeDB Enterprise pods. |
|
|
Kubernetes tolerations of taints on nodes. Example
|
|
|
Kubernetes node affinities. |
|
|
Kubernetes pod affinities. |
|
|
Kubernetes pod anti-affinities. |
|
|
Whether a pod should share nodes with other TypeDB Enterprise instances from the same Helm installation. Warning: changing this to false and making no anti-affinities of your own will allow Kubernetes to place multiple cluster servers on the same node, negating the high-availability guarantees of TypeDB Enterprise. |
|
|
Kubernetes pod labels. |
|
|
Number of TypeDB Enterprise servers to run. |
|
|
Kubernetes resources specification. |
|
|
How much disk space should be allocated for each TypeDB Enterprise server. |
|
|
Whether TypeDB Enterprise should use a persistent volume to store data. |
|
|
Whether TypeDB Enterprise uses an in-flight encryption. |
|
Encryption settings for client-server communications. |
|
|
Encryption settings for cluster management, e.g., creating a database on all replicas. |
|
|
Encryption settings for data replication. |
|
|
|
Check whether the |
|
|
Whether TypeDB Enterprise pushes logs into Logstash |
|
|
Hostname and port of a Logstash daemon accepting log records |