# Data pipelines

This reference covers the usage of data pipelines in TypeQL. Data pipeline read and modify data in stages.

## [](#pipeline_kinds)Pipeline kinds

*   A **write pipeline** may comprise stages that modify data in the database (such as `insert` or `delete`).
    
*   A **read pipeline** only comprises stages that read data from the database but do not modify it.
    

## [](#_data_manipulation_stages)Data manipulation stages

[Match stage](match/index.md)

Used to match existing data instances in the database.

[Fetch stage](fetch/index.md)

Used to format the output of a data pipeline into JSON.

[Insert stage](insert/index.md)

Used to add new data instances into the database.

[Delete stage](delete/index.md)

Used to remove existing data instances from the database.

[Update stage](update/index.md)

Used to modify existing data instances in the database.

[Put stage](put/index.md)

Used to add new data instances to the database if they do not already exist.

## [](#_stream_manipulation_stages)Stream manipulation stages

[Select operator](select/index.md)

Used to keep specified variables for each element of the data stream and remove the rest.

[Require operator](require/index.md)

Used to remove elements from the data stream that do not contain specified optional variables.

[Distinct operator](distinct/index.md)

Used to remove duplicate elements from the data stream.

[Sort operator](sort/index.md)

Used to order the elements of the data stream based on the value of specified variables.

[Limit operator](limit/index.md)

Used to keep a specified number of elements of the data stream and remove the rest.

[Offset operator](offset/index.md)

Used to remove a specified number of elements from the data stream.

[Reduce operator](reduce/index.md)

Used to perform reduction operations on the data stream according to specified groupings.

## [](#_preamble_stage)Preamble stage

[Preamble](with/index.md)

Used to define functions on an ad-hoc basis for use in the data pipeline.

## [](#_query_terminator)Query terminator

[End clause](end/index.md)

When writing multiple queries together (for example in TypeQL console scripts), used to explicitly mark the end of a query.

[Redefine](../schema/redefine/index.md) [Match](match/index.md)

[Edit on GitHub](https://github.com/typedb/typedb-docs/edit/3.x-development/typeql-reference/modules/ROOT/pages/pipelines/index.adoc) Edit this page on GitHub.