Your Postgres data in ClickHouse, kept in sync, with proof both sides agree.
chlift takes a snapshot of your Postgres tables into ClickHouse, keeps them current with live change data capture, and checks that both sides hold the same data. It is for teams that run their own Postgres and need analytics without hurting it.
chlift will be free and MIT licensed when it is released. The only paid offers are installation, support and training: [email protected]
What we measured
A recorded run in a fresh environment on our dev box, with synthetic data: a Postgres with three tables (8M, 4M and 2M rows) moved by chlift into two ClickHouse replicas, then kept in sync with live changes.
Install from empty nodes
chlift install set up the ClickHouse replicas and their coordination from empty machines (3 nodes, install only, timed with time); all of its checks passed.
14,000,000 rows into both replicas
The snapshot from migrate start. Measured by polling every 15 s, so this is an upper bound.
Equal after live changes
After 50,000 inserts, 1,000 updates and 5,000 deletes, migrate check --checksums found Postgres and both replicas equal: 8,047,000 / 4,000,000 / 1,998,000 rows, with 0 of 64 primary-key ranges differing per table.
- A dev box with synthetic data, not a vendor benchmark. Your data and queries are the real test.
- Query speed-ups: not shown until the benchmark is re-run with its raw samples published next to the numbers.
Two silent data bugs we caught, and what chlift does about them
Replication that fails loudly is annoying. Replication that quietly gives you wrong numbers is worse. Both of these produced no error anywhere.
Long text blanked on updates
With Postgres's default replica identity, unchanged long text/JSON columns were written as empty strings on every UPDATE that did not touch them. In a soak run: 374 rows with empty notes in ClickHouse, 0 in Postgres.
Deleted rows that stayed alive
On a table sorted by time first, delete markers landed under a different key, so deleted rows stayed visible in ClickHouse. migrate check caught it: 1,998,000 rows in Postgres, 2,000,000 in ClickHouse.
What chlift does
migrate plan finds every table at risk (updated tables with long text/JSON columns, and tables whose sort key does not start with the primary key) and gives you the ALTER TABLE … REPLICA IDENTITY FULL to run before anything is copied. migrate check then compares both sides.
What works today, and what doesn't yet
Built and verified on our dev setup
Install on a ClickHouse cluster; migrate plan, start, status and check (with checksums); and the bench.
Being verified now
Cutover (switching readers over to ClickHouse) is written and being verified. A longer replication reliability run is in progress; this page stays marked preview until it passes.
Install today
There are no public binaries yet. Until there are, deemwar installs chlift with you: tell us about your setup below.
Talk to us
Tell us about your Postgres and what you want in ClickHouse. A person replies, usually within a working day. Or email [email protected].