Installation and Migrations

How to add Cyclone to a Go project, connect to Postgres, and apply the schema migrations.

Requirements

• Go 1.26.3 or later. This release's go.mod declares that minimum.
• PostgreSQL 14 or later, the supported floor. CI uses PostgreSQL 14 or later.
• A Postgres instance that is up and that your application can reach.

Nothing in the schema or the queries needs 14 specifically. FOR UPDATE SKIP LOCKED and ON CONFLICT are 9.5 features. JSONB is a 9.4 feature. GENERATED ALWAYS AS IDENTITY is a 10 feature. 14 is the floor because it is the oldest major that still receives upstream security fixes. It is widely available on managed providers.

The schema and the queries also work on Postgres 12. Postgres 12 is out of upstream support. That result is a data point. It is not a supported configuration. No one has tried a version older than 12. No CI job covers Postgres 12. TestServerVersionMeetsSupportedFloor skips with a message. The test does not pass. You decide whether to use Postgres 12.

Install the Package

go get github.com/wafer-labs/cyclone

For the test harness:

go get github.com/wafer-labs/cyclone/cyclonetest

cyclonetest depends on Testcontainers for Go. Docker, or a compatible OCI runtime, must be available on the test host. Set CYCLONETEST_DSN to skip Testcontainers and reuse an existing Postgres instance.

Connect with pgxpool

Cyclone uses pgxpool.Pool as its database handle:

import (
    "context"
    "os"

    "github.com/jackc/pgx/v5/pgxpool"
    "github.com/wafer-labs/cyclone"
)

func main() {
    ctx := context.Background()

    pool, err := pgxpool.New(ctx, os.Getenv("DATABASE_URL"))
    if err != nil {
        log.Fatalf("connect: %v", err)
    }
    defer pool.Close()

    // Pool size must satisfy: MaxConns >= txSlots + 6.
    // For 8 workers, default txSlots=8, so MaxConns >= 14.
    // Configure in the DSN or via pgxpool.Config:
    // DATABASE_URL='postgres://...?pool_max_conns=25'
}

Run cyclone.Migrate

Call Migrate before you create the client. It is idempotent. Call it at every process start.

if err := cyclone.Migrate(ctx, pool); err != nil {
    log.Fatalf("migrate: %v", err)
}

Migrate applies pending migrations and returns. Migrate skips migrations that are already applied. The first call creates all tables, indexes, and views.

It does three things:

1. It reads cyclone_schema_version to find the highest applied version.
2. It applies only migrations with a version higher than that.
3. It returns nil immediately when nothing is pending.

Concurrent calls serialize on a session-level Postgres advisory lock. That is safe when the connection goes directly to Postgres. It is also safe through a session-mode pooler.

What Migrations Create

Object	Type	Purpose
cyclone_schema_version	Table	Tracks applied migration versions
cyclone_runs	Table	One row per run (workflow or standalone job)
cyclone_steps	Table	One row per step or action within a run
cyclone_deps	Table	Dependency edges between steps
cyclone_schedules	Table	Durable cron schedule definitions
cyclone_schedule_fires	Table	Exactly-once fire log per (schedule, slot)
cyclone_signals	Table	Durable mailbox for signals delivered with Client.Signal
cyclone_v_runs	View	Run summary with step roll-up
cyclone_v_steps	View	Step detail with lease_expired flag
cyclone_v_kinds	View	Per-(kind, state) aggregate
cyclone_v_queues	View	Per-queue aggregate

Autovacuum settings on cyclone_steps and cyclone_runs:

• autovacuum_vacuum_scale_factor = 0.02
• autovacuum_vacuum_threshold = 1000
• fillfactor = 90

These settings make Postgres vacuum dead rows promptly under high churn.

When to Migrate

Call Migrate in main before NewClient. That is the usual path. A deployment applies migrations at most once.

Start a one-off migration binary before you deploy the main service when:

• Migrations must finish before old workers stop reading the old schema.
• You want explicit control of migration time in a rolling deploy.
• The deploy pipeline already has a dedicated migrate step.

Use a dedicated migration connection. Send it directly to Postgres, or through session-mode PgBouncer. Do not call Migrate through transaction-mode PgBouncer. Tern's advisory lock is session-scoped. Review every pre-1.0 migration for compatibility and locking before deployment.

Schema and Search Path

Cyclone creates its objects in the public schema. If the instance uses a non-default search_path, include public. Or set search_path in the DSN:

DATABASE_URL='postgres://user:pass@host/db?search_path=myapp,public'

Or configure the pool:

config, _ := pgxpool.ParseConfig(os.Getenv("DATABASE_URL"))
config.ConnConfig.DefaultQueryExecMode = pgx.QueryExecModeCacheDescribe
pool, _ := pgxpool.NewWithConfig(ctx, config)

The cyclonetest harness creates its own schema per test (SET search_path = cyclone_test_<random>). Test migrations stay isolated from the development database.

Permissions Required

Use separate migration and runtime roles where possible. The migration role owns the schema objects and needs:

-- Minimum required:
GRANT CONNECT, CREATE ON DATABASE mydb TO cyclone_migrator;
GRANT USAGE, CREATE ON SCHEMA public TO cyclone_migrator;

-- After migration, grant only runtime data access:
GRANT CONNECT ON DATABASE mydb TO cyclone_runtime;
GRANT USAGE ON SCHEMA public TO cyclone_runtime;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO cyclone_runtime;
GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO cyclone_runtime;

Arrange equivalent default privileges. Or repeat the runtime grants after a migration that creates new objects. Neither role needs PostgreSQL superuser privileges.

Cyclone does not enforce application authorization or tenant isolation. A runtime role can inspect and mutate every Cyclone run visible in its schema. Use separate schemas or databases when you must isolate mutually untrusted tenants. Or use database-enforced policies. Validate payload sizes before you accept untrusted signal input or workflow input.

Migration Safety in Production

Migrations are transactional. DDL can still take strong table locks. Constraint validation can scan existing rows. The migration files do not use IF NOT EXISTS. Tern prevents a second apply of an already-recorded version. Review the SQL. Test it against production-scale data. Set a lock timeout and a statement timeout. Use a low-traffic window when the migration requires one.

Troubleshooting Common Migration Errors

permission denied for schema public

The database user lacks CREATE on the schema. Grant it. Or call the migrations as the database owner.

relation "cyclone_schema_version" already exists

Expect this when a previous version already applied migrations. Migrate reads this table to decide what to skip.

pq: SSL is not enabled on the server

Add sslmode=disable to the DSN for local development:

DATABASE_URL='postgres://postgres@127.0.0.1:5432/mydb?sslmode=disable'

Migrations hang / lock wait timeout

Another process holds a lock on a table that the migration creates. Check pg_stat_activity for blocking queries.

Related

• First Workflow: full setup walkthrough
• Migrations: migration versions and rollout notes
• Schema: full schema reference
• Database Maintenance: autovacuum and maintenance