Cyclone

Appearance

Unique Jobs ​

Use WithUnique when callers can request the same work more than once. Only one active run can exist at a time. Typical cases are an account sync, a webhook retry, an invoice send, and a daily report.

Enqueue with a Business Key ​

go
runID, err := cyclone.Enqueue(ctx, client, SyncAccount,
    SyncAccountInput{AccountID: 42},
    cyclone.WithUnique("account:42"),
)

Cyclone enforces uniqueness on an active job by (kind, key). The second enqueue fails with DuplicateError when another sync_account job with key account:42 is pending, available, running, or paused.

Completed, discarded, and cancelled runs do not hold the slot.

Handle Duplicates ​

A duplicate is an expected result. Treat it as work already in progress. Do not treat it as a crash.

go
runID, err := cyclone.Enqueue(ctx, client, SyncAccount,
    SyncAccountInput{AccountID: 42},
    cyclone.WithUnique("account:42"),
)
if err != nil {
    var dup *cyclone.DuplicateError
    if errors.As(err, &dup) {
        log.Printf("account sync already running as %s", dup.RunID)
        return dup.RunID, nil
    }
    return "", err
}

return runID, nil

dup.RunID is the existing active run. Show it on an admin page, wait for it, or return it to the caller.

Pick Useful Keys ​

Describe the work in the key. Do not put the HTTP request id in the key.

go
cyclone.WithUnique("account:" + strconv.FormatInt(accountID, 10))
cyclone.WithUnique("invoice:" + invoiceID)
cyclone.WithUnique("daily-report:" + date.Format("2006-01-02"))
cyclone.WithUnique("webhook:" + eventID)

Keys that are too narrow let duplicates through:

go
// Bad: each click gets a different key.
cyclone.WithUnique("account:42:request:" + requestID)

Keys that are too broad block unrelated work:

go
// Bad: one account sync blocks every account sync.
cyclone.WithUnique("account-sync")

Keep a key short enough to read in a log. 256 bytes is the hard limit. Do not treat 256 bytes as a target length.

Key Rules ​

A key must be non-empty. A key must be valid UTF-8 with no NUL. A key must be at most 256 bytes. When a key breaks a rule, the enqueue returns an error. The enqueue inserts nothing.

The empty-key rule is the important one. A partial index on (kind, unique_key) enforces uniqueness. Every empty key is the same key. An empty key would be one global lock for that kind.

go
// Bad: order.ID is the zero value here, so every second job of this kind
// is rejected as a duplicate of the first. Cyclone now returns an error instead.
cyclone.WithUnique(order.ID)

Retries Keep the Slot ​

A retrying job is still active. It keeps the unique slot until it completes, is cancelled, or is discarded.

go
_, err := cyclone.Enqueue(ctx, client, DeliverWebhook, in,
    cyclone.WithUnique("webhook:evt_123"),
)

// If the first run is backing off after a transient error, this still returns DuplicateError.
_, err = cyclone.Enqueue(ctx, client, DeliverWebhook, in,
    cyclone.WithUnique("webhook:evt_123"),
)

This is the usual result. One broken webhook delivery can retry. Ten copies of the same delivery do not queue behind it.

Delayed Jobs Hold the Slot ​

WithProcessIn holds the unique slot when the row is inserted. WithProcessAt does the same. A worker cannot claim the job yet. The slot is already held.

go
_, err := cyclone.Enqueue(ctx, client, SendReminder, in,
    cyclone.WithUnique("invoice:inv_123:reminder"),
    cyclone.WithProcessIn(24*time.Hour),
)

A second enqueue with the same (kind, key) fails during that 24-hour wait.

Testing Duplicates ​

Use a slow handler, or use WithAutoStart(false). The first job then stays active during the second enqueue.

go
env := cyclonetest.New(t, cyclonetest.WithAutoStart(false))
env.Mux().Register(SyncAccount)

runID, err := cyclone.EnqueueAction(ctx, env.Client(), SyncAccount,
    SyncAccountInput{AccountID: 42},
    cyclone.WithUnique("account:42"),
)
require.NoError(t, err)

_, err = cyclone.EnqueueAction(ctx, env.Client(), SyncAccount,
    SyncAccountInput{AccountID: 42},
    cyclone.WithUnique("account:42"),
)

var dup *cyclone.DuplicateError
require.True(t, errors.As(err, &dup))
require.Equal(t, runID, dup.RunID)

Next ​

Cyclone stores each run in Postgres.