Sequences (FIFO per Key)

Some standalone jobs must execute one at a time for one entity. Examples are every job for account X, in sequence, and every write to document Y. Jobs for other entities stay concurrent. WithSequence gives that order. Each key has a FIFO queue. The backlog has no limit.

_, err := cyclone.Enqueue(ctx, client, ApplyLedgerEntry, entry,
    cyclone.WithSequence("account-"+accountID))

At most one job for a sequence key is active at a time. The other jobs wait in enqueue order. A waiting job becomes claimable only when the active job reaches a terminal state. Jobs with different keys stay concurrent. A job with no key stays concurrent.

A key must be non-empty. A key must be valid UTF-8 with no NUL. A key must be at most 256 bytes. WithUnique uses the same rules. Cyclone rejects an empty key. Cyclone does not store an empty key. An empty key would put every such job into one global sequence. That sequence would order work that is not related.

How It Differs from WithUnique

• WithUnique rejects a second active job with the same key. The error is a *DuplicateError. Use it when only one active job with that key can exist.
• WithSequence accepts every job. WithSequence orders the jobs. Cyclone rejects nothing. The backlog has no limit. The jobs become claimable in enqueue order. Use it to execute every job, one at a time, in order.

Ordering and the Gate

The claim query holds the gate. The gate applies to every worker.

• A sequenced job is claimable only when no job of its key is in state running. The job must also be the oldest job for that key that is not terminal, by enqueue order.
• Different keys are independent. A job with key a in state running does not block key b.

// These three run in order 1 → 2 → 3, never overlapping, even on a 50-worker fleet.
for _, e := range entries {
    cyclone.Enqueue(ctx, client, ApplyLedgerEntry, e, cyclone.WithSequence("account-42"))
}

Failure and Retry Policy

The head of a sequence keeps its position until it reaches a terminal state.

• While the head retries, the backlog stays blocked. The head failed an attempt. The head is in backoff. No other job for that key becomes claimable until the head reaches a terminal state.
• When the head is discarded, the sequence does not stop. The head is discarded when it uses every attempt. The head is also discarded when the failure is permanent. The next job becomes claimable. The sequence continues. One bad job does not block the key with no end.

When a failed head must stop the whole sequence, do not use only a flat sequence. Gate the later jobs on the success of the head. For example, use a workflow with OnDepFailure(FailRun).

Testing

Sequenced jobs are ordinary runs. The cyclonetest harness drives them with no special setup. Enqueue several jobs for one key. Call env.RunUntilIdle(). Check that the handler saw the jobs one at a time, in order. See Testing with cyclonetest.

See Unique Jobs for the variant that rejects a duplicate. See Queues and Priority for routing across the worker fleet.