Exactly-Once Effects

How to write side effects that happen exactly once, even when a handler executes more than once.

What This Is For

Cyclone executes a handler at least once. This guide explains how to make the observable consequences of a handler happen exactly once. A consequence can be a database row, a ledger entry, or a webhook delivery. Retries, crashes, and lease reclaims do not change that count.

Prerequisites

You need ctx.CompletionTx() and the at-least-once execution contract. See Idempotency.

Cyclone Is At-Least-Once

A handler can execute more than once:

• The worker crashes after the side effect and before the completion transaction commits.
• A lease expires and the recovery sweep reclaims the step. The attempt stays the same. The handler executes again.
• A per-attempt timeout fires and a new attempt starts.

This is the durability contract. It is not a bug. Your handlers must be ready for it.

What "Exactly-Once Effects" Means

Exactly-once effects does not mean that the handler executes exactly once. It means the observable consequence happens exactly once, even when the handler executes many times. A consequence can be a charge on a ledger, a row in an events table, or a sent message.

The mechanism depends on where the effect lands:

• For a Postgres write, use ctx.CompletionTx() for an atomic commit. Or use ON CONFLICT DO NOTHING for deduplication.
• For an external system, use (ctx.RunID(), ctx.Node()) as an idempotency key.

Use ctx.CompletionTx() for Postgres Writes

ctx.CompletionTx() is the completion transaction. A write through it commits atomically with the step's state change to completed. Either the write and the completion both commit, or both roll back.

var ChargeAction = cyclone.NewAction("charge", func(ctx cyclone.Context, in ChargeInput) error {
    _, err := ctx.CompletionTx().Exec(ctx, `
        INSERT INTO ledger (run_id, node, amount, charged_at)
        VALUES ($1, $2, $3, now())
        ON CONFLICT (run_id, node) DO NOTHING`,
        ctx.RunID(), ctx.Node(), in.Amount)
    return err
})

ON CONFLICT DO NOTHING covers one case. A previous execution committed the insert. A reclaim took the step after that commit. That case is rare with ctx.CompletionTx(). The clause is still good practice. The (run_id, node) pair is your deduplication key.

Write Business Changes and Completion Atomically

The key property is one rollback. If the worker crashes before the commit, the business write rolls back. The step completion rolls back with it. The recovery sweep reclaims the step. The handler starts again from the top. The new execution uses a fresh transaction.

var FinalizeInvoice = cyclone.NewStep("finalize_invoice",
    func(ctx cyclone.Context, in InvoiceInput) (InvoiceOutput, error) {
        // These writes commit atomically with step completion.
        // If the process crashes here, both are rolled back.
        var invoiceID int
        err := ctx.CompletionTx().QueryRow(ctx, `
            INSERT INTO invoices (order_id, total, finalized_at)
            VALUES ($1, $2, now())
            ON CONFLICT (order_id) DO UPDATE SET finalized_at = now()
            RETURNING id`,
            in.OrderID, in.Total,
        ).Scan(&invoiceID)
        if err != nil {
            return InvoiceOutput{}, err
        }
        return InvoiceOutput{InvoiceID: invoiceID}, nil
    },
)

Use (RunID, Node) as the Idempotency Key

For deduplication, always key on (ctx.RunID(), ctx.Node()). Never include ctx.Attempt().

The attempt number changes across retries. A key that includes it makes every retry look like a new operation. That defeats deduplication. (RunID, Node) stays stable for the whole life of the node.

var WriteEvent = cyclone.NewAction("write_event", func(ctx cyclone.Context, in EventInput) error {
    key := ctx.RunID() + ":" + ctx.Node() // stable key
    _, err := ctx.CompletionTx().Exec(ctx, `
        INSERT INTO outbox (idempotency_key, type, payload)
        VALUES ($1, $2, $3)
        ON CONFLICT (idempotency_key) DO NOTHING`,
        key, in.EventType, in.Payload)
    return err
})

Example: Ledger Charge

This payment handler charges exactly once, even when the handler executes many times:

var ProcessPayment = cyclone.NewAction("process_payment",
    func(ctx cyclone.Context, in PaymentInput) error {
        // Use ctx.CompletionTx() so the ledger INSERT and step completion
        // are in the same transaction.
        _, err := ctx.CompletionTx().Exec(ctx, `
            INSERT INTO payments (run_id, node, user_id, amount_cents, created_at)
            VALUES ($1, $2, $3, $4, now())
            ON CONFLICT (run_id, node) DO NOTHING`,
            ctx.RunID(), ctx.Node(), in.UserID, in.AmountCents)
        return err
    },
)

If the worker crashes after this INSERT and before the transaction commits, the INSERT rolls back. On the next execution, the INSERT succeeds and commits with the step completion. The payment row exists exactly once.

Example: Webhook Outbox

Write a webhook delivery record. Let a separate relay process pick it up. The record is written exactly once:

var QueueWebhook = cyclone.NewAction("queue_webhook",
    func(ctx cyclone.Context, in WebhookInput) error {
        _, err := ctx.CompletionTx().Exec(ctx, `
            INSERT INTO webhook_outbox (run_id, node, url, payload, created_at)
            VALUES ($1, $2, $3, $4, now())
            ON CONFLICT (run_id, node) DO NOTHING`,
            ctx.RunID(), ctx.Node(), in.URL, in.Payload)
        return err
    },
)

The relay can see the webhook outbox row only after the step completes. The transaction keeps delivery order.

Example: Follow-Up Work

When the follow-up is itself Cyclone work, enqueue it through the completion transaction. Do not use an outbox table for that case. EnqueueActionCompletion inserts the run in the same transaction as the step completion. EnqueueCompletion does the same for a step. The job exists exactly once, and only if the step completes:

var ChargeOrder = cyclone.NewAction("charge_order", func(ctx cyclone.Context, in ChargeInput) error {
    if _, err := ctx.CompletionTx().Exec(ctx, `
        INSERT INTO payments (run_id, node, order_id, amount_cents)
        VALUES ($1, $2, $3, $4)
        ON CONFLICT (run_id, node) DO NOTHING`,
        ctx.RunID(), ctx.Node(), in.OrderID, in.AmountCents); err != nil {
        return err
    }
    _, err := cyclone.EnqueueActionCompletion(ctx, client, SendReceipt, ReceiptInput{OrderID: in.OrderID})
    return err
})

A crash rolls the receipt job back with the payment row. A failed completion does the same. A completion that lost its lease does the same. See Transactional Enqueueing.

What Not to Put in ctx.CompletionTx()

ctx.CompletionTx() is a Postgres transaction. Use it only for Postgres writes. Do not:

• Make an HTTP call through ctx.CompletionTx(). It is not an HTTP context.
• Start another Postgres transaction. Tx does not support BEGIN. The methods omit BEGIN on purpose.
• Use it for an operation that holds a database connection for a long time. Keep the write quick.

For an external API call, an email, or another effect outside Postgres, write a record with ctx.CompletionTx(). The record can be an outbox row or an event-log row. Do the delivery separately.

External Side Effects Still Need Idempotency

If you must call an external API directly from the handler, there is no outbox. Use (RunID, Node) as the idempotency token:

var SendSMS = cyclone.NewAction("send_sms", func(ctx cyclone.Context, in SMSInput) error {
    idempKey := fmt.Sprintf("%s:%s", ctx.RunID(), ctx.Node())
    return smsProvider.Send(ctx, sms.Message{
        To:             in.Phone,
        Body:           in.Body,
        IdempotencyKey: idempKey,
    })
})

If the SMS provider supports idempotency keys, a second call uses the same key. The provider returns the result that it already delivered. It does not send the message again.

Crash Scenarios

Timing	ctx.CompletionTx() write state	Step state	What happens
Crash before ctx.CompletionTx() write	Not attempted	running → reclaimed	The handler starts again. The write starts fresh.
Crash after write, before commit	Write rolled back	running → reclaimed	The handler starts again. The handler attempts the write again. ON CONFLICT does not trigger.
Crash after commit	Write committed	completed	There is no second execution. The step is done.
Crash on every run	Write rolled back each time	running → reclaimed, repeatedly	The handler starts again until WithMaxReclaims discards the step (reclaim_limit). Reclaims have no bound by default.
Timeout fires	Depends on where in handler	running → new attempt	A new attempt starts. ON CONFLICT deduplicates any previous write.
Output exceeds the payload limit	Write rolled back	running → discarded	Cyclone can never persist the output. Cyclone cannot mark the step complete. The business writes must not commit without that completion. The failure is permanent. A retry would produce the same oversized value. See Payload Limits

Related

• Idempotency: the at-least-once contract
• Transactional Enqueueing: transactional enqueue
• Transactional Workflow: full walkthrough