Per-Branch Failure Policy

By default, a run is all-or-nothing. When a node uses every attempt and is discarded, the run fails at once. Cyclone cancels every other node that is not terminal. In a wide DAG, that result is often too broad. One of 40 fetches can fail. Cyclone then cancels the other 39.

OnDepFailure lets a node declare the result when one dependency fails:

verdict := cyclone.Do(b, Aggregate, cyclone.FromAll(fetches...),
    cyclone.OnDepFailure(cyclone.CancelBranch))

Two policies:

• FailRun is the default. A discarded dependency fails the whole run. This is the historical behavior. A node with no OnDepFailure keeps FailRun.
• CancelBranch: a discarded dependency cancels this node. It also cancels the branch downstream of this node. An independent branch continues. The run can still complete.

Containment Rule

When a node is discarded, Cyclone walks the branch that depends on it.

• When every dependent node declares CancelBranch, the failure stays in that branch. This set includes a node that depends on the discarded node through other nodes. Cyclone cancels that branch. The run continues. The run completes when no other node is in progress.
• When any dependent in that branch declares FailRun, the discard fails the run. FailRun is the default. The run fails at once, as it did before this option. The strictest declared policy wins.
• A discarded node with no dependents is always fatal. A leaf is such a node. No downstream node can declare tolerance.

var IngestWorkflow = cyclone.Define("ingest", func(b *cyclone.Builder, in Batch) error {
    // 40 independent fetches.
    var results []*cyclone.Future[Doc]
    for _, src := range in.Sources {
        results = append(results, cyclone.Do(b, Fetch, cyclone.Lit(src)))
    }
    // If any fetch dies, cancel only the aggregation branch; the other fetches
    // still complete their side effects and the run completes.
    cyclone.Do(b, Aggregate, cyclone.FromAll(results...),
        cyclone.OnDepFailure(cyclone.CancelBranch))
    return nil
})

A run contained this way ends as completed. ListSteps shows the discarded node. The cancelled branch has state cancelled.

When the cancelled branch holds the result node of a cyclone.DefineResult workflow, the run completes with no result. run.Result returns an error. run.Output returns an error. The error matches cyclone.ErrNotCompleted. The call does not return a zero value.

Part of the def_version Fingerprint

The policy is part of the declared graph. A change to the policy changes the def_version of the workflow. A run that has already started continues with the fingerprint from its submit. A new deploy that sets a node to CancelBranch affects only a new run. See Renaming and Versioning Steps.

See Retries and Permanent Failures for how a node becomes discarded.