In the pantheon of distributed computing, privacy stands not as an afterthought but as the architectural cornerstone upon which all trust is built. The Bazaar platform implements an unprecedented privacy infrastructure—eight core services orchestrating four fundamental technologies to create what security researchers have called "the most comprehensive privacy-preserving compute platform in production today."

The numbers speak with authority: sub-millisecond differential privacy operations, 100-millisecond zero-knowledge proof generation, Byzantine fault tolerance up to 33% malicious nodes, and triple-encrypted Tor circuits established in under 300 milliseconds. These are not theoretical benchmarks but operational realities, battle-tested across thousands of compute hours and millions of privacy-preserving operations.

Every request entering the Bazaar platform undergoes a sophisticated privacy transformation, passing through multiple validation and protection layers before reaching its destination. This lifecycle, measured in milliseconds yet comprehensive in its security guarantees, represents the practical implementation of theoretical privacy primitives at scale.

The gradient privacy processing flow represents the confluence of multiple privacy-preserving technologies working in concert. Each gradient update undergoes clipping to bound its L2 norm, receives carefully calibrated Gaussian noise to ensure differential privacy, and is committed using Pedersen commitments before zero-knowledge proof generation.

The implementation of Groth16 SNARKs on the BN254 curve represents a masterclass in applied cryptography. With proof sizes of merely 200 bytes and generation times averaging 100 milliseconds, the system achieves the holy grail of zero-knowledge systems: practical efficiency without compromising security.

HoneyBadgerBFT, the Byzantine fault-tolerant consensus protocol at the heart of the bulletin board system, achieves what was once thought impossible: asynchronous Byzantine consensus with optimal communication complexity. With tolerance for up to 33% malicious nodes, the system maintains consistency and availability even under adversarial conditions.

The protocol operates in three distinct phases: reliable broadcast ensures all honest nodes receive the same messages, binary agreement reaches consensus on message inclusion, and the commit phase constructs Merkle trees for cryptographic verification. This elegant dance of distributed agreement occurs in approximately one second, a remarkable achievement for Byzantine consensus at scale.

Through the mathematical elegance of Shamir's secret sharing, the platform enables multiple parties to jointly compute functions over their private inputs without revealing those inputs to each other. This is not theoretical cryptography but practical privacy, enabling federated learning across untrusted nodes while maintaining complete confidentiality of individual contributions.

The privacy architecture's true power emerges from the seamless integration of its components through the Kong API Gateway. Three critical plugins—privacy-budget, anofel-provenance, and msi-degraded—form the first line of defense, validating every request against privacy policies before it enters the system.

The privacy budget ledger implements a sophisticated accounting system for differential privacy resources. Each operation consumes a portion of the privacy budget (ε, δ), tracked with microsecond precision and enforced through distributed consensus.

The Security Vault maintains eight critical tables for managing cryptographic materials and audit trails. Built on BadgerDB for performance with PostgreSQL for compliance tracking, it supports RSA, ECDSA, ED25519, AES, and ChaCha20Poly1305 operations with hardware security module integration.

The distributed GPU network, while revolutionary in its ambitions, rests upon a foundation of carefully orchestrated components. This technical appendix documents the actual implementation—a system battle-tested across thousands of reservations, millions of compute minutes, and hundreds of provider nodes.

At its core, the architecture separates concerns across three primary layers: the control plane for orchestration and economic coordination, the provider network for compute execution, and the settlement layer anchored in Arbitrum smart contracts for trustless financial settlement.

The PriceIndexOracleService implements uniform-price bucket auctions with surge multipliers and per-entity capacity caps. Demand configuration accepts micro-SCM requirements plus reserve buffers; supply submission accumulates offers sorted by price. Bucket finalization executes a modified uniform-price clearing: offers are sorted ascending, demand is filled sequentially, and the marginal price becomes the clearing price for all accepted units.

Surge multipliers apply when utilization exceeds 95%, scaling linearly from 1.0× at 95% to 1.5× at 100%. Entity caps prevent single providers from capturing more than 30% of any bucket's supply—a critical anti-manipulation guard. The resulting BucketResult contains clearing price, utilization basis points, surge multiplier, and filled micro-SCM, all persisted locally before being mirrored to the on-chain PriceIndexOracle contract.

The VRACUScheduler implements Dominant Resource Fairness with attained-service scoring. Each provider maintains a running total of attained service minutes; new allocations favor providers with lower historical utilization. The fairness score combines expected job duration with attained service ratio, preventing long-running providers from monopolizing capacity while ensuring hardware constraints (VRAM, interconnect class) are satisfied.

NVIDIA MIG (Multi-Instance GPU) support extends the scheduler's capacity model. Provider registration records MIG profile, partition count, and per-partition memory. The scheduler normalizes SCM rates on a per-partition basis, treating each MIG slice as an independent scheduling unit. This enables fine-grained multi-tenancy without sacrificing fairness or resource isolation.

The MeteringService provides strict idempotency guarantees via SHA-256 content hashing. Each meter slice contains job ID, bucket ID, sequence number, SCM delta, and the price index at time of execution. Duplicate submissions (identical hash) succeed silently; conflicting payloads (same sequence, different hash) return rejection with reason code.

Settlement aggregation computes SCM-weighted time-averaged pricing (TWAP) across all slices for a job. The formula: Σ(minutes × price) / Σ(minutes). Burn amounts apply ceiling rounding to micro-ACU units, ensuring providers never receive fractional tokens. Hold fractions (0.0–1.0) split burn amounts between immediate provider payout and refund escrow, enabling dispute resolution without blocking settlement.

The DualSignatureService produces cryptographic attestations for every settlement receipt. Primary signatures use Ed25519 with embedded public keys (32-byte seed expanded via SHA-512). Secondary signatures employ a post-quantum envelope: 64-byte secrets processed through SHAKE-256 XOF, yielding Dilithium3-compatible signing material.

Both signatures cover the JCS-canonicalized receipt JSON (sorted keys, minimal whitespace). The control plane persists signatures alongside receipts in SQLite, enabling offline verification without blockchain round-trips. The GET /settlement/receipt/{job_id} endpoint returns the canonical receipt, both signature envelopes, and current Mirror-Mint escrow state.

Modularity, not monoliths. Unlike traditional job schedulers that hardcode workload types into platform logic, the distributed compute network employs adapters— protocol-based transformers registered at runtime via a plugin architecture. Each adapter converts high-level job specifications into resource profiles and execution plans, enabling the same user code to run across Docker, Ray clusters, or Kubernetes without modification. Third-party developers extend the platform by registering custom adapters without touching core infrastructure code.

Registry-Based Architecture

The AdapterFactory pattern decouples adapter implementations from scheduler logic. A global registry maps adapter names to factory functions, allowing hot-swapping of adapters without redeploying the launcher service. The registry initializes with five core adapters (training, inference, quantization, rendering, federated), but any module can call register_adapter(name, factory) to inject custom transformation logic.

The adapter protocol defines two primary operations: prepare(job_spec) transforms declarative requirements into a (ResourceProfile, ExecutionPlan) tuple, while map_metrics(raw) normalizes heterogeneous telemetry into standardized metering signals. This abstraction allows the control plane to allocate providers based on resource constraints without understanding framework-specific details.

Training Adapter

The TrainingAdapter prepares distributed training jobs with multi-GPU coordination strategies. It accepts job specs containing image references, command arrays, VRAM requirements, and interconnect preferences. The adapter injects environment variables for DDP (DistributedDataParallel) or FSDP (Fully Sharded Data Parallel) rendezvous, configures volume mounts for dataset access, and sets priority metadata for queue ordering.

Inference Adapter

The InferenceAdapter targets long-running model serving deployments. Unlike batch training jobs, inference workloads require service-oriented execution: health probes (readiness/liveness), autoscaling configurations, load balancer exposure, and rolling update strategies. The adapter generates Kubernetes-compatible execution plans with service ports, replica counts, and horizontal pod autoscaling parameters.

Health probes default to HTTP GET requests against /health endpoints, with configurable initial delays and check intervals. Service types (ClusterIP, LoadBalancer, NodePort) control network exposure. Autoscaling policies define CPU/memory thresholds triggering replica scale-up, enabling elastic capacity matching demand spikes.

Federated Learning Adapter

The FederatedAdapter prepares multi-party training with privacy-preserving aggregation. Job specs include: world_size (participant count), committee parameters (K-of-M threshold for Shamir secret sharing), differential privacy budgets (epsilon/delta), and bulletin board backend configuration (Redis, S3, or IPFS).

The adapter injects environment variables controlling aggregation behavior: FED_PACKET_BYTES (fixed-size message padding, default 128KB), FED_ROUNDS (training iterations), FED_DP_EPSILON (privacy budget), FED_BB_BACKEND (bulletin board type), and FED_ROUND_SECRET (shared secret hex for key derivation). These parameters enable secure gradient aggregation without trusted coordinators.

Quantization and rendering adapters follow similar patterns, specializing resource requirements and metric extraction for their respective workloads. The quantization adapter handles model compression tasks (GPTQ, AWQ, bitsandbytes), while the rendering adapter manages visual workloads with GPU rasterization demands.

The network's privacy guarantees emerge from three complementary layers, each addressing a distinct threat model. Hardware attestation prevents operators from inspecting workload data. Encrypted input/output pipelines protect data in transit and at rest. Cryptographic secure aggregation prevents peers from observing individual contributions during federated training. Together, these mechanisms enable confidential computation on untrusted infrastructure.

Hardware Attestation

Provider nodes collect attestation evidence proving workloads execute inside hardware-protected enclaves. Three attestation technologies integrate via composite providers: NVIDIA Confidential Computing (CC-On), Intel TDX (Trust Domain Extensions), and AMD SEV-SNP (Secure Encrypted Virtualization with Secure Nested Paging).

Intel TDX provides VM-level isolation with encrypted memory and integrity protection. Attestation quotes prove execution inside a Trust Domain, with measurements covering firmware, kernel, and initial ramdisk. Control plane verification checks quote signatures against Intel's root keys, ensuring authenticity.

AMD SEV-SNP extends SEV with stronger memory integrity guarantees. SNP reports include platform measurements and VM guest policy, preventing malicious hypervisor tampering. Combined with encrypted memory, SNP isolates guest execution from host observation.

The CompositeAttestationProvider aggregates evidence from multiple sources, enabling hybrid deployments (e.g., NVIDIA GPU inside Intel TDX VM). Async evidence collection occurs during provider attach, with challenge-response protocols ensuring freshness. Stale evidence (> 24 hours) triggers re-attestation before accepting privacy-tier workloads.

Encrypted Input/Output Pipeline

Job artifacts (datasets, model checkpoints, configuration files) never touch provider disks in plaintext. The launcher service generates per-job Data Encryption Keys (DEKs), encrypting all artifacts with AES-GCM authenticated encryption. DEKs transfer to provider sidecars via TLS mutual authentication, with client certificate fingerprint validation preventing unauthorized access.

The sidecar's decrypt shim fetches DEKs at job start, decrypts artifacts inside the secure enclave, and launches the workload. Operators observe only ciphertext blobs—plaintext exists exclusively within hardware-protected memory. Output encryption reverses the flow: results encrypt before leaving the enclave, with DEKs accessible only to job initiators.

Federated Learning with Secure Aggregation

The network implements committee-based secure aggregation combining Shamir secret sharing, differential privacy, and bulletin board coordination. Unlike naive averaging (where aggregators observe raw gradients), this protocol ensures no party—including the coordinator—sees individual contributions.

Each training round proceeds as follows: participants add Gaussian noise to gradients (satisfying (ε,δ)-differential privacy), encode noisy gradients as field elements over GF(4,294,967,291), split into M shares via Shamir's scheme (K-of-M threshold), encrypt shares with per-pair AES keys, post ciphertexts to bulletin board, collect K shares addressed to them, decrypt and reconstruct via Lagrange interpolation, average reconstructed gradients, and broadcast the aggregate.

Committee selection employs deterministic random sampling seeded by round ID, ensuring all participants agree on the committee without coordination. Key derivation uses HKDF-SHA256 with context strings encoding round, sender, and recipient identities: fed-round:{round}:from:{sender}:to:{recipient}. This generates unique AES keys for each communication pair per round.

The bulletin board abstraction supports three backends: Redis (RPUSH/LRANGE for ordered streams), S3 (timestamped objects with lexicographic ordering), and IPFS (content-addressed immutable logs). Fixed-size message padding (default 128KB) prevents size-based traffic analysis.

This multi-layered approach achieves end-to-end confidentiality: hardware attestation proves code integrity, encrypted I/O protects data in motion, and secure aggregation prevents gradient leakage. The combination enables privacy-preserving machine learning on commodity hardware without trusted third parties—a capability previously requiring specialized secure enclaves or multiparty computation protocols.

Two tokens, distinct roles, unified economy. The network employs a dual-token architecture where ACU (Actual Compute Units) serves as the fixed-supply settlement currency, while AVL (Availability Token) functions as the inflationary utility token rewarding provider liveness. This separation creates economic pressure: ACU scarcity drives value appreciation, AVL emissions incentivize capacity contribution, and the ConversionRouter bridges them via oracle-priced burns—transforming availability into settlement rights.

ACU: The Settlement Token

ACU implements a fixed-supply ERC20 (18 decimals) with no mint function post-deployment. The total supply (S_MAX) initializes at construction and remains immutable—every ACU that will ever exist mints to the treasury address during contract deployment. This design choice transforms ACU into a deflationary settlement currency: as compute demand grows, fixed supply creates scarcity pressure.

Users deposit ACU into the MirrorMintPool escrow contract for job execution. Each job receives an isolated escrow account tracking: deposited_micro_acu, burned_micro_acu, released_micro_acu (provider payments), refunded_micro_acu, and held_micro_acu (dispute buffer). Settlement burns protocol fees to the treasury while routing provider payouts and refunding unused balances—all operations occur in micro-ACU (1e-6 ACU) precision to minimize rounding losses.

AVL: The Availability Token

AVL implements an ERC20 with role-gated minting and burning. The contract enforces a MAX_SUPPLY cap but allows addresses holding the MINTER_ROLE to create new tokens below this ceiling. Daily emissions distribute AVL to providers via Merkle airdrops proportional to their availability scores— the longer a provider maintains liveness (passing heartbeat checks), the more AVL they earn.

Providers stake AVL in the AvailabilityStaking contract to signal commitment. Staked amounts act as economic bonds: misbehavior (failed jobs, missed heartbeats) triggers slashing via the SLASHER_ROLE, burning a percentage of the stake. Unstaking requires a cooldown period preventing providers from exiting immediately before slashing events. The staking mechanism creates skin-in-the-game: providers risk capital to participate, and penalties enforce service quality.

ConversionRouter: The Bridge

The ConversionRouter contract implements one-way conversion: burn AVL → mint ACU. The oracle-determined exchange rate reflects market-discovered pricing: as compute demand increases relative to provider supply, the ACU price (denominated in AVL) rises. The router enforces ACU_MAX_SUPPLY—cumulative mints cannot exceed this ceiling—preventing infinite inflation even as AVL emissions continue.

The conversion mechanism creates economic alignment: providers earn AVL through availability (passive income), accumulate stakes, then convert to ACU when settlement demand materializes (active income). Users purchasing ACU on secondary markets indirectly reward past provider contributions. The dual-loop structure— AVL emissions incentivize long-term capacity, ACU scarcity rewards immediate execution—balances supply-side growth with demand-side sustainability.

The settlement layer anchors economic finality in Arbitrum Nitro—a Layer 2 optimized for EVM execution with sub-second confirmation times and negligible gas costs. Seven Solidity contracts form the on-chain substrate: ACUToken, MirrorMintPool, PriceIndexOracle, BurnGovernor, ProtocolFeePool, AvailabilityToken, and ConversionRouter.

ACUToken implements a fixed-supply ERC-20 representing Standard Compute Minutes. Total supply is immutable post-deployment; no mint/burn functions exist, preserving the supply invariant. The treasury holds initial allocation; governance controls rescue functions for accidentally transferred tokens.

MirrorMintPool manages job escrow with burn/release/hold state machines. The depositForJob function accepts micro-ACU deposits, recording deposited amounts per job ID. Settlement authorities (authorized by governance) invoke settleJob with burn amounts, provider addresses, and receipt hashes. Burn amounts route to treasury; provider payments execute immediately; hold fractions freeze pending governance review.

PriceIndexOracle records bucket configurations and clearing results. Demand oracles (governance-authorized) call configureBucketDemand with micro-SCM requirements and commit deadlines. Supply submitters post offers before finalization. The control plane executes clearing off-chain, then publishes results via finalizeBucket, emitting clearing price and surge multiplier events.

BurnGovernor mediates between settlement receipts and Mirror-Mint escrow. The settleJob function accepts job IDs and receipt payloads, extracting burn/provider amounts and forwarding to MirrorMintPool. Governance can pause settlements system-wide via emergencyPause, halting all burns without touching escrow state.

ProtocolFeePool accumulates burned ACU and distributes protocol revenues. Governance proposals withdraw to specified addresses; spending requires timelock execution (48-hour delay). The pool maintains immutable audit trails via Withdrawal and FeeAccrued events.

ConversionRouter implements trustless ACU↔AVL swaps using the price oracle as reference. Conversion applies basis-point slippage caps; governance adjusts spreads based on liquidity depth. The router maintains no internal state—all pricing derives from on-chain oracle snapshots.

The provider network transforms heterogeneous GPU hardware into fungible compute units through a three-layer abstraction: backend runners (Docker, Ray, Kubernetes), capability publishing, and heartbeat coordination.

Provider nodes begin life via alien attach—a CLI tool consuming join tokens from the directory API. The attach flow redeems tokens for provider IDs, runs microbenchmarks to calibrate SCM rates, collects attestation evidence, and publishes capabilities to the control plane.

Backend runners isolate workloads via container runtimes. The Docker backend executes jobs as privileged containers with GPU passthrough, SSH-tunneling logs and metrics to the control plane. The Ray backend bootstraps Ray clusters, submitting jobs via the Ray Jobs API with custom resource specifications (GPU count, VRAM, placement strategies). The Kubernetes backend provisions k3s clusters with NVIDIA device plugins, deploying jobs as pods with GPU requests/limits.

The heartbeat agent maintains provider liveness via periodic POST /heartbeat calls. Each heartbeat includes: provider ID, current load (running jobs, available VRAM), calibration drift (actual vs. advertised SCM rate), and attestation refresh timestamps. The directory API marks providers unavailable after three missed heartbeats (90-second timeout).

Privacy-preserving execution leverages attestation and encrypted inputs. Providers with SGX or TPM capabilities generate remote attestation quotes during attach; the control plane verifies quotes against manufacturer root keys before approving privacy-tier workloads. Encrypted job inputs decrypt inside secure enclaves, ensuring operators never observe plaintext data or intermediate activations.

The relay mechanism enables NAT traversal for home providers. Providers behind firewalls establish persistent WebSocket connections to relay servers; the control plane routes job submissions via relay endpoints. Bi-directional tunneling supports both job dispatch and real-time log streaming without requiring public IPs or port forwarding.

Production infrastructure demands resilience mechanisms beyond optimistic execution paths. The system implements circuit breakers, capacity guards, regional isolation, and chaos injection to maintain SLAs under adversarial conditions.

The primary price breaker monitors clearing prices against configured mint prices. When a bucket clears below mint_price - epsilon, the breaker opens, halting new demand configuration until price recovery. This prevents cascading under-pricing that could destabilize provider economics.

The capacity breaker tracks filled SCM across recent buckets (configurable window, default 10 buckets). If filled capacity falls below (demand + reserve) × (1 - buffer_pct / 100), the breaker trips, signaling insufficient supply to meet committed demand. The scheduler rejects new reservations until capacity recovers.

Regional isolation supports active-passive multi-region deployments. Control plane instances operate in three modes: NORMAL (full read-write), ISOLATED (local writes queued, remote reads blocked), and MERGE_REPLAY (reconciling queued writes post-outage).

During regional failures, operators invoke POST /region/isolate, transitioning the affected region to isolated mode. Local writes persist to SQLite; remote API calls receive 503 Service Unavailable. Recovery initiates via POST /region/merge, which replays queued writes against primary state, resolving conflicts via last-write-wins timestamp comparison.

The resilience controller monitors job health and triggers automatic reallocation. Jobs exceeding SLA thresholds (95th percentile latency, failure rate > 5%) receive priority reallocation to higher-tier providers. The controller maintains a reallocate queue; operators approve/reject proposals via POST /resilience/approve/{job_id}.

Observability surfaces breaker state, queue depths, and regional health via Prometheus metrics and Grafana dashboards. Critical alerts fire on: breaker open > 5 minutes, queue depth > 1000 tasks, missed heartbeats > 10% of fleet, settlement failures > 1% of volume.

While on-chain settlement handles provider payouts and protocol fees, enterprise users require fiat on-ramps. The payment stack bridges traditional finance via Stripe webhooks, double-entry ledger accounting, and ACU wallet provisioning.

The Stripe service listens for checkout.session.completed webhooks, validating HMAC signatures before processing. Successful checkouts trigger: credit issuance to user wallets, ledger debit/credit pairs (fiat → ACU), and escrow deposits for immediate job execution.

The ledger service implements immutable double-entry accounting. Every transaction creates two ledger entries: one debit, one credit, summing to zero. Account types include: fiat:stripe (external fiat inflows), acu_wallet:* (user balances), escrow:* (job deposits), protocol:treasury (burned fees), and provider:* (payout accounts).

Monthly reconciliation queries aggregate ledger entries, verifying: Σ(debits) = Σ(credits), user wallet balances match sum of deposits minus escrow, and protocol treasury equals cumulative burns. Discrepancies trigger alerts and halt settlements pending manual review.

Escrow orchestration coordinates off-chain credits with on-chain deposits. When users initiate jobs, the payment processor: debits ACU wallets, credits escrow accounts, invokes MirrorMintPool.depositForJob, and persists transaction hashes for audit trails.

Provider payouts reverse the flow: settlement receipts trigger wallet credits, escrow debits, and optional fiat conversions. Providers configure payout rails (on-chain ACU, Stripe transfers, wire) via PATCH /providers/{id}/payout. The payout service batches settlements daily, minimizing gas costs via Merkle batching.

The Phase 4 SDK encapsulates reservation loops, metering, settlement, and receipt verification behind Pythonic interfaces. Machine learning engineers integrate distributed compute with minimal infrastructure knowledge.

The ControlPlaneClient provides authenticated HTTP transport. Retry logic handles transient failures (exponential backoff, jittered delays); rate limit detection (HTTP 429) triggers automatic back-pressure. SSL context support enables custom certificate validation for private deployments.

The ReservationLoop orchestrates multi-phase workflows. Phase 1 submits supply offers to the oracle. Phase 2 waits for bucket finalization. Phase 3 invokes the scheduler, receiving provider allocation. Phase 4 executes workloads, streaming meter slices to the control plane. Phase 5 polls settlement status, retrieving signed receipts upon job completion.

Integration examples demonstrate Ray, Modal, and Kubernetes adapters. The Ray integration submits jobs via ray.job_submission.submit, tailing logs for metering signals. The Modal integration wraps Modal functions with VR-ACU reservation context, transparently routing compute through the provider network. The Kubernetes integration generates pod specs with GPU resource requests, applying VR-ACU annotations for cost attribution.

The CLI tool exposes SDK functionality via terminal commands. vracu reserve initiates reservations, vracu meter ingests manual slices, vracu settle forces settlement, and vracu receipt verifies signatures. Shell completion scripts support bash, zsh, and fish.

Production observability leverages Prometheus metrics, structured logging, and distributed tracing to surface system health, performance bottlenecks, and failure modes.

Prometheus metrics export from GET /metrics endpoints across all services. Control plane metrics include: queue depth (async tasks pending), HTTP latency histograms (p50/p95/p99), breaker state (binary open/closed), oracle finalization durations, and settlement batch sizes.

Provider metrics expose: GPU utilization percentages, VRAM allocated/free, job counts (running/queued/failed), heartbeat intervals, and attestation refresh timestamps. Grafana dashboards aggregate fleet-wide statistics, alerting on: utilization < 60% (underutilized), failures > 5% (reliability degradation), heartbeat gaps > 90s (connectivity issues).

Structured logging emits JSON lines to stdout, ingested by log aggregators (Loki, Elasticsearch). Log entries include: trace IDs (for correlation), log levels (DEBUG/INFO/WARN/ERROR), component names, and contextual metadata (job IDs, provider IDs, transaction hashes).

Distributed tracing via OpenTelemetry instruments HTTP handlers, database queries, and blockchain transactions. Trace spans propagate across service boundaries via W3C Trace Context headers. Jaeger UI visualizes request flows, surfacing latency waterfalls and failure attribution.

Alerting rules fire on SLO violations, breaker openings, and anomaly detection. PagerDuty integration routes critical alerts to on-call engineers; Slack webhooks notify teams of warnings. Alert fatigue mitigation groups correlated alerts (multiple breakers from same root cause) into single incidents.