In one sentence. Machines continuously check whether a model is technically working, humans report whether it's actually any good, and every report — founded or not — leaves a permanent record.
Picture it like this. Restaurant inspection has two halves. An inspector can measure the fridge temperature and count violations; only a diner can tell you the food was terrible. Neither substitutes for the other, and a system built with only one of them fails in an entirely predictable direction.
Why it matters. A centralized platform can simply investigate and delist. In a permissionless network where anyone can register anything, the interesting question isn't who polices quality — it's what structure makes lying unprofitable when nobody is in charge.
Curious how reputation is actually scored? Skip to the EMA algorithm — the "never holds a grudge forever" property is the elegant bit.
The Fundamental Problem
A decentralized AI routing network has an asymmetric information problem. Send a prompt to model X through provider Y and three things can go wrong. The provider silently downgrades the model, serving GPT-3.5 to someone who paid for GPT-4. The provider fabricates responses without running inference at all. Or the model itself produces harmful or systematically broken output.
A centralized gateway like OpenRouter has an answer: the platform operator investigates and delists bad actors. In a permissionless network where anyone can register any model, there is no central authority to play judge.
ComputeFlux solves this through a dual-mechanism governance system: community-driven reports (ModelReport) and system-executed audits (ModelAudit). This builds directly on the governance machinery introduced in Article 17: Governance — Gov Track Proposals. Where Article 17 covered how the network decides what the rules are — track periods, deposits, voting, and Sudo-gated execution — this article covers how those rules get enforced against individual models and providers on an ongoing basis. The duality between reports and audits is not an accident of implementation — it reflects a deliberate architectural choice about the distribution of trust, the scalability of moderation, and the economic incentives that make honest participation rational.
Why Two Mechanisms Instead of One?
Consider the alternatives. A purely community-driven system, reports only, runs straight into the Sybil problem: a malicious provider spins up thousands of fake accounts to report competitors and drowns the genuine signal in noise. A purely system-driven one, automated checks only, can confirm an API endpoint is reachable and returns syntactically valid JSON. It cannot tell whether the responses are semantically correct. That takes human judgment about context, tone, and factual accuracy.
The dual mechanism splits the verification burden along a natural boundary. Community reports handle the qualitative dimension: "this model is returning gibberish," "the responses don't match GPT-4 quality," "the provider is censoring certain topics." These are judgments that automated systems cannot reliably make. System audits handle the quantitative dimension: "the endpoint is returning 500 errors," "latency exceeds SLA thresholds by 3σ," "response format does not conform to the OpenAI schema." These are checks that scale infinitely without human intervention.
The interplay creates a feedback loop. A spike in community reports triggers an automatic system audit (via AuditTriggerReport = 1). If the audit confirms the issue, the model's reputation score drops, which deprioritizes it in the scheduler. If the audit finds no technical fault, the reports are resolved with valid=true — meaning the model passed the objective check, and the community's concern is dismissed (or at least not corroborated by automated verification). This prevents reputation attacks where competitors mass-report a model with vague or fabricated complaints.
Beyond report-triggered audits, the system also supports random spot checks (AuditTriggerRandom = 2). These serve as a baseline quality surveillance mechanism: even if no user files a report, models are periodically tested against known prompts with verifiable outputs. The random audit rate can be tuned by governance — higher rates consume more network resources but catch degradation faster. This is conceptually similar to how blockchain light clients randomly sample block headers: the probability of detection compounds over time, making sustained misbehavior economically irrational.
The Report Lifecycle as a State Machine
The lifecycle of a report is deliberately constrained to prevent abuse in both directions — neither false reporters nor malicious providers should be able to game the system. The state machine has three logical phases, though the implementation collapses the last two into a single boolean flag:
CREATED (Resolved=false)
│
▼
PENDING REVIEW — report sits in a queue visible only to system auditors.
│ Public queries (GetModelReports) filter to Resolved=true,
│ preventing premature reputational damage from unverified claims.
│
▼
RESOLVED — Sudo calls ResolveReport(reportId, valid, detail)
│
├── valid=true → AuditModel(passed=true) → reputation reinforced
│ (Report was investigated, model found working correctly;
│ the EMA score moves toward the passing value, same as any other passed audit)
│
└── valid=false → AuditModel(passed=false) → reputation penalized
(Report was confirmed, model has quality issues)
A critical design decision is that reports cannot be "dismissed" in the traditional sense. Every report must result in an audit record. Even if a report is determined to be unfounded (valid=true), the system still creates an AuditModel entry with Passed=true. This creates a complete, immutable paper trail: for any model, you can trace every report, every audit, and every reputation adjustment back to its origin. This auditability is essential for a system that aspires to replace centralized trust with verifiable computation — the same principle underpinning the Gov Track's requirement in Article 17 that every proposal, deposit, and vote leaves a permanent on-chain record.
Contrast this with a hypothetical "dismiss without audit" flow: if Sudo could simply set Resolved=true without creating an audit record, the system could silently suppress legitimate complaints. The current design makes suppression computationally visible — any gap in the audit trail is itself evidence of censorship.
The privacy model for pending reports is deliberate. GetModelReports (the public query) filters with QueryAND{Resolved: uint64(1)}, returning only resolved reports. The internal GetModelReportsAll bypasses this filter but is restricted to GatewaySecret — meaning only the system's TEE execution context can view pending reports. This prevents two failure modes: (1) malicious actors cannot scrape pending reports to identify vulnerable models before they're investigated, and (2) providers cannot preemptively retaliate against reporters whose claims haven't been verified yet.
Sybil Resistance: The Missing Layer and Its Implications
In its current implementation, the report system has no explicit Sybil resistance. Any authenticated caller — meaning anyone who holds a valid API key — can submit a report. There is no staking requirement, no minimum reputation threshold for reporters, and no cost to filing a report beyond the transaction gas fee.
This is a deliberate trade-off with identifiable risks and compensating controls. The argument for low-barrier reporting is that false negatives are more dangerous than false positives in a quality enforcement system. If a model is genuinely returning harmful or fraudulent output, making it hard to report means users suffer in silence. The compensating control is the Sudo-gated resolution step: a flood of false reports doesn't automatically damage a model's reputation because reports are invisible to the public until resolved. The system bears the operational cost of processing spam reports (Sudo must call ResolveReport on each one), but the model's public reputation score is unaffected.
That said, a more mature system should introduce stake-weighted reporting. The simplest version: a reporter must lock a small amount of tokens with their report. If the report is resolved as valid=false (confirmed valid), the stake is returned and the reporter may receive a bounty drawn from the provider's slashed deposit. If the report is resolved as valid=true (unfounded), the stake is burned or transferred to the provider as compensation for the investigation cost. This creates a simple, game-theoretically sound incentive: honest reporters have positive expected value (bounty > gas cost), while dishonest reporters face a certain loss.
The stake amount must be calibrated carefully. Too high, and legitimate users won't bother reporting (the cost exceeds the personal benefit). Too low, and Sybil attackers can still afford to spam. A dynamic stake based on the model's total deposit or revenue volume could help: reporting a high-value model requires a proportionally larger stake, making mass false reports against premium providers economically prohibitive.
Auditor Identity and Accountability
The current implementation uses caller.T == 0 as the Sudo gate — meaning only the system's own contract execution context can call ResolveReport and AuditModel. In the transition period before full DAO governance (the same transition arc discussed for governance proposals in Article 17), this means the core development team, or whoever controls the validator set, acts as the sole auditor.
This creates an obvious accountability problem: what prevents the auditors from colluding with providers to suppress legitimate reports or to falsely penalize competitors? The answer lies in the TEE-based execution environment. The Sudo caller (T == 0) is not a human operator with a private key — it is the CometBFT consensus engine executing within an SGX enclave. The audit logic runs in the same TEE that processes all other state transitions. This means that:
- Audit decisions are deterministic — if the audit is automated (e.g., checking endpoint health, verifying response schema), the outcome is computed programmatically within the enclave, not subject to human discretion.
- The audit trail is immutable — every
ModelAuditrecord is stored in the PebbleDB-backed state, replicated across all validator nodes via CometBFT consensus. Tampering with audit history would require compromising ≥2/3 of the validator set. - Remote attestation provides verifiability — any observer can request a TEE quote that proves the enclave is running the correct audit code. If the code is open-source, anyone can verify that audit decisions follow the published algorithm.
However, for the qualitative judgments that currently require human Sudo intervention (e.g., "is this model returning gibberish?"), the TEE guarantee is weaker. The enclave guarantees that the Sudo caller's decision is faithfully recorded, but it cannot guarantee that the decision itself is correct. This is where the transition to DAO governance matters. Spreading the auditor role across stakeholders with conflicting incentives — providers, users, token holders — cuts collusion risk through the same mechanism that makes BFT consensus secure. A malicious outcome requires a supermajority of independent actors to agree on it.
The roadmap envisions a multi-auditor committee where resolution requires a threshold signature from a rotating set of randomly selected validators. Each auditor stakes tokens that can be slashed if their audit decisions deviate significantly from the committee consensus, creating a Schelling-point oracle similar to how Kleros and UMA operate in the DeFi space.
Economic Incentives: Why Participate Honestly?
For the report-and-audit system to function without constant Sudo intervention, it needs to be a self-sustaining game where honest behavior is the dominant strategy for all participants. Let's analyze the incentive structure for each role:
Reporters (users): In the current system, the incentive to report is primarily altruistic — you help maintain network quality. The personal benefit is indirect: if bad models are removed, your future API calls are more likely to succeed. A mature system should add direct incentives: a bounty paid from the provider's slashed deposit when a report is confirmed valid. The bounty amount should be calibrated so that P(valid) × bounty − P(invalid) × stake_burn − gas_cost > 0 for honest reporters and < 0 for dishonest ones.
Providers: The incentive is straightforward. A low reputation score is a public signal that makes a model less attractive to route to, and less routing means less revenue. Even where that signal isn't yet wired into automated scheduler weighting, it still surfaces in any query of the model's track record.
The EMA algorithm sharpens the incentive. A new provider with few audits can watch their score collapse from a single failure, because n is small and one result=0 audit carries enormous weight. The lesson is to be good early: build a track record of passing audits so that later failures barely move the number.
Auditors (Sudo/DAO): In the current Sudo model, the auditor's incentive is aligned with the network's success — if the network loses credibility due to unchecked bad models, the value of the platform (and any tokens held by the team) declines. In the DAO model, auditors stake tokens and earn a share of audit fees. Honest auditing is profitable because: (a) audit fees exceed gas costs, and (b) slashing penalties for deviating from consensus exceed any bribe a provider might offer. The Schelling-point dynamic works because each auditor assumes others will vote honestly, making honest voting the safest strategy.
The system's economic security ultimately rests on the same principle as proof-of-stake consensus: the cost of corruption must exceed the value that can be extracted through corruption. If a provider can earn $1M by serving a downgraded model for a month before being caught, the total staked value of the auditor committee must exceed $1M (plus a security margin) for the slashing threat to be credible.
The EMA Algorithm: Mathematical Intuition
The reputation scoring algorithm uses an exponential moving average with a window cap of 100:
new_score = (old_score × (n-1) + result) / n
where n = min(TotalAudits, 100)
result = 10000 if passed, 0 if failed
The window cap of 100 is not arbitrary. With n=100, each new audit contributes 1% to the score. This means that after 100 audits, a provider's reputation is a weighted average where the most recent 100 audits each contribute equally (1%), and all older audits are effectively discarded. This gives the system a "memory" of approximately 100 audits.
For a provider audited once per day, this means reputation fully reflects the last ~3 months of performance. A provider that was excellent for a year but starts degrading will be caught within weeks, not months. Conversely, a provider that was poor initially but improved will see their score recover within 100 audits — the system doesn't hold grudges indefinitely.
The binary scoring (10000 or 0) is deliberately extreme. There is no partial credit for "mostly working but slow." This forces the audit criteria to be precisely defined: what exactly constitutes a "pass"? The answer must be specified in the governance proposal that defines the audit parameters — routed, naturally, through the same Gov Track mechanism from Article 17 — and it must be a verifiable predicate that can be evaluated programmatically within the TEE. This binary approach avoids the ambiguity and disputability of subjective scoring, though it may miss nuanced quality degradation that doesn't trip the pass/fail threshold.
Conclusion
The ModelReport/Audit system is a microcosm of ComputeFlux's broader design philosophy: replace centralized trust with verifiable computation, use economic incentives to align participant behavior, and provide escape hatches (DAO governance) for the edge cases that automation cannot handle. Its current Sudo-gated implementation is a bootstrapping mechanism, not the end state. The roadmap to DAO-based multi-auditor committees will transform it from a benevolent-dictator quality control system into a truly decentralized quality market — one where the cost of lying exceeds the profit, and the truth emerges from the equilibrium of self-interested actors.
Key Takeaways
- Two mechanisms, because they cover genuinely different ground. Automation scales infinitely and cannot tell whether an answer was any good. People can judge quality, don't scale, and can be faked by the thousand. Either one alone fails in a predictable way.
- Reports cannot be quietly dismissed. Even an unfounded one produces an audit record, which means suppression would show up as a gap in the trail. The paper trail is the accountability mechanism.
- Pending reports are hidden, blocking two attacks at once: scraping the queue to find vulnerable models before they're fixed, and retaliating against a reporter before their claim has been checked.
- A named weakness: filing a report currently costs nothing and has no Sybil resistance. The compensating control is that unresolved reports can't damage anyone's reputation. Stake-weighted reporting is the acknowledged next step, not a shipped feature.
- The reputation score forgets on purpose. With a memory of roughly the last hundred audits, a provider who improves recovers, and one who degrades is caught within weeks. No permanent grudges — and no permanent immunity earned by a good past.
- The honest limit of the hardware guarantee. The enclave guarantees a decision was faithfully recorded; it cannot guarantee the decision was correct. That gap is exactly why the human half of this system needs to move to a distributed committee.
Governance decides the rules; reports and audits enforce them at the model level. But both of those assume the contract code doing the enforcing is itself sound.
Next — Article 19: Contract Security Audit: turning the audit lens around, onto ComputeFlux's own code.