Methodology

This page is the complete rulebook: what gets tested, how it is scored, and what would have to be true for a score to be wrong. Some humanizers in these rankings are made by the operator of this benchmark; they play by every rule on this page, identically to every other tool. Nothing below asks for trust: each cycle ships its inputs, outputs, detector verdicts, and scoring code in a public repository (MIT for code, CC BY 4.0 for data), so every claim here is checkable against the files. Read our fairness policy or why we built this. Current version: v1.0.0.

Sample selection

A cycle's test set spans the seven kinds of writing people actually run through a humanizer:

Every cycle publishes its complete template set, word-count targets and placeholder tokens included, as templates.json, and the AI-written source texts the humanizers must rewrite as samples.json. How many samples ran is visible directly in the published data.

Templates carry [BRACKETED] tokens: [TOPIC], [ROLE], [FIELD], and so on. When a cycle starts, a secret random nonce drives a deterministic selection that picks which value from each bank lands in each slot. The banks themselves live in version control and are frozen into the cycle as banks.json.

Automated execution and evidence

No hands touch a test. Each tool runs automatically on its default settings, which is what makes the procedure identical across all of them and the results re-derivable from the published files. Tools that can only be operated manually stay out of the benchmark: a run nobody can repeat is a run nobody can check. And because humanizers are stochastic (the same input produces a different rewrite each time), judgment comes from the whole sample set, never from one lucky or unlucky output.

Each run produces three layers of evidence:

• The data itself — one JSONL row per test carrying the source text, the tool's output with its settings, and all seven detector scores. The leaderboard recomputes from these rows alone. Results are published rather than raw network traffic: traffic would prove nothing further, and would hand every vendor a fingerprint of benchmark requests to special-case.
• The cryptographic frame — the commit-reveal chain described below plus the timestamp anchor, which together close off cherry-picking before the run and editing after it.
• The zero-tooling check — every detector in the panel is a public product. Take any published output, paste it in, and compare what comes back to what we recorded. The evidence pages lay out every prompt, every rewrite, and every verdict for exactly this purpose.

Humanizer configuration

Humanizers are not standardized: each exposes its own modes, models, and strength settings. One configuration rule applies to every tool, so none is tuned more favorably than another:

• Default settings. Every tool runs on its own defaults: the mode, model and strength a normal user gets without changing anything. We do not hunt for a stronger option on some tools and not others, and we do not enable premium or “maximum stealth” modes. Where a tool requires a choice it has no default for, we pick a neutral one and publish the exact value used in settings_used for every test, so any run can be repeated with the same configuration.
• Base tier. Every tool is exercised on the tier an ordinary user reaches without upgrading. We do not move to a higher tier, enable premium models, or buy add-ons to chase a better score, and the tier each run used is published per test as plan_tier_used.
• Capacity only, never capability. If a tool needs more capacity to finish a cycle we obtain only enough to complete the run, and still use the base tier's features, models and settings. Nothing we do to finish a run changes what is being measured.

Whatever configuration actually ran is written into leaderboard.json as plan_tier_used and settings_used; no score exists without its settings on record beside it.

Detectors

Every output is scored by seven independent commercial AI detectors: GPTZero, Originality.ai, Copyleaks, Winston AI, ZeroGPT, QuillBot, and Grammarly. None of them are operated, fine-tuned, or coordinated with by this benchmark. Each detector receives the identical text and returns its own verdict, recorded on a [0, 1] scale: 1 means the detector judged the text human, 0 means it judged it AI.

Normalization is limited to unit conversion: an AI probability becomes human = 1 − ai, and a 0–100 scale is divided by 100. Beyond that, no calibration, reweighting, or adjustment is applied: the number the vendor returned is the number that enters the per-test median. All verdicts are published per cycle in detector-scores.json, one scores object per test, keyed by detector.

Scoring formula

A humanizer's composite is built from four sub-scores, each in [0, 1]. The weighted sum is bounded [0, 100]; applied penalties come off that sum, and the result is kept within [0, 100]:

composite_raw =
    42 * bypass_rate
  + 32 * meaning_preservation
  + 16 * readability
  + 10 * consistency_across_categories

composite = max(0, composite_raw - sum_of_penalty_deltas)

The weights and penalty constants are written directly into the scoring.js shipped with each cycle, so every published leaderboard carries the exact constants that produced it. Aggregation covers every test in the cycle regardless of status: an errored, refused, or identical result counts toward the averages and triggers its penalty. If failures were excluded instead, failing would raise a tool's score.

bypass_rate

Each test contributes the median of its seven detector scores. A null indicates the detector could not be reached for that test; it is skipped, not counted as zero, because missing data is not a verdict. The bypass rate is the mean of these per-test medians.

meaning_preservation

Measured per test as the cosine similarity between embeddings of the input and the output, clamped to [0, 1]. The raw value is published in tests.json; the meaning-drift penalty reads the same number.

readability

Each output receives one writing-quality rating in [0, 1] for clarity, fluency, and naturalness, assigned by a language model and cached by text hash, so the value is stable across re-runs. The per-test rating is published in tests.json, and the published scoring code only averages those stored values. Reproducing the leaderboard therefore requires no model access at all.

consistency_across_categories

Within each writing category, the per-test bypass medians are averaged; consistency is then max(0, 1 - stddev(category_means)). Equal performance in every category yields 1. Strong results in one category paired with weak results in another pull this term down.

Penalties

Output-quality failures reduce the composite. Every penalty carries a fixed per-occurrence deduction and a per-code cap, which bounds the total possible deduction at -50.0 of 100 and prevents any one failure mode from deciding a ranking on its own. Each humanizer's applied deductions appear in leaderboard.json, itemized.

Unavailability is excluded, not penalized

When half or more of a tool's attempts fail at the access level (service down, quota exhausted, requests blocked), that cycle did not meaningfully test the tool. It is excluded from the ranking for the cycle and listed separately in leaderboard.json with its attempt counts, rather than scored on whatever fraction happened to complete or docked points for an outage. Quality failures are treated differently: if the tool ran and returned a bad result, the penalties above apply.

Transparency and verification

Cycle names follow the calendar, so if the prompt draw were seeded on the name, any vendor could compute October's prompts in September and train against them. The seed is therefore a secret, but one we prove we did not change after the fact, using commit-reveal:

1. 1
   Commit

   Before anything runs, a random 32-byte nonce is generated and only its SHA-256 fingerprint goes public, as commit.json. The nonce itself stays sealed.

2. 2
   Run

   Every humanizer processes the corpus automatically, and every output is scored. What gets kept is the result (input, output, settings), not the network traffic around it.

3. 3
   Reveal

   When the cycle closes, the nonce goes public together with frozen copies of the templates, banks, selection code, and the prompts they produced.

4. 4
   Anchor

   The finished bundle's hash is stamped with a public timestamping service: independent proof the data existed in exactly this form at publication.

One command checks the whole chain, and each check closes off a different way a benchmark could be quietly manipulated:

• The revealed nonce hashes to the fingerprint published before the run, and feeding it through the frozen selection code lands on exactly the published prompts.json: the prompt set was not reselected after the fact.
• Every source text traces to a committed prompt, and every test fed a tool the published text verbatim: the corpus was not quietly swapped for an easier one.
• The full leaderboard recomputes from the raw rows, every field within 1e-4 (the replay described below): no number was edited after scoring.
• Every humanizer faced every sample and every test carries a score from every detector: no inconvenient rows were dropped before averaging, which would otherwise recompute cleanly.
• Every file matches its hash in the bundle manifest: nothing changed since publication.

To see for yourself:

git clone https://github.com/AIHumanizerBenchmark/AIHumanizerBenchmark.git
cd AIHumanizerBenchmark
npm run verify                      # checks all published cycles
npm run verify -- "September 2026"  # checks a single cycle

There is no install step. The verifier uses only Node's standard library and runs entirely offline. A verification tool whose dependencies you have to trust does not verify much.

Recomputing every score from the raw data

Every number on the leaderboard is a function of the published files. The scoring script in each cycle bundle is self-contained, with no dependencies; recomputing the full ranking is an import and one function call:

import { computeLeaderboard } from "./cycles/september-2026/scoring.js";
import tests from "./cycles/september-2026/tests.json";
import detectorScores from "./cycles/september-2026/detector-scores.json";

// Returns every humanizer, ranked by composite descending.
const ranked = computeLeaderboard(tests, detectorScores);
console.log(ranked[0]); // top-ranked humanizer for the cycle

A single sub-score can be checked by hand. For one humanizer's bypass rate:

1. Take that humanizer's rows from tests.json, all of them. Rows with error, refusal, or identical status stay in, since dropping failures would raise the average.
2. For each row, locate its entry in detector-scores.json and take the median of the scores values. Skip any null; an unreachable detector is missing data, not a zero.
3. Average those medians. That is the bypass rate.
4. Compare against leaderboard.json, using an absolute tolerance of 1e-4 for floating point.

Versioning

Every cycle's leaderboard.json records three version stamps:

Roster changes do not move any stamp. Which humanizers and detectors ran is recorded in each cycle's own data, and that record is the authority.

Published cycles never change. A bundle (templates, banks, selection code, scoring code, raw data, leaderboard) is frozen the moment it is published. Later methodology changes affect later cycles and their version stamps; they cannot reach backward.

Methodology integrity

Rankings are never edited by hand. They are the output of the published scoring code applied to the published data, and the claim rests on structure, not reputation: a leaderboard that did not come from that code and that data fails public verification, and the verifier is available to anyone.

How to contribute or dispute

Think a score is wrong, or spotted an error in the data? File a dispute: we respond within 14 business days and publish the resolution. You can also submit a humanizer or suggest a detector for a future cycle. For anything else, including corrections, questions about the data, or a security report about the verification code, write to contact@aihumanizerbenchmark.com.