# Methodology

**Version 0.2.0 — DRAFT. Nothing has been published. Source agreements are archived; verification of figures is in progress.**

This document is the contract with the reader. It states where every number
comes from, what was assumed to make numbers from different agreements
comparable, and what this project will not do. It is versioned, and every
change is logged at the bottom.

Items marked **[OPEN — ROMAN]** are product decisions that have not been made.
They are listed rather than guessed. Nothing publishes while any decision that
affects a displayed number is still open.

---

## 1. What this is

A cited, side-by-side comparison of compensation terms for aircraft mechanics
across a small number of US carriers, built only from public documents.

**Scope:** US carriers with represented mechanics, expanding in waves —
launch wave: American, United, Southwest (plus Delta as an honest
employer-published gap); wave 2 in progress: Alaska, Allegiant; queued: UPS
Airlines (IBT 2727), Hawaiian (IAM), Spirit (representation being verified).
Carriers whose mechanics have no union (Delta, FedEx) appear with their
non-CBA status stated plainly — the absence of a contract is itself
information. Mechanic and related classifications only. Event-driven updates,
not continuous.

**Not in scope:** historical archives, clause-level benchmarking across dozens
of agreements, grievance deadline tooling, individual advice.

## 2. Sources

Every source document is downloaded once, hashed with SHA-256, and archived
under `data/sources/`. The archive is immutable: files are stored read-only and
are never overwritten. If a document changes upstream, the new copy is stored
alongside the old one and both are kept.

The authoritative source list is `data/sources/manifest.json`, which records
for every retrieval: URL, SHA-256, retrieval timestamp (UTC), carrier,
agreement label, page count, and whether the PDF has a real text layer.

<!-- Generated from manifest.json at publish time. Empty until src/fetch.py runs. -->

| Document | Carrier | Retrieved | Pages | SHA-256 |
|---|---|---|---|---|
| _none archived yet_ | | | | |

### 2.1 Source quality is not uniform

A ratified collective bargaining agreement and an employer-published pay scale
are not the same kind of evidence. The dataset records which one each figure
came from (`agreement.kind`), and the site labels employer-published figures
wherever they appear. A carrier whose mechanics are not represented has no CBA;
its numbers are weaker and are presented as such.

### 2.2 What a citation points at

Every number carries a source reference naming the document, the **1-based PDF
page**, and the section as the agreement itself labels it (e.g. "Article 4,
Section (c)"). Where the printed page number differs from the PDF page — common
in CBAs numbered by article — both are recorded and the printed one is shown to
the reader.

## 3. How numbers get into the dataset

**Current process: machine-drafted, human-confirmed.** Vision-model reads of
the archived page images produce a **draft layer** (`data/draft/`), each value
carrying its page citation and, where two independent documents state the same
figure, a two-source consensus flag. The draft layer is a review queue in a
format deliberately incompatible with the production dataset: nothing moves
from draft to `data/agreements.json` without a human confirming it against the
cited page, at which point it is entered with that human's verification block.
Machine text extraction (`src/extract_text.py`) additionally produces
page-anchored reading aids, banner-marked as such. No value reaches the
published dataset on machine authority alone.

Every value in the dataset carries:

- a **source reference** — document, page, section
- a **verification reference** — who read it, when, and by what method

### Verification tiers (amended v0.3.0)

| Method | Meaning |
|---|---|
| `human_read` / `human_read_second_pass` | a human read the figure off the archived page |
| `machine_extracted_human_confirmed` | machine-drafted, then human-confirmed against the page |
| `machine_consensus` | two independent machine readers — separate contexts, different models, neither shown the other's output — transcribed the same archived page and agree exactly; the cell also passed the deterministic checks (range, monotonicity, cross-document agreement where a second document states the same figure) |

**Every published value displays which tier verified it.** Nothing is
presented as human-verified that was not.

**Decision record.** v0.2.0 stated no method omitting human confirmation
would be added. On 2026-08-08 the operator, after the residual-risk tradeoff
was stated, directed full autonomy; `machine_consensus` was added under that
authority. The change is disclosed here rather than made silently. Residual
risk: independent published evaluations of two-reader consensus report
roughly 0.25%–1.5% error on numeric extraction; the two readers here share a
model family, so correlated misreads are possible. Mitigations: deterministic
checks, cross-document corroboration, the public corrections process (§7),
and ongoing human spot-checks that upgrade cells to
`machine_extracted_human_confirmed`. Cells where readers disagree, or any
check fails, are **never published** — they go to the flag queue.

### 3.1 Zero, unknown, and not published are three different things

| State | Meaning | How it displays |
|---|---|---|
| a value | a human read this figure on the cited page | the number, with its citation |
| `not_published` | a human read the agreement and the figure is not disclosed | "not published" |
| `not_applicable` | the concept does not exist at this carrier | "n/a" |
| `pending` | nobody has read this topic yet | "not yet verified" |

Conflating "not published" with zero is the single most likely way to produce a
comparison that is technically sourced and materially false. The schema makes it
impossible to write a zero where the correct answer is "not published", because
an absence must carry its own citation proving somebody looked.

## 4. Normalization assumptions

Carriers name and number things differently, so comparison happens at a fixed
canonical topic vocabulary, never at article level. The vocabulary is closed:
adding a topic is a deliberate schema change. Each topic also records
`carrier_label` — what that agreement actually calls the thing — so a reader can
find it in their own contract.

Assumptions that will be stated here once the agreements have been read:

- **Premium treatment.** Whether a license premium is a separate adder or
  already inside the base scale is recorded per carrier (`treatment`). Adding a
  premium that is already folded into base double-counts it.
- **Shift differential form.** Flat cents versus a percentage of base.
- **Retirement.** Employer match, non-elective direct contribution, or both,
  recorded separately rather than summed into one "retirement" figure.
- **Health contributions** reduce effective compensation and are recorded as a
  cost, per plan and per coverage tier.
- **[OPEN — ROMAN]** Whether overtime, profit sharing, and premium-pay-in-OT-base
  are modelled at launch or deferred. They move the number a lot and they are
  the easiest place to be accused of picking a favourable assumption.

## 5. What "total compensation" includes and excludes

**Fixed at v0.2.0 (2026-08-08), under the operator's delegated authority.**
These assumptions are identical for every carrier, are implemented verbatim in
`src/calc.py` (A1–A7), and never change to suit a result. Changing any of them
is a versioned methodology change with a changelog entry.

1. **Straight-time only.** No overtime and no holiday premium in any headline
   number. Overtime and holiday terms are shown as components with citations.
2. **2080 paid hours/year** converts annual and monthly amounts to hourly.
3. **Retirement counts as compensation**: employer non-elective contributions
   in full, and the employer match valued at full capture (the employee is
   assumed to defer enough to earn the entire match). Both computed on the
   cash hourly rate.
4. **Health is subtracted**: the cheapest employee-only tier employee
   contribution reduces effective compensation.
5. **Profit sharing is excluded** — variable and backward-looking.
6. Per-diem, tool, uniform and similar allowances are **excluded** from the
   headline number and shown as components where published.
7. **The component table is primary.** The combined figure is presented as the
   output of these stated assumptions, never as "the" number.

A component whose structure defeats these assumptions (e.g. a tiered premium
with no tier selected) is excluded from the total and flagged visibly rather
than silently approximated. "Not published" is never counted as zero (§3.1).

## 6. Commitments

- **Individual mechanics are never charged.** This is a neutrality commitment,
  not a pricing decision, and it does not change.
- **No position on representation.** This project endorses no union, opposes
  none, and makes no recommendation on any vote, campaign, or contract
  ratification.
- **No commentary on people.** Numbers and clause text only. No commentary on
  any union officer, company officer, or negotiator.
- **No advice.** Nothing here is legal advice or a representation about anyone's
  individual pay, seniority, or rights. The agreement governs, not this site.
- **Public documents only.** No internal company system, no internal grievance
  data, no non-public document.
- **No tracking.** No analytics, no cookies, no third-party scripts, no
  accounts, no personal data collected.
- **Excerpts, not republication.** Clause quotes are capped at 600 characters
  and exist to support a citation, not to reproduce the agreement.

## 7. Corrections

If a number here is wrong, it gets fixed and the error stays on the record.

**To report an error: corrections@amtindex.com.** Please include the carrier,
the figure, and the page of the agreement where the correct figure appears.

Every correction is logged in the dataset itself (`corrections`) and rendered on
the site with: what was published, what is correct, when it was reported, when
it was fixed, and the root cause. Corrections are never quietly edited away.

A wrong published pay figure is treated as a stop-everything event: the figure
comes down, the correction goes up, and the reason the verification step missed
it gets written down before anything else ships.

## 8. Field reference

Data lives in `data/agreements.json`, validated against
`schema/agreements.schema.json` (JSON Schema Draft 2020-12).
`schema/EXAMPLE.fictional.json` shows a filled-in record — **every number in it
is invented, and the carrier does not exist.**

### Shape

```
carrier
└── agreement            kind: collective_bargaining_agreement | employer_published_scale
    └── agreement_version
        ├── sources       key -> { document_id, page, printed_page, section }
        ├── verifications key -> { method, verified_by, verified_at }
        └── topics        the closed canonical vocabulary
```

`sources` and `verifications` are declared once per version and referenced by
key. Every value still names its own source and verification explicitly; it just
names it by reference instead of repeating it. Rendering expands the references
back out, so the reader sees the full citation on every figure.

### The measure

No bare numbers. Every quantity is:

```json
{ "value": 54.32, "unit": "usd_per_hour", "src": "s_scale", "ver": "b1" }
```

or a verified absence:

```json
{ "status": "not_published", "src": "s_lic", "ver": "b2", "note": "..." }
```

`unit` is a closed enum. A value with no unit, no source, or no verification is
rejected by the schema, as is anything claiming both a value and an absence.

### Topic vocabulary

`base_scale`, `step_progression`, `license_premium_ap`, `license_premium_ia`,
`shift_differential`, `lead_premium`, `overtime_rules`, `holiday_pay`,
`retirement`, `health_employee_contribution`, `scope_outsourcing`,
`vacation_accrual`, `sick_accrual`, `grievance_deadlines`.

`grievance_deadlines` is captured because it is in the agreement and cheap to
record while reading. **This project does not compute anyone's deadline and does
not build grievance tooling.**

### Rate tables

A pay scale is a grid, not a list of loose numbers: ordered `step_labels`, the
`step_years` at which each step is reached, and one `rate_table` per effective
date. A rate table declares a default `src` and `ver` covering its cells,
because a rate table is one table on one page read in one sitting. A cell may
override either. `validate.py` resolves this and rejects any cell that ends up
with no source or no verification — the schema alone cannot express inheritance,
so that rule lives in exactly one place in code.

### Checks

`validate.py` enforces, beyond the schema:

- pay steps increase monotonically within a rate table
- no rate below $15/hr or above $120/hr without an explicit override
- percentages between 0 and 100
- every cited page is within that document's actual page count, per the manifest
- every `src`/`ver` reference resolves; no orphans
- effective dates ordered, no overlapping versions
- every cell resolves to a source and a verification

It exits nonzero on failure and runs before every publish.

---

## Changelog

| Version | Date | Change |
|---|---|---|
| 0.3.0 | 2026-08-08 | Verification tiers formalized; `machine_consensus` added by operator decision with full decision record and risk disclosure (§3). Disagreements and check failures never publish. |
| 0.2.0 | 2026-08-08 | Source documents archived (11 documents, 4 carriers). §5 total-compensation assumptions fixed (A1–A7, delegated authority). Draft-layer process documented: machine-drafted, human-confirmed. |
| 0.1.0 | 2026-08-08 | Initial draft. Schema, archival, and text extraction only. No agreement read, nothing published. |