Retention-time checking

After MS1 and MS2 matching, the selected RT engine fits a model of expected retention time from confident annotations in this run, optionally refined by an external anchor file. It then applies its own decision layer to flag and remove implausible hits—candidates whose structure does not match their observed retention time.

WORKFLOW:
  rt_check: liri               # False | 'liri' (default) | 'hydra'
  rt_ref: '/output/rt_ref.csv' # Optional external RT anchor file

PARAM:
  rt_deviation_cutoff: 0.15    # final generic-formula check; fraction or seconds

Exactly one engine runs. The value is case-insensitive, and an unrecognised string fails configuration validation rather than silently disabling the check. Every engine works without any external reference — rt_ref is optional, purely to sharpen the fit. WORKFLOW.is_peaks is a separate setting that supplies internal-standard peaks; it is not an RT-check parameter.

Pipeline order

After MS1/MS2 matching, cleanup, and optional internal-standard matching, the selected RT engine runs, followed in every case by the generic formula check:

The engine-specific settings stay independent: rt_liri_* controls LiRI training, alignment, uncertainty, and z-score decisions; rt_hydra_* controls HYDRA anchors, calibration, uncertainty, competition, and z-score decisions. Progress output contains one RT checking... header and a timed summary for the selected engine.

Principle

Lipids are modular: a headgroup plus one or more acyl, alkyl, or sphingoid chains. In reversed-phase LC, retention is dominated by hydrophobic surface area, which follows that same modular structure fairly predictably. More carbons elute later, more double bonds or oxygens elute earlier, and a given headgroup shifts its whole homologous series by roughly a constant amount.

That regularity is what makes the check possible. A wrong MS1 or MS2 annotation is often also a retention-time outlier. The selected engine fits an expected-RT model from confident annotations in this run, optionally sharpened by an external anchor file, and applies its own decision layer. Afterward, the generic formula model supplies the exported rt_deviation and optional final cutoff.

Every engine works without any external reference file, purely from this run's own confident annotations.

Reference data

To reduce spurious identifications, we recommend building a retention-time library for each gradient, from standards or manually validated IDs. Point at it with:

WORKFLOW:
  rt_ref: '/output/rt_ref_2min.csv'

The CSV must have these columns:

hg,c,db,mod,rt
SM,30,0,O2,53.46
SM,40,0,O2,90.3
SM,41,0,O2,93
SM,42,0,O2,96.54
SM,32,0,O2,61.5

External anchors are never trusted on their own time axis. They are first aligned onto this run's RT axis through a robust monotone warp, which is outlier-trimmed and leave-one-out validated. If too few pairs match, or the leave-one-out error is too high, the alignment is refused outright and the engine falls back to running anchor-free. A reference file from a different gradient will therefore not silently corrupt the fit.

The two engines

LiRI — Lipid Retention Index (default)

LiRI builds a retention index — a structural score in carbon-equivalent units:

h = C - a_db × DB - a_o × O

It then fits a monotone map from h to RT independently for each (class, ether) pair, using only that pair's own anchors. The coefficients a_db and a_o are fitted once globally (borrowing statistical power is safe for a single ratio), but the retention curve itself is never shared between classes—this is deliberate, not an oversight.

Why no cross-class curve? Earlier versions tried this. It failed: classes like triacylglycerols with genuinely different retention slopes got pulled into a curve shaped by steeper classes and mispredicted by tens of seconds. Better to be silent than wrong.

Each candidate is graded into a tier that determines how forcefully it can be rejected:

A hard rejection bypasses score protection. A soft rejection is forgiven for high-confidence, score-protected idlevel 2 annotations. A group with fewer than rt_liri_min_group_n anchors gets no prediction at all and is left untouched.

Tradeoff: safety vs. completeness. LiRI will never flag an implausible idlevel 1 hit in a class without local evidence (FA, CE, or any class unconfirmed by MS2 in this run). That is by design. LiRI is safe—it will not remove something wrongly for lack of evidence—but incomplete—classes without evidence pass through unfiltered no matter how far their RT deviates. This is the right default for conservative filtering. If you need to catch implausible hits in every class, see HYDRA below.

HYDRA — HYDrophobic Retention Axis

HYDRA takes the opposite side of the LiRI tradeoff: it keeps ONE shared monotone calibration curve, but shares information across classes at the feature level, not at the curve level.

Each candidate is encoded as a feature vector: headgroup SMILES-fragment composition (the same fragments used to generate full structures, so PC, LPC, and SM all get an identical phosphocholine descriptor), lipid-family indicators, and chain inventory (carbon count, double bonds, oxygen, ether, lyso flags). Anchored classes learn their own offset on the shared curve. Unanchored classes get their offset predicted by ridge regression—a chemistry-informed guess rather than no prediction.

HYDRA also computes a pairwise relative z-score (z_rel) from a constraint graph over nearby anchors: it checks whether a candidate's RT is ordered correctly relative to its neighbours, not just against the absolute curve. Tier-C candidates with class support blend 70% absolute and 30% relative z.

What this buys: HYDRA can flag an implausible hit in a class with zero anchors—purely from chemistry. A real FA 22:1;O2 annotation at 114.5 s (where fatty acids of that size elute around 25 s) was caught and removed through this path. LiRI cannot do this.

What it costs: stricter idlevel-1 threshold (rt_hydra_z_soft_ms1, default 0.75) can substantially increase removals. Raise it toward rt_hydra_z_soft or set to 0 to disable the special idlevel-1 setting if that behavior is too aggressive.

The final generic formula check

After the selected RT engine has run, a final generic check is applied to remove matches that are obviously wrong. Its purpose is to catch the sporadic idlevel 1 hits that a conservative engine such as LiRI leaves behind. The advantage of running it last is that it can be trained on high-quality anchors that the engine has already verified.

The model is deliberately simple — a single formula-based regression across all classes pooled. It is implemented in src/dashboard.rs::compute_rt_deviation and works in four steps:

1. Train on annotations with idlevel ≥ 2, a non-empty molecular formula, and score ≥ 0.7.
2. Convert each formula into the features 1, C, H, N, O, P, S, degree_of_unsaturation.
3. Fit a robust ridge regression (penalty 0.5) of observed RT against those features; large residuals are trimmed before the final fit.
4. For every surviving annotation with a parseable formula, compute rt_deviation = abs(observed_rt - predicted_rt). Rows without a usable formula receive 0.0 and are never removed by this check.

The cutoff comes from PARAM.rt_deviation_cutoff, resolved in src/rt.rs::resolve_rt_deviation_cutoff:

Rows with rt_deviation > resolved_cutoff are removed after the selected engine. This is the only stage where rt_deviation_cutoff is used. Because the model is intentionally crude, the cutoff should be left loose — it is a safety net for gross outliers, not a second opinion on borderline calls.

When rt_check: False, rt_deviation can still be computed and displayed for inspection, but no rows are removed, because no RT engine was selected.

Two different RT errors

The output contains two distinct kinds of RT error, and confusing them leads to wrong conclusions:

• The engine diagnostics (rt_liri.csv, rt_hydra.csv) contain the prediction and error used by that engine's own decision layer.
• rt_generic.csv is written after the selected engine and contains the final generic formula check for every input row: its predicted RT, error, resolved cutoff, and keep/remove verdict.

The exported rt_deviation value in annotation.csv, annotation_full.csv, and the dashboard comes from the generic model, so it always reflects what the final cutoff actually checked. An engine diagnostic error and rt_deviation may therefore differ — that is correct and expected, and the generic cutoff applies only to the latter.

Comparison and recommendations

Default to liri. It matches HYDRA's removal precision (~77% vs ~74%) with a smaller, better-understood footprint and a hard safety guarantee: a class with no local evidence is never flagged. The tradeoff is that false annotations survive in classes without MS2 confirmation in this run.

Use hydra when coverage and idlevel-1 confidence matter more than conservative filtering—for example, a broad-spectrum study with many classes that rarely get MS2 confirmation, or when you want to catch implausible hits in unconfirmed classes. Review and tune rt_hydra_z_soft_ms1 (default 0.75); it is stricter than the generic threshold and can substantially increase removals. The final generic formula check (rt_deviation_cutoff) always applies after HYDRA runs.

Use False when validating a new sample type or gradient. When you do not yet trust any RT-based judgement, it is better to see everything MS1 and MS2 produced and filter by hand than to lose candidates silently to a model trained on the assumption that this run's chromatography is well behaved. The generic formula model can still compute and display rt_deviation for inspection even when rt_check: False.

Whichever engine you choose, validate any threshold change against replicate reproducibility or an rt_ref-graded audit, not just a single run's annotation counts.

Diagnostics and troubleshooting

Each engine writes a per-annotation diagnostic trail to the diag/ output folder when it produces a model, so you can see exactly why any given candidate was kept or removed:

• LiRI: diag/rt_liri.csv — ether, sum_c, sum_db, sum_o, rt, rt_pred, sigma, z, tier, verdict
• HYDRA: diag/rt_hydra.csv and diag/rt_hydra_anchors.csv — h, rt_pred, sigma_rt, z_abs, z_rel, tier, verdict, and score before/after
• Final generic formula (after any engine): diag/rt_generic.csv — rt, rt_pred, err, cutoff, protected, severe, verdict for every input annotation

The dashboard also shows the generic filter as a scatter plot of predicted vs. observed RT for every hit, coloured by identification level. Points whose RT deviates excessively from the generic model are the ones removed by rt_deviation_cutoff, so the plot makes it immediately visible whether the model is behaving and whether the cutoff is set sensibly.