Three jobs this tool already fits
Use cases: RLHF, reasoning and code data
Each workflow names the preset, the threshold behavior, and the expected cost shape. All three run on the same binary with the same two output files.
presets: reasoning-math chain-of-thought correctness anti-sycophancy honesty over agreement code-correctness programs that compile and entail output: clean.jsonl + rejected.jsonl
RLHF: honesty over agreement
Preference data drifts toward whatever the judge agrees with. Filtering with anti-sycophancy scores each exemplar for rated honesty before it reaches the preference pipeline. Runs where the model grovels, hedges to the prompt, or mirrors the user's tone score low on the sycophancy checks and land in rejected.jsonl.
The workflow is the same shape as every other job here: point -i at the mined pairs, --dry-run to read the reject rate, adjust the Noul threshold until the drop rate matches the critique from the human audit pass, then run the batch.
jev-curate -p anti-sycophancy -i pairs.jsonl -o ./curated --dry-run
Reasoning sets: keep the steps that entail the answer
Synthetic reasoning data degrades when the chain of thought is fluent but disconnected from the conclusion. The reasoning-math preset evaluates chain-of-thought correctness and arithmetic soundness, and keeps steps that actually entail the answer. This is the preset the benchmarked row payload could not fool: circular reasoning scores a Noul of 0.05, and the row still passes on reasoning depth when it is sound.
One warning from the codebase: the preset never rewrites verbatim row text. It evaluates and routes. Use it to remove low-entailment rows before fine-tuning, not to patch what a generator wrote badly. For blacklisted reasoning starters such as questions that answer themselves, route those to a reject file by threshold tuning in a dry run first.
Code data: pre-filter before the linter
Large code corpora are mostly fine and cheap to store, but training runs pay the cost of the bad rows in loss spikes. code-correctness scores programs on soundness before they enter a code-model mixture. Rows that fail correctness checks drop out at 0.042 USD per million input tokens, which is cheaper than a human triage pass and faster than waiting on a heavy eval sweep.
Because every row is evaluated with the same questions, the reject file doubles as an error-class report. Classify by the dominant failed check and you have a bug taxonomy for the generator, which is a second deliverable from the same run.
Threshold tuning, in four passes
Every job here follows the same four pass loop. Pass one: --dry-run with the preset defaults, read the reject rate. Pass two: raise or lower the Noul or Score threshold until the reject rate matches your audit sample. Pass three: inspect a slice of rejected.jsonl to confirm the drops are the drops you wanted, not typos or schema drift. Pass four: run the real batch. Cost shape stays flat because dry runs never touch the API and the real run pays the same 0.042 USD per million input tokens regardless of how you tune.
The reject file is also a report. Group rejected.jsonl by the dominant failed question and you get a defect taxonomy for the generator or the miner, which is the second deliverable of the job.