Skip to content
Lyncat Tria
Documentation pages

tria eval

tria eval tells you how often Tria would get it right with your rules. It takes a sample from an archive you’ve already sorted, asks Tria to sort it again and compares what it proposes with where each document really is.

Evaluating sends the sample documents to your provider and spends from your key. Before it starts, Tria tells you how many documents it will evaluate, with which model and how much it will cost, and waits for your yes.

How it works

  1. Picks a random sample of the documents already inside some folder of the archive, leaving out those in folders Tria doesn’t touch. 200 by default.
  2. Copies each one to a temporary folder under a random name, so the name gives nothing away.
  3. Reads them with the archive’s TRIA.md and PERFIL.md and with the folder list, which leaves out the sample documents.
  4. Compares the folder and the name it proposes with the real ones.
  5. Deletes the copies when it’s done.

It never touches the originals. It doesn’t use the cache: every evaluation reads the sample again.

bash
tria eval ~/Documents/Business --sample 50tria eval --synthetic# with made-up documents

The folder needs its own TRIA.md, or you can give it a different one with --rules. With --synthetic you don’t need any documents of your own: it uses a fictitious company’s archive that comes with the CLI.

The result

  • Folder accuracy: in how many documents it proposes the right folder.
  • Exact and similar name: how many names match exactly and how many come close.
  • Pending and unreadable: what Tria would leave in the inbox and what it can’t read.
  • By confidence: how many documents come out with high, medium and low confidence, and how accurate each one is.

It ends with the tokens and the cost. The high-confidence line is the one that tells you whether automatic mode, which only files high-confidence documents, is right for you.

Options

OptionWhat it does
--syntheticUses the made-up archive that comes with the CLI.
--rulesA rules file instead of the folder’s TRIA.md.
--sampleSample size. 200 by default.
--seedThe sample seed: same seed, same sample. 42 by default.
--outSaves the report as JSON to a file.
--baselineCompares with a saved report and exits with 1 if accuracy drops.
--jsonWrites the report as JSON.
-yDoesn’t ask before spending.

So it doesn’t get worse

Save a baseline report and compare against it every time you change the rules or the model:

bash
tria eval ~/Documents/Business --sample 50 --out baseline.jsontria eval ~/Documents/Business --sample 50 --baseline baseline.json# exits with 1 if it drops

With the same seed, the same size and the same archive, the sample is the same, so the comparison is fair. If the evaluation is cut short, for example because your account runs out of credit, the report only counts what was finished and isn’t compared with the baseline.