All work

Job Search

An LLM-assisted pipeline for reading job postings honestly

  • Model output is never trusted raw
  • Degrades to a working app with no key
  • Two models, chosen per task

Applying for work is mostly reading. Fifty postings that use the same words to describe materially different jobs, and the part that decides yes or no — is this remote really, is "lead" a title or two direct reports — is usually one clause somewhere in the middle.

So this is a tracker with an LLM in the loop: capture a posting, score it against what I actually want, and keep the pipeline visible. The interesting engineering is not the prompting. It is everything around the model.

What I want, as prose

Fit scoring needs something to score against. That is a written profile, not a form:

The preference profile. Prose rather than checkboxes, because the things that decide a yes or no are not checkboxes.

Checkboxes cannot express "React acceptable if the team is strong on fundamentals" or "product company over agency", and those are exactly the criteria that settle most decisions. Prose in, prose compared — the model is good at that, and a filter is not.

The model output is never trusted raw

Every call constrains output with a JSON schema — output_config: { format: { type: 'json_schema' } } — so the response is shaped before it arrives. That is necessary but not sufficient, so each response then goes through a coerce* function at the boundary: coerceScore clamps to 0–100 and normalises the arrays, coerceSummary shapes the signals object, coerceExtraction maps the sentinels a model returns for "not stated" onto real nulls.

Those coercers are the most-tested code in the project, and they are pure functions taking a plain object — so they are unit-tested with no key, no network and no mocking of the SDK. Of 57 tests, none of them call the API. The parts that could be wrong in a way that matters are the parts that don't need it.

A score is worthless without its reasoning

A number on its own invites you to either trust it or ignore it. So a score arrives with the rationale, what matched, what is missing, and the stack it found:

The AI panel: a score, the reasoning behind it, matches and gaps, and the extracted stack.

The gaps are the useful half. "On-call expectations unclear" is not a reason to reject a posting — it is the question to ask on the call, which is a better output than a verdict. And when the rationale is visible, a bad score is obviously bad, which is the only practical way to keep trusting the good ones.

Scores are also stamped with the profile they were scored against, so editing what I want does not silently leave stale numbers looking current.

Two models, chosen per task

Scoring, extraction and summarising run on Haiku — they are high-volume, structured, and cheap by design. Drafting a cover letter runs on Sonnet, because that is prose someone will read and the difference shows. Both are environment-overridable, because pinning a model in source is how you end up on last year's model forever.

The board, sorted by fit. Scores are colour-banded, so triage is a glance rather than a read.

It has to work with no key at all

isAiEnabled() is a presence check on the API key, and every AI surface is behind it. With no key the app is a complete tracker: capture, kanban, timeline, reminders, documents, search. The AI controls simply are not rendered.

That is not politeness about optional dependencies. An app that half-works without its API key is one that fails confusingly the first time a key expires — and it made the whole AI layer testable without one, which is why the coercers have the coverage they do.

The dashboard: funnel, applications per week, and what needs chasing.

Capture, in three tiers

Paste a URL and it tries the cheap thing first: schema.org JobPosting, then OpenGraph, and only then the model on the raw text. Most postings are structured data and cost nothing to read properly. A pluggable scraper framework (RSS and generic HTML) feeds a staging inbox with dedup, so nothing auto-enters the pipeline without being looked at.

Ordering those by cost rather than by capability is the whole trick. Reaching for the model first would work, and would be slower and more expensive on the majority of inputs that never needed it.