applied ai engineer
I build LLM products in Python, TypeScript and Swift, and the evals that show whether they work. Founder of Shelfie. Before that, five years as a senior data analyst at Reshet 13, building its forecasting and pricing models.
open to applied ai engineer and forward deployed engineer roles · new york or remote · us permanent resident, no sponsorship needed
cost per solved incident, between settings at 77-84% with no detected accuracy difference
Under this grader, Haiku 5.5 was not detectably less accurate than Opus 5.5 at high effort, for about one twenty-fifth of the cost. Gaps under about 19 points would not have been detected. Most of the 25x is price per token: Haiku was billed about one thirtieth as much per token and used about 25% more tokens.
read the benchmark →rootcause · 525 graded runs · 7 settings
shelfie
A vision-powered kitchen app. Swift on the phone, Postgres and TypeScript behind it.
- A camera sweep builds a live kitchen inventory that drives expiry tracking, recipe matching, nutrition and grocery restocking.
- Recipes rank availability-first, by what's on the shelf and expiring soon, to cut food waste and grocery spend.
- Recognition learns from use: each household's corrections feed its priors and tune the auto-save bar.
- On-device first, cloud vision only when needed, every cloud call through one gateway with spend limits and caching.
shelfie.food →
founder · ios · pre-launch
work
Six projects carry the argument; all 13 are public, with CI in each repo or in sous. Each tile opens its build log. The figures on them are copied from each repo's own results, and the push dates are fetched from GitHub when the site builds.
rootcause
An agent that works out why an incident happened or a CI run failed. No framework underneath, and every claim about how well it works comes from graded runs you can rerun from the repo.
sous
The checks I run coding agents under. It blocks destructive commands and secret reads, fails a test run that executed nothing, and flags tests that were weakened to pass.
Ships with scoped (file locks for concurrent claude code sessions), deuce (post-merge cleanup for agent worktrees) and blast shield (shows what a risky command will change), from the same marketplace.
Litmus
Mutation testing for eval suites. It breaks the skill on purpose and reports how many broken copies the evals caught. My rootcause suite caught 0 of 12, one run per mutant.
cited
Fetches every page an LLM cites and flags claims whose words, numbers or names it can't find there, before anything is published. On its own 29-claim benchmark it caught 10 of 15 unsupported claims; swapped words still get through.
Acme Robotics raised a $12M Series A led by Sequoia
cited page, excerpt: Acme Robotics builds warehouse automation software for mid-size logistics companies.
- before
- low_match, topical 0.40, allowed to ship
- after
- unsupported: 12, 12 million, Series A, Series, Sequoia not on page
LLM Gateway Kit
A Swift package between an app and its LLM providers: a semantic cache, provider fallback, and a daily spend cap that refuses the call instead of sending it. The patterns come from Shelfie's gateway.
Signal Scout
Give it a startup's URL and it returns a shortlist of likely first customers, each tied to a public source it re-fetched. A prospect whose cited evidence is clearly missing from the page is dropped; partial matches are flagged for review.
checks that proved less than they claimed
Three pull requests where my own test or eval turned out not to test anything, and what each one does now.
The only concurrency test ran on one connection. Racing real processes let two sessions edit the same file; the PR fixes both causes.
scoped#1 →
A made-up funding round scored as a near match. Numbers and names in a claim now have to be on the page.
cited#1 →
The eval job graded hand-written outputs, so no change could fail it. It now runs the pipeline on fixtures, and three planted bugs each turn it red.
signal-scout#3 →
Each repo keeps its own record of what it does not show. Decision records, with the alternatives I rejected: rootcause, sous, cited, litmus. Limitations sections: rootcause, cited, architecture-lint, scoped, litmus, rag-generation, llm-gateway-kit, blast-shield, deuce, ny-noise.
Also on GitHub: ny-noise (where new york is loud, and next week by zip), retrieval eval (bm25, dense and hybrid on a labelled query set), architecture lint (module boundary linter with a ratchet), signal-skills (agent skills that mine podcasts and reviews). All 13 projects. Everything else is at github.com/OrenSegal.
about
I'm an applied AI engineer in Brooklyn. Most of my work is the part of an LLM system nobody demos: the eval that says how often it's right, what each correct answer costs, and the check that stops an agent from reporting work it didn't do.
Shelfie is the product. I own its architecture, the review and the checks, across the Swift app, the Postgres and TypeScript backend and CI. The open-source projects here came out of that work: measuring an incident agent, mutation-testing eval suites, checking citations against their sources, and keeping parallel agent sessions off the same file.
Before this I spent five years as a Senior Data Analyst at Reshet 13, an Israeli TV network, building the audience analytics system and the forecasting and pricing models the business ran on. How I got a pricing model adopted there. I moved to New York in 2023 and retrained in AI/ML, and in 2025 I started Shelfie. Since 2025 I have also run the data side of KashCash, a consumer transactions business: a BigQuery warehouse that loads each day with no one in the loop, a sales forecast that flags unusual days, and a watch list of the biggest accounts against their own three-month average.
hooks, skills and tools i wrote
- Hooks
- sous guard · scoped edit lock
- Skills
- signal-scout · test-audit · deuce · podcast · review-scout · ideation
- MCP servers
- rootcause telemetry · scoped
- Commands and subagents
- sous commands · sous-reviewer
- Agent loop and evals
- framework-free loop · litmus result · sous evals · cited evals
Python · TypeScript · Swift · SQL · Postgres · pgvector · SQLite · Bash · Claude API · Claude Code · Gemini API · MCP · OpenTelemetry · GitHub Actions · pandas · scikit-learn · BigQuery · Looker Studio · regression and forecasting · bootstrap intervals and power analysis
contact
Email is the fastest way to reach me. Happy to talk shop about evals, gateways, or anything to do with kitchens.