Back to work
LLM Gateway Kit
Cost and reliability layer for Swift apps, with patterns from the Shelfie gateway · AI Infrastructure
A Swift package that sits between an app and its LLM providers. It caches paraphrased prompts and similar images, falls back across providers, and refuses a call once a daily spending cap is hit. Its patterns come from the gateway in Shelfie, which runs its own server-side version.
View source on GitHubKey features
- Prompts are matched on embedding similarity, images on a perceptual hash
- A cheap model runs first; a circuit breaker, a thrown error or a low confidence score escalates to the next tier
- The budget check runs before the request goes out, with separate caps for identified and anonymous callers
Impact
A runaway loop or a burst of concurrent calls runs out of calls at the cap, because each call reserves its estimate first. The kit has no published cost benchmark, and the README says savings depend on your traffic.
Tech stack
Swift, Swift Package Manager, GitHub Actions