Bring your own key: how we got AI coaching costs below $2 a month

Last month a friend asked me what IntelliPrompter costs to run. I pulled up my own OpenAI usage dashboard and showed him the number: $1.74 for the previous month. Thirteen real meetings, live coaching on, the transcript going through our full RAG pipeline. One dollar and seventy four cents.

He assumed I was lying, or that I was running some stripped-down demo mode. I wasn't. That's the actual bill. The comparable tools in this space charge $30 a month, flat. I want to explain why we can charge less and how we got there, because the answer is more interesting than "we found a cheaper vendor."

If you're a meeting AI company, your biggest variable cost is the AI itself. Every meeting is thousands of transcript tokens flowing into a model, often several times a minute. If you're routing all of it to GPT-4 or Claude Opus, your per-user API bill lands somewhere between $8 and $20 a month for an active user. You charge $30, you keep the rest, you grow.

That math works. It also forces decisions I don't like. You have to batch calls to save money, which hurts latency. You have to cap features for lighter users to subsidize the heavy ones. And you own the API key, which means you're the one with a copy of every transcript sitting in a vendor's logs. The business model dictates the architecture.

I wanted a different business model, so we built a different architecture.

You sign up, you plug in your own OpenAI or Anthropic or Gemini API key, and the app routes calls directly from your machine to the provider. I never see your key. I never see your transcripts. You pay the provider pennies, you pay us a modest subscription for the app itself.

I think this is the honest way to do it. Your conversations are the most sensitive data you generate at work. The fewer parties sitting in the middle of them, the better. And if you're already paying the model vendor for other things, there's no reason to pay a second markup on top.

Here's the tradeoff, stated plainly: this isn't "sign up and go." You need an API key. Getting one takes about four minutes on OpenAI's dashboard. For a lot of people that's a non-starter and that's fine. We're not for everyone. If you'd rather pay a flat $30 and skip the setup, Fathom and Fireflies both do that well.

Bring-your-own-key only works if your-key doesn't get hammered. If I naively forwarded every transcript chunk to GPT-4, you'd be looking at a $25 bill at the end of the month and you'd (rightly) quit. So the real engineering problem was: how do we get the actual per-meeting cost down to a few cents?

Three things, in rough order of impact.

Smart routing by task. Not every AI call needs a flagship model. A meeting generates a lot of small, boring questions: "is this chunk of transcript on-topic or chit-chat?" "does this look like a question the user should answer?" "classify this segment." Those run on gpt-4o-mini or local Llama via Ollama and cost essentially nothing. The hard reasoning passes (should we interrupt with a coaching suggestion, and if so what's the single most useful thing to say right now) go to Claude Sonnet or GPT-4o. We route per task, not per app. The gateway that does this lives at src/main/ai/provider-selector.ts if you're curious.

Prompt caching. Our coaching prompts have a lot of fixed structure: the system prompt, your preferred frameworks, the RAG context from your personal knowledge base. That part doesn't change meeting-to-meeting. Anthropic and OpenAI both now charge roughly a tenth of the normal price for cached prefix tokens. We structure every prompt so the stable part comes first, and the cache hit rate across a meeting runs north of 85%. That alone cut our per-call cost by about 40%.

Local-first for high-volume work. If you've got Ollama installed, classification and summarization both default to local. These are the two tasks that fire most often during a live meeting, so moving them off the paid API is a bigger deal than it sounds. On my laptop, with Llama 3.1 8B running locally, roughly 70% of the AI calls in a meeting never touch a paid provider. The cloud flagship only shows up for the moments that actually need it.

Here's what the math looked like for me in September. Ten meetings, averaging 42 minutes each. Coaching on the whole time. I ran local Ollama for classification and let the cloud handle the reasoning:

• Classification + topic detection (local Ollama): $0.00
• Summarization + segment tagging (gpt-4o-mini): $0.31
• Coaching suggestions (Claude Sonnet, cached): $1.47
• Rehearse / structured topic prep (Sonnet, uncached): $0.02

Total: $1.80 on my OpenAI + Anthropic bills combined. Add our app subscription and the whole stack cost me under $20 that month. Comparable incumbents would have been $30 to $40 for the same volume, with my transcripts sitting in their logs.

Your mileage will vary. Heavier users with longer meetings and no local Ollama will probably see $4 to $6 on the provider side. If you only use cloud flagships and skip caching, you'll land closer to $10. The number is still well under the flat-$30 competitors, and you're the one holding the key.

A few things I'll own up to.

First, the setup friction is real. Four minutes on OpenAI's dashboard is four minutes more than most consumer products ask for, and we lose some signups because of it. I'm making the bet that the people this product is for will tolerate the setup once they understand the privacy and cost tradeoff. If I'm wrong, we'll have to add a managed-key tier later.

Second, routing complexity is a tax we pay. Every time a new model drops, I have to re-benchmark which tasks it should take over. The router has gotten more opinionated over time, not less. That's fine for us, but it means I can't promise the exact numbers above will hold forever. They'll move. Hopefully down.

Third, local Ollama is not free, it's just not metered. It costs you RAM and some battery. If you're on an 8GB machine, you probably want to leave it off and eat the small cloud bill for classification. We default to cloud if we don't detect Ollama running.

If a $2 month on your own API key sounds more honest than a $30 month on someone else's, we're probably a fit. The full pricing breakdown is at intelliprompter.com/pricing, including the per-meeting cost calculator so you can plug in your own usage pattern and see what your bill would look like before you commit.

I'd rather build a product you can audit than one you have to trust. The math is on the page. Come kick the tires.

Jeff