gdhaliwal23

You're right that Langfuse and Helicone are retrospective — they show traces but don't roll up to "feature X costs $0.12/invocation." A middle ground that's worked: tag every API call with a feature ID during prototype runs and compute per-call cost from token counts using current pricing. Within a day or two of testing you have a real cost distribution per feature — enough to decide if the architecture is viable before building it out fully. We open-sourced the cost calculation part: `ai-cost-calc` on PyPI (https://pypi.org/project/ai-cost-calc/).

Tag every API call with a customer ID and feature name, then compute cost per call from token counts against current model pricing. That gives you per-customer cost attribution instead of just an aggregate bill. Budget caps per customer bound the risk — a runaway loop hits the cap instead of your margin. We built this as MarginDash (https://margindash.com) — the cost calculation piece is also available as a free open-source SDK if you just want that part.

I built MarginDash so teams can stop over-budget AI requests before they’re sent.

  What it does:
  - Set budgets at company, customer, and feature level
  - SDK checks budget first and blocks requests that exceed limits
  - Your app still calls OpenAI/Anthropic/etc directly (no proxy/gateway)
  - Prompts and outputs go directly between your app and the AI API provider
  - MarginDash only receives usage metadata (token counts)
  - TypeScript and Python SDKs

Flow: Checks limit you set for customer/feature -> sends AI call if within that limit -> records cost

I’d love feedback on any missing enforcement scope you’d need in production

Docs: https://margindash.com/documentation

Founder here. Gemini actually suggested I build this. I was brainstorming SaaS ideas and it pointed out the gap between AI cost tracking and revenue attribution. I dug into it, agreed, and built it.

The hardest part was the cost simulator. Comparing price-per-million-tokens across models is misleading — different models burn different amounts of tokens for the same task. So we normalize token counts to estimate what a swap would actually look like. When we recommend an alternative, we filter out anything that drops more than 10% on any benchmark or can't handle your context window size. Still improving this.

The SDK never sees your prompts or responses — just model name, token counts, and a customer ID. Limitations: simulator recommends from six vendors only, no custom/fine-tuned models, USD only.

Stack is Rails and Postgres. Happy to answer anything.

Founder here. Gemini actually suggested I build this. I was brainstorming SaaS ideas and it pointed out the gap between AI cost tracking and revenue attribution. I dug into it, agreed, and built it.

The hardest part was the cost simulator. Comparing price-per-million-tokens across models is misleading — different models burn different amounts of tokens for the same task. So we normalize token counts to estimate what a swap would actually look like. When we recommend an alternative, we filter out anything that drops more than 10% on any benchmark or can't handle your context window size. Still improving this.

The SDK never sees your prompts or responses — just model name, token counts, and a customer ID. Limitations: simulator recommends from six vendors only, no custom/fine-tuned models, USD only.

Stack is Rails and Postgres. Happy to answer anything.