Thanks to Amazon Bedrock, We Stopped Losing Money

Most startups building on cloud solutions divide total cloud spend by total revenue to get a blended margin. If that looks healthy, they don’t dig deeper. The problem with that calculation is that it hides how cost is distributed across your customers. Before LLMs usage patterns were easier to model and per-customer cost didn't swing much. With LLMs, that's changed. Inference cost varies enormously from one customer to the next, and you can't price intelligently if you don't know your cost to serve each individual customer.
It's still early days for Trovia, and after we launched our first batch of customers we found one of them was running about 40% of our total inference cost while paying the same rate as everyone else. At scale, we'd be spending roughly four times what we were charging them. Thanks to Amazon Bedrock, we caught this early.
Our engineering team came from AWS so watching cost per customer, not just cost in aggregate, is a habit we brought with us. The setup is simple. Here are the four takeaways for other AI builders:
Pick a provider that tracks model usage per customer
Tag every workload for the business, not just for engineering
Aggregate the data somewhere you actually look
Do this early
1. Pick a provider that tracks model usage per customer
One of the main reasons we chose Bedrock over going direct to the model provider is that Bedrock lets you create inference profiles, meaning that you can track model usage per customer. Without Bedrock, per-customer attribution is something we'd have had to build and maintain ourselves, which would’ve been costly and complicated. Instead, using AWS’s cost allocation tags, these costs show up automatically in our AWS Cost and Usage Report.
2. Tag every workload for business, not just engineering
Most teams only track user-facing query traffic. Each of our customer profiles tag the following:
Ingestion
Validation and evals
Product queries
Scoring
Agent fan-out
The true usage cost per customer isn’t just their traffic, but whatever AI usage was executed by our team on the customer’s behalf or in order to service them.
3. Aggregate the data somewhere you actually look
By using Bedrock, the customer inference profile data is already in our workflow.
The Cost and Usage Report lands in our warehouse and populates our customer table tracking per-customer cost over time, by workload. Here is where this particular customer surfaced. Since it's after-the-fact by a day or two, we pair it with AWS Budgets and anomaly detection that alert us when something unusual happens. By setting up a profile per customer, usage data like latency, errors, throttling, and token throughput, show up in CloudWatch as separate customer dashboards.
The best part is that neither of these require manual input. As long as the inference profiles are tagged with the labels you want to track, the data will show up in the dashboards you’re already using.
4. Do this early
Most teams don't think about this in the early days because cost-per-customer usage does not directly affect the end-user experience. When you are starting out, you are iterating quickly and more customer usage seems like a signal in the right direction.
This thinking becomes unsustainable once you have a few key customers. Without sufficient visibility, pricing and operations can get thorny really fast.
For pricing, once we could see the dispersion, we moved off the flat rate and repriced this customer in line with what they actually cost to serve. Now we can price every account against its real cost, something we couldn’t do before. For operations, if something goes wrong, we’ll already know exactly which customers are affected and how badly.
These recommendations are for any team building an AI product. Set up the following things early: have a way to accurately assess how much each customer costs, clear stats on usage per customer, and an easy way to track them.