ToMO by VedAGI
ToMO is a way for companies' AI apps to use fewer AI tokens and spend less on them. It checks each answer for quality before it goes out, and every response comes with a receipt. ToMO means Token Minimization Orchestrator. It is an enterprise drop-in OpenAI-compatible proxy: changing the base URL is the only required client change.
How it works
ToMO is a way for companies' AI apps to cut how many AI tokens they use and what they spend on them. It checks each answer for quality before it goes out, and every response comes with a receipt. It is also an enterprise drop-in OpenAI-compatible proxy: changing the base URL is the only required client change.
Every request runs through nine stages: auth and tenant policy, budgets and rate limits, normalize, exact cache, semantic cache, context compiler, complexity classification, cascade routing, and a receipt and audit record. Quality gates are not a separate final stage; they sit inside semantic cache, the context compiler and cascade routing. A cache hit stops early and goes straight to the receipt.
Every request follows this required pipeline. Callers cannot turn it off by default. Model tiers are shown as simple, medium and complex only. This walkthrough is an illustration: it never sends a request, calls a model, computes savings or produces a real receipt.
Checks the API key or virtual key and loads that tenant's policy.
Checks organization and team budgets. On a hard breach it returns a 429 with a receipt, or moves the request to a cheaper model tier if tenant policy allows.
Puts messages in a stable order and strips whitespace that does not matter, so the request hash stays consistent.
Looks up the normalized hash. A hit returns the cached answer with a receipt and exits early.
Embeds the user's intent. A similar past request counts as a hit only if similarity passes the gate (default 0.92). A failed gate is a miss. A hit exits early to the receipt.
Keeps the system message and final user message, converts uniform data tables to compact TOON format, drops low-relevance text, trims old turns and reduces tool output. A quality floor (for example 0.85) backs compression off if too much meaning would be lost.
Labels the request simple, medium or complex and picks the starting model tier.
Starts at the lowest allowed tier. A quality gate checks the answer; if it fails, the request moves up one tier, at most 2 times.
Writes eligible caches, returns a receipt and stores an audit record. Every override and gate decision appears in the receipt.
Receipt and audit record. A receipt can show the request id, time, tenant, cache hit and similarity, whether compression ran and its method, token counts before and after, starting tier, tier used, cascade attempts, estimated tokens saved, quality gate result (passed, escalated, overridden or exhausted), overrides, policy rules and shadow mode. It is returned in a response header and through a receipts endpoint. Evaluate savings from receipts and evaluation output. This explanatory walkthrough reports no savings result. It is not a benchmark.
Required, not optional
The pipeline is forced. Callers cannot switch off these steps as optional extras. Savings must be demonstrable from receipts and evaluation output, not asserted as a headline percentage.
Overrides exist only where tenant policy allows them: force a model tier, skip compression or skip cache. Every override and every gate decision is recorded in the receipt. No gate is skipped silently.
Caches, embeddings, logs and receipts stay in your environment by default. Tenants are isolated, full prompts are redacted from logs by default, and ToMO is designed with an air-gap install path in mind.
For teams evaluating LLM token minimization, model cascade routing or a semantic cache LLM proxy, ToMO combines these controls in one required request path.
Purpose
ToMO helps companies cut AI token use and spend. Quality checks and a receipt on every response make results reviewable. Savings must still be demonstrated from receipts and evaluation output.
ToMO is built for enterprises, not for individual home use.
Benefits
Semantic-cache research reports API-call reductions of up to 68.8 percent on repetitive query workloads. Source: arXiv 2411.05276.
Industry research from LMSYS RouteLLM reports over 85 percent lower estimated cost than GPT-4 alone on MT Bench, while keeping about 95 percent of GPT-4’s quality. Source: LMSYS, RouteLLM.
Target: ToMO is designed to cut enterprise AI model costs by 60% or more while keeping at least 90% of output quality. That is our build target, based on published caching and routing research showing 68.8% and 85% reductions.
Roadmap
Combining company-configured answer templates with reusable answers from everyday use. Ask us about availability for your deployment.
Clarification: Answer-cache reuse exists today. Company-configured templates are planned and are not available today. “Intelligent learning” means reusing business information, not training an AI model or autonomous learning.
Compliance
ToMO is designed to support EU AI Act logging duties by issuing a per-request receipt that records model choice, cache outcome, and quality-gate results so operators can keep automatic event records for at least the six-month Article 19 floor. Sources: EU AI Act Article 12 and EU AI Act Article 19.
ToMO is designed to support FinOps-style AI cost governance, where 98% of surveyed teams now manage AI spend, by enforcing budgets and quotas and attributing each call through receipts that make token and model spend allocable. Sources: FinOps Foundation data and FinOps for AI overview.
ToMO is designed to support GDPR Article 5 data-minimisation principles by compiling and reducing prompt context before model calls so only what is needed for the task is sent and retained in the request path. Source: GDPR Article 5.
Discuss ToMO for your enterprise.
VedAGI. Intelligence for Humanity.