Workload-aware LLM admission control

Keep batch LLM work from starving user-facing traffic.

MoFlux makes capacity a workload-aware decision before execution. Tyr protects configured interactive capacity and sheds lower-priority work first when the provider gets tight. In eight counterbalanced live OpenAI overload runs, MoFlux preserved 99.4% of interactive requests versus 71.9% with an undifferentiated static cap.

Live OpenAI overload: 99.4% interactive success through Tyr versus 71.9% with the static cap; MoFlux led in all 8 counterbalanced seeds.MoFlux shed more batch work to preserve interactive capacity. No latency-win claim.

Independent founder project. Seeking one first design partner.
Admission under contention Local
InteractiveUser-facing requests
8 protected
Batch / backgroundDelay-tolerant work
4 protected
UpstreamLLM provider
Interactive capacity stays protected.When slots run out, lower-priority work is shed first.
Fleet authorityLatchflo stays off the request path
Control plane

Overload begins before the 429

By the time an upstream rejects traffic, your service may already be carrying too many large requests, retries, open streams, and waiting callers.

Replica-local limits multiply

A limit that is safe for one process can become unsafe when every replica independently assumes it owns the same shared capacity.

Queues can turn pressure into latency

Waiting is not free. Under sustained load, queues can trade obvious rejection for timeouts, retry amplification, memory pressure, and a less predictable user experience.

Measured progress

Live overload proof.
Changing fleet.

Two different questions now have direct evidence. The live sweep tests whether protected concurrency changes who succeeds under real provider scarcity. The membership run tests whether Latchflo converges cleanly as the fleet changes.

Counterbalanced live OpenAI overload
99.4%

Protected interactive requests completed under overload.

Static concurrency completed 71.9%. MoFlux led in all 8 matched seeds, with a median paired advantage of +25.5 points. Both controlled arms kept provider 429s at zero.

Open reviewed OpenAI evidence
Current-release fleet proof
5/5

Dynamic membership runs converged cleanly.

Zero heartbeat revision churn, duplicate members, stale members after convergence, or revision regressions. Replacement joins propagated in 11.1ms p50; timeout removal was 1.40s p50.

This benchmark validates Latchflo membership and routing-topology convergence. It does not yet prove uninterrupted routed request traffic while peers churn.
Market position

Not another gateway.
Not just another limiter.

MoFlux is a specialized safety layer for the gap between “this request is allowed” and “this request is safe to start right now.” It is intended to integrate with an existing gateway and observability stack.

Adjacent category

AI gateway

Primary job
Routes requests and often owns provider access, authentication, observability, and broad API policy.
Boundary
That is a wider job than admission control, and many teams already have this layer.
Relationship
MoFlux is designed to integrate with it rather than replace it.
Adjacent category

RPM / TPM rate limit

Primary job
Controls request or token volume over a time window.
Boundary
A request can fit the current rate window and still be too large or too concurrent to start safely now.
Relationship
MoFlux adds an in-flight capacity decision before execution.
Adjacent category

Generic concurrency limiter

Primary job
Caps local in-flight work, sometimes with a queue.
Boundary
It protects the upstream, but an undifferentiated cap does not know whether a slot is being used by a waiting user or delay-tolerant background work.
Relationship
MoFlux adds protected workload classes, LLM-sized reservations, fleet authority, and progressive accounting.
Defensible category

A focused capacity firewall for LLM traffic.

Pre-execution reservation, local admission, protected service-class capacity, fleet-level grants, and decision provenance—without a control-plane lookup on every request.

Operational value

Built around the failure modes platform teams actually see

The goal is narrow: make capacity an explicit pre-execution decision, then leave ingress, provider governance, and broader gateway policy where they already live.

Protect scarce execution capacity

Admit, reject, or observe a request before it consumes provider-facing concurrency and expected in-flight token headroom.

Coordinate a fleet

Turn one capacity envelope into expiring local authority without placing the control plane on every provider request.

Protect service classes

Map trusted identities to a bounded class table, apply hard ceilings and protected concurrency/token floors, and expose protected versus borrowed usage.

Explain the decision

Surface rejection reason, policy revision, admission identity, grant ID, controller epoch, and bounded metrics for tuning and incident review.

Measured evidence

Live-provider overload.
Distributed-system proof.

moflux-bench v0.29.0 is public under Apache-2.0. The reviewed eight-seed OpenAI sweep now carries the protected-concurrency overload claim. The separately published August 26 synthetic ladder remains the evidence for token pressure, adaptive lending, and coordinator-distance behavior. Keeping those claims separate prevents one benchmark from pretending to prove the whole platform.

Live OpenAI 99.4% interactive · Tyr 0.28.0Membership 5/5 · Latchflo 0.13.0Synthetic overload 8 seeds × 4 rungs · Tyr 0.27.0 / Latchflo 0.12.4adaptive-28-4 · 28/4 protected pool slotsAdaptive proof 32/32 · no recorded applied over-allocationBrowse public evidence
Protected interactive success99.4%

795/800 interactive requests completed through Tyr under live OpenAI overload.

Static-cap interactive success71.9%

575/800 completed with the same 36-request global concurrency ceiling and no workload protection.

Paired advantage+25.5 pts

Median MoFlux advantage across eight matched seeds; every seed was positive (17–58 points).

Controlled-arm provider 429s0

Both static and MoFlux kept provider 429s at zero; direct OpenAI produced 280 under the same offered workload.

What changed under scarcity+27.5 pts

pooled interactive success versus the static cap

Consistency8/8

counterbalanced seeds favored MoFlux on interactive success

Tradeoff-8.48 pts

median paired batch-success delta versus static

Scope36 slots

same global concurrency ceiling; Tyr protected 8 interactive / 4 batch

Synthetic ladder · separate evidence

Four approaches under controlled overload.

The synthetic ladder is still useful for token pressure, Redis coordination distance, and adaptive fleet behavior. It is no longer the primary proof that protected concurrency changes interactive outcomes under a real provider.

Published coordinator ladder, 0ms rung, median across eight matched seeds; smaller text is the observed minimum–maximum range.
MetricNo controlStatic capRedisMoFlux
Interactive successCompleted interactive logical requests81.1% · 75.8%–85.3%93.7% · 87.3%–98.7%92.7% · 88.3%–95.7%
Interactive goodputSuccessful interactive requests per second4.79 · 4.38–5.135.52 · 5.07–5.915.50 · 4.89–6.04
Interactive p50End-to-end latency among successful requests7.39s · 6.53s–8.51s5.77s · 5.07s–7.09s4.22s · 2.89s–5.64s
Interactive p95End-to-end latency among successful requests14.87s · 12.70s–17.67s13.23s · 11.75s–15.14s10.30s · 7.67s–12.89s
TTFT p50Time to first streamed output among successful requests2.96s · 1.27s–3.95s1.93s · 1.11s–2.53s0.79s · 0.56s–1.13s
TTFT p95Time to first streamed output among successful requests7.23s · 6.44s–7.79s5.77s · 4.84s–5.98s2.92s · 2.14s–3.44s
Retry amplificationAttempts per logical interactive request1.8281.4571.478
Batch successCompleted batch requests; the adaptive policy guarantees liveness, not maximum batch throughput55.1% · 50.0%–69.8%23.0% · 17.2%–30.2%17.4% · 11.4%–23.3%
Local rejectsAttempt-level work shed before provider execution0295 · 196–443306.5 · 231–420
Token-budget rejectsRequest size was a binding admission constraint0086 · 13–123
Upstream 429sProvider-side rejection after capacity was spent375 · 241–47100
Peak provider occupancyActive requests against the 32-slot envelope32/3232/3232/32 · 30/32–32/32
Request-size-aware admission

Token pressure changed real admission decisions on every 0ms MoFlux seed.

Interactive request sizes varied by roughly 24.9×–32.0×. A median 50.6% of MoFlux capacity-bound rejects were token-bound, with 40.5 median token-budget rejects. No control and the static cap do not make request-size-aware token admission decisions. MoFlux and Redis exercised token-aware admission on all eight 0ms seeds.

What the alternatives taught us

The current result supports a differentiated product, not a universal winner.

Static caps and Redis both work. The benchmark is useful precisely because it shows where MoFlux adds value and where another control is currently better.

MoFlux completed more protected interactive work than Redis

At the 0ms rung, MoFlux improved paired interactive completion by +4.0 pts and goodput by +4.2%. Across all four rungs, its median interactive-success advantage stayed between 3.42 and 4.72 percentage points.

MoFlux shed fewer attempts and retried less

At 0ms, median local rejects were 264 for MoFlux versus 306.5 for Redis — about 13.9% fewer attempt-level local rejects. Retry amplification was 1.339× versus 1.478×. Local rejects are attempt-level, so completion remains the stronger reliability outcome.

Redis still delivered faster successful-request latency

At the 0ms rung, Redis median TTFT p95 was 2.92s versus 5.46s for MoFlux, and Redis median end-to-end p95 was 10.30s versus 11.54s. These percentiles include successful logical requests only, so they should be read together with completion rate.

Redis also served more batch work in this policy

At 0ms, Redis median batch success was 17.4% versus 9.2% for MoFlux. MoFlux restored and served the protected batch floor on every seed, but this adaptive policy prioritizes interactive reliability over maximizing batch throughput.

Coordinator distance · 8 seeds × 4 rungs

Per-request coordination pays for distance on the request path.

The published ladder injects 0, 5, 20, and 50ms into the Redis coordinator path. Redis admitted-decision time rose 1.0141ms for each 1ms of injected distance, with all 8 seeds degrading. Tyr's admitted-decision slope was -0.0001ms/ms with a 95% interval from -0.0005 to +0.0002. The benchmark directly measures Tyr synchronous decisionDuration separately from queue wait. The near-zero MoFlux decision slope supports coordinator-distance independence for that measured local decision path; it does not mean end-to-end TTFT is zero-cost or invariant.

Redis admitted-decision slope

1.014 ms/ms

Median paired slope across eight seeds; 95% interval 1.00921.0183.

MoFlux admitted-decision slope

-0.0001 ms/ms

Measured Tyr decisionDuration excluding queue wait; 95% interval -0.0005 to +0.0002.

Interactive reliability

+3.42 to +4.72 pts

MoFlux median interactive-success advantage over Redis across the four tested coordinator-distance rungs.

Recorded adaptive proof

32/32 rung-seeds

Every rung recorded 8/8 passing seeds for the configured interactive target, batch-liveness floor, zero-upstream-429 requirement, and handoff safety checks.

Pool-level demand-aware lending + reclaim

The current stack proves the handoff, not just the lending event.

All eight 0ms seeds observed and committed the reclaim handoff, restored the protected batch guarantee in controller and data-plane evidence, proved commit-before-admission with exact Tyr admission provenance, and recorded no applied over-allocation. Interactive traffic actually consumed borrowed slots on six of eight seeds. Controller-level lending was observed on all eight, so some seeds made capacity available without needing to occupy it above the protected floor. The median first batch admission window was 0.36–0.68s after commit. The roughly 14.28s commit-to-response-headers interval includes downstream simulated service for the large batch prompt and should not be read as capacity-control recovery latency.

Safe handoff proof

8/8 seeds

Every seed committed in safe event order, before batch admission and fallback lease expiry, with no applied over-allocation.

Drain → commit

0.81s median

Where demand timing was available, drain began immediately; acknowledgment dominated the handoff, while acknowledgment-to-commit was 0.01s at the median.

Borrowed occupancy

6/8 seeds

Interactive traffic actually consumed borrowed capacity on 6 of 8 seeds, reaching as many as 4 extra concurrency slots.

Post-commit admission bound

0.36s0.68s

Median lower/upper bound for first batch admission after commit. The roughly 14.28s commit-to-response-headers interval includes downstream simulated service for the large batch prompt—queue/prefill/TTFT-equivalent work—not capacity-control recovery.

Progressive reservation reconciliation

Conservative reservation does not have to stay stranded until completion.

MoFlux recovered a median 60.6% of reserved tokens net of overruns, and 83.4% of refunded tokens returned while streams were still active. Median overrun was 133.5 tokens. This is earlier reuse of existing headroom, not newly created capacity.

Net reservation recovery

60.6%

Median across eight matched 0ms seeds.

Released before completion

83.4%

Share of refunded tokens returned mid-stream.

Protected admission classes · prior five-seed proof

The isolation mechanism has direct proof, but not yet on the newest runtime.

This earlier run used Tyr 0.22.0 and Latchflo 0.8.0. Across five matched seeds, the protected policy was exercised successfully, both classes continued to receive service under contention, and every arm recorded zero upstream 429s. Against class ceilings alone, premium median success moved from 72.5% to 75.4% and contended goodput from 2.76 to 2.88 requests/s. Premium TTFT was effectively unchanged versus ceilings-only.

Proof gate

5/5 seeds passed

Same trace, both classes observed and served under contention, protected policy exercised, minimum noisy-class progress met, and zero upstream 429s in every arm.

Premium vs ceilings

+2.93 pts · +4.3% goodput

Protected floors improved premium success and goodput in four of five seeds, but the effect is incremental rather than dramatic.

TTFT vs ceilings

1.003× · +0.3%

Median protected/ceilings premium p95 TTFT ratio and change. Protected TTFT was better in only 2/5 seeds, so the benchmark does not establish a TTFT advantage over ceilings-only.

Noisy-class throughput

+0.67 pts success · 16 vs 18 completions

Noisy median success edged up from 37.5% to 38.2%, but median contended completions fell. The floor guarantees progress; this run does not show a noisy-class throughput win.

The shared arm is an important counterweight: it reached 98.8% median premium completion and 4.52 premium contended goodput, both well above protected. Protected instead cut premium contended p95 TTFT to 0.535× shared at the median — about 46.5% lower — while reserving capacity for the noisy class. That is an isolation trade-off, not a universal performance win.

What the site will not hide

The interesting part is the boundary of the claim.

The newest evidence strengthens the safety claim substantially: across the published four-rung ladder, all 32 rung-seed combinations passed the recorded adaptive proof, and the 0ms evidence includes exact Tyr admission provenance for safe reclaim. Current Tyr/Latchflo releases also support opt-in demand-aware class-floor lending, but the class-performance numbers below are still from the earlier static protected-floor proof and should not be relabeled as current-stack performance evidence.

This is a workload-specific benchmark, not a universal ranking

The provider is simulated and the policy intentionally prioritizes interactive traffic. The result supports claims about this tested overload regime, not a blanket statement that MoFlux is faster or more reliable in every workload.

Static caps remain a credible low-complexity control

At 0ms, static caps reached 93.7% median interactive success with zero upstream 429s. MoFlux reached 97.4% and adds token-aware admission, adaptive lending, fleet authority, provenance, and coordinator-independent local decisions — at the cost of more system complexity.

Redis wins responsiveness in this comparison

Redis had substantially lower median successful-request TTFT at 0ms and lower median end-to-end latency. MoFlux's value in this run is higher interactive completion, lower retry amplification, fewer attempt-level local rejects, and a request path that does not inherit coordinator RTT.

Batch fairness is bounded, not maximized

All 32 rung-seed combinations met the configured batch-liveness floor, but at 0ms MoFlux median batch success was 9.2% versus 17.4% for Redis. The site therefore describes the batch guarantee as non-starvation/liveness, not superior batch throughput.

Queue wait is measured separately from local decision time

The published Tyr 0.27.0 ladder exports synchronous admission decision time separately from queue wait. The coordinator-distance claim uses decisionDuration excluding queue wait; end-to-end TTFT still includes retry/backoff, queueing when exercised, provider prefill, and first-token service.

Protected-class proof is from the previous stack

Protected class floors remain shipped, but the five-seed class proof cited here was generated with Tyr 0.22.0 and Latchflo 0.8.0. It is prior isolation evidence, not a current-runtime performance rerun.

Reservation recovery is reuse, not new capacity

MoFlux returned a median 60.6% of reserved tokens net of overruns, with 83.4% of refunded tokens released while streams were still active.

Reproducibility and limits

The harness is public. Licensed product evidence is clearly labeled.

The benchmark harness is at v0.29.0. The August 26 adaptive coordinator ladder cited here is published under video-seed-sweep in the public results tree. Reproducing the MoFlux arms still requires licensed Tyr and Latchflo images. The older protected-class performance proof remains separately disclosed because it predates the current runtime.

What you can check

  • publicmoflux-bench v0.29.0 source and verification suite
  • publicReviewed counterbalanced live OpenAI overload sweep
  • publicDynamic fleet-membership benchmark driver
  • runnableProvider simulator plus no-control, static-cap, and Redis arms
  • publicReviewed August 26 overload ladder at results/video-seed-sweep.json
  • licensed images requiredTyr + Latchflo benchmark arms
  • not committedProtected-class proof cited on this page (older stack)

Inspect the published reviewed result or clone mofluxhq/moflux-bench and run npm run verify. The current ladder driver is npm run demo:coordinator:adaptive; the protected-class driver is npm run demo:classes. Licensed Tyr and Latchflo images are required to reproduce the MoFlux arm.

The live overload result is real-provider evidence

The openai-live-overload-sweep artifact sends real requests to OpenAI under deliberate request-rate overload. Arm order is counterbalanced across eight seeds and each arm starts behind the same rate-limit headroom gate. The older August 26 Universal Scalability Law ladder remains separate synthetic evidence for coordinator distance, token pressure, and adaptive fleet behavior.

The live OpenAI claim is deliberately narrow

MoFlux completed 795/800 protected interactive requests versus 575/800 under the static cap, with a positive paired advantage in every seed. It shed more batch work to do so, and Single-node Tyr protected concurrency only. This sweep does not exercise Tyr in-flight token-budget admission or Latchflo fleet coordination.

Dynamic membership proof stops at topology convergence

5/5 membership runs passed with no revision churn, duplicates, stale members after convergence, or revision regressions. This benchmark validates Latchflo membership and routing-topology convergence. It does not yet prove uninterrupted routed request traffic while peers churn.

The current ladder is reviewed public evidence

The August 26 ladder 20260826T204430Z was promoted under video-seed-sweep. The site links directly to the reviewed evidence in the public moflux-bench results tree rather than relying on an unpublished local corpus.

Runtime provenance is current-stack evidence

The published ladder is associated with moflux-bench 0.27.0, Tyr 0.27.0, Latchflo 0.12.4, async-bulkhead-llm 3.16.0, and async-bulkhead-ts 1.0.1.

Coordinator distance has direct admission timing

Redis admitted decisions rose 1.0141ms per 1ms of injected coordinator distance. Tyr admitted decisions had a median slope of -0.0001ms/ms with a 95% interval of -0.0005 to +0.0002. This measures local decisionDuration excluding queue wait, not end-to-end response latency.

Admission timing is separated from response timing

At 0ms, the safe handoff took a median 0.81s. First batch admission was bounded to 0.36s–0.68s after commit at the median, while response headers arrived 14.28s after commit because the large batch request still had downstream simulated service to do.

Protected-class evidence is older than the ladder

The reviewed npm run demo:classes run used Tyr 0.22.0 and Latchflo 0.8.0. It remains useful correctness/isolation evidence, but it is not presented as a current-stack performance measurement.

Peer-routing causality is not isolated here

Peer routing ships in the current Tyr release. The published ladder measures the full MoFlux arm with routing enabled, but it does not isolate routing as a standalone causal effect.

Current architecture

Fleet capacity authority.
Local request decisions.

Latchflo allocates bounded, expiring authority for physical pools and fixed admission classes, and now derives versioned routing topology from live agent membership. Tyr atomically applies complete desired state, reports bounded pool and per-class demand, acknowledges restrictive physical or class grants with bounded occupancy, and publishes fresh post-ACK evidence; Latchflo keeps staged authority unavailable until that evidence proves the drain is safe. Tyr still makes the final admission decision locally without a Latchflo round trip on every provider call.

MoFlux current-state architectureLatchflo control plane 0.13.0 owns versioned, expiring fleet authority for concurrency and in-flight token capacity. It distributes complete desired-state grants, protected admission-class floors and hard ceilings, supports opt-in per-class demand-aware floor lending, coordinates acknowledged physical capacity-group reclaim, and now also coordinates class-only acknowledged handoff for returning protected floors and restrictive class transitions. Authority expansion remains staged until drain installation plus fresh post-ack pool or class occupancy proves safe attrition; lease expiry is the conservative fallback and active work is not preempted. Tyr admission controller 0.28.0 atomically applies higher-revision desired-state batches, reports bounded pool and admission-class demand, includes bounded pool/class occupancy in applied acknowledgements, publishes immediate post-ack evidence heartbeats for restrictive transitions, fails managed pools closed when authority expires, and makes the final request admission decision locally. Latchflo now derives a versioned routing-topology snapshot from live agent membership and includes that fleet state in managed desired state. Tyr can route a request once using fresh request-specific concurrency, token and class headroom; the destination Tyr performs authoritative admission. Current public membership evidence proves topology convergence, not uninterrupted routed traffic during churn. Latchflo does not receive prompts or caller identity. Client applications call Tyr on the Anthropic Messages or OpenAI Chat Completions-compatible routes; Latchflo remains off the live provider request path.Latchflo control planev0.13.0 · capacity + live membership · acknowledged handoffFleet + class policyPool guarantees + adaptive class floorsLive membership → versioned topologyHandoff coordinatorPool/class drain · applied ACKFresh occupancy proof → commitDesired state + topology ↓ACK + pool/class demand + occupancy ↑Tyr admission controllersv0.28.0 · local admission + managed topologyAtomic apply · fail closed on lease expiryPool/class demand + post-shrink proof heartbeatInstance Alocal admit / routeInstance Blocal admit / routeInstance Nlocal admit / routeClient appsagents, jobsAnthropicMessages APIOpenAIChat CompletionsLatchflo receives capacity, demand, and endpoint metadataPrompts and caller identity stay out of the control planePeer routing stays Tyr-local; no control-plane lookup per request
Current state: Latchflo 0.13.0 owns versioned pool/class authority, tracks live membership, publishes routing-topology snapshots, coordinates acknowledged handoff, and retries pending reclaim when fresh telemetry makes it actionable. Tyr 0.28.0 keeps the request admission decision local and destination-authoritative.

Versioned desired state

Controller epoch, revision, grant ID, expiration, complete pool/class limits, apply acknowledgement, and immutable provenance make installed authority attributable.

Proof before reuse

Pool/class drain → applied ACK → fresh post-ACK occupancy proof → commit staged authority. A restrictive-grant ACK proves installation; fresh post-ACK pool or class occupancy proves the transferred authority is actually safe to reuse.

Dynamic fleet membership

Live agent registration and timeout drive a versioned routing-topology snapshot without placing Latchflo on the provider request path. The current membership proof validates topology convergence; routed traffic continuity during churn remains a separate benchmark target.

Fail-closed managed mode

Valid leases survive transient control-plane failures until expiration; when authority expires, Tyr applies the reserved zero-capacity revision rather than admitting indefinitely.

Integration position

Keep the gateway you already have.

MoFlux can sit on a selected provider path while your existing platform continues to own ingress, broader authentication and authorization, secrets, routing, and AI governance. Tyr can optionally verify JWT/JWKS identity when admission classes or role-based policy need it.

Inside one Tyr admission controllerTyr admission controller 0.28.0 contains JWT and JWKS identity with bounded role and admission-class selection; an admission engine built on async-bulkhead-llm 3.16.0; first-class Latchflo managed mode; and optional one-hop peer routing. The admission engine enforces physical-pool and admission-class concurrency and in-flight token limits, protected floors and hard ceilings, shared-capacity borrowing, exact request reservations, progressive streaming reconciliation, observe or enforce mode, and bounded retry hints. Managed mode validates complete desired state, applies higher-revision batches atomically, reports pool and admission-class demand, acknowledges applied grants, includes bounded pool and admission-class occupancy in applied acknowledgements, publishes fresh post-ack evidence for physical and restrictive class transitions, preserves a valid lease through transient control-plane failures, and fails managed pools closed on expiry. After a physical or class drain Tyr emits an immediate evidence heartbeat and can temporarily use a bounded 500 millisecond cadence until an exact sent snapshot proves the transition safe. Managed state can carry versioned routing topology derived from Latchflo membership. Peer routing remains off path, forwards at most once using authenticated internal headers, and leaves final admission to the destination Tyr. Current public evidence validates topology convergence separately from routed traffic continuity during churn. /healthz and /readyz are public probes; /stats and /metrics require operator authority. Caller identity is not forwarded upstream, while the caller's provider credential is preserved.Tyr admission controller v0.28.0Local request decision; managed fleet authority arrives off pathIdentity + class selectionJWT / JWKS · configurable headerRoles + bounded admission classesIdentity stays Tyr-local for LatchfloDefault class when no rule matchesAdmission engineasync-bulkhead-llm v3.16.0Pool + class concurrency / token headroomProtected floors · hard ceilings · borrowingProgressive reconciliation · admit / rejectLatchflo managed modeComplete desired state + routing topologyApplied ACK + bounded pool/class occupancyPool + per-class demand heartbeatsPool/class post-shrink proof · fail closed on expiryActive work shrinks by attrition, never preemptedCapacity-aware peer routingVersioned membership topology + private secretFresh request-specific pool + class headroomForward at most once; no automatic retryDestination Tyr performs authoritative admissionPublic proof validates topology convergencePublic probes/healthz /readyzOperator authority/stats /metricsIdentity stripped before provider forwarding; caller provider credential preserved
Inside Tyr 0.28.0: identity-aware admission classes, protected floors and hard ceilings, bounded pool/per-class demand, atomic Latchflo desired-state application, applied ACK with bounded pool/class occupancy plus immediate post-ACK proof heartbeats for restrictive transitions, progressive reconciliation, and one-hop peer routing all stay in the data plane. Latchflo 0.13.0 now publishes versioned routing topology from live membership; routed churn remains a separate end-to-end proof target.
Current implementation

A substantial technical core,
still before production v1.

The current releases are credible for technical evaluation, architecture review, and a tightly controlled design-partner pilot. Tyr and Latchflo are still pre-1.0 and should not be sold as a broadly hardened enterprise platform yet.

Admission data planev0.28.0

Tyr

Makes local token-and-concurrency admission decisions, reports bounded pool and per-class demand, atomically applies managed desired state, acknowledges restrictive physical and class grants with bounded occupancy, publishes ordered post-ACK proof heartbeats, enforces protected class floors and hard ceilings, progressively reconciles streaming usage, and can route once toward a peer with better request-specific headroom.

Capacity control planev0.13.0

Latchflo

Allocates versioned, expiring fleet grants, protected class floors and hard ceilings; tracks live agent membership and publishes versioned routing topology; coordinates acknowledged physical and class-only drain → applied ACK → fresh occupancy proof → commit; and retriggers pending reclaim when fresh member telemetry makes the transition provable — without entering the live provider request path.

LLM-aware admission enginev3.16.0

async-bulkhead-llm

Provides token estimation, exact reservation previews, protected admission-class floors, shared-capacity borrowing, progressive reconciliation, observe mode, and atomic versioned limit updates.

Concurrency primitivev1.0.2

async-bulkhead-ts

Provides the lower-level fail-fast and bounded-queue admission foundation used beneath the LLM-aware policy layer.

Current boundaries

Where the current boundary is

The project is moving quickly, but these constraints remain part of the architecture discussion from the start.

Single-controller control plane

Latchflo currently persists to SQLite behind one controller. PostgreSQL-backed HA and hardened multi-region operation remain roadmap work.

Founder-stage operations

There is no 24/7 support organization or enterprise SLA today. A pilot is direct, technical, and intentionally bounded.

Capacity policy, not billing

Token budgets govern expected in-flight exposure. They are not monthly spend limits, invoices, or billing-grade accounting.

Focused protocol and identity support

Tyr currently targets OpenAI Chat Completions and Anthropic Messages and supports direct JWT/JWKS identity. Broader provider protocols and dynamic authorization policy are not yet generalized.

Membership convergence is proven; routed churn is next

Latchflo 0.13.0 now publishes versioned routing topology from live membership. The current 5-run proof validates clean remove/join convergence with no duplicate or stale members, but uninterrupted routed request traffic during peer churn remains a separate benchmark target.

Adaptive class lending now has acknowledged pre-expiry restoration

Latchflo 0.13.0 with Tyr 0.28.0 can lend a fully observed idle class's protected floor while keeping hard ceilings fixed, then restore that floor before the old lease expires when ordered applied-ACK plus fresh class-occupancy evidence proves safe attrition. Older/incapable agents or missing evidence fall back to lease expiry; active work is never preempted.

Batch fairness remains unfinished

At the 0ms rung, the adaptive policy restored the batch pool guarantee safely on all 8 seeds, and first batch admission was bounded to 0.36s–0.68s after commit at the median. Batch completion was still only 9.2% versus 17.4% for Redis, so the current policy proves liveness rather than superior batch throughput.

Coordinator distance now has direct evidence

Across eight matched seeds at 0/5/20/50ms, Redis admitted-decision time rose 1.014ms for each 1ms of injected coordinator distance. Tyr's measured local admitted-decision slope was -0.0001ms/ms with a 95% interval spanning zero; queue wait and end-to-end response time remain separate metrics.

First design partner

Bring one real capacity problem. Measure it together.

The strongest pilot has several AI-service replicas, a shared constrained provider or inference pool, and a workload where interactive traffic, batch jobs, large prompts, or retries compete for capacity. The goal is not a broad rollout; it is a defensible answer to one capacity question.

A bounded non-critical or canary workload Access to latency, rejection, and usage telemetry An engineering owner who can review behavior directly
01

Choose one bounded workload

Pick a service, model route, environment, or batch flow where overload behavior is visible and reversible.

02

Start in observe mode

Exercise the real estimator and policy path without turning capacity-related decisions into hard rejection yet.

03

Compare policy with traffic

Review prospective rejects, latency, completion, token reservation accuracy, class behavior, and provider-throttling exposure.

04

Enforce only if the evidence supports it

Move to a narrow enforcement canary only when the observed trade-offs and operational model are acceptable to your team.

Commercial status

Founder-led, direct, and early.

MoFlux is currently built and supported by one founder and is not yet an incorporated company.

Before the first paid client engagement, I will form an LLC and obtain any business licences required for the engagement.

I am looking for one technically engaged design partner with a real AI-capacity problem and a bounded pilot workload.

Direct contactmofluxhq@gmail.com

No sales team, contact form, or automated funnel. Your email goes directly to the person building the system.

Start the conversation