ttft

Zero-config local LLM optimization for Ollama, LM Studio, and Apple Silicon MLX. Reduces TTFT by 40%, wall time for local agents by 46%, and RAM usage by 3x.

批量测试符合 OpenAI 协议和 Anthropic 协议的 AI 模型性能指标。支持 TTFT（首字节时间）、TPS（吞吐量）、网络延迟等关键性能指标的测量，提供多模型对比测试和详细的性能报告生成功能。

A Go CLI tool to benchmark local LLMs via Ollama, measuring Time To First Token (TTFT) and throughput on your specific hardware.

High-precision edge and on-device LLM inference profiler calculating TTFT, TPOT, tokens/second throughput, and percentile jitter distributions (P50/P90/P99).

OpenCode TUI plugin that shows generation speed (tok/s), cache hit rate, and average TTFT in the sidebar

High-precision edge and on-device LLM inference profiler calculating TTFT, TPOT, tokens/second throughput, and percentile jitter distributions (P50/P90/P99).

Pi Coding Agent extension for real-time execution turns, steps, LLM/tool durations, TTFT, and TPS in the footer status bar

LLM inference benchmarking toolkit. Measure TTFT, inter-token latency, throughput, and P50–P99 across concurrency levels.

The only voice agent context manager with a TTFT feedback loop

Linux kernel and systems fast path for LLM inference: eBPF tracing, runtime hints, cgroups, NUMA/GPU locality, KV-cache memory policies, TTFT boost, and experimental kernel primitives.

An asynchronous, neuro-symbolic VLA (Vision-Language-Action) orchestration stack for edge autonomy. Fuses probabilistic Qwen2-VL visual reasoning and faster-whisper ASR with deterministic PX4/MAVSDK flight-control loops and HSV color guardrails.

Mesure les métriques d'inférence LLM (TTFT, TPOT, débit, coût, VRAM) sur n'importe quelle API OpenAI-compatible. Inclut infer-serve pour héberger un GGUF via llama.cpp en une commande.

Open-source MLX runtime that runs local LLMs faster on Apple Silicon and NVIDIA CUDA - same tokens as the reference, every claim backed by measurement.

How to benchmark LLM inference: 3,000 measured DigitalOcean requests, raw JSON, rebuildable charts, and a 15-point disclosure checklist (August 24, 2026).

LLM inference observability sidecar for vLLM in Python: FastAPI middleware exports TTFT/TBT/E2E, KV-cache and queue-depth metrics to Prometheus + Grafana; Docker Compose stack; 32/32 pytest passing. Demo: TTFT p50 84ms / p99 244ms, TBT p50 17ms, E2E p50 384ms.

Prometheus exporter for LLM API monitoring — probes OpenAI, Anthropic, Google Gemini, Azure OpenAI and any OpenAI-compatible endpoint, collecting TTFT, latency, token usage and availability metrics.

A professional concurrent stress testing tool for Large Language Models. Test P99 latency, TTFT (Time To First Token), and token generation speed of your deployed models. 一个专业的大语言模型并发压测工具。测试部署模型的P99延迟、首字延迟(TTFT)和Token生成速度。

Zero-dependency CLI: benchmark latency (TTFT) and pricing across OpenAI-compatible AI providers

Dynamically benchmarks every model on NVIDIA NIM — type, context window, reasoning/cache, TTFT & token rate — and renders an interactive, dated HTML report.

A CLI tool for real-time comparison of Chinese LLM API response speeds.实时对比国产大模型 API 响应速度的 CLI 工具