# LLM latency calculator

Estimate how long an AI response takes

Use it at https://www.quicktoolz.com/tools/llm-latency-calculator. From QuickToolz.

An AI response takes two steps: waiting for the first token, then generating the rest. Total time is the time to first token, plus the output tokens divided by tokens per second. With 500 ms to start and 300 tokens at 50 a second, a full answer takes about 6.5 seconds.

## How it works

total time = time to first token + output tokens ÷ tokens per second

Streaming shows words as they arrive, so the time to first token is what people feel most.

Long prompts raise the time to first token, and long answers raise the total.

Worked example: With 500 ms to the first token and 300 tokens at 50 a second: 500 ms + 6 s = 6.5 s.

## Examples

| First token | Tokens a second | Output | Total |
| --- | --- | --- | --- |
| 300 ms | 100 | 100 | 1.3 s |
| 500 ms | 50 | 300 | 6.5 s |
| 1000 ms | 30 | 1000 | 34.3 s |

## What this tool doesn't cover

- Real speeds change with the model, the load and your network.
- The words figure assumes about 0.75 words a token, which is rough.

## Questions

### What is a good time to first token?

Under a second feels quick for chat. Test your own model and region.

### How can I make responses faster?

Ask for shorter answers, use a smaller model, or stream the output.

### Where do I find tokens per second?

Measure it from your own calls, as published figures vary.

### What affects AI response time?

The model size, the length of your prompt, how much it writes, and how busy the service is.

### Why does streaming feel faster?

People see words as soon as they’re written, so they start reading while the rest is on its way. The total time doesn’t change.

## Sources

- [OpenAI: latency optimization](https://platform.openai.com/docs/guides/latency-optimization)