Why I built llm-queue: one local LLM (Ollama) behind an OpenAI-compatible HTTP API, so every process shares one serialized priority queue instead of thrashing the model.
Tag
LLM
1 post tagged “LLM”.
Tag
1 post tagged “LLM”.
Why I built llm-queue: one local LLM (Ollama) behind an OpenAI-compatible HTTP API, so every process shares one serialized priority queue instead of thrashing the model.