Why I built llm-queue: one local LLM (Ollama) behind an OpenAI-compatible HTTP API, so every process shares one serialized priority queue instead of thrashing the model.
Tag
Side project
1 post tagged “Side project”.
Tag
1 post tagged “Side project”.
Why I built llm-queue: one local LLM (Ollama) behind an OpenAI-compatible HTTP API, so every process shares one serialized priority queue instead of thrashing the model.