#ai #LLM #homelab experiments/silliness today: writing a tiny inference gateway.I want one place on my tailnet that any ...

#ai #LLM #homelab experiments/silliness today: writing a tiny inference gateway.I want one place on my tailnet that any agent can use for inference, across model providers, that also gives upstream quota info when relevant and model info from models.dev (specifically context limits). I don't want to use litellm because it keeps getting hit with supply chain attacks and other stuff like bifrost seems way too heavy.The workflow is silly. This all happens in https://paseo.sh with https://pi.dev agents. One top level orchestrator agent, mostly idle, launches another orchestrator agent for each task in the epic.That agent launches an implementer using a local model then sleeps until implementation is done. It then launches a reviewer agent with a frontier model.The reviewer and implementer bounce back and forth until they're satisfied, then the orchestrator makes a PR.Paseo automatically closes the worktree when I merge the PR, then the top level orchestrator launches the next ready task.Th...