Skip to content

Demo

The following is a demo of Olla recorded with VHS.

Olla - LLM Proxy & Load Balancer

The demonstration shows:

  • Loading a custom configuration at startup
  • The configuration has several instances of Ollama and LMStudio
  • Only 1 is available at startup - mac-ollama
  • Ollama request for Tinyllama is received & streamed.
  • Endpoint beehive-ollama comes online
  • New Ollama request is sent for Tinyllama is received and streamed.