How to Run Basaran on Runpod: An Open-Source Alternative to OpenAI’s Completion API

Basaran is an open-source alternative to the OpenAI text completion API, and this guide provides instructions on how to run it on Runpod using a predefined template. Users can customize the model by editing the MODEL environment variable during pod creation, allowing for the use of different models than the default bigscience/bloomz-560m. The guide outlines steps such as creating a pod, modifying the environment variable if necessary, and resetting the pod to apply any changes. Once the pod is running, users can access the Basaran interface via HTTP to generate text completions or utilize the API by appending /v1/completions to the proxy URL. The guide concludes by encouraging users to reach out for support via Runpod's Discord or other community channels if needed.