LiteLLM is a versatile library designed to serve as a unified proxy, providing seamless access to over 100 large language models (LLMs) from various providers. It simplifies integration by acting as a single interface for diverse AI services.
Requirements
- Mandatory Add-On: Requires the “ChatGPT for Translator++” add-on to be installed first.
- Requirement Points: 1000
- Minimum Translator++ Version: 6.10.30
API Keys:
- API keys from your chosen LLM provider(s) are required (e.g., OpenAI, Anthropic, Cohere, etc.).
- Some services may require paid subscriptions. Obtain keys directly from the provider’s website.
Setup Guide
The LiteLLM add-on will automatically install and configure all necessary dependencies.
Prerequisite: Ensure you have an active subscription with your LLM provider and a valid API key.
- Click the Settings/Options menu (wrench and screwdriver icon 🔧).
- Go to the Translator & Language tab.
- Choose LiteLLM Unified AI Translation from the engine list.
- Enter your provider-specific API key (e.g., OpenAI API key).
- Input the model name (e.g.,
gpt-3.5-turbo,claude-2). Refer to your LLM provider’s documentation for supported models. - Start LiteLLM Server
- Under LiteLLM Proxy Configuration, click Start LiteLLM Server.
- Verify no errors appear in the console. A success message confirms the server is running.
Additional Notes
- Testing: After setup, run a test translation to ensure connectivity.
- Troubleshooting:
- If the server fails to start, check port availability (default:
8890) or firewall settings. - Monitor the console for detailed error logs.
- Advanced Configuration: Customize model parameters (temperature, max tokens) via LiteLLM’s advanced settings.
For a full list of supported LLM providers and models, visit the LiteLLM Documentation.