> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hymalaia.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom Inference Provider

> Configure an OpenAI-compatible language model provider

## Guide

If you want to use a language model provider that is not supported directly by Hymalaia,
you can configure a custom inference provider.

> [!TIP]
>
> Your custom provider must provide OpenAI-compatible API endpoints.

**1. Setup your Custom Inference Provider**

Determine your provider's API base URL.
It should look something like `https://yourprovider.com/v1` or `http://localhost:12345/v1`.

**2. Navigate to Language Models**

Access the Admin Panel from your user profile icon, then navigate to **Configuration → Language Models**.

**3. Configure Custom Inference Provider**

Select **Add Custom LLM Provider** from the available providers.

![Custom provider configuration](https://mintcdn.com/hymalaia/8ykKHOjeXvJ7RJZr/images/ai-models/custom_provider_config.png?fit=max\&auto=format\&n=8ykKHOjeXvJ7RJZr\&q=85\&s=1e1f4fa0b530d1c7f9f4b1dd388a76c3)

Give your provider a **Display Name**.

Enter your model's **Provider Name**.

> [!NOTE]
>
> The **Provider Name** must match Litellm's [list of supported providers](https://docs.litellm.ai/docs/providers).

In this example, the provider name is `vertex_ai`.

![Custom inference provider name](https://mintcdn.com/hymalaia/8ykKHOjeXvJ7RJZr/images/ai-models/litellm_provider_name.png?fit=max\&auto=format\&n=8ykKHOjeXvJ7RJZr\&q=85\&s=fc5f0a83fbf050431900a3c385684648)

**4. Configure Optional Fields and Models**

Enter the provider's **Base URL**.

Fill out the other optional fields if applicable.

In the **Model Configurations** section, enter each model you want to make available through this provider.

**5. Designate Provider Access**

Lastly, you may select whether or not the provider is public to all users in Hymalaia.

If set to private,
the provider's models will be available to Admins and User Groups you explicitly assign the provider to.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.