> ## Documentation Index
> Fetch the complete documentation index at: https://docs.abliteration.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Use LiteLLM with Abliteration

> Call Abliteration through the LiteLLM Python SDK or a local LiteLLM gateway.

Use LiteLLM's built-in `abliteration/` provider with your Abliteration API key. No custom base URL is needed.

## Install

Use Python 3.12 or later and create an isolated environment:

```sh theme={"system"}
python3 -m venv .venv
source .venv/bin/activate
pip install "litellm==1.106.4"
```

Create an `ak_...` key in the [console](https://abliteration.ai/console), then set it in the same terminal:

```sh theme={"system"}
export ABLITERATION_API_KEY="ak_YOUR_API_KEY"
```

## Send a request

Save this as `chat.py` and run `python chat.py`:

```python theme={"system"}
from litellm import completion

response = completion(
    model="abliteration/abliterated-model-large-v2",
    messages=[{"role": "user", "content": "Reply with hello."}],
    extra_body={"reasoning_effort": "low"},
    max_tokens=512,
)
print(response.choices[0].message.content)
```

Prefix any supported [model ID](https://docs.abliteration.ai/models.md) with `abliteration/`. For images, use `abliteration/abliterated-model-large-flash` or `abliteration/abliterated-model`. Large v2 accepts text only.

## Stream a response

Save this as `stream.py` and run `python stream.py`:

```python theme={"system"}
from litellm import completion

stream = completion(
    model="abliteration/abliterated-model-large-v2",
    messages=[{"role": "user", "content": "Reply with hello."}],
    extra_body={"reasoning_effort": "low"},
    max_tokens=512,
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)
    if getattr(chunk, "usage", None):
        print("\n", chunk.usage)
```

The final chunk includes token usage, including cached input tokens when available. See [prompt caching](https://docs.abliteration.ai/capabilities/prompt-caching.md) for how to improve cache reuse.

## Run a local gateway

In the same environment, install the proxy and create a key for local gateway clients:

```sh theme={"system"}
pip install "litellm[proxy]==1.106.4"
export LITELLM_MASTER_KEY="sk-$(openssl rand -hex 32)"
printf '%s\n' "$LITELLM_MASTER_KEY"
```

Copy the printed gateway key for the client terminal. It is separate from your Abliteration API key.

Save this as `config.yaml`:

```yaml theme={"system"}
model_list:
  - model_name: abliteration-chat
    litellm_params:
      model: abliteration/abliterated-model-large-v2
      api_key: os.environ/ABLITERATION_API_KEY
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
```

Start the gateway:

```sh theme={"system"}
litellm --config config.yaml --host 127.0.0.1 --port 4000
```

In a second terminal, set the gateway key you copied:

```sh theme={"system"}
export LITELLM_MASTER_KEY="sk-YOUR_GATEWAY_KEY"
```

Send a request:

```sh theme={"system"}
curl --fail-with-body http://127.0.0.1:4000/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "abliteration-chat",
    "messages": [{"role": "user", "content": "Reply with hello."}],
    "extra_body": {"reasoning_effort": "low"},
    "max_tokens": 512
  }'
```

A successful request returns an assistant message in `choices[0].message.content` and token counts in `usage`. Gateway clients use `abliteration-chat` as the model name and the gateway key for authentication. Keep your Abliteration key on the gateway process.

## Tools and reasoning

Pass `tools`, `tool_choice`, and `response_format` as in [OpenAI Chat Completions](https://docs.abliteration.ai/api/openai-compatibility.md).

In LiteLLM 1.106.4, put `reasoning_effort` in `extra_body`, as shown above. A top-level `reasoning_effort` is not forwarded. See [reasoning effort](https://docs.abliteration.ai/capabilities/thinking.md) for supported levels.

## Responses and Messages

LiteLLM translates Responses and Anthropic Messages requests into Chat Completions for this provider. For endpoint-specific features such as Anthropic web fetch, use Abliteration's [native APIs](https://docs.abliteration.ai/compatibility-matrix.md). See [LiteLLM's provider guide](https://docs.litellm.ai/docs/providers/abliteration) for its translated interfaces.

## Troubleshooting

| Error | Action |
| - | - |
| Unknown provider or model | Check the `abliteration/` prefix and installed LiteLLM version. Compare the unprefixed ID with [the model catalog](https://docs.abliteration.ai/models.md#list-models). |
| `401` | Check `ABLITERATION_API_KEY` on the gateway or SDK process. For gateway authentication, check the client's `LITELLM_MASTER_KEY`. |
| `402` | Check your Abliteration credit balance in the [console](https://abliteration.ai/console). |
| `429` | Follow the retry guidance in [rate limits](https://docs.abliteration.ai/api/rate-limits.md). |

LiteLLM needs a current model catalog to estimate costs and recognize model limits. Estimates can differ from your bill because of web search charges and subscription discounts. Check [Abliteration pricing](https://docs.abliteration.ai/pricing.md) and Console usage for billing.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.