Use LiteLLM with Abliteration

Use LiteLLM’s built-in abliteration/ provider with your Abliteration API key. No custom base URL is needed.

Install

Use Python 3.12 or later and create an isolated environment:

python3 -m venv .venv
source .venv/bin/activate
pip install "litellm==1.106.4"

Create an ak_... key in the console, then set it in the same terminal:

export ABLITERATION_API_KEY="ak_YOUR_API_KEY"

Send a request

Save this as chat.py and run python chat.py:

from litellm import completion

response = completion(
    model="abliteration/abliterated-model-large-v2",
    messages=[{"role": "user", "content": "Reply with hello."}],
    extra_body={"reasoning_effort": "low"},
    max_tokens=512,
)
print(response.choices[0].message.content)

Prefix any supported model ID with abliteration/. For images, use abliteration/abliterated-model-large-flash or abliteration/abliterated-model. Large v2 accepts text only.

Stream a response

Save this as stream.py and run python stream.py:

from litellm import completion

stream = completion(
    model="abliteration/abliterated-model-large-v2",
    messages=[{"role": "user", "content": "Reply with hello."}],
    extra_body={"reasoning_effort": "low"},
    max_tokens=512,
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)
    if getattr(chunk, "usage", None):
        print("\n", chunk.usage)

The final chunk includes token usage, including cached input tokens when available. See prompt caching for how to improve cache reuse.

Run a local gateway

In the same environment, install the proxy and create a key for local gateway clients:

pip install "litellm[proxy]==1.106.4"
export LITELLM_MASTER_KEY="sk-$(openssl rand -hex 32)"
printf '%s\n' "$LITELLM_MASTER_KEY"

Copy the printed gateway key for the client terminal. It is separate from your Abliteration API key. Save this as config.yaml:

model_list:
  - model_name: abliteration-chat
    litellm_params:
      model: abliteration/abliterated-model-large-v2
      api_key: os.environ/ABLITERATION_API_KEY
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

Start the gateway:

litellm --config config.yaml --host 127.0.0.1 --port 4000

In a second terminal, set the gateway key you copied:

export LITELLM_MASTER_KEY="sk-YOUR_GATEWAY_KEY"

Send a request:

curl --fail-with-body http://127.0.0.1:4000/v1/chat/completions \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "abliteration-chat",
    "messages": [{"role": "user", "content": "Reply with hello."}],
    "extra_body": {"reasoning_effort": "low"},
    "max_tokens": 512
  }'

A successful request returns an assistant message in choices[0].message.content and token counts in usage. Gateway clients use abliteration-chat as the model name and the gateway key for authentication. Keep your Abliteration key on the gateway process.

Tools and reasoning

Pass tools, tool_choice, and response_format as in OpenAI Chat Completions. In LiteLLM 1.106.4, put reasoning_effort in extra_body, as shown above. A top-level reasoning_effort is not forwarded. See reasoning effort for supported levels.

Responses and Messages

LiteLLM translates Responses and Anthropic Messages requests into Chat Completions for this provider. For endpoint-specific features such as Anthropic web fetch, use Abliteration’s native APIs. See LiteLLM’s provider guide for its translated interfaces.

Troubleshooting

Error	Action
Unknown provider or model	Check the `abliteration/` prefix and installed LiteLLM version. Compare the unprefixed ID with the model catalog.
`401`	Check `ABLITERATION_API_KEY` on the gateway or SDK process. For gateway authentication, check the client’s `LITELLM_MASTER_KEY`.
`402`	Check your Abliteration credit balance in the console.
`429`	Follow the retry guidance in rate limits.

LiteLLM needs a current model catalog to estimate costs and recognize model limits. Estimates can differ from your bill because of web search charges and subscription discounts. Check Abliteration pricing and Console usage for billing.

Integrations

Security

Agents

AI testing and red teaming

SDKs & frameworks

API routing & provider switching

Runtimes

Use LiteLLM with Abliteration

Install

Send a request

Stream a response

Run a local gateway

Tools and reasoning

Responses and Messages

Troubleshooting

​Install

​Send a request

​Stream a response

​Run a local gateway

​Tools and reasoning

​Responses and Messages

​Troubleshooting

Install

Send a request

Stream a response

Run a local gateway

Tools and reasoning

Responses and Messages

Troubleshooting