Platform

WARNING

Here for the Spark Hackathon? Then this page isn't for you. Check out the dedicated Spark section above!

So you want to use the inference grid?

The easiest way to use the inference grid is to use Platform (opens new window) which provides an OpenAI-compatible API endpoint. This service allows you to sign up for an account, generate an API key, and fund your account with a credit card, Spark wallet, or UMA Auth.

import json
from openai import OpenAI

model_spec = {
    "tier_selector": ">3",
    "flags": [
        "uncensored", # Support open source, uncensored models.
    ],
}

client = OpenAI(
    base_url="https://relay.inferencegrid.ai/",
    api_key="..."
)

client.chat.completions.create(
    model=json.dumps(model_spec),
    messages=[
        {
            "role": "user",
            "content": "Tell me a joke!",
        },
    ],
)

The Platform service is great for most use cases, but what if you want more control? What if you want complete control over your funds and to make sure that no one can censor or block your requests? For that, we recommend running the adapter and connecting directly to the grid!