Inference Grid

# Configuration

DANGER

The Operator program is currently invite-only. If you're interested in running an inference node, please reach out to us at hello@inferencegrid.ai.

# Keys

To become an operator, you need to generate a private/public key pair.

openssl genpkey -algorithm ED25519 -out private.pem
openssl pkey -in private.pem -text -noout | grep 'priv:' -A3 | tail -n +2 | tr -d '[:space:]:' | tr -d '\n'

This will output a 64 character hexadecimal string. Keep it somewhere safe - this is how your node will prove its identity to the network. To get the public key, run:

openssl pkey -in private.pem -pubout -outform DER | tail -c +13 | xxd -p -c 32

When joining the network, your node broadcasts the following information:

{
    "public_key": "...",
    "name": "My Node (optional)",
    "website": "https://my-node.com (optional)",
}

When processing requests, your node will sign all requests with your private key to ensure that no one can impersonate you. This is automatically handled by our SDK if you use it to deploy your node.

As your node(s) process requests, the reputation score associated with your public key will be updated based on your performance, both in terms of speed and quality.

# Wallets

Spark (opens new window) is a non-custodial wallet which enables you to send and receive money on the Lightning Network. Your Spark wallet will enable you to receive payments from users and then withdraw those funds to your bank account using services such as UMA (opens new window), Coinbase (opens new window), or CashApp (opens new window).

To generate a Spark wallet, you can use the following script:

import { WalletSDK } from "wallet-sdk";

const sparkWallet = new WalletSDK("MAINNET");
const mnemonic = sparkWallet.generateMnemonic();
console.log("Mnemonic:", mnemonic);

This will output a 12 word mnemonic which can be used to initialize your Spark wallet. Keep this mnemonic somewhere safe - anyone who has access to it can send or withdraw funds. This is automatically managed by our SDK if you use it to deploy your node.

WARNING

Currently, we exclusively support Spark wallet integration. However, we actively welcome community contributions to expand support for additional Lightning Network wallets.

# Models

The Inference Grid supports a wide range of models, from language models to vision models. You can serve any model you want, but if you're not sure which model to run, we recommend running one of the following models since they are popular and in high demand.

Model Minimum Specs Description
pixtral-12b 40GB GPU Vision+language model.
llama-3.2-11b-vision-instruct 40GB GPU Vision+language model.
dolphin-mixtral-8x7b 30GB GPU Language model. Uncensored.
pixtral-large-2411 300GB GPU Vision+language model.
llama-3.2-90b-vision-instruct 300GB GPU Vision+language model.

Note that you are responsible for complying with the appropriate licenses for the models you choose to serve.

← Validation →