From clone to first request.

Clone the MIT-licensed SolanaLM repo from GitHub. You need Python 3.12 or newer.

git clone https://github.com/cryptuon/solanalm.git
cd solanalm

Install the gateway, SDK, and node runtime dependencies with Poetry.

poetry install
poetry shell

Set your role (inference, training, or proxy) and declare the models you can serve. The quick-start script can provision a testnet wallet for development, so you do not need to know Solana to start.

cp config.example.toml config.toml
# edit config.toml: role = "inference"
#   models = ["meta-llama/Llama-3.1-8B-Instruct"]

Bring up the gateway plus a node locally with Docker Compose (or run them directly with Uvicorn). The node joins the registry on its first heartbeat.

docker compose up -d
# gateway on :8000, node registered on first heartbeat

Point the OpenAI-compatible client at the gateway and pass your Solana wallet address as the API key. Existing OpenAI SDK code works unchanged.

from solanalm_client import OpenAICompatibleClient

client = OpenAICompatibleClient(
    base_url="http://localhost:8000/v1",
    api_key="your-solana-wallet-address",
)
resp = client.chat.completions.create(
    model="meta-llama/Llama-3.1-8B-Instruct",
    messages=[{"role": "user", "content": "Explain CRDTs."}],
)
print(resp.choices[0].message.content)