fireworks-ai-inference

Fireworks AI -- Fast Inference & Fine-Tuning

Fastest open-model inference platform with serverless and on-demand GPU deployments, OpenAI-compatible API, and built-in fine-tuning (SFT, DPO, RL).

When to Use Fireworks AI

Use Fireworks AI when:

• Need fast serverless inference for open-source models (Llama, Qwen, DeepSeek, Mixtral)
• Want OpenAI SDK drop-in replacement with open models
• Need fine-tuning without managing infrastructure (SFT, DPO, RL)
• Require structured output / JSON mode / function calling with open models
• Need dedicated GPU deployments with predictable latency
• Require SOC2 or HIPAA compliance
• Want prompt caching and batch inference for cost savings

Use alternatives instead:

Need	Use Instead
Self-hosted inference (full control)	vLLM, TensorRT-LLM
Cheapest serverless inference	Groq (free tier), Together AI
Managed LoRA fine-tuning (no infra)	Tinker
Closed-model APIs (GPT-4, Claude)	OpenAI, Anthropic direct
GPU instances with SSH access	Lambda Labs, RunPod
Multi-cloud orchestration	SkyPilot

Credential Setup

Credentials are auto-injected by openscience when connected via the dashboard.

# Verify credentials
[ -n "$FIREWORKS_API_KEY" ] && echo "FIREWORKS_API_KEY set" || echo "NOT SET"

If not set: add your Fireworks AI key in Customize → Models or export FIREWORKS_API_KEY locally.

Quick Start

Install

pip install fireworks-ai openai

Set API key

import os
os.environ["FIREWORKS_API_KEY"] = "fw_..."  # from https://fireworks.ai/api-keys

Basic chat completion

from openai import OpenAI

client = OpenAI(
    base_url="https://api.fireworks.ai/inference/v1",
    api_key=os.environ["FIREWORKS_API_KEY"],
)

response = client.chat.completions.create(
    model="accounts/fireworks/models/llama-v3p3-70b-instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain gradient descent in two sentences."},
    ],
    max_tokens=256,
    temperature=0.7,
)
print(response.choices[0].message.content)