together-ai-inference

Together AI — Serverless Inference & Fine-Tuning

Together AI is an AI cloud platform providing serverless inference on 200+ open-source models through an OpenAI-compatible API. It supports chat completions, embeddings, fine-tuning, image generation, and batch processing at https://api.together.xyz/v1.

When to Use Together AI

Use Together AI when:

• You need fast serverless inference on open-source models (Llama, DeepSeek, Qwen, Mistral)
• You want an OpenAI-compatible API so you can swap providers with a single line change
• You need to fine-tune open-source models without managing GPU infrastructure
• You want cost-effective inference with pay-per-token pricing
• You need function calling, JSON mode, or structured outputs from open-source models
• You want batch processing at 50% lower cost for non-urgent workloads
• You need embeddings or image generation alongside chat completions

Use alternatives instead:

Need	Use Instead
Managed LoRA fine-tuning with training platform	Tinker
Self-hosted inference with full control	vLLM, TensorRT-LLM
Dedicated GPU instances	Lambda Labs, RunPod
Serverless GPU with custom containers	Modal
Multi-cloud cost optimization	SkyPilot
Proprietary models (GPT-4o, Claude)	OpenAI, Anthropic directly

Credential Setup

Credentials are auto-injected by openscience when connected via the dashboard.

# Verify credentials
[ -n "$TOGETHER_API_KEY" ] && echo "TOGETHER_API_KEY set" || echo "NOT SET"

If not set: add your Together AI key in Customize → Models or export TOGETHER_API_KEY locally.

Quick Start

Install

pip install together openai

Set API Key

import os
os.environ["TOGETHER_API_KEY"] = "your-api-key"

# Or export in shell:
# export TOGETHER_API_KEY="your-api-key"

Get your API key from https://api.together.xyz/settings/api-keys

Basic Chat Completion

from together import Together

client = Together()

response = client.chat.completions.create(
    model="meta-llama/Llama-3.3-70B-Instruct-Reference",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain gradient descent in one paragraph."},
    ],
    max_tokens=256,
    temperature=0.7,
)

print(response.choices[0].message.content)