local-slm-edge-ai-expert

Local SLM & Edge AI Expert (WebGPU & In-Browser Intelligence)

English | Bahasa Indonesia

---

English

Orchestration & Integration

• Model Routing & Cost Optimization: llm-finops-router, adaptive-model-cascade, frontier-ai-models-expert
• Edge Runtimes & Frontend: senior-frontend, pwa-offline-first-expert, wasm-edge-computing-expert
• Local Vectors & RAG: vector-db-rag-expert, agentic-memory-architect, session-memory-manager
• Privacy & Safety Guardrails: compliance-gdpr-privacy-expert, ai-safety-governance-expert, zero-trust-secret-vault
• Lifecycle & Governance: zero-to-prod-orchestrator, anti-slop

Purpose & Overview

Production-grade architectural guide for running Small Language Models (SLMs) and embedding models directly inside client browsers and edge runtimes. Eliminates cloud API costs, guarantees 100% data privacy (zero cloud data leakage), and enables instant, offline-capable AI features using WebGPU, Transformers.js v3, WebLLM, and ONNX Runtime Web.

Key Capabilities

1. Client-Side Model Execution: Running quantized 1B–4B SLMs (Llama 3.2 1B/3B, Gemma 2 2B, Phi-3.5 Mini, Qwen 2.5 1.5B/3B) entirely inside the user's browser via WebGPU.
2. In-Browser Embeddings: Fast client-side vector embeddings with models like all-MiniLM-L6-v2 or bge-small-en-v1.5 using Transformers.js v3.
3. Hybrid Edge-Cloud Fallback: Gracefully falling back to server-side LLMs when client hardware lacks WebGPU or sufficient VRAM.
4. Zero-Latency PII Masking: Anonymizing sensitive user data locally on the client before sending queries to external LLMs.

---

Production Implementation Recipes

Recipe 1: In-Browser Semantic Embedding Generation with Transformers.js v3

import { pipeline, env } from '@huggingface/transformers';

// Configure cache and worker settings
env.allowLocalModels = false;
env.useBrowserCache = true;

let embedder: any = null;

export async function getLocalEmbedding(text: string): Promise<number[]> {
  if (!embedder) {
    embedder = await pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2', {
      dtype: 'fp32',
      device: 'webgpu', // Accelerate via WebGPU if supported
    });
  }

  const output = await embedder(text, { pooling: 'mean', normalize: true });
  return Array.from(output.data);
}