walkeros-understanding-flow

Understanding walkerOS Flow

Overview

walkerOS follows a Source → Collector → Destination(s) architecture for composable, modular event processing.

Core principle: Separation of concerns. Each component has one job. Components are composable and replaceable.

The Flow Pattern

[Source.before] → Sources → [Source.next] → Collector → [Dest.before] → Destinations → [Dest.next]
(Preprocessing)  (Capture)  (Pre-chain)   (Processing) (Post-chain)   (Delivery)     (Post-push)

Consent-exempt:                                                                        Post-consent:
- Decode                    - Validation   - Event creation - Validation   - Push       - Audit logging
- Validate format           - Enrichment   - Enrichment    - Enrichment   - Send       - Response parsing
- Authenticate              - Redaction    - Consent check - Routing      - Store      - Webhooks

Key Concepts

Composability

A Flow combines components. You can:

• Use multiple sources feeding one collector
• Route events to multiple destinations
• Swap components without changing others

The Flow Type

See packages/core/src/types/flow.ts for the canonical interface.

// Conceptual structure (see source for full type)
interface Flow {
  stores?: Record<string, Store>;
  sources?: Record<string, Source>;
  transformers?: Record<string, Transformer>;
  destinations?: Record<string, Destination>;
  collector?: Collector.InitConfig;
}

Universal Push Interface

All components communicate via push functions:

Component	Push Signature	Purpose
Source	push(input) → events	Capture external data
Collector	push(event) → void	Process and route
Destination	push(event, context) → void	Transform and deliver

The elb() function is an alias for collector.push - used for component wiring.