Rusty Beam Architecture

This document provides an in-depth look at Rusty Beam's architecture, covering the core server design, request processing pipeline, plugin system, and key architectural decisions.

Architectural Overview
Core Design Principles
Request Processing Flow
Plugin System Architecture
Memory Model & State Management
Configuration System
Key Components
Security Architecture

Architectural Overview

┌─────────────────┐     ┌──────────────────┐     ┌─────────────────┐
│  HTTP Client    │────▶│   Rusty Beam     │────▶│  HTML Document  │
└─────────────────┘     │     Server       │     └─────────────────┘
                        └──────────────────┘
                                 │
                    ┌────────────┴───────────┐
                    │   Plugin Pipeline      │
                    ├────────────────────────┤
                    │ • WebSocket            │
                    │ • Selector Handler     │
                    │ • Authentication       │
                    │ • File Handler         │
                    └────────────────────────┘

Rusty Beam is built as an asynchronous, event-driven HTTP server using Tokio and Hyper. The architecture emphasizes:

Modularity: Core functionality separated into plugins
Async-First: All I/O operations are non-blocking
Document-Centric: HTML documents are first-class citizens
Standards-Based: Uses standard HTTP headers creatively

Core Design Principles

1. Plugin-Based Architecture

Everything is a plugin. The core server only handles:

HTTP connection management
Plugin loading and pipeline execution
Configuration parsing
Basic request/response handling

2. Async Pipeline Processing

Requests flow through plugins sequentially but asynchronously:

pub async fn process_request_through_pipeline(
    req: Request<Body>,
    app_state: Arc<AppState>,
    pipeline: Arc<Vec<Arc<dyn Plugin>>>,
) -> Result<Response<Body>, Box<dyn std::error::Error + Send + Sync>>

3. Immutable Request Pattern

Plugins receive requests by value and must return modified versions, ensuring clean data flow and preventing side effects.

4. HTML as Configuration

Configuration uses semantic HTML with microdata, making it self-documenting and queryable via the same selector API.

Request Processing Flow

Detailed Request Lifecycle

Connection Accepted: Hyper accepts TCP connection
Request Parsing: HTTP request parsed into Request<Body>
Host Resolution: Determine which host pipeline to use

Pipeline Execution:

for plugin in pipeline {
    match plugin.handle_request(req, app_state).await? {
        PluginResponse::Continue(new_req) => req = new_req,
        PluginResponse::Done(response) => return response,
    }
}

Response Generation: Final plugin generates response
Response Pipeline: Plugins can modify response in reverse order
Send Response: Response sent to client

Plugin Response Types

Response Type	Description	Use Case
`Continue(Request)`	Pass modified request to next plugin	Authentication, logging, header modification
`Done(Response)`	Generate response and stop pipeline	File serving, error handling, API endpoints

Plugin System Architecture

Plugin Trait Definition

pub trait Plugin: Send + Sync {
    fn name(&self) -> &str;
    
    async fn handle_request(
        &self,
        req: Request<Body>,
        app_state: Arc<AppState>,
    ) -> Result<PluginResponse, Box<dyn std::error::Error + Send + Sync>>;
    
    async fn handle_response(
        &self,
        response: Response<Body>,
        _app_state: Arc<AppState>,
    ) -> Result<Response<Body>, Box<dyn std::error::Error + Send + Sync>> {
        Ok(response)
    }
}

Dynamic Plugin Loading

Plugins are loaded as dynamic libraries via FFI:

Plugin compiled as cdylib with create_plugin export
Server loads .so/.dll/.dylib at runtime
Plugin instance created and added to pipeline
Type erasure via Arc<dyn Plugin>

Plugin Communication

Plugins communicate via:

Request Extensions: Type-safe data attached to requests
Shared State: AppState with thread-safe collections
Headers: Custom headers for metadata

Memory Model & State Management

AppState Structure

pub struct AppState {
    pub connections: Arc<RwLock<HashMap<String, Vec<Connection>>>>,
    pub config: Arc<RwLock<Config>>,
    pub plugin_state: Arc<RwLock<HashMap<String, Arc<dyn Any + Send + Sync>>>>,
}

Concurrency Model

Read-Heavy Optimization: RwLock for shared state
Arc Everything: Reference counting for shared ownership
Async Everywhere: No blocking operations in request path
Type-Safe Plugin State: Plugins store typed state via Any

Memory Safety Guarantees

No unsafe code in core server
Plugin FFI boundary is the only unsafe interface
All shared state protected by locks
Request/response ownership prevents data races

Configuration System

HTML Configuration Loading

pub fn load_config_from_html(path: &str) -> Result<Config, Box<dyn std::error::Error>> {
    let html = fs::read_to_string(path)?;
    let doc = dom_query::Document::from(html.as_str());
    
    // Extract server config via CSS selectors
    let server_config = doc.select("[itemtype='https://rustybeam.net/schema/ServerConfig']")?;
    
    // Parse microdata properties
    let bind_address = server_config.select("[itemprop='bindAddress']")?.text();
    let bind_port = server_config.select("[itemprop='bindPort']")?.text().parse()?;
    
    // Load host configurations and plugins...
}

Hot Reload Architecture

Configuration reloading via SIGHUP:

Signal handler receives SIGHUP
New configuration loaded in background
Atomic swap of configuration
New requests use new config
In-flight requests complete with old config

Key Components

Core Server Files

src/
├── main.rs              # Entry point, server setup
├── config.rs            # Configuration parsing
├── plugin.rs            # Plugin trait and types
├── utils.rs             # Path canonicalization, utilities
└── lib.rs               # Public API exports

plugins/
├── selector-handler/    # CSS selector manipulation
├── file-handler/        # Static file serving
├── websocket/           # WebSocket support
├── basic-auth/          # HTTP Basic Authentication
└── [other plugins]/     # Additional functionality

Critical Functions

Function	Location	Purpose
`handle_request()`	src/main.rs:316	Main request entry point
`process_request_through_pipeline()`	src/main.rs:169	Execute plugin pipeline
`create_host_pipelines()`	src/main.rs:70	Build plugin pipelines from config
`canonicalize_file_path()`	src/utils.rs	Security-critical path validation

Security Architecture

Path Traversal Prevention

pub fn canonicalize_file_path(
    root: &Path, 
    requested: &str
) -> Result<PathBuf, Box<dyn std::error::Error>> {
    let normalized = requested.trim_start_matches('/');
    let full_path = root.join(normalized).canonicalize()?;
    
    // Ensure path is within root
    if !full_path.starts_with(root) {
        return Err("Path traversal attempt".into());
    }
    
    Ok(full_path)
}

Plugin Isolation

Plugins run in same process but isolated via trait boundary
No direct memory access between plugins
All communication through defined interfaces
Plugin crashes don't affect server (with panic catching)

Authentication & Authorization

Pluggable auth system via basic-auth and authorization plugins
Request extensions carry authentication state
Each plugin can enforce its own authorization
Fail-secure: no auth means no access

Performance Considerations

Optimization Strategies

Zero-Copy Where Possible: Use Bytes for body handling
Lazy Parsing: Only parse HTML when selectors used
Connection Pooling: Reuse WebSocket connections
Async All The Way: Never block the runtime

Scalability

Horizontal scaling via multiple instances
Shared nothing architecture
Plugin pipeline overhead is minimal
Memory usage proportional to active connections

Future Architecture Directions

Planned Enhancements

Plugin Sandboxing: WASM-based plugins for isolation
Distributed State: Redis/etcd for multi-instance coordination
gRPC Support: Binary protocol for high-performance scenarios
Native HTTP/3: QUIC-based transport

Architectural Invariants

These principles will remain constant:

HTML documents as first-class citizens
Plugin-based extensibility
CSS selectors for addressing
Configuration as HTML
Progressive enhancement philosophy