Observability Architecture: Building Systems You Can Understand

The Three Pillars

Observability means understanding what's happening inside your system. The three pillars—logs, metrics, and traces—provide different views into system behavior.

Structured Logging

class OrderService
{
    public function place(Order $order): void
    {
        Log::info('Order placement started', [
            'order_id' => $order->id,
            'customer_id' => $order->customerId,
            'total' => $order->total,
        ]);

        try {
            $this->process($order);

            Log::info('Order placed successfully', [
                'order_id' => $order->id,
                'processing_time_ms' => $this->timer->elapsed(),
            ]);
        } catch (Exception $e) {
            Log::error('Order placement failed', [
                'order_id' => $order->id,
                'error' => $e->getMessage(),
                'trace' => $e->getTraceAsString(),
            ]);
            throw $e;
        }
    }
}

Metrics Collection

class MetricsCollector
{
    public function increment(string $metric, array $tags = []): void
    {
        $this->client->increment($metric, $tags);
    }

    public function timing(string $metric, float $ms, array $tags = []): void
    {
        $this->client->timing($metric, $ms, $tags);
    }

    public function gauge(string $metric, float $value, array $tags = []): void
    {
        $this->client->gauge($metric, $value, $tags);
    }
}

// Usage
$metrics->increment('orders.placed', ['payment_method' => 'stripe']);
$metrics->timing('orders.processing_time', 234.5);

Distributed Tracing

class TracingMiddleware
{
    public function handle(Request $request, Closure $next): Response
    {
        $span = $this->tracer->startSpan('http_request', [
            'http.method' => $request->method(),
            'http.url' => $request->url(),
        ]);

        $response = $next($request);

        $span->setTag('http.status_code', $response->status());
        $span->finish();

        return $response;
    }
}

Health Checks

class HealthController
{
    public function check(): Response
    {
        $checks = [
            'database' => $this->checkDatabase(),
            'cache' => $this->checkCache(),
            'queue' => $this->checkQueue(),
        ];

        $healthy = collect($checks)->every(fn ($check) => $check['status'] === 'healthy');

        return response()->json([
            'status' => $healthy ? 'healthy' : 'unhealthy',
            'checks' => $checks,
        ], $healthy ? 200 : 503);
    }
}

Conclusion

Build observability into your architecture from the start. Structured logs, business metrics, distributed traces, and health checks give you the visibility needed to operate systems confidently.