Ryan Mitchell

Bastion Security, 2023

Bastion's threat detection polled cloud APIs on a 45-second batch interval. Two enterprise prospects walked during POC evaluations because of the detection gap. I proposed and led the migration to a Kafka-based streaming architecture with event subscriptions from CloudTrail and cloud provider webhooks.

The interesting part was the stateful stream processor. It correlates events across multiple cloud sources in real-time with exactly-once processing guarantees via idempotent consumers. I ran a 3-week parallel deployment where both the old and new systems processed live traffic so we could validate correctness before cutting over.

Alert latency dropped from 45 seconds to under 3 seconds. We re-engaged both lost prospects and one converted to a $280K ARR contract. The streaming architecture also unblocked a "live threat map" feature that product had deprioritized as technically infeasible.

Multi-Tenant API Gateway

Bastion Security, 2023

The existing API layer was a single Flask app with tenant routing in middleware. A noisy tenant incident caused a 12-minute partial outage across all customers. At 800 req/sec, p99 was already 420ms and degrading.

I built a new gateway in Go with tenant-scoped connection pools, adaptive rate limiting, and circuit breakers per downstream service. Rolled it out as a gradual traffic shift using weighted DNS, migrating 5% at a time over two weeks while watching latency percentiles. Credentials rotate automatically every 72 hours with zero downtime.

The gateway now handles 2,400 req/sec at p99 of 68ms. Noisy-tenant incidents: zero since launch. Processes $18M in ARR worth of customer traffic.

ECS to Kubernetes Migration

Bastion Security, 2024

14 microservices on ECS with manually maintained task definitions. Deployments took 18 minutes, rollbacks required SSH, and infrastructure costs had grown 40% YoY without matching traffic growth. I wrote Helm charts for all 14 services with standardized health checks, resource limits, and auto-scaling.

Built a GitOps pipeline with ArgoCD for declarative deployments. Migrated in 4 waves over 6 weeks, starting with the lowest-risk internal services. Ran hands-on workshops so every engineer could deploy and debug independently.

ML Model Serving Layer

Tensor Cloud, 2021-2022

Tensor Cloud's inference layer was a prototype Flask service. At 400 req/sec, response times exceeded 500ms and GPU utilization was below 35% because requests queued sequentially.

I designed a two-tier architecture: a Go request router handling connection management and batching, fronting Python inference workers with shared GPU memory pools. Adaptive batching groups requests by model version. An LRU response cache with TTL hits 23% of requests, giving free headroom. Health-based routing avoids workers mid-model-load.

The serving layer handles 1,800 req/sec at p99 of 92ms. GPU utilization jumped from 35% to 78%. The architecture became the template for Tensor's next two product lines.

Event-Driven Billing Migration

Tensor Cloud, 2022

Billing lived inside a Django monolith. Race conditions between usage tracking and the billing cron caused double-charges -- $34K in refunds over 6 months. Customer trust was eroding.

I extracted billing into a standalone service. Usage events publish to Kafka with idempotent producers. The consumer processes with exactly-once semantics using the transactional outbox pattern. I built a real-time usage dashboard so customers see charges accumulate live, and wrote property-based tests (Hypothesis) to verify billing invariants under concurrent load. Zero billing discrepancies in 14 months since launch.

GPU Cost Attribution Dashboard

Tensor Cloud, 2022

$230K/month GPU bill and nobody could tell which teams or projects consumed what. Finance was manually attributing costs from AWS billing exports in spreadsheets.

Built the dashboard in React with D3.js. Designed a cost attribution model using Kubernetes labels to tag GPU hours by team, project, and model type. Wrote an ETL pipeline enriching AWS Cost Explorer data with pod metadata every 15 minutes. Drill-down views, trend lines, anomaly highlighting for spend spikes. Adopted by all 8 engineering teams in the first week. Identified $51K/month in idle GPU instances.

Time-Series Query Engine

Pinpoint Analytics, 2020

Pinpoint's analytics dashboard was slow for high-volume customers. P95 load time was 4.2 seconds, with some customers hitting 8+. The root cause was full table scans on a 900M-row time-series table. The team didn't have bandwidth for a new database.

I stayed within PostgreSQL and implemented a multi-pronged strategy: time-based monthly partitioning, materialized views for the 5 most common aggregations on staggered refresh schedules, partial indexes on customer_id + time range, and raw SQL replacing ORM queries where the ORM generated suboptimal partition pruning plans.