Thirteen NestJS services, one Postgres cluster, a Kafka broker, Redis and an API gateway. On a laptop, one docker compose up starts all of it. The tempting plan is to run the same command on one EC2 host and call it production. The tempting objection is that "real" microservices run on Kubernetes.
Both skip a step. Look at two problems the same Compose stack can hit.
Problem one. The stack boots, and the first call from one service to another fails with ECONNREFUSED. Neither service is broken. Compose started them in the order you declared; it just waited for the wrong thing.
Problem two. You ship a fix with docker compose up -d order, and for a few seconds the gateway answers 502. Nothing crashed. Compose replaced the container exactly as designed.
The first is a configuration bug that Compose fixes in a few lines. The second isn't a bug. It's what a single host is, and no Compose flag removes it. Telling those two kinds of problem apart is most of the Compose-or-Kubernetes decision.
A note on evidence: the platform is a prototype. What follows describes the repository (Compose files that boot the whole stack, Terraform and a pipeline for the next step), not production results. I'll mark exactly where that line sits.
What does a deployment actually have to do?
Think about deliveries. One van with one driver can load parcels, drive the route and restart the engine when it stalls. What one van can't do is keep deliveries going while it's in the workshop, or after it ends up in a ditch. For that you need more vans and a dispatcher: someone who knows which vans are running, sends a spare out before calling one in, and reroutes when a van goes quiet.
Deployment splits along the same line, into two job lists.
List one: describe and start.
- Build one image per service.
- Give each container its config and secrets.
- Start things in dependency order, and wait until each one is ready, not just running.
- Let services find each other by name.
- Restart a process that crashes, and keep the logs.
List two: keep serving while things change and break.
- Replace a version without dropping requests.
- Survive the loss of a whole machine.
- Spread containers across machines, and add machines under load.
- Send traffic only to instances that pass a health check.
- Roll back on its own when a new version never becomes healthy.
Compose on one host does list one well and almost nothing on list two, because every item there needs a second machine and a dispatcher deciding what runs where. ECS and Kubernetes are both dispatchers. So the real question isn't "Compose or Kubernetes?" but "do I need list two yet?". If not, Compose is enough. If yes, you pick a dispatcher, and Kubernetes is rarely the first answer.
What does Docker Compose give you on one host?
The repository has two Compose files. compose.yml runs only the infrastructure: Postgres 17, a single Kafka 4.1 node in KRaft mode, Redis 8.2 with persistence off, and the gateway (Kong in DB-less mode). compose.dev.yml adds all thirteen services, each from its own image, plus a one-shot migration container per service. Three habits in those files are worth copying.
One anchor for shared defaults. Every service inherits a restart policy, a network and a dependency on a healthy Postgres:
x-service-defaults: &service-defaults
restart: unless-stopped
depends_on:
postgres:
condition: service_healthy
networks: [dev]Ports bound to loopback. Every published port looks like 127.0.0.1:3005:3005. On an EC2 host, that means a database port you forgot to remove isn't reachable from outside, even if a security group is too loose.
Required variables fail loudly. Secrets with no safe default use ${NAME:?NAME is required}, so docker compose up stops with a message instead of starting a service with an empty key.
Is "started" the same as "ready" in Docker Compose?
No, and that's problem one. In the dev file, the infrastructure containers have health checks and the services don't. So a service that calls a peer can only wait for condition: service_started, which means "the process exists", not "it is listening". A NestJS service needs a few seconds to connect to its database and bind its ports.
Here is the race on its own. Save it as race.yml and run docker compose -f race.yml up --abort-on-container-exit:
services:
auth:
image: node:24-alpine
# Listens only after five seconds, like a service warming up.
command: ["node", "-e", "setTimeout(() => require('node:http').createServer((req, res) => res.end('ready')).listen(3000), 5000)"]
healthcheck:
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:3000"]
interval: 1s
retries: 30
order:
image: node:24-alpine
depends_on:
auth:
condition: service_started
command: ["node", "-e", "fetch('http://auth:3000').then((res) => res.text()).then(console.log)"]order dies with fetch failed, caused by ECONNREFUSED. Change service_started to service_healthy, run it again, and it prints ready about five seconds later.
The real fix needs something to probe, and every service already has it. A shared NestJS controller serves /health/live, which only says the process is up, and /health/ready, which returns 503 until the service's gRPC health status reports SERVING. Point a Compose health check at /health/ready, make peers depend on service_healthy, and problem one is gone. Keep the two apart: liveness answers "should I restart it?", readiness answers "should I send it traffic?".
Why do migrations run in their own container?
Each service has a twin, such as order-migrate, built from the same Dockerfile but a different stage (migration instead of runtime). It runs prisma migrate deploy and exits; the runtime image never migrates on boot. Once there are two replicas, both would otherwise race to migrate. A one-shot job makes "change the schema" and "start the code" two steps you order on purpose, and the ECS pipeline later keeps the same split.
What else does a single production host need?
Docker enabled at boot, so the restart policy survives a reboot. Log rotation, because the default json-file driver doesn't rotate unless you set max-size. A memory limit per container, or one leaking service lets the OOM killer pick a victim from the whole host, possibly Postgres. Backups that leave the box. All of it is still list one.
What do you give up on a single host?
Now problem two. Probe any route that reaches a service through the gateway, then redeploy that service:
# Terminal 1: one request every 200 ms, print the status code.
URL=localhost:8000/orders/health/ready # any route that reaches the service
while true; do
printf '%s %s\n' "$(date +%T)" "$(curl -s -o /dev/null -w '%{http_code}' "$URL")"
sleep 0.2
done
# Terminal 2: ship a new version of one service.
docker compose up -d --force-recreate orderYou'll see a run of 5xx codes between the old container stopping and the new one becoming ready. Compose recreates a container by stopping the old one first, and with one container per service there's nowhere else for traffic to go.
You can script around it with two replicas recreated one at a time, but that script is the first piece of a dispatcher written by hand.
The rest of the price:
- The host is the failure domain. One instance, one availability zone, one disk. Lose it and every service, the database and the broker go down together. A restart policy can't restart a machine that's gone.
- Kafka has no replicas. One broker means replication factor 1, and restarting it pauses every producer and consumer. The transactional outbox keeps that pause from losing events, since they wait in Postgres, but it's still a pause.
- The database shares the box. The database-per-service split already sits on one Postgres cluster. On one host, that cluster also shares CPU, memory and disk with thirteen Node.js processes.
- Scaling is vertical only, and resizing an EC2 instance means stopping it.
For a prototype, an internal tool or a first few users, that price is often fine. Pay it knowingly, and decide in advance what would make you stop.
Docker Compose vs ECS vs Kubernetes: what actually changes?
| Compose on one EC2 | ECS on EC2 | Kubernetes (EKS) | |
|---|---|---|---|
| Failure domain | One host | Many hosts and AZs | Many nodes and AZs |
| Redeploy | Stop, then start | Rolling, health-gated | Rolling, health-gated |
| Bad new version | You notice | Circuit breaker rolls back | Rollout stalls; you roll back |
| Service discovery | Compose DNS | Cloud Map | Cluster DNS |
| You operate | One VM | Instances, task definitions | Upgrades, add-ons, manifests |
In van terms, Compose is one van. ECS is a depot whose dispatcher follows AWS's rules: you hand over a manifest and accept their way of working. EKS rents you the dispatch office, but you still hire and train the dispatchers: an ingress controller, secret injection, node autoscaling, upgrades on a schedule. Kubernetes gives you the most control over dispatch, and that control only pays if you'll use it. (The rollback cell is deliberate: a stalled Deployment keeps its old pods serving, but Kubernetes doesn't undo it for you.)
When should you move off a single host?
Move when a named signal shows up, not when the diagram starts to feel small:
- The redeploy gap becomes visible. Users notice the
5xxburst, or you start deploying at night to hide it. - Losing the host becomes unacceptable. Someone writes down a recovery-time target that "rebuild the box from backup" can't meet.
- One machine stops being enough. You've resized once, and the next size up costs more than spreading the load.
Any of these points to the next rung: a managed dispatcher.
Why ECS before Kubernetes?
The step to any dispatcher covers most of list two, and ECS gets there with the least new machinery: AWS runs the scheduler, there's no control plane to upgrade, and it uses the IAM, load balancers and secrets you'd use anyway.
Climb to Kubernetes when you need something only it has: operators that run stateful systems for you, manifests you want to carry across clouds, or tooling your team already knows. "We might need it later" isn't a signal. The same image runs on all three rungs, so when you climb you rewrite the deployment description, not the code.
That holds only if every image is honest, booting from its own artifact with nothing borrowed from the workspace. It's why the Nx monorepo copies every build out and boots it alone in CI.
What does the ECS setup look like in the repository?
The second rung is written as Terraform in four root modules: bootstrap (state bucket and pipeline identity), ecr (image repositories), platform (network, ECS cluster, load balancer, RDS, secrets) and services (one ECS service per application, plus Kafka, Redis and the gateway). Tasks run on an Auto Scaling group of EC2 instances, each task with its own network interface; a validation even rejects burstable t instances, which don't support the ENI trunking that so many interfaces require. Rung one never asks you to know facts like that.
How does ECS redeploy without a gap?
Each application service defaults to one task, a minimum healthy percent of 50, a maximum of 200, and a circuit breaker that rolls back:
resource "aws_ecs_service" "service" {
for_each = local.service_catalog
desired_count = var.service_desired_count # 1
deployment_minimum_healthy_percent = var.deployment_minimum_healthy_percent # 50
deployment_maximum_percent = var.deployment_maximum_percent # 200
deployment_circuit_breaker {
enable = true
rollback = true
}
}Do the arithmetic, because this is where the dispatcher earns its keep. ECS rounds the minimum up and the maximum down. With one desired task, 50% rounds up to 1, so the old task must keep running; 200% is 2, so a second task may start beside it. ECS starts the new version, waits for it to pass health checks, and only then stops the old one. The spare van leaves the depot before the old one is called in. If the new version never becomes healthy, the circuit breaker rolls back to the last deployment that completed.
Each container's health check calls /health/live every 30 seconds (5-second timeout, 3 retries, 30-second start period), the load balancer checks the gateway's /health, and services find each other through a Cloud Map private DNS namespace.
Which service still has the gap?
Kafka. It runs as a single task with the reverse settings, minimum 0 and maximum 100: stop the old broker, then start the new one, because two copies of one broker can't run side by side. The second rung doesn't make Kafka highly available; it makes the services around it roll cleanly. Multiple brokers or managed Kafka is a separate decision with its own price.
Is written Terraform the same as built infrastructure?
No. The pipeline that would build all of this has twelve stages:
quality → plan_ecr → apply_ecr → publish_images
→ plan_platform → apply_platform → bootstrap_database → migrate_services
→ plan_services → apply_services → verify → protected_operationsTwo things in it I'd copy anywhere. Plan to a file, apply that file: each plan job writes -out=<name>.tfplan as an artifact and the matching apply consumes exactly that file, so a newer commit can't slip in between. One apply at a time: the platform and services applies, database bootstrap and migrations share a resource group named after the environment, so two pipelines can't change the same environment at once.
More than a dozen terraform test files run against mock providers. They prove that expressions evaluate and validations fire, not that AWS accepts a request or that a task ever becomes healthy.
Here is the line I promised. There's no apply on record. The repository holds no environment outputs and no record of verify passing. The decision record for deployment topology still says Planned and defers the platform choice until service count, traffic, recovery targets and ownership are agreed. A stage that's defined isn't a stage that ran, and "running on ECS" would be a claim with nothing behind it.
The gap I'd close before the first real apply is the one at the bottom of the diagram. Today the apply jobs run automatically on protected branches. I'd put a human approval between plan and apply, tied to the digest of the exact plan file, mandatory for any plan that deletes resources or touches IAM or networking. The mock tests stay in front of that gate; they just can't replace it.
What does each option cost?
Compose on one host is the cheapest to run and to understand; you pay in redeploy gaps and manual recovery. ECS adds a load balancer, a NAT gateway, RDS, spare capacity for the second task during each deploy, and a lot of Terraform: the platform module alone declares 13 security groups and 25 rules. EKS adds a per-cluster control-plane fee and the people-time to run Kubernetes itself.
The in-between stage costs something too: two descriptions of one system that can drift. They already disagree once. The ECS task definitions health-check every service; the dev Compose stack doesn't.
What would I do differently?
- Health-check the services in Compose too, against
/health/ready, with peers onservice_healthy. The local stack would then behave like the target. - Check Compose and Terraform against each other in CI: same service list, ports and variable names. Drift should fail a build, not surface during an incident.
- Stop "live" from answering early. ECS gives a container one health check that both replaces tasks and gates rollouts. Pointing it at
/health/liveavoids killing tasks when a dependency blinks, but lets a new task look healthy before it's ready. I'd have each service bind its HTTP port only once its dependencies are up. - Price Fargate before committing to EC2 capacity. It removes the trunking and instance-sizing questions at a higher price per task.
- Keep the decision record in step with the code. The record says "defer"; the Terraform beside it has already chosen ECS on EC2. Either is defensible. Both at once confuses the next reader.
Key takeaways
- Deployment is two job lists. Compose covers "describe and start". "Keep serving while things change and break" needs more than one machine and a scheduler.
- Started isn't ready. Health checks plus
service_healthyfix startup races in Compose; keep liveness and readiness separate. - On one host, Compose stops the old container before starting the new one. That gap is the shape of a single host, not a bug.
- Move on named signals: a visible deploy gap, a recovery target one box can't meet, a machine that's maxed out.
- Choose ECS before Kubernetes unless you need operators, portable manifests or tooling only Kubernetes has.
- Written Terraform isn't built infrastructure. Plan to a file, apply that file, gate it, and never claim a stage that hasn't run.
Telling a stage that exists from a stage that ran gets harder when AI agents write the pipeline, which is where verifying an agent's work with evidence picks up.
Well done