Skip to main content

Production Install Runbook (Fresh Server)

This runbook is the manual, production-grade process to bring up a brand-new server from scratch.

If the server is already bootstrapped and reachable as externaladmin on the hardened SSH port, do not use this runbook. Use the deploy/update flow referenced from Deployment.

Assumptions​

  • You have a Debian/Ubuntu VPS with a public IP.
  • DNS is under your control.
  • You start with provider access: SSH root@<host> on TCP port 22 using a temporary password.
  • Final access will be: SSH externaladmin@<host> on TCP port 36987.

Inputs you must have​

  • IP_OR_DOMAIN: the server IP or DNS name (example: sd1.desarrolloselectronicos.com)
  • Provider temporary root password (used only for bootstrap): .env.secrets REMOTE_ROOT_PASSWORD
  • Desired admin password (used for sudo during deploy): .env.secrets REMOTE_EXTERNAL_ADMIN_PASSWORD
  • Your SSH public key (recommended): .env.secrets REMOTE_SSH_PUBLIC_KEY_PATH

1) What you must do first​

Before you run any install commands, do this checklist in order.

DNS (your DNS provider)​

You must create DNS records in your DNS provider/zone (this repo cannot do that for you).

Print the hostnames you need to create from your current config:

./infra.sh remote domains

Create A/AAAA records for each hostname (pointing to the server public IP).

Wait until they resolve:

host <any-endpoint>.<domain>

Provider firewall / security group​

Allow inbound:

  • TCP 22 (temporary; bootstrap only)
  • TCP 36987 (final SSH port)
  • TCP 80, 443 (HTTP/HTTPS)
  • any other public ports you intentionally expose (optional)

If this checklist is not green, stop here and fix it first. Otherwise you'll hit noisy, misleading failures later.

2) How to run the process​

Configure local env + secrets​

These are the canonical sources of truth used by scripts and Ansible:

  • Topology/ports (non-secret): deployment/topology.env (generated from template)
  • Public endpoints/domains (non-secret): deployment/remote.env
  • Secrets (never commit): .env.secrets

Choose ONE path: Option A OR Option B (do not do both).

Option A (recommended): guided wizard (standard flow). This walks you through the same steps as the manual path (env -> server registration -> bootstrap -> deploy) and then runs basic verification checks.

./infra.sh remote init

If you choose Option A, you can skip the manual steps below.

Option B: run the manual steps below:

Generate and validate locally:

./infra.sh env generate
./infra.sh env check

Important note:

  • These env files are used from your local machine to drive Ansible and generate configs.
  • remote bootstrap does not "generate env on the server".
  • remote deploy syncs the repo and seeds missing per-project .env files from .env.example/.env.template when needed.

Register the server (inventory)​

./infra.sh server add <IP_OR_DOMAIN> --user root --port 22
./infra.sh server list

Bootstrap (one-time hardening)​

Bootstrap uses the initial provider access (root@22 + temporary password) to harden the server and enable key-based access.

./infra.sh remote bootstrap --host <IP_OR_DOMAIN>

After bootstrap, you should be able to connect on the hardened port:

ssh -p 36987 externaladmin@<IP_OR_DOMAIN>

Deploy (repeatable)​

./infra.sh remote deploy --host <IP_OR_DOMAIN>

3) How to verify it worked (and lock it down)​

Prefer SSH keys (no password) for server access​

Before you consider the server "production-ready", ensure you can SSH with keys (no password prompts).

If you provided REMOTE_SSH_PUBLIC_KEY_PATH (or SSH_PUBLIC_KEY), bootstrap should have installed your key. Verify:

ssh -p 36987 -o BatchMode=yes externaladmin@<IP_OR_DOMAIN> "echo ok"

Operational checks​

On the server:

ssh -p 36987 externaladmin@<IP_OR_DOMAIN>
sudo ufw status verbose
sudo systemctl is-active docker nginx fail2ban
sudo fail2ban-client status sshd
sudo nginx -t
docker ps

From your local machine:

./infra.sh remote deploy --host <IP_OR_DOMAIN>
./infra.sh remote logs <project> --host <IP_OR_DOMAIN>

If public endpoints exist, validate:

  • HTTPS certs (issuer + expiry)
  • app pages return expected content
  • only expected ports are open

Lock down bootstrap access​

After everything is stable:

  • close inbound TCP 22 at the provider firewall (keep only 36987)
  • keep a break-glass procedure using provider console access

Final post-install checks​

After the first successful deploy, review Backups and confirm the scheduler/snapshot process.

Confirm the cert renewal timer:

ssh -p 36987 externaladmin@<IP_OR_DOMAIN> sudo systemctl status certbot.timer

Common install failures (quick fixes)​

  1. Bootstrap fails to connect as root

    • provider firewall blocks port 22, or wrong password in .env.secrets
    • verify: nc -zv <host> 22

  2. Deploy fails on certificates

    • DNS not pointing to the server yet, or port 80/443 blocked
    • temporary workaround: ./infra.sh remote deploy -e enable_letsencrypt=false

  3. Deploy sync is slow or fails

    • check disk space and network on the server
    • verify: df -h and journalctl -u ssh -n 200 --no-pager