Cloud Infrastructure from Zero to Production for a Data Analytics SaaS

Challenge

• No dev or production environments; every deployment was manual
• Data processing in Azure Databricks had to read and write a PostgreSQL database in AWS privately
• No access rules, no backups, no encryption policy
• No view of what the infrastructure costs
• An internal ML tool needed a large server that would sit idle most of the day

Solution

• Azure: Data Lake Gen2 storage and a Databricks workspace with VNet injection and Unity Catalog; users, roles and Key Vault access; a budget alert on the subscription
• A private Site-to-Site VPN between Azure Databricks and AWS RDS (PostgreSQL), taken down again in April together with every gateway and route it had needed
• AWS: separate dev and production networks, EC2, RDS, S3, ECR, IAM roles, KMS encryption, Route 53 domains, TLS certificates that renew automatically
• CI/CD in GitHub Actions: test → build a Docker image to ECR → deploy over AWS SSM. No SSH access and no long-lived AWS keys: the pipeline signs in with GitHub OIDC
• A health check after every production deploy with automatic rollback, plus a manual rollback workflow by commit
• Backups: snapshot policies for the servers; the production database and servers created from snapshots
• S3 access narrowed to the product's own domains; SSM sessions encrypted with KMS
• An internal ML tool on a large server (8 vCPU, 32 GB) that starts on demand from a link and stops by itself after an hour idle
• Email deliverability for the client's domain: SPF, DKIM, DMARC
• Incident work: traced repeated hangs of the dev backend to a single-threaded dev server overloaded by frontend polling, and a CPU spike to one unindexed database query polled every two seconds — and gave the backend team the evidence and the fix to make
• Cost tracking per service and per environment from AWS Cost Explorer, updated monthly
• Architecture documentation for both clouds and a handover document with the full resource inventory

Result

Every release goes through one pipeline and rolls itself back if the health check fails. Costs are tracked per service and per environment, and the whole setup is documented and handed over to the client.