cloud providers

The main cloud providers for the ’20 decade are aws, gcp and azure but digitalocean, linode and others are used too. Here is a good overview on aws.

App deployment on-prem vs AWS

On prem: I launch an instance from my cloud provider, I log into it with ssh, I install the libraries and launch the application, install and configure nginx, create certificate with certbot and rerun nginx. AWS:

filesystem

Filesystems are a great option for backups, cheap cold storage. Big files are usually stored in parquet

query engine

orchestration

spark

storage

relational

document

instances

ML ops

warehouse, query

load balancing

data integration, ETL, batch processing

logs, tracking

cloud monitoring

cloud formation

messagging

business intelligence