Operational Excellence is all about doing things The Right Way, and that's much different for this small personal site than a corporate intranet or public webapp. It would be fun to overbuild this site: we could add users through Cognito, and give them very specific RBAC permissions, and slurp in all their information into Firehose and deposit that data in a data lake, and send alerts each time an action happens... Use your imagination!
But that's not Operationally Excellent.
AWS’s current OE principles are:
observability
What's important about a small personal site like this? Mostly just that it's up.
Every 5 minutes a lambda GETs https://www.devney.net/ and https://www.devney.net/about.html. Both must be 200. There is a custom User-Agent so you can spot it in access logs. Rather than monitor if a service is up, we directly confirm that the page is accessible. There are several other alarms and metrics, all showing in a dashboard. SNS allows any alarms to fan out to me myself and I.
safely automate
The environment is code. All AWS resources are managed by terraform, and all code is stored in GitHub. Any change to the environment is a change to the code, then a PR, then a review, then a merge. Rollout is done via terraform apply. EventBridge runs the boring backend.
small reversible changes
Each page is a file, so any change is a small change. The terraform-based deployment is fast and safe and reversible. S3 versioning provides a second avenue of recovery.
refine procedures
It's just me. I have tried to build this site according to best practices. There's just not much to refine until something changes.
anticipate failure
The code for this website anticipates at least 11 failure modes. There are also some failure modes that are identified but not resolved. None of this has gone through a gameday or DR exercise yet.
use managed services
This site does almost nothing on its own. It's mostly a small wrapper around AWS managed services such as S3, Lambda, EventBridge, CloudFront, ACM, CloudWatch, SNS, etc. Operational Excellence is about doing the things that are worth doing, and not necessarily doing everything that's possible.
use best practices
The code for this website is written partially as an example of best practices. To that end I have made the source available on GitHub so we can all learn from each other.