3 July 2021
Once, in a rush to get feedback from an external partner, I deployed a change that seemed pretty innocuous. I then headed out for 5 days of camping and rafting the Snake River in Idaho, and was unreachable. My code brought the site down, and my colleagues had to clean up my mess while I was incommunicado. Upon my return, I was horrified to read the Slack backlog account of their discovery of the issue and subsequent fixing of it. My colleagues were gracious; all of them mentioned how it was a bit of a rite of passage. Since then, we've consistently added layers of precautions to make it harder for new devs to have the same deleterious effect.