A year ago I couldn’t tell you what a PVC or ingress was. Today I’m recovering from broken clusters, migrating core services, and — most importantly — actually feeling confident. 🚀

#DevOps #learnk8s #100DaysOfCode #100DaysOfDevOps #homelab #kubernetes

- Stumbled upon mailgun, and am now using their free tier for my SMTP needs. Which aren't much.
- Deployed Komodo to monitor my docker containers.
#DevOps #learnk8s #100DaysOfCode #100DaysOfDevOps #homelab #kubernetes

I was experimenting with Longhorn and accidentally corrupted a few PVCs, including the one for Pi-hole. Since Pi-hole handles DNS inside my cluster, its downtime meant the entire cluster lost DNS. As a result, Infisical couldn’t resolve DNS, and my apps couldn’t access their secrets.

- Moved PiHole and Infisical (Secrets Manager) to external VMs outside of my K8s cluster. Not sure why I didn't do this when I initially built my cluster 🤷🏾‍♂️ . So what happened? #DevOps #learnk8s #100DaysOfCode #100DaysOfDevOps #homelab #kubernetes

- Fully migrated my K8s stack to RKE2. Having my apps "GItOps"ed made the process less painful then what it could've been.

#DevOps #learnk8s #100DaysOfCode #100DaysOfDevOps #homelab #kubernetes

In the end, I resolved it by just manually recreating the secret pihole was looking for.
✅

My secrets are managed by External Secrets, which was also failing because it couldn't resolve the dns name for Infisical because.... Pi-hole was down

After bringing my cluster back online, services were recovering except for a few. Pi-hole was complaining about a missing secret.

Oh wow! Thanks for sharing!!