So hand-wavy ; you still need #OnCall for critical services with defined SLAs :

“Breaking Up With On-Call”, Alexey Karandashev (https://reflector.dev/articles/breaking-up-with-on-call/).

Via HN: https://news.ycombinator.com/item?id=43378671

howdy, #hachyderm!

over the last week or so, we've been preparing to move hachy's #DNS zones from #AWS route 53 to bunny DNS.

since this could be a pretty scary thing -- going from one geo-DNS provider to another -- we want to make sure *before* we move that records are resolving in a reasonable way across the globe.

to help us to do this, we've started a small, lightweight tool that we can deploy to a provider like bunny's magic containers to quickly get DNS resolution info from multiple geographic regions quickly. we then write this data to a backend S3 bucket, at which point we can use a tool like #duckdb to analyze the results and find records we need to tweak to improve performance. all *before* we make the change.

then, after we've flipped the switch and while DNS is propagating -- -- we can watch in real-time as different servers begin flipping over to the new provider.

we named the tool hachyboop and it's available publicly --> https://github.com/hachyderm/hachyboop

please keep in mind that it's early in the booper's life, and there's a lot we can do, including cleaning up my hacky code.

attached is an example of a quick run across 17 regions for a few minutes. the data is spread across multiple files but duckdb makes it quite easy for us to query everything like it's one table.

hello, #SRE friends. looking for good resources for SRE-related content, specifically for #observability please share your fave resources, TY

Saw this very nice article on slack about making standalone #python scripts that use #uv to automatically create their own virtual environments and install their dependencies there.

https://thisdavej.com/share-python-scripts-like-a-pro-uv-and-pep-723-for-easy-deployment/

#sre @sre #devops @devops

There'd be a lot less whining from devs about 'needing' root in prod if K & R had named that account janitor

@sre @devops #sre #devops

Okay, I've been asked this question to which I was not prepared: where do you go if you want to learn Kubernetes from the ground up? Consider a person who doesn't even know how to run "docker" in their own laptop, but they want to rise to a point where they can actually manage a Kubernetes cluster. Thoughts?
#HomeLab #Kubernetes #SysAdminLife #k8s #SRE

Anyone looking for a developer with crazy fullstack experience (back to the 1990s!), about two decades professional experience with both #Python and #Django, very solid with JS/frontend, some #SRE background, an absurd love for #Lisp, and who's an enthusiastic loony for new tech?

I'm your girl!

I'm looking to jump ship to somewhere much more fulfilling, and I'd love to check you out as long as you're diversity-friendly.

#fedihire #fedihired #getfedihired

My younger daughter wants to interview for a Lead Staff SRE position at a new company (but for an old boss). She'd like some ideas of what she might expect.

This is work she's currently doing at a large company, but due to the way her career has unfolded, she's never actually done a coding interview before.

There's 2 rounds, qualifying, and final.
Qualifying is leetcode stuff, data structures etc.
Final one is 4 parts, each one hour:

• behavioral
• Python/code review
• system design
• Linux

She's worried about coding for someone for the first time. And what algorithms/data structures they'll want (she doesn't have a formal CS background). She also isn't sure (nor am I) what the Linux segment would be likely to cover.

I know she has the skills and can talk through the problems well. She's survived lots of layoffs because she's the one who makes everything work and builds strong teams with focused goals. But now she's the last person left in her group and training folks in India, so the clock is ticking.

I can DM a link to the job description if that helps. Would rather not publish details publicly.

Thanks.

OH on slack:

every day a fresh yak to shave and while the ones from yesterday still have mullets

@sre @devops #sre #devops

OH:

If the computer can lie to me, it's moral to lie to it

#devops #sre @sre @devops #ai

howdy, folks - it's been a bit since our last #hachyderm infra check in.

stuff in motion:

- ditching #terraform cloud & tf for #opentofu and #atlantis. we are just about to import our dev environment and put it through its paces.
- bringing #postgresql under ansible management. the team has been doing awesome work, and we've started to spin up dev nodes using the new playbooks. soon: production!
- moving #DNS zones away from AWS route 53. we chose bunny DNS as our provider and have been doing basic tests in dev. we'll likely prep our records for production this week with a plan for a cutover in one of the coming weekends.

and if you filled out our volunteer form and haven't heard from me in a bit - you're still on the list. we'll onboard a new batch of folks in the next couple of weeks.

@hachyderm

Life of #SRE as a Salesperson

https://tech.trivago.com/post/2024-12-20-life-of-sre-as-a-salesperson

My employer is an EFF-approved independent ISP, and we're hiring a Senior Systems Engineer https://grnh.se/a6b53dbd1us
Key technologies involved: Linux (RedHat), MySQL/MariaDB, VMware,Proxmox, Ansible, Perl, Python and Shell.

Salary starts at $145k. Can be fully remote (within US), but preference if able to work in our northern california office one day a week.

See link for details (and better accuracy).
#syseng #sysadmin #sre #devops #hiring

Bueno, venga, va, nunca he hecho #presentación a pesar de que me he mudado más veces de las que puedo contar.

Soy #DevOps, #SRE o #PlatformEngineer, depende de cómo vaya el día. Mi trabajo no me define, pero me flipan los ordenadores desde lechón, así que un poco si.

Me gusta la broma y la chanza, pero si te ofendo con algún chiste, avísame para cambiarlo.

No me interesaba la política hasta que LOS PUTOS FASCISTAS me obligaron.

Para lo demás, la bio, o pregunta.

OH on slack, re why some things never come out of backlog

We can't say "your crap is not important enough". We say "look there are more important/urgent tasks right now. We'll keep yours on the backlog and come back to them"

@sre @devops #sre #devops

oh on slack:

code isn't an investment, it's a liability. It rots and requires maintenance for new security vulns, to work with the latest software, etc.

#devops #sre @devops @sre

Hahaha
https://github.com/learnk8s/xlskubectl

Nothing makes my day like a great post incident review!!!

And that's better than any metric you can shake a stick at.