themcgruff

I built a similar Ruby based agent development kit that has a different focus and feature set:

https://github.com/tweibley/legate

I did it to learn about rlms. But also I've tested it a bit with some openclaw style bots and it works well.

Multiple things recently:

https://github.com/tweibley/rubyrlm - MVP Ruby implementation of Recursive Language Models (RLMs) that uses Gemini as the model backend and a Ruby REPL for iterative reasoning.

https://github.com/tweibley/legate - Framework for building AI agents in Ruby with dynamic tool selection, multi-step planning, and session management

https://insightclips.com - Create personalized event videos (with optional CTA) — from first promo to final recap automatically.

https://freshdeck.app - Presentations that design themselves.

https://yourslidessuck.com - Get your pdf/slide deck roasted for free.

https://seriouslygreatjob.com - Put someone on the "news" for free.

https://www.sayayeaye.app - IOS/Mac OS client for Google's Jules.

(I run our Ops team.) The time burden goes down with the more people we have. It's still a burden though -- and our compensation (in theory) reflects that.

FWIW We're doing everything we can to make this "work hours only" M-F, which we could solve by hiring tons of people immediately, but we also have other ideals like keeping the company as small as possible that we want to realize too.

There's an open and ongoing discussion about making improvements in this area and I'm thankful that David and Jason have been receptive to many of the suggestions I, or anyone else on our team has had.

My personal stance is that we should do everything we can to give Ops a 40 hour work week that's during regular working hours and no more, even if that means people get cut a lot of extra slack to recover after a late night page, etc. (Hopefully our team would back me up in saying I encourage people to take reasonable time to make up "lost" hours.)

(Also fwiw, I participate equally in the on call rotations.)

Thanks for your feed back. It's understandable that this might be too much for some people.

To date, we've had a single person skip those questions and still submit. This person was not a qualified applicant.

Based on the applications we've received this very minimal list of requirements appears to be working as desired. We're interested in employees who are going to do the best work of their careers. I'm okay with having a slightly higher bar (in terms of application effort) for that type of person.

Basecamp | Ops/Sysadmin| Chicago, IL | REMOTE, Full-time, https://basecamp.com/

Basecamp solves the critical problems that every growing business deals with. We say it’s the saner, calmer, organized way to manage projects and communicate company-wide.

Basecamp Ops is responsible for infrastructure across 3 colocation sites in the United States and use both Google Cloud and Amazon Web Services too. We're heavily a Ruby on Rails shop though there's a few other languages hanging around in our deployments. If you are passionate about delivering fast and reliable sites at an awesome company that will respect you and help you grow personally and professionally please get in touch: https://basecamp.workable.com/j/A5A189B311. (Oh yeah we have amazing benefits too: https://m.signalvnoise.com/employee-benefits-at-basecamp-d2d...)

(Full disclosure I work with the author, Noah, at Basecamp.)

Not sure if more important than math, but you are right it's extremely important.

One of the biggest on going problems we have isn't getting data in -- it's helping everyone get data out. Even with training sessions, documentation, and some fairly fleshed out "self help" tools, there is still confusion about where to look and how to "find" and "combine" the right data to answer a given question.

One of the ways we've partially solved this problem is through Tableau which is a commercial solution. (I was skeptical about whether we would stick with solutions like Tableau but it has been worth every penny.)

Basecamp (formerly 37signals) - Chicago IL (But you can be anywhere. REMOTE, FULL TIME)

Android Lead Dev - https://basecamp.com/jobs/android Marketing Designer - https://signalvnoise.com/posts/3841

We treat all environments as production. However we add additional security to staging since it's the most experimental. (Additional security meaning it can only be accessed via VPN and after two factor authentication.)

To be very clear: Beta, production, rollout and staging environments are secure. They run all the same front end security provisions, same data center, same patches, etc. The difference is they run differing versions of the code base based on development.

(I do ops at 37signals...) We have two top of the line ("high end") load balancers at each site. They malfunctioned. When they both malfunction at once, it doesn't matter whether it's active/active or active/passive or anything like that.

In the operating room, where some would argue where procedural checklists like this might most count, they use a pre-op timeout procedure. Usually this is to ensure the right patient is being operated on, in the right place(s), and that the right operation is being performed. The same happens before the patient is "closed". A count is taken of every bit of material / tooling used in the procedure to make sure nothing is "left behind" (in the patient). Sources: 1.) http://www3.aaos.org/member/safety/guidelines.cfm for more information. 2.) My wife who was a surgery resident.

We talked with some popular "mysql consultants" about things like mysql-master-ha and there was always a lot of hand waving murmuring about how existing solutions were incomplete / had lots of bugs / were not to be trusted in production.

It looks to me like they do mostly the same thing. We wrote our script in Ruby because it's what we know and have the most expertise with, which makes it easy for us to debug.

mysql_role_swap is not black magic, it doesn't do automatic failover, it doesn't run in the background / daemonize itself. It fails gracefully in a way we find predictable and useful.

We write out the cluster config and update the script using Chef.

In an emergency situation, chef-client is a wee bit slow ;)

Also we like having the script standalone so there's just one moving part.

The BMC is horrible. It needs a cold reset frequently, and it often doesn't work at all when access from a non Windows machine. Otherwise they work really well. I like that the sleds are able to be removed from the front too.

(I'm the author of the post).

These aren't blades and I dislike blades for the hardware/vendor lockin reasons you mentioned. Blades usually share power, network, etc. These only share power + baseboard management controller. We can lose a single power supply and still keep on going. We've distributed the applications over multiple chasis, just in case though.

Mutex issues and other problems with high insert / update rates. Google "mysql query cache issues" and "mysql query cache disable" for lots of posts about the various costs / benefits.

Most, if not all production mysql installations have this cache disabled anyways. (Based on my experience at Engine Yard for 3 years and other places the last 5 years.)

There's two of us at 37signals building projects on top of the HttpLua / ngx_lua module. The OpenResty (http://www.openresty.org) project is absolutely worth a look if you are willing to live on the edge and you wan't incredibly fast performance. I can't say enough good things about the work agentzh and chaoslawful have been up to lately -- just check out their Github profiles.

Copied from Noah's comment on the post (I asked him first):

"We do track open rates for emails that are already HTML formatted and making remote requests for images, but you’ll never get 100% accuracy with that metric because many people use plain text emails or don’t load images. Our experience is that the best you’ll ever see is between 60-70% “open” rate because of this. Some of our applications in some contexts only send plain text emails as well, so we don’t track open rate there at all.

Why is remote server acceptance rate important anyway? Some thoughts:

1) First, because hard bounces really do happen a lot, and at our scale, a 1% difference in hard bounce rate means 160k messages per week that aren’t making it to users, which means a poor experience for many and many support requests coming in to us. Based on all the information we’ve been able to find, we’re pretty sure a 0.7% hard bounce rate at our scale is pretty good.

2) Second, because it is a relative metric of overall deliverability. Our experience has been that when we do get on a blacklist, servers start hard bouncing our mail until we get off of it. As we’ve improved our SpamAssassin type scores over the last few years, we’ve seen an improvement in hard bounce rate. We also see a strong correlation between hard bounce rate and the number of email delivery related support requests we get. While it’s not the perfect measure, it’s the best measure we have available that we can reliably monitor.

Again, there’s no perfect way that I know of to reliably tell whether an email is getting to a user, since read status isn’t particularly accurate. We use whatever we can (hard bounce rate, open rate, number of support requests relating to email) to get as close to that as we can."

You comment reminds me that we left out a biggie: We include a "stop sending me these messages" link in almost every email we send. The link actually works too.

I know this wasn't clear from the post... We include a "fake" image (with a unique has for that specific message) in all html email so we know if it's "really been delivered" (ie it was actually opened).

FWIW we haven't done any "paid" whitelisting. That's too mafia-esque for us.

SES requires each sending address to be verified upfront. For us that's an issue.

I'll talk with Noah about open sourcing it. Right now it would take a little bit of effort, but it's probably worth it for the good of the community.

It's not worth outsourcing. We just add a tiny gif that has a unique hash. When that file is requested record the hash and mark the identified message as read.

That should read we "check a repo". We don't actually do a git checkout.

We use this script: https://raw.github.com/37signals/37s_cookbooks/edefbd17eeb8f... to check spamhaus, abuseat, sorbs, etc.

Yes. Sorry used the package name(dkim-filter). It's a milter though.

At 37signals:

We send 40M - 100M emails from our applications each month. We run 3 outbound relays on top of Postfix. We sign about 95% of our outbound mail with Domain Keys using dkim-filter. We also use SPF records for all of our sending IPs / domains. We monitor our sending IP reputation with via Nagios checks and manual review. Use also have checks setup for the most common black lists. Every email that goes out from our applications has an "opt out" link. (This is really important!)

We used to send all of our mail via one IP address. That meant any given spam report could really hurt our deliverability. Today we send from about 10+ IP addresses, divided amongst the apps. If the reputation of one IP goes down because of a false report, we just decrease the volume on that IP until the reputation goes back up.

For our "company" mail we use Google Apps. For our "marketing" mail use Mailchimp.

As DHH commented on the post: The file name was only shared because it was funny and not identifiable with anything, and just seen passing through the logs. We do not look at people’s files and we do not share any personal information of any kind.

Had it been BankruptcyPreparation.docx, or any other sensitive file name I would never have shared it. Going forward I won't be sharing dog.gif either.

The file was named cat.jpg and that was logged, which was what we saw. We do not look at user’s files.