HN user

Nilithus

78 karma
Posts0
Comments20
View on HN
No posts found.
Neovim Pack 11 months ago

I switched from packer to lazy after I noticed the maintainer of packer stopped using packer in his neovim config. Also I believe he has now officially marked it as unmaintained and recommends lazy or pckr

I don’t think that’s really fair. They are highlighting some pretty serious security flaws in MCP tools that are allowed to do some pretty privileged things.

They don’t even mention their product till the very last section. Overall think it’s an excellent blog post.

I found Amazon's Builders Library to have a very insightful list of how to handle building multi tenant queue systems https://aws.amazon.com/builders-library/avoiding-insurmounta...

I think one of the big tools is Shuffle Sharding. The article talks about standard sharding by itself as not being enough to provide robustness in multitenant queues. But Shuffle Sharding I.E. assigning users to virtual groups of underlying queues and enqueueing to the queue with the smallest size gets you pretty far. It can limit throughput for individual users but implementing some simple work stealing logic on the consumer helps make sure you keep your throughput up.

The Cloud Computer 3 years ago

I guess there must be a largish market for this since AWS introduced Outpost to provide the "cloud" to onprem industries. I feel like this is competing with that market.

Since many of those use cases probably already run extensive on-prem infrastructure this could appeal to them. AWS outpost talks about industries like healthcare, telecom, media and entertainment, manufacturing, or highly regulated spaces like financial services. I've heard of media companies that process through things like IMAX cameras that have just tons of TB's of data sometimes just for 5 minutes worth of footage. That would simply be too cost prohibitive - in bandwidth alone - to try and move around in the cloud and you don't want to have to wait for things like AWS snowball or whatever.

While I think the space is "niche" those niche spaces are not small. Big companies with big budgets.

The Cloud Computer 3 years ago

Perhaps this changes the calculus. If you don't need as much of an ops team to go private cloud. Then you might not need to be as large before it makes sense.

I was wondering anyone could shed some light on the "These hash functions should be pairwise independent" part. I don't know what pairwise independent means. My background is mostly in web so I am familiar with things like `sha256`, `sha1`, `md5` things like that. Would you use these kind of functions for Count-min Sketch?

I saw in your socklimit project looks like `fasthash64` and `hashlittle` I'm not familiar with those any insight or recommended reading to understand these hash functions?

p.s. googling pairwise independent hash functions did get me some college class reading but doesn't mention any named hash functions developed out in the world.

Terraform 1.0 5 years ago

"Luckily" Terraform's performance is so bad that you need to split the stacks anyways

Not sure what about terraforms performance is so bad. Seems hard to blame a tool who's main execution path is potentially 100's of network IO requests with 3rd party API's. Most of the "split stacks" I've seen is more for code organization and security reasons rather than performance. Seems safer to know 100% that deploying infra for my app isn't going to mess with my VPC settings and can be executed with a lower privileged role.

Furthermore you can no longer effectively plan any change that affects the boundary between the two stacks.

That's fair -- you do end up with these "foundational" modules a lot of the time. Like an 'aws-account basics' module or something that other modules expect the account to be setup with that base for being able to query data objects for subnets ect... planning changes if that changes be difficult but not impossible. Good versioning is critical. Feels in the same vein as apps that need to manage framework updates and things like that. (though can be made more difficult or easier based on how you've broken up using your cloud provider -- multiple accounts by buisness unit or all in one).

I think a lot depends on the environment you're working in / migrating from. If you're working from an on-prem datacenter with VM's and you're not moving to the cloud for whatever reason. I think your point of not having to hand roll config managment, secrets management volumes ect.. is a good one and kubernetes makes a lot of sense in that environment. Or using kubernetes as a target for vendor apps that may need to run in unknown or exotic environments.

But if you're already in the cloud most of the k8 solutions already exist as individual tools, with little operational maintenance needed. "I need 4GB RAM, 2 cores, I need access to the postgres secret to talk to it, and here is my hostname" is all achievable with lambda's or ecs fargate task definitions and parameter store. Managed services for queues and databases lower operational burden. All of this while not having to risk a cluster upgrade bringing everything down or now having to increase operational overhead by running multiple clusters to have some operational blast doors.

All of this to say if kubernetes is working for you thats great! Congrats. But I'm on the hill with OP. Most platform/infrastructure engineers that I've heard pitch kubernetes because they are trying to solve one problem I.E secrets management or deployment ect... not all of the problems at once and bring in kubernetes complexity when there is already a fully managed, safer and cheaper solution to migrate to doesn't usually make sense.

There's no 100% right or wrong answer but I think you really have to evaluate what you're migrating from when evaluating moving to kubernetes