alien_

This is the tool I use to generate and prepare for sending the PDF invoices for my freelance work.

It takes a bunch of JSON files that define info about my clients, as well as my own business details.

It then uses some of those JSON files to generates a PDF for the invoice, which it attaches to an email pre-populated with all the details, including the client’s finance email address, etc., which I just need to double check and send.

Might be useful for freelancers having multiple clients billed using various billing methods (hourly, results based, etc.)

Sorry about this, I hadn't noticed.

Unfortunately Beehiv added their tracking query strings which will break the site because of the way we host it from a static content bucket.

It's enough if you just delete the beehiiv query string.

I'll try to extend the CloudFlare worker we use for the redirect of / to index.html to also handle this.

Thank you for reporting this, I'll have a look ASAP.

This is besides the point, the AutoSpotting and EBS Optimizer tools are not even in the scope of this release, I just mentioned them for a bit of extra context about myself and what I've been building previously.

Feel free to criticize them, but that code saved thousands of customers over $100M over the years in aggregate, and was launching at some point a non-negligible percentage of the total AWS Spot capacity.

Regarding the OSS repos, I said they're also still available as open source, but evolved a lot since in the commercial version as stated in the readme of AutoSpotting.

This is the list of changes only available in the commercial version of AutoSpotting: https://github.com/LeanerCloud/AutoSpotting/discussions/489

For EBS Optimizer, the OSS code base evolved into the Optimizer tool released under the ONCE model as part of this bundle and it's been all but rewritten.

Regarding the lack of tests, the code base has seen many changes since, and critical areas have unit tests now.

When it comes to IaC drift, in case of AutoSpotting and EBS Optimizer the ASGs and volumes are expected to be tagged as per the expected tags, and you can do that through IaC to avoid drift.

The instances launched by the ASG can be replaced by AutoSpotting without creating any IaC drift, because the infrastructure code does not have any reference to the instances.

The same is true for EBS Optimizer, it is expected to run against EBS volumes created when launching instances. The IaC will not show drift because it's not keeping track of the underlying EBS volumes, only launching the instances.

But the Optimizer tool part of this ONCE release will indeed cause drift because it's changing the instance type of RDS databases and Elasticache clusters, which is captured in the IaC code. The drift will need to be resolved after the fact, which is what I do as part of my service work when using this tool for my customers.

These tools saved me a lot of time when saving my customers money, and the TF building blocks are just examples of a few patterns that I used for delivering infra for my customers and I've been thinking it may save some people time from building the same from scratch.

If you're not interested in these, fair enough, some people already bought these.

I'm a solopreneur doing AWS cost optimization tools and services, much like a Pieter Levels in the niche of Corey Quinn.

For many years I've been building self-hosted serverless headless AWS cost optimization tools:

- AutoSpotting.io - for adopting Spot instances in ASGs without configuration changes, no vendor lock-in, and with automated diversification and failover to on-demand - started as OSS alternative to the Spot.io Elastic Group product, still available as OSS at https://github.com/LeanerCloud/AutoSpotting.

- EBS Optimizer for GP3 converting EBS volumes to GP3, also available as OSS at https://github.com/LeanerCloud/EBS-Optimizer.

(I'm currently working on a few more, for example a NAT Gateway alternative and a tool to automate Savings plans purchases.)

Over the last year I started to offer cost optimization as a service, mainly in order to get more ideas for such tools.

Over multiple customer engagements I built almost a dozen CLI tools that can be used to optimize many of the AWS services at my customers. Using these tools I was able to deliver some 70% cost optimizations in average at my services customers over the supported resources, and sometimes as high as 60% of their entire AWS bill.

I also did a few DevOps engagements, out of which I developed a bunch of Terraform building blocks that can be used to create ECS Fargate or Lambda microservices with persistence on RDS Aurora Serverless v2, with automated CI/CD and other goodies included out of the box.

My main FinOps CLI tool named Optimizer was evolved out of EBS Optimizer over multiple customer engagements, and now does automated conversion to GP3 for EC2 and RDS storage volumes, as well as rightsizing and conversion to Graviton for RDS, ElastiCache and OpenSearch, among other things.

I recently started to offer Optimizer under the ONCE Source-Available license used by 37signals for Campfire, and I will gradually release all my CLI FinOps tools and Terraform building blocks under a large bundle of tools and building blocks.

(The Terraform building blocks scheduled to be released in a couple of days)

If you're looking to optimize cloud costs and/or build serverless microservices on AWS, you may want to check this out.

After the recent layoffs at Google that impacted open source maintainers of some prominent projects, I was curious to see what's the state of other projects where the community had to step up involvement after the initial project ceased their Open Source contributions.

I saw a few people made similar comparisons for other forked projects in the past, and as an infra guy myself, I immediately thought about OpenTofu, now that we're more than half a year after the fork from Terraform, they recently had a massive release, and also had some interesting shenanigans with Hashicorp's lawyers accusing them of stealing their precious IP.

Sharing my rollercoarter ride of monetizing a formerly quite popular AWS cost optimization OSS project I started 8 years ago and venting about my latest MRR drop.

Ask me anything!

For sure, I use the AWS APIs to gather the data but there's no single API for all this, I aggregate it from different services and across regions.

AWS also has some dashboard in the AWS console similar to this but it doesn't have cost information for the resources, it's not easy to track the resources that generate the costs, and you can't extract the data out of it for further automated processing.

They also have visibility for this in the Cost Intelligence dashboards, but you need those deployed in the account.

This tool runs from your local machine and you can point it to different accounts with just using a different AWS CLI profile, and you don't need to deploy anything in the target account.

I show it all on a single view across services(looks and feels much like a spreadsheet), each resource with its price tag next to it and also showing some metrics, like load balancer traffic numbers that should inform consolidation decisions.

Data export functionality is not there yet but we may add it soon if there's demand for it or someone contributes it.

And also why we had to deploy NAT Gateways in our public subnets.

Thanks for the comment!

You're right, it's amazing at that, I've had a similar experience.

At small scale you can also do much of that pretty well also with ECS, we use GitHub Actions for driving our deployments using Terraform.

The OP here, thanks for the comment!

It was definitely not about being contrarian but about offering first and foremost a more cost effective but still relatively simple, scalable and robust alternative to their current setup.

They have a single small team of less than a dozen people, all working on a single application, with a single frontend component.

Imagine instead this team managing a K8s setup with DNS, ALB and SSL controllers that each set up a single resource. I personally find that overkill.

The OP here.

What you say may make sense for a large corporation with hundreds of developers from many teams, all sharing a single cluster, but remember this is a pre-revenue startup with a single dev team of less than a dozen people.

But then with a large cluster you will struggle with splitting the costs. In such scenarios I'd rather give each team its own AWS account and have some devops people set up everything from the landing zone.

In this particular case, every service is set up from less than 100 lines of Terraform, which includes Docker image build and push, as well as the task and service definition that deploys that docker image.

yes, they need to handle Terraform, but it's really not so different from the previous Docker-compose YAML file, not to mention the way it would look if converted to K8s YAML.

The OP here, thanks for the comment!

Why does the developer need to care about the certs and ALBs? The devops engineer you need to set up all those controllers could as well deploy those resources from Terraform.

As I showed in the diagrams from the article this application has a single ALB and a single cert per environment and the internal services only talk to each other through the rabbit MQ queue.

DNS, ALB and TLS certs could be easily handled from just a few lines of Terraform, and nobody needs to touch it ever again.

With EKS you would need multiple controllers and multiple annotations controlling them, and then each controller will end up setting up a single resource per environment.

The controllers make sense if you have a ton of distinct applications sharing the same clusters, but this is not the case here, and would be overkill.

The OP here, thanks for your comment.

To be honest I wasn't hired to challenge their entire setup, only to make it more cost effective.

So I chose the most straightforward way I could think of that would allow us to come up with a cost effective setup that will be scalable, fault tolerant and simple to maintain later on.

It all probably started with such a single instance running Docker compose, but then over time it evolved into this setup.

The ideal setup I mentioned would have been also cost effective, scalable and resilient.

This is related to a recent conversation and article on the same topic, but also giving a few solutions for Spot users.

- https://news.ycombinator.com/item?id=35802157 - https://pauley.me/post/2023/spot-price-trends

Another alternative is to also diversify over multiple On-Demand instances, just like for Spot.

This is no longer the case since 2018, see my comment above.

Except for Karpenter and EMR, no other AWS services (including Spot fleets) failover between Spot and OnDemand when losing Spot capacity.

You simply run Spot from other instance types, and if no Spot capacity is available you run at reduced capacity.

Alternative solutions such as AutoSpotting.io are doing it and allow you to run with failover capacity.

This is no longer the case. Back in 2018 AWS changed the pricing model so now there's no more bidding.

This means "bidding" more won't protect your capacity, AWS will just take it if they need it.

Prices are now relatively flat, and changing slightly over time by a fraction of a cent per day with a cap at the OnDemand price, while previously there used to be lots of from 0.1x to 10x the on demand price and with swings even multiple times within a day.

The author of AutoSpotting here, glad to see it mentioned in here and happy to help clarify such issues.

Are you sharing this as an experience of using AutoSpotting or other mechanisms to launch Spot instances?

What you described may still happen occasionally and it happened a lot with older versions, but the recent versions of AutoSpotting, especially the commercial edition available on the AWS Marketplace (although the community edition available on GitHub also does it to some degree) should avoid such situations in the default configuration should be much more reliable than before.

Actually when it comes to capacity, when Spot capacity is not available and we failover to on-demand, the latest AutoSpotting version will failover between multiple instance types with the same diversification used for Spot instances so you're pretty much guaranteed to have capacity even better than your initial single-instance ASG.

We also use the capacity-optimized-prioritized allocation strategy with a custom priority based on lowest cost, but also preferring recent instance types if available for more performance and lower carbon emissions.

Let me know if you have further questions about AutoSpotting, Im happy to help.

For now it's manual but I'll automate it ASAP, that would be a great problem to have

I love this tool, I use it and recommend it to my own customers to see how little my AutoSpotting.io tool implemented in Lambda is costing them to run and also to easily see the logs.

I wrote the EBS Optimizer tool mentioned in the article, I'll gladly answer any questions about that.

There's no risk, Amazon recommends this to customers, see https://aws.amazon.com/blogs/storage/migrate-your-amazon-ebs...

My tool mainly automates the process described in this blog and does it continuously in case you run instances that start with suboptimal configuration. It also detects premium IO1 and IO2 volumes that could be converted to GP3 and changes them for up to 85% in savings, which often amount to hundreds of dollars.

In the future I'll also implement additional functionality such as informing you that there was a suboptimal volume so perhaps you can update your IaC to avoid it going forward, or switching detached volumes to much cheaper magnetic and back to the initial settings when re-attached to instances.

Let me know what you think about this and if you have any further questions I'd love to help you out.

Thanks, I wasn't aware of this, I'll ask the marketplace support about it.

Try to search for "EBS Optimizer" in the AWS console and you should find one of my tools most likely to be useful for most people.

If you run EC2 or any other services that use it under the hood, such as EMR, ECS or EKS, it's going to save you a few bucks. It's free of charge below about $14 savings monthly and charges up to 5% of the savings thereafter, $0.7 for each $14 it saves you.

I have an M1 Macbook Pro and it beats any other laptop I've seen/had when it comes to performance and battery life. I'd love to also have Linux run on it and apparently enough people do as well.

Yes, the hardware is peculiar in many ways and Apple didn't bother to document it (it would have costed them some resources for no clear benefit to their business model) but they also didn't bother to completely lock it down, like they did with the iPhone and iPad. This is what allows other OSes to be ported to it if they are able/willing to reverse engineer and develop support for it.

This reverse engineering work was done so far by just a handful of people who would also like to have Linux run on it, partly sponsored by a few hundreds that share this desire, or just doing it voluntarily as a challenge/interesting project.

The m1n1 bootloader/hypervisor built by the team allows running macOS and logging all the hardware access.

Once the reverse engineering is done for a given component, all the other OSes can implement support for it.

The fact that a lot of the functionality is implemented in firmware blobs makes it easier to support at the OS level, essentially each OS driver only needs to give high level instructions to the firmware, without having to implement all the minutiae done in the firmware.

"The average user prints exactly one colour page during the lifetime of their colour printer. It is usually the test page."

So true!

There's a lot to be done to address digital divide especially for the elderly.

The other week I witnessed first hand how multiple 50-something folks who own smartphones struggled with something as easy as filling a form on a website for making an appointment (in this case for getting a passport) and confirming the appointment by clicking an email.

Or think about the thousands of vulnerable/old people who will die of COVID-19 because they failed to schedule their vaccinations even though they could otherwise access it. A friend who works in Berlin on radiotherapy for cancer patients said lots of them aren't vaccinated because they just can't sign up for it and their doctor doesn't do it for them.

Anything that requires using technology, even as simple as using a browser and getting a confirmation email should be probably be also made available by more familiar/accessible tools such as SMS/whatsapp/facebook or a call center.

That's okay, it doesn't need to be a full-time job, but more like a "4 hour work week" mini-job that pays the bills with minimal ongoing effort.

The rest of the time could be spent doing other more interesting things that just bring joy and/or benefit humankind.

They'll probably also have further potential for monetization, and so on, sustaining a life of leisure and doing meaningful work.

It's just a numbers game. The total number of people who consume porn is probably many orders of magnitude larger than the audience of your respectable software, so even if you're tapping into a very small niche of porn consumers, it can be enough to overtake your other software.