heldsteel7

Every provider has an attribution mechanism. AWS has Inference Profiles. Azure tracks by deployment. GCP has request labels. But each one has a blind spot.

Hyperscalar prices will eventually rise due to AI Tax.

This comprehensive guide covers right-sizing, smart code optimization, event filtering, and efficient monitoring techniques. Learn how to reduce Lambda costs without sacrificing performance, implement best practices, and avoid common pitfalls. Perfect for developers and architects looking to build cost-efficient serverless applications.

AWS NAT Gateway is essential for managing traffic from private subnets, but it can silently inflate your cloud costs through data processing fees. Every AWS service call—whether to S3, DynamoDB, or other services—incurs a $0.045/GB charge when routed through NAT Gateway.

Each AWS service call from private subnets could be adding unnecessary NAT Gateway charges ($0.045/GB) on top of standard data transfer fees. With 50TB monthly traffic, that's $2,250 in avoidable costs. Analysis of how VPC endpoints eliminate this double-charging, with performance implications and implementation patterns.

As your startup scales on AWS, managing access control becomes crucial. This blog post provides a roadmap for securing your cloud environment. You'll learn about the limitations of basic IAM users, the benefits of centralized identity management, and the capabilities of AWS IAM Identity Center with Just-In-Time access.

From mere labelling in 2010 to the sophisticated tagging services of today, AWS tagging has seen a rapid evolution. These updates have provided users with additional functionality, greater control over their resources, and more effective monitoring, auditing, and management of their AWS infrastructure.

CloudYali (https://www.cloudyali.io) provides insights into best practices and effective strategies for managing IAM users at scale.

CloudFormation StackSet allows for automated onboarding of any number of member accounts, while also providing an exceptional user experience.

AWS S3 bucket creation date may be reported differently in different regions. To get the S3 bucket creation date correctly call list api in us-east-1.

CloudYali makes AWS IAM monitoring for large accounts a breeze.

It is critical to understand that almost anything in the cloud can assume an identity. Resources can have direct and indirect relationships, some of which are not obvious.

absolute gem!

A complimentary read https://nedbatchelder.com/text/unipain.html

Great idea! How do you choose papers? Is it all free??

Would be good if you could further narrow down from the topic to subtopics.

A small glitch, when I deselect 'Select All' it doesn't clear topics.

Congratulations on your launch. I'm don't have technical background to understand the nitty gritty, but reading through all the comments itself is an immense learning on the topic.Good luck.

Awesome! Thanks for sharing.

Thanks, can you please suggest improvement points?

Very valid points Brad.

Currently it is third party service collecting the data, we reduct the sensitive information. The data is encrypted at rest and we follow best security rules.

Though our existing model is SaaS based, we are contemplating to separate the control plane into SaaS model, while keeping data plane within the users account. We want to work with our users feedback for this.

Please do try and share your valuable feedback.Thanks!

I’m not surprised you asked how CloudYali is different from AWS Config. We have been asked this earlier as well.

As we know AWS Config, when debuted in 2014 it promised to be the configuration auditor for AWS. The mandate for any team working in AWS workload management, is security and cost effectiveness along with operational efficiency. This requires a great deal of visibility into the cloud workloads. These workloads consists of cloud assets (or resources) from multiple AWS services. As a configuration auditor, AWS Config is supposed to provide a complete visibility into the cloud assets, keep the configuration history of changes and help in achieving the regulatory compliance.

Today AWS has more than 200 services, and more than 500 cloud assets types. In 2017, AWS Config supported ~26 services and 72 resource types. In 2021, the coverage has reached barely 103 resource types. This is ~20% coverage across all the asset types. These assets have relationships which may cross the service boundaries. These relationships are important to be understood, from security point of view (lateral movement attacks). Similarly you need to know about all your assets from cost point of view. You never know if someone accidentally started any expensive service, say macie and you realise it only when the monthly bills arrive. Problem with AWS Config is lack of coverage and even slower addition of newer resource types into coverage.

Yet another issue is usability. AWS Config is a regional service, which needs to be configured across all regions and accounts. This is a sizeable effort. Recently only AWS Config started with Organizations support. When you’re in midst of incident resolution, the important t for any cloud team is access to historical information about the configuration changes. AWS Config UI and CLI is not something you want to deal with at that moment.

For CloudYali, the mission is to provide visibility across all the cloud with usability in mind. CloudYali works on multiple accounts and all regions by default. This means, by default you are looking at the cloud assets across your whole cloud. We have currently added a support for ~ 250 asset types (and we will keep adding).

CloudYali, also maintains the lifetime of the assets, and the history of configuration changes. It is like applying version control to the cloud. The configuration changes are visualised on the timeline, where you can choose to see configuration attributes at any given time. We clearly mark the assets as Live/Active or Deleted if asset no longer exists. It is also possible to filter assets based on the accounts, regions, or time duration so that it is easier to pinpoint assets. This makes troubleshooting easier for cloud operations. Consider for example, rules in load balancer, min/max for auto-scaling group. These things are often changed manually during production issues and if you don't have backups, it's quite easy to mis-configure and then spend time trying to recall previous values. CloudYali intends to be helpful in such scenarios.

I hope this answers your query.

Thanks,

Hi HN,

CloudYali is a service to instantly discover/inventory AWS Cloud assets across accounts and regions.

I have worked in AWS cloud for last 6 years, and many times I had to get an inventory of assets in one place. The tools I found were not adequate, including AWS Config.

Two major issues I found that needs to be fixed: 1. Coverage: Many these tools cover a limited set of AWS Resources. e.g. AWS Config covers merely ~103 resources across ~230 AWS services. The coverage is less than 20%. 2. Usability: These tools are either designed at regional or account level. Eg. AWS Config is at regional level, and requires configuration changes to make it work across regions.

CloudYali, is designed to work across multiple accounts and regions by default. To begin with we have added support for ~250 resources (and will add more in going time).

It is also possible to search for the resources based on Account IDs, regions, resource types. (Search based on resource attributes is in development currently)

CloudYali, also maintains the resource configuration change history and shows a visual timeline for the changes. Which means, you can compare the configuration changes in UI head-to-head.

CloudYali, also tracks the asset lifetimes, so that you can clearly see what assets are Live/Active and what assets are Deleted (and what time)

CloudYali is easy to setup and is a readonly service.

We have now launched a Live Demo of the service with real AWS inventory data, discovered and inventoried by CloudYali. The Live Demo link is available on the landing page.

We are looking for cloud engineers who are willing to try our service and provide feedback.

As a token of appreciation we are giving two months of service to early users at no cost.

Thank you!

The AWS Application discovery is for on-premise server inventory. This information is used as base for on-premise to cloud migration.

CloudYali, on other hand does the Cloud resource discovery. The cloud resources include EC2 instances, RDS, Lambda functions, CA Certificates and many more.

CloudYali, is comparable to asset inventory functionality of AWS Config. In fact the limitations of AWS Config and similar such tools prompted us to write this service. I had previously collected information of such tools in a blog post: https://heldsteel7.medium.com/aws-cloud-scanners-10024e4bb16...

The problem, with AWS Config and other tools is that the coverage is very minimal. In fact AWS Config supports only ~100 cloud resources. Considering that Cloud resource inventory is basis of security (resource misconfigurations), cost (orphaned/unused resources), it is critical that we have a comprehensive inventory. With CloudYali we are attempting to do that.

Another issue with existing tools is usability, which we are trying to address by bringing the information from multiple AWS accounts and regions at a single place. At the same time we keep the change history, which could be useful during troubleshooting.

Please do let me know if you have further queries/doubts. You may also reach to us: support@cloudyali.io

We don't use AWS Key and instead use cross-account IAM role with read-only permissions. We use CloudFormation to create this IAM role.

This looks super cool!

Hi HN, CloudYali is a service to discover AWS Cloud resources across accounts and regions instantly. I have worked in AWS cloud for last 6 years, and many times I had to get an inventory of resources in one place. The tools I found were not adequate, including AWS Config. Two major issues I found that needs to be fixed: 1. Coverage: Many these tools cover a limited set of AWS Resources. e.g. AWS Config covers merely ~103 resources across ~230 AWS services. The coverage is less than 20%. 2. Usability: These tools are either designed at regional or account level. Eg. AWS Config is at regional level, and requires configuration changes to make it work across regions.

CloudYali, is designed to work across multiple accounts and regions by default. To begin with we have added support for ~250 resources (and will add more in going time).

It is also possible to search for the resources based on Account IDs, regions, resource types.

CloudYali, also maintains the resource configuration change history and shows a visual timeline for the changes. Which means, you can compare the configuration changes in UI head-to-head.

CloudYali, also maintains the information about the "dead" or inactive resources. These are the resources which no longer active or live.

CloudYali is easy to setup and is a readonly service.

This post is for the AWS users who might be interested in such service. We are looking for cloud engineers who are willing to try our service and provide feedback.

For the early signers we are giving 2 months free with multiple AWS account support. Please note that the product is slated to launch in September'21. The Live working demo will be functional by next week.

Thank you!

Thanks for sharing the link to strip.

I worked on a use case which was clearly in graph in nature. So graph database seemed a natural choice, of course Neo4j was one of the most heard product even back then.

We did evaluate Neo4j, but put down due to its complex query language (cypher) and slowness. It was really an awkward language, super awkward.

We also evaluated arangodb and we found it much better than Neo4j. Performance was good and its query language was better too.

What we realised in the process is, using graph databases is more of a cultural transformation as well. SQL is much well understood, well adopted and well supported by community.

Ultimately we implemented the use case in Postgres, and thank God we did it that way. IMO, we can still get all the benefits of graphs we SQL databases with little efforts.

Writing forces presenter to think about the idea in much more detail. This is major benefit IMO. Many times, where document culture is prevalent, people come up with vague ideas and expect others to clean it up. Definitely see a value in this approach. Thanks for the write up.