HN user

antoncohen

3,401 karma

Anton Cohen

Linux sysadmin

http://www.antoncohen.com/

Posts66
Comments414
View on HN
opensource.microsoft.com 1y ago

DocumentDB: Open-Source Announcement

antoncohen
9pts0
techcrunch.com 2y ago

Redis Switches Licenses

antoncohen
3pts1
cloud.google.com 2y ago

Infrastructure Manager: Provision Google Cloud Resources with Terraform

antoncohen
4pts0
blog.pragmaticengineer.com 3y ago

Compensation at Publicly Traded Tech Companies

antoncohen
1pts0
blog.cloudflare.com 4y ago

Cloudflare config staging and versioning with HTTP applications

antoncohen
2pts0
techcrunch.com 4y ago

Google Cloud launches AlloyDB, a new fully managed PostgreSQL database service

antoncohen
9pts1
www.bbc.com 5y ago

'Hovering ship' photographed off Cornish coast by walker

antoncohen
2pts0
apnews.com 5y ago

In Nevada desert, a technology firm aims to be a government

antoncohen
6pts0
scitechdaily.com 5y ago

Math Riddle from the 1980’s Finally Solved – Could Be Used to Improve Computers

antoncohen
3pts2
www.theverge.com 5y ago

Epic says Apple threatens 'catastrophic' response if Fortnite doesn’t comply

antoncohen
1pts0
about.kaiserpermanente.org 5y ago

Covid-19 vaccine generates immune response, well tolerated

antoncohen
2pts0
www.cnbc.com 6y ago

Google announces scholarships for certificates in data analytics, PM and UX

antoncohen
3pts0
www.hashicorp.com 6y ago

HashiCorp Cloud Platform

antoncohen
2pts0
jalopnik.com 6y ago

The Case for Ending All Traffic Stops

antoncohen
4pts1
www.bbc.com 6y ago

Solutions to US Police Problems

antoncohen
2pts0
www.governor.ny.gov 6y ago

Army Corps of Engineers to Build Temporary Hospitals in NY

antoncohen
416pts508
opensource.com 6y ago

What I learned going from prison to Python

antoncohen
4pts0
www.scylladb.com 6y ago

Gemini: An Open Source Automated Random Testing Suite for Scylla and Cassandra

antoncohen
4pts0
www.sfgate.com 6y ago

Estimated $64M loss as SF street conditions and costs drive out Oracle OpenWorld

antoncohen
23pts9
www.docker.com 6y ago

Docker Restructures and Secures $35M

antoncohen
2pts0
en.wikipedia.org 6y ago

A/UX: Unix-Based OS by Apple

antoncohen
2pts0
www.cockroachlabs.com 7y ago

We Run Managed CockroachDB on Kubernetes

antoncohen
2pts0
status.cloud.google.com 7y ago

Google Cloud outage post-mortem

antoncohen
8pts2
status.cloud.google.com 7y ago

Post-mortem of March 12 Google outage

antoncohen
204pts107
vitess.io 8y ago

Vitess: database clustering system for horizontal scaling of MySQL

antoncohen
2pts0
www.motortrend.com 8y ago

Tesla Model 3 Teardown

antoncohen
4pts0
www.apple.com 8y ago

Trade in with Apple GiveBack

antoncohen
1pts0
cloudplatform.googleblog.com 8y ago

Kayenta: An open automated canary analysis tool from Google and Netflix

antoncohen
4pts0
medium.com 8y ago

Bringing Waymo self-driving technology to trucks

antoncohen
1pts0
kubedb.com 8y ago

KubeDB, production-grade databases on Kubernetes

antoncohen
3pts0

The debit card behavior is probably bank specific. I had fraudulent transactions on my debit card. The bank caught it after a few transactions, alerted me, and shutdown the card when I told them it wasn't me. I didn't get charged for any of the fraudulent transactions. I also had a restaurant charge my debit card for my bill and another customer's bill (honest mistake, not fraud). The restaurant wouldn't refund the transaction, so I disputed it with my bank, who reversed the transaction. The bank was fully set up to dispute debit card transactions from their website.

My rude-ass car 3 years ago

can anyone explain why in the US if I click unlock on the remote, it just unlocks the front door? This keeps happening on rental cars. You have to click twice on unlock to get all the doors to open.

This is likely configurable. Instructions should be in the owner's manual. It is configurable for both my and my wife's vehicles, one via the touchscreen and the other by pressing and holding unlock for 5 seconds.

My rude-ass car 3 years ago

This is vehicle specific. Not all 2023 vehicles behave this way. I have a current generation vehicle (same generation and tech as 2023).

- Doors stay unlocked. Eventually the engine won't start without pressing unlock on the key fob again, but the doors remain physically unlocked forever.

- Trunk is manually operated.

- It doesn't ding when starting the engine if my seatbelt it on. And I have a programmer that lets me disable the dings when my seatbelt it off. There are no dings when turning the engine off.

- Blind spot warning is configurable: Off, lights, lights + chime. The chime warning doesn't seem annoying.

- No lane keeping assistant.

- Tire pressure monitors work well. They are accurate (same pressure as multiple physical gauges I've tried). Tire pressure increases slightly when driving due to heat. They have never triggered a warning.

- I don't recall ever having to accept terms of service. It certainly hasn't happened multiple times.

I have physical knobs for volume, fan speed, and tuner. Physical buttons for everything else. No controls use resistive touch buttons. No controls are via touch screen (touchscreen has information and setting like blind spot, but not actual controls that don't have physical buttons).

I also have a 1990s vehicle, with an aftermarket touchscreen installed to support Android Auto and Apple CarPlay. The current generation vehicle is no more annoying than the 1990s one.

My wife has a 2023 model year vehicle. Many of the complaints in the post are enabled by default (auto re-lock, blind spot chime that gets confused by multiple lanes). But many of the annoying things are also configurable, including auto re-lock and blind spot.

So it is possible to pick a vehicle that isn't annoying. And I suspect most of the annoying things can be disabled.

That is true, I was thinking specifically about the metadata and SSH keys. But DHCP can also set DNS servers, NTP servers, and other things that can either cause disruptions or be used to facilitate a different attack.

There might be a persistence issue, it seems like part of this attack was that the IP was persisted to /etc/hosts even after the real DHCP server took over again. But even just writing to /etc/hosts could open the door redirecting traffic to an attacker controlled server.

While there are a series of vulnerabilities here, none of them would be exploitable in this way if the metadata server was accessed via an IP instead of the hostname metadata.google.internal.

The metadata server is documented to be at 169.254.169.25, always[1]. But Google software (agents and libraries on VMs) resolves it by looking up metadata.google.internal. If metadata.google.internal isn't in /etc/hosts, as can be the case in containers, this can result in actual DNS lookups over the network to get an address that should be known.

AWS uses the same address for their metadata server, but accesses via the IP address and not some hostname[2].

I've seen Google managed DNS servers (in GKE clusters) fall over under the load of Google libraries querying for the metadata address[3]. I'm guessing Google wants to maintain some flexibility, which is why they are using a hostname, but there are tradeoffs.

[1] https://cloud.google.com/compute/docs/internal-dns

[2] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance...

[3] This is easily solvable with Kubernetes HostAliases that write /etc/hosts in the containers.

Eusociality typically requires two other conditions. ... And since the ferns spread asexually on shared roots, they don’t actually exhibit an active system of resource acquisition typical of brood care.

One key question is what defines an individual fern. If a colony can begin with a small plume of strap fronds sticking up from a few nest fronds and then spread asexually on the same roots, perhaps it is a single plant

There is a lot of talk about the amazing cooperation of these plants. And then they say it is actually one plant. Is it interesting if one plant grows different leaves at different heights?

This is effectively required when open sourcing something that was previously internal. I've done it at a company. There could be company internal things that leak in old revisions of code and commit messages. Even if the latest commit on master is clean, it is really hard to know that every revisions, throughout the whole history is clean.

Pulumi 3.0 5 years ago

Open Source SDK

The SDK is a CLI and collection of libraries for defining and deploying cloud apps and infrastructure in code.

https://www.pulumi.com/pricing/

With Terraform the open source CLI and libraries for defining and deploying infrastructure are fully usable, even for large systems.

What I would like to know is, how usable Pulumi is without a paid subscription?

I thought when I first looked at Pulumi, the only non-local state backend was the paid Pulumi Service. But looking at it now, they seem to support the normal object store backends that Terraform does (https://www.pulumi.com/docs/intro/concepts/state/). Though the talk at lack of concurrency control seems to imply they don't support locking like Terraform does.

Everyone's needs are different. To me GCP is easier to get started with.

I don't want white glove service where they try to architect my architecture for me. Pushing me into cloud specific features (like everything in Lambdas stitched together with API Gateways). I don't need that level of help.

Also, of the three startups where I have heavily used AWS (personally "owned" the AWS infra), I never had that level of service offered. The only time I've had that "we'll help you architect it" service from AWS was at a startup that used GCP, and there were high level (CEO and CTO) discussions with AWS about partnerships. Then AWS wanted to help us architect (with Lambdas and API Gateways).

For me, GCP is way easier. Their products are more complete products. They work together out of the box. They are optimized for the 90% use case to just work. It is far easier to get a really good complete architecture working with GCP. Using their products push you to industry standard ways of writing and packaging applications (12-factor, Docker containers, etc.).

The startups I've seen build on AWS often have very bad setups. Hand rolled and poorly managed EC2 messes. Bad logging and monitoring because AWS doesn't have good logging and monitoring. Having to manage everything themselves because AWS doesn't/didn't have good managed platforms (Elastic Beanstalk has enough problems that people don't use it, and EKS barely counts as a managed service).

I've seen multiple startup with hand configured EC2 instances, running some generic web service. These systems create huge burdens and risks as the company grows. The app then grows and becomes locked into the special snowflake hand configured mess. One company I joined couldn't deploy for 6 months because their last deployment (a manual process) took 7 hours of downtime. When I see those systems I wish they started on Heroku or GCP App Engine (2nd gen). They would be way better off with a fully automated platform and 12-factor apps. Once you grow and hire people with infrastructure expertise you can take that clean 12-factor app and move it somewhere else. Non-experts trying to use AWS results in scary things.

GCP gives you that easy on-ramp. You can start with App Engine. If you need something a bit more custom, you can use Cloud Run (same basic tech as App Engine v2 but with Docker containers). If you need to move to something even more customizable for growing special needs, you can use GKE for a fully managed Kubernetes. With all those services you can write normal apps that use environment variables and log to stdout/stderr. All three give you logs in an easy to use logging platform. All three give you metrics in an easy to use monitoring/metrics/dashboard platform.

At every level GCP is easier. Cloud Pub/Sub is easier than SNS+SQS, Amazon Kinesis, or Amazon MSK (Kafka). It is multi-region and just works and scales. GCP VPCs and networking are easier because GCP VPCs are global (not regional), and subnets are regional (not zonal). No need to stitch VPCs together. GCP even has Shared VPCs so you can have a single address space shared across multiple projects (GCP projects == AW accounts). GCP authentication is simple and sane, even across multiple projects, no federation and role assumption needed. GCP IAM is far easier than AWS IAM. Cloud Datastore provides a really simple yet powerful NoSQL store, totally simple, no tunables, just works and scales, and even has some multi-region options available. Want a multi-region consistent database that spans large geographic regions? Amazon doesn't have an option, GCP has Spanner. Want a multi-region consistent blob/file store? Amazon doesn't have an option (S3 is single region), GCP offers GCS in multi-region, dual-region options that provide consistent multi-region file storage. Do you have some weird old app that writes to disk, and you would like it to be able to survive a zone failure? AWS EBS is single zone, GCP persistent disks can be regional, and because subnets are regional that means an instance in another zone can take over the disk and the IP.

For me, GCP is way easier to get started with. And will result in a better system in the medium term. In the long term, when you have thousands of engineers and experts in infrastructure engineering, it probably doesn't matter.

Do you trust Debian LTS? As much as RHEL? The documentation about Debian LTS always made me think it is not a fully fledged thing. I've always felt like Debian releases reached EOL on their EOL date, not their LTS EOL date.

Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success.

https://wiki.debian.org/LTS

Possibly, thanks, I'll look into it. One key is that I'm not looking for an abstraction, I'm looking to provide an abstraction. I run infrastructure teams that provide the infrastructure as a product to other engineering teams, including build and deployment systems. So I need a system that is flexible enough that I can create the abstraction I want to provide. Most of the tools in this space either abstract away everything, in an opinionated way that I don't agree with. Or they abstract away too little, so I would have to write a heavy abstraction over the abstraction. Waypoint, with its plugin system and if the HCL syntax is expanded to support reusable templates/modules, may provide a good building block for me.

This is great. I'm glad a good company is backing an open source project like this. I've built abstractions like this at two companies, and I hope one day I can stop writing my own. Some comments:

I see "promotion" on the roadmap, so this is probably in the works. But the current "Workspaces" each have their own build. I really want to be able to promote the same artifact from one environment (staging) to another (production). I'd also like to be able to choose the artifact build/version to deploy, not just what happens to be in my local repo.

The concept of multiple environments also brings up the need to vary things by environment. App config (env vars) are obvious. But also settings like number of replicas or auto scaling min/max (auto scaling is also required).

The biggest thing most of the tools in the space lack, like all the ones that try to copy the docker-compose.yml syntax, is standardized reusable app settings. Imagine I have two common types of app, "API services" and "background workers". They have common settings, like maybe they all default to using /healthz for health checks and auto-scaling at 70% CPU. Then within each group they vary, "API services" use internal load balancers and "background workers" don't.

I don't want every individual "API service" app to have to say "my health check endpoint is /healthz" and "I run on port 8080 and need a load balancer". Those are the defaults, and if the app uses the defaults it shouldn't need to be configured. But at the same time I want the app to be able to override the defaults. Within a well standardized environment 90% of app infra settings (not env vars) are the same, or can use the same template (like the image is docker.example.com/svc/{app_name}:{build_id}). I want to be able to reuse the settings.

This is awesome! The interface looks great, it is the UX I want. It boggles my mind why the major cloud providers who have parameter/secret management don't optimize their UX for the 90% use case of "I have an app, it runs in multiple environments, I want to vary the config by environment, and expose the config as environment variables, all with a simple and easy to audit interface".

On the feature request front, I'd like to be able to vary the config by location (e.g., region, but could be zone, rack, etc.). It is common to have a production app deployed to multiple regions (as Doppler itself does), and it is likely that 80% of the config will be the same between regions, but there may be region specific settings.

Which leads to the next thing I want, a hierarchy of config precedence: app default -> app+env -> app+env+location. So that the common settings don't need to be duplicated. Right now my guess is that to use Doppler with multiple regions I'd create environments like "prod-us-central1" and "prod-us-east1", but then 80% of the config will be the same between them.

Another thing that can be nice is to have a canonical value, and have multiple apps point to that value instead of having their own copy of the value. For example if you have a "production DB host" you can set that once, and multiple apps can point their DB_HOST or DATABASE_HOST at the "production DB host" canonical value. That way when the "production DB host" changes, it only needs to be changed in one place.

The App Platform is one of the few PaaS products built on a shared Kubernetes platform.

I wonder how they are doing this from a security standpoint. For customer workload isolation is every container actually a VM? The pricing/sizing kind of makes it look like that.

App Platform provides predictable, easy-to-understand pricing

How? Outbound network is charged per-GB. I know autoscaling isn't supported yet, but hopefully they support setting a max number of instances. One of the things people like about DigitalOcean over other cloud services is that you can sign up to pay $X per month and know that is what exactly what you will pay, no surprises.

Upcoming features

The list of things not supported yet includes auto-scaling and VPC. It is hard to imagine using a PaaS without autoscaling. And I wouldn't want to build out a microservices-like architecture without VPC.

Overall I really like the concept of the offering. Simple PaaS, with custom container support, Kubernetes (for what it is worth in a PaaS, I don't know), and predictable pricing. That all sounds really good.

Have you ever filled out an actual job application in the US? I'm pretty sure every job application I've filled out until San Francisco passed the Fair Chance Ordinance has asked if I have any convictions, and some have asked about arrests (which is crazy). And every job I can remember has done a background check.

Move to San Francisco. Employers aren't allowed ask about arrests or conviction on a job application. They aren't allowed to do background checks until they have offered you the job. They can only consider convictions directly related to the job. And if they reject you because of a background check, they must notify you, allow you to respond, and reconsider based on the response.

https://sfgov.org/olse/fair-chance-ordinance-fco

https://sfgov.org/olse/sites/default/files/FCO%20poster2020....

For context on why this park restriction comes across as racist, East Palo Alto is a different city from Palo Alto, and these are the demographics:

Palo Alto[1]: 60.6% White, 27.0% Asian, 6.2% Latino, 1.8% Black, 0.2% Pacific Islander

East Palo Alto[2]: 64.5% Latino, 15.8% Black, 7.4% Pacific Islander, 6.2% White, 3.6% Asian

It comes across as a law that prevents Latino, Black, Pacific Islander people living in the area from visiting their local park.

[1] https://en.wikipedia.org/wiki/Palo_Alto,_California#Demograp...

[2] https://en.wikipedia.org/wiki/East_Palo_Alto,_California#Dem...

That is interesting. There is some magic networking going on if Google allows every customer to allocate an IP range of their choice, and the customer can use all the IPs in that range, and Google runs multiple customers on the same network (same VPC and subnet).

A project can contain multiple VPCs. And a VPC can contain multiple subnets, but not with overlapping ranges.

https://cloud.google.com/sql/docs/mysql/configure-private-se...

One of our interesting findings was the iptables rules, since when you enable Private IP access (Which cannot be disabled afterwards), access to the MySQL port is not only added for the IP addresses of the specified VPC network, but instead added for the full 10.0.0.0/8 IP range, which includes other Cloud SQL instances.

Therefore, if a customer ever enabled Private IP access to their instance, they could be targeted by an attacker-controlled Cloud SQL instance. This could go wrong very quickly if the customer solely relied on the instance being isolated from the external world, and didn’t protect it with a proper password.

I'm not convinced by this, I'm not sure it is vulnerable in the way the author is suggesting "they could be targeted by an attacker-controlled Cloud SQL instance".

First of all, GCE has firewall rules outside of iptables. But the main thing is that the way Cloud SQL does Private IP is via VPC peering. Google creates a VPC on their side, runs MySQL in it, and peers that VPC with your VPC. You actually tell Google what CIDR range to use in the their VPC (the Cloud SQL VPC).

I don't think is it fair to assume that all customers are in the same VPC, and same subnets, with routes between them, and no GCE firewall rules blocking them.

Asgard from Netflix[1] basically did what you are asking for. But it has been deprecated[2] and replaced with Spinnaker[3], which has a bit of a different focus.

But really, with couple dozen VMs, you don't have a cloud management problem. I would take a step back and figure out what the problem is, and where you want to be. Why are people creating VMs? What are the problems with them creating VMs? How do you mitigate those problems? If it is just tracking, then tags + education can help solve the issue. If it is security and standardization, an opinionated deployment system could help.

With a couple dozen VMs, I'd say you probably shouldn't be running any VMs you manage yourself, and you should be using fully manage solutions. Then focus on the tooling that makes those solutions efficient to use for your users, while meeting your security needs.

[1] https://netflixtechblog.com/asgard-web-based-cloud-managemen...

[2] https://github.com/Netflix/asgard

[3] https://spinnaker.io/

That is a result of how Homebrew built and packaged it. There are downloads at https://osquery.io/downloads, and for a Homebrew installation that page recommends installing the Cask.

    $ brew cask info osquery
    osquery: 4.4.0
    https://osquery.io/
    Not installed
    From: https://github.com/Homebrew/homebrew-cask/blob/HEAD/Casks/osquery.rb
    ==> Name
    osquery
    ==> Artifacts
    osquery-4.4.0.pkg (Pkg)
    ==> Analytics
    install: 120 (30 days), 198 (90 days), 206 (365 days)

People are confusing new versions of things, or new APIs, or new products, or not recommenced anymore, with actually removing products. From this thread people are complaining about:

- Datastore -> Firestore in Datastore Mode: This was a seamless transition with no API change, Google changed the underlying tech.

- Firebase -> Firestore: Never happened. They are different products. The Firebase Realtime Database still exists.

- gcloud CLI updates: It is a CLI tool, it gets new versions.

- App Engine with Python 2.7: It still exists, you can still use it. There is a new App Engine v2 that supports Python 3.

- Cloud SQL v1 to v2: They had a tool to automatically upgrade, it set up replication and switched over.

AWS has new versions too. RDS Aurora v1 (MySQL 5.6) can't be upgraded in-place to RDS Aurora v2 (MySQL 5.7)[1]. AWS Lambda runtimes get deprecated[2].

[1] https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...

[2] https://docs.aws.amazon.com/lambda/latest/dg/runtime-support...