HN user

martius

2,783 karma

Site Reliability Engineer, enjoying System & Network programming.

https://marti.us/ (personal webpage) https://marti.us/w/ (index of my articles)

github: martiusweb bluesky: @marti.us

Posts18
Comments149
View on HN
medium.com 7y ago

Upgrading a large Kafka infra from 0.10 to 2.0

martius
9pts0
code.google.com 11y ago

Forcing Wordpress sites to use https even when not directed

martius
52pts60
www.bbc.com 11y ago

French parliament approves new surveillance rules

martius
5pts0
plus.google.com 11y ago

Google Handwriting input for Android

martius
5pts0
www.openwall.com 11y ago

CVE-2015-0235 – GHOST: glibc gethostbyname buffer overflow

martius
531pts241
marti.us 12y ago

I made a patch for Mozilla, and you can do it too

martius
319pts36
berjon.com 12y ago

Web 2024: Where will we be in ten years from now?

martius
3pts0
promenadenocturne.withgoogle.com 12y ago

Interactive night walk in Marseille

martius
3pts1
plus.google.com 12y ago

What should we expect from a modern init system?

martius
62pts95
medium.com 12y ago

Get to know the friendly hacker who brews his beer and grows his beard

martius
3pts0
github.com 12y ago

NavigationController: API proposal for offline webapps (FF, Blink)

martius
1pts0
translate.google.fr 13y ago

French ISP adds an ad-blocker in its modem/router (translated from french)

martius
1pts0
translate.google.com 13y ago

Facebook Phone OS based on Firefox OS

martius
2pts1
xkcd-map.rent-a-geek.de 13y ago

Xkcd "Click and Drag" in a "map" interface

martius
844pts86
on.ft.com 13y ago

Facebook ventures into online gambling

martius
19pts21
www.isbullsh.it 14y ago

The anti-tech startup pattern

martius
5pts0
www.isbullsh.it 14y ago

Over-engineering considered useful

martius
21pts19
www.html5rocks.com 14y ago

How Browsers Work

martius
633pts36

That's not my experience.

First, there are two SRE ladders: SRE-SWE and SRE-SysEng. SRE-SWE have the same interviews and hiring bar as SWEs. SysEng have less coding interviews but I think the interview questions are more practical and less algorithm oriented.

Still, SREs are subject to the same rules and policies as SWEs when it comes to submitting code.

And in the end, I don't see why using python on some projects would be a bad engineering practice

The reason is quite simple: why spend engineering headcount on a less successful product?

Some wouldn't have been viable, sure. Others were probably too ingrained in Google's hardware/software ecosystem to be separated out (although I wonder if nowadays everything Google runs on its cloud offering, which would make it simpler, just change the billing).

Google Cloud is built on top of Google's tech ecosystem, not the other way around.

Google must diversify its source of revenue, that's the purpose of hardware, Cloud, Youtube premium and others. It is also why they sell access to APIs.

Yes, the later is what Nolan avoids.

But when he says "the movie is done without CGI shots", he can't ignore that most people will assume the movie is also made without digital editing and compositing.

Nolan not using CGI is like Tom Cruise doing Mission: Impossible stunts himself. These affirmations are ambiguous and misleading.

In the case of Oppenheimer (and other Nolan movies) they use a lot of practical effects but they are eventually digitally composited, color corrected and edited.

The director was not attempting to claim that there was no CGI in Oppenheimer at all. He was instead stating that there are no shots in the movie that were entirely created using visual effects.

see https://screenrant.com/how-much-oppenheimer-real-cgi/

In the first case, she's mentioning descriptors to make the same point as for "this" with javascript: you can bind a set of related data to a function.

Exposing "descriptors" is a latter thing, but their default implementation was already a thing before they were visible to python users.

I have zero sympathy for Macron, but the article seems to miss an important aspect: it was mentioned in front of ~200 mayors of cities where the riots are happening.

I believe he said this to satisfy his audience, and maybe to try to put pressure on social media so that they cooperate with authorities (to identify leaders and/or limit coordination between rioters).

It's bad because he's still pushing the overton window to the right, pretending this is OK for a the leader of a democracy to consider this. But I don't think they are seriously thinking about cutting social media.

I've seen a lot of critics complaining about using direction technique (editing, music, etc) to make audience cry, as if only the story, screenplay and acting is pure enough to be allowed to trigger these emotions.

On the other hand, no problem for triggering laughs. It's actually the reverse: you need good direction and good editing (because rhythm matters a lot) to make a joke impactful.

I've never really understood that. Is it shameful to be emotionally affected by a movie, even if you don't really like all of it?

I've seen that a lot about Spielberg movies, and more recently with J.A. Bayona (in particular with The Impossible and A Monster calls).

Not that I know of, we have some virtual filesystems for specific things, but in general Drive is for shared docs, videos (recorded meetings/presentations) and things like this.

We don't use drive to store other files. Actually, we don't really "store files" since almost everything we need is remote.

See for instance this discussion: https://news.ycombinator.com/item?id=13561096

True, I implicitly included the MTTR in the "severity", but this is actually a different thing (severity is more about the impact radius).

But I don't think it changes my point: knowing what/how Google Cloud designs regions or zones is still an implementation detail, what matters is what MTTR they are targeting and this should be known ahead of time.

There are so many "implementation details" that customers are not aware of, because they are always changing, non contractual, or just hard to make sense of, what matters is meaningful abstractions.

I am not saying it's OK if the zones are in the same building or not, I don't know and I was really surprised when I discovered this a few years ago. But this information gives you a mental model of "what could go wrong" that is biased towards some specific risks, and in my experience, relying on these very practical aspects make the risk analysis and design decisions harder to make.

Otho, one thing that may be problematic too (and biasing) is that the common understood definition of a "zone" is the one people know from AWS, so using the same term without being very explicit about the differences will also lead to incorrectly calculated risks. I find the public documentation of Google Cloud too vague in general (and often ambiguous).

But does it really matter that the incident is a flood or a cascading software failure if the likelihood and severity is the same?

Being in the same building is an "implementation detail" from a customer perspective, what matters is the consequences of this decision.

For example, maybe this decision allows for better network connectivity at a lower cost for inter-zones traffic, while, on the other hand, not protecting against some classes of risks.

In the end, you can have a similar multi-zone outage keeping the region down for an extended period of time just because of a bad network config push (see the massive facebook outage in 2021). As a customer, I don't care if it's a flood or a network outage.

Imho, what matters the most is a clear documentation of how these abstractions work for users and the corresponding contractual agreements (costs, SLAs, etc). Users can thus decide if they are ready to pay the price of protecting themselves against an extended outage impacting a single region.

From https://cloud.google.com/docs/geography-and-regions#regions_...

Regions are independent geographic areas that consist of zones. Zones and regions are logical abstractions of underlying physical resources provided in one or more physical data centers. (...) A zone is a deployment area for Google Cloud resources within a region. Zones should be considered a single failure domain within a region. To deploy fault-tolerant applications with high availability and help protect against unexpected failures, deploy your applications across multiple zones in a region.

You should use "region" and "zone" as abstract concepts with shared properties like network topology, local peering, costs, and availability. AFAIK no cloud provider discusses (nor provides guarantees) against specific threats or correlated failures.

There is no guarantee that a given risk will not impact multiple zones, but this risk is lowered by the implementation of various safeguards (for example, rollouts are not happening in multiple regions at the same time).

Google doesn't say "put your VMs in more than one zone because you can be sure we won't have all zones in a region down at the same time", but rather "by putting your VMs in multiple zones in the same region, you can target better SLOs that the SLOs in one zone".

Note that it's different from the concept of "availability zone" of AWS which explicitly says that AZs are physically separated:

AZs are physically separated by a meaningful distance, many kilometers, from any other AZ, although all are within 100 km (60 miles) of each other.

https://aws.amazon.com/about-aws/global-infrastructure/regio...

europe-west9-a and west9-c:

Description: Water intrusion in europe-west9-a led to an emergency shutdown of some hardware in that zone. There is no current ETA for recovery of operations in europe-west9-a, but it is expected to be an extended outage. Customers are advised to fail over to other zones in europe-west9 if they are impacted.

Cloud Console is also impacted:

Customer using Cloud Console globally are unable to open and view the Compute Engine related pages like: Instance creation page Disk creation page Instance templates page Instance Groups page

https://status.cloud.google.com/incidents/dS9ps52MUnxQfyDGPf...

From what I can remember, this is a rumor, not something Google "announced".

From the beginning of the article:

The Google unit, (...) is under pressure from top management to pass Amazon or Microsoft—currently first and second, respectively, in cloud market share—or risk losing funding.

I'm sure that there are discussions about the strategy, the expected growth and revenue and what to do if these targets aren't met. It totally makes sense to reduce the rate of investments to mitigate risks if the results aren't what was expected, but this doesn't mean "shutting down" Cloud.

From the article:

The group even talked about—and eventually dismissed—the idea of leaving the market entirely,

Seriously, Google could even decide to leave the Cloud business eventually, branching it off the rest of the company. But "shutting it down" wouldn't suddenly happen because a spreadsheet shows that Azure has more market shares than Google.

It's something I've heard and read many times, inside and outside of the company, but it's something I haven't really witnessed in my 4 years here.

There are many ways to be promoted and launching something new is definitely not the easiest way.

I understand that it's common to see people leave a team or product once they shipped a promo project, but products don't just die because one or two senior people leave a team.

What happens to the people who bet on them? Do they even have a point of contact if e.g. an integration breaks? That’s the point. They’re half-baked products being put out and pulled with no care in the world. That’s the brand.

So on one hand Google doesn't innovate anymore and is too shy to release anything new, and on the other hand Google can't just get things out and experiment because it would let down people who bet on these experiments.

There is probably an in-between.

a list which apparently now includes its cloud.

Two years ago Google Cloud was 37k employees[1], which is about 20% of the company. Even if the 12k layoffs were on Cloud that would still be 10-12% of the company.

Calling it a side project and comparing it to a "smart tag embedded in clothing" or even Stadia is just FUD.

[1] https://www.bloomberg.com/news/features/2021-07-26/google-fa...

Short answer, yes, a PM should have been consulted and give their approval for the change.

In the scenario I described above (again, it's a guess, I don't know what actually happened), it's possible that the PM was bypassed because the engineers for a reason they thought was good. For example:

* they didn't even think about involving a PM because that's something you have never done,

* the people who wrote/reviewed the change assumed the conversation already happened,

* they were pressured to move fast to mitigate an imminent or existing problem (performance, scalability, ...),

* the PM who should have made this call "left" the company and didn't (get a chance to) hand-off their responsibilities.

I guess what I mean with these comments is that sometimes there are misses like this. Maybe it's a sign of a systematic failure and that internal processes should be improved, but I don't think it means that the company is fundamentally unable to handle these changes correctly or can't make the right technical or product decisions.

Disclaimer: SRE at Google, not on Drive.

I didn't look into this and don't know what really happened, but I can guess.

It's more likely a tradeoff. Either you set this limit and protect your service from an identified scalability limit with the current architecture, or you plan for a (possibly long and expensive) redesign to get rid of this bottleneck.

My guess is that a group of SRE and devs identified the risk, listed their options, evaluated the impact on users (eg: which fraction of users have more than 5M files) and assumed that the change would be mostly unnoticed and that the error message would be enough to push users with more than 5M files in their drive account to do some cleanup.

This group of people underestimated the impact and maybe chose to not involve product managers. It's also possible that the decision was rushed because of recent layoffs or any other random event which pushed engineers to act quickly.

With the bad press, leadership got involved, decision to rollback was taken. In my team, we would have a retrospective doc to discuss the issue (not exactly a postmortem, as this process has specific requirements which would not be applicable to this case).

I think this is a easy mistake to do even with very good intentions, and I can see myself doing it.

Are they unrelated?

The article is about a tradeoff between security and user experience, claiming that a given practice is bad experience without any security gain.

The Ashley Madison leak shows that there are plenty of vulnerable users who are not educated to even "the most basic" things to do to protect their privacy.

It's also a question of user experience to protect users against themselves, or against threats they don't know about.

Saying "it's up to the user to do the right thing" isn't really helpful in the context of discussing account creation/login UIs.

There is also the question of leaking whether a given user has an account on a website or not.

Maybe I don't want my employer to know that I have an account on competitor-service.com, or my partner to know that I have an account on kinky-thing.website.

It might not be a security issue, but it could be a privacy issue.