mooted1

as someone that has complained about a lot of uber's resume driven development (see comments), I can attest that this system is

a.) necessary, and saves the company millions

b.) a simplification over the ... unusually ... architected system that came before and did not scale.

there's a lot of junk technology at many large tech companies, but scaling problems /are/ real when you have hundreds of millions of users and have demanding performance and reliability requirements. RAMEN is a differentiated and necessary part of Uber's infrastructure, even if that can't be said of a lot of uber eng blog posts.

This is the rare actually funny comment on hn.

I'm going to assume you're a developer?

No, but thanks for explaining my career to me and helping me empathize with toxic people.

bai

I appreciate so many of you are willing to crusade for free speech. But the reality is that platforming isn't free, and the process of deciding who gets platforms is political. The first amendment doesn't govern this process, nor can it—no one can reasonably consume every piece of intellectual content, from every angle, and process it.

My point is, cancel culture definitely exists. There has always been, and always will be, political processes for selecting views we consider acceptable discourse. Arguments against cancel culture that don't grapple with this reality are missing the point. Canceling isn't about using the power of the state to crack down on free speech. It's about deciding who gets access to a scarce resource: their platform.

The way this get decided is unavoidably political, since it's about who wields power.

A good example is this website. While most posts are technical, Hacker News hosts a significant amount of discussion regarding social issues. If you read this site using hckrnews.com, which preserves posts flagged off the front page, it's easy to notice that there are political trends in how moderators and users select what posts get prominence. This post, which espouses a more centrist perspective on cancel culture, was briefly flagged, then restored. Many, many other posts espousing more progressive views are systematically downvoted or flagged (search for DEAD on hckrnews.com), many of which are far longer and more carefully developed than Taibbi's piece. Regardless of how you feel about confederate statues, I think it's hard to consider that this 538 post (https://projects.fivethirtyeight.com/confederate-statues/) isn't comparable to this post in terms of quality, sophistication, and merit to be on the front page of HN.

For those of you who agree with Taibbi... do you think those posts aren't "canceled"?

If you think this example is contrived, because the nature of this cancellation doesn't involve a visible mob, that's because of power differences.

The nature of "cancellation" against minorities looks different. Mob justice isn't necessary when people in power can just fire you. And progressives are fired or otherwise censored for advocating for inclusion, against police violence, for progressive social policy all the time. You don't see mob justice in these cases because this kind of canceling is often enforced by institutions. A mob isn't necessary. The effect is the same: people are regularly censored and excluded from mainstream prominence because their views are considered unacceptable.

The reason this latest wave of (attempted) cancellations (nyt editorial, jk rowling, adam rapaport, countless people resulting from #metoo etc) have garnered attention is twofold:

1. They require popular support. Lots of people support holding rapaport, the founder of crossfit, Tucker Carlson, etc to account. No institution wanted to punish these people, so a popular movement formed.

2. They cancel views formerly considered to be within acceptable bounds. Anti-trans speech, implicitly discriminatory pay practices, military crackdowns on protestors, even if they weren't popular, are believed by many to be "newspaper publishable". It's shocking to people like Taibbi that people who share their views can get fired and singled out on social media—things that happen to people with progressive views on a regular basis.

3. Oftentimes, the people making the speech are civil and ostensibly in good faith, even if their views are onerous.

To understand cancel culture, you need to define cancellation as "people whose views are censored", and you need to look at people who espouse progressive views. That "cancel culture" is prominent today means that our definition of acceptability is changing, that powerful people require mobs to be held accountable, and that we increasingly believe that how you present the argument (good intent, good faith etc) is less important than what you present.

Lastly, I'd be remiss if I didn't acknowledge that cancellations are sometimes off the mark. Mob justice is often unfair and harmful. It is often disproportionate (the shor incident is the best example I can think of). Holding people accountable for hate speech or discriminatory practices should be the job of institutions. But they're not doing the job—this is what progressives are protesting.

If you want to stop mob culture, change institutions to fairly enforce the injustices that the mob is necessary to enforce today.

Oftentimes, these tools are developed for much more complex sites than the people who author these rants have ever maintained. Of course it doesn't solve meaningful problems for sites that can be maintained by a handful of developers and whose feature set evolves slowly.

That's fine. Don't use them.

It'd be nice if rants like this also reflect on why these frameworks do exist. Then people would actually learn something.

Here are some of the tradeoffs I considered when evaluating JAMStack vs flask/jinja server side rendering.

1. Low latency interactivity unlocks all kinds of user interaction optimizations for us. The size of the initial download is amortized over hundreds to thousands of lightening fast interactions.

2. It's difficult to manage complexity around large html/css deployments, since CSS is a global namespace. Devs use all kinds of conventions for modularizing CSS. You can definitely succeed with these strategies, but when you're juggling tens of thousands of lines of CSS, it's a lot easier to solve these problems using a real programming language and its module system. We anticipated thousands of lines of CSS over time.

3. Immutability makes whole classes of race conditions unlikely to impossible. Do things still race in React? Of course. Are they vastly less likely to happen? Yes.

4. Managing interactivity through server-client interaction shifts a ton of complexity to the backend that the browser could otherwise handle. If you're designing a multi-stage form, for example, you often have to save incomplete user submissions in servers, depending on use case. Offloading this to localstorage on the frontend is a lot easier to author and maintain.

5. If you host your frontend statically on CDNs, that's several orders of magnitude less traffic you have to scale your web servers for. The vast majority of my company's traffic is handled by Cloudfront, making our operational footprint tiny.

Are there also reasons the modern stack sucks? Yes.

1. Configuring react and webpack is incredibly, incredibly hard. Webpack exposes a vast array of knobs that most applications don't benefit from tuning.

2. Time to first load is absurdly slow and server side rendering is, again, very complex to implement. React is not the right choice for a blog or anything that is sensitive to bounce rates.

3. MUI's documentation and APIs are painful, as OP notes.

For these reasons, and because most of our application was interactive, not read-only, we went with JAMStack.

I definitely think there's a place for the traditional HTML/CSS/jQuery stack. Read-only sites, for starters, with limited feature complexity. Everything I've built in the last several years has grown in complexity and features at a faster-than-linear rate, however, and these new technologies make managing such codebases vastly simpler.

Your suggestion for the problems with this project is "Why don't you just fork it and maintain your own?"

Get off your high horse. It doesn't matter what it was originally intended for if hospitals are over capacity, and opening up the ship will save hundreds, maybe thousands.

I think you make some good points here, and RLS is an ideal data security model in many respects.

The parent's point is that tooling around this is weak, which undermines its value. Not being able to review and easily test authz changes will almost certainly lead to issues.

Having a JWT or whatever authn is irrelevant. The problem is an insufficient authz model.

Exposing databases through APIs is not the problem. Exposing relations without authorization is.

Under these frameworks, you can add an innocuous relationship between two models that entirely compromises security without even touching the API code. Not only that, but the graph of relations and their associated ACLs is complex. Every time you add a relation, you need to create a graph of your data model and ensure that it's safe. There are far, far more surface area to make a critical error, allowing attackers to exfiltrate large volumes of your data.

I've written these bugs in similar modeled systems (there were GQL like systems before GQL). I've fixed these bugs. I've caught these bugs in code review.

These bugs are orders of magnitude less likely to happen with a simpler authz model where you don't need to lock down every relation, just the table itself. This is why the GraphQL creators themselves encourage users to put authz at the business layer.

2 out of three examples above literally have no framework authz support. Postgraphile requires setting up row level security policies, meaning you have no control over what layer of code you want authz policies to live; they must be in the database. Even if you are ok with that sacrifice, you still have to find ways to manage this in version control and test, for which there is scarce tooling.

Hasura seems to do the right thing here, provided you opt into it. It's not clear if it allows you to easily version control or test your ACLs.

And that's all assuming the GraphQL server has a public IP, which is far from a certainty (just like REST).

Security doesn't stop at your VPN. At several hundred engineers, organizations begin implementing internal controls.

In fact, my example was an employee comp manager :|.

For the love of god, this is never how GraphQL was intended to be used. The official graphql website is very clear:

https://graphql.org/learn/authorization/

Delegate authorization logic to the business logic layer

If you take security or performance debugging seriously, you should never expose database models through APIs directly in a production app.

To illustrate, say you have an Employee model:

  query {
      employee(userId=uuid) {
          name
          salary
      }
  }

The next day, someone creates a Manager object, a relation from employee to Manager, and Manager to employee.

Now, without having consider security for a second, you've granted all employees the ability to query each other:

  query {
      employee(userId=uuid) {
          name
          salary
          manager {
              employees {
                  name
                  salary
              }
          }
      }
  }

Please, please stop encouraging people to directly expose their databases through an API.

Did you actually read the post?

The case for big tech today is still the economy of scale and not network effects (maybe facebook have those, but it exists only if the interface to facebook does not change).

This is only true if you believe that the greatest cost of developing software is running hardware. The greatest cost of developing software is developing software. Not only are economies of scale in compute management negligible except at massive scale, the cost of compute has declined dramatically as the companies you've described have made their datacenters available for rent through the cloud. Yet the tech giants persist.

Facebook, Google, Netflix, Amazon all have considerable network effects that you're not considering. For each of these companies, having so many customers provides benefits that accrue without diminishing returns, giving them a firm hold on market share. See https://stratechery.com/2015/aggregation-theory/

Ben is saying that the only way to topple the giants is by working around them and leveraging new computing technologies better than them. He makes the (admittedly speculative) case that this is no longer possible because we can't bring compute any closer to the user than the mobile devices.

However, with Kubernetes operators, there is a way to move those capabilities into any Kubernetes cluser.

Kubernetes, at the scale of technologies we're discussing, is a minor optimization. Introducing k8s costs more than it helps far until far into a company's infra maturity. Even if most companies deployed k8s in a manner that significantly reduced costs, it's not enough to overcome the massive advantages existing tech companies have accrued. Not to mention all of the big tech companies have internal cluster managers of their own.

If you read more of Ben's writing, he talks extensively about how software companies dominate market share through network effects and vertical integration.

You don't hear him talk about economies of scale because marginal costs are negligible for software companies. Besides, network effects and vertical integration are sufficiently powerful to control the market.

In addition, new tools and new processes for software has increased the productivity times fold, which means that you need fewer developers for new software.

There are other barriers to entry besides the cost of writing software, like product, sales, operations, and most importantly, network.

Disappointed to see the usual onslaught of HN cynicism.

"Product doesn't meet my narrow expectations; is bad product."

1. We use Slab. It's a fairly ok product in a crowded space. Needs maturity. Far better than confluence.

2. I have had no experience with the lag that another commenter insists is a product killer, and I write and edit docs all day.

3. The niche this product and others like it solve is to keep your entire company's docs organized and discoverable. Remembering to file things away correctly, share content, and do full text search on google drive is... not a good experience.

4. Editor is pretty solid and responsive. If you know markdown it's a breeze. Literally can't understand why you'd want to edit raw markdown when the WYSIWYG reflects markdown syntax as well as it does.

I think it's tempting to attribute to structure what can be better explained by accident.

You obviously have more context, but from my experience developing product, there are almost always ways to change direction incrementally and in a manner that allows you to get your toe wet to test the temperature, and not dive in unshielded.

Although I realize now you may be saying that change was inevitable, but catastrophe wasn't.

Have you ever worked in a polyglot ecosystem with rapidly evolving schemas?

Tools like protobuf and thrift were designed to facilitate schema evolution since interfaces in these ecosystems evolve quickly and independently. Generics undermine this by creating strict dependencies on a few types, making it difficult to evolve a single type without breaking things.

Poorly implemented generics would undermine one of the design goals of this project. In addition, there aren't nearly as many opportunities for generics in an IDL as in a programming language, so what would the upside even be?

Security isn't black and white. SMS doesn't neutralize the value of 2FA since the effort required to compromise it are still considerable, such that the highest risk methods outlined by NIST require physical proximity to the phone.

While SMS is inappropriate for high value targets such as employees with infrastructure access or people with government security clearances, it remains an excellent option for general consumer security.

By contrast, the vast majority of people who use login.gov likely don't understand nor will they ever install an authenticator app. It makes sense to deprecate support for this vector if, presumably, few people are using it.

+1 that the code is, unfortunately, most likely worthless.

My company has one chief competitor.

If that competitor went out of business and offered me their source code, I'm unlikely to pay for it. Code tends to be highly coupled to the specific assumptions, structures, workflows, architecture, and environment it was designed for, unless the team has gone out of their way to make it reusable. Most often, it would be easier to rebuild the same functionality from scratch than integrate someone else's code into your product.

What is valuable are the more generalizable and often intangible assets my competitor has accrued:

1. Its relationships with potential clients. 2. The lessons they've learned in building the business so we might avoid them. 3. Their employees.

Do you have any of the above to offer to companies tackling similar problems?

That said, isolated parts of a codebase can be useful, if it's easy to reuse and solves an extremely challenging problem and has required a large amount of investment to develop and mature. If such a part of your codebase exists, you could try finding a buyer for that.

What middleman do you want to cut out? My proposal doesn't add any dependencies. It's also not possible to prevent duplicate charges if a user POSTs a form twice without idempotency keys, so idempotency is necessary.

I'm really not sure what you're actually proposing?

Also, I think the answers you're getting keep telling you to build distributed transactions because

a.) they didn't read your post and are overindexing on "microservices" b.) this isn't an atomicity problem, it's a workflow problem. When people say atomicity, they're usually referring to operations over data. In your case, it sounds like you need a way to coordinate execution of side-effectful operations (like creating a charge).

That's not what the OP is asking--they're using external services like Stripe. Creating a Stripe charge isn't something you can roll back.

Having built similar applications in microservice environments, I think there are usually simpler answers than distributed transactions. And if you do need distributed transactions, this is often a sign that your service boundaries are too granular.

In fact, since the services you're describing don't know about each other, distributed transactions aren't an option.

I think the only solution to this problem is idempotency. Idempotency is a distributed systems swiss army knife—you can tackle 90% of use cases by combining retries and idempotency and never have to worry about ACID. Yes, it adds complexity. No, you don't have a choice.

I'm also not sure why this requires a lot of complexity. Can you explain how you're implementing idempotency? The simplest approach is to initialize an idempotency key on the browser side which you thread across your call graph. Stripe has built in support for idempotency keys so in that case, no additional logic is required. For providers without idempotency support, you'll need a way to track idempotency keys atomically, but this is usually trivial to implement. When a particular provider fails, you can ask users to retry.

* If you need a particular operation to succeed only if another succeeds (creating a stripe charge, for example), make sure that it runs after its dependencies.

* If you don't like the idea of forcing users to retry, you can ensure "eventual consistency" using a durable queue + idempotency.

I'm not a fan of HN comments that trivialize problems, but if you have to build complex distributed systems machinery to solve the problem you're describing, I feel strongly that something's going wrong.

This was under consideration at one time by the Seattle build team and was a recurring point of discussion during infra summits. Unclear if we actually broke ground.

I worked at Uber from 2014-2018.

Can confirm this. It was a very poorly managed org because it was extremely grassroots driven—leadership was underempowered to say no to front line teams or hold groups accountable.

The overlaps and poorly considered projects described above are not an exaggeration. Many of them were designed for building the lead engineer's open source brand, not for company needs. Half of the projects described above have since been canceled.

Google:

- Hyperbahn

- Cherami

- Schemaless

- Peloton

- uDeploy

- m3db

- piper

- XYS

Contrary to their glossy open source and tech blog presentations, these projects were highly contentious internally and widely viewed as inferior to industry counterparts. Each of these projects had 5-20 engineers working on them for over a year; many of them are being EOLed or phased out currently. And this is just the list that I can publicly talk about. This development came at extraordinary cost for the org and, in the case of the people who were laid off, cost to people's careers.

No. He wasn't. He was /a/ product manager. There are various other leads in the area, all of which had higher rank than this dude.

This kind of analysis nearly always fails to account for why these businesses aren't able to recoup expenses. While many startups are will never find a large enough market to be profitable, many others choose to invest earnings and take on debt in order to improve their product further and acquire market share.

Could Uber or Lyft be profitable if they slashed incentive spending, cut R&D efforts (self driving cars, improved matched algorithms, etc), closed up shop in cities too small to be profitable, and fired everyone that wasn't critical to keeping the business running?

Maybe. But this guy certainly doesn't know enough to tell.

I work at Uber. I was on the wifi yesterday. I was successfully using blind. If business insider would like video evidence, I'm happy to work with their reporters 1:1.

There's a lot of problems at the company and it's been a difficult week for many of us here. Not having access to blind is not one of them.

Most of the comments here are missing the point. The functionality and architecture of git aren't what's being criticized, but the way this functionality is presented. In fact, gitless if built on top of git's architecture.

As far as I can tell, gitless still embraces git's distributed and incremental core. It just packages it in a way that's easier to digest. If you're a veteran and know your way around native git flawlessly, gitless probably presents little value. But if you're a notice just learning the tool, gitless can probably accelerate your learning curve without sacrificing functionality—you can still bypass gitless's abstraction layer when you need it.

Sharding your application geographically is a quite a bit of complexity and requires a lot of work developing support infrastructure to manage load balancing, failover, and placement. One of the advantages of SOA is that different services can have different architectures.

To be precise, we do do geographic sharding in the services that benefit from it, but avoid it in the services that don't.

Also note that the assumption of region based partitioning doesn't extend to all applications. Analytics, for example, may want to dice and slice the data along different dimensions. Partitioning is a convenient abstraction for managing marketplace scale, as you mentioned, but inconvenient elsewhere :).

When I first started interviewing, I got turned down at a lot of companies because they were concerned about my self taught background, often in spite of strong project work and interviews. In spite of so many rejections (2 offers after 17+ interviews), I've been wildly successful in my current job—I received a promotion in the first 6 months and have since held down tech lead roles.

Look, the bottom line is that companies optimize for false negatives. In order to achieve a high accuracy rate, [tests must have exceptionally low false positive rates](https://www.math.hmc.edu/funfacts/ffiles/30002.6.shtml), just based on stats. I won't work out the math for you—you sound like you're perfectly capable of plugging numbers into bayes theorem—but that implies that even very good engineers are likely to get false negative rejections at many companies. It does not mean, however, that those companies are necessarily judging you based on unfair criterion, and I don't thin it's fair, thoughtful, or mature to indicate otherwise—(especially because some of my strongest coworkers at Uber are from less prestigious schools in the midwest...)