skepticATX

Why aren't employees more honest about AI and concerns regarding it

Because it’s being pushed by the c-suite and board, so most people aren’t in a position to push back. And it’s a quasi-religion now, so even pushing back at all can result in being branded as a “skeptic”.

People do raise concerns, though. And the responses aren’t very satisfactory. For example, the biggest concern that has been raised thus far is open PRs, and the answer has always been that all we need is AI code review.

Yes, I am exhausted. Most of my company is obsessed with agents, because everyone wants to be seen as AI first. There is little thought going into usage. No care for long term maintainability and quality. Our product is actively worse by many metrics, but no one cares because we marketing can say “agents”.

The sad part is that this technology is incredible. It’s us choosing to turn it into a slop cannon (and the labs sure seem to encourage this).

I want to leave the industry as soon as I can.

Maybe I am missing it, but I don’t see where AI solved anything in this anecdote. Rather, a team of medical professionals and a patient willing to advocate for herself did.

The reason that I’m looking for an out is that it’s turned everyone I work with into imbeciles.

Nobody wants to think anymore. Coworkers are now just intermediaries for their LLMs. Talking to them is just talking to the LLM - sometimes directly copied and pasted, sometimes minimal effort to conceal what they’re doing. It is so disheartening.

And the sad part is, LLMs are incredible and can enable you to do much better work if you can stay in the loop, and stop focusing only on shipping speed. But from what I have observed, very few people care to do this. Who cares about substance when middle management thinks your productivity is 10x?

You generally don’t assign work to an intern just for the output, though.

Reviewing code is absolutely different from writing it, and in my opinion much harder if the goal is more than surface level understanding.

This is what I am still grappling with. Agents make more productive, but also probably worse at my job.

One explanation is that this is effectively a quid pro quo, given Brockman’s enormous financial support of the current president.

So much software just flat out doesn’t work that people don’t even notice how bad X has gotten.

I actually think that plateauing is the best case scenario for big labs.

I think there are three broad scenarios to consider:

- Super-intelligence is achieved. In this scenario the economics totally break down, but even ignoring that, it’s hard to imagine that there are any winners except for the the singular lab that gets here first.

- Scaling laws hold up and models continue to get better, but we never see any sort of “takeoff”. In this scenario, models continue to become stale after mere months and labs have to spend enormous amounts of money to stay competitive.

- Model raw capabilities plateau. In this scenario open source will catch up, but labs will have the opportunity to invest in specific verticals.

I believe that we’re already seeing the third scenario play out, but time will tell.

This seems roughly equivalent to liability insurance.

It’s much easier to accept fatalities caused by other humans because there is someone to hold responsible. Will autonomous vehicle companies be held responsible when they cause fatalities?

It also goes beyond just the total number of fatalities. Just like we don’t accept DUIs, we shouldn’t accept negligence or laziness from autonomous vehicle developers even if their product is safer than human drivers.

I think the more interesting question is who will be on the panel?

A group of ex frontier lab employees? You could declare AGI today. A more diverse group across academia and industry might actually have some backbone and be able to stand up to OpenAI.

How is this not a terrible deal for Microsoft? I’m not confident that an “expert panel” will prevent OpenAI from prematurely declaring AGI.

This was really a bad release for OpenAI, if benchmarks are even somewhat indicative of how the model will perform in practice.

Never heard of those either. And I took the highest level math courses offered. The only competition I can remember is participating in is Academic Decathlon.

I’m in the US but this was a while back, in the south. It was a highly ranked school and ended up producing lots of PhDs, but many of the families were blue collar and so there just wasn’t any awareness of things like this.

OpenAI’s systems haven’t been pure language models since the o models though, right? Their RL approach may very well still generalize, but it’s not just a big pre-trained model that is one-shotting these problems.

The key difference is that they claim to have not used any verifiers.

I’m not trying to take away from the difficulty of the competition. But I went to a relatively well regarded high school and never even heard of IMO until I met competitors during undergrad.

I think that the number of students who are even aware of the competition is way lower than the total number of students.

I mean, I don’t think I’d have been a great competitor even if I tried. But I’m pretty sure there are a lot of students that could do well if given the opportunity.

Has anyone independently reviewed these solutions?

My proving skills are extremely rusty so I can’t look at these and validate them. They certainly are not traditional proofs though.

Access to cheap goods may lead to short term satisfaction, but will never lead to long term fulfillment.

Where are the expectations coming from? The major labs continually claim that these models are now PhD level, whatever that even means.

The fact that so many feel the same way about this technology (I do too!) is an indictment of humanity, not the technology itself.

We _could_ use this to empower humans, but many of us instinctively know that it will instead be used to crush the human spirit. The end result of this isn’t going to be an expansion of creative ability, it’s going to be the destruction of creative jobs and the capture of these creative mediums by a few large companies.

The industry only has themselves to blame. When you promise literal utopia and inevitably don’t deliver, you can’t be surprised by what happens next.

Listening to Satya in recent interviews I think makes it clear that he doesn’t really buy into OpenAI’s religious-like view of AGI. I think the divorce makes a lot of sense in light of this.

Sad that 18F is gone and that USDS is effectively gone. I’d long had in the back of my mind that I’d take take a short sabbatical at some point and work for one of these organizations - looks like that’s not happening.

How many still believe that scaling up base models will lead to AGI?

They'd absolutely report on it. I consume news from many different conservative and liberal sources. NYT newsroom is still one of the best and while bias can't be completely eliminated, they are fair. The opinion section of course leans to the left, and I find little value in it, but that has no bearing on the newsroom.

There are more oddities for sure. For example, when they cancel a multi-year contract they are considering the entire value of the contract as savings even if a significant chunk of the money has already been distributed.

I think this was an honest mistake. But it demonstrates exactly why it’s so dangerous for Elon Musk and our president to make sweeping accusations before understanding the situation.

Lives are being ruined right now - federal employees, citizens who rely on the programs being cut, and people around the world who benefited from US generosity.

The sad part about all of this is that clearly in a system as large as the federal government, there is waste. Elon has the opportunity to make a real impact, but he’s demonstrated that he just cares about optics and will not put in the effort required.

Yes, there’s practically no chance that this is true.