serverholic

I really don’t understand the people who have a problem with rust. Do you not value the increased memory safety? Now that Microsoft and Google are adopting rust and reporting significant decreases in memory related bugs it’s pretty clear that rust does make a difference.

Thanks, can you explain what you mean by “vouched”? I’ve noticed that my comments have been getting much less engagement recently and sometimes they don’t show up.

Perhaps the biggest issue is the mental framework that people use to approach AI. I've found that there are so many assumptions in people's thinking and these assumptions are strange and/or don't match up with the evidence we have so far.

First of all, you have to ask the question "what is intelligence?". What I've found is most people think intelligence is deeply connected to humanity or that intelligence is synonymous with knowledge. Really, intelligence is the ability to reason, predict, and learn. It's the ability to see patterns in the world, learn and act on those patterns. It doesn't have to be human-like. It doesn't mean emotions, wants, dreams, or desires. It's cold, hard logic and statistics.

Secondly, you have to ask "do I think it's possible for computers to be intelligent?". A lot of people have issues with this as well. The thing is that if you say "no, computers can't be intelligent" you are basically making a religious claim because we have brains and brains are intelligent. We can literally grow intelligence inside a human being during pregnancy. It might be difficult to program intelligence, but saying it's impossible is a bold claim that I don't find very convincing.

Third, you have to ask "if a computer is intelligent then how does it act?". So far the closest thing we have to general intelligence is an LLM model like GPT and even then it's questionable. However, reports indicate that after initial training these models don't have a moral compass. They aren't good or evil, they just do whatever you ask. This makes sense because, after all, they are computers right? Again we have to remember that computers aren't humans. Intelligence also means OPTIMIZATION, so we also have to be careful we don't give the AI the wrong instructions or it might find a solution that is technically correct but doesn't match up with humans wants or desires.

Four, you have to ask "can we control how these models act?" and the answer seems to be kinda but not really. We can shift the statistics in certain ways, like through reinforcement learning, but as many have found out these models still hallucinate, and can be jail broken. Our best attempts to control these models are still very flawed because basically an LLM is a soup of neural circuits and we don't really understand them.

Fifth, you have to ask "ok, if a computer can be intelligent, can it be super intelligent?". Once you've gotten this far, it seems very reasonable that once we understand intelligence we can just scale it up and make AI's super intelligent. Given the previous steps we now have an agent that is smarter than us, can learn and find patterns that we don't understand, and act in ways that appear mysterious to us. Furthermore, even if we had solid techniques to control AIs, it's been shown that as you scale up these models they display emergent behaviors that we can't predict. So this thing is powerful, and we can't understand it until we build it. This is a dangerous combination!

Finally, add in the human element. All along the way you have to worry about stupid or evil humans using these AIs in dangerous ways.

Given all of this, anyone who isn't a bit scared of AI in the future is either ignorant, superstitious, or blinded by some sort of optimism or desire to be build a cool sci-fi future where they have space ships and robots and light-sabers. There are so many things to be worried about here. The biggest point is that intelligence is POWER, it's the ability to shape the world as one sees fit whether that's the AI itself or humans who program it.

I’m curious why you are so confident in your assertion. It seems to me that an advanced statistical model of the world is an essential component of AGI. How do you know that we aren’t a few breakthroughs away from AGI?

Some recent papers have shown significant performance improvements when these models are allowed to respond to their own outputs.

How do you know that putting an LLM in a fancy loop with access to external memory and tools isn’t AGI?

I don't think that's necessarily proof against the Bayesian brain. It seems reasonable that the brain is also using its statistical models to assess the relevance of new evidence. So it's not just "new evidence, I need to update" but more like "new evidence, how likely is this true? I'll update according to the magnitude of the likelihood."

I'm skeptical that RNNs alone will outperform transformers. Perhaps some sort of transformer + rnn combo?

The issue with RNNs is that feedback signals decay over time, so the model will be biased towards more recent words.

Transformers on the other hand don't have this bias. A word 10,000 words ago could be just as important as a word 5 words ago. The tradeoff is that the context window for transformers is a hard cutoff point.

Yes. Capitalism.

It’s interesting how the first benefit he lists for AI is productivity. It was a similar message in one of OpenAI’s recent blog posts.

It is definitely not as good as GPT-4. It's much less creative and detailed in its answers. The answers are more surface level and shallow.

I don’t understand why the threshold is “never”. Isn’t it entirely possible that the AI is learning a model of chess but this model is imperfect? What if AIs don’t fail the same way as humans?

This is likely related to data-oriented design

https://en.m.wikipedia.org/wiki/Data-oriented_design

The idea is that you use data structures and algorithms to make better use of CPU cache.

I don’t think you really understood what I said.

If your goal is population distribution then you are inevitably hiring based on attributes other than skill.

For example, women don’t make up 50% of the engineering talent pool so if your goal is 50% women then you have to lower standards to achieve that.

That sounds nice and all but the distribution of engineers doesn’t match population distributions.

This leads to organizational pressure to hire based on population distribution. Doing so inevitably means hiring based on attributes other than skill.

More than that, it puts your data into your hands. If someone kicks you off a server for whatever reason you still have your account and you can visit other servers. With mastodon you’re at the mercy of whoever owns the server.

Same with mastodon. Your id is tied to the server so if you get banned you have to create a new id on another server.

Mastodon is worse because your data is in the server owner’s hands. At least with Bluesky your identity and server are separate so you still have your data.

He literally said that's what he's looking for when he announced his company. He also shows off his work stats on twitter.

Politics is just decision making in groups of people. Similar analyses can be used whether you're talking about government, schools, your workplace, open-source projects, your household, etc.

As a general rule I tend to always interpret these discussions as "how humans interact with each other" unless the discussion itself is about the law.

Wow. It's clear nobody gives a fuck. We're all doomed aren't we.

I’m inclined to believe you. In my experience men are socialized to downplay mental illness whereas women are the opposite, they wear their illnesses on their sleeve.

Because then my choices are:

1) give in and pretend to go along with it.

2) look like an asshole at a wedding.

Sorry if I don’t want to ruin someone’s wedding lol.

Just so you know, if I ever met you at a party I probably secretly resent you and I'm guessing I'm not alone.

I DONT LIKE DANCING. I DONT LIKE THE THING THAT YOU LIKE.

My issue is the claim that these people are poorly adjusted.

Perhaps these people are responding to other peoples sense of superiority. These other people don’t get called out because they are “normal” and we assume that normalcy is automatically superior.

Or perhaps people are different and they find themselves pressured and bullied by “normal” people.

Whenever someone has done that I end up pretending it’s fine until it’s socially acceptable to stop.

Yeah I just don’t find dancing fun. That’s it, end of story.

Wherever there’s dancing at a wedding or something I just think of it as something I have to get through so I don’t appear rude.

one touches people's emotions, the other doesn't.

Yes and then someone comes along and makes a change that breaks something seemingly unrelated and it’s less likely to be caught because there is no type system.

And please don’t say tests solve that because they don’t. They help but they don’t solve it.

See this is what I’m scared about in my current job. We have some people who want to use dynamically typed languages and some who want to use static.

I don’t think being able to brain dump code is a good thing and it leads to an unmanageable mess when you have a large codebase. Types are essential as far as I’m concerned, but I constantly have to get into arguments with the dynamic type fans because their “flow” Is being restricted.

It looks like this guy has created multiple accounts and posted the same URL multiple times.

A little unsafe is better than all unsafe.