wvoch235

going to need to spend a lot more money than currently to support that...

i have significantly more faith in school children regulating an insane asylum

IMO attempts to make it low paid work will fail, just like almost every STEM profession. But... the number of engineers that we need who operate as "power multipliers" on team will continue to decrease. Many startup and corporate teams already aren't needing junior/mid level engineers any longer.

They just need "drivers", senior/lead/staff engineers that can run independent tracks. AI becomes the "power multiplier" in the teams who amplify the effects of the "driver".

Many people pretend that 10x engineers don't exist. But anyone who has worked on an adequately high performing team at a large (or small) company knows that skill, and quite frankly intelligence, operate on power laws.

The bottom 3 quartiles will be virtually unemployable. Talent in the top quartile will be impossible to find because they're all employed. Not all that unlike today, though which quartile you fall into is largely going to depend on how "great" of an engineer you are AND how effectively you use AI.

As this happens, the tap of new engineers who are learning how to make it into the top quartile, will cutoff for everyone except for those who are passionate/sadistic enough to programming without AI, then learn to program WITH AI.

Meanwhile the number of startups disrupting corporate monopolies will increase as the cost of labor goes down due to lower headcount requirements. Lower head counts will lead to better team communication and in general business efficiency.

At some point the upper quartile will get automated too. And with that, corporate moats evaporate to solo-entrepreneurs and startups. The ship is sinking, but the ocean is about to boil too. When economic formulas start dividing by zero, we can be pretty sure that we can't predict the impact.

The problem with the European mindset on this, is it's always involves bureaucrats taking their taxpayers money and allocating it in smarter ways than American investors who are doing it with their own money.

If that seems unlikely to work to you, then you possess critical thinking.

The US spends more on R&D (Private and Public) than the next 5 countries combined. Public research is and since the 70s has been a small fraction of research spending in the US. That's why their companies actually innovate.

If Europe doesn't change the inventive structures that are preventing investment in R&D, no amount of government money is going to fill that void...

Go to a typical Nordic community gathering space and speak English.... I'll wait. Are they treating you like a local yet?

And if those people they bring in who knows what they’re doing aren’t using AI after a certain point, that company isn’t going to make it either.

OP is talking about learning, we all did it at some point, and yes you were a liability too. Learning done through an LLM is the closest you can get to 1:1 training without having it.

The article though also largely talks about more experienced engineers not using it as a tool to increase their leverage. In which case it’s not all that different than throwing a task off to a JR engineer and reviewing this work. Even a JR Engineer who is well read on the latest research.

But most people don’t do research. Considering 70% of software engineers mainly write crud or mobile apps these days… this liability argument is really looking shaky.

Liability from what? taking out staging or even making a bug out to prod? Should you be learning space flight control systems as you code them? No. Should a jr engineer us it to learn how to do a left join that’s getting code reviewed anyway? Yeah that’s probably going to be faster than them polling team resources. Just the same as hitting google.

Startups for instance are usually more incentivized to move fast than deliver a bug free product. Large companies usually are too. That’s why software always needs updates.

I am sure we can all enumerate fields and projects where it’s problematic and dangerous to human life to accept ai output. But the vast majority of people on this site don’t work in those fields, and companies in those fields should already have (and likely do already have) control processes in place.

If you’re a company with no control processes, and you’re terrified by the prospect of ai code because of legitimate danger to your users… and you think it’s you… the engineer… holding back the gates… your company also is not going to make it.

Can we agree:

Claim 1: A “sustainable birth rate” is bounded by the efficiency of resource extraction and the repair rate of the environment for the damage the extraction does to it.

Claim 2: It would appear that technology has accelerated so far very closely at the same rate as global population growth.

Claim 3: Our efficiency of resource extraction has sky rocketed.

Claim 4: Our damage to the environment has exceeded its repair rate as this has increased.

Claim 5: A collapsed society would not immediately loose all of its population, and would likely be the most damaging to the environment in the shortest period of time. We won’t forget how to burn oil, but we won’t be using high efficiency well maintained engines when we do it.

With that. Seems like there are two options before humanity:

1. Burn it all down, reduce population growth rate (doesn’t matter if it’s controlled [Mao] or uncontrolled [The West Today]), eventually loose genetic entropy, and before that loose (or automate and replace) the labor force that allowed for the resource extraction.

The outcomes here are:

a) Society collapses (genetic entropy, climate change damage that has already been done, destabilization caused by population reduction measures, etc)

b) We automate labor and humans either are replaced entirely

c) or a small oligarchy of humans exist to rule that automation. That population must keep genetic entropy through gene editing (requires a lot of novel advancement which is harder with fewer people, unless we’re curating)

- or -

2. Address the reasons why people aren’t having kids in the west to maintain labor and genetic entropy, which mainly has to do with economic opportunity of young adults. And ensure that our resource extraction continues more efficient and that we either develop ways to limit damage or ways to accelerate the repair.

Outcomes are:

A) We fail to accelerate repair or reduce damage, and society collapses

B) We succeed and we’ve managed not to damage genetic entropy and don’t risk a conflict over population control.

The problem with option 1 is the implementation won’t be uniform, and population reduction globally cannot be achieved without some type of concerted effort. Consider for a moment how that is supposed to be implemented.

If things go wrong in that effort, that leads to conflict. It seems very likely that society collapses as well.

The other issue is considering the population number as the solution is it is a very short jump to the justification of genocide or some other form or population reduction. That also seems like it would accelerate conflict and therefore society collapse.

The problem with option 2 is there is only so much time before our damage exceeds allowable levels.

As an aside, and as someone who doesn't own a vision pro (non US pleb): While it is interesting to me if people find utility. I can't help but feel that the narrative on places outside of HN is a strong "no".

But, that is to be expected, the form factor isn't convenient yet. When mobile phones weighed 2KG few people used them on a daily. When it's miniaturized into the form factor of glasses, we'll all be daily users. That seems to me more like a question of when, not if.

Update: The SEC is blocking it https://decrypt.co/230085/bitcoin-wallet-exodus-deeply-disap...

Most private underground pedestrian tunnels are basements of existing buildings. Do you think the government should be using tax payer money to be cease/buy basements instead? Seems like a really odd use of resources just to not be able to kick out people who aren’t using the path for the intend purpose… but more so: Seems like something most local governments in North America would be too inefficient to handle without it turning into a project that takes 50 years and millions of dollars to complete 1 mile.

If German chancellor or US president visited Russia, and there were enough desire to arrest them, they would be arrested. I don't see practical means to prevent it.

Invasion. It would lead to invasion.

As much as I am against Putin… this article and headline are silly.

If Putin visited Germany he would be surrounded by his own armed security. Any attempt to arrest him would lead to a standoff that would immediately end in him leaving the country or risk a military confrontation that would almost certainly lead to WWIII.

Imagine the German chancellor, or US president were captured by Russian police, even if they justified it through some international organization they helped setup.

The headline and article should read “Putin is unable to visit Germany without making a geo political disaster.”

Unfortunately heads of nuclear states, and their inner circles, are truly above the law. The only way to enforce it, is to risk escalation.

The nature of the scam has evolved, and possibly the parties involved have changed, but the scams were always there.

or NORAD

Either this is a missing layer, and we’ll get there soon, or one could say verification is happening through the statistical model. We need to see if we can train hallucinations out by having it rely on a stable data store to retrieve facts rather than trying to pull facts from the model itself. This is still similar to how the brain has discrete components for memory storage and retrieval.

Honestly, yes, I don’t think it’s that far off. LLMs are a series of relatively simple transformers chained together. Through which we can simulate thought to the point that it not only passes the Turing test but it’s useful.

This is a bit of extrapolation but I would say the reason why we’ve been unable to locate “consciousness” in the brain is because it’s the same thing. Relatively simple neurones, chained together, to create thought.

On a philosophical level: this doesn’t make any claims for idealism or materialism, “experience” could exist at a more fundamental level of reality than matter. But IMO that would mean that the LLM is “experiencing” as well.

Absolutely, voice isn’t the only output.

As if that is so unlike what is happening in the electro-chemical computer that is our brain.

I agree with 99% of the statements made here, but I think a lot of them are now problems.

I think the big thing to consider is: We're still in the early days and there is a lot of low hanging fruit. It is possible that the number of potential injection attacks is innumerable, but it seems more likely to me that these will end up following patterns that will eventually be able to be classified into a finite number of groups (just with all other attack vectors), though the number of classifications might be significantly higher than structured languages.

That doesn't mean we won't find zero days, but it does mean that it won't be nearly as easy as it is today and companies will worry less about repetitional damage. If we could reliably have a human moderator determine if message is prompt injection or not, that should be able to be modelled.

I also think key to the approach is not to necessarily catch the injection before it's sent to the model, instead we should be evaluating the model response along with the input and block outputs that violate the rules of that service. That means you'd still waste resources with an injection, but filtering the output is a much simpler task.

Even as models get more capable and are able to do more and more tasks autonomously, that is most likely going to look like an LLM returning a code block that has a set of commands that are sandboxed. Like the LLM returns 'send-email <email> <subject> <message>`, which means there still will be a chance to moderate before the action is actually executed. Unless something changes significantly in the architecture of LLMs (which of course will happen at some point), this is how we would approach this today, and judging by bing's exfiltrated prompt, appears to be how they're doing it with search.

Also think, for things like Bing, and what most people are doing prompt injection for, the interest in this will subside once open source models catch up. This will also mean a new era for all of us because the genie will be fully out of the bottle.

I'm starting to wonder if the most effective way to protect against prompt injection is to use an additional layer of (hopefully) a smaller model.

As in, another prompt that searches the input and/or output for questionable content before sending the result. The question will be if that is also susceptible, but I suspect fine tuning an LLM only to do the task of filtering and not parsing will be easier to control.

This was my first thought as well. I'm kind of surprised the media is making such a big deal out of this. I would assume, considering how cheap it is to make a stratospheric balloon, it is either fairly common and the US has an equivalent program or its completely inferior to satellites.

The technology is pretty mature, Google ran a whole fleet of them with Project Loon back in 2011. https://en.wikipedia.org/wiki/Loon_LLC

See https://news.ycombinator.com/item?id=34340079, I would venture to say that this isn't any different to the reason why you know that 2+2 is 4.

I get what you're saying, and it's a database if you're talking about facts, but breaking down mathematics and being able to perform logic isn't a database in the true sense, it's still able to generate novel responses based on a set of rules.

My point here is even if it only "gets" the semantics, it has the ability to perform logic. It's just not very efficient. And, I'd say this isn't that far off from what is happening in our brains.

Do we really "get" logic, or do we rely on heuristics? Why do we know that 12 * 2 is equal to 24? Either because we remember our multiplication tables (look up from a table) or because we break it into smaller steps until we're left with pieces of the problem that we inherently know (including 12 * 2 means 12 + 12 which means take 12 and increment the number +1 for 12x times, or 10 + 10 = 20 and 2 + 2 = 4 so 12 + 12 = 24).

I don't see why that couldn't reasonably scaled up to advanced calculus.

I think the point that you're hitting on is that LLMs aren't "full brains" that have all of the components that human brains have, and that's true. But LLMs appear to essentially have the ability to simulate (or replicate depending on how far you want to go) executive function. As others have hit on, from that, you can either command other specialized components to be able to do specific tasks (like recall facts, perform logic, etc) or you can have it break down the problem into steps. Then take the output from that processing and form a coherent response.

The structure here is flexible enough that I'm struggling to find the upper limit, and if all research in the development of LLMs stopped tomorrow, and everyone focused on building with LLMs, I still don't think that limit will be found.

I would wager that it will eventually be able to do math and logic, and may already be able to with the right prompting. It can follow instructions quite well, and multi step problems can be solved by breaking the problem into small digestible steps, and having it write out each step, just the same as humans do. If it can't do this yet, it's likely just that it doesn't have enough parameters yet. But I don't see why this couldn't be achieved with fine tuning.

But ultimately I think it's just too computationally heavy to do math, remember obscure facts, or track rapidly changing data, with an LLM, and it's far more efficient to pass it off to a piece of specialized software, just the same as humans do.

Not an answer to the question but an aside related to it: After driving around in the Central Valley and looking at the massive agricultural industry there, it becomes very apparent that we're pumping a significant amount of water out of the ground and allowing it to evaporate into the atmosphere. I wonder how all of this additional moisture is impacting climate change as well. The scale of it all leads me to believe it's not insignificant, and the Central Valley is not the only place where this goes on.

This is true, but the renewable energy that was used to mine could have supplied the grid to reduce reliance on non-renewables.

If the renewable generation capacity was built entirely for the purpose of crypto mining and in other words, wouldn't have ever been built without it, you could argue that it's zero emissions (or more accurately, some miners are running effectively with zero emissions while others aren't).

However, since the supplies of renewable energy equipment are not unlimited, demand is already high (so you can't argue that you're driving innovation in the sector), and the production of said equipment is very much carbon positive (from mining to processing to transport), it's just not a very strong argument. If anything, mining with renewables reduces further investment in renewables for the grid as it drives equipment prices up, while still indirectly polluting the environment.

No matter how your slice it, terawatt hours of electricity are being used each year to run the networks, when the biggest problem we face as humans is a shortage of energy.

And I say all of this as someone who supports decentralized digital currency. We have a problem in our society where we obscure and mangle the narrative when something we dislike is true.

We should stop lying about what makes us uncomfortable and start focusing on the solutions to the problems. And there are plenty of reasonable options here:

1. Use the excess heat generated from mining to generate more electricity (with considerable efficiency loss) to supply the grid or to put back into the operation, or run any useful chemical process that requires heat.

2. Use the excess heat generated from mining to heat homes, buildings, etc.

3. Move to a more power efficient method of mining such as a PoS instead PoW. Or an entirely new method that hasn't been thought of yet.(Which, IMO, is the only way to prevent centralization once Quantum Computing or Fusion become commercially available).

Yes, and once this happens it significantly lowers the cost of automation in general, which then will continue to drive exponential acceleration. It will take time to get there but I believe it to be a lot closer than most believe.

Friend clearly didn't live very far away

It's a tool that significantly improves productivity. I've been a programmer ~15 years and the amount of time it saves me from looking shit up on the internet is insane. I think most people who say it doesn't work either aren't giving it a fair shot or don't know how to use it (in a real project, writing code with comments).

This productivity improvement should be advancing the open source community even faster, but instead most people are just pissing on it for various ideological reasons.