Silicon Valley is turning into its own worst fear (2017)

We've had isolated demos of this and that for many decades, all of varying difficulty of course, but I for one definitely agree with the post. I don't feel like I could go and buy a generic home assistant robot that can perform a variety of basic tasks anytime soon. Demos are nice, but they're kinda like news about new batteries, always a few years away from the real thing.

add-sub-mul-div• 2 years ago

I'd bet many more than a few years away from a robot that can arbitrarily navigate your house, gather laundry from several baskets, the floor, etc. Divide into whites/colors or warm/cold loads if desired. Load them into an arbitrary washer, measure out detergent, use the right washer settings for the given load. Then come back and move them to the dryer, then come back and unload, fold, put away, etc. And don't get me started on sheets.

It's much easier than it should be to fool people with demos.

fragmede3• 2 years ago

solid state batteries starting to actually hit the market, as in you can buy one today.

https://yoshinopower.com/collections/portable-power-stations

ValentinA232• 2 years ago

10 years ? 20 ?

https://mobile-aloha.github.io/

(note: in some videos the robot is teleoperated)

catskull2• 2 years ago

but how far are we from some bot ordering you a breakfast burrito on DoorEats and having it delivered? I can 100% envision a bug in one of the current crop of "AI" products resulting in mass burritoing.

willsmith72• 2 years ago

The bot already does that, you command it through your web or mobile interface

pixl97• 2 years ago

Heh we're all standing on the tracks looking at the incoming light here at HN, but don't worry, all the down voters are yelling it's very far away over the ever increasing rumble.

JKCalhoun• 2 years ago

This summer, Elon Musk spoke to the National Governors Association and told them that “AI is a fundamental risk to the existence of human civilization.”

And here I was going to suggest that billionaires, unbridled mega-corporations were the fundamental risk to the existence of human civilization.

Musk gave an example of an artificial intelligence that’s given the task of picking strawberries.

Also odd since it's more likely that a corporation, in the name of maximizing profits, would make decisions that threaten humanity. We can start with Bhopal, India. If you find fault with that example I am sure there are plenty of others, some probably a good deal more subtle, that others can suggest.

Me, not worried at all about AI.

lupire2• 2 years ago

it's more likely that a corporation, in the name of maximizing profits, would make decisions that threaten humanity.
Me, not worried at all about AI.

Now imagine AI in the hands of a corporation building murderbots and propaganda bots.

ein0p• 2 years ago

You don't have to imagine. Both the corporations and USG have been doing that for the last decade at the very least. If you want to test this, try posting anything controversial (AKA not 100% in line with the neoliberal agenda) in the YT comments section. Gone within seconds. Those aren't human moderators.

WorkerBee28474• 2 years ago

And here I was going to suggest that billionaires, unbridled mega-corporations were the fundamental risk to the existence of human civilization.

Depending on the measure [0], it's possible that the richest billionaires and least-bridled corporations are both well in the past. Human civilization survived, and will continue to survive.

[0] https://en.wikipedia.org/wiki/List_of_richest_Americans_in_h...

jprete2• 2 years ago

The idea that AGI doom scenarios are really late-stage capitalism is interesting and strikes me as fundamentally correct. The difference between the AGI takeover and capitalism is just the choice of metric to optimize on.

But, I think, it's the act of trying to optimize on a metric itself that is the source of the destruction. Unmeasurable human values can't survive an optimization process focused on measurable ones.

catskull2• 2 years ago

It makes me question if I've ever seen a metric employed by a company that didn't have a detrimental effect on the happiness of the people involved. We are all unique but we're also really, really similar and it's in those uniquenesses that we find real happiness. But if we're only a number of sales/dollars/tickets/loc then the distinction is a zero-sum game; I only stand to lose a dollar that you make.

fzeroracer• 2 years ago

I personally have never seen a metric that was actually beneficial for a company. Because the moment a metric exists, all rationality and reason is thrown out the window in favor of optimizing that metric. It doesn't matter if it destroys the company or customer trust in your product or runs out valuable employees. None of those are part of The Metric.

It's the whole 'we increased shareholder value by 1% even if we won't exist next year' scheme. The notion of any sort of sustainability or long term planning has long vanished.

nradov2• 2 years ago

Business is not a zero-sum game, especially not technology businesses that constantly innovate. All sides of the transaction can benefit. Optimizing for profit or similar metrics doesn't inhibit those benefits.

tmpz22• 2 years ago

We lose what we cannot measure and sometimes that means losing something very important.

wazoox• 2 years ago

As Bob Kennedy said in 1968 :

Our Gross National Product, now, is over $800 billion dollars a year, but that Gross National Product - if we judge the United States of America by that - that Gross National Product counts air pollution and cigarette advertising, and ambulances to clear our highways of carnage. It counts special locks for our doors and the jails for the people who break them. It counts the destruction of the redwood and the loss of our natural wonder in chaotic sprawl. It counts napalm and counts nuclear warheads and armored cars for the police to fight the riots in our cities. It counts Whitman's rifle and Speck's knife, and the television programs which glorify violence in order to sell toys to our children. Yet the gross national product does not allow for the health of our children, the quality of their education or the joy of their play. It does not include the beauty of our poetry or the strength of our marriages, the intelligence of our public debate or the integrity of our public officials. It measures neither our wit nor our courage, neither our wisdom nor our learning, neither our compassion nor our devotion to our country, it measures everything in short, except that which makes life worthwhile. And it can tell us everything about America except why we are proud that we are Americans.

kortilla3• 2 years ago

The difference between the AGI takeover and capitalism is just the choice of metric to optimize on.

Not specific to capitalism. The same thing occurs with any government/economic structure that is optimizing for a metric. Think of a communist government using “happiness” to drive decisions that just silently kills or drugs everyone who isn’t happy.

cmrdporcupine2• 2 years ago

The fundamental difference of capitalism vs the other instrumental forces you are describing or intimating is that capitalism is self-reproducing. Through the explosion of the profit motive and the harnessing of wage labour it continually opens new markets, and demolishes limits to growth.

jprete2• 2 years ago

I agree with you that this isn't really specific to capitalism, and is specific to using metrics as proxies for values.

cmrdporcupine2• 2 years ago

"AI" in the current hype cycle (LLM generative ML yada yada) is just classic capitalist industrial automation, applied to one of the few areas that still had echoes of the pre/proto-capitalist "craft economy"

Prepare, nerds, to be put through the same process as seamstresses, weavers, blacksmiths, lumberjacks, etc.

So much of what people identify as post-modern or late-stage or whatever is just the continuation of the same intrinsic process that has been happening since the end of the 18th century.

All that is solid melts into air.

EasyMark• 2 years ago

Except we can eat the rich, we can't eat the machines.

mattmaroon• 2 years ago

How unfortunate that this article aged so well.

gkoberger• 2 years ago

If you’re not familiar with Ted Chiang, make sure you put Story of Your Life and Others on your reading list.

throw0101d• 2 years ago

Story of Your Life

Adapted into the movie Arrival, directed by Denis Villeneuve:

* https://en.wikipedia.org/wiki/Arrival_(film)

jackschultz• 2 years ago

Then I realized that we are already surrounded by machines that demonstrate a complete lack of insight, we just call them corporations. Corporations don’t operate autonomously, of course, and the humans in charge of them are presumably capable of insight, but capitalism doesn’t reward them for using it. On the contrary, capitalism actively erodes this capacity in people by demanding that they replace their own judgment of what “good” means with “whatever the market decides.”

Excellent quote. We say we're in a rational world where we make rational decisions in our societal game we've all been told we have to play - capitalism where money is the determinant of success vs failure for corporations, families, individuals.

But step back and when looking at the question of whether it's rational to us as humans be playing this game and it is not rational at all. Why are we not deciding that food and places to live for everyone is the determinant of success of a country, society? Or happiness?

60 Minutes has a segment about Bhutan from a couple weeks ago [0] about this. They lived by something they named "Gross National Happiness". Which feels weird to type but again stepping back, it's because our whole lives we're told that "Gross Domestic Product", overall money, is the determinant of "best" and that's so engrained for us.

On a different note, Ted Chaing's short story books [1][2] are incredibly, incredibly good. I'm reading them again and read "Story of Your Life" earlier today. Being able to write fiction like that makes it much more trusting to listen to what someone has to say on other topics. And saying that seems like another topic - how we're told to downplay fiction compared to non-fiction, when our brains evolved for stories. But that's for another comment.

[0] https://www.youtube.com/watch?v=7g_t1lzn-1A

[1] https://en.wikipedia.org/wiki/Stories_of_Your_Life_and_Other...

[2] https://en.wikipedia.org/wiki/Exhalation:_Stories

bamboozled3• 2 years ago

Why are we not deciding that food and places to live for everyone is the determinant of success of a country, society? Or happiness?

I think a lot of countries do, just not the USA. Money is truly king in the USA. But in Europe, developed parts of Asia, people are definitely not as rich, but decent quality basics are much much more accessible and considered a basic right.

lupire2• 2 years ago

People in USA have an extremely rose-tinted view of the rest of the world.

hotstickyballs• 2 years ago

Because once that’s the priority it becomes easier to game the system rather than delivering food and places to live.

ponector2• 2 years ago

Btw, USA has more accessible housing than EU. Food is also cheaper.

What is screwing Americans - healthcare and urge to buy things people don't need.

nicbou• 2 years ago

Bhutan is about to create a special economic zone where this idea is relaxed, because they are facing a brain drain. The Bhutanese also want shiny things.

roenxi• 2 years ago

Why are we not deciding that food and places to live for everyone is the determinant of success of a country, society?

That is what people do decide. That is why everyone slowly converged on capitalist economies with some level of social safety net - it is the system that best resists mass starvation and provides material comfort for as many people as possible. There is no alternative that is better at feeding (or housing, for that matter) people; otherwise that'd be the economic system of choice.

nradov2• 2 years ago

Bhutan is having some serious problems.

https://www.cbsnews.com/news/bhutan-emigration-crisis-60-min...

kortilla3• 2 years ago

it's because our whole lives we're told that "Gross Domestic Product", overall money, is the determinant of "best" and that's so engrained for us.

I grew up in the US and literally never heard this. It’s a way to measure economic behavior but nobody actually cares about that number. For the people who are money focused, they look at things like PPI and individual incomes. I’m not making a value judgement on how much focus should be put on that, but the point is that GDP is irrelevant to citizens.

For a concrete example, Biden/Harris got backlash about the economy sucking despite GDP being great because of the massive erosion in purchasing power during 2022/2023. Them pointing to GDP to show how strong the economy was even made it worse.

ValentinA232• 2 years ago

when our brains evolved for stories. But that's for another comment

Go ahead !

deadbabe• 2 years ago

Gross National Happiness is a made up statistic that means nothing, it’s just used to distract from other atrocities. Don’t be naive.

nextworddev2• 2 years ago

Great article. Ted Chiang's "Understand" also happens to be my favorite short story of all time.

tetrisgm• 2 years ago

Agreed

sourcepluck2• 2 years ago

Exceptionally good. Wow, going to be reading some of this fellow!

nextworddev2• 2 years ago

Ted's writing feels like it compresses so much into each sentence.

He clearly practices extreme craftsmanship and is a very deep thinker.

kortilla3• 2 years ago

This article compares a strawberry picking machine killing all of humanity to increase strawberry fields with current corporations. This is stupid because corporations do have guardrails both internally and externally from society. All of the mega Silicon Valley corporations are not expanding by murdering people. Even in the ones that are expanding in ways that people question (social media), they are filled with humans who actually think they are doing the right thing.

Humans with morals are still very much in the decision chain and there is obviously a lot of debate about their morals, but them being there makes such a vast difference that the comparison to the strawberry AI is completely invalid. The strawberry AI isn’t even considering humans.

The article then builds on that false comparison for the rest of the article so there isn’t much to gain from the rest of it.

You can make the same lazy comparison to a completely socialist, centralized decision making by a government optimizing for a single metric (voter approvals, poverty levels, whatever). It has nothing to do with capitalism or the economic system.

TLDR; article says mega corps are the same as dangerous AI because they make optimizations in favor of profit that some people disagree with.

homefree• 2 years ago

I love his science fiction but he was wrong on AI in 2017 and is even more wrong on today.

ozim• 2 years ago

Problem is that for strawberries or diamonds or SV startup stuff there is a line where if you make something infinitely available it will also loose the value.

You cannot equate paper clip maximizer with SV companies. Because fitness functions are different.

Not saying that diamonds industry didn’t make world worse off but still somehow it didn’t take over the world to make people eat diamonds for breakfast. So the same there will be no entity that can make all people everywhere eat strawberries for breakfast all year long.

Scary part is finance industry that basically already is self conscious with all the rules baked in and no single person being able to grasp it.

Finance with AGI could already become paper clip optimizer - but it actually needs energy only. It doesn’t need humans anymore. So it would most likely fill in whole world with power plants and erase all other life just to have the electricity.

efitz• 2 years ago

The problem is not capitalism, it’s corporatism.

Or more precisely, it is that we allow a group of people, under the control of a single person or very small number of people, amass incredible power, and do so in an amoral framework in blind service to a goal of stock price growth.

The problem is a scale problem in my mind. If you limit the scale then all the other problems become manageable.

We wouldn’t need antitrust laws anymore if our tax laws made it unprofitable to own shares in a company with, for example, 50% of search market share or online retailing share.

notacoward• 2 years ago

The problem is not capitalism, it’s corporatism.

I can't agree with this, even though I think the corporate form has done more harm than good. While the idea of limited liability in exchange for a public good has some possible merit - building transport infrastructure was one of the original reasons they were created - giving away limited liability for nothing was a mistake. The shareholder value myth and corporate personhood have done even more damage. Nonetheless, not all capitalist ills are corporate ones. Many of the companies with the most capital doing the most damage are "shadow banks" - hedge funds, PE firms, etc. - which are run as partnerships, not corporations. They don't even issue stock, let alone care about its price. Also, nobody qualified to be opining about this could be unaware of the damage hyper-rich individuals are doing and will continue to do regardless of whether corporations are involved. Infinite accumulation of capital (and political power to go with it) is a problem, the corporate form is a problem, there's some overlap and they're both anti-free-market, but neither should be dismissed to focus on the other.

Silicon Valley is turning into its own worst fear (2017)

Related Stories