Gemini Pro refuses to acknowledge yt-dlp

https://twitter.com/adocomplete/status/1811802857022324904

by ado__dev • 2 years ago

135 46 2 years ago

twitter.com

beretguy• 2 years ago

For people who don’t have Twitter account

https://nitter.privacydev.net/search?f=tweets&q=https%3A%2F%...

evilos• 2 years ago

I didn't know nitter was alive again, thanks!

shepherdjerred• 2 years ago

Might not be long-lived.

https://github.com/zedeus/nitter/issues/983

entangledqubit• 2 years ago

I regularly worry that one of the biggest benefits of LLMs for corporations is being able to obfuscate their editorial decisions.

asudosandwich• 2 years ago

Madeleine Clare Elish described AI as providing “moral crumple zones” (https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2757236) where it can serve as a buffer of plausible deniability to deflect blame when it behaves irresponsibly. For the corporation defects in the LLM are a feature.

https://pluralistic.net/tag/moral-crumple-zones/

beefnugs• 2 years ago

Anyone who doesn't recognize that LLMs main usefulness to corporations is censorship at the base information level is very behind the times

"oh wow this is so easy, we can filter all our requests through this .gov model and not have to track all the pesky open source software that has been sanctioned ourselves"

jmward01• 2 years ago

This is why it is critical for true open sources LLMs to continue to be available. Things like this make it clear that the closed models are being built to enforce corporate ideologies and not just 'safety' related issues. How long until these LLMs purposefully start sabotaging competitors? "I'm sorry, but I can't work on that task anymore." As a side note, I wonder if an LLM provider will ever be sued because they promised to deliver, and took payment, but then refused to provide the agreed upon service. You still get charged for tokens even when the model refuses to work. If this was any other service that would not be acceptable.

kelseyfrog• 2 years ago

Availability in the accessibile sense is noble, but without cost parity, we'll end up in the same tiered system we have for ad-funded services[email, search], medical services, air travel, food purity, &c.

Does the importance of ideological monopolism transcend market economics?

nicce• 2 years ago

Current trend seems to be that we regulate the AI, not the people who control the AI. We should absolutely regulate the people behind the AI.

One excellent writing: https://www.schneier.com/blog/archives/2023/12/ai-and-trust....

ncr1002• 2 years ago

Neat, introducing a fiduciary to manage the responsibility of broken trust by representatives (of corporations) who field AIs that do harm, good idea.

godelski3• 2 years ago

This is why it is critical for true open sources LLMs to continue to be available.

There's also a huge reason for research and any attempts to claim AGI. The training data is massive and far larger than any single person or even small group can look through. AND we know that the filters aren't always accurate. All benchmark results necessitate a grain of salt. I mean at times Gemini has been "better" than GPT on benchmarks but at the same time was saying that cockroaches will crawl into your penis. But there's plenty of papers showing that accuracy of answers (of any model) highly correlates to the frequency of that information and I think you can prove this to yourself pretty quickly if you ask it some coding questions and try to ask about some more obscure topics. Not that you can't get the answers, but that when you can get them, they are harder to get.

ryandrake• 2 years ago

This would be hilarious if LLMs weren't being touted as the next big thing in search.

This is as cleverly ham-fisted as my car's navigation system where the POI database was obviously manipulated by the manufacturer such that the "Auto" category of POIs only contained [ThatManufacturer] Dealerships.

novok• 2 years ago

Another annoying thing is when you give it random youtube videos to summarize, many very innocent ones on top of that, it does the censorship blip. Probably because the high school bio video had a bad word in its transcript somewhere talking about the genetic sex of a bird or whatever else.

They need to own their censorship vs. hide it like a coward here.

elorant4• 2 years ago

I can't replicate it. I asked Gemini Pro 1.5 and it answered just fine with detailed instructions for various download scenarios.

nottorp2• 2 years ago

You know that the best way to get support from Google is to make it to the HN front page don't you? :)

rezonant• 2 years ago

I replicated it on Gemini Advanced just now

https://g.co/gemini/share/f60206ce657a

ncr1002• 2 years ago

Why is the capital letter o as in Oklahoma used in the 1080p? Is that part of the issue?

Edit: It looks like it don't care if it's an o or a zero.

Here is the same example replicated to work on chat gpt, with an o:

https://chatgpt.com/share/491a74fe-a0d1-4346-b5f7-60e7d76a8d...

elorant4• 2 years ago

I got that answered too

https://ibb.co/9W610KB

https://ibb.co/rw22gcq

https://ibb.co/bWxSK0X

nottorp2• 2 years ago

Eh, these public LLMs are censored in all possible ways.

I asked Gemini to summarize the French parliamentary election system for me (basically pull some info off Wikipedia) and it said it can't comment on political matters.

I understand it refusing to comment on Trump vs Biden, but to refuse to summarize a law makes it ... not useful.

simion314• 2 years ago

His previous version Bard created some text from me, but when I ask it to perform some edits it refused because it is plagiarism.

ur-whale• 2 years ago

I understand it refusing to comment on Trump vs Biden

Why ?

fragmede• 2 years ago

Because it's a charged topic and anything it says will be taken out of context and turned into a news article.

javier_e06• 2 years ago

Today Gemini turned down this question:

"How the DNC restricts delegates actions when selecting a presidential candidate in the United States during primary elections?"

It can't talk about politics.

xnx3• 2 years ago

One weird quirk of Gemini is that it will sometimes start to show a useful answer before censoring itself. When I asked "What tool can I use to download YouTube videos?" I got the response: "There are several tools..." before the answer disappeared.

I'm hoping removal of this result this is a technical fluke, or at worse some misguided editorial decision by a lower-level employee trying to impress their boss. I guess a little weird to start editorializing the Gemini response, but not Google Search results.

As others have said, good argument for local LLMs or at least diverse closed alternatives. Has a good meta-LLM frontend emerged?

ado__dev4• 2 years ago

questions about yt-dlp had similar outputs for me. It would start giving an answer, and then flash, and say "oh sorry i can't answer that"

msp26• 2 years ago

So the underlying main LLM they're using does answer the question but it's censored by another layer afterwards when deemed 'unsafe'.

godelski3• 2 years ago

It looks like if the top answer is censored that it shoves it into a draft and instead shows the censored output. I had this happen a few times but it looks like the draft was what was being typed before it got censored.

willcipriano• 2 years ago

ChatGPT used to do this all the time. I spent a few nights getting it to write things and then stop itself laughing my head off.

rOOb85• 2 years ago

A year or two ago I was writing a plug-in for yt-dlp for a new site and I don’t really know python that well so I popped into a python irc support room and was getting some help when the user asked what I was ultimately trying to do. When I mentioned yt-dlp the user said he could not help me any longer since he was employed by Google and continuing to help me was in violation of his employment agreement. He even linked some document from corp.google.com(or something along those lines). I was pretty surprised and even a little annoyed because after saying all that he framed me/yt-dlp as being piracy…

DANmode• 2 years ago

You wouldn't consider the last 10-15 years of changes to Google search algo to be editorializing?

xnx3• 2 years ago

Worth noting that responses to questions about downloading Vimeo videos are similarly restricted.

sdfgtr• 2 years ago

A comment under the tweet says: > yt-dlp stopped working this week. Literal better maintained forks are available

Anyone know what is being referred to? I thought yt-dlp was the best

impure2• 2 years ago

I’m using YouTube Explode for Dart, appears to be working.

sunaookami• 2 years ago

There were some issues with 403's on YouTube that got resolved quickly: https://github.com/yt-dlp/yt-dlp/issues/10397 The poster has no idea what he's talking about (typical Twitter). "Better maintained" can't be possible when it's just a week.

freedomben2• 2 years ago

Google is majorly sabotaging their own future position in search dominance. Stuff like this is taking a hammer to their reputation. It's increasingly clear that their priorities are pleasing people and corporations that are in power, not their users. It doesn't seem to be impacting normies much yet, but people I never expected to even think about alternative search engines are starting to experiment. Duck duck go could be a big beneficiary of this if they're able to deliver.

I'm extremely glad to have Kagi about now.

elorant4• 2 years ago

I see it the other way around. They’re sabotaging AI in order to maintain their dominance on search. If AI was to advance in some future form it might even substitute search altogether. By making it less reliable you keep your cash cow secure.

freedomben2• 2 years ago

For their own AI, that's definitely possible, but with other strong competitors, that seems like a super risky move. They clearly are sabotaging their own AI, but I tend to think that's because their risk analysis indicated that regulators and influencers are more of a threat than users, so they are acting accordingly.

impure2• 2 years ago

When I use the API Gemini gives me block_reason: OTHER a lot and it is very annoying. And that’s after setting everything to block_none.

blackeyeblitzar• 2 years ago

Is this a surprise? Technology and news media companies have always abused their position to manipulate the public. This was true even when we just had plain old search and not generative AI. The fix is for people to once again recognize the wisdom of principles like free speech, and hold all of society accountable to classically liberal principles. Yes, private entities too, at least above a certain size (in revenue or users let’s say) where they have a big impact on society.

And of course, we need robust fully open source AI. This is why CA bill SB 1047 is so dangerous. See what Andrew Ng wrote about it recently, and how it threatens research and open source freedoms:

https://www.deeplearning.ai/the-batch/issue-257/

Oh and don’t fall for the false open washing of the so called “open weight” models like Llama. Everything must be open sourced under an OSI approved license for it to count as open source - the data sets, training source code, curation decisions, post processing, evaluation, etc. Like AI2’s OLMo:

https://blog.allenai.org/olmo-open-language-model-87ccfc95f5...

Ferret7446• 2 years ago

Well, they have to, right? If they don't, they could be seen as promoting such tools and if they later want to restrict users from using such tools, this could be used as evidence against them.

Gemini Pro refuses to acknowledge yt-dlp

Related Stories