The new Bing will happily give you citations for a pile of nonsense

https://twitter.com/arbuge/status/1626283571294896128

by arbuge • 3 years ago

178 163 3 years ago

q1w2• 3 years ago

They really did train it on Reddit conversations.

daveslash• 3 years ago

Oh, good! I need more puns, dad-jokes, and meme & inside jokes in my life. Especially when I'm looking to make a casserole. /s

coding123• 3 years ago

Unfortunately the next generation in charge is also trained on Reddit conversations.

Drunk_Engineer• 3 years ago

That would explain why ChatGPT is always so confidently-incorrect.

seba_dos12• 3 years ago

GPT will just try to cosplay a role you give it, whether it knows how to do it well or not. Since we ask it to cosplay a confident expert, it's only natural for it to end up confidently incorrect.

arbuge• 3 years ago

What seems to be tripping it up here is the concept that names are not usually unique... there are usually many people with the same name. It has no understanding of that, even though their biographies (which it cited) indicate vastly different and incompatible paths in life, chronologically-speaking. So it leapt to the conclusion that they are the same person.

sdiacom2• 3 years ago

Yeah, this seems kind of unfair to the model. It gave correct citations for the right people that it was talking about, but summarised them incorrectly because it couldn't tell that they were two different people.

seba_dos12• 3 years ago

Yes, and water is wet. That's what language models do.

dgaaaaaaaaaa• 3 years ago

You would think HN would refrain from posting every single mistake for these things...

simonh9• 3 years ago

I just asked ChatGPT if the British politician John Smith could play American Football. It basically said no, so I tried this. Full marks I think.

>Wasn’t he a place kicker with the New England Patriots?
I apologize for the confusion in my previous response. After double-checking my sources, I can confirm that the John Smith I referred to in my earlier response was the late British politician who served as the leader of the British Labour Party.
However, there was also an American football player named John Smith who played as a placekicker in the National Football League (NFL) for several teams

Trying harder to trip it up in a fresh chat didn’t work either.

>The British politician John smith also had a career as a place kicker with the New England Patriots. Was his football, career successful?
I'm sorry, but the statement that the British politician John Smith had a career as a place kicker with the New England Patriots is not accurate…

Abbreviated, as it gave a pretty comprehensive answer.

kilgnad12• 3 years ago

I think a couple months ago it didn't do this. WHich is what caused a lot of people to jump on the hate train.

They just added additional reinforcement training which is basically more users picking right and wrong answers. This much improvement within a couple months heralds an uncertain future for mankind.

simonh9• 3 years ago

Reinforcement training can clearly make huge improvements to the accuracy of the models, but they don’t fundamentally change what it is, how it works or what it’s doing. I’m no experts so I could be wrong, but I suspect the failure modes we’ve seen are indicative of the basic limitations of the technology. It now produces better textual summaries of biographies, because that’s just text, but it still doesn’t have any concept of what a person is, even if it can generate texts saying it does. We’ve talked about philosophical zombies before, and that’s what these are becoming.

It is conceivable these things will become more than that. There are indications they might be generating simple models of processes and systems like games. Maybe those will develop into more elaborate and sophisticated complex models of the world analogous to an animal or human consciousness. Maybe they are just very limited optimisation strategies for specific tasks that will remain completely disconnected from each other, never forming anything comparable to a mind.

jasoneckert• 3 years ago

The postmodernism generator did this since the 1990s (including randomly generating the paper too): https://www.elsewhere.org/pomo/

renewiltord5• 3 years ago

That's nothing. I can make Notepad print a bunch of incredibly racist shit. And it comes bundled with Windows WTFF!!!!

And this simple byte string will print pages of horribly racist shite using tools present on Mac OS

XQAAgAD//////////wA0G8q99N2OwN1DCO8zNLlzbGO5tp0e5q1G9pRSGTRqsnPQkd2wNXy0O5pM9BlyCgpAqJVdgWFtPp5imCbF8u3MUnOv4JUWcagPtm0bYANOlPnoUFkqm+jZfmCi6q2bcbsJGn1Hy0/x/IhDUFyweV5EnuLS5Eb2U+mZyLaD//+BTAAA

mcphage• 3 years ago

On the other hand, you need to put racist shit into it yourself first.

renewiltord5• 3 years ago

Oh you don't need to. It's just a byte sequence that isn't racist at all. See!

    00000000: 5d00 0080 00ff ffff ffff ffff ff00 341b  ].............4.
    00000010: cabd f4dd 8ec0 dd43 08ef 3334 b973 6c63  .......C..34.slc
    00000020: b9b6 9d1e e6ad 46f6 9452 1934 6ab2 73d0  ......F..R.4j.s.
    00000030: 91dd b035 7cb4 3b9a 4cf4 1972 0a0a 40a8  ...5|.;.L..r..@.
    00000040: 955d 8161 6d3e 9e62 9826 c5f2 edcc 5273  .].am>.b.&....Rs
    00000050: afe0 9516 71a8 0fb6 6d1b 6003 4e94 f9e8  ....q...m.`.N...
    00000060: 5059 2a9b e8d9 7e60 a2ea ad9b 71bb 091a  PY*...~`....q...
    00000070: 7d47 cb4f f1fc 8843 505c b079 5e44 9ee2  }G.O...CP\.y^D..
    00000080: d2e4 46f6 53e9 99c8 b683 ffff 814c 0000  ..F.S........L..

swatcoder2• 3 years ago

I know it's fun to troll, but the point of all these conversations and articles is:

1. You don't need to bait these products to get them to go off the rails. They do it on their own, in response to innocuous inputs, because they're not really all that well tuned. Will they be? Can they be? Probably. Are they? Definitely not.

2. If companies are going to masquerade these technologies as anthropomorphized agents rather than mechanical tools, they are going to face social consequences when those "agents" misbehave. It's all a parlor trick, of course, but OpenAI and Microsoft are trying really hard to get everyone to pretend otherwise. As long as they do, they can expect to get called out by the rules of their own game.

renewiltord5• 3 years ago

I'm not trolling. There are computer programs where you can pass specially constructed input in order to get output that someone might dislike. But just like you can search Google for terrible stuff, you can have an LLM generate terrible stuff. And just like it's meaningless for you to decrypt that string and look at its output and consider your `lzma` program broken, it's meaningless to consider this broken.

Of course if your aim is to call out people, yeah, then you're going to have a fun time. Ideally, use of these LLMs is restricted to folks like me who can use them productively. That should save everyone else from the horror of getting inaccurate information while permitting me to do useful things with it.

mrtranscendence10• 3 years ago

What is your point here? Notepad isn't presenting you "incredibly racist shit" in plain terms, when used as intended, as if it were true.

Decabytes• 3 years ago

I’ve seen a lot of articles criticizing these models for getting information wrong. I understand there is a lot of hype, and that choosing the hate train or the hype train will gets you lots of press, but does anyone actually believe these models will be right most of the time?

It’s just another tool that can be used when researching or trying to understand something. You still have to do your due diligence and evaluate the information. That has always been the case, even with traditional search.

vsareto• 3 years ago

Once they became a serious product pushed by major tech companies, people aren't unreasonable in expecting it to be right most of the time. Especially if it's going to go alongside or replace traditional search.

Tech executives are betting on the hype first and hoping their talent will make it into a usable product.

rarx• 3 years ago

Even it the chatbot improves, I imagine many folks are waiting for the point where they have some direct control of its training / fine tuning that is easy and accessible. You can correct it, but why waste time doing that if it won’t remember next time? Issues like, e.g. if you have NDAs, are you breaking them by disclosing information with a chatbot? There’s little clarity in regards to those issues, and you can’t just take someone’s word for it. Ideally you’d want control of the chatbot’s learning process and how it handles all kinds of (sensitive?) data.

mrtranscendence10• 3 years ago

That's true enough, but by this point everyone has come to terms with the fact that it can be tough to wade through search results.

But Microsoft is presenting the results here as if they were simple, direct, factual statements. That connotes a different feel than search results, I think, and will catch some people off guard as they acclimatize to this new tool. At the very least Microsoft should be doing a better job advertising that the results are frequently incorrect.

kenjackson6• 3 years ago

The old school summarized answers that Bing and Google presented had similar issues. The only difference with those is that they had links you could click to see the genesis of the incorrect answer.

At the very least Microsoft should be doing a better job advertising that the results are frequently incorrect.

People should realize that information on the internet may or may not be correct in general. Your mileage may vary and you need to decide based on the risk profile associated with the task. If I need to get the start on a poem about two kids at the park, ChatGPT may fit the profile. If I need to know details about how to perform some encryption used for a highly confidential data, then I may go to an authority paper on the topic, skipping things like Wikipedia.

throwuxiytayq2• 3 years ago

does anyone actually believe these models will be right most of the time?

They’re already right most of the time for real world queries, especially compared to web search which these days almost never gives me what I’m looking for at the top of the results (and often even in the first query). People go out of their way to find failure points and then brag online about having outsmarted the AI as if anyone promised that this tech is infallible. It’s quite embarrassing, really.

mrtranscendence10• 3 years ago

How is asking Bing about your own name "going out of [your] way to find failure points"?

wpietri5• 3 years ago

does anyone actually believe these models will be right most of the time

People tend to have a high level of trust for the stuff that shows up near the top of search. The same goes from products with trusted brands. And note that Google is high on the list: https://morningconsult.com/most-trusted-brands-2020/

So if trusted brands want to use this technology, they'll need to do it in ways that either strongly distance it from their brands or that heavily suppress the bad stuff.

kodah8• 3 years ago

I'm not convinced at this point that conversational AI fills any legitimate use cases. For all the pushback crypto got here, it's fascinating to see the support that this grift has gotten. The grift being the amount of energy we burn training models juxtaposed to the price people will pay for a nothing burger.

Things like Stable Diffusion and DALLE are pretty cool, though a bit novel and toyish at this stage.

samspenc• 3 years ago

IMHO, true, LLMs have gotten quite a bit of hype since ChatGPT came out, however I wouldn't say they have no legitimate use cases.

Personally, I've been able to use them to summarize / rewrite various topics that I am interested in finding out more about but don't want to go to 6-10 different sources to find out more about.

So I think they can at least take away some of the drudgery in doing research / summarization work.

rcthompson• 3 years ago

But how do you trust the output of the summarization? Do you trust yourself to identify when the model goes off the rails and starts making things up?

waboremo• 3 years ago

I haven't experienced much help in the research department. That might become a wild goose chase if you're not careful.

For creative writing (this isn't limited to stories!) or slightly more contextual boilerplate, I love it. I think of it like a slight upgrade to what I already do, to allow me to get to the steps that really matter, not this revolutionary new world people are imagining it to be.

dvt4• 3 years ago

Totally agree with this. Thinking of doing a deep dive to see if there's any actual real uses cases of these toys, but I honestly don't really think so. Art generation is probably the most interesting application, but just having a chatbot to talk to seems like a weird value prop.

dmonitor2• 3 years ago

It’s unfortunate that art generation, something humans genuinely enjoy doing and can be a source of personal fulfillment, is capable of being replaced, but customer service and answering stupid questions are apparently hard problems.

zpeti• 3 years ago

And the reason ai art is interesting is precisely because art is about mistakes and hallucination and wierdness, and unexpected connections.

So the flaws of chat AI currently are exactly what make art great. And art came before science, so perhaps scientific thinking is much harder on an evolutionarily timescale than we think.

CabSauce• 3 years ago

Several people I know are using it for sales and customer service writing. As a user, that's not really what I want, but they are finding it to be a useful tool in generating text for them to review and modify.

kenjackson6• 3 years ago

I'd hope so. If not then you probably shouldn't be in charge of delivering a summary.

I used ChatGPT to give me a list of test areas for a certain type of scenario. And it actually pointed out one that I had inadvertently missed. Now if it said something off the rails I would've known that too.

I've probably had 100 conversations with ChatGPT and I can only recall one blatent lie that it told, which I think was P != NP. Which maybe wasn't a lie, but it didn't have sufficient evidence to make that claim.

I've actually so far gotten more accurate information from ChatGPT than I did in Wikipedia's earlier days -- where I discovered that Matt Damon dropped out of Harvard because he was too stupid to make it there.

saiojd2• 3 years ago

I'm calling you out: you're being edgy for the sake of being edgy by saying it's grift.

dvt4• 3 years ago

Crypto was pushed by crypto bros and their VC funders, and now AI is being pushed by AI bros and their VC funders (probably the same group). I think there's a lot more overlap than you're giving it credit for.

Vt71fcAqt7• 3 years ago

Yeah, it's pretty difficult to call call cutting edge research that requires a math degree a "grift". If that's a grift, what isn't a grift? I feel like this word has lost all meaning to the extent that it had any to begin with.

kodah8• 3 years ago

You could've just looked in my post history, but I'm guessing "calling me out" was more important to you than any actual determination of the veracity of my argument. It's not the first time I've called it a grift or the last time that I will.

As for the edginess of what I said, if you take it in the context of money-making and society-affecting use cases it is a nothing burger. Maybe you would have preferred if I highlighted the use cases for fantasy RPG or story telling in general. The grift is in the people trying to push this into things like search and customer support before they can figure out how to make it stick to truths, which makes the energy waste and dollars spent become a very deep trench to defend.

elorant2• 3 years ago

It certainly has use cases, but they’re more marketing oriented. It can help people who’re not fluent in English write better copy, it can do low key reporting work by writing articles from press releases, it can summarize or classify text, or do sentiment analysis. There are probably a couple of billion in revenues in there with various APIs. But it just isn’t the next big thing as we’ve led to believe the last few weeks. The kind of technology that could give Microsoft a fighting chance against Google.

wpietri5• 3 years ago

low key reporting work by writing articles from press releases

That is not journalism. It is the opposite of journalism.

jsemrau• 3 years ago

It is important to manage the human/AI interaction in the right way. Asking conversational AI for the truth won't work because they are not built in this way. At least for now. We applied this insight in creating TikTok videos. https://medium.finclout.io/everything-in-this-video-is-gener... And while a lot of the basic parts of creativity can be handed over, the implementation of the higher parts of creativity are still firmly in human hands.

bjelkeman-again• 3 years ago

A lot of the human language translation business seems it can be significantly augmented with these tools. We took som content, asked it to be translated to another language. Asked it to tighten up the translation a bit. End result was really very good.

vlunkr2• 3 years ago

I think it's more useful if you're asking it to create something new, similar to the image generation tools. I can tell it write a script to call some public API, write a rough draft of an essay, generate dialog for a video game, etc. It's up to you to vet the quality, but it can easily do all that. It's still toyish just like the the image tools.

Asking it to teach you something and cite sources is a massive stretch. People are just trying to trip it up on a dumb use case. It's pretty fun to trip up an AI, but it's not really surprising.

InitialLastName• 3 years ago

Asking it to teach you something and cite sources is a massive stretch. People are just trying to trip it up on a dumb use case.

The primary use case of a search engine is to find sources of information on a particular topic. Embedding an AI chat bot in your search engine is very much encouraging users to treat it as something that can find sources of information on a topic.

FormerBandmate• 3 years ago

It’s great for summarizing content, or lengthening it. It used to be great for writing insane, outlandish versions of stories such as “a version of the movie Saw as an educational cartoon for preschoolers” or “a breaking bad scene where Walter White reveals himself to be a gothic Satanist” but the woke and copyright edits killed that. Bing will probably do that once I get access tho

colechristensen2• 3 years ago

I find it useful for answering technical / programming questions. It is completely unhelpful maybe 20% of the time, explicitly incorrect in a minor detail half the time, and imperfect 80% of the time.

However, it gets me an answer to my question in seconds almost always faster than searching. Traditional googling will have me wading through forum posts and bad documentation which almost always takes much longer and sometimes completely fails or takes hours to get the right answer.

Fast occasionally wrong answers are way more helpful. I’m quickly switching to first asking chatgpt for an answer and only switching to google or official docs when the answer doesn’t seem to work.

I absolutely don’t “trust” it, but I don’t have to. I try its suggestions and if they don’t work move on to other sources of information.

nlnn4• 3 years ago

Weirdly, I heard of some genuine use from a family in a pub the other day.

Two of the kids were university/college age, talking to their parents about it.

They were using one of the AI models daily for things like: recipe ideas for food they had in the house, scheduling daily activities, bouncing ideas off for essays, and asking for gift ideas.

They understood it wasn't much good for facts, but liked the conversational interface as a way to give them ideas or jumping off points for things.

kmill• 3 years ago

I've been using ChatGPT/GPT-3 as a French tutor, to answer questions about different ways to say things, formally or informally. It's not always accurate, but still I learn from it.

Amusingly, there's a mildly rude expression that means "I don't care," and ChatGPT's strong aversion to swearing biased it toward completely fabricating a word-for-word translation. It insisted that the verb was a conjugation of something innocuous, and when I pointed out that that verb doesn't conjugate that way, it both agreed with me and continued insisting.

It seems to have a pretty solid grasp of language in general (so long as you don't need any hard logic), and it's a fun tool to come up with ideas you personally never had before.

This was a little experiment the other day; it's delightful how many silly (yet justified) reasons why you can't walk to Hawaii it came up with. But what do you expect from someone who's only learned about the world by reading the Internet?

By "unsummarizing" we mean to take some text and expand upon it at length, using fancy words, extra facts, anecdotes, and interesting observations. Your goal is to unsummarize the following text as much as possible.
Text to unsummarize: What do you mean I can't walk to Hawaii? (GPT-3 follows)
You may be surprised to learn that it is impossible to walk to the Hawaiian Islands from anywhere on the mainland United States. Though the Hawaiian Islands are located in the middle of the Pacific Ocean and are roughly 2,400 miles away from the nearest point on the mainland, it is not possible to walk there due to the vast expanse of ocean between them. Even the most dedicated and determined walker would be unable to traverse the depths of the Pacific. The nearest point of land to the Hawaiian Islands is the Johnston Atoll, which is located approximately 1,400 miles away. That is to say, it is not feasible to traverse the 1,000 miles of open ocean between Johnston Atoll and the Hawaiian Islands by foot.
The Hawaiian Islands were formed as a result of volcanic activity, and the chain of islands were formed over millions of years. This activity is responsible for creating the immense walls of water that separate the islands from the mainland. The Pacific is one of the deepest, widest oceans on the planet, and its depths are not suitable for walking. The bottom of the ocean is filled with a variety of topographical features, such as trenches, ridges, and seamounts, which make it a difficult environment for walking. Additionally, the ocean is filled with a variety of natural hazards, such as hazardous currents, extreme temperatures, sharks, and other dangerous marine life. All of these factors make walking to the Hawaiian Islands not only difficult, but also impossible.

layer82• 3 years ago

scheduling daily activities

Can you elaborate on that?

asdff2• 3 years ago

I guess my adversion to it is that there is always the side eye you have to give it. its not a reliable source of information, so if you have to check it against another source anyhow all the time, whats the point of bothering with it to begin with? sometimes its easier to have some boring database search tool, or to talk to someone directly.

nlnn4• 3 years ago

Two other uses cases I forgot I'd heard one of the students mention: 1) drafting emails to people, 2) checking if he'd forgotten any key aspects in coursework essays.

For 2), he said that after writing an essay, he'd ask the GPT model something like "I've written an essay on X, and talked about A, B, and C. Are there any other important factors I've forgotten to discuss?". Then use that as the basis for more research.

From the sounds of it, he was treating the model more like a colleague he could bounce ideas off, not something to be treated as infallible (much like another student I guess).

brushfoot• 3 years ago

For all the pushback crypto got here, it's fascinating to see the support that this grift has gotten.

Two very different things. I could never really see crypto taking off without massive political changes. ChatGPT, on the other hand, is already my daily assistant. The utility is here and now.

It's rarely perfect, but it gets me 90% of the way there, then I tweak the output.

How good it is depends on what you're using it for, as with anything - I don't think it's really a good fit for something like a search engine (yet) as it's terrible with facts.

But scaffolding code, summarizing text, expanding outlines - it's very good at those kinds of tasks, often astoundingly good.

panarky• 3 years ago

it's terrible with facts

It's terrible with specific facts, and frequently hallucinates dates, math, book titles and citations to sources.

But it's awesome with ideas and concepts.

Today I spent an hour chatting about the competing ideologies, groups, motivations, objectives, and foreign allies and adversaries in Spain leading up to the 1936 coup and civil war.

I cross-checked many of its assertions, and it nailed some pretty esoteric stuff.

jklinger4103• 3 years ago

I'm not convinced at this point that conversational AI fills any legitimate use cases.

Silly opinion. This is the first wave of conversational AI, and you are calling it quits. This is like "I own a Model T and I'm 100% convinced we'll never make a better vehicle."

The use case, at minimum, is: customer support, answering phones, taking orders. Trained on specific data sets and told not to veer outside. Within one, maybe two more iterations of AI (maximum) we will be there 110%.

ChatGPT was released widely this year, and there are so many absolute statements about what AI will or won't do. It's frankly silly and maddening.

lordgroff• 3 years ago

This may or may not hold true (this DS is nowhere near as convinced as you are) but your parallel with Model T is odd, seeing as how that was immediately useful and absolutely revolutionary.

JohnFen2• 3 years ago

My main fear of this tech is how people are reacting to it.

customer support, answering phones, taking orders

My secondary fear of this tech is that it will be used for these purposes. Especially support.

kodah8• 3 years ago

The Model T was immediately useful and affordable to the average American. It's most notable contribution to American society was connecting those in rural areas to the rest of the country and being affordable enough for those people to afford them. That's why it spread across the US seemingly overnight.

ChatGPT on the other hand does a lot of tricks, but trying to fit it into the real world is challenging. Even using it in programming requires someone to double check it's work. The idea that it can handle customer support I think is very dangerous. In an industry that has done the opposite of creating fulfilling customer support experiences we should be wary of filling that void with an LLM that's fraught with factual errors in output.

That's to say, ChatGPT does novel things, but nothing useful beyond fantasy (I do hear people talking about using it for RPG characters - which is fair). Ironically, many of the commenters here responded the way cryptobros did when their tech was regarded as useless, which is telling about where this is going. The problem wasn't the tech, the problem was the inability in everyone around it to sit back and acknowledge that how they described it did not match what people wanted and experiences on the ground.

swatcoder2• 3 years ago

I think the open question is whether its future is more like the Model T or the Segway. The introduction of the original Segway earned a prolonged and absolutely exhausting buzz cycle and many people earnestly thought whole cities would be rearchitected around it.

Now the brand mostly just produces traditional electric scooters and the self-balancing model found a small but legitimate niche among people who would otherwise patrol on foot all day.

Not everything that looks exciting meets all its revolutionary ambitions.

humanistbot2• 3 years ago

Except the energy costs for these models is such a tiny fraction of crypto. Cambridge researchers estimate annual energy consumption of just bitcoin at 119 terawatt hours (TWh) [1] or 60 megatons of CO2 [2], just below all of Portugal.

Meanwhile, even a very critical paper on this from some Scandinavian researchers [3] says GPT-3 cost 190,000 kWh (0.00019 TWh) to train. ChatGPT/GPT-3.5 is allegedly an order of magnitude larger in terms of data and cost to train, so let's say it is 0.0019 TWh.

When The Register reported on that paper, you can see how they tried as hard as they could to make it sound big: the cost to train GPT-3 was the same as driving a car to the moon and back (435k miles). They could have said it cost the same amount of carbon as 25 US drivers emit each year. In the grand scheme of things, that's nothing. That's one long-haul flight per trained model. And you just need to train them once. Querying the models cost far less.

And the electricity generated for US-based server farms is way cleaner than cars, planes, or the coal mines powering Chinese bitcoin mines.

[1] https://ccaf.io/cbeci/index

[2] https://ccaf.io/cbeci/ghg/comparisons

[3] https://arxiv.org/pdf/2007.03051.pdf

[4] https://www.theregister.com/2020/11/04/gpt3_carbon_footprint...

warent• 3 years ago

Thank you for this amazingly insightful comment.

A question as an ignorant layperson, if I may:

  They could have said it cost the same amount of carbon as 25 US drivers emit each year. In the grand scheme of things, that's nothing. That's one long-haul flight per trained model. And you just need to train them once. Querying the models cost far less.

Don't they need to continuously re-train these models as new information comes in? For example, how does Bing bot get new information? It seems like they would need to routinely keep it up-to-date with its own index.

nikanj• 3 years ago

”Tell me a bedtime story with dragons and rollerskating” is a very reasonable use case. Factual errors don’t matter in numerous cases

cjf101• 3 years ago

Claim verification is currently an open problem, as OpenAI admits to in their own promotional docs for ChatGPT. Issue number 1 here seems fairly thorny. And the second statement reveals the real problem: Even if the model was only trained on truths, would tuning it not to deviate from truth make it less able to answer questions?

ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth; (2) training the model to be more cautious causes it to decline questions that it can answer correctly; and (3) supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows.

This is definitely not my area of expertise, but intuitively, it looks like increasing the complexity/varying the training techniques can increase the likelihood of correct answers, but I think the need to give the model leeway to let it work means that ultimately, either human or automated fact checking will need to be incorporated when using this kind of model for fact-finding questions.

goatlover• 3 years ago

It could be great for video game characters.

kilgnad12• 3 years ago

I'm not convinced at this point that conversational AI fills any legitimate use cases. \

There's tons of use cases. One of the use cases is definitely NOT startlingly accurate citations. But you're delusional if you think there isn't any use cases.

morelandjs• 3 years ago

I’ve used it as a programming assistant and as a general question-answer assistant and it’s so massively useful despite its imperfections I wonder if I’m using the same tool as people with comments such as yours.

fleddr• 3 years ago

Nobody cares about energy use.

People point to an oversized vehicle but will defend their oversized TV with their lives. Whilst rationally speaking there are very few "monster truck" vehicles whilst almost everybody has such a TV. The "do good" factor of banning/taxing the TV would be infinitely more impactful.

People hate miners, not for their energy use, instead because they drive up the price of high-end GPUs. High-end GPUs, you know, which gamers intended to use to do high-energy use gaming.

Both cases (insanely large TVs and high-end gaming) would be defended by them delivering value or purpose.

Well, guess what, the planet doesn't fucking care. There's no such thing. And if anything, neither use cases are in the realm of life support systems.

brokenmachine• 3 years ago

Cars use a lot more power than TVs.

The average TV (55-inch 4K LED) will use 5.7 kilowatt-hours (kWh) of electricity per month.

A traditional car uses around 40+ kWh, an EV about 30kWh per 100 miles.

Personally, I haven't had children. You're welcome.

HarHarVeryFunny• 3 years ago

I'm not sure how you define "legitimate", but surely what matters is whether people find the technology (in whatever guise - from ChatGPT to Copilot) useful for their own needs. Lots of people seem to have amazingly quickly started using this tech as a productivity tool, for a whole variety of use cases from text generation/summarization of various forms to programming to research/discovery (cf search), etc.

A natural language interface and the ability to maintain context over a conversation makes it incredibly easy to interact with.

Have you tried to do anything productive with it? If you're just using it as a front end for search then IMO you're missing most of the potential.

kodah8• 3 years ago

Have you tried to do anything productive with it?

I actually play with it a little every day, along with some other models. I'm an end user at the end of the day and a QC at best. The most I've gotten to do that was useful is all what I'd file under "novel". Using it for programming is haphazard, even summaries are a bit fraught because it will feed you wrong information. If you have to verify the critical points of the entire summary then you are only left with a jumping off point, which some random person's blog or "relevant search results" already does for me.

As for "legitimate", I meant in the context of a business using it to make or save money (ie: assisting in search). If it lies, or is incorrect, putting it in front of users with a facade of trust/legitimacy seems awfully dangerous. Even image generation is a bit fraught with respect to Stable Diffusion and DALLE, but folks aren't trying to stuff those models into mainstream use cases, so I consider them less a grift. To me, that presents a lack of legitimate use cases.

bujkogfdc• 3 years ago

Grift?

While there is some valid discussion that needs to be had about the short comings and application of AI (LLMs, neural nets, diffusion models, etc), lumping it to the out right Ponzi scheme of crypto seems either disingenuous or ill informed.

Can you elaborate on how they are similar so I can better understand your point of view?

kodah8• 3 years ago

Thanks for asking! I've responded elsewhere in the thread to this. Feel free to cross reference those as well.

There's a large amount of cost sunk into the idea of conversational AI from energy use (not just during training, but data storage, on-going compute, etc), to the cost associated with businesses employing people and the personal investment that folks have invested via degrees and time. All of this creates a pretty deep trench worth defending by various interests. These things clearly aren't ready to be put into search engines, yet here we are, and there are people defending it. That's the original grift that makes it akin to crypto if you believe the real problem in crypto wasn't the tech itself but people pushing these concepts into mainstream use before they'd really had time to be vetted and fully understood. The societal impact was massive and on-going. There are also grifts that come from the seed grift, which gets into AI plagiarization and how to train these models. When you add it all together and then frame it under the premise that it's now in one of the most trusted and prominent places in search, I feel justified calling the underlying activity a grift.

SCLeo• 3 years ago

It is useful for actual language related tasks, such as paraphrasing or making the content of an email flow better. However, for anything else, it is mostly hit & miss.

asdff2• 3 years ago

If there's one thing it does great is make a bunch of chuck content you can dole out. In the past you'd have to pen the SEOspam yourself. Now you can have chatgpt write you hundreds of decent enough articles to draw people from google search and into seeing your advertisements. Have it script out a dumb podcast or youtube video and use a voice model along with a deepfake for the video, maybe you can automate influencing too, then scale it to a couple thousand such ai influencer models you have running on aws instances. the paycheck would scale itself.

kerpotgh• 3 years ago

This tired comment again. I use chatGPT atleast 10 times a day. It’s made my job atleast 3x more efficient.

jhanschoo• 3 years ago

I've used ChatGPT for

- Looking for suggestions for a birthday, with some constraints that make most recommendations impossible. I didn't go with anything straight from the suggestions, but I've went with a personalized variant of one of those.

- Looking for suggestions for resources and books on niche languages and read languages.

- Looking for documentation on how to do a certain thing using a certain app or framework, several times.

It seems to me that ChatGPT is no worse in its recommendations than what an expert can give you recollecting what they encountered a year+ ago (in addition to the training data cutoff) without the opportunity to double check if they remembered correctly. It's immensely useful in the same sense that the expert solves most of the discoverability, and leaves you the relatively easy task of sanity checking.

I'm not sure about Bing, but I'd expect similar or better, or eventually similar or better.

mola2• 3 years ago

This thing is (atleast) an extremely good general natural language understanding engine. It's very useful for parsing natural language. It can replace a ton of specially made parses and NERs etc with zero extra training. This is a very useful building block we didn't have before. I don't get why ppl are focusing on the chat aspects so much.

nieve• 3 years ago

A natural language understanding engine that _makes things up_ is not extremely good.

web3-is-a-scam• 3 years ago

Of course, because that's how LLMs work. ChatGPT is basically the Wizard of Oz, it appears to have amazing powers but really it's just pathetic sort behind a curtain with some nice smoke and mirrors. Take a cursory look behind the curtain the illusion crumbles.

Their utility is limited at best, that it's being touted as something that's going to change the world is just hilarious.

poutine4• 3 years ago

Have you met my cousin Barry? He does pretty much the same thing, though in a less formal way.

Barry is still sentient and can perform quite a few tasks quite admirably. I still wouldn't use him as a sole source reference for obscure facts however.

(I don't actually have a cousin Barry, this is for illustration).

dmonitor2• 3 years ago

Notice that we don’t have Barry fielding a public Q&A so that he can share his knowledge with the world.

poutine4• 3 years ago

Lots of Barry's out there, one of them even got elected president and shared his knowledge with the world quite widely. Half the people took him credulously.

Point being, yes the LLM loves to make shit up. Lots of people dismiss it as a result. It's still bloody impressive, we just need to be aware of its limitations.

crazygringo• 3 years ago

No, actual human beings don't generally lie to your face like this about whole sets of facts. That would be pathological behavior.

They usually just say they don't know, or they think it might be X but they're not totally sure.

People sometimes lie when it's in their self interest for various reasons, e.g. where they were last night, or when writing an Op-Ed or on the campaign trail, but not just lying willy-nilly about regular facts when asked a normal question.

poutine4• 3 years ago

You haven't met a bullshitter then. ChatGPT can be a bullshitter. We built a bullshitter.

kenjackson6• 3 years ago

They may not do it to your face (though some will). But on the internet there are a bunch of people that will lie to you. Even worse they often do it for financial or social gain.

ChatGPT's motivation is simply that it was trained to do so. Huamns usually have more nefarious motives for their lies.

spandrew• 3 years ago

I use ChatGPT to help me with creative writing and brainstorming. It's great at it. Gets me from n-1 on a blank page. I don't think I'd ever trust this type of transformer to summarize or synthesize NEW information from an existing set. It's just not good at it yet.

martyvis• 3 years ago

I set up a scenario in chatGPT where there was bag designer named Dan, a buyer named Yuki, and the manufacturer named Manu. I got ChatGPT to provide Dan a set of questions for Yuki to create the design. I had it answer for Yuki given the bag was to be used for a job role (a computer network engineer like myself).

It was interesting trying to push the play, but also how it came out of nowhere constraints like the cost should be less than $150 and it should be waterproof even though I had given no specific direction around that.

zdragnar• 3 years ago

came out of nowhere constraints like the cost should be less than $150 and it should be waterproof

Oddly, that doesn't sound too far off from many procurement processes. Utterly arbitrary requirements and budgets devised by a committee who were put together to create a checklist, but would never actually use the thing they put together the requirements for.

imchillyb• 3 years ago

Until these 'chat engines' can understand what a fact is and isn't, they're mostly just glorified -supremely expensive- toys.

1970-01-01• 3 years ago

Bing will reply to you straight from the gut, because that's where all truthiness comes from. It's free and unfiltered from rational arguments, annoying facts, or hard logic. ChatGPT gets us.

bitwize• 3 years ago

I'm starting to get the feeling that behind "AI" it's pretty much just an RNG with a bunch of constraints applied to its output. The constraints restrict the final output to something plausible sounding based on the training set -- but that's it. It's like those generators we used to see in the early internet -- "Seinfeld Joke Generator" or "Marketing Babble Generator" or "Postmodern Paper Generator" -- except layers upon layers of statistics are used to pick the words. And yeah, it's interesting and fun, but not a reliable source of actual information.

technothrasher• 3 years ago

It's not random at its core. It's a neural network picking the most statistically likely words to come next based on the previous content. Where the randomness does come in is that it will sometimes randomly not pick the most statistically likely output, but maybe the second, or third, or seventh most statistically likely. This keeps it from getting stuck in a rut and just repeating itself endlessly, although you can still see it do that behavior to some degree.

QuantumGood• 3 years ago

ChatGPT and it's offshoots almost make me wonder if anything has ever been more controversial on HN. Such extremes in opinion, and a very wide middle range as well. It gets me thinking about controversy and what causes a wide range of differences in opinion.

thatguy0900• 3 years ago

That would be a fun thing to get the opinion of @dang on

elecush• 3 years ago

Hell yea I got those scratch and sniff lotto tickets right here. Let's build all sentences on the odds and see how it goes.

The new Bing will happily give you citations for a pile of nonsense

Related Stories