https://news.ycombinator.com/item?id=41058194
What this implies is future models will be even better at sounding smart but even more likely to hallucinate and give you wrong answers.
The future is r-slurred. 
Training AI on AI generated output leads to model collapse
- 62
- 62
Top Poster of the Day:
J
Current Registered Users: 26,837
BROWSE EFFORTPOSTS
SITE GUIDE
HOLES
PING GROUPS
Live commit: 441c078
Jump in the discussion.
No email address required.
it's amazing how people know 1 thing about LLMs, that they hallucinate, and go around dropping that fact to sound incredibly smart. I'm going to offer a quick heuristic, any time you see someone mention that LLMs hallucinate, in a sort of "I'm here to educate you rubes" way, you can assume that person is an r-slur.
And that's also the only thing people have to say about LLMs, hallucinate, hallucinate, did you know they hallcuinate? here's a new thing hope they fix the hallucinate issue. Lol. There's no other conversation to be fricking had. LLMS HALLUCINATE FOLKS IT'S SERIOUS OUT HERE.
Jump in the discussion.
No email address required.
why do they hallucinate and what does it mean for ai to hallucinate?
Jump in the discussion.
No email address required.
It just means they make stuff up. Llms basically just produce the next word that is statistically likely given all the previous words, starting with the prompt. They don't actually "know" anything
Jump in the discussion.
No email address required.
This sounds r-slurred fake and straight even to my non expert ears.
Jump in the discussion.
No email address required.
Sometimes the model falls apart and you can see behind the curtain. Case in point, "SolidGoldMagikarp".
This was a username on a counting forum, all they did was count numbers. They posted enough that "SolidGoldMagikarp" made it into the training data as a distinct word. At some point the devs realised this was useless data and removed it, but they didn't update the tokens, so you get nonsense like this:
Jump in the discussion.
No email address required.
Do you have a source for the counting forum thing you mentioned? Interested in seeing the
on display
Jump in the discussion.
No email address required.
Oh, is it this shit?
https://old.reddit.com/r/counting/comments/cum60c/2845k_counting_thread/
https://old.reddit.com/r/counting/comments/55ixip/1394k_counting_thread/
There's a person named SolidGoldMagikarp mentioned in these posts, but their account is deleted
I assumed you meant an independent forum. Not surprised it's a subreddit.
Jump in the discussion.
No email address required.
More options
Context
More options
Context
based.
Jump in the discussion.
No email address required.
More options
Context
More options
Context
Lol yeah, it really is a miracle that they work as well as they do. Here's an interesting layman's article on how it all works:
https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/
Jump in the discussion.
No email address required.
No i mean like, it gives the probability of each word in the context of the question asked right?
Jump in the discussion.
No email address required.
The question asked is a series of words that statistically come before the first word in the response
Jump in the discussion.
No email address required.
yeah then it makes more sense. Not all the sense, but definitely more sense. So AI solutions is just the probability field being off by that 1 in a million correct word, which is why more data is giving more accurate and fewer hallucination answers?
Jump in the discussion.
No email address required.
Linguochads who can phrase true facts 1000 ways are all that stand between us and oblivion
Jump in the discussion.
No email address required.
More options
Context
More options
Context
More options
Context
Think of it like auto complete. The model is called iteratively to get the next word starting with the question asked. Then it keeps doing that until the model determines the most likely "next word" is a stop token and the response is done. You can even see this in the chatgpt interface words pop up sequentially in the UI as the model runs
Jump in the discussion.
No email address required.
Still crazy but more believable now.
Jump in the discussion.
No email address required.
More options
Context
More options
Context
More options
Context
When GPT3 first came out I had a mini existential crisis based on the fact it worked well enough to "converse" with. Made me question how much of my own thought was really just iterating on the most likely words to spit out.
Jump in the discussion.
No email address required.
BIPOC
Jump in the discussion.
No email address required.
More options
Context
More options
Context
More options
Context
More options
Context
More options
Context
More options
Context
More options
Context