Jump in the discussion.

No email address required.

>I kind of am skeptical that GPT-4 is just scaling up. GPT-3 was 175B params, which is already absurd, and that recent FAIR paper showed that after a certain point better data is better than more params.

>Also considering they didn't release the parameter count, it might be to mislead competitors and have them try many absurd parameter counts, while they figured out some other better tricks. They did mention that it was the most stable train they had out of all the GPT models, which I feel like means it's more than just scaling up of the model.

>But it's just a hypothesis.

Holy fricking shit, it's been THREE years and these r-slurs still haven't realised the bitter lesson. Thank god I left that shithole sub.

Jump in the discussion.

No email address required.

Link copied to clipboard
Action successful!
Error, please refresh the page and try again.