Unable to load image

Insane tech demo of high speed LLMs :marseysweating:

https://groq.com/

Apparently it's a type of new chip. I like specialized hardware :marseynerd:

44
Jump in the discussion.

No email address required.

Always funny to see all these r-slurred optimizations (which are basically shit approximations)

https://i.rdrama.net/images/17084588499320116.webp

Jump in the discussion.

No email address required.

this is the fastest I have ever seen any LLM work lol. its kind of mindblowing that it's possible

Jump in the discussion.

No email address required.

Jump in the discussion.

No email address required.

fp16 usually isn't a problem for inference, it's a very good compromise for most ML shit

Jump in the discussion.

No email address required.

:marseynotes:

I only know ML in the context of multiple regressions, so if you run thousands of these, and fp16 is less "accurate," then how do you know which regressions were done correctly?

Jump in the discussion.

No email address required.

Link copied to clipboard
Action successful!
Error, please refresh the page and try again.