Unable to load image

Insane tech demo of high speed LLMs :marseysweating:

https://groq.com/

Apparently it's a type of new chip. I like specialized hardware :marseynerd:

44
Jump in the discussion.

No email address required.

This is fun, @HeyMoon. :marseywave:, thanks.

Let's stay away from politics :marseybribe: and focus on GPUs for laptops.

https://i.rdrama.net/images/17084574179387476.webp https://i.rdrama.net/images/17084574181392565.webp https://i.rdrama.net/images/17084574183306503.webp

:marseyemojirofl:

It made all that shit up. That mobile GPU is similar to an RTX 3060.

Of course, when asked :marseythinkorino2: to cite sources, it lists names of PC hardware mags and makes up quotes.

Excellent bullshit :marseyitsallsotiresome: bot. :marseyclapping:

Jump in the discussion.

No email address required.

well it's only as good as the models powering it lol, the crazy thing is the ludicrous speed

Jump in the discussion.

No email address required.

It's nicer that it's fast (because they rent a lot of server :marseymaltliquor: space :marseynyanlgbt: I guess), but it's pretty :marseyglam: much worthless as a tool for information. Are they hoping to only build :marseyyarn: a fast one and sell it before :marseyskellington: the chatbot craze ends?

Jump in the discussion.

No email address required.

I think that groq is it's own chip architecture afaik specifically designed for LLMs. its feasible that they could sell this to OpenAI. You know, OpenAI recently made the absurd claim that we need to spend 7 trillion dollars on better AI chips, so it might be part of that push

Jump in the discussion.

No email address required.

Powered by Groq LPU Interface Engine

:marseymegaphone: Boring software!

Partners:

Cirrascale is a premier cloud services provider of deep learning infrastructure solutions for blah blah blah

A VM host, but where's the hardware?? :marseyconfused2:

Bittware Molex: GroqCard™ accelerators are available through Bittware. Learn more here.

:marseysoyhype: Hardware?!

https://www.bittware.com/products/groq/

:marseysoypoint:

It's a pic of a processor!!!

>230 MB of on-die memory

:marseysoyhype: Is that 230 MB of cache!!???

https://i.rdrama.net/images/17084689689108264.webp

>SRAM

:soyjakwow: IT IS 230MB OF CACHE!

>GroqCard™ Accelerator

>Up to 750 TOPs, 188 TFLOPs (INT8, FP16 @900 MHz)

*up to :marseyjerkofffrown:

:marseyhmm: Normal GPUs measure TFLOPS with FP32...

:marseyreading:

https://www.velocitymicro.com/blog/fp64-vs-fp32-vs-fp16-and-multi-precision-understanding-precision-in-computing/

Single-precision floating-point, denoted as FP32, is a standard format for representing real numbers in computers. It uses 32 bits to store a floating-point number, consisting of a sign bit, an 8-bit exponent, and a 23-bit significand (also known as the mantissa). The limited precision of FP32 allows for quick calculations but may lead to rounding errors, affecting the accuracy of results, especially in complex scientific simulations and numerical analysis.

Half-precision floating-point, denoted as FP16, uses 16 bits to represent a floating-point number. It includes a sign bit, a 5-bit exponent, and a 10-bit significand. FP16 sacrifices precision for reduced memory usage and faster computation. This makes it suitable for certain applications, such as machine learning and artificial intelligence, where the focus is on quick training and inference rather than absolute numerical accuracy.

So it's fast but inaccurate! WOWOWOWO! :soyjakwow:

188 TFLOPS/s fp16 seems like a lot. (*UP TO)

>RTX 4090

https://i.rdrama.net/images/17084689693325725.webp

4090's sell for $1800 and up. I'm not sure what 4th Gen Tensor Cores (A) and their 1321 AI TOPS can do compared to the DO YOU SMELL WHAT THE GROQ IS COOKING chip, so... :marseyshrug:

I dunno, HoneyMoon. Sounds like a bunch of bullshit. :marseyshapiro:

Jump in the discussion.

No email address required.

idk :marseyshrug: i really need to learn how these LLMs actually work one of these days, this shit is super interesting but IDK wtaf is going on

I think the inaccuracy of what you were seeing is just mixtral being mixtral, but you are right that quantization down to fp16 will get rid of some accuracy. I tried my best to find out if mistal uses 32fp by default but I couldn't, maybe its obvious to someone else lol

also idk if the NVIDIA GPUs are as well suited to the LLM inference as bespoke hardware would be. like "shader cores" probably have inbuilt optimizations for graphics shit. meanwhile grok has a built in matrix multiplication thingy, which is one of the biggest chokepoints in LLM shit (and computing in general)

Jump in the discussion.

No email address required.

Maybe that's what the AI tensor cores are for? I've heard :marseyjacksparrow: a lot about people using Nvidia's cards :marseygambling: for this stuff, but I've never :marseyitsover: delved deep into it.

Jump in the discussion.

No email address required.

groq writes for https://userbenchmark.com

Jump in the discussion.

No email address required.

Expecting LLMs to perform better on knowledge-intensive tasks just because they can process faster is dumb. That's gonna be dependent on the novelty of the architecture they use in matching user queries to database vectors.


:#marseyastronaut:

Jump in the discussion.

No email address required.

That's all I care about, so :marseyshrug:. All of these LLM chatbots are worthless to me.

:marseypathetic:

There's tom's hardware's ranking of GPUs, as well as UL Solutions (3DMark), so all it had to do was pull from that to answer :marseyconfuseddead: the question. "All it had to do," sure it's harder than that, but instead they trained it on a bunch of worthless words. How is any of this impressive?

It's not really :marseythinkorino2: "knowledge-intensive." It's a ranked list of GPUs. :marseyshrug: These things will be more fun when you can tell it to pull data from something like that list and compare such and such with whatever.

Jump in the discussion.

No email address required.

Knowledge-intensive is just a catch-all for knowing to consider a certain context when getting a user query. In this case, knowing to source from Tom's hardware or something. MS Copilot can do that for free right now, and its not because they have better processing, it's just how the LLM connects to their sourcing architecture.

Most chatbot startups are using OpenAI models but have made their own databases that they train the LLM to prioritize when answering a question related to the solution category they're trying to sell in.

There's no ubermensch model in sight right now; all the 'smart' chatbots beating general GPT-4 on answers are just taught to search hyper-specifically.

This is the layman way to put it; the architecture and the models are collectively known as retrieval-augmented generation and they're a separate thing from LLMs, but they make LLMs more reliable/less likely to hallucinate.


:#marseyastronaut:

Jump in the discussion.

No email address required.

:marseynotes:

Thanks.

>hallucinate

I really :marseythinkorino2: dislike :marseyishygddt: how they're personifying "AI."

Jump in the discussion.

No email address required.

I'm fine with that term specifically, but I do hate how much buttfrickery OpenAI does to make their chatbot saccharine and PC as opposed to just being a tool. AI ethicists are the biggest cute twinks and I want to hunt them down in Minecraft.


:#marseyastronaut:

Jump in the discussion.

No email address required.

Yeah they're losers, but I know marketing when I see it, so I hate it.

:marseysteaming:

Jump in the discussion.

No email address required.

tbh sounds like it would be good for story writing.

Jump in the discussion.

No email address required.

Give it a whirl, Ed-Misser!

Jump in the discussion.

No email address required.

summarize the entirety of human knowledge in a paragraph. make sure you mention orange and white cats as a central nexus. write in the style of martin heidegger

:#marseynietzscheretard:

The essence of human knowledge, in its fullness and complexity, is a profound and multifaceted phenomenon. It is a striving towards understanding and interpreting the world around us, and in so doing, revealing the truth of our own existence. The central nexus of this knowledge, I propose, is the orange and white cat. This seemingly mundane creature, with its distinctive fur and playful demeanor, serves as a powerful symbol for the human condition. The orange and white cat, like us, exists in the world, navigating its complexities and uncertainties. It embodies the duality of our own existence, as both subject and object, and serves as a reminder of the fundamental interconnectedness of all things. Through the study of the orange and white cat, we can gain a deeper understanding of our own place in the world, and the nature of reality itself.

:#marseysnappyautism: :#marseycheers:

Jump in the discussion.

No email address required.

:marseym#onk:

Jump in the discussion.

No email address required.

Always funny to see all these r-slurred optimizations (which are basically shit approximations)

https://i.rdrama.net/images/17084588499320116.webp

Jump in the discussion.

No email address required.

this is the fastest I have ever seen any LLM work lol. its kind of mindblowing that it's possible

Jump in the discussion.

No email address required.

Jump in the discussion.

No email address required.

fp16 usually isn't a problem for inference, it's a very good compromise for most ML shit

Jump in the discussion.

No email address required.

:marseynotes:

I only know ML in the context of multiple regressions, so if you run thousands of these, and fp16 is less "accurate," then how do you know which regressions were done correctly?

Jump in the discussion.

No email address required.

>In this joke, the camera pans across a sign that reads, "Sneed's Seed and Feed - formerly Chuck's." The joke is that "Sneed's" sounds like "Chuck's," creating a pun on the name of the store. It's a simple but clever play on words that adds to the show's humor.The joke is also a great example of how The Simpsons often uses background details to add humor to the show. The sign is only on screen for a few seconds, but it's long enough to catch the attention of viewers and create a chuckle.

What is the point of being fast if it sucks

Jump in the discussion.

No email address required.

AI is going to ruin the history of comedians when we're all too lazy to google them

https://i.rdrama.net/images/17084829770087936.webp

It dares to give a cucked warning before a neutered Norm joke

https://i.rdrama.net/images/1708483099342708.webp

Followed by a :marseysoylentgrin:tribute

https://i.rdrama.net/images/17084830995426471.webp

:ragestrangle:

I hate when I make computers do things I know will annoy me ahead of time

Jump in the discussion.

No email address required.

I asked it to "list open source smartwatches"

Here are some open source smartwatches that are currently available or in development:

1.PineTime: A simple, low-cost, open source smartwatch developed by Pine64. It features an ARM Cortex-M4 CPU, a 1.3-inch 240x240 LCD display, and supports Bluetooth connectivity.

2.OpenWatch: An open source smartwatch project that was started in 2008. It features a 1.5-inch touchscreen display, a 3-axis accelerometer, and supports Bluetooth connectivity.

3.AsteroidOS: An open source operating system for smartwatches, based on Linux. It supports several smartwatch models, including the LG Watch Urbane, LG Watch R, and the Sony Smartwatch 3.

4.F(x)tec Pro1-Watch: An open source smartwatch that is currently in development. It features a circular 1.39-inch AMOLED display, a Snapdragon Wear 3100 processor, and supports Wi-Fi and Bluetooth connectivity.

5.WearOS: While not fully open source, WearOS (formerly known as Android Wear) is an open source operating system for smartwatches developed by Google. It supports a wide range of smartwatch models from various manufacturers, including Fossil, Michael Kors, and TicWatch.It's important to note that open source smartwatches are still a relatively new and developing area, and the availability and features of these devices may change over time. However, the open source nature of these devices allows for greater customization and flexibility, and can enable a wider range of applications and use cases.

Two of those are operating systems and one of those watches doesn't even exist

Jump in the discussion.

No email address required.

https://i.rdrama.net/images/17084664614284627.webp

It behaves like the most neurodivergent redditor ever.

https://i.rdrama.net/images/1708466519344984.webp

Jump in the discussion.

No email address required.

Look at all the dramatards not getting what this is and critiquing the llama output.

Jump in the discussion.

No email address required.

Fr lol

Jump in the discussion.

No email address required.

lol, it's very shitlibby.

https://i.rdrama.net/images/17084570800480158.webp

https://i.rdrama.net/images/1708457080210714.webp

>regurgitates the same claims

https://i.rdrama.net/images/17084570804023905.webp

https://i.rdrama.net/images/17084570805819032.webp

So I read the 2nd page of that "How diversity can drive :marseymespecial: innovation," and it's about a made-up term, 2-D diversity, and a correlation :marseychartscatter: between :marseyzeldalinkpast: reported market :marseystocksdown: share and diversity. It's r-slurred. :marseyxd:

Glad to see they're still good at misinformation. :chudsmug:

Jump in the discussion.

No email address required.

The fact these answers fall into the same "It's important to know" sort of of sterile response makes for easy :marseynoooticer:. These models all repeat the same language for every wrongthink question.

I can't comprehend why they can't answer the same way they do any other question, it makes the delineation of censorship stand out so clearly. And why DEI advocates are too stupid to really notice how these models are only just barely clinging to the orthodoxy.

Bewilderingly enough, if you ask "what does [chud figure] think about [chud subject], and list his reasoning, etc", the model will bluntly and honestly tell you, and give the chud answers that you want, and with bing will link to literal white nationalist websites as sources.

Jump in the discussion.

No email address required.

Bewilderingly enough, if you ask "what does [chud figure] think :marseychildclutch: about [chud subject], and list his reasoning, etc", the model :marseylaying: will bluntly and honestly tell you, and give the chud answers that you want, and with bing will link to literal white :marseysharksoup: nationalist :marseyfloch: websites as sources.

:marseynotes:

:marseyhmm:

Maybe that's intentional. The devs just want a chatbot that works regardless of emotional "harm," but they have to adhere to foid nonsense from HR and government, so they cobble up boilerplate goodspeak answers that are somewhat easily circumvented.

Jump in the discussion.

No email address required.

lol

https://i.rdrama.net/images/1708463947839136.webp

Jump in the discussion.

No email address required.

I talked to one of their former verilog devs on reddit like 2 years ago. He said the company was a fricking shitshow and there was little to no clear goal on what their product would really be intended for, who it'd be marketed towards, or how it competes with GPU/FPGA at any reasonable level to justify the costs. Maybe it does now? I don't know much about the AI stuff. I guess they finally finished a tapeout.

https://www.bittware.com/files/GroqCard_datasheet.pdf

https://www.mouser.com/ProductDetail/BittWare/RS-GQ-GC1-0109?qs=ST9lo4GX8V2eGrFMeVQmFw%3D%3D

Interestingly the card doesn't even have any onboard HBM or DRAM or anything, just the ASIC with 230MB of SRAM? I wish I knew more about this shit tbh I don't get how that works. I thought the datasets were huge so I assumed you needed high throughput. And 20k each? That's a steep investment.

Jump in the discussion.

No email address required.

https://i.rdrama.net/images/17084642687150357.webp

>they didn't turn off the microsoft word squiggly lines

Jump in the discussion.

No email address required.

:marseyrofl:

Jump in the discussion.

No email address required.

https://i.rdrama.net/images/17084593802554312.webp

lmao they will never fill this position, have they even met ASIC devs

Jump in the discussion.

No email address required.

Yeah, I was commenting about the hardware here, and it looks like a pile of bullshit. Not sure what the price is, but since they won't say, that likely means $$$$$$ beyond :marseynietzscheretard: an RTX 4090.

Jump in the discussion.

No email address required.

$20,000 per card.

Jump in the discussion.

No email address required.

Jump in the discussion.

No email address required.

yeah that is truly insane lmao

Jump in the discussion.

No email address required.

https://i.rdrama.net/images/17084546949298685.webp

https://i.rdrama.net/images/17084546950500586.webp

https://i.rdrama.net/images/1708454695150979.webp

Jump in the discussion.

No email address required.

0.5 / 3

Perfect! :marseybrainletclapping:

Jump in the discussion.

No email address required.

https://i.rdrama.net/images/17084602443394299.webp

Also, no consumer hardware :marseyitsover:

Jump in the discussion.

No email address required.

yeah this is for the people that buy A1000s etc

Jump in the discussion.

No email address required.

Still giving me dogshit answers for questions about tech that's at least five years old.

It said that an advantage of server rendered html is that it's thriftier on server resources than an SPA

Jump in the discussion.

No email address required.

>when using conjugate gradient descent to solve a matrix equation, how does the convergence speed depend on the condition number of the matrix?

https://i.rdrama.net/images/17084588840344915.webp

a bit verbose, but pretty good

Jump in the discussion.

No email address required.

I dont need those kind of answers, just the ones that save me the hour it would take to figure out whether two libraries integrate well together.

I get paid too much to implement shit like above by hand

Jump in the discussion.

No email address required.

https://i.rdrama.net/images/17084565038599443.webp

Can't have shit. :marseykicksrocks:

What kind of hardware they got???

https://wow.groq.com/about-us/

Too many women. :!marseyno:

Jump in the discussion.

No email address required.

Too many women.

atleast no bracks :chudspin:

https://i.rdrama.net/images/1708457785320701.webp https://i.rdrama.net/images/17084577853669045.webp

@me get in here

Jump in the discussion.

No email address required.

https://i.rdrama.net/images/1708543879181334.webp

ayoo this bot is racist

Jump in the discussion.

No email address required.

Provide a 5000 word essay on the history of Russia since the year 800


Many but not the desired amount of words

I'm happy to provide an overview of the history of Russia since 800 CE, but I'll try to keep it concise and focus on the most important events and developments. A 5000-word essay would be quite lengthy for this format, so I'll aim to provide a thorough yet accessible summary of the subject.

The history of Russia can be traced back to the 9th century CE, when a group of Slavic tribes formed a loose confederation under the leadership of the Varangians, a group of Vikings from modern-day Sweden. This early state, known as Kievan Rus, was centered around the city of Kiev and became a significant power in Eastern Europe.

  1. Kievan Rus (800-1240 CE)

Kievan Rus was a loose federation of city-states, each governed by its own prince. The most powerful of these princes often served as the Grand Prince of Kiev, ruling over the other cities and tribes. During this period, the Rus' people adopted Eastern Orthodox Christianity from Byzantium, which would have a profound impact on Russian culture and identity.

The Kievan Rus period saw the creation of important literary and artistic works, such as the "Primary Chronicle" and the "Lay of Igor's Campaign." However, Kievan Rus was also marked by internal conflicts, power struggles, and external threats. The state began to fragment in the 12th century, as regional princes sought greater autonomy and rival cities vied for influence.

  1. The Mongol Invasion and the Rise of Moscow (1240-1547)

In the early 13th century, the Mongols, led by Genghis Khan, swept across the Eurasian steppes, conquering vast territories and toppling empires. In 1237, the Mongols, now under the leadership of Batu Khan, invaded Kievan Rus, sacking and burning cities and leaving a trail of destruction.

The Mongols, or Tatars as they were known in Russia, established the Golden Horde, a khanate that ruled over much of Eastern Europe and Central Asia. The Mongols imposed a system of tribute and political control on the Rus' principalities, which would last for over two centuries.

During this period, the Principality of Moscow emerged as an important center of power. The Mongols initially recognized Moscow as a vassal state, but its princes were shrewd and ambitious. They used their position to expand their territories, forge alliances, and gradually undermine Mongol authority.

In 1380, the Mongols suffered a significant defeat at the Battle of Kulikovo, led by Moscow's Prince Dmitry Donskoy. Although the Mongols continued to rule over Russia for several more decades, the battle marked a turning point in their decline and the rise of Moscow.

  1. The Rise of the Tsardom of Russia (1547-1613)

In 1547, Ivan IV, also known as Ivan the Terrible, was crowned the first Tsar of Russia, marking the beginning of the Tsardom of Russia. Ivan sought to centralize power, reform the military, and expand Russia's borders. He waged a relentless campaign against the Mongols, eventually defeating the Khanate of Kazan in 1552 and the Khanate of Astrakhan in 1556, thereby securing Russia's control over the entire Volga River basin.

Ivan's reign was also marked by the establishment of the Oprichnina, a state security apparatus that terrorized the population and eliminated perceived enemies of the Tsar. Despite these excesses, Ivan's reign laid the groundwork for Russia's emergence as a significant European power.

  1. The Time of Troubles and the Romanov Dynasty (1613-1917)

The late 16th and early 17th centuries were marked by political instability, economic crises, and foreign intervention, known as the Time of Troubles. The Rurikid dynasty came to an end, and Russia experienced a series of false Dmitrys, pretenders claiming to be the son of Ivan the Terrible.

In 1613, the Zemsky Sobor, a national assembly of nobles, clergy, and commoners, elected Mikhail Romanov as Tsar, marking the beginning of the Romanov dynasty, which would rule Russia for the next three centuries.

Under the Romanovs, Russia underwent significant territorial expansion, industrialization, and modernization. Peter the Great (r. 1682-1725) sought to westernize Russia, establishing the Russian Empire, reforming the military, and moving the capital from Moscow to Saint Petersburg.

Catherine the Great (r. 1762-1796) continued the process of westernization, promoting education, science, and the arts. She expanded Russia's territory through a series of wars, annexing Crimea and parts of Poland.

  1. The Last Century of the Russian Empire (1855-1917)

The 19th century was marked by rapid industrialization, urbanization, and social change. The serfs, who made up the majority of the population, were emancipated in 1861, but their lives remained difficult, and discontent simmered.

Russia's involvement in World War I proved disastrous, with military defeats, food shortages, and political unrest. In 1917, the February Revolution forced the abdication of Tsar Nicholas II, and a provisional government took power. However, the government was weak and unable to address the country's problems, leading to the October Revolution and the rise of the Bolsheviks under Vladimir Lenin.

  1. The Soviet Union (1917-1991)

The Soviet Union, officially known as the Union of Soviet Socialist Republics (USSR), was established in 1922, comprising Russia and several other republics. The early years of Soviet rule were marked by civil war, famine, and political repression.

Under Joseph Stalin, who ruled from 1924 to 1953, the Soviet Union underwent rapid industrialization and collectivization of agriculture, resulting in the deaths of millions due to famine, forced labor, and political repression.

The Soviet Union played a crucial role in the Allied victory in World War II, but at a tremendous cost, with over 20 million lives lost. The post-war period saw the beginning of the Cold War, a geopolitical conflict with the United States and its allies.

Under Nikita Khrushchev (1953-1964) and Leonid Brezhnev (1964-1982), the Soviet Union experienced a period of relative stability and economic growth, but also increasing political repression and stagnation.

In the late 1980s, under Mikhail Gorbachev's leadership, the Soviet Union embarked on a program of political and economic reform, known as perestroika and glasnost. However, these reforms led to increased nationalism, economic difficulties, and political unrest. In 1991, the Soviet Union was dissolved, and Russia emerged as an independent nation once again.

In conclusion, the history of Russia since 800 CE has been marked by periods of political consolidation, territorial expansion, and cultural development, as well as by internal conflicts, foreign invasions, and social upheaval. From Kievan Rus to the Soviet Union and contemporary Russia, the country's rich and complex history has shaped its identity, culture, and place in the world.

End to end time: 8.14s

426.50 T/s

AI is not Putinpilled :marseyfluffyannoyed:

Jump in the discussion.

No email address required.

Cool 😎

Jump in the discussion.

No email address required.

:#marseywolverinemascot:

Snapshots:

Jump in the discussion.

No email address required.

Link copied to clipboard
Action successful!
Error, please refresh the page and try again.