Unable to load image

Yandex open sources 100B GPT-like model :marseysaluteussr::marseyrussiaglow:

https://old.reddit.com/r/programming/comments/vit8xs/yandex_open_sources_100b_gptlike_model

								

								

PSA: Yandex is a multi-billion Moscow based company, finances the Russian war of aggression in Ukraine, and is one of the main Kremlin's tool in spreading propaganda and suppressing dissent.

:#soyjaktalking:

YaLM 100B is a GPT-like neural network for generating and processing text. It can be used freely by developers and researchers from all over the world.

The model leverages 100 billion parameters. It took 65 days to train the model on a cluster of 800 A100 graphics cards and 1.7 TB of online texts, books, and countless other sources in both English and Russian.

Training details and best practices on acceleration and stabilizations can be found on Medium (English) and Habr (Russian) articles.

Make sure to have 200GB of free disk space before downloading weights. The model (code is based on microsoft/DeepSpeedExamples/Megatron-LM-v1.1.5-ZeRO3) is supposed to run on multiple GPUs with tensor parallelism. It was tested on 4 (A100 80g) and 8 (V100 32g) GPUs, but is able to work with different configurations with ≈200GB of GPU memory in total which divide weight dimensions correctly (e.g. 16, 64, 128).

https://github.com/yandex/YaLM-100B

https://news.ycombinator.com/item?id=31846593

65
Jump in the discussion.

No email address required.

This is based, AI companies are gating access to AIs to prevent people from misusing them. Glad Russia is giving out a reasonably powered AI and telling people to have fun

Jump in the discussion.

No email address required.

AI companies are gating access to AIs to prevent people from misusing them.

They are doing it to stop anyone from having any fun.

Jump in the discussion.

No email address required.

Agreed, we should be able to do all sorts of fun trolling with AIs.

Jump in the discussion.

No email address required.

If trolling is like fishing, AI trolling is using hand grenades. So of course people who don't understand fishing will hate it.

Jump in the discussion.

No email address required.

![](/images/16560023375810962.webp)

Jump in the discussion.

No email address required.

The art of fishing, perfected :marseycool2:

Jump in the discussion.

No email address required.

Lmao those old dudes got fricked up. Did the one in back go overboard?

Jump in the discussion.

No email address required.

Yeah, but it looks like the guy in front blew his own head off.

Jump in the discussion.

No email address required.

:#karlmarxey:

Jump in the discussion.

No email address required.

"Misusing" them how exactly? Making them say BIPOC?

Jump in the discussion.

No email address required.

A bunch of wet blanket "AI ethicists" (ie. foids who can't code but want to nanny an emerging field) are worried that being able to fake human interaction at scale/deepfakes will harm society. They are r-slurred though.

Jump in the discussion.

No email address required.

Tbf, the idea of social media platforms shadowbanning people by putting them in a closed ecosystem populated entirely by AI so they can't tell is concerning. A future full of people even more atomized and isolated because our elites have replaced even online interactions with a simulacrum designed to keep people docile and controllable.

Or even more effective astroturfing being used to shill and create a fake consensus. It's already happening, and imagining how much better it could get is concerning.

It's not really a genie we can keep in the bottle though. Keeping it out of the general public's hands is less of a concern than what governments and corpos will do with it.

Jump in the discussion.

No email address required.

And the longer you lock the tech away, the stronger will the first attack be against a naive population when AI finally becomes open

Jump in the discussion.

No email address required.

agreed, but for different reasons. Languages models are 100% being used to dilute wrongthink forums with human-like gibberish, but ethicists are still r-slurred. They think that gatekeeping ai will stop it from being misused, when really all it does is stop the proles from participating in the fun (ironic since most of these types are leftists)

Heres proof that my :marseyleftoidschizo: idea is used IRL (even if only for shits and giggles)

Jump in the discussion.

No email address required.

they aren't wrong the problem is that the harm has already been done when the internet became accessible to the public.

Jump in the discussion.

No email address required.

"Only approved context setters are allowed to do that!!"

:#soyjaktalking:

Jump in the discussion.

No email address required.

even libgen is Russian right?

Jump in the discussion.

No email address required.

OMG who fricking cares, go into a fricking politics board.

People care. Programmers are people.

Programmers aren't people.

Jump in the discussion.

No email address required.

Never were

Jump in the discussion.

No email address required.

We are the next step in human evolution

Jump in the discussion.

No email address required.

Syntax error: Did you mean devolution?

Jump in the discussion.

No email address required.

>"will you post the same kind of comment when a US-based company releases ML models?"

>"Once US illegally invades a country, yes, probably."

:#soyjakanimeglasses:

:#marseyflagiraq::#marseyflagafghanistan: -> :#marseydisintegrate:

Jump in the discussion.

No email address required.

It's always going to be ironic to me that the only unfiltered search engine left is Russian.

Jump in the discussion.

No email address required.

Yandex is probably the best way to troubleshoot PC problems avoiding all those "Just reset your cache bro" AI generated articles, and it's a shame that the only competent search engine in the world is probably going to be banned because it comes from Mordor

Jump in the discussion.

No email address required.

It's definitely not unfiltered, it just doesn't care about filtering anything relevant to you. For the same reason I use Chinese-owned Opera.

Jump in the discussion.

No email address required.

The size of these models always amazes me. 200GB just for the parameter weights!

Jump in the discussion.

No email address required.

just for the parameter weights

Ya that’s not even including the 0.0000001% of other stuff!

Jump in the discussion.

No email address required.

Anyone else find it difficult to remember learned experiences?

Jump in the discussion.

No email address required.

Proud of our IT magicians. :marseysaluteussr:

Jump in the discussion.

No email address required.

Google was made by Russian Jew

Jump in the discussion.

No email address required.

unironically some of the smartest people on the planet. Oh wait, there's a wikipedia article on that

Jump in the discussion.

No email address required.

:#marseywhitemagepat:

Jump in the discussion.

No email address required.

Reported by:
  • Unbroken : Misinformation, Russia is currently not engaged in any war

I am pretty sure twinks who contributed ain’t pro war

![](/images/16559919443167787.webp)

![](/images/16559919443776333.webp)

Jump in the discussion.

No email address required.

:#marseyhanger:

Snapshots:

Jump in the discussion.

No email address required.

I'm not pregnant sweaty

:#marseychonkerfoid:

Jump in the discussion.

No email address required.

Fat mods can also be pregnant sweety

:marseytrans2:

Jump in the discussion.

No email address required.

Russian bots (human) on suicide watch.

Jump in the discussion.

No email address required.

>only trained on english and russian

Is this the most white supremacist AI ever invented?

:!#marppyenraged:

Jump in the discussion.

No email address required.

Lets start raising money for a more sentient Snappy 2.0

Jump in the discussion.

No email address required.

bleeble blabble reset your modem

Jump in the discussion.

No email address required.

Link copied to clipboard
Action successful!
Error, please refresh the page and try again.