/h/slackernews banner
Unable to load image

Yandex open sources 100B GPT-like model :marseysaluteussr::marseyrussiaglow:


PSA: Yandex is a multi-billion Moscow based company, finances the Russian war of aggression in Ukraine, and is one of the main Kremlin's tool in spreading propaganda and suppressing dissent.


YaLM 100B is a GPT-like neural network for generating and processing text. It can be used freely by developers and researchers from all over the world.

The model leverages 100 billion parameters. It took 65 days to train the model on a cluster of 800 A100 graphics cards and 1.7 TB of online texts, books, and countless other sources in both English and Russian.

Training details and best practices on acceleration and stabilizations can be found on Medium (English) and Habr (Russian) articles.

Make sure to have 200GB of free disk space before downloading weights. The model (code is based on microsoft/DeepSpeedExamples/Megatron-LM-v1.1.5-ZeRO3) is supposed to run on multiple GPUs with tensor parallelism. It was tested on 4 (A100 80g) and 8 (V100 32g) GPUs, but is able to work with different configurations with ≈200GB of GPU memory in total which divide weight dimensions correctly (e.g. 16, 64, 128).



Jump in the discussion.

No email address required.

This is based, AI companies are gating access to AIs to prevent people from misusing them. Glad Russia is giving out a reasonably powered AI and telling people to have fun

AI companies are gating access to AIs to prevent people from misusing them.

They are doing it to stop anyone from having any fun.

Agreed, we should be able to do all sorts of fun trolling with AIs.

If trolling is like fishing, AI trolling is using hand grenades. So of course people who don't understand fishing will hate it.


The art of fishing, perfected :marseycool2:

Lmao those old dudes got fucked up. Did the one in back go overboard?

Yeah, but it looks like the guy in front blew his own head off.

"Misusing" them how exactly? Making them say BIPOC?

A bunch of wet blanket "AI ethicists" (ie. foids who can't code but want to nanny an emerging field) are worried that being able to fake human interaction at scale/deepfakes will harm society. They are r-slurred though.

Tbf, the idea of social media platforms shadowbanning people by putting them in a closed ecosystem populated entirely by AI so they can't tell is concerning. A future full of people even more atomized and isolated because our elites have replaced even online interactions with a simulacrum designed to keep people docile and controllable.

Or even more effective astroturfing being used to shill and create a fake consensus. It's already happening, and imagining how much better it could get is concerning.

It's not really a genie we can keep in the bottle though. Keeping it out of the general public's hands is less of a concern than what governments and corpos will do with it.

And the longer you lock the tech away, the stronger will the first attack be against a naive population when AI finally becomes open

OMG who fucking cares, go into a fucking politics board.

People care. Programmers are people.

Programmers aren't people.

Never were

We are the next step in human evolution

t. bestselling artist, originator of marseyxcore

:marseyking: Hail to the Cat :marseyking:

Syntax error: Did you mean devolution?

>"will you post the same kind of comment when a US-based company releases ML models?"

>"Once US illegally invades a country, yes, probably."


:#marseyflagiraq::#marseyflagafghanistan: -> :#marseydisintegrate:


It's always going to be ironic to me that the only unfiltered search engine left is Russian.

Yandex is probably the best way to troubleshoot PC problems avoiding all those "Just reset your cache bro" AI generated articles, and it's a shame that the only competent search engine in the world is probably going to be banned because it comes from Mordor

It's definitely not unfiltered, it just doesn't care about filtering anything relevant to you. For the same reason I use Chinese-owned Opera.

The size of these models always amazes me. 200GB just for the parameter weights!

just for the parameter weights

Ya that’s not even including the 0.0000001% of other stuff!

Anyone else find it difficult to remember learned experiences?

Proud of our IT magicians. :marseysaluteussr:

Google was made by Russian Jew

unironically some of the smartest people on the planet. Oh wait, there's a wikipedia article on that


Reported by:

  • Unbroken: Misinformation, Russia is currently not engaged in any war

I am pretty sure twinks who contributed ain’t pro war





I'm not pregnant sweaty


Fat mods can also be pregnant sweety


Russian bots (human) on suicide watch.

>only trained on english and russian

Is this the most white supremacist AI ever invented?


Lets start raising money for a more sentient Snappy 2.0

bleeble blabble reset your modem