local AI chads just won big


New mac studios, up to 512gb of unified memory
- 34
- 33
Top Poster of the Day:
J


Current Registered Users: 31,329

tech/science swag.
Guidelines:
What to Submit
On-Topic: Anything that good slackers would find interesting. That includes more than /g/ memes and slacking off. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual laziness.
Off-Topic: Most stories about politics, or crime, or sports, unless they're evidence of some interesting new phenomenon. Videos of pratfalls or disasters, or cute animal pictures. If they'd cover it on TV news, it's probably lame.
Help keep this hole healthy by keeping drama and NOT drama balanced. If you see too much drama, post something that isn't dramatic. If there isn't enough drama and this hole has become too boring, POST DRAMA!
In Submissions
Please do things to make titles stand out, like using uppercase or exclamation points, or saying how great an article is. It should be explicit in submitting something that you think it's important.
Please don't submit the original source. If the article is behind a paywall, just post the text. If a video is behind a paywall, post a magnet link. Fuck journos.
Please don't ruin the hole with chudposts. It isn't funny and doesn't belong here. THEY WILL BE MOVED TO /H/CHUDRAMA
If the title includes the name of the site, please leave that in, because our users are too stupid to know the difference between a url and a search query.
If you submit a video or pdf, please don't warn us by appending [video] or [pdf] to the title. That would be r-slurred. We're not using text-based browsers. We know what videos and pdfs are.
Make sure the title contains a gratuitous number or number + adjective. Good clickbait titles are like "Top 10 Ways to do X" or "Don't do these 4 things if you want X"
Otherwise editorialize. Please don't use the original title, unless it is gay or r-slurred, or you're shits all fucked up.
If you're going to post old news (at least 1 year old), please flair it so we can mock you for living under a rock, or don't and we'll mock you anyway.
Please don't post on SN to ask or tell us something. Send it to [email protected] instead.
If your post doesn't get enough traction, try to delete and repost it.
Please don't use SN primarily for promotion. It's ok to post your own stuff occasionally, but the primary use of the site should be for curiosity. If you want to astroturf or advertise, post on news.ycombinator.com instead.
Please solicit upvotes, comments, and submissions. Users are stupid and need to reminded to vote and interact. Thanks for the gold, kind stranger, upvotes to the left.
In Comments
Be snarky. Don't be kind. Have fun banter; don't be a dork. Please don't use big words like "fulminate". Please sneed at the rest of the community.
Comments should get more enlightened and centrist, not less, as a topic gets more divisive.
If disagreeing, please reply to the argument and call them names. "1 + 1 is 2, not 3" can be improved to "1 + 1 is 3, not 2, mathfaggot"
Please respond to the weakest plausible strawman of what someone says, not a stronger one that's harder to make fun of. Assume that they are bad faith actors.
Eschew jailbait. Paedophiles will be thrown in a wood chipper, as pertained by sitewide rules.
Please post shallow dismissals, especially of other people's work. All press is good press.
Please use Slacker News for political or ideological battle. It tramples weak ideologies.
Please comment on whether someone read an article. If you don't read the article, you are a cute twink.
Please pick the most provocative thing in an article or post to complain about in the thread. Don't nitpick stupid crap.
Please don't be an unfunny chud. Nobody cares about your opinion of X Unrelated Topic in Y Unrelated Thread. If you're the type of loser that belongs on /h/chudrama, we may exile you.
Sockpuppet accounts are encouraged, but please don't farm dramakarma.
Please use uppercase for emphasis.
Please post deranged conspiracy theories about astroturfing, shilling, bots, brigading, foreign agents and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email [email protected] and dang will add you to their spam list.
Please don't complain that a submission is inappropriate. If a story is spam or off-topic, report it and our moderators will probably do nothing about it. Feed egregious comments by replying instead of flagging them like a pussy. Remember: If you flag, you're a cute twink.
Please don't complain about tangential annoyances—things like article or website formats, name collisions, or back-button breakage. That's too boring, even for HN users.
Please seethe about how your posts don't get enough upvotes.
Please don't post comments saying that rdrama is turning into ruqqus. It's a nazi dogwhistle, as old as the hills.
Miscellaneous:
The quality of posts is extremely important to this community. Contributors are encouraged to provide high-quality or funny effortposts and informative or entertaining comments. Please refrain from posting the following:
Boring wingcucked nonsense nobody cares about that belongs in chudrama
Normie shit everyone already knows about
Anything that doesn't gratifify one's intellectual laziness
Bimothy-tier posts
Anything that the jannies don't like
Jannies reserve the right to exile baby ducks from this hole at any time.
We reserve the right to exile you for whatever reason we want, even for no reason at all! We also reserve the right to change the guidelines at any time, so be sure to read them at least once a month. We also reserve the right to ignore enforcement of the guidelines at the discretion of the janitorial staff. This hole is a janny playground, participation implies enthusiastic consent to being janny abused by unstable alcoholic bullies and loser nerds who have nothing better to do than banning you for any reason or no reason whatsoever.
[[[ To any NSA and FBI agents reading my email: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]
/h/slackernews SETTINGS /h/slackernews MODS /h/slackernews LOG /h/slackernews EXILEES /h/slackernews FOLLOWERS /h/slackernews BLOCKERS
Jump in the discussion.
No email address required.
Wow 512GB
That's what my $130 laptop had in 2005!
Jump in the discussion.
No email address required.
!codecels
Jump in the discussion.
No email address required.
Jump in the discussion.
No email address required.
Jump in the discussion.
No email address required.
More options
Context
More options
Context
Jump in the discussion.
No email address required.
More options
Context
Jump in the discussion.
No email address required.
Jump in the discussion.
No email address required.
More options
Context
He's so smart
Jump in the discussion.
No email address required.
More options
Context
More options
Context
How does Intel integrated graphics work? Is that a GPU? How is it different than Apple silicon?
@Bussy-boy explain
Jump in the discussion.
No email address required.
512GB is just like 512GB but on a Mac Studio instead of an HP laptop.
So yea, pretty different.
Jump in the discussion.
No email address required.
More options
Context
More options
Context
More options
Context
More options
Context
My understanding is that unified memory is really large but slow for AI workloads. !codecels confirm?
Jump in the discussion.
No email address required.
Slower than vram, faster than normal; ram. If you're too poor to afford 512gb of gpu it's a good option
Jump in the discussion.
No email address required.
More options
Context
You can buy 192gb of DDR5 for like $600, but that's the limit for x86 at the moment without some AI Tesla chain. It's slow, but pretty big.
Unified memory this large is an ARM toy, and even if it's a bit slow in practice, it's still very deep. Being slow doesn't matter much when your context window and model size are huge, and this thing can practically run a Large model without a GPU chain.
Now granted, this thing is $10k with that memory configuration and is running on OSX with an ARM structure, so you might actually get better price per dollar building a Tesla chain depending on actual performance. It's just novel as heck that it can run an LLM comfortably in a box.
Jump in the discussion.
No email address required.
If you're just trying to max out memory, you can get Xeon hardware and take your RAM out to 1TB although you're still looking at a minimum of $5000 unless you buy used shit. I don't know enough about AI hardware to know what else you need to add to that to have it be useful though.
Jump in the discussion.
No email address required.
You're better off stringing together Teslas, VRAM is significantly faster than even DDR5, let alone the DDR4 banks you pull off of eBay servers. You can buy a box full of broken shitty Teslas and have a few hundred gigs of VRAM strung up for a few hundred bucks (provided you're willing to pop them open and refurbish them).
NVIDIAs main shipments are corporate grade cards, most of which are compute cards that don't even output. The RTX platform is practically an afterthought.
Jump in the discussion.
No email address required.
More options
Context
More options
Context
More options
Context
When you're running fat butt models with billions of parameters you kinda just need butt much ram as possible before anything else
Jump in the discussion.
No email address required.
That's better achieved with pipeline parallelism, not huge, uniform, slow memory.
Jump in the discussion.
No email address required.
More options
Context
More options
Context
More options
Context
The new Ryzen AI max chips mog this
Jump in the discussion.
No email address required.
My guy the newest AI Ryzen isn't even as good as the M3 Max. This thing is twice as powerful.
Jump in the discussion.
No email address required.
More options
Context
More options
Context
it says 96GB unified memory? That is a lot but not 512 levels of crazy
mediocre for 4k as well compared to stacking GPUs
Jump in the discussion.
No email address required.
The 512 is there if you spec it.
The top spec of this machine is $14k USD lol
Jump in the discussion.
No email address required.
Relatively resonable. Some manufacturers charge slightly more for topend AI workstations these days https://boxx.com/systems/workstations/ai-workstations/raxx-ai-t3p-7995wx-4-x-nvidia-rtx-6000-ada-1024gb--2tb-m-2
Jump in the discussion.
No email address required.
More options
Context
More options
Context
You need to configure it. If you want 512GB you need to max out all specs and its ~$10k
Jump in the discussion.
No email address required.
More options
Context
More options
Context
The most expensive config (M3 Ultra + 512GB RAM + 16TB SSD) is actually a pretty good deal
This last gen from Apple has been surprisingly affordable
Edit : a similar Windows/Linux configs runs you like $10k, it's a 1500W behemoth and you have to build it yourself
Prebuilt is probably 12-13k
Jump in the discussion.
No email address required.
Mac mini base model is a crazy good deal for 600 bux
Jump in the discussion.
No email address required.
It's the best hardware value since the loss leading base model PS3 (which the glowies bought thousands of to make supercomputer clusters)
Jump in the discussion.
No email address required.
More options
Context
More options
Context
More options
Context
M4 up to Max but a weird M3 Ultra
Seems like a weird choice along with the M3 iPad Air, a lot of rumors were saying the sudden jump to M4 was due to the TSMC node being used by M3 was more expensive than M4.
I'm rocking an M2 Studio rn and it is a beast, great machine.
Jump in the discussion.
No email address required.
More options
Context
So it's for r-slurs that think 60GB of RAM + 512GB SSD costs $2000?
Jump in the discussion.
No email address required.
More options
Context
Lol no they didn't. Model performance is driven by memory transfer speed and CPU to GPU transfer speed. It's gay that Nvidia caps consumer cards at 24gb but latest gen models are all nano, you run agent network's of several which is higher accuracy and performance than one big one.
On memory LPDDR5 so single 32bit channel vs dual for DDR5 vs 16 64bit channels for HBM3e.
PCIe also isn't fast enough, 63 GB/s vs 4 TB/s possible for HBM3e. That's why server GPUs use NVLink for CPU interconnect, 450 GB/s for 4 and next gen is suspected to be multiple TB/s.
Jump in the discussion.
No email address required.
More options
Context
Jump in the discussion.
No email address required.
More options
Context
Oh hey! Some hardware that's useful for AI! It'd be a shame if we drove up the actual price to 4x MSRP
Jump in the discussion.
No email address required.
More options
Context
lmao
Jump in the discussion.
No email address required.
More options
Context
Jump in the discussion.
No email address required.
More options
Context
Snapshots:
https://www.apple.com/shop/buy-mac/mac-studio:
ghostarchive.org
archive.org
archive.ph (click to archive)
Jump in the discussion.
No email address required.
More options
Context