not only does it convert back losslessly it also has the noise removed
— moderate rock (@lookoutitsbbear) May 27, 2024
I am not an electrical or comp sci engineer but I have had some experience on the electrical side of things lately and have started becoming familiar with signal processing, so bear with me for minor mistakes if you are one of the nerds familiar with it. For those of you who are too or not deep enough on the spectrum to understand signal processing, all you need to know is this: noise is not data. It is never data. A good signals engineer must attempt to minimize noise as much as possible.
Neuralink put out some request recently asking for some insane level of compression on a file (wanting 200x compression) with a transmitted signal, and that this must be lossless (meaning the original file data can be perfectly recovered from the compressed file data). Neurodivergents online immediately set out working on this to determine what the max amount of compression is. Enter our king and hero, moderate rock (user @ lookoutitsbbear).
MR states that so far he's found a way to compress the initial data file by a factor of 4.1, the most seen so far.
Curious as to how, people start asking how well this actually works.
https://x.com/lookoutitsbbear/status/1794962035714785570
And this is where all heck breaks loose. See, Neuralink had technically asked for the original file to be compressed without any data lost. To the average midwit and/or software engineer, this means taking the original file and just making it smaller. But MR did what any good signal engineer would do, and worked on filtering the signal to get rid of unwanted and unnecessary information (the noise) so that he could do a better job of compressing the data. Midwits do not understand that noise is not relevant for a signal's information. And because of this, MR has sinned and for this, he must be dunked on. So the beatings commence.
"Heh kid, just google it. You idiot. You moron."
There are multiple midwits continuing to repeat the "noise is data" line, as though repeating it makes it true.
Once again, noise is not a relevant part of a signal.
The midwit bonanza continues, once again acting as though if noise is an important part of a signal.
Quote tweets even gain major traction dunking on him even though they're all wrong.
MR makes a quote tweet that at least gains traction with people who understand him, and the semantics argument becomes a bit more present.
Yes, on a technical level it is not lossless, as MR removed "information." But the "information" he removed is not relevant. The original signal present in the original file is recoverable after the compression. So he has done it correctly. MR also points out that the Neuralink engineers are being stupid because they should be working on getting rid of the noise before working on the compression.
Of course, the midwits can't admit they're wrong and continue to argue the semantics.
Someone who actually makes a decent enough analogy to understand what he's getting at.
There is a lot of continued insults and attempts at dunks here (too many for me to add at this point) and midwits continue try to dunk on him and he continues to shrug it off.
Adding to the hilarity, a PhD looks at the Neuralink thing and comes to the same conclusion on his own, that removing noise from the signal will help compression.
https://x.com/CJHandmer/status/1795486204185682315
https://x.com/kindgracekind/status/1795577979952845220
(some cope in the replies of this one that MR was "wrong" because of how he stated it, even though he wasn't)
Finally, our king decides to take a rest, having survived his beatings and coming out stronger
So remember dramatards: your average midwit has no clue what they're talking about, software engineers should stick to learning to code, and you should always listen to the neurodivergent savants
Jump in the discussion.
No email address required.
Even if you believe this, there's no objective way to tell the difference
Anything subjective obviously doesn't meet the request
I'm honestly not sure if you're baiting or not, so well done for that
Jump in the discussion.
No email address required.
The noise is not the data. Objectively.
Jump in the discussion.
No email address required.
No not objectively. Objectively (from a physics standpoint) it is all vibrations in a medium. I know electrical engineers have a different perspective but then that goes back to it not being objective dont it?
Jump in the discussion.
No email address required.
From a physics standpoint if you capture data beyond the precision of your measurement device it is noise. Introducing noise in the signal path doesn't magically make it data from the original measurement.
Jump in the discussion.
No email address required.
How is the tool of measure capable of capturing data beyond it's own ability? Or if your tool of measure measures/records/picks up data and you don't like the result, does that make the data noise? There still has to be an element of discrimination to label certain data sets as either "noise" or "information" don't it?
Jump in the discussion.
No email address required.
More options
Context
More options
Context
But it is objective,
Jump in the discussion.
No email address required.
What is?
Jump in the discussion.
No email address required.
the noise not being the data
Jump in the discussion.
No email address required.
Well there is no objective difference between the two, unless you consider yourself objective?
Jump in the discussion.
No email address required.
maybe you just lack understanding
Jump in the discussion.
No email address required.
how about you tell me what you are actually thinking instead of all this vague shit?
Jump in the discussion.
No email address required.
More options
Context
More options
Context
More options
Context
More options
Context
More options
Context
More options
Context
More options
Context
In the context of the challenge, everything in the file is data, there's no noise there.
Jump in the discussion.
No email address required.
proof?
Jump in the discussion.
No email address required.
You can't perfectly recover the original file if you remove noise. Do you often have difficulty understanding the things you read?
Jump in the discussion.
No email address required.
you can perfectly recover the original. more than perfect actually since you got rid of noise.
Jump in the discussion.
No email address required.
No, you can't perfectly recover the original. The new file will be different from the old.
Jump in the discussion.
No email address required.
No it won't, it'll be compressed and better than the old file perfectly capturing the original signal
Jump in the discussion.
No email address required.
No one cares about the signal, you sperg.
Jump in the discussion.
No email address required.
More options
Context
More options
Context
More options
Context
More options
Context
More options
Context
More options
Context
More options
Context
Noise is data. It's not the data they're interested in, but it's data
Jump in the discussion.
No email address required.
noise is not data, is noise, should never have been there in the first place.
Jump in the discussion.
No email address required.
Should we get the movie nerds in here to argue about film grain
Jump in the discussion.
No email address required.
More options
Context
More options
Context
More options
Context
More options
Context
If you know the performance of the ADC used you can determine a noise floor. In this case, a 10 bit adc has a noise floor of ~60 dB. By extension, all the high frequency data below 60 dB can't be from the original signal.
Imagine you task someone with reading an analog thermometer that has markings per ones digit. They return to you a notebook with measurements to the one billionth and complain that it will take too long to transcribe to an excel sheet. Obviously, you can disregard the data after the first decimal point because reading 1/100th of a digit on an analog thermometer only marked to one digit is impossible to do accurately. This analogy is not great (ignores that the input signal is very low and clipped) but I hope you understand why, objectively, some noise can't be signal data.
This entire "problem" is a complete waste of time when the problem is incorrectly stated. It isn't a software issue, it is a signal processing issue.
See: https://x.com/lookoutitsbbear/status/1795562684462624806
Jump in the discussion.
No email address required.
it isn't either of these. the whole question is a non-starter. might as well be asking about the chicken or the egg
Jump in the discussion.
No email address required.
More options
Context
More options
Context
Of course there is. Noise is normal distributed, data is not.
Jump in the discussion.
No email address required.
More options
Context
More options
Context