John Coltrane - My Favorite Things (Live at Newport '63)

Judging from the YouTube numbers and my personal experience most people really do not get this music, despite quite a bit of popularity in later instrumental artists (like Pink Floyd), and even classical music.

I think the reason for this is that it's simply easier to understand a genre of music when you've been surrounded by that genre from birth, e.g. if a musician deliberately plays a really cliche line and sends it in an unexpected direction, or quotes an unrelated piece, or makes a musical joke, you'll be able to understand it, which you obviously wouldn't be able to if you hadn't heard hundreds of songs in that genre before. The end result of this is that you need to listen to a lot of jazz and put in some effort before you can fully appreciate it, at least more than other genres. Then there's also other barriers of entry like the fact that a lot of the older stuff is poorly recorded, percieved musical elitism, and that most people don't like listening to multiple different versions of the same song.

As for why people listen to and/or are able to understand classical music when the same isn't true for jazz, I think there's a few reasons:

1. A lot of people don't actually understand it, and just treat it like background music (the recent trend of cringeworthy Youtube videos and Spotify playlists titled things like "classical music to make you feel like a 17th century villain" is more than enough proof of that), with "relaxing" being the first word used to describe pretty much every really famous piece other than Mars, Bringer of War and 1812 Overture.

2. People are much more likely to put the effort into something understanding a genre of music if it has some element of prestige attached to it, with the aforementioned element of percieved musical elitism (along with regular elitism) making a certain subsection of people more likely to put in the effort so they can "get" it.

3. There are a lot more people in the US who are surrounded by classical music from birth than by jazz music; lots of parents who fall into category 2 are into the idea of their kids playing some kind of musical instrument, and even if their kids don't fall into category 2 when they grow up they'll still be able to understand the intricacies of classical music better than someone who never learned violin/piano/whatever.

TL;DR: Rock is ubiquitous so everyone understands it, jazz is different and complicated so not a lot of people understand it, classical music is different and complicated but has enough prestige attached to it that people are either willing to put in the effort to understand it or don't get it and like it anyway (also it is ubiquitous for certain people).