Patriarchy is at fault for

That's historically illiterate. Men used to be far more into astrology. Virtually all early astronomers believed in astrology including the ones we think as very rational, like Galileo. Once men have definitely empirically proven that astrology is bullshit, women got into it.