I was blessed to grow up listening to good music. Almost by accident. My dad, just your typical Midwestern farmer, had a set of CDs he would play over and over again at home. (As well as listen to NPR, always being sure to switch it off before The Voice of Perdition All Things Considered came on.) A lot of these songs and composers I couldn't name until years afterward. But it did help me cultivate an idea of what kind of music is - or isn't - "irritating to the mind". I listen to a lot more music post-1950 nowadays, but it's good to have classical music as a reference point for listening to new music - almost like an oasis. I can't speak for people who learned the other way around (growing up with modern music and then 'going backwards') but it's really helped me understand the fundamentals of how music works. Especially with Gregorian chant. I didn't really become proficient at reading music until I delved into chant.