Among a certain demographic in the US, there's a lingering concept of adulthood which suggests that anything people actually enjoy and that makes us happy is childish; that, beyond a few, specific, pre-approved hobbies, our lives are not sufficiently "adult" if we're not constantly miserable.
I kind of feel like this thing you've noticed about books is in the same ballpark. Reading is not one of those "approved" hobbies, so the best books are the books that make us sad, upset, or otherwise disgruntled. If they don't, they're not serious and adult enough. Which is why various parties did a Big Concern back in the late 90s when Harry Potter first got popular and a ton of, gasp, adults were reading it. Local news stations bemoaned the phenomenon as evidence of all sorts of uncouth things, from taking stabs at the adult literacy rate to pondering what factors made people not want to "grow up". Anecdotally, I endured similar complaints from multiple people in my own life, including older co-workers and my ex-wife (this pattern being one of the first times I noticed a generation-based values divide).
Considering that the top literature reviewers, publication editors, literature professors, and award committees are more likely to belong to the same demographic, it's not surprising that sad, "serious" books get all the good press and books that are actually fun to read get panned.