You make some great points. I think of that line from Yeats' poem: "The best lack all conviction, while the worst are full of passionate intensity."

I'm glad you mentioned the Dunning-Kruger effect, but there are some other causes that might be explored such as American individualism and populism (democratic anti-elitism); the self-destruction of modern humanistic values via postmodern hyperskepticism; and consumerist infantilization (a function of capitalism and First World self-entitlement).