Those are some great questions and observations. I've thought and written a lot about this, and I have some upcoming articles that address it (in the context of Neil deGrasse Tyson's prejudice against philosophy).

If I had to put it in a nutshell at the moment, I might say it's a case of ambivalence. Philosophy is socially harmful but it's also admirable because it condemns that which is too weak to withstand scrutiny, like the hypocrisies of mass cultures. So philosophy is bad for imperfect cultures--a fact which often marginalizes "philosophers" in this broad sense, meaning intellectuals, including artists, that is, those who are obsessed with ideas even if that's at the expense of persons, places, and things. But philosophy is good for the soul in that it makes us potentially transhuman. Philosophy burns away nonsense and gives us intellectual integrity. But this is a cold, possibly self-destructive kind of maturity, the kind we might attain before a moment of bathos or our ironic downfall.

Yet there's a sci-fi vision, at least, of an advanced, godlike species, of one that's made its peace with the unsettling facts of life's existential condition in the universe. This is a secular version of the old religious meanings of life. Whether it's attainable or still faith-based are open questions. But if this brighter, more honourable future is at least possible, philosophy won't be wholly absurd or destructive. Philosophy would destroy only our flaws.