No, I don't think that follows. We've talked about this before, I believe, when you shared your view of how colours are produced.

The phenomenon of colour blindness shows that private experiences can differ despite our use of the same labels. Labels are highly general in that they're like buckets in which we plop lots of things, abstracting from some differences to track real patterns (categories) that nevertheless hold.

We can both talk about apples because even though we may experience the fruit differently, not just with possible colour blindness or because you're looking at an apple at a different angle, but because we each associate "apple" with different background concepts. Perhaps I had a traumatic experience of apples as a child, when I saw a worm squirm out of one I was about to eat. Thus, "apple" would mean something different for me, distinguishing my private experience of apples from yours.

I don't see how the commuicative capacity of language is going to eliminate those inevitable differences in inner experience. But I take it this somehow challenges your monism.