It seems like it’s the word “elite” that’s annoying you. The standards wouldn’t be just about who fits into a vague elite, but who is intellectually, ethically, or spiritually superior. Again, we can take the easy example of who’s a lifelong criminal and who’s a law-abiding citizen, and work backwards to discern the standards we’re implicitly using to distinguish between those classes of people.

The idea that we have a divided nature is well-established. You can look at it biologically, since the brain has roughly three stages for processing information: the primal, reptilian layer (basal ganglia), the emotions (limbic system), and the latest to evolve, the neomammalian layer (cerebral cortex, for language, long-term planning, reasoning).

So those who behave more like animals and betray their higher calling would succumb to the temptation to use mainly their more primitive brain functions as they react mindlessly to circumstances (committing crimes, etc) and don’t plan for the long-term. Or they don’t combine the parts of their brain with much creativity, thinking in cold-hearted, left-brain terms and not empathizing emotionally with their victims.