What groups made up the planter aristocracy? Why did their influence far exceed their numbers?

The planter aristocracy was a group of people who owned large plantations, full of slaves. They owned over 800 acres of land each and would have 50 or more slaves on their plantations. Their influence was huge because they had immeasurable wealth so everyone did what they said because they had the money.