How were wealthy Southerners able to avoid serving in the Confederate Army despite its dire need for soldiers?

The answer is:

Because of the Twenty-Slave Law.

Explanation:

Confederate congressmen passed a variety of exemptions to maintain industrial and agricultural production.

By authorizing the exemption of one white man per plantation with twenty or more slaves. The law also allowed an overseer exemption for two or more plantations within five miles of each other with collectively twenty or more slaves.