This article from your friendly Georgia Cooperative Extension folks has good info and even some diagrams to help you prune:
we have trimmed ours back a lot a few years ago and they have been doing very well since. I don't think they are as delicate as many people make them out to be.
The general rule is not to cut a plant back by more than one-third at a time. But if your azaleas are really out of shape, you can do a rejuvenation pruning, in which they are cut back severely, to say within 6 inches of the ground. If you do this, be prepared for it to take a couple of seasons for the plants to fill back in.
I know there are rules to pruning azaleas, but if they are well establishes you can tale back to the ground, and they will come back looking new..have done it several times with 30+ year old azaleas.
Azaleas in the deep south withstand all kinds of pruning butchery but the kind I really hate are those people that prune them like hedges or topiaries, give them flat-tops or round into balls. They live and bloom but look so much prettier if gently lightly pruned to a nice natural shrub shape.