I didn't start formal training until I was 13. I started informal lessons when I was six, but growing up in a family where there was always music and dancing was expected at parties I've been doing it since I could walk.
I personally believe informal lessons are best5 at an early age...
These days you have all sorts of "movement" classes for children that are chock-full of games and exercises but none of the formality or structure of a ballet class. These are marvelous. They teach social interaction, coordination, musical interpetation etc. This is the stuff I learned just being around family members. I started with vernacular jazz when I was six from my grandparents just showing things and then as hip-hop slowly emerged I was part of the burdgeoning scene of cardboard carrying kids who spun, popped and locked at the malls. I learned by watching, we taught each other moves and invented new ones.
It wasn't until I was 13 that I chose to take formal dance lessons... personally I wish it had started earlier, but I wouldn't trade my informal instruction for it, so much as I wish I had the opportunity to do both.
The long and short of it is that as long as it is approached in a fun manner, something they enjoy there is no minimum age to begin an introduction to dancing. Formal classes will be dependant on the maturity and personality of the child. Don't push, but encourage.