Yeah, I think I agree with tsb... until you find a dance that you really enjoy doing (and thus motivating to do), it's all going to feel equally difficult.
But as time goes on, there are definitely favorites of mine (VW). Usually that's a reflection with the confidence I have with the figures that I have learned. Among the dances I really don't know about doing it well, for me it's samba and jive. Samba has a different feel of hip motion that I am used to with rumba (I think because of speed). Jive breaks just about every rule I know about ECS, lindy, and WCS that it's just awful for me to do jive well without looking like I'm on a pogo stick.
Then there are the standard dances when I already have American smooth down. When a tango is playing, even if I'm in an international class (just starting out), I will still default to American figures because I am extremely comfortable with American tango.
So I guess to corrupt Yogi Berra said, ballroom is 90% mental and the other half is physical.