Not just the cultural feel, but also the sense of the dialogue would be lost. I found the mixing of English into the Japanese to be interesting.
Tamako-sensei would sometimes count in English, sometimes in Japanese. Plus her "slow, slow, quick quick". And Mia's [name?] final invitation in English of "Shall we dance, Mr. ___?" (sorry, forgot his name) would have also been lost if it had been dubbed.
Years ago I saw a French movie that had been dubbed into English. In one scene, the guy meets with his tutor for an English lesson. Needless to say, the whole scene didn't make any sense at all to an English-speaking audience.
Similarly, I saw Cabaret in Germany dubbed into German. Remember when he first meets Sally and she stumbles through broken German before bursting out in English that she's just dying for a cigarette. Same thing in the German version, only she goes from broken German to fast and fluent German.
I can't understand why people hate subtitling so much. The only disadvantage I see to it is that when I'm watching one at home I have to keep my eyes on the screen, whereas if it's in English then I can work on something and still follow what's happening by listening.
Up-side to subtitling: One Scandanavian country -- Sweden, I think -- shows programming from other countries all subtitled, even cartoons for the children. They have one of the highest literacy rates.