You could have a couple things. First, for picking your size, did you go by a off the rack size? If you're normally a size 6, you'll be a completely different size in patterns. And each pattern maker sizes differently. So definitely match your measurements to the specific pattern makers size guide.
If it's a drapey top, it's very likely it's cut on/off the bias (whichever way means diagnonally, I can't remember).
If you stretch fabric along the lines of the threads, you'll get a certain amount of stretch. If you stretch diagonally across the threads, you'll get a lot more stretch. Most fabrics will also "relax" and become bigger along any lines cut across that diagonal. (It's why using pi and perfect geometric measurements to create a circle skirt will leave you with a giant waistband. Fabric will grow on the diagonal.) It makes things drape beautifully, but can be difficult to deal with when you're trying to make something have a tailored fit.
Add the diagonally relaxing to a fabric that's already got some stretch (and running stretch fabric through the sewing machine will tend to stretch it a bit, even if you don't think you're stretching it.) Well all that can end up with your creation being a few inches bigger than you originally planned. Especially if you're sewing with stretch fabric on a pattern that was designed for non-stretch fabric.
I think you've mentioned that you're on the larger chested side. Depending on the drape of the top, you might be able to just pinch it in at the waist a bit to make it workable. Which is a lot easier than altering darts and such. So I'll cross my fingers that will work for you. If not, every project is a learning experience that will make the next project turn out better!
