Do you seem them at practicas? In nyc, practicas are more casual and for actual practicing and not like lite-milongas, so its possible to just verbally ask them to practice to sort of break the ice if you don't get the chance to even dance with them at milongas, and also ask them their thoughts on your dance. If they reject you at practicas (happened my first few years...which I find very rude), then yeah you can put them on your no dance list.
There's no actual practicas in London, to the best of my knowledge, they're essentially milongas that masquerade as practicas, but you do get the occasional dancers practising something. Would be nice to experience one, it's been a while!
I don't find it rude if an (unknown) follower rejects me at practicas or milongas — it's definitely their right to do so, and I don't take it personally even though rejection hurts, and they do it in full knowledge that I will probably not invite them again. I think that's fair. What I do find rude is if they're dismissive, snap out a sharp "No", things like that that make a lack of respect clear. For the followers who've danced with me at least once and decided that they didn't enjoy it enough, that's 100% okay with me, it's not like I've never danced with someone and decided it wasn't worth dancing with them again after all.
Followers go on my no-dance list if they have a "黑臉". Hard to translate all the connotations but essentially a stern, bitter or displeased expression, lit. "black face". Maybe they're frustrated after ironing their seats, maybe they've had a bad day, but I figure that if I can make my own entertainment at the milonga when I haven't danced in hours without looking visibly vexed, they have no excuse.
Or if I've tried to cabeceo twice or thrice at the same milonga, and then once more at a different milonga. Or if I really don't think dancing with them is better than sitting down for most tandas.
I bear followers who reject me no ill-will, I just don't see the point of making their acquaintance (except by chance) or acknowledging their existence beyond a simple smile and nod in passing. Way more worth it to focus on my people and things.