Oxford is to Swindle Town as Southend is to Col Ewe, as I recall. I can see why they might be miffed. It's not rational for them to be so (after all, to most footballers, a club is merely a place of employment rather than something close to their hearts), but it is understandable.
I dont think a player wants to be linked with a rival for very rational reasons. Imagine if we found out one of our players said he wanted to join col u, he would be public enemy no.1 and if it didnt come off he would have to live with the stick.