A couple of things here...
If they are unfit for office, it’s not necessarily easy to give them that reason as it may lead to a legal challenge etc etc... they may win on a technicality and you end up with someone if it for public office. If this were a reason, I don’t have a problem with it. There maybe stuff that can’t be said, or they may just be hideous to work with.
If it is because they want someone who better fits the image of the party then what’s the problem? Under corbyn they wanted more left wing people, of course they did, it was perfectly reasonable that the leadership (then as now) gave direction to this....