granting rights to a group can absolutely diminish rights of another
if a group has a right to claim government funding due to specific needs of that group, and then a new group is allowed access to those funds, now there is less money for both of them...or, if more money is allocated to keep up, that can remove funding from other groups out there
if women have a right to feel safe and comfortable in their own spaces, and then you say those spaces have to be opened to trans women, that can trample their right to feeling comfortable, you're saying that right is less important than being more accommodating
or, you gaslight them and tell them they never had a right to feel comfortable to begin with