You’re still free to remove what you don’t like, but you’re not entitled to have the mod hosted on any site you want. The site owners decide that, and they don’t want it on their site. That’s not infringing on your freedom, but forcing it to be hosted on their site would indeed infringe on the site owner’s freedom.
Nexus Mods Fine With Bigots Leaving Over Removed Starfield ‘Pronoun’ Mod (kotaku.com) angielski