A single rando can deny your interaction on their instance, not on a bunch of other instances. Shared blocklists notwithstanding, but I would hope if you're following a shared blocklists you're vetting it to an extent.