When you create a PR part of the message is like this:

When this pull request is ready to be reviewed, an RFR email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label (add|remove) "label" command.

This suggests that the label needs to be in quotes, but:

/label remove "foo"

results in an error as quotes are not needed or allowed.

@dholmes-ora Usage: /label <add|remove> [label[, label, ...]]wherelabel` is an additional classification that should be applied to this PR. These labels are valid:
...