I’m not sure where to start here, so here are two equally important building blocks.

First, aside from other reasons the Nazi/minority is wrong, you are comparing a label somebody gets for existing the way they were born with a label somebody gets for actions they take that harm other people.

Second, some kind of mishmash of the terms “social contract” and “paradox of tolerance.”