That is your boundary. Feel free to have it. But also reflect on why you have that feeling. Just because it is your personal boundary does not make it exempt from bigotry. I question all the time why my gut reaction goes a certain way and when I first started doing it, I found that a lot of the things that I felt were based in some sort of bigotry. With gay men in particular, a lot of those feelings that we think are internal are actually due to a shitload of toxic masculinity in our culture.