Every day, it gets a little easier. But you have to do it every day, that's the hard part. But it does get easier.
Madness string 101920
It turns out your visual system has really strong views about whether faces should be inside-out or not, and it's willing to execute a hard override on perception if it doesn't like what it sees.