This sounds like 'The Call of the Void'. If I remember correctly, it's a real thing and you aren't insane.
Madness string 102526
If you're overdressed people will just assume you're coming from somewhere better or going somewhere better.