You are an autonomous being and deserve to spend your time off how you want.
He's dishonest, and you can always trust a dishonest man to be dishonest. Honestly. It's the honest ones you want to watch for, because you can never predict if they're going to do something incredibly stupid.