The skin on a man's face is significantly oilier than a woman's. This causes larger pores where free-radicals, environmental toxins, sweat (that turns to bacteria) sit
— a recipe for grime.
The 21st century man takes good care of their hygiene, an effort that is very attractive to 21st century women. (Queue, forward this to your men-friends, ladies.) It is not a bit emasculating to use good deep cleansing face wash as well as an exfoliant. Like I tell the ladies, find an scrub without ground nuts and shells that are harsh and cause microscopic tearing that causes wrinkles and sagging.
For best results, cleanse daily and exfoliate once a week if you can manage... twice a month is fine if that better suits you.