I do not believe that we are jacks of ALL trades, but I also do not believe that we should stubbornly stick to our ‘specialty’. I like to think of myself as a trainer who has trained enough older adults and people with special medical concern, that this has become me ‘specialty’. That does not mean that I cannot or do not want train a younger healthy person (in fact, it’s a welcome break).
However, for me it means that I recommend that a person look for another trainer if the potential client is a child or a teenager. I have also never trained a woman who is pregnant or who is post-partum, and since I have never been pregnant myself, I would refer out.
I also do not consider myself the best trainer for a competitive athlete; but that issue never arises because I would probably not be their first choince anyway.
I do not see your distinction between men’s and women’s fitness that I would consider those to be individual specialties (except as stated in the case of pregnancy).