Many resorts offer fitness classes, so fitness is part of the options that come with the destination. I taught at one of these for a week.
Certain tour operators offer fitness retreats. Mostly yoga retreats all over the world, but I’ve seen some general fitness retreats as well.
Lastly, there are travel companies that specialize in adventure travel. While not directly “fitness,” they require a certain fitness level to attend, and there is a certain amount of activity expected along with that particular vacation. For example, my husband and I went on a 4 day guided rafting tour on the Snake river through Hell’s canyon. That required a moderate fitness level. The tour company ranks each of its tours based on the amount of hiking and rafting (and the difficulty thereof) in its brochures.