Recovery, phyiologically, describes the biochemical and structural processes that occur following exercise. Of course the extent is directly related to the intensity, type, and duration of the effort. Following glycogen depletion effort, for example, cycling 50 miles to exhaustion, it may take 2-3 days for the targeted muscles to fully repack their glycogen stores, that’s assuming adequate carbohydrate and water intake. Following maximal power effort to temporary muscle fatigue (like with a maximal effort, major muscle, free weight workout, (8-12 RM), it may take at least 2-3 days for the streaming of the contractile protein actin/myosin/tropomyosin/troponin complex in skeletal muscle cells to return to normal structure.
The point is that rest and recovery (including hydration and sensible eating) are a critical part of the conditioning process, for both fitness and performance. Taking “at least” a day off can be the best advice.