If the difference is severe, then a medical workup is needed. There are nerve compression syndromes that can cause neurogenic muscle atrophy, but that’s usually severe.
Otherwise, it may be a dominant side vs. non-dominant side issue. Any other muscle asymmetries? Do you have a job where one side is working harder than the other?