For me personally, as soon as I feel that I can’t keep an eye on everyone safely, the class is too big. But this is dependent on the participants and usually the number of novice bootcamp/class participants. For something like boot camp, I generally cut off the enrollment at 20. That is pushing it a little, but if I have a class that big, I really go into extra high monitoring mode. Which is pretty high already as I really focus on monitoring participants.
I think that as with any group training that “too big” is a product of what you’re doing (i.e. how complex the workout is), how experienced your group is (i.e. a group consisting of newer members who may not know the particular exercises or what to do, needs more careful monitoring), the age of the participants (i.e. of course kids need more monitoring than some adults), and the ever important trainer-to-client ratio that you’re comfortable maintaining without adversely effecting client safety, your ability to be effective etc.
A lot of factors to consider.
I hope that this helps.