Not necessarily, the boats are rated for certain weight classes so they have different buoyancy

Rowing for men is separated into two categories: heavyweight and lightweight. Lightweight is when the average weight of the people in a boat is 70kg with a maximum of 72.5kg, anything above that is heavyweight.

You're at an advantage when you're big and tall in rowing but there is some variety, depending on the type of boat and the person's position in the boat. This can lead to some difference in weight within the heavyweight category. For example, in an 8 person boat ("an eight") you can have someone at the front of the boat ("bow") who can be relatively small and light compared to the group in the centre of the boat (positions 3, 4, 5, and 6). It wouldn't be unusual for men in these positions to be >100kg.

Rowing is definitely not about raw strength. Like any sport, there's plenty of nuance and technique. The smaller person you tend to see in films "calling the strokes" is a misrepresentation of the job a coxwain or "cox" does. They are not present in all boat types. When they are, they steer the boat, act like a second coach during training, call strategic pushes and provide encouragement/feedback during races. They don't call strokes, or at least they don't at any level beyond your first day in a boat if they do at all.