HO scale figures are often said to be 1/72, not far from 1/96. N scale figures are smaller, though I forget the exact ratio. What is good in contexts like this is that every HO and N scale figure maker (and those who make figures in the similarly sized 25mm scale or 15mm scale) do so in a slightly different fashion.
There are a fair number of sailor figures in HO scale and by looking around you can hopefully find a set that looks right for your 1/96 ship.
I also recall seeing this model in versions that actually included figures. Perhaps an internet search will find such a model or, better yet, a model manufacturer who will sell you only the figures.
Good luck
Skallagrim