I mixed 1:450 and 1:500 people on a diorama, but that is a bit less of a difference than 1:350 and 1:400. Still, I found the mix fine- did not really notice the difference. One of the sets I used had a number of children, so that helped- you could see a mix of sizes anyway.
A thought- you might want to consider the 1:450 people. I believe I got them from a Japanese importer- there is a new micro size model railroad scale at 1:450 and accessories include people and cars. These are civilian people, though, but you could probably just repaint adult males.