Some were used sparingly by some stateside training units. Some were used in the Burma theater. Some (a few dozen?) were given to US engineering units from UK stocks in the build up to the invasion of Southern France.
Rare and far away places. Think about it -- the engine required an entirely different supply/spare/repair system than the other, more commonly used Shermans. Go with a UK, Polish, or French scheme. Lots of interesting markings for non-US M4A4s.