Frankly;
I don,t know if the little uns will be chewing on their hats. I would use low VOCs paint from Rustoleum. They are sprays and you can then clearcoat in Matte or Gloss. I am sure that they would work. The average Hobby paints won't have the longevity you need. Saliva and Greasy baby hands will remove most of them Including Humbrol!
I've tried the painters touch sprays from rustoleum but they're so hard to control, I'm useless with spray cans. Plus masking the foam is impossible, nothing sticks properly and if you get spray paint on it it goes hard and brittle, no good against babies skin.