I didn't serve so I can't answer your question. I do however think that when I see someone wearing one, I assume it was their ship.
No harm, no foul in my opinion but a little confusing.
I have a couple that say more than the ship name and hull number. For instance I have one for the USS Pampanito because I contribute to the cause. It says "USS Pampanito Breakfast Club" since that event where they hooked me, a breakfast on board, was where I was given the hat.
A solution might be to add the phrase "Dad's Ship" or some such.
Just my 2 cents.