While I have not had lots of difficulties/bad results by not priming, I can say I have had zero issues when I do prime. Now it's the standard for me, priming before paint is routine.
A. When the primer dries it will reveal areas that need further work, like filling or sanding. Better to discover that with a prime coat, rather than problems under paint.
B. Primers like Alclad and Stynylrez offer a very durable first layer and adhere well to the original surface. That can help the paint process stick positively to the surface, and help prevent lifting when tape masking is removed.
C. Primers sand well when preparing for finish paint coats, paint not so much. That makes any corrections you may have to make much easier and with better results.
Seems no matter how closely I looked when ready for paint, I missed some things and had to make corrections. With primer the remaining flaws are much easier to see and address, before final paint. The plus is that the primer is much better to work with for corrections, finish paint coats are not as good.
Priming highly recommended by me.
Patrick