IME, acrylics work best when layed on top of a coat of primer, always had probs without a primer.
Said that I often apply a top-coat between layers too so that I can remove mistakes without touching the underlying paint layers. Needed for some of the kits I do.
If only the paint is lifting off and not the primer than i would say that your primer is either incompatible with your paints or that you are getting finger oil residue onto the primer.
Most parts/sub-asemblies that I paint I attach to wooden dowels so that I can handle the parts without having to touch them.
HTH.