A couple of issues here. 1) Most PE frets have some kind of releasing or finishing agent on them, so they need to be cleaned. That also helps to give them "tooth" so the paint sticks well. I always give mine a bath in plain old white vinegar (5% acidity) to remove anything on the frets and to mildly etch them, then a good rinsing in cool water and thorough air drying.
2) CA glues, as many of you have discovered, are great for holding things together as long as no lateral strain is put on the parts after joining them (which makes it great for 1/700 ship railings and such). You MAY have better luck with the gel-type CA glues, which is what I use since they also have slower drying times, and they tend to fill in any irregularities between the two bonding surfaces to give a tighter overall bond. I use Pacer Sheet Zap, PT-30, in the blue bottle. It's actually intended for the R/C crowd but I need that extra few seconds of working time. I've never had problems with parts popping off or seperating after using this stuff, quite the contrary ... there are times when I wish I could get them to separate after experiencing one of those " Oh !" moments we all have right after we have glued the wrong parts together.