You're quite right in thinking this is an important topic. The dozens - even hundreds - of rope coils around the weather decks are vital to the look of a sailing ship.
Different modelers find that different techniques work for them. I've always had good results from a slightly modified version of the direct approach. I leave the line plenty long, then, when it's time to belay it, put a dollup of Elmer's white glue at the base of the belaying pin and simply drag the line through the glue as I form the coil. Two pairs of long tweezers are a big help in controlling the size of the coil. Take the bitter end through the middle of the coil; that will help keep it from unwinding. In the process of forming the coil the thread gets sufficiently permeated with the glue not only to keep it coiled up but to make it droop realistically over the pin. Often I'll come back with the tweezers ten or fifteen minutes later, when the glue is almost but not quite dry, and sort of teaze the coil into place.
When the glue dries it's virtually invisible. If it looks a little shiny, that can be taken care of by a drop of solvent based (not water based, lest the glue soften) clear flat. Some kinds, and some colors, of thread darken more than others in the process, but with few exceptions the original color comes back when the glue dries.
Another benefit to this technique is that it can be undone easily. If you ever have to re-rig the line (should it get busted, become slack, or whatever), just touch the glue spot with a small, water-dampened brush and the glue will come loose.
This seems to be another of those ship modeling operations in which there's a rather lengthy "relearning curve." The first couple of coils take me ten minutes and about a dozen cuss words apiece, but by the end of the evening the thread recognizes who's boss and behaves itself. Folks who are new to ship modeling often make the mistake of thinking something is "too difficult" because it's so hard the first time they try it. Frequently it starts seeming easy by attempt number four or five.
A similar trick works for coils of rope that are just lying on the deck - if the deck's made of wood. (On a plastic deck I'd do something different.) Stores that cater to butterfly collectors sell extremely fine, extremely sharp steel pins called "insect pins." I shove a couple of insect pins into the deck where I want the coil, lay down a blob of Elmer's around them, wind the thread around the pins, take the bitter end under the last full turn, and press the coil down into the glue. When the glue dries I yank out the pins. The wood grain usually closes up around the pin holes; if it doesn't I give them a tiny bit of water to make it swell.
Good luck. Sounds like you're into the home stretch of a terrific model.