There are quite a few books about ship modeling, but I know of only four that really concentrate on its history. The oldest is called The World of Model Ships and Boats, by Guy Williams. It appeared, if I remember right (as I frequently don't these days), in the mid-seventies. It was a pretty nice picture book with generally reliable text, but awfully sketchy in its coverage.
Norman Boyd, whom Mr. Blevins mentioned above, wrote a relatively small work on the subject about twenty years ago. I don't recall the title, but I can't recommend it. It was full of mistakes and invalid generalizations; much of it was devoted to promoting the work of a handful of professional ship modelers whom the author happened to know. (All of them were good modelers - but there were plenty of others.)
Mr. Boyd's second work (the one Mr. Blevins cited) is far superior in terms of content and accuracy. It's essentially a "coffee-table book," with all that implies: fine illustrations, beautiful reproduction, large format, and pretty superficial text.
The book that looks at first glance like the definitive work on the subject is Ship Models: Their Purpose and Development From 1650 To the Present, by Brian Lavery and Simon Stephens. The authors both work at the National Maritime Museum in Greenwich, and know what they're talking about. My big reservation about the book (a reservation that I've heard several other Americans express) is that it's intensely Anglo-centric. The vast majority of the text and illustrations deals with British subjects - mainly from the NMM. An appendix does list "major" model collections in other countries. (As for those in the U.S., the authors acknowledge the museum at the "United States Naval College" in Annapolis. AARRRGGGHHH!!!) It's best considered an excellent catalog of the model collection in the NMM - arguably the best, and almost certainly the biggest, model collection in the world. But the world is still waiting for a really authoritative general history of the ship model.
As it happens we've taken up the topics of Viking ships and ancient galleys recently in this forum. The galley discussion is under the title "Zvezda Greek Trireme," and the Viking ship one under "Building a wooden ship." Bottom line (including a generous dose of personal opinion, with which other forum participants may well disagree): there's one good, reasonably accurate Viking ship kit. It was made by Revell in the 1970s, and is no longer generally available. As for Greek and Roman vessels - this is a topic I don't know much about. Several plastic kits purporting to represent ancient galleys are on the market, but I have reservations about them. Scholarly thinking about what those ships looked like has undergone some major changes in the past two or three decades, largely due to the construction of a full-size, thoroughly researched replica under the auspices of the Hellenic Navy. I have the impression that most (probably all) of the model kits representing such ships were designed before that project - but I may be mistaken. The photos of the replica (its name, I believe, is Olympius) sure don't look much like any of the kits I've seen.
Hope this helps. It's a fascinating subject, and there's room for a great deal more to be written about it.