Greetings,
As I build scale dios for HO RR and 1/35th armor, etc, I have found many methods for both ends of the hobby, model RR water and this, Acrylic caulk and clear glossy Polycrylic coating. Here is the method:
1.) Using several tubes of acrylic caulk which can be clear or tan, apply the caulk onto the base surface which can be styrofoam as you have used, apply with a putty knife and as the caulk starts to set up (harden) make the waves stand out with a toothpick or small paintbrush. Also you can feature the rotor wash by making circles in the caulk at this stage and making each circle smaller in diameter than the next in toward the Helo.
2.) Next, let the caulk set up overnight and then paint with acrylic colors appropriate to the sea conditions, the wave tips and froth can be painted white along with the greenish grey of water in the case of your diorama. The reasons for using acyrlic paints are nurmerous as I would point out many advantages to the use, the main one is water solubility and ease of use. These paints come in glossy as well as flat and match any color for a military or RR use, as I have always matched completely for the closest details possible and the cost is a fraction of the bottled- matched paints. Sold as craft paints in many outlets they are numerous and airbrush- usable as well. ( More on that in some of my posts on this forum) Airbrush the paints in this diorama and if you cannot, handpainting with a little water mixed in will work just as well.
3.) After the paint sceme has completely dried, apply thin coats of the acrylic gloss coating, this will take several applications and look very realistic for any river, swamp and ocean scene. In fact one can use the gloss coat for rivers, swamps and beach landing zones on its own. It dries in about a day and by applying additional coats will look extremely realistic when used over a base color for river bottoms as an example. The gloss acrylic coat will run somewhat but if you dam it up with a framework around your diorama, which I would suggest as it will build up quite nicely to look finished as a photograph would.
4.) Placing any models in the water should be done as the caulk is setting up and to get a realistic affect be sure to have waves splashing up onto the boat, in your dio. Having been to sea on several ships, as that's what the Navy is all about, I can attest to the fact that the waves can be unforgiving so the diorama should make someone feel the sea-illness effect! (politely put)
Sorry I have not seen your post before today, but you are on the right track and it does look great, just pitch the cotton and save it for other applications.
Keep up the great work and let us all know about the results,
johncpo