I have to disagree a little with CapnMac82. A square-rigged sail when the ship is running before the wind (i.e., with the wind blowing from directly astern) isn't an airfoil. In fact I don't think it functions as an airfoil under any circumstances. It's just a big, more-or-less flat, more-or-less vertical surface with the wind blowing on it. (It's worth noting that, as technology advanced, the designers worked hard at stretching the square sails tighter, so they couldn't be blown into anything like an airfoil configuration. In the fifteenth and sixteenth centuries, sails were deliberately cut and sewn so they'd be baggy, on the (false) theory that they'd "gather up the wind." Photos of the last twentieth-century windjammers show that the square sails were set so they were almost perfectly flat.)
The airfoil phenomenon comes into effect in a fore-and-aft-rigged vessel (such as a modern yacht), which can utilize that effect to sail almost into the wind. The wind blowing across the sail creates an airfoil, which, if the yacht is well-designed, can make the hull move forward. But letting the wind blow across a square sail accomplishes almost nothing.
In a square-rigged ship, the only limiting factor in how far the yards can swing is the rigging below them. Investigator, you've got enough rigging on your Batavia to work it out. Hold the fore- or mainyard in its position and swing it around. It will swing a good bit before it bumps into the first shroud. (Ships often slacked off their shrouds so the yards could swing farther.) It wasn't at all unusual, when the ship was working to windward, for the yards to be swung 30 degrees or more from the centerline. (In such a case, if the wind was really blowing, the ship would be heeling over with the water frequently breaking on her lee rail.)
Here's a famous painting by the Dutch master Willem van de Velde the Younger: http://www.christies.com/lotfinderimages/d52594/d5259430l.jpg .
This ship is beating to windward in a storm. The wind is blowing from a few points aft of the port beam. The topsails are in their lowered positions, with their sails furled, and squared to the ship's centerline. The topgallants have been struck and sent down to the deck. The fore and main yards are swung around a long way from their "normal" positions squared to the keel, and they've been lowered several feet in an attempt to keep the center of the wind's force relatively low. (This painting dates from 1678. A century later the topsails would probably be taken in after the courses.) The lateen-rigged mizzen is set so the wind is blowing directly into it.
The flags are blowing directly away from the wind, or nearly so. (The fact that the ship is moving forward, and the flags presumably are flapping, lets the artist show the designs of them more clearly.) He's also painted them as though they're almost transparent. Big flags were really big in those days, and were generally made of what we'd call a very coarse, heavy gauze. If they'd been made of canvas, they would have messed up the ship's navigation.
It was perfectly possible for the same wind to drive two square-rigged ships in opposite directions. But a square-rigged ship can't sail anywhere near directly into the wind. A fore-and-aft rig ship can come a lot closer to doing that.