Avila Beach is where you want to go. Barbara by the sea, and the restaurant out at the end of the pier. Theres a reason its called SLO town, but... there are the Poly Dollies.
You will be out at Camp Cuesta.
If you have time, which it sounds like you don't , the reason for being of SLO is the Southern Pacific Railroad, about which I have far more invested in time, modeling money and interest than a/c-Gee you could tell.
The SP comes south from my way, goes 200 miles south total: along the Peninsula, then thru the Salinas valley over the Cuesta grade after an 100 mile march from the sea, south and up 3000 feet. I drops that same altitude in about 25 miles, maybe 15 miles on the map, into SLO by a series of fairly tortuous horseshoe loops, trestles and steep grades, plus tunnels. If you can, and its not far from where you are, get over to Stenner Creek road, which is off of Hwy 1 not far at all from where you are, there's a trestle that was brought in from the east coast in pieces, and completed the Los Angeles to San Francisco Coast Division. Scramble up there and watch big freights come down.
In the day, they were a mile long, longest in the world sometimes, with a 3 to 7 engine front locomotive, 5 in the middle and 5 at the rear. It was a big deal for the crews to keep the train from bowstringing over open space. If you go downtown, there is an historic art deco depot, a small amount of sidings, an important historic watertower. Amtrak Coast Starlights come thru north and south bound around 1-2 pm. Theres a siding up on the hill called Goldtree where they used to pass in the days when it had to be on schedule, but now who knows. Good railfanning.