yep, you'd use something like nginx or apache traffic server and set that up to serve SSL.
One special case though, if you have multiple servers that serve your content load balanced, and if these servers are in different colocations, then you probably need to run any sync between them over SSL. Even if you do control the link between the two boxes, there's that off chance that your link goes down and the IP layer automatically routes traffic through a different set of routers.
Edit: It looks like the best way might be to do SSL between the client and nginx on my side acting as a reverse proxy, and then non-SSL internally on my side? Not sure how that setup compares to Varnish in general, but it's probably fine for my purposes.