The Coast Guard has long been responsible for port security. TSA does administer TWIC, the Transportation Worker Identification Credential program, which is a biometric identification system go access to secure port facilities.

Ports did indeed have very little security before 2001 (compared to now). See ISPS code: https://en.wikipedia.org/wiki/International_Ship_and_Port_Fa...

Yeah, it was actually a lot of fun. I worked in a port-related area and often we would just cruise around the port and look at the ships. If it looked cool we would yell loudly and ask if we could come aboard. The seamen were usually thrilled to show us around their massive ships and would often invite us to a barbecue. With the introduction of the ISPS all of that was over in an instant.