Tackled source control and issue tracking first since they were low hanging fruit. Next came basic (manual) backups. My background is not in putting together general office-wide IT solutions, so I did not feel qualified to recommend any specific package. After that, it was on to a sane build and deployment process that involved a dedicated build machine and included QA. After a long time I could start safely making code changes to clean the actual codebase up.

I think you'll see this a lot in manufacturing companies who sell physical hardware. The embedded software is a critical part of the product, yet it's not treated with much more care or seriousness than anything else on the BOM. Night and day difference between working at a company where software _is the product_ vs where software is just one of the many components that goes inside the product.