To your point, the first college football game in the US was, simultaneously, the first college soccer game because it took place before the split between soccer and rugby, and the rules were those drafted by the Football Association in England. The rugby rules were adopted by Canadian colleges, who went on to play against American colleges, who adopted the rugby rules and added the initial changes that created American football.