Assuming that you still want offroad capability, you basically just described a cyclocross bike. And going on an ultralight tour on a modified CX bike sounds super fun!