What I learned from summarised nuggets of Winnicott and Bowlby [0,1] is that time spent as they grow is an investment in relationship building for the future. It doesn't have to be that much for fathers, but it needs to be consistent and regular. By the time you want to go skateboarding and bike riding with them, when you have more spare time and money and mum gives them more independence, there needs to be a solid foundational attachment bond. It may not seem like playing with a 1 or 2 year old is "useful or productive", but your mere presence is foundation building.

(dad of 7/8 yo)

[0] https://en.wikipedia.org/wiki/Donald_Winnicott [1] https://en.wikipedia.org/wiki/John_Bowlby