When I was ten, I had no idea about any of this stuff and yet when somebody showed me how to do arithmetic, variable assignments, comparisons and goto in QBasic (pretty much equivalent of Babbage's machine) I was able to write a simple drawing program and tic-tac-toe which checked whether one of the players won.
Add some IO and I would write a program which reads series of transactions and computes your bank account balance. Tell me what a matrix is and I would implement LAPACK for you.
Without that they would have been at a disadvantage. I don't see what is so hard about this concept to you.
Regardless, the simple historic fact remains that Babbage and Ada were both unable to build the machine, and unable to verify their suspicion that the machine was special, and unable to effectively communicate to their peers this suspicion. The prerequisite math for all three of these tasks did not yet exist.
They could have pulled some impressive stuff off I am sure, but it would have all been intuition and stabbing in the dark. The tools necessary to structurally reason about algorithms had not yet been created, nor even the tools necessary to create those tools...
Ada had the notion that the Analytical Engine was something special, something more than just a calculator... but that was conjecture based on genius insight. To actually discuss that idea in a rigorous manner would require several more decades of advances in mathematics.
Could they have programmed? Yes, obviously. That's not even hypothetical, since they did. Would they have been at a distinct disadvantage? Without anything reasonably resembling modern mathematics, absolutely.