vrnm
Nowadays, performance depends almost exclusively on memory bandwidth usage and spatial and temporal memory locality, i.e. the only thing that matters is how much memory you are moving and how you are traversing it (the number of additions/divisions/branches doesn't matter that much).
In other words, your machine prefers data structures with good memory locality which determine which algorithms you can use and which problems are fast to solve. That is, in real life (as opposed to white board computer science) the order is inverted.
The question is: are you constrained at all by the machine? Because if you can solve the problems you want to solve without knowing about memory, then you obviously don't need to know about memory. On the other hand, if you are constrained by computing time, power consumption and so on, then you either learn about memory or can't solve your problem.
Again in other words, if you can get the job done in ruby/python/haskell/java/lisp... then do it! You'll do it faster, nicer, probably more maintainable... However, if you cannot get it done there, you'll need to use C++/C/Fortran/D...