You fail to appreciate that there is quite a bit of skills development and problem solving training. The first two years are more memorization heavy, as you have to learn the language and basics. Years 3 and 4 are more focused on navigating the health care system, interacting with patients, performing technical skills, physical examination, use of imaging, diagnosis, etc. Memorization is only a fraction of what you do to become a competent physician.