In the Default App panel, if I go into Photo Viewer and click choose a default, I will only see Paint, Pain 3D and "Look for an app in the Store". No choice for a dialog with a complete list of app. Again you are probably referring to an earlier version of Windows 10.
I'm on build 15063, and it is there.
> In the Default App panel, if I go into Photo Viewer and click choose a default, I will only see Paint, Pain 3D and "Look for an app in the Store".
It shows the apps that are registered as photo viewers. The fact that IrfanView doesn't show up there means that it doesn't do that. For me, that dialog lists Paint.NET and other non-Store apps. Again, this is on build 15063.
Indeed, even per your description, it lists at least one non-Store app - that being Paint. So it's clearly not locked to the Store.
Now, I don't know exactly what it takes for the app to be considered a "photo viewer", but it seems that you need to have it registered (not as a default, but just generally as a handler) for .jpg and .png? I googled a little bit, and the discussion here indicates that after people did that association in IrfanView settings, it did show up in the app list for Photo Viewer for them:
https://www.tenforums.com/software-apps/35248-cant-add-defau...
I don't know what you consider "the primary menu", but Settings | App | Defaults lets you choose any app that's registered for the purpose - e.g. for me it shows Chrome for web browser and mail, and Paint.NET for photo viewer.
> and you cannot change the default program from a contextual menu on the file anymore
Sure you can. If you click "Open With" in Explorer, you get the list of several most used apps registered for this file type, both Store and non-Store. If you then click "Choose another app", you get a dialog with the complete list of apps (again, both Store and non-Store), and at the bottom it has a checkbox to always open this type with the selected app