I would have loved to go through the setup again and see where exactly I got confused. For this purpose I uninstalled Plex and re-installed it. Sadly for the last 20 minutes it only shows:

<Response code="503" title="Maintenance" status="Plex Media Server is currently running startup maintenance tasks."/>