So if the Ubuntu security team does its job properly then you shouldn't have a reason to worry.
(However given the number of security vulns these days it's often challenging for LTS distributions to backport all security fixes. There are already breakdowns of the LTS concept, e.g. sticking with latest upstream versions for some packages like chromium where backporting is not realistic.)
You can see the status of all known CVEs and which .deb updates patches them here: https://security-tracker.debian.org/tracker/source-package/c...
The best way to stay on top of things is to subscribe to your distro's security advisory mailing list, for example https://lists.debian.org/debian-security-announce/
You are trusting Ubuntu's judgment that the remaining 12 CVEs aren't that important. Ubuntu's security team is pretty good, but I don't think there is any distro that is extremely good. In part this is because a distro is on the hook for compatibility of all the software they ship, and expected to prioritize compatibility over security. Anything other than targeted security fixes can cause regressions.
Where is the fault in this chain? How can I as a maintainer of a few servers be sure my servers are secure without manually patching every package?
[1] http://packages.ubuntu.com/xenial/libcurl3 [2] http://oss-security.openwall.org/wiki/mailing-lists/distros
EDIT: changed "12 CVEs" to "at least 15 CVEs". The changelog don't have CVE-numbers in the title for all of them.