I'm guessing it's because of memory access.

The 6510 processor can accesses memory only on every other clock cycle. So half of all clock cycles can be used by the VIC to get display data out of memory, without fearing contention with the CPU. But this means that pixels must be output at the rhythm of the processor's clock. If you wanted to make the pixels 10% wider, you'd have to slow down the clock driving the VIC, but then it wouldn't be able to interleave memory access with the processor anymore.

Not entirely sure this is the true reason since my knowledge is patchy, but it's my guess.

https://en.wikipedia.org/wiki/Overscan: “Overscan is a behaviour in certain television sets, in which part of the input picture is shown outside of the visible bounds of the screen. It exists because cathode-ray tube (CRT) television sets from the 1930s through to the early 2000s were highly variable in how the video image was positioned within the borders of the screen. It then became common practice to have video signals with black edges around the picture, which the television was meant to discard in this way.”

Basically, you couldn’t trust two televisions to show a picture at the same size, or even a single television to show an image at constant size if it heated up.

Computer users would object if their screen chopped of some letters at the edge of their screens, so they had to accept the reverse: wasted space around the screen’s contents.

You have borders because you want the image to be inside of the title safe area and then the layout is constrained because the chip is designed to show a 40x25 character area of characters 8 pixels wide, and outputting 8 pixels takes exactly as much time as one clockcycle of the cpu, they execute in lockstep.

And of course you can’t just zoom in a little because a character is 8x8 pixels and the video signal has a fixed number of lines on the screen so 25 characters of 8 lines tall is just a fixed part of the screen (which is 312 lines tall on PAL).

In hindsight it is a bit silly that there is no easy way to display sprites over the borders though, to do that you have to wait until the video chip is displaying the last character and then configure the screen a bit less wide/tall so it oversteps the end and never stops outputting.

It is not a matter of quantity of memory. (One screen worth of content is just 1000 bytes - 40x25 chars)

Do not forget that the physical pixels sit on real rasterbars on a cathodic ray tube machine.

If you made the centered content bigger, first of all the effect would simply be ugly - stretched;

if you expanded the content per bar (over the 40ch, 320px) but keep the same ratio, you'd have less warranties that the set would make it visible;

similarly for using more rasterbars than 200 (taking into account that compatibility with NTSC basically imposes a limit at 240): will every set display all the lines or cause frustrated users?

(And of course, given the shape of the screen, if you filled it up with a rectangular matrix you would surely lose displaying the content at the corners.)

As others said in a different way: there was a safe area.