In order to go higher, you must enclose a larger volume of lifting gas. Eventually, the weight of your envelope will exceed the amount of lift you're able to generate with the enclosed volume.
The Japanese currently hold the world altitude record (among all professional and amateur balloons), IIRC. Their balloon was 3.4 micrometers thick, 60,000 m^3 of helium. They reached 53 km.
Out of curiosity, why use helium instead of hydrogen? It's not like human lives are at risk in these experiments.
I imagine you'd get more lift with hydrogen.
People do use hydrogen, but most balloons still use helium because it does provide a slight safety advantage when launching. (It's also sometimes an insurance requirement.)
I expect to see an increase in the number of hydrogen-filled balloons as helium prices rise in the next few years.
(PS: I am aware the Kármán line actually has to do with aerodynamic lift (and therefore may not really be applicable to lighter than aircraft). It just chose it because it's the commonly accepted "edge of space")