Glibc Buffer Overflow in Iconv

Re:Glibc Buffer Overflow in Iconv by masfuerte on Sunday April 21, 2024 @07:18AM (#40104758)

Rust still has its warts. When dealing with an archive, say, you can find yourself needing to deal with Windows strings on Unix, or vice versa, but Rust only provides the string type for the platform you are running on. It could do with UnixString and WindowsString in addition to OsString.

MFC had a similar problem for years. It had a CString class which was ANSI or Unicode, depending on how your compiled your app, but moderately often you needed the other one, so it should have had a CStringA and CStringW too, with nice conversions between them.

[ Parent | on HN ]

Re:Glibc Buffer Overflow in Iconv by tialaramex on Sunday April 21, 2024 @07:27AM (#40104800)

OsString isn't "the native string type", it's a container for whatever was convenient for Rust's internals on that OS, there are convenience functions so it probably feels like it's "UnixString" or "WindowsString" but it's neither.

In most of these file format cases what you've got is &[u8] or &[u16] and maybe it's the NonZero variant instead, so I think it's fine to be explicit that's what is going on and maybe in the process remind you to check - is this data UTF16LE? UTF16 with a BOM? UCS2 with a nod and a wink? Just arbitrary 16-bit integers and good luck?

But like I said, I favoured "picking nits" long before I learned Rust, so mileage may vary.

[ Parent | on HN ]

Re:Glibc Buffer Overflow in Iconv by pornel on Sunday April 21, 2024 @10:08AM (#40105911)

IMHO Rust got OSString wrong – it indirectly promised that UTF-8 can always be cast to it without any copying or allocations, so on Windows it has to use WTF-8 rather than UCS2/UTF-16. Instead of being OS’s preferred string type, it’s merely a trick to preserve unpaired surrogates.

[ Parent | on HN ]

Re:Glibc Buffer Overflow in Iconv by tialaramex on Sunday April 21, 2024 @11:55AM (#40106716)

Huh. Where is the "indirect promise" ? Is there a (conventionally hinted as "free", so it's not good if there's one which isn't very cheap) as function? Like as_os_string or something?

[ Parent | on HN ]

Re:Glibc Buffer Overflow in Iconv by LegionMammal978 on Sunday April 21, 2024 @01:44PM (#40107613)

It's because str implements AsRef<OsStr> [0]. The function signature promises that whenever you have a borrowed &'a str, the standard library can give you a borrowed &'a OsStr with the same data.

Since references can't have destructors (they don't own the data like an OsString does), it means that the standard library can't give you a newly-allocated string without leaking it. Since obviously it isn't going to do that, the &OsStr must instead just act as a view into the underlying &str. And the conversion can't enforce any extra restrictions on the input string without breaking backward compatibility.

The overall effect is that whatever format OsStr uses, it has to be a superset of UTF-8.

[0] https://doc.rust-lang.org/1.77.2/std/ffi/struct.OsStr.html#i...

[ Parent | on HN ]

Re:Glibc Buffer Overflow in Iconv by tialaramex on Sunday April 21, 2024 @03:27PM (#40108468)

Ah, an AsRef<OsStr>, yep that would do it. Thanks.

[ Parent | on HN ]

Re:Glibc Buffer Overflow in Iconv by jcranmer on Sunday April 21, 2024 @02:26PM (#40107984)

I wouldn't say it's necessarily wrong; it may be (accidental) foresight. Windows has added a UTF-8 code page, which means you can get as near enough as makes no difference full support with the A functions instead of the W functions.

That said, even now in 2024, it's not clear how much of a bet Windows is making on UTF-8 versus UTF-16.

[ Parent | on HN ]

Re:Glibc Buffer Overflow in Iconv by pornel on Sunday April 21, 2024 @02:53PM (#40108192)

I would be surprised if the UTF-8 support in Windows was anything deeper than the A functions creating a W string and calling W functions, which is what Rust is doing already.

[ Parent | on HN ]

Re:Glibc Buffer Overflow in Iconv by tialaramex on Sunday April 21, 2024 @08:26PM (#40110444)

The question is whether that's all they ever do, or whether gradually some parts of Windows begin to decide to be UTF-8 first and their W functions now translate the opposite way. This must be tempting for some networking stuff, where you're otherwise taking a perf hit compared to Linux.

[ Parent | on HN ]

Re:Glibc Buffer Overflow in Iconv by pjmlp on Sunday April 21, 2024 @10:32AM (#40106050)

The first time I had to deal with this was on Turbo Pascal for Windows, where we got C strings alongside Pascal ones.

Then when 32 bit came to be we had the other variations on top.

Got to love leaky abstractions.

[ Parent | on HN ]