Re:The sorry state of Java deserialization
by antonhag on Sunday September 22, 2024 @02:31PM
(#41618960)
Aren't the versions that take 10% of the time only reading each city name once, and then doing an array lookup rather than a hashmap lookup?
Re:The sorry state of Java deserialization
by marginalia_nu on Sunday September 22, 2024 @02:41PM
(#41619037)
Nope, see for example "Custom 1":
var buffer = ByteBuffer.allocate(4096);
try (var fc = (FileChannel) Files.newByteChannel(tempFile,
StandardOpenOption.READ))
{
buffer.flip();
for (int i = 0; i < records; i++) {
if (buffer.remaining() < 32) {
buffer.compact();
fc.read(buffer);
buffer.flip();
}
int len = buffer.get();
byte[] cityBytes = new byte[len];
buffer.get(cityBytes);
String city = new String(cityBytes);
int temperature = buffer.getShort();
stats.computeIfAbsent(city, k -> new ResultsObserver())
.observe(temperature / 100.);
}
}
Re:The sorry state of Java deserialization
by antonhag on Sunday September 22, 2024 @03:30PM
(#41619349)
My bad - I got confused as the original DIS+BIS took ~60s on my machine. I reproducing the Custom 1 implementation locally (before seeing your repo) and it took ~48s on the same machine. JFR (which you honestly can trust most of the time) says that the HashMap lookup now is ~50% of the time and the String constructor call being ~35%.
It doesn't seem like it can be true that 90% of the time is spent in string parsing and hash lookups if the same operation takes 10% of the time when reading from a filechannel and bytebuffer.