News for hackers. Stuff that matters.

A comment on Notes on Writing Web Scrapers. Parent comment

Re:Notes on Writing Web Scrapers by marginalia_nu on Friday December 03, 2021 @12:17PM (#29432452)
There is absolutely a performance benefit to reducing your row sizes. It both reduces the amount of disk I/O and the amount of CPU cache misses and in many cases also increases the amount of data that can be kept in RAM.

You can map meaning onto the column in your code, as most languages have enums that are free in terms of performance. It does not make sense to burden the storage layer with this, as it lacks this feature.

[ Parent | on HN ]
Re:Notes on Writing Web Scrapers by cushychicken on Friday December 03, 2021 @10:53PM (#29438553)
You can map meaning onto the column in your code, as most languages have enums that are free in terms of performance. It does not make sense to burden the storage layer with this, as it lacks this feature.

Was just looking at how to do this with an enum today! Read my mind. :)

[ Parent | on HN ]
Re:Notes on Writing Web Scrapers by yakshaving_jgt on Friday December 03, 2021 @12:24PM (#29432529)
The performance benefit is negligible at the scale the author of the article is operating at. You already alluded to this point being context-dependent earlier when you said:

> if you have a sufficiently large database

Roughly 360,000 rows per year is not sufficiently large. It's tiny.

[ Parent | on HN ]