Re:Why I Use Nim instead of Python for Data Processing
by cb321 on Saturday September 25, 2021 @02:26PM
(#28655117)
@brabel - The Nim compiler actually builds a relatively large `system` package every time. (They are also working on speeding up compiles.) So, compile time does not scale as badly as you think. E.g., you might have to 50..100x the "user level" source code to double the time.
Also, @benjamin-lee this version of the Nim program is a bit lower level, but probably much faster:
import memfiles as mf
var gc = 0
var total = 0
var f = mf.open("orthocoronavirinae.fasta")
for line in memSlices(f):
let n = line.size
let cs = cast[cstring](line.data)
if n > 0 and cs[0] == '>': # ignore comment lines
continue
for i in 0 ..< n:
let letter = cs[i]
if letter == 'C' or letter == 'G':
gc += 1
total += 1
echo(gc.float / total.float)
mf.close(f) # not really needed; process about to end
Compile with -d:danger and so on, of course. { On a small 30kB test file I got about a 1.7x speed-up over that of the blog post. I also could not find the 150 MB file. Multiplying up the tiny 30 KB file like @brabel, I got only a 1.25x speed-up down to 0.5 seconds. So, might not be worth the low levelness, but a real file might tilt more towards the 1.7x end. }
Also, @benjamin-lee this version of the Nim program is a bit lower level, but probably much faster:
Compile with -d:danger and so on, of course. { On a small 30kB test file I got about a 1.7x speed-up over that of the blog post. I also could not find the 150 MB file. Multiplying up the tiny 30 KB file like @brabel, I got only a 1.25x speed-up down to 0.5 seconds. So, might not be worth the low levelness, but a real file might tilt more towards the 1.7x end. }