Stagit – A static Git page generator

Re:Stagit – A static Git page generator by wrigby on Tuesday August 01, 2017 @10:38PM (#14907503)

Looks pretty nice! The readme says it's not useful for repos with >2000 commits, but I wonder if we could hack together some sort of cache for it to facilitate delta updates?

[ Parent | on HN ]

Re:Stagit – A static Git page generator by sdesol on Wednesday August 02, 2017 @05:13AM (#14909066)

> we could hack together some sort of cache for it to facilitate delta updates?

Crunching diffs at scale is actually an extremely I/O intensive operation and I can say this from experience, since my product is doing just that. You can see an example of what I mean at

https://public.gitsense.com/insight/github?r=Microsoft/vscod...

If you scroll down, you can see how the indexed information is used. The red, green, and blue bars, denotes how many lines were deleted, added and changed. The red, green and blue bars in the second row, shows how many lines were deleted, added and changed, which were non blank/comment lines.

If you want to crunch diffs at scale, you really have to throw CPU cores, RAM and multiple SSDs at the problem. The more CPU cores, translates to more diffs that can be processed in parallel. The more RAM, translates to better Kernel page caching. And multiple SSDs in RAID 0 configuration, translates to better read and write speeds.

[ Parent | on HN ]

Re:Stagit – A static Git page generator by mrmondo on Wednesday August 02, 2017 @06:18AM (#14909364)

Do you think it’s possible to calculate the relationship between the depth of the git data and the number of I/O operations?

This is purely out of interest, our storage performs at around 2-4M random 4K read IOP/s per node and I was wondering (just for fun) how it might look thrown a good I/O crunch task like this.

[ Parent | on HN ]

Re:Stagit – A static Git page generator by sdesol on Wednesday August 02, 2017 @06:26PM (#14915249)

> Do you think it’s possible to calculate the relationship between the depth of the git data and the number of I/O operations

It's actually pretty straight forward since you know exactly what operations has to be done. With very good I/O, you may find the CPU will become your bottleneck.

[ Parent | on HN ]

Re:Stagit – A static Git page generator by mrmondo on Thursday August 03, 2017 @01:10AM (#14917090)

Yeah then it becomes even more interesting to me ;) especially when it comes to NUMA etc...

[ Parent | on HN ]