News for hackers. Stuff that matters.
Re:Colly: Fast and Elegant Scraping Framework for Golang by feelin_googley on Friday October 06, 2017 @02:36PM (#15419208)
Improve the CSV:

   # add to end
   \
   |exec tr '\15' '\12' \
   |exec awk 'NR==1{a=$0}NR==3{b=$0}NR==5{print}NR>=6{print "\42"a"\42,"b","$0}' 

   # update CSV header line 
     \"sitetitle\",\"url\",\"pageID\",\"pagetitle1\",\"pagetitle2\",\"jsonurl\"/;};
Probably would be better to change "jsonurl" to "jsonfile" and just save the filename. The location (URL) of this file may be in several locations or it may change over time.

There are no comments on this comment yet.