Show HN: Herodotus – An IRC bot that logs a channel's activity to JSON

Re:Show HN: Herodotus – An IRC bot that logs a channel's act... by koolba on Saturday February 20, 2016 @01:14PM (#11141152)

    var array = [];
    
    client.addListener('message', function (from, to, text) {
      var time = Math.floor(new Date() / 1000);
      var obj = { nick: from, message: text };
      var item = '\"' + time + '\"\:' + JSON.stringify(obj);
    
      array.push(item);
    
      fs.writeFile(file, '{' + array + '}\n', function() {
        // console.log("updated");
      });
    });

This will buffer all the messages in memory (in the variable "array") and rewrite the file on every message. Besides the lack of atomicity mentioned in another comment, that's also really inefficient and will eventually run out of memory. Plus 1000 incoming messages will lead to 1000 sequential rewrites of the same file!

[ Parent | on HN ]

Re:Show HN: Herodotus – An IRC bot that logs a channel's act... by IanCal on Saturday February 20, 2016 @02:08PM (#11141404)

A useful file format to deal with this kind of problem is JSONL, which is just a list of newline separated json blobs. You can then simply append things to the file rather than having to write the whole thing each time.

[ Parent | on HN ]