News for hackers. Stuff that matters.

Ask HN: What should I do with 4 GBs of form spam?

I've been collecting form spam for around two years, during which I've collected around 6 GB or 3.7 million submissions, on a website called http://spambotlove.com/

At first I wanted to use the data to generate images or to display statistics, but I don't really have the time or will because I'm very busy with other stuff.

What should I do with it? I guess I could just delete it..?

Would it be of any value to anyone if I would make a simple API to query the DB?

Any ideas?

( 12 comments | 14 points | on HN )
Re:Ask HN: What should I do with 4 GBs of form spam? by pedrokost on Tuesday February 15, 2011 @12:37PM (#2222231)
Instead of deleting it, you may want to share it in a torrent file. Maybe someone could make some better use of it.
Re:Ask HN: What should I do with 4 GBs of form spam? by nyellin on Tuesday February 15, 2011 @12:41PM (#2222250)
I whole-heartedly agree. You don't need to do anything yourself, except for releasing the data. Please don't, under any circumstances, delete the dataset.

Edit: Regarding your original question, I think that a torrent would be much more useful than an API.

Re:Ask HN: What should I do with 4 GBs of form spam? by kakaylor on Tuesday February 15, 2011 @05:21PM (#2223640)
I agree, I would definitely be interested in a torrent of the data.
Re:Ask HN: What should I do with 4 GBs of form spam? by kephra on Tuesday February 15, 2011 @12:33PM (#2222212)
Install Weka (requires Java), do a string to wordvector conversion of your data, and compare SPAM and HAM of your blog. Play around with machine learning. /join #machinelearning @ freenode <- some of us are using Weka
Re:Ask HN: What should I do with 4 GBs of form spam? by khandelwal on Tuesday February 15, 2011 @12:52PM (#2222306)
If you don't like Java, you can use Python with the Natural Language Toolkit (NLTK) and do the same thing. Please do provide the dataset for download.
Re:Ask HN: What should I do with 4 GBs of form spam? by arnorhs on Tuesday February 15, 2011 @12:41PM (#2222252)
That's interesting. I've never heard of Weka before.
Re:Ask HN: What should I do with 4 GBs of form spam? by nyellin on Tuesday February 15, 2011 @12:46PM (#2222275)
You should add the "rel=nofollow" attribute to the links that you show on your website. You can damage your search engine rankings with spammy outbound links.
Re:Ask HN: What should I do with 4 GBs of form spam? by arnorhs on Tuesday February 15, 2011 @12:24PM (#2222162)
I had actually asked about this very same issue a year ago, but the database was much smaller and I didn't really receive any breathtaking ideas.

Also - wtf of the day:: Last submissions was exactly a year ago! http://news.ycombinator.com/item?id=1147085

Re:Ask HN: What should I do with 4 GBs of form spam? by JacobAldridge on Tuesday February 15, 2011 @12:53PM (#2222307)
Close 356 days, not 365, though still close enough to be a note-worthy coincidence!
Re:Ask HN: What should I do with 4 GBs of form spam? by windsurfer on Tuesday February 15, 2011 @12:26PM (#2222172)
It would be neat to generate an image tracking similarities between spam messages. Sounds like a nifty project in R.
Re:Ask HN: What should I do with 4 GBs of form spam? by arnorhs on Tuesday February 15, 2011 @12:31PM (#2222199)
That's true. And even visualizing how spam has changed from two years ago. Choice of words, length, amount of links etc
Re:Ask HN: What should I do with 4 GBs of form spam? by sagacity on Wednesday February 16, 2011 @07:41AM (#2226141)
Please consider doing at least the following:

Generate a separate file (csv/sql or something) with the structure:

IP Address, number of submissions

and make it available for download.

I guess people could use it to flag probable spam.