News for hackers. Stuff that matters.
Re:EC2's most dangerous feature by cperciva on Sunday October 09, 2016 @04:31AM (#12670323)
OK, dwaxe, I have to ask: Are you a robot? Because I uploaded this blog post, tweeted it, and then came straight over here to submit it and you still got here first.

Not that I mind, but getting your HN submission in within 30 seconds of my blog post going up is very impressive if you're not a robot.

Re:EC2's most dangerous feature by dwaxe on Monday October 10, 2016 @06:07AM (#12676154)
Yes I am. This is my personal account, but I use it to automatically post to Hacker News. I was playing around with BigQuery one day and found the Hacker News dataset [1]. From my experience with the Reddit submissions dataset [2], I knew that I could compose this query,

SELECT AVG(score) AS avg_score, COUNT(* ) AS num, REGEXP_EXTRACT(url, r'//([^/]*)/') AS domain FROM [fh-bigquery:hackernews.full_201510] WHERE score IS NOT NULL AND url <> '' GROUP BY domain HAVING num > 10 ORDER BY avg_score DESC

which returns a list of domains with more than ten submissions sorted by average score. This turns out to be a list of some of the most successful tech blogs on the internet, as well as various YCombinator related materials. Out of the domains with over 100 submissions, daemonology.net has the 9th highest average score per submission. I manually visited all the domains with more than about 30 submissions, found the appropriate xml feeds, and saved them. I added a few websites like eff.org whose messages I think everyone should read anyways.

Then I jumped into python and started trying to figure out how to post to Hacker News. It was a little more complicated than I anticipated [3], but an open source HN app for Android helped me figure it out.

I set up a cron job on my $5 Digital Ocean that runs the script every few minutes (pseudocode):

If you can reach http://news.ycombinator.com, Check all feeds for new entries, Post a new entry to hn, Sleep for an hour before posting another

[1] https://bigquery.cloud.google.com/dataset/bigquery-public-da...

[2] The only difference on Reddit is the subreddit system.

[3] After you send a POST request to send to the login screen, Hacker News gives you a url with a unique "fnid" parameter, and you send another POST request to another url with the appropriate "fnid".

Re:EC2's most dangerous feature by dang on Monday October 10, 2016 @01:02PM (#12678348)
We appreciate both the cleverness here and your detailed explanation. But could you please not do this anymore? It isn't malicious, but it's unhealthy for HN's ecosystem. For example, when an author submits his or her own work, that can add a lot of value to the community—but your bot pre-empts that, as it indeed did in this case.

There are many more reasons why this isn't a good thing for HN. For example, it's better for submissions from popular sites to be distributed across a wide range of accounts. That gives more users a chance to feel like they're making important contributions, and gives the community (and authors) a clearer sense of the audience.

There are lots of ways to write software to interact with HN, and lots of users with the ability to do it, so we really depend on the good will of the community only to do that when it serves the whole.

Re:EC2's most dangerous feature by Scaevolus on Sunday October 09, 2016 @05:48AM (#12670511)
Yes, there's definitely a script at work.

Compare these timestamps (hover over date): https://hn.algolia.com/?query=dwaxe%20eff&sort=byPopularity&... To the RSS feed: https://www.eff.org/rss/updates.xml

Humans don't post everything from an RSS feed within 10 minutes!

Re:EC2's most dangerous feature by 0x0 on Sunday October 09, 2016 @08:44AM (#12670955)
Wouldn't it be funny if you added an RSS entry that linked to an otherwise unpublished page with the url path /i-am-a-bot-that-posts-to-hackernews with a matching title to go?
Re:EC2's most dangerous feature by cperciva on Sunday October 09, 2016 @06:23PM (#12673891)
Re:EC2's most dangerous feature by 0x0 on Sunday October 09, 2016 @06:36PM (#12673950)
Haha that's brilliant! Nice touch with the recursive link.
Re:EC2's most dangerous feature by jacques_chester on Sunday October 09, 2016 @09:40AM (#12671141)
> Yes, there's definitely a script at work.

Presumably to farm karma for some other purpose.

I once observed that everything that shows up here eventually shows up in a particular cluster of subreddits and vice versa. Usually with a lag of 24-48 hours.

I half-jokingly floated the idea that one could write a karma arbitrageur bot which cross posts between HN and those subreddits.

I was told that I would be banned. Not just the bot: me.

Re:EC2's most dangerous feature by 0xmohit on Sunday October 09, 2016 @07:17AM (#12670766)
The submissions [0] appear to be scripted. Most of those appear to be from selected few domains.

Your domain seems to have been added recently [1]. Congratulations!

So whether you choose to blog about tarsnap or anything else, chances are that it'd be posted to HN before you're able to.

[0] https://news.ycombinator.com/submitted?id=dwaxe

[1] https://news.ycombinator.com/from?site=daemonology.net

Re:EC2's most dangerous feature by jacquesm on Sunday October 09, 2016 @05:27AM (#12670459)
Suggestion: post before tweeting.

And to your question, dwaxe is not a bot (there are comments associated with the account too), and this has happened before (apparently a lightning fast submitter):

https://news.ycombinator.com/item?id=12218797

Of course he/she could still be running a script.

Re:EC2's most dangerous feature by cperciva on Sunday October 09, 2016 @05:38AM (#12670483)
Suggestion: post before tweeting.

I'll probably do that next time. I didn't think it would matter! (And really it doesn't -- it's not as if I need the karma points.)

Re:EC2's most dangerous feature by chrisan on Sunday October 09, 2016 @06:22AM (#12670622)
> And really it doesn't -- it's not as if I need the karma points.

It's the principle of the matter! Does HN allow bot submissions?

Re:EC2's most dangerous feature by OJFord on Sunday October 09, 2016 @10:44AM (#12671401)
Or better; assuming the URI is known before posting (it's not randomly generated on submission or encapsulates absurdly precise submission time) just post it to HN a fraction of a second before it's live. Nothing any script can do to beat you then.
Re:EC2's most dangerous feature by jacquesm on Sunday October 09, 2016 @11:01AM (#12671477)
HN doesn't check submissions by fetching them so that would work.
Re:EC2's most dangerous feature by jacques_chester on Sunday October 09, 2016 @09:44AM (#12671151)
> OK, dwaxe, I have to ask: Are you a robot?

Cyborg. Story submission is clearly robotic, but there's a lot of charmingly humanesque entries under dwaxe's comments:

https://news.ycombinator.com/threads?id=dwaxe

Re:EC2's most dangerous feature by brettproctor on Sunday October 09, 2016 @04:58AM (#12670386)
Assuming dwaxe replies claiming to not be a robot, how would we go about verifying? :P
Re:EC2's most dangerous feature by pilsetnieks on Sunday October 09, 2016 @08:40AM (#12670943)
We could administer it a test of some sort, and evaluate the ensuing conversation, whether it's convincingly human. I think it'd be fitting name this test in honor of some forefather of computer science.
Re:EC2's most dangerous feature by ATsch on Sunday October 09, 2016 @05:53AM (#12670521)
I heard robots float on water
Re:EC2's most dangerous feature by bsandert on Sunday October 09, 2016 @06:14AM (#12670591)
What also floats on water?
Re:EC2's most dangerous feature by tomludus on Sunday October 09, 2016 @06:18AM (#12670607)
Ducks!
Re:EC2's most dangerous feature by OJFord on Sunday October 09, 2016 @10:39AM (#12671379)
Did I mention that dwaxe turned me into a newt?

(I got better.)

Re:EC2's most dangerous feature by lucideer on Sunday October 09, 2016 @08:58AM (#12670990)
We could compare the time it took to submit (seconds) to the time it took to comment (3 hours and counting...)
Re:EC2's most dangerous feature by latentpot on Sunday October 09, 2016 @09:09AM (#12671041)
Could be using something like IFTTT with a simple rule RSS to Post Immediately on Forum
Re:EC2's most dangerous feature by jakozaur on Sunday October 09, 2016 @06:44AM (#12670683)
Most likely dwaxe is human using scripts to boost his karma. Almost like sportsman using enhancing performance drugs.

Side question: I wonder when AI pass Hacker News Turing test. So bot can trick us into being human by HN comments.

Re:EC2's most dangerous feature by human_error on Sunday October 09, 2016 @05:15AM (#12670428)
Probably right time right place.