News for hackers. Stuff that matters.
Re:239,621 Hacker News datasets from the past 5 years by massanishi on Tuesday November 17, 2020 @11:16AM (#25125476)
OP: Hackernews offers an excellent API via Firebase: https://github.com/HackerNews/API. Unfortunately, their fetch method doesn't offer any way to filter by item types. So collecting all posts take excessively long thanks to all the comments.

Based on the official item API, this focuses only on the main posts with at least 2 engagements. The posts in datasets are about 1.5% of the items returned

With posts as a starting point, you can easily trace the hierarchy from "kids" fields.

Potential Usage:

- Generate popular titles.

- Collect comments in a hierarchical order for further training.

- Analyze popular topics in the engineering community.

- Identify the best time to post for maximum engagement.

Re:239,621 Hacker News datasets from the past 5 years by fsflover on Tuesday November 17, 2020 @01:24PM (#25127484)
I wish someone would use GPT-3 to generate automatic replies to comments based on previous HN history. I'm sure discussions repeat themselves a lot.
Re:239,621 Hacker News datasets from the past 5 years by aaron695 on Tuesday November 17, 2020 @03:10PM (#25129052)
The only thing GPT-3 is good for is producing useless text that humans find hard to process.

So it's great for blog spam.

GPT-3 would produce comments that would get up voted but contribute absolutely nothing.

You'd be better crowdsourcing your idea with a page to point people to explaining how the threads been done before and the common answers.

Re:239,621 Hacker News datasets from the past 5 years by lowdose on Tuesday November 17, 2020 @03:21PM (#25129244)
Would love to see some examples of what you are talking about, do you have access to the OpenAI API?
Re:239,621 Hacker News datasets from the past 5 years by aaron695 on Tuesday November 17, 2020 @06:11PM (#25131161)
There's a silicon user on HN, but no idea if they are/identify as a GPT-3. They have some training particular to HN

But here is a outed Reddit user -

https://www.theregister.com/2020/10/09/reddit_gpt3_bot/

https://www.reddit.com/user/thegentlemetre/