News for hackers. Stuff that matters.
Re:Show HN: A Twitter crawler for app ideas by mulhoon on Thursday April 08, 2021 @01:41PM (#26741612)
I spent a few hours building this. I’d love to hear your thoughts.
Re:Show HN: A Twitter crawler for app ideas by nickelcitymario on Thursday April 08, 2021 @02:14PM (#26741954)
I love this and have bookmarked it for daily review.

If I can make one recommendation, add a search function. That way users can find all the recommendations that touch on some field that's important to them.

For example, I'm working on an app for ebook authoring. It would be awesome to search through these for any ebook related app requests. It could help me to spot patterns related to my field, and give me ideas not just for apps but for features.

If I can make a second recommendation, make those search results subscribe-able... like a Google Alerts for Twitter-based app requests.

Re:Show HN: A Twitter crawler for app ideas by fransiscoli on Thursday April 08, 2021 @02:12PM (#26741930)
This is awesome! It would also be really nice if the scraper would organize the ideas based on keywords in a given tweet.
Re:Show HN: A Twitter crawler for app ideas by josefresco on Thursday April 08, 2021 @03:54PM (#26743002)
As someone who built a website called (roughly) "$5 business ideas" about 12 years ago this gives me great joy! It was mostly satire but still fun to build and populate with terrible ideas.
Re:Show HN: A Twitter crawler for app ideas by bckr on Thursday April 08, 2021 @02:04PM (#26741849)
I think this is pretty cool and a great idea. One thing that I ran into within a couple minutes is that there are several entries that don't have enough context. It will just say something like "is there an app?".
Re:Show HN: A Twitter crawler for app ideas by asdev on Thursday April 08, 2021 @04:01PM (#26743079)
What's the tech stack behind it? Just curious, it takes me days/weeks to get any ideas out due to tech choice paralysis
Re:Show HN: A Twitter crawler for app ideas by carmen_sandiego on Thursday April 08, 2021 @04:31PM (#26743389)
Looks like Nuxt/Vue sitting on S3 with CloudFront as CDN. Presumably with some script on the backend to update the list of Tweets on S3.

/vaguely informed guessing

Re:Show HN: A Twitter crawler for app ideas by mulhoon on Thursday April 08, 2021 @04:44PM (#26743529)
Yup, It's one simple lambda function that reads the JSON file from S3, searches twitter for new tweets since the latest, appends new ones and writes back to S3. Cron'd every minute.
Re:Show HN: A Twitter crawler for app ideas by tracyhenry on Thursday April 08, 2021 @11:31PM (#26746396)
This is awesome! What is the strategy to identify the idea tweets? It looks like some string matching, but I'm not sure.
Re:Show HN: A Twitter crawler for app ideas by mulhoon on Saturday April 10, 2021 @10:54AM (#26761903)
Yes, nothing too special. There are 6 or 7 string matches e.g. "I need an app that" (minus retweets). But also retrieving referenced tweets.
Re:Show HN: A Twitter crawler for app ideas by MarkMc on Sunday April 11, 2021 @07:44AM (#26769605)
A few hours? That is a fantastic return on investment!

By the way, you should definitely add this to your CV. It's the sort of technical thing that really stands out from other candidates.

Re:Show HN: A Twitter crawler for app ideas by nickysielicki on Thursday April 08, 2021 @04:52PM (#26743625)
You should go one level of abstraction further and make a site where users can aggregate arbitrary twitter search queries like this. AFAIK you can't actually "follow" a search query or hashtag on twitter.