News for hackers. Stuff that matters.
Re:Show HN: Wishminer – Mining social media for app ideas, m... by fiatjaf on Sunday March 25, 2018 @07:00PM (#16675147)
Seems very good for a weekend -- for an year, even.

How do you do it?

Re:Show HN: Wishminer – Mining social media for app ideas, m... by julvo on Sunday March 25, 2018 @07:10PM (#16675198)
Thank you. I'm using Firebase as a database and for hosting, which makes development really fast.

For getting the data, I'm using a worker written in Python, which queries the Twitter and Reddit APIs and does some regex filtering before writing it to the database. In the future, it could be interesting to experiment with more advanced NLP techniques for filtering/grouping/classifying.

Re:Show HN: Wishminer – Mining social media for app ideas, m... by parsthis on Monday March 26, 2018 @01:58AM (#16676931)
Very interesting, thanks for sharing. Im impressed by what you've done with just regex for the filtering. Which makes it possible to curate a dataset of wishes to do some unsupervised clustering and fitering.

Filtering could benefit from knowing what other syntax ( phrases ) would map to wishes, broadening the coverage.

Would love to work on this --let me know if you're open to that.

Either way, this is a useful project

Re:Show HN: Wishminer – Mining social media for app ideas, m... by andai on Monday March 26, 2018 @07:21AM (#16678035)
Re: finding other syntax, have you seen https://answerthepublic.com ?

That site lets you type in some key words and then it finds what questions people are searching for on Google based on those keywords (with prefixes like "who", "what", "where", and search suggestions).

Then it makes a nice big graph of the questions.

Re:Show HN: Wishminer – Mining social media for app ideas, m... by julvo on Monday March 26, 2018 @09:34AM (#16678700)
That's an amazing resource, thanks for sharing. Using autosuggestions might actually be really useful.
Re:Show HN: Wishminer – Mining social media for app ideas, m... by philippz on Wednesday March 28, 2018 @07:21PM (#16703003)
Hey guys, we're working on a platform that literally collects wishes from any touchpoint and it has quite some traction, especially in the gaming industry. Could you send me an email to hn{@}stomt.com? Would love to catch up on this. We also do a lot of research in the fields of NLP and clustering.
Re:Show HN: Wishminer – Mining social media for app ideas, m... by julvo on Monday March 26, 2018 @09:29AM (#16678671)
Unsupervised learning could be really interesting for this. Let's have a chat about how we could work together. Would you mind dropping me an email either via the website or to julian{at}wishminer.com?
Re:Show HN: Wishminer – Mining social media for app ideas, m... by 2data222 on Sunday March 25, 2018 @11:11PM (#16676329)
Hey, I'm thinking about doing something similar with baseball JSON apis of all things. Would you mind sharing your firebase/python implementation details with me? Thanks!
Re:Show HN: Wishminer – Mining social media for app ideas, m... by worldexplorer on Monday March 26, 2018 @04:58AM (#16677578)
I am curious to know how you did this filtering with regex. Can you share an example or any tutorial for reference?
Re:Show HN: Wishminer – Mining social media for app ideas, m... by andai on Monday March 26, 2018 @07:16AM (#16678020)
Search for tweets containing "I wish there was". Remove the "I wish there was" part. (Profit?)
Re:Show HN: Wishminer – Mining social media for app ideas, m... by julvo on Monday March 26, 2018 @09:37AM (#16678723)
Pretty much what undai says. It then comes down to building a dictionary of search queries and a 'blacklist'. The regex itself is relatively simple.