News for hackers. Stuff that matters.
Re:Show HN: I built a service to discover rapidly growing G... by jhow15 on Friday July 19, 2019 @09:16AM (#20478350)
Hey everyone, Josh here the creator. As a developer, I’m always looking out for an emerging market trend on which to bootstrap my own SaaS product. This is since market awareness and timing are often so critical, on top of execution skill of course!

So I built Trennd. Under the hood it continually monitors the web for interesting keywords & topics, classifies them using Google Trends data and packages everything as a neat web app where other people can contribute too.

The app itself is built with the Next.js React framework along with Express, Bootstrap and MongoDB. Next.js was new to me, but made sense since it comes with so much out of the box, including server side rendering.

Let me know if you have any questions, and any feedback is much appreciated!

Re:Show HN: I built a service to discover rapidly growing G... by ashelmire on Friday July 19, 2019 @02:21PM (#20481166)
These results aren't filtered in some way? I find it suspicious that almost all of the top ~100 results are tech related.
Re:Show HN: I built a service to discover rapidly growing G... by concert-gilled on Friday July 19, 2019 @02:59PM (#20481519)
> I have scripts that are crawling the web in places like twitter, reddit and facebook to generate keyword ideas. So the trends that it detects is biased towards the corners of the web that I am monitoring. Which skew heavily tech!
Re:Show HN: I built a service to discover rapidly growing G... by deadcoder0904 on Friday July 19, 2019 @10:04AM (#20478714)
You said you are monitoring interesting keywords & topics but how do you know which keywords & topics to monitor?

Also, if you are choosing the keywords then wouldn't the data be inaccurate for predicting trends as trends occur randomly?

Re:Show HN: I built a service to discover rapidly growing G... by jhow15 on Friday July 19, 2019 @10:13AM (#20478802)
I have scripts that are crawling the web in places like twitter, reddit and facebook to generate keyword ideas.

So the trends that it detects is biased towards the corners of the web that I am monitoring. Which skew heavily tech!

Re:Show HN: I built a service to discover rapidly growing G... by mooreds on Friday July 19, 2019 @11:41AM (#20479540)
This is a great service. It would be great for you to take suggestions on other locations to crawl.
Re:Show HN: I built a service to discover rapidly growing G... by jhow15 on Friday July 19, 2019 @12:20PM (#20479936)
That's a brilliant idea, will add somewhere for people to make suggestions. Thank you!
Re:Show HN: I built a service to discover rapidly growing G... by yuy910616 on Friday July 19, 2019 @11:28AM (#20479397)
Hi Josh, Very interesting stuff. Can I ask what is the math behind defining the trend? How did you calculate the most trendy words, etc?

I'm working on something similar for a internal tool, so I'm curious to see what your solution is.

Thanks!

Re:Show HN: I built a service to discover rapidly growing G... by jhow15 on Friday July 19, 2019 @12:19PM (#20479916)
Of course - I have scripts constantly monitoring the web for potentially interesting keywords. I then check google trends data for that topic/keyword and analyze the gradient of the line of best fit over different timeframes. If the gradient is higher than a certain (arbitrary) amount, then I class it as a trend!
Re:Show HN: I built a service to discover rapidly growing G... by peterhil on Friday July 19, 2019 @07:38PM (#20483296)
Is the gradient just a linear least squares fit?
Re:Show HN: I built a service to discover rapidly growing G... by yuy910616 on Tuesday July 23, 2019 @12:37PM (#20508036)
Not the author. Seems like it, it doesn't look like Google Trend gives other variables, so just simple 2 var regression I'd guess
Re:Show HN: I built a service to discover rapidly growing G... by omarhaneef on Friday July 19, 2019 @11:01AM (#20479180)
Nice work Josh.

It can probably be used for lots of things: startups, as you mentioned, but also journalism ideas, investing ideas, job hunting and many more.

Re:Show HN: I built a service to discover rapidly growing G... by jhow15 on Friday July 19, 2019 @11:08AM (#20479248)
Thank you! Yeah totally agree, I hadn't thought of job hunting though!!
Re:Show HN: I built a service to discover rapidly growing G... by kevinguh on Friday July 19, 2019 @10:43PM (#20483872)
Pretty neat, I see the most granular time scale currently supported is a 3-month window -- are there any plans to increase granularity towards a (near) real-time system? I get your crawler might be rate-limited at that scale and you would likely have to rethink the entire system architecture, but I think real-time data would open up a whole new dimension of possibilities for detecting/predicting trends perhaps even before public awareness catches up. I'd imagine there would be parties interested in paying for that kind of capability if that's the direction you're planning on taking this idea
Re:Show HN: I built a service to discover rapidly growing G... by jhow15 on Saturday July 20, 2019 @05:31AM (#20484857)
I'd love to go as granular as possible, at least 30 days. Real-time would be ideal, but as you note - Google Trends rate limiting soon becomes a hurdle especially when you potentially multiply by different geographies.
Re:Show HN: I built a service to discover rapidly growing G... by maxencecornet on Friday July 19, 2019 @10:37AM (#20478993)
What could be a really neat addition is having trends sorted by countries
Re:Show HN: I built a service to discover rapidly growing G... by jhow15 on Friday July 19, 2019 @10:45AM (#20479064)
Great idea, thank you!
Re:Show HN: I built a service to discover rapidly growing G... by tasssko on Friday July 19, 2019 @07:29PM (#20483269)
Awesome I enjoyed making some trend charts, and the results are fun (i.e I added DevOps, Erlang, Elixir). I think there is a risk it could conceal too many errors and biases for it to be useful for trend analysis. Using services like Twitter and Reddit might introduce biases into your data skewing trends. The challenge is using data and not understanding the intent (is it positive, negative or neutral/sarcastic potentially negative/positive). Google Adwords is good at this. I applaud your effort; I hope you keep going but its a tough nut to crack. I use markets like Google Adwords Tools to do this at the moment, have you looked at it? I think more info around time, intent and volume.
Re:Show HN: I built a service to discover rapidly growing G... by charlesism on Friday July 19, 2019 @12:03PM (#20479742)
I don’t have much feedback, except to say I think this is a really good, useful concept.
Re:Show HN: I built a service to discover rapidly growing G... by jhow15 on Friday July 19, 2019 @12:17PM (#20479896)
Thanks so much!
Re:Show HN: I built a service to discover rapidly growing G... by nurettin on Friday July 19, 2019 @11:16AM (#20479313)
Hi, it would be nice if the keywords had short explanations. I had to find out what an affinity designer and an enneagram was. Bubble tea was also quite new.
Re:Show HN: I built a service to discover rapidly growing G... by markdown on Friday July 19, 2019 @05:49PM (#20482725)
OT but...

> built with the Next.js React framework

Wait wait wait... do you mean to say there's now a framework for a framework? Javascriptland continues to confound.

Re:Show HN: I built a service to discover rapidly growing G... by hashbig on Friday July 19, 2019 @06:06PM (#20482841)
I wouldn't call React a framework, it's more like a library.
Re:Show HN: I built a service to discover rapidly growing G... by simonebrunozzi on Friday July 19, 2019 @12:33PM (#20480095)
Hey Josh, very interesting concept. Thanks for building it!

I tried to use the "technology" category, but the results are not what I expected. Perhaps it would be more useful if I could narrow it down to more specific things, e.g. programming languages, or databases.

Keep up the good work!

Re:Show HN: I built a service to discover rapidly growing G... by taneem on Saturday July 20, 2019 @04:37AM (#20484755)
Hi Josh - it would be incredible to have this filtered by location, and I think you can get that from Google Trends.
Re:Show HN: I built a service to discover rapidly growing G... by jhow15 on Saturday July 20, 2019 @04:58AM (#20484799)
Absolutely agree! How specific/granular would be the most valuable to you - at the city level perhaps?
Re:Show HN: I built a service to discover rapidly growing G... by taneem on Saturday July 20, 2019 @11:44AM (#20486412)
Yes - City would be great for people doing stuff around activities, events and local places. But Country would be more powerful for e-commerce I suspect (my usecase).
Re:Show HN: I built a service to discover rapidly growing G... by gondo on Friday July 19, 2019 @02:07PM (#20481035)
would be interesting to also to see the opposite, retreating trends
Re:Show HN: I built a service to discover rapidly growing G... by vernie on Friday July 19, 2019 @02:08PM (#20481041)
It seems to already have that (the "gradient" arrow)