Show HN: I made ChatGPT search 15,000 Reddit posts for side project ideas

Re:Show HN: I made ChatGPT search 15,000 Reddit posts for si... by Jkroeger123 on Monday January 08, 2024 @10:44AM (#38913591)

Hey HN,

Im an indiehacker who loves to solve for niche pain points

I realized I spend a lot of time just scrolling through subreddits looking for user complaints, insights, etc

I decided to have ChatGPT do the heavy lifting for me, and consolidate a more curated list of posts with insights for me to read

Stats:

Posts Processed: ~15k Insights Found: ~1K API Costs: ~$150

I hope a few of you may find this useful, would appreciate any feedback

~Justin

[ Parent | on HN ]

Re:Show HN: I made ChatGPT search 15,000 Reddit posts for si... by XCSme on Monday January 08, 2024 @05:14PM (#38918991)

~1K API Costs: ~$150

$150 for 1000 API requests? Isn't that way too high?

[ Parent | on HN ]

Re:Show HN: I made ChatGPT search 15,000 Reddit posts for si... by Jkroeger123 on Monday January 08, 2024 @05:21PM (#38919100)

ah sorry poor formatting. insights found was 1k. api costs were $150 for the 15k posts processed

[ Parent | on HN ]

Re:Show HN: I made ChatGPT search 15,000 Reddit posts for si... by XCSme on Monday January 08, 2024 @05:48PM (#38919462)

Still, isn't that high? With $150 you can rent a powerful server for a month and have a big LLM like Llama 2 running on it.

[ Parent | on HN ]

Re:Show HN: I made ChatGPT search 15,000 Reddit posts for si... by Jkroeger123 on Monday January 08, 2024 @06:13PM (#38919807)

yea definitely not cheap, to break it down further:

I first ran each post title through gpt-3 to classify as either likely or unlikely to be insightful to founders. Then, if it is likely to be insightful I pass the entire post content including up to 100 comments into gpt 4.

This comes down to around 13-14 million tokens used for the 15k posts, which for the latest gpt-4 model is ~$150.

Im sure this could be done cheaper with something like Llama 2 but tbh I am fine with this for now since I was aiming for speed with the initial list to gauge interest :)

[ Parent | on HN ]