News for hackers. Stuff that matters.

Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring'

https://www.kadoa.com/joblens

There are existing HN job aggregators, but I thought we could take it a step further. Inspired by an insightful comment on a previous thread (https://news.ycombinator.com/item?id=36163021), I built a tool that aggregates job postings and intelligently categorizes them based on user-specific preferences:

* Country and remote work preferences

* Employer type (e.g., startup, corporation, government)

* Industry

* Technologies used

* Role type (developer, architect, product owner, etc.)

* Salary range (where available)

One of the superpowers of LLMs is reformatting information from any format X to any other format Y. We leverage this to map all the unstructured job postings into the same unified strucutre. The new GPT functions feature and the extended context windows are really helpful for this. Instead of having to build a custom NER pipeline, it works very well with GPT out-of-the box.

One challenge to keep the filters consistent is the merging of duplicates. Embeddings help with that.

What's next:

* Integrate additional sources. We can generate web scrapers and data processing steps on the fly that extract and transform data into the same structure.

* Add location distance filters.

* Expand beyond jobs to monitor personalized data like events or real estate. Imagine using AI to rate local events from multiple sources based on your preferences, considering factors like your interests and distance from home.

* Smaller improvements based on your feedback :)

( 18 comments | 2 points | on HN )
Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by rahimnathwani on Monday July 03, 2023 @03:39PM (#36577828)
This is cool. I'm curious about this part:

  We leverage this to map all the unstructured job postings into the same unified structure. The new GPT functions feature and the extended context windows are really helpful for this.
Does this mean that, rather than using the gpt-3.5-turbo chat API, and using a system prompt that describes the output format you want (JSON?), you instead describe your output format in the same way you would if you were going to call a function?
Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by hubraumhugo on Monday July 03, 2023 @04:14PM (#36578176)
Yes, you can essentially use GPT functions to reliably output JSON based on a predefined schema. See https://github.com/openai/openai-cookbook/blob/main/examples...
Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by boxcarr on Monday July 03, 2023 @04:03PM (#36578062)
JobLens is cool! This past month we had the same idea -- it's been a fun project: https://hnjobs.u-turn.dev

ChatGPT does an incredible job parsing, but then lots of effort goes into normalizing and deduping each field. Long story short, your results look quite good to me!

Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by hubraumhugo on Monday July 03, 2023 @04:11PM (#36578149)
Your project looks quite impressive as well, especially the extracted URLs to apply and the candidate profiles, didn't get that far yet :) Automating tedious work like data extraction and transformation is a great use case for LLMs.
Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by boxcarr on Monday July 03, 2023 @05:38PM (#36579188)
Not sure if you found this as well, but gpt-3.5-turbo-016 does a poor job following instructions other than parsing. So, to work around this, we prompt gpt-3.5-turbo with the rules we want applied say an extracted field and then go back to gpt-3.5-turbo-016 to parse with chatgpt functions.

Bottom line, every single post requires approximately 10 different prompts to refine the extraction.

Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by hubraumhugo on Tuesday July 04, 2023 @03:30AM (#36583408)
yeah I had the same experience, divide and conquer works better than trying to do everything with a single prompt. Are you interested in comparing notes? Feel free to ping me via the email in my profile :)
Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by SCUSKU on Monday July 03, 2023 @04:27PM (#36578306)
Wow this is amazing. This is more or less what I was trying to build https://hnresumetojobs.com to look like, I am reconsidering whether or not to continue now.

I tried using the OpenAI functions API to do structured text extraction, but found that it would hallunicate a lot of things. Do you guys have that problem? How did you guys go about solving that?

Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by OccamsMirror on Monday July 03, 2023 @07:27PM (#36580138)
Ask ChatGPT to improve your prompt to stop it hallucinating. I’ve had a lot of wins doing that.
Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by SomewhatLikely on Monday July 03, 2023 @07:32PM (#36580195)
Having the 'unchecked' visual actually have a checkmark threw me off. I was thinking, oh goodness, I have to deselect all of these.
Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by timhigins on Monday July 03, 2023 @04:37PM (#36578448)
Site is down for me, maybe the HN hug of death? Anyone else seeing this?
Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by baxtr on Monday July 03, 2023 @04:45PM (#36578563)
Nice! Is there something similar for freelancing gigs?
Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by version_five on Monday July 03, 2023 @07:04PM (#36579954)
Hi, just a thought, I backed out as soon as I saw I had to pick from a giant list of industries. Most people don't have rigid criteria like that, and if the do, a conventional search is easy. Conversational AI has a potential to more naturally arrive at interesting jobs, either through asking for a freeform description of what you want, or maybe a back and forth discussion.
Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by great_psy on Monday July 03, 2023 @11:50PM (#36582012)
I got turned off by that as well, but I submitted the form anyway, and it loads correctly. I guess no fill in means *
Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by hubraumhugo on Tuesday July 04, 2023 @01:12AM (#36582477)
You're right, I'll improve this.
Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by digitalsanctum on Monday July 03, 2023 @07:09PM (#36579982)
I was going to build something similar for funsies but now I don't have to :)

I'm curious the page mentions not being able to scrape sites that require a login but then I saw LinkedIn as an example. Will LinkedIn eventually be supported but require a login or something?

Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by Laaas on Monday July 03, 2023 @07:10PM (#36579997)
All I want is a list of jobs that involve functional programming.
Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by mmaia on Monday July 03, 2023 @07:38PM (#36580247)
Looks like it's getting some fields wrong. For examples, "Global" location is matching other text not related to the job location.

Still, a good example of OpenAI's functions. I was using direct prompt for YAML parsing before and migrated to functions for my job board https://aijobnetwork.com/

I'm also using OpenAI's embeddings for recommendations: https://medium.com/@mauricio/how-to-add-recommendations-to-a...

Re:Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring' by dsgnr on Tuesday July 04, 2023 @09:30AM (#36586381)
Need a category for design. Right now all design jobs are scattered around, mostly in product management. Also as others have mentioned, having to make all these decisions before searching is hard to get through without abandoning.