Show HN: JobLens – AI-Powered Job Search for 'Who's Hiring'
* Country and remote work preferences
* Employer type (e.g., startup, corporation, government)
* Industry
* Technologies used
* Role type (developer, architect, product owner, etc.)
* Salary range (where available)
One of the superpowers of LLMs is reformatting information from any format X to any other format Y. We leverage this to map all the unstructured job postings into the same unified strucutre. The new GPT functions feature and the extended context windows are really helpful for this. Instead of having to build a custom NER pipeline, it works very well with GPT out-of-the box.
One challenge to keep the filters consistent is the merging of duplicates. Embeddings help with that.
What's next:
* Integrate additional sources. We can generate web scrapers and data processing steps on the fly that extract and transform data into the same structure.
* Add location distance filters.
* Expand beyond jobs to monitor personalized data like events or real estate. Imagine using AI to rate local events from multiple sources based on your preferences, considering factors like your interests and distance from home.
* Smaller improvements based on your feedback :)
ChatGPT does an incredible job parsing, but then lots of effort goes into normalizing and deduping each field. Long story short, your results look quite good to me!
Bottom line, every single post requires approximately 10 different prompts to refine the extraction.
I tried using the OpenAI functions API to do structured text extraction, but found that it would hallunicate a lot of things. Do you guys have that problem? How did you guys go about solving that?
I'm curious the page mentions not being able to scrape sites that require a login but then I saw LinkedIn as an example. Will LinkedIn eventually be supported but require a login or something?
Still, a good example of OpenAI's functions. I was using direct prompt for YAML parsing before and migrated to functions for my job board https://aijobnetwork.com/
I'm also using OpenAI's embeddings for recommendations: https://medium.com/@mauricio/how-to-add-recommendations-to-a...