News for hackers. Stuff that matters.

tompec

Re:Show HN: Turn any website into a knowledge base for LLMs on Thursday August 01, 2024 @12:05AM (#41126050)
It doesn't embed images, no. But that's a great idea for the roadmap!
Re:Show HN: Turn any website into a knowledge base for LLMs on Wednesday July 31, 2024 @10:50PM (#41125745)
Having looked at a lot of HTMLs, I noticed that sections are not really the default. I rely on headings (h1, h2, ...) to chunk each pages. Each chunk has its heading hierarchy attached to it. There are a lot of optimizations that could be done at that level.
Re:Show HN: Turn any website into a knowledge base for LLMs on Wednesday July 31, 2024 @08:19PM (#41124959)
I chunk pages and generate embeddings for each chunk. So there's no real size limit per page.
Re:Show HN: Turn any website into a knowledge base for LLMs on Wednesday July 31, 2024 @07:22PM (#41124591)
Gotta start somewhere :)
Re:Show HN: Turn any website into a knowledge base for LLMs on Wednesday July 31, 2024 @05:53PM (#41123878)
Sorry about that, a bit too much load at the moment
Re:Show HN: Turn any website into a knowledge base for LLMs on Wednesday July 31, 2024 @05:47PM (#41123821)
Thanks! I'm still figuring things out about pricing, but there will be small plans available.
Re:Show HN: Turn any website into a knowledge base for LLMs on Wednesday July 31, 2024 @05:43PM (#41123782)
It does respect robots.txt when crawling. I'll add more details about this in the docs.
Re:Show HN: Turn any website into a knowledge base for LLMs on Wednesday July 31, 2024 @05:33PM (#41123688)
Currently just a cloud-toy.
Re:Show HN: Turn any website into a knowledge base for LLMs on Wednesday July 31, 2024 @05:25PM (#41123628)
Thanks! The chat demo is actually just a small thing I put together as a preview of what can be done, but the main product is the API. But seeing that most users seem to like that, there's probably something there... If you want to email me at support at embedding.io with some requirements, I can see how to make that work for you.
Re:Show HN: Turn any website into a knowledge base for LLMs on Wednesday July 31, 2024 @05:20PM (#41123587)
You can group as many websites as you want into a collection. Then query that collection. Not sure what you mean by exporting; you would like to export the vectors themselves? Or just the chunks of text from the websites?
Re:Show HN: Turn any website into a knowledge base for LLMs on Wednesday July 31, 2024 @05:13PM (#41123527)
Apologies, please email me at support at embedding.io. If you have something you'd like embedded, please also mention it so I can set it up for you.
Re:Show HN: Turn any website into a knowledge base for LLMs on Wednesday July 31, 2024 @05:06PM (#41123466)
It currently will try to find a sitemap on its own. But I have on the roadmap to let users add their own.
Re:Show HN: Turn any website into a knowledge base for LLMs on Wednesday July 31, 2024 @06:10AM (#41117788)
It works with pretty much any website, and works well with docs hosted on GitBook yes, I have embedded a website that's hosted there.
Tech stack is a mix of serverless Laravel, with Cloudflare and AWS functions, and some Pinecone for vector storage. Still experimenting on a few things but don't want to over-engineer unless I know where I'm going.
Unless you own those sites, I'm afraid that's not going to be possible.
Give it URLs or domains, and it will crawl and extract their content, embed them in a vector database, and give you an endpoint that you can then query when doing RAG stuff or semantic search.
I built this tool because I wanted a way to just take a bunch of URLs or domains, and query their content in RAG applications.

It takes away the pain of crawling, extracting content, chunking, vectorizing, and updating periodically.

I'm curious to see if it can be useful to others. I meant to launch this six months ago but life got in the way...

Re:Show HN: Imgsrc – Generate Beautiful Open Graph Images on Friday January 05, 2024 @04:10AM (#38877160)
This is great! Congrats on shipping this. I made a basic version of this years ago (imageee.com) and would love to find a replacement. Something I could use in the same way. ``` <meta property="og:image" content="https://api.imgsrc.io/basic?title=This is a title&description=blahblahblah&logo=https://example.com/image.png"> ```
Re:Show HN: ChatGPT for HelpScout on Tuesday April 04, 2023 @02:07AM (#35435720)
I made this service to solve my own customer support problem. GPT-4 generates draft replies using previous responses as a knowledge base. I've been using it for a few months and most replies don't need editing. Thoughts?
Re:Show HN: Netflix Prices in 245 Countries on Sunday January 29, 2023 @02:37AM (#34565806)
The content depends on where you stream from or the VPN you use. If you subscribed from India and then move to the USA, you will see content available in the USA for the price of Netflix India.
Not yet sorry :) Maybe one day!
Will do, thanks for the tip!
Glad you like the idea! Categories are on our list of features to implement. We're thinking about hashtags like Twitter does. Yeah, Signup with Twitter was mostly to accelerate the development time but an email signup will be introduced in the future as we're aware that some people prefer this way.
Hi there, My friend Nathan and I recently launched Predibly.com: a social platform to publicly share your predictions of the future and have interesting conversations about them. Nathan had the idea last week, I build the MVP and we'd love to know your thoughts!

Older »