News for hackers. Stuff that matters.

dmitrykan

dmitrykan on Hacker News

Re:Ask HN: What Are You Working On? (December 2025) on Sunday December 14, 2025 @04:29PM (#46267137)
On a Research Assistant: https://muves.io/

It helps to comprehend research papers (and not only papers - any document on any language) faster.

The tool is free to use, because we have credits from GCP. I guess at some point we'll need to introduce some level of subscription fee to keep it alive and useful, as it uses LLMs and vector search quite a bit.

Feedback is welcome!

Re: on Saturday February 08, 2025 @02:13AM (#42981110)
Leo Boytsov, my guest on the Vector Podcast, made an honest and an eye-opening claim about vector search being intellectually rewarding, but professionally undervalued. What picked my attention was how he gives credit to people who actually deserve it, and how he speaks modestly about his own achievements. When professionally as a researcher, he accumulated over 1800 citations by now and helped to create the famous HNSW vector search algorithm.
Re:The Rise, Fall, and Future of Vector Databases: How to Pick the One... on Thursday January 16, 2025 @04:28PM (#42731127)
My take on whether we've lost the category of vector databases.
Re:OTranscribe: A free and open tool for transcribing audio interviews on Friday August 09, 2024 @07:56AM (#41200973)
I'm working on the tool, that includes AI. My original target is to test it on my https://www.youtube.com/c/VectorPodcast by offering something that Lex Fridman does for his episodes.

Current features: 1. Download from YT 2. Transcribe using Vosk (output has time codes included) 3. Speaker diarization using pyannote - this isn't perfect and needs a bit more ironing out.

What needs to be done: 4. Store the transcription in a search engine (can include vectors) 5. Implement a webapp

If anyone here is interested to join forces, let me know.

Re:Ask HN: Could you share your personal blog here? on Wednesday July 05, 2023 @03:12AM (#36596696)
https://dmitry-kan.medium.com/

I write about vector search, ANN algorithms, neural search frameworks, search engines and algorithms in general and publish episodes of the Vector Podcast.

Re:What Are Embeddings? on Monday June 26, 2023 @05:08PM (#36485877)
Nice post, thanks for sharing! Learnt about new dbs / solutions for vector search from it.

Vicki mentions this survey I wrote some time ago: https://towardsdatascience.com/milvus-pinecone-vespa-weaviat...

Hoping it'll be useful as well!

Re:Neural Search Frameworks: A Head-to-Head Comparison on Saturday December 24, 2022 @04:44AM (#34115396)
thank you, Doug! It is mutual. Had a fantastic discussion with you on the podcast: https://www.youtube.com/watch?v=Kpua1Euc-B8
Re:Vector Podcast – Journey of Vespa from Sparse into Neural Search on Tuesday April 12, 2022 @12:07PM (#31004523)
It was great to discuss with Jo Kristian on these topics and more:

- History of Vespa - Tensor data structure and its use cases - Multi-stage ranking pipeline - Game-changing vector search in Vespa - Approximate vs exact nearest neighbor search tradeoffs - Misconceptions in neural search - Multimodal search is where vector search shines - Power of building fully-fledged demos - How to combine vector search with sparse search: Reciprocal Rank Fusion - The question of WHY (my favourite)

I wonder what topics are interesting to HN community -- it would help me focus on these topics / embed into my questions in new episodes.

Re:The next Google on Wednesday April 06, 2022 @02:33AM (#30929074)
Thanks for the article, I've learnt new search engines despite spending a couple of years recently in web scale search. I think you may consider https://usearch.com/ as another dimension in web scale search, where query log is learnt from the data, making it quite unique.
Re:Show HN: We made an open-source personalization engine on Wednesday March 23, 2022 @04:27PM (#30782896)
Great project! Elasticsearch / OpenSearch / Solr have their own learning to rank plugins. Have you considered integrating Metarank with such systems? Or is your vision to provide a reranker layer, that can be independent of the underlying search engine architecture?
Re:Vectorizing the entire eCFR for $3.00 and 22 minutes on Wednesday March 16, 2022 @12:20PM (#30700759)
This is fantastic work, Max! I like where you're going with this unit economy.

It is increasingly more and more common that companies need ways of productizing their deep learning models with tight SLA constraints and tight budgets. I think, one of the logical steps could be integration with vector databases / frameworks and measuring gains there in client setups.

Re: on Wednesday December 22, 2021 @07:59AM (#29648685)
5th episode of Vector Podcast with Tom Lackner, VP Engineering at classic.com explaning the choise of Qdrant vector DB for his projects.
Re:Semantic search through Wikipedia with the Weaviate vector search e... on Monday November 29, 2021 @11:50AM (#29382016)
thanks!
Re:Semantic search through Wikipedia with the Weaviate vector search e... on Monday November 29, 2021 @11:49AM (#29382014)
To learn about vector search and get practical with it, you can check out my blog as well: https://dmitry-kan.medium.com/list/vector-search-e9b564d1427...

I'm also hosting a podcast around these topics: https://www.youtube.com/channel/UCCIMPfR7TXyDvlDRXjVhP1g

Re:Semantic search through Wikipedia with the Weaviate vector search e... on Monday November 29, 2021 @10:39AM (#29381052)
11M+ Wikipedia articles indexed for similarity search.
Re:Vector Search on Friday November 12, 2021 @03:52PM (#29204389)
From code to podcast to billion-scale ANN challenge and statistics
Re:Ć Programming Language on Sunday October 10, 2021 @06:24AM (#28817270)
Thanks! One practical target I imagine having is: 1. write the program in Python. 2. translate it into Ć ("interlingua") 3. compile Ć into Go -- for instance there is a belief around me, that Python is not suitable for web scale loads, while Go is more suitable.

Go source code / binary in this case are of less importance for code readability, because they are meant for production deployments. Something that happened in GWT: write in Java, compile into JavaScript.

Re:Ć Programming Language on Saturday October 09, 2021 @07:15AM (#28808841)
Thanks for sharing! The practical value of such languages could be to build an interlingua and support Python->Ć->Go. The question is: if Ć->Python works seamlessly, how easy it is to make the opposite translation? For instance, take some code that uses external library, like spacy.

If this becomes ever possible, it is going to be revolutionary.

Re:Vector Podcast for Vector Search: people, businesses, products on Tuesday October 05, 2021 @02:27PM (#28763509)
A related discussion and a detailed study about vector databases can be found here: https://news.ycombinator.com/item?id=28727816
Re:Vector Podcast for Vector Search: people, businesses, products on Tuesday October 05, 2021 @11:33AM (#28760853)
Vector Podcast is here to bring you the depth and breadth of Search Engine Technology, Product, Marketing, Business. In the podcast we talk with engineers, entrepreneurs, thinkers and tinkerers, who put their soul into search.

Depending on your interest, you should find a matching topic for you -- whether it is deep algorithmic aspect of search engines and information retrieval field, or examples of products offering deep tech to its users.

"Vector" -- because it aims to cover an emerging field of vector similarity search, giving you the ability to search content beyond text: audio, video, images and more.

"Vector" also because it is all about vector in your profession, product, marketing and business.

SoundCloud: https://soundcloud.com/vector-podcast-dmitry-kan/greg-kogan-...

Apple Podcasts: https://podcasts.apple.com/us/podcast/greg-kogan-pinecone-on...

Re: on Sunday October 03, 2021 @11:31AM (#28737955)
sure!
Re: on Sunday October 03, 2021 @06:58AM (#28736168)
Yes, for larger datasets of vectors, having the search do a linear scan will likely be slow. So you could take a look at KNN/ANN (K nearest / Approximate nearest neighbors), like https://faiss.ai/ But if you prefer to offload this complexity to a database, you can pick and evaluate one from my blog post.

You have 4 out of 6 open source DBs, and 2 commercial ones give you the managed service. All 6 can scale to quite large numbers of vectors.

My goal is to systematically study each DB through the lens of a specific search task, which will not be (only) text based.

If you think we could collaborate in some way on your dataset, that would be fantastic and probably a learning experience to both sides.

Re: on Sunday October 03, 2021 @04:56AM (#28735613)
Thanks for more detail, this sounds quite interesting. So basically the 16-digit chromosome records you added to each document would work as nearest neighbors? Did this satisfy all your search needs for an algorithm?
Re: on Sunday October 03, 2021 @01:36AM (#28734788)
btw, if would be cool to see some example of a genetic algorithm input and resulting vector? And how do you use the vectors in down-stream tasks?
Re: on Sunday October 03, 2021 @01:35AM (#28734781)
Such a great question! I think vector search is designed to solve a specific task, like finding K nearest neighbors for your given vector embedding. If you have a few hundred thousand, a million, it might work fine with a linear scan. Again, this depends on your use case. But if you have tens of millions, hundreds of millions or even a billion of vectors, you will need something more robust to store, index and search over this data.

An example in NLP world is BERT-like NNs, that allow you to embed your text into a dense vector representation.

Re: on Saturday October 02, 2021 @04:37PM (#28731866)
Hi! It is a metered post atm, will consider making it more public over time. If you want to get practical with vector search in Solr/Elasticsearch/Opensearch you can try out this repo: https://github.com/DmitryKey/bert-solr-search

Older »