News for hackers. Stuff that matters.

tomhartke

On the discovery progress, we can't look at all 200M papers with the LLM, so we prioritize some of them (the first 100 most promising) for deep analysis. Within those, we find a few that are relevant. But the rate at which we discover these tells us roughly what will happen if we read the next 100 (if we're discovering new relevant papers all the time, we will likely continue). We need a better explanation on the website, but we can statistically model this to quantitatively predict how many papers we would find if we exhaustively searched the whole database.

More info here: https://www.undermind.ai/static/Undermind_whitepaper.pdf

Here's some others focused at least partly on search: Elicit https://elicit.com/ Lumina-chat https://www.lumina-chat.com/ Consensus https://consensus.app/ OpenEvidence (medical focused) https://www.openevidence.com/ sciscape: https://typeset.io/ Epsilon-ai https://www.epsilon-ai.com/ scholarAI https://scholarai.io/
Here's the results for Emmitt's request on research using interventions to improve reaction time: https://www.undermind.ai/query_app/display_one_search/e1bcab...
While the time/cost of using GPT-4 is not ideal, GPT-4-level classification is absolutely crucial for the entire adaptation process to succeed. With 3.5 guiding the adaptation, we find that errors quickly accumulate. It can't identify complex ideas correctly.

3-6 minutes for results takes getting used to, but we've found most people don't complain if it solves a problem that is actually impossible to solve without hours of digging, ie if you use it on something truly hard. Low latency is more crucial for public search engines like Google (0.5s delay -> 20% traffic loss) where there are convenient, fast alternatives.

Preliminary search is a blend of semantic embeddings on 100M+ papers and keyword search, citation links, etc. Reasonably accurate, but full of noise for complex queries.

Show HN: Undermind – Deep scientific paper search with adaptive LLMs

https://www.undermind.ai/home/

We can now build drastically higher quality search because we can use LLMs in algorithms that mimic a human's systematic research process, instead of just roughly recommending results based on semantic embeddings or term frequency.We built a deep search LLM pipeline that takes a few minutes to carefully search all the scientific literature. You describe your complex goal, as you would to a...

( Read More... | 14 comments | 4 points | on HN )

Show HN: Deep search engine for science

https://www.undermind.ai/home/

Undermind is a deep, systematic search engine to discover incredibly complex topics in scientific papers.We optimized everything for accuracy and comprehensiveness. That’s the bottom line that matters to scientists, doctors, etc. (not compute cost/time required).How it works: You type in what you’re looking for, in all its complexity - whatever you need to solve your problem. Then, we find...

( Read More... | 1 comment | on HN )
Re:Show HN: Deep search of all ML papers on Wednesday January 24, 2024 @02:29PM (#39121611)
FYI system is a bit delayed because of traffic levels. May take a bit longer to generate results at the moment (usually takes ~10 min).
Re:Show HN: Deep search of all ML papers on Wednesday January 24, 2024 @02:12PM (#39121416)
Appreciate the advice. Re: timing, it's bottlenecked by the sequential nature of the search. To be comprehensive, we discover a few papers, and use that info to choose where to look more closely next.
Re:Show HN: Deep search of all ML papers on Wednesday January 24, 2024 @01:28PM (#39120855)
The goal is to be systematic and handle complex topics. ChatGPT + keyword search can't handle complex topics at all, and isn't systematic either.
Re:Show HN: Deep search of all ML papers on Wednesday January 24, 2024 @10:23AM (#39118391)
Here's an example report on: tokenization-free large language model architectures, which have been shown to achieve compute/accuracy tradeoffs comparable to or better than traditional token-based models https://app.undermind.ai/query_app/display_one_search/05f0b8...

Show HN: Deep search of all ML papers

https://app.undermind.ai/home/

Built an automated system to run a deep search of ArXiv and carefully find all the precise papers that exist on a complex topic.It's different from simple RAG because it searches, classifies, and adapts based on relevant papers it uncovers, and then continues until it finds every paper on a topic (trying to mimic the human research process). Benchmarked 10x higher accuracy and total retrieval...

( Read More... | 25 comments | 37 points | on HN )
Re: on Monday April 03, 2023 @12:49PM (#35427116)
Why don't we track scientific authorship trends like financial markets?

We should track how experienced scientists switch fields, since this is probably a strong signal for important science.

I tracked author movement between fields from all ArXiv papers over the last two decades. The results reveal some obvious trends (ie. the growth of AI), and can more generally provide valuable context for career choices.