tom_hartke
tom_hartke on Hacker News
A decent fraction of research is open access these days (it's highly dependent on the field, though). That fraction is also growing. We're building systems to access full texts whenever possible.
That makes sense. You should be able to right click - save as PDF and it will preserve the links, etc.
We'd love to talk and see if we can provide this. Seems like it would be really helpful. Can you email us (
[email protected])?
Hadn't seen that. Very complementary. We don't address staying up to date or pull in community value metrics (other than total citations). It's a somewhat different goal to broadly gather emerging ideas and stay informed.
Can you be more specific? Like break it out into AND and OR statements? Or just more iteration back and forth? We find people more familiar with the system learn better strategies than the LLM can suggest.
If it's easy/fast to check the literature, hopefully people's instinct changes. If we had time, we should be doing that for any small idea.
Just save the report url! It will persist.
Also, on the usual system if you're logged in (instead of the HN free link), all your searches automatically get saved on a "history" page.
I think we associate learning/discovery with those moments because they happen together, not because they're causally related.
I think this is somewhat equivalent to how we used to have to learn 100 different integrals and derivatives in calculus. That's somewhat helpful. I learned to see patterns in math like that, the same way I learned a decent bit by browsing irrelevant abstracts and follow citation trails. But physically memorizing 100s of integrals is mostly a waste, and so are the irrelevant abstracts. You'll be much better at math (and hopefully science) if you can learn ~10 key integrals or read ~10 abstracts, and then spend the rest of your time understanding the high level patterns and implications by talking with an expert. Just like I can now ask GPT-4 to explain why some integral formula is true, which ones are related, and so on.
And that's the last point - these literature search tools aren't developing in isolation. We will get to have a local "expert" to discuss what we find with. That changes the cost-benefit analysis too.
Thanks! That's helpful to hear. Honestly just did numbers because the LLM has no trouble remembering which is which, and it's easier to programmatically parse out the citations to build hyperlinks (compared to names/years, where little variations creep in).
Emphasis is meant to be on "mind", like supporting better reasoning, but point taken.
Not a bad idea. Want to avoid too much complexity in pricing, though. Decision fatigue.
Latency, compute required, and lack of full texts (paywalled publisher content).
In short, yes, though it's geared toward topic search.
From a strategy perspective, we designed it for topic search because it makes more sense to find everything on a topic first, then filter for the most recent, if recent is what you want. That's because there is a lot of useful information in older articles (citation connections, what people discuss, and how), and gathering all that helps uncover the most relevant results. Conversely if you only ever filtered on articles in the last year, you might discover a few things, but you wouldn't have as much information to adapt to help the search work better.
So, you can ask for articles on coffee (though ideally it should be something a bit more specific, or there will be thousands of results). Our system will carefully find all articles, then you can filter for 2024 articles or look at the timeline.
The few minute time delay is primarily because of the sequential LLM processing steps by high quality LLMs, not database access times. The system reads and generates paragraphs about papers, then compares them, and we have to use the highest quality LLMs, so token generation times are perceptible. We repeat many times for accuracy. We find it's impossible to be accurate without GPT-4 level models and the delay.
Ours is slow, but accurate, even for complex topics. The rest are fast, but generally can't handle complex topics. (There's more nuanced explanations in other comments)
It's worth highlighting that first result is
exactly what you asked for, given all 4 of your criteria:
1. It's on adults.
2. It's longitudinal over multiple years.
3. It studies variations in brain volume.
4. It focuses on healthy individuals.
You can see the full results for that search text here: https://undermind.ai/query_app/display_one_search/e1a3805d35...
Semantic Scholar seems more focused on
1. being the data provider/aggregator for the research community, and
2. long term, I think they plan to develop software at the reading interface that learns as a researcher uses it to browse papers (a rich PDF reader, with hyperlinks, TLDRs, citation contexts, and a way to track your interactions over time, and remind you of what you've seen or not).
Their core feature now is a fast keyword search engine, but they also have a few advanced search features through their API (https://api.semanticscholar.org/api-docs/) like recommendations from positive/negative examples, but neither KW search nor these other systems are currently high enough quality to be very useful for us.
FYI our core dataset for now is provided by Semantic Scholar, so hugely thankful for their data aggregation pipeline and open access/API.
I think the biggest difference is our focus on search quality, and being willing to spend a lot on compute to do it, while they focus on systematic extraction of data from existing sources and on being fast. It's a bit of an oversimplification (they of course have search, and we also have extraction).
Feature-wise, we definitely have a lot of work to do :) What crucial pieces do you think we're missing?
Potentially. Given the latency and the cost/compute we put into each result, it doesn't fit the usual API mechanics.
What use case are you thinking of?
Agreed, exa is great - particularly, it's the best thing I've found for fast web retrieval of slightly more complex topics than Perplexity, Google, etc can handle.
I ran these two example searches we have on our homepage on AnswerThis:
(3D ion shuttling)
https://undermind.ai/query_app/display_one_search/b3767fb7b6...
(laser cooling to BEC)
https://undermind.ai/query_app/display_one_search/c5f77f862a...The results from their website aren't sharable, but their lists of references do not seem relevant (ie. they miss the fact that shuttling needs to be in 3D, and the list of experiments for laser cooling to BEC is missing all of the relevant papers).
I think, like other research tools, they're more focused on the summarization/extraction of information, rather than the discovery process (though they are similar to us in the way they say they do multi-stage retrieval and it takes some time).
For a meta-analysis, you might want to try the "extend" feature. It sends the agent to gather more papers (we only analyze 100 carefully initially), so if your report might say "only 55% discovered", could be useful.
(Also, if you want, you can share your report URL here, others will be able to take a look.)
A few possibilities:
- We only use abstracts for now. Have to make sure you ask for something present there.
- Did you ask for a scientific topic? (Sometimes people ask for papers by a specific author, journal, etc. The system isn't engineered to efficiently find that).
Regarding citations: we use them, but only for figuring out which papers to look at next in the iterative discovery process, not for choosing what to rank higher or lower at the end (unless you explicitly ask for citations). It's ranking based on topic match.
If you're comfortable, posting the report URLs here can let us debug.
We're trying to bias the system toward more autonomous execution, rather than a "copilot"-like experience where you iterate back and forth with the system. That lets us run more useful subroutines in parallel in the backend, as long as you specified your complex goal clearly.