brendanashworth
brendanashworth on Hacker News
An intriguing case of "exceptional resilience" against dementia
Sandboxing isn't about having the perfect devex environment, most dev sandboxes (looking at codex cloud) don't actually have full verification available. Sometimes Github actions CICD is all you can get!
Recursive self-improvement is possible for apps, too
Effects of Full-Fat and Fermented Dairy Products on Cardiometabolic Disease
Use of Bayesian Methodology in Clinical Trials of Drug and Biological Products
Salesforce Acquires Qualified
GCJ-02, China's "Mars Coordinates"
Do I Need Kubernetes?
Tech Model Railroad Club at MIT
Self-hosted slippy maps, for novices (like me)
Map of Palaeohispanic Coins and Inscriptions
Self-hosting HyperDX for fun and profit
Guelta
The Universal Transverse Mercator (UTM) geographic coordinate system
American Latitude
Greek Language Question
Undo executive order 14172 (OSGeo/GDAL)
The Online Guide to U.S. Map Collections
Ironic Process Theory
Two Parameter Model for Running Performance
Topographic Prominence (2017)
Use "instruction @web @Flask" to search the internet and Flask docs before answering, for example.
(1) The "baseline" comparison was to PyPDF + Naive RAG. For the LlamaParse evaluation, you appear to have used a different RAG pipeline, called "recursive retrieval." Why not use the same pipeline to demonstrate the improvement from LlamaParse? Can you share the code to your evaluation for LlamaParse?
(2) I ran the benchmark for the PyPDF + Naive RAG solution, directly copying the code on the linked LlamaIndex repo [1]
I got very different numbers: mean_correctness_score 3.941 mean_relevancy_score 0.826 mean_faithfulness_score 0.980
You reported: mean_correctness_score 3.874 mean_relevancy_score 0.844 mean_faithfulness_score 0.667
Notably, the faithfulness score I measured for the baseline solution was actually higher than that reported for your proprietary LlamaParse based solution.
[1] https://github.com/run-llama/llama-hub/tree/main/llama_hub/l...