News for hackers. Stuff that matters.
Re:Leveraging machine learning to fuel new discoveries with ... by iandanforth on Wednesday August 05, 2020 @11:27AM (#24061878)
This is really cool. I worked on the AI Index Report and had to bug karpathy to get his copy of arxiv papers to do analysis. (He's been slowly collecting papers for arxiv sanity for years). Getting them all from the arxiv API would have taken months.

This will enable tons of useful stats gathering about the fields represented on arxiv. Hopefully it will also lead to new scientific insights as well!

Re:Leveraging machine learning to fuel new discoveries with ... by newman8r on Wednesday August 05, 2020 @12:18PM (#24062527)
arXiv also offers bulk access to the papers, you can download them via S3 'requester pays'

https://arxiv.org/help/bulk_data

Re:Leveraging machine learning to fuel new discoveries with ... by etaioinshrdlu on Wednesday August 05, 2020 @12:05PM (#24062345)
Did you process the full text of the papers?