News for hackers. Stuff that matters.
Re:RWKV: Reinventing RNNs for the Transformer Era by ftxbro on Monday May 22, 2023 @10:28PM (#36039099)
I think it's just, everyone in their discord channel.
Re:RWKV: Reinventing RNNs for the Transformer Era by Buttons840 on Monday May 22, 2023 @10:38PM (#36039144)
What channel? I have an application for sequence models I think might be novel, and I'd like to be able to get credit and help research it if possible. Probably somebody has already done it, but I cannot search well enough to find related literature.
Re:RWKV: Reinventing RNNs for the Transformer Era by ftxbro on Monday May 22, 2023 @10:45PM (#36039188)
Write it and submit it yourself. Sole authorship is given more weight these days for whatever reason. In a multi author publication, even if you are first author or listed as equal contribution, if there is a more famous person on the paper everyone will assume they did it.
Re:RWKV: Reinventing RNNs for the Transformer Era by Buttons840 on Monday May 22, 2023 @11:34PM (#36039478)
Write it and submit it to who? I'm not familiar enough with the field to find any prior work or related work.
Re:RWKV: Reinventing RNNs for the Transformer Era by ftxbro on Monday May 22, 2023 @11:39PM (#36039526)
find the most closely related paper that you know of, even if it's not very close, and submit your idea to the same journal that published that other idea
Re:RWKV: Reinventing RNNs for the Transformer Era by rahimnathwani on Monday May 22, 2023 @11:13PM (#36039351)
Re:RWKV: Reinventing RNNs for the Transformer Era by stellaathena on Monday May 22, 2023 @11:25PM (#36039433)
Our discord servers (a primary one, a spin-off for RWKV, another spinoff for BioML) have tens of thousands of people between them :) So not quite everyone. But this was a community effort with a public call for contributions
Re:RWKV: Reinventing RNNs for the Transformer Era by ftxbro on Tuesday May 23, 2023 @12:05AM (#36039687)
Are you the Stella from the Acknowledgements section? Are you in one of those spinoffs like BioML, or you aren't on the author list because you have some conflict with your work where you can't legally do something because of intellectual property laws or license things or NDA? Also when you talk about 'our discord servers' which ones do you mean? Is it ones run by Eleuther?
Re:RWKV: Reinventing RNNs for the Transformer Era by stellaathena on Tuesday May 23, 2023 @12:42AM (#36039901)
Yes I am (apparently) in the acknowledgments. I was not an author on the paper because I didn’t have time to contribute too much. I also try to err on the side of not being added to papers, as my position (I run EleutherAI) tends to encourage people to be overgenerous with offers. I anticipate having more time this coming month and being on the version that’s submitted for peer review, but we’ll see.

BlinkDL has been working on this project for two-ish years, originally in the EleutherAI discord and then created his own to house the project.

I wasn’t thinking too hard about my exact wording, but yes I was thinking of EleutherAI and its various spin-off servers. EleutherAI doesn’t /run/ any of the other servers, but we all have a close collaborative relationship. I’m sure there’s a lot of duplication of membership (e.g., I’m in all of them) but quickly adding up the membership of each server comes out to around 70,000. EleutherAI and LAION are the largest at 25k each, with the others typically having around 5k each. I would expect at least 30k of those users to be unique though.

Re:RWKV: Reinventing RNNs for the Transformer Era by ftxbro on Monday May 22, 2023 @11:51PM (#36039599)
I'm a boomer and I never used discord except to go to a guild channel for an online game like hello kitty island adventure. If I start going to other discords where I might start participating more professionally, are they all going to see my history as a hello kitty island adventure guildie or can I undo that somehow? I heard they are all linked together for some reason. Also I saw some news headline where discord was exposing some private stuff of its users. Should I be afraid of it? I'm from the days before anyone was supposed to give any personal information to the internet.