News for hackers. Stuff that matters.
Re:RWKV RNN: Better than ChatGPT? by textninja on Friday March 24, 2023 @12:16AM (#35285027)
The moat between OpenAI and projects like this is access to expensive computing resources, an organizational mandate to invest in aforementioned compute, and finally, access to high quality training data. They were first to market so they’re only getting further ahead thanks to the invaluable real world usage data.

I don’t think their moat ever had anything to do with innovative architecture. If anything, projects like this will widen the chasm since it’s easier for OpenAI to implement new architecture in their models than it is for an independent researcher to scale their ideas.

If projects like this are seeds then the problem is that OpenAI owns all the land.

Re:RWKV RNN: Better than ChatGPT? by pffft8888 on Friday March 24, 2023 @01:12AM (#35285320)
That's a bit defeatest, don't you think?

OpenAI is not the end all and be all, and projects like this will only inspire more challengers. OpenAI copying this architecture will delegitimize them as hardcore innovators, whom they're not.

I see this is a credible challenge because it most certainly is.

Re:RWKV RNN: Better than ChatGPT? by textninja on Friday March 24, 2023 @01:26AM (#35285378)
I agree, I sounded more defeatist than I intended. No doubt the moat is expansive, but it’s not impossible to cross. At least until the regulators get involved. [I did it again, didn’t I.]
Re:RWKV RNN: Better than ChatGPT? by d0mine on Friday March 24, 2023 @03:07PM (#35293989)
regulators exists so that big guys won't be worried that small guys will out innovate them. If somebody asks for regulation, they became too big to innovate themselves.
Re:RWKV RNN: Better than ChatGPT? by lallysingh on Friday March 24, 2023 @08:40AM (#35288243)
I don't think so. OpenAI doesn't do anything but AI. You can't ask GPT to actually do anything for you but generate output.

Siri, the crap it is, can actually do things for you. Alexa can do things for you.

The real value of this system is as a control interface and OpenAI has nothing to control.

Re:RWKV RNN: Better than ChatGPT? by fnordpiglet on Friday March 24, 2023 @09:22AM (#35288699)
But Microsoft does.
Re:RWKV RNN: Better than ChatGPT? by flangola7 on Friday March 24, 2023 @09:37AM (#35288896)
Did you see the plugins announcement?
Re:RWKV RNN: Better than ChatGPT? by lallysingh on Saturday March 25, 2023 @08:05PM (#35308971)
Yeah (after my comment) but the big players will compete instead of doing plug-ins.