> But keep in mind our code gen is still on its infancy, and is nowhere as mature as SOTA compilers like GCC and GHC.
Second paragraph of Bend's GUIDE.md:
> While cool, Bend is far from perfect. In absolute terms it is still not so fast. Compared to SOTA compilers like GCC or GHC, our code gen is still embarrassingly bad, and there is a lot to improve. That said, it does what it promises: scaling horizontally with cores.
Limitations session on HVM2's paper:
> While HVM2 achieves near-linear speedup, its compiler is still extremely immature, and not nearly as fast as state-of-art alternatives like GCC of GHC. In single-thread CPU evaluation, HVM2, is still about 5x slower than GHC, and this number can grow to 100x on programs that involve loops and mutable arrays, since HVM2 doesn’t feature these yet.
Yeah exactly. I read most of the readme and watched the demo, but I'm not interested in installing it so I missed this. I would recommend moving this to the first section in its own paragraph.
I understand you might not want to focus on this but it's important information and not a bad thing at all.
We'll add the disclaimer before the install instructions instead!
I think you can avoid this kind of criticism by setting expectations better - just plastering a banner at the top saying that it’s in early stage development and not optimized, but that the future is bright, for example. The current headline saying it’s the “parallel future of computation” isn’t really enough to make people understand that the future isn’t here yet.
Same goes for the README, the fact that it’s not production ready per-se really ought to be at the top to set people’s expectations properly IMO, since a lot of people will not read the whole wall of text and just jump straight into trying it out once they’re on your GitHub page.
They’re critical since they are led to have much higher expectations than what actually exists today.
That said, this is a cool project and wish you the best in making it really good!
Based on how you've made a nice marketing page and README that sounds like you want people to actually use this tool in practice, within that context correctness is a minimum requirement/table stakes for a language to be usable at all, but that alone doesn't make it "production ready" if it fails to practically achieve anything you'd realistically want to do with it better than old-school languages that people already know how to use.
I am not a Python dev, but it seems that CPython's goal is not to be as fast as C, but just that it is a default runtime for Python [1] and the fact that C is in its name is just an implementation detail. Very different.
So the criticism leveled at the project appears to be valid.
And that's comparing a first-version interpreter against a SOTA runtime deployed in all browsers around the world and optimized by all major companies over 20+ years. If that's not useful to you, that's useful to me, which is why I wanted to share so it can be useful to more people.
If you add some disclaimer like "Note: Bend is currently focused on correctness and scaling. On an absolute scale it may still be slower than single threaded Python. We plan to improve the absolute performance soon." then you won't see these comments.
Also this defensive tone does not come off well:
> We published the real benchmarks, checked and double checked. And then you complained some benchmarks are not so good. Which we acknowledged, and provided causes, and how we plan to address them. And then you said the benchmarks need more evaluation? How does that make sense in the context of them being underwhelming?