vikrantrathore

On this topic wrote a small book https://insanai.github.io/aiexportcontrols/

You can also download offline PDF at: https://insanai.github.io/aiexportcontrols/books/en/open-sou...

It also shows using game theory that export control leads to overall failure of both the sides.

> There are many use cases where "bots" are there specifically to fulfill human needs, and yet we ban them.

Definitely you can use bots if they solve human problem, but then let them work on human scales. In Sibuna its based on PoW which does not mean bots are banned, its just they need to show proof of work which takes 20ms to 30ms, humans might not be able to perceive it even. But for bots who create 100's of copies and want to dig inside the content without care for bandwidth or compute costs of a small provider its fair trade to do some PoW before access.

>Worse, many of us outsource the control over who can access their website to a third party

This is the precise reason I created Sibuna as I mentioned earlier you can have it without much fuss and supports similar features as cloudflare and even includes a high performance console.

Many of this tools primary aim is to even the scales. If bot spends less energy its a cost for the provider. If bots spend same or more energy compared to provider who designed it for human consumption then its fair. Its about balancing the scales and for a small website, ecommerce or content provider who runs a small scale operation, its imperative to control the costs as they have limited compute, memory and bandwidth budget.

This is one of the motivation I created sibuna to be able to help them.

You can also look at https://github.com/insanai/sibuna, for it you can do PoW and can get access via an a Token. Also it has a wasm based solver for itself which works with users. You can also run it on local machine with wasm runtime no need of browser. Probably it can also help with cloudflare. I am the author of sibuna.

Great to see another way to do specification. I also created So created guidedog: https://github.com/insanai/guidedog for working with multiple AI providers, inspired by PEP, RFC and Oxide RFD.

I worked a lot of Python PEP which was adopted by rust and java later and working with RFC and RFC process since 1990's. Then later saw RFD by Oxide based on RFC. Learning from all of them created guidedog, the philosophy is if its good for human it is automatically very good for AI (most LLM are trained on it).

guidedog supports RestructuredText (since its specification) as core markup with support for markdown. Its modeled after sphinx but can work for documentation, discussions (what you call as Deliberation Records) and can be a static site generator. So you don't need 3 tools one single tool a cli which can do all three. For doing this I worked by porting docutils and then building own parser and compiler in rust.

This process so far worked extremely well for me as it is not just very good for AI, but provides complete control to humans. Also I like completely offline reading experience on paper. So this has first class pdf support.

If you look at repos in insanai many projects used this process with AI and worked well. Just one hard work this process needs for human agency is to really read each discussion properly and sometimes they can be long. But its a very small work which can help later with verification and even writing documentation for what's being built.

I do not use book for a context, book is the outcome of the development. I am still old school I feel any product should have good documentation and should be available offline, also personally I like physical things so many of them will print and read. I follow a rigorous process I used to follow since 1998, without AI, which is write something like PEP or RFC before starting work on things. So if you look at the whole project it has design records and each contains details. I read through the docs, even though I use AI from time to time. I used to write a lot of restructuredtext (RST) docs. So I wanted to move the whole pipeline to rust, I used to use latex for it earlier but then wanted to move to typst. So for it wrote another system which helps me with website, html, pdf. You can take a look if you like the project is https://github.com/insanai/guidedog, sibuna also I ported from typst pipeline to guidedog which uses rst markup. If you want to look at development part I especially use dsc (short for discussions) so if you type `guidedog dsc --help` to see the details.

Its funny, when I was reading part time parliament paper (paxos) and while learning zig and odin language created a pure paxos library and then the first application for that library was to build a multi-master sqlite. The two approaches I took are very different in case of zig I build zaxonlite (https://github.com/insanai/zaxonlite) and have a benchmark on it that includes rqlite. It uses WAL replication as strategy. The second attempt I tried in odin is sqlodin (https://github.com/insanai/sqlodin), in this the system does not use WAL but run sql query on each node, as result subset of sql query in sqlite are not supported.

This is what happens when an important position to head the Leading AI lab of the company is given to a startup marketing guy, who pretends to be an Lead AI scientist, whose biggest credential is to sale the same startup at exhorbitant valuation to the the same company. This is the price for embracing better marketers over real innovators.

In your bot detection systems I do not see open source system like anubis, safeline or sibuna (I am the author of sibuna). Sibuna is built as self-hosted alternative to cloudflare with console UI providing OWASP CRS, JA4 TLS signature from ingress, these are still necessary besides bot detection and cloudflare offer it as enterprise offering.

I have a smaller benchmark for them in my book at https://insanai.github.io/sibuna/. If you can provide your benchmarking code as open source will incorporate this report into sibuna book.

Yes name was anubis backwards and thats the only resemblance between the 2 projects and also I used anubis, safeline for benchmarks compparison. Each of the design records are in the repo if you want to see the evolution and algorithms used.

Its not a rewrite and is more inspired by https://github.com/chaitin/safeline. Wanted a web interface and a distributed waf similar to cloudflare as an alternative. It supports OWASP CRS, J4A TLS trrmination handling. Comes with console-ui and also the proof of work used a different algorithm which is proven to be resistant to quantum attacks.

In the spirit of open source thought about it and created sibuna (https://github.com/insanai/sibuna)

It works similar to cloudflare WAF but is an open source software. It can be on your own infrastructures even if its a $5 VPS.

The idea is same let the bot or AI agent do more work than it takes to serve that request. Also it needs to follow rules, similar to cloudflare enterprise web application firewalls. It was OWASP CRS and J4A TLS support.

You can check two projects on top of sqlite using paxos and implement reciprocal rank fusion (RRF) and distribution-based fusion score (DBSF) on top of sqlite with sqlite's FTS and sqlite-vec extensions.

Project zaxonlite: https://github.com/insanai/zaxonlite

Project sqlodin: https://github.com/insanai/sqlodin

Just you need to remember sqlite support single writer at a time so both needs to work with that limitation. PostgreSQL uses MVCC, so performance for concurrent writes from multiple clients might be better. Did not do benchmark against it so cannot say.

Initially I posted my project on ziggit forum. Respecting the rules, in my post I did mention use of LLM extensively on those projects. One of the post asked "If I used LLM", I did not get a chance to replay, right away my account was blocked on ziggit for 1 year. Thats ok I accepted it, I thought I should respect the community.

In the second incident I opened a small PR with an issue which I fixed for my own project, later I created a PR. I did understand that zig community do not accept LLM written post or PR on their official repository, so I went ahead and wrote the issue myself, also for PR I wrote the code, run the test locally on my own PC, tested if it compiles and pass the test on my osx and linux. Since it was my first PR, did not expect it to be accepted and thought may be some core developer will pick the code and merge or write another code.

Also I created one more issue which was closed by the core developer with a message its fixed in 0.17, I thanked the person.

I saw one issue from another person facing similar issue as mine. I give a comment and also put a code snippet for him to try on his own machine, since on my machine it compiled correctly and created windows cross compiled library on my linux machine. But then suddenly I cannot see my issues, my comments and PR. I initially thought issue with my own system it was not so. Then thought may be codeberg blocked it, but that was not the issue either. Then I spend significant amount of time going through forgejo documentation to find out that a person running a project can remove the person from a repo and their whole history and work will be gone from that repo. I have other repo on codeberg which worked fine with my account, so then tried to create new issue in zig repo to test if I am being blocked.

This is when I realized then my account has been banned by some zig core developer deliberately without a single message, including no message that my account has been blocked. This is like a tyranny of minority and community do not build like this.

A community requires a due and a transparent process even if its benevolent dictatorship. Since it happened twice in a span of just 2 weeks, it really felt like zig is not an open community and breeds strong disdain towards people working in AI or LLM. I then realized that I made too early decision on bun incident, where I took the side of the zig core team, now could understand the other side of the coin.

I have been following zig since it was started, just didn't take the plunge, but once I took the plunge and met hostile community, personally I thought I should move on. Still I believe in diversity of opinion so will still be happy to see zig thrive as a language because I liked its simplicity and also diversity breeds innovation.

Thanks for the reply alex. I send an email to andrew, but not to reinstate my account just to send the details of the incident before I move on.

I wish all the success to zig because diversity keeps the innovation alive.

The reason I moved on is, after 2 times consecutively facing the same issue, I felt I will be helpless in zig community and felt like my voice will be muzzled without any reason done in an opaque and abrupt manner. I prefer to err on the side of openness than caution.

I would understand if its by error but based on how it was banned it did feel deliberate. Also I have written email to core member before making a decision to leave the community. I think a community can only thrive when they are nice to humans. Here it felt like zig is worse then a oligopoly where few core members can block anyone without any reason and don't even have a courtesy to send a message. I worked with python community since its beginning, same worked with linux community since 1992, never seen such attitude even though both of the projects were driven by benevolent dictators, but they did care about the community around it. I still have one major work [1] in zig, which I will still release but after this won't work on zig. Zig community felt like a hostile ologipoly of few core members without proper due process, when it comes to treating humans.

[1] https://github.com/insanai/sibuna

It is not codeberg, I can work on other repository properly on it. Its only zig repository where my account is blocked. My initial thought was same as you, only later realized it. Also on ziggit these people did the same even though based on policy when I put the post i clearly mentioned use of LLM. This all without any proper communication. After working on zig for over 6 months, now I am moving slowly to odin [1] language. This decision I did not take lightly, I had spend significant resources in zig earlier and now moving away.

[1] https://odin-lang.org/

I would be fine if they reject the PR, given I was first time raising it to fix POSIX related issue. It was just few lines of code. But my account was banned all issues removed my own PR not visible and looked like tyrannical approach not a benevolent dictator approach. I think being nice to another human being is the basis for building community. Here felt like if you are among the few who dictate the terms there is no process just my way or highway.

This is from my own first hand experience both on ziggit forums as well as raising bugs in zig repository. My account on codeberg is still banned on zig repository without any single comment, my issues or pull request raised on that project just went in ether without being accessible, if you see the issue yourself could have made the decision if it was warranted to ban the account, but that right also taken by the core contributor out of spite and disdain for LLM.

I have been in open source world with linux kernel since 1992, never every had seen a community so hostile, especially towards humans who spend time and efforts just out of curiosity, inquisitiveness and trying to support some simple alternative when odds are already against them.

Facing hostile behavior from inside the community made me switch to odin language where they also do not use LLM but are pragmatic about people using it. Cannot comment on Mitchell because I do not contribute with money.

Congratulations to zig team. Good to know they are trying pragmatic approach to LLM now. I left the zig eco-system due to zig core members hostile behaviour toward humans not just LLM, so good to see the change they are becoming pragmatic. For me I am slowly porting the same work to odin language [1].

Background is I created issue and one pull request to fix them in zig compiler version 0.16.1 issue numbers 36812, 36811 (you cannot access them as my account is banned can see my fork at [2]). Respecting the community’s no AI stand. For these specific issues I wrote the issue and code myself and not let AI write it. Spend a lot of time on it. Subsequently without any notice my account was banned because my projects on github using zig uses LLM. This was done without message or any information. My account was banned on zig repository.

I can now understand the other side of coin how bun team might have been treated with disdain when they used LLM.

I wrote an email and left the zig community, have many work in zig but slowly moving them to odin.

I feel personal disdain should not be spilled on to people who are pragmatic on using LLM. I was a very big evangelist of zig for their no LLM stand and promoted them among my community, but with poor treatment by community I just left. You can still see projects I wrote in zig [3].

I have worked with postgreql community since 1997 and python community since 1998. Never felt such hostile community. So all the best and I wish zig continue its progress

[1] https://github.com/insanai/sqlodin

[2] https://codeberg.org/vyomtech/zig

[3] https://github.com/insanai/zenfmt

In some of your other writing can see use of emdash probably thats also intentional, emacs might have very special key combo for it to use unicode emdash instead of ASCII “-“. I commented about double dash (“--“) because because thats what LLM does when asked not to use emdash for writing.

“Another promising approach is to conduct on-site inspections regularly—say, every six months— to make sure chips are not just installed to fool inspectors and then removed and smuggled, as has happened before. However, conducting on-site inspections at this scale would likely require additional budget for export enforcement, or for US authorities to implement a third-party export auditor program.”