superboum
But that's not true. Computers, including RAM, are used for many different things that do not involve AI at all. video games, office work, communications, access to culture, etc.
So, what Altman really says is "AI is the only thing that matter for computing, and thus, for the success of his company, we must allocate all the RAM to his company, and forbid any people from accessing it".
And, IMO, we should not accept that kind of discourse :s
I know many people will think that's business as usual, and that business are for profits and that's their only _raison d'être_, that winning the AI race is all what matters, and trusting any promise from a company/CEO/executive make you a fool. But not everyone think like that, and OpenAI/Anthropric/etc. are allowed to exist because many people expect some positive outcomes from their work. And a healthy society needs a bit more than "business as usual/any lie is ok as long as we are not caught" to work. And asking for the same level of responsibility/accountability as any other business would be, in my opinion, a good signal.
And the very first step could be, indeed, to ask them to be accountable here, and the question could be "hacking is about celebrating creativity with computers, creativity is what make life enjoyable, doing creative work is something we can enjoy, why do you want to kill it so much? Why are you so dismissive about people for which creativity is the core of their work? Copyright does not seem to be the target here, as the rent collector is the editor, not George R.R. Martin or any other author, so what's wrong with authors, painters, designers according to you? Similarly, you're only talking about making creative work disappear - not helping creative work in the way it was framed by Steve Jobs - "the computer is a bicycle for your mind". Could you frame your ideal society? What role does culture play in this society?"
I would really like to hear them on these topics. Sincerely.
[Tools do not exist in the void but in a society]. You can't study tools without studying the context that created it. Some tools work in some society, some others don't. One example, IIRC, are Ski-Doo with some native population that have Potlatch-like practices. Some anthropologist gave them Ski-Doo but you have to sacrifice something precious to give back, and soon they had to be burnt. In our society, anything expensive that require maintenance and repair is a bad fit as both "don't have the time" and the skills. We prefer "disposable" objects at the cost we know. That argument alone also explain why if both Switzerland and USA have the same amount of guns per inhabitant, they are less gun murder in Switzerland than in USA: different societies, with different culture, with different level of poverty, with different actors, etc.
[Tools create potentials, society may realize them]. As hackers, we often like to create new potentials (eg. with Bluesky ATProto, or the anonymous Vuvuzela chat, etc.). We also envisioned that with electricity, then with 3D printing, that people would build many objects of their daily life directly at home. But, from all the created technologies, some potentials are realized, some not. Community Memory, a San-Fransisco pre-Internet electronic board, wrote that more often the potentials that favor people in power in our society are realized. That's Palantir - strong - business model.
[Tools convey intentions]. I said above that tools create potentials. These potentials are not limited to what the tool actually is, but it should encompass all the micro-choices made by ones creating it. To illustrate this idea, take a knife. Knife designed to cook on one side, and knife designed for hunting in the other side, do not look the same. Youtube, by not displaying a big bar telling you how much you already uploaded to their server convey the idea of an infinite storage space, etc. Additionally to the raw utility of your tool, you can convey ideas, intentions, suggestion, affordance, either consciously or unconsciously. Back to guns in Switzerland & USA, both how people get the gun, what are the narrative, what the guns look like, whether or not you personalize it, etc. has a huge impact too.
[Our tools maximize efficiency at the expense of everything else]. Efficiency, in this case, could be defined as reaching a specific goal, in a stable & defined context. The critics of efficiency improvement are multiple: often, what really matter is to reach an acceptable threshold on many many different goals. Additionally, most efficiency gain make the process more and more dependent of a stable defined context, each micro-change in the environment and the whole tool / production chain is disrupted. It could be summarized as the opposition between "industry" and "craft". It's also one argument of the luddite: they also had tools to build cloth, that were less optimized in term of speed to build a cloth, and harder to master. But on other goals, that were not considered at that time, like quality, they were better. They thought the better quality of their cloth will save them, when people will find how poor quality are the cloths made by the new automated machine. But it did not happen. Still, from a French perspective, crafts still exist there; it's what luxury brand sells, because rich people know how much better they are. William Morris experimented and published a lot on these topics, it's an interesting reference.
With the 4 ideas (tools do not exist in the void, tools create only potentials, tools convey intentions, we build tools to maximize efficiency only), I think we have some basis to build a stimulating critic of both AI, cars, and knife. But this part is left as an exercise for the reader :-)
People interested in these topics may read Ivan Illich, André Gorz, or Jacques Ellul.
First bias is "perception", which is influenced by how media want to communicate and who own them - in France, mostly right wing billionaires. There is no actual correlation with actual, real, statistics.
Second, by who answer the survey - often people concerned about insecurity, which is a topic from the right.
Finally, there is no control on how many times you can vote, and some people demonstrated that with very few knowledge, you can completely change the results by sending thousands of vote [1].
The fact that Nantes is deemed highly insecure in France is also the consequence of this city being socialist and the place where police killed a guy named Steve during a party. So these attacks on Nantes being dangerous can also be interpreted as a backlash[2].
Please Hacker News, you're better than this, don't fall in this trap...
[1]: https://xcancel.com/dbertho/status/1574761634840592384 [2]: https://www.index.ngo/en/news/steve-maia-canico-trial-of-pol...
It seems the first "grid scale batteries" were derived from EV batteries, and are planned for 1 or 2 decades[1].
Basically, we are discussing battery ageing here, which is a complex problem[2].
According to the different studies on the topic I found, mentioning specifically "large-scale" installation like the ones discussed here, the answer is definitely and deceptively the same: between 10 and 20 years[3][5]. More precisely.
From [3]:
> To address the global effort to decrease carbon emissions, many consumers, corporations, and energy providers are adopting the use of electric vehicles and stationary energy storage systems paired with renewable electricity generation. These systems often utilize large-format lithium-ion batteries [...]. Real-world battery lifetime is evaluated by simulating residential energy storage and commercial frequency containment reserve systems in several U.S. climate regions. Predicted lifetime across cell types varies from 7 years to 20+ years, though all cells are predicted to have at least 10 year life in certain conditions.
From [5]:
> In the 2020 report, calendar life for both LFP and NMC Li-ion systems was stated as 10 years. The 2022 report takes additional information from long-term laboratory work (Saft, 2021) and product data into account (Baxter, 2021b) to establish new calendar lives of 16 years for LFP and 13 years for NMC. The calendar life is unchanged for 2030.
I also claim that battery are not renewable. One might argue that, if we can recycle batteries like we recycle regular glass, it could be considered as renewable. However, today there are 2 industrialized processes that are not satisfying (pyrometallurgical and hydrometallurgical processing) which "require high energy, and/or complex wet-chemistry steps"[4]. Some explored processes called "direct recycling"[4], which also has severe drawbacks but at least is more promising.
Which makes me think: we are, at least, making huge bets on the future here, as we risk 1) having huge amount of aged batteries in 1 or 2 decades, 2) no more mineral resources to extract.
[1]: https://www.quora.com/How-long-do-grid-storage-batteries-las...
[2]: https://cea.hal.science/cea-01791260/document
[3]: https://www.sciencedirect.com/science/article/abs/pii/S23521...
[4]: https://www.sciencedirect.com/science/article/pii/S2352152X2...
[5]: https://www.pnnl.gov/sites/default/files/media/file/ESGC%20C...
Don't get me wrong: the article is very interesting. I really learnt something: I discovered "system inertia", I was not aware of stability issues linked to inverters, and did not know about grid-forming & grid-following inverters, and the research about finding the minimal amount of grid-forming to keep a power-grid stable in case of an issue in a given power plant. All of these topics are very interesting.
But making a connection between inverters and ecology through the "net zero" terms seems either off topic, misleading or irrelevant. First because this "net zero" term is heavily criticized as it means carbon are still emitted but companies are paying for carbon credits (that are not compensating at all the carbon emitted for many reasons [1]). Here building solar panels, wind turbines & batteries emits CO2, and their lifespan is relatively short (at most 10 years for batteries, ~25 years for wind turbines & solar panels, compared to hundreds of years for a dam[7]). Second because climate change is not the only concern about ecology, there are concerning questions about mineral resource extraction, like lithium[2] that is heavily used in batteries, but more generally, we are already extracting the whole Mendeleev periodic table[3]: we don't have alternative mineral resources for batteries or other technologies, the only solution is to extract, produce & consume less. Third, if your only goal is to reduce carbon dioxide equivalent (eqCO2), you should advertise nuclear power plant as the solution. Depending on studies, they produce the same amount or less eqCO2 compared to a wind turbine without batteries[4]. Of course, often eqCO2 is not the only important subject here (being renewable/sustainable is also important, and uranium is a limited resource). And finally, the fact we use renewable energy more and more did not lead to a worlwide energy transition, but an addition. Having a transition will require way more than technologies[5], something that is also not discussed here.
Speaking about solutions to pack a higher percentage of Intermittent renewable energy sources (IRES)[6] in a power-grid through the help of batteries and inverters would have been more accurate in my opinion. Maybe "Why we were not able to achieve 100% renewable energy before?" if you want to be catchy, and it's not perfect, as you are still hiding that you rely on lot of batteries, that are far from being renewable.
As a conclusion, I would say it would be great to be careful when engineers (here IEEE) discuss specific technologies (here power-grid inverters) to not draw conclusion too quickly (having a positive environmental impact), as it's far from being obvious. I know they want to be read, I know that the title must be catchy to attract readers, but it's not an excuse as illustrated above.
[1]: https://demandclimatejustice.org/wp-content/uploads/2020/10/...
[2]: https://www.cnbc.com/2023/08/29/a-worldwide-lithium-shortage...
[3]: https://www.euchems.eu/euchems-periodic-table/
[4]: https://www.edfenergy.com/media-centre/news-releases/over-it...
[5]: https://www.sciencedirect.com/science/article/abs/pii/S22146...
[6]: https://en.wikipedia.org/wiki/Variable_renewable_energy
[7]: https://www.power-technology.com/data-insights/power-plant-p...
A Suggested-by: tag indicates that the patch idea is suggested by the person named and ensures credit to the person for the idea.
Please note that this tag should not be added without the reporter's permission, especially if the idea was not posted in a public forum.
That said, if we diligently credit our idea reporters, they will, hopefully, be inspired to help us again in the future.
It could have been more appropriate to the situation, I think it's convey better the idea that you have found a solution to a problem, but because you are not familiar with the project, the exact syntax of your patch has not been kept.Ref: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
My idea when I installed Haiku was to make my own version of the "old computer challenge"[1], with an emphasis on using GUI apps.
Similarly to @probono (a FOSS dev), I also found Haiku "shockingly good"[2] at being a lightweight, responsive, easy-to-use desktop OS.
After some patching, I was even able to compile Tectonic[3], a modern LaTeX engine written in Rust, and Quaternion a Matrix client supporting E2EE[4]. All that running on a single core Athlon 64 and 1.5GB of RAM.
I posted some screenshots in a Mastodon threads if you are curious[5] (but my posts are in french sorry :/). And of course this comment is posted from Haiku!
[1]: https://dataswamp.org/~solene/2021-07-07-old-computer-challe...
[2]: https://medium.com/@probonopd/my-first-day-with-haiku-shocki...
[3]: https://tectonic-typesetting.github.io/en-US/
[4]: https://github.com/quotient-im/Quaternion
[5]: https://mastodon.tedomum.net/@tgoldoin/109554115997967651
I think an interesting comparison, for both Linen and Matrix, would be to compare these 2 approaches: Linen natively indexed conversation and this Matrix "static" client. I would be especially interested by what additional features Linen provides in term of indexing compared to this static client.
[1]: https://github.com/matrix-org/matrix-static [2]: https://view.matrix.org/
I also work on a Rust IMAP server that is far from being as feature complete as yours. I also chose your `mail-parser` library to parse RFC822/5822, but we observed that in many cases, we did not have enough information to build some BODY/BODYSTRUCTURE responses. We also discovered that line count and many details are not very obvious on IMAP, did you run some tests to compare your IMAP server outputs to existing servers? Or, more generally, what is your approach to ensure compatibility / integration with the existing email ecosystem?
In any case, congratulation for your project, we will follow it closely! I experimented how big these protocols became with all their extensions, this is an impressive work!
On the plus side, it survived Hacker News Hug of Death. Indeed, the website we linked is hosted on our own Garage cluster made of old Lenovo ThinkCentre M83 (with Intel Pentium G3420 and 8GB of RAM) and the cluster seems fine. We also host more than 100k objects in our Matrix (a chat service) bucket.
On the minus side, this is the first time we have so much coverage, so our software has not yet been tested by thousands of people. It is possible that in the near future, some edge cases we never triggered are reported. This is the reason why most people wait that an application reaches a certain level of adoption before using it, in other words they don't want to pay "the early adopter cost".
In the end, it's up to you :-)
Due to our design choice, you can add and remove nodes without any constraint on number of nodes and size of the storage. So you do not have to overprovision your cluster as recommended by MinIO[0].
Additionally, and we planned a full blog post on this subject, adding or removing a node in the cluster does not lead to a full rebalance of the cluster. To understand why, I must explain how it works traditionally and how we improved on existing work.
When you initialize the cluster, we split the cluster in partitions, then assign partitions to nodes (see Maglev[1]). Later, based on their hash, we will store data in its corresponding partition. When a node is added or removed, traditional approaches rerun the whole algorithm and comes with a totally different partition assignation. Instead, we try to compute a new partition distribution that minimize partitions assignment change, which in the end minimize the number of partitions moved.
On the drawback side, Garage does not implement erasure coding (as it also the reason of many MinIO's limitations) and duplicate data 3 times which is less efficient. Garage also implements less S3 endpoints than Minio (for example we do not support versioning), the full list is available in our documentation[2].
[0]: https://docs.min.io/minio/baremetal/installation/deploy-mini...
[1]: https://www.usenix.org/conference/nsdi16/technical-sessions/...
[2]: https://garagehq.deuxfleurs.fr/documentation/reference-manua...
When failures occur, repair is done through workers that says when they launch, when they repair chunks, and when they exit in the logs. We also have `garage status` and `garage stats`. The first command displays healthy and non healthy nodes, the second one displays the queue length of our tables and chunks, if their values are greater than zero, we are repairing the cluster. We are documenting failure recovery in our documentation: https://garagehq.deuxfleurs.fr/documentation/cookbook/recove...
For the near future, we plan to integrate opentelemetry. But we are still discussing the design and information we want to track and report. We are currently discussing these questions in our issue tracker: https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/111 https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/207
If you have some knowledge/experience on this subject, feel free to share it in these issues.
I just deleted the AAAA entry for this machine. In the meantime, if the result is cached for you, you can pass the `-4` argument to force IPv4:
git clone -4 [email protected]:Deuxfleurs/garage.git
And, in a second time, we will work on a better/working IPv6 configuration for our Git repository and all of our services (we use Nomad+Docker and did not find a way to expose IPv6 in a satisfying way yet).For a Raft-based object store, your gateway will receive the write request and forward it to the leader (+ 100ms, 2 messages). Then, the leader will forward in parallel this write to the 9 nodes of the cluster and wait that a majority answers (+ 100ms, 18 messages). Then the leader will confirm the write to all the cluster and wait for a majority again (+ 100ms, 18 messages). Finally, it will answer to your gateway (already counted in the first step). In the end, our write took 300ms and generated 38 messages over the cluster.
Another critical point with Raft is that your writes do not scale: they all have to go through your leader. So on the writes point of view, it is not very different from having a single server.
For a DynamoDB-like object store (Riak CS, Pithos, Openstack Swift, Garage), the gateway receives the request and know directly on which nodes it must store the writes. For Garage, we choose to store every writes on 3 different nodes. So the gateway sends the write request to the 3 nodes and waits that at least 2 nodes confirm the write (+ 100ms, 6 messages). In the end, our write took 100ms, generated 6 messages over the cluster, and the number of writes is not dependent on the number of (raft) nodes in the cluster.
With this model, we can still provide always up to date values. When performing a read request, we also query the 3 nodes that must contain the data and wait for 2 of them. Because we have 3 nodes, wrote at least on 2 of them, and read on 2 of them, we will necessarily get the last value. This algorithm is discussed in Amazon's DynamoDB paper[0].
I reasoned in a model where there is no bandwidth, no CPU limit, no contention at all. In real systems, these limits apply, and we think that's another argument in favor of Garage :-)
> Why not using Riak and adding an S3 API around it.
Riak was developed by a company named Basho that went bankrupt some years ago, the software is not developed anymore. In fact, we do no need to add an S3 API around Riak KV, Basho even released "Riak Cloud Storage"[0] that exactly does this: provide an S3 API on top of Riak KV architecture. We plan to release a comparison between Garage and Riak CS, Garage has some interesting features that Riak CS does not have! In practice, implementing an object store on top of a DynamoDB-like KV store is not that straightforward. For example, Exoscale, a cloud provider went this way for their first implementation of their KV store, Pithos[1], but rewrote it later as you need special logic to handle your chunks (they did not publish Pithos v2).
> Most apps don't have S3 support
We are maintaining in our documentation an "integration" section listing all the compatible applications. Garage already works with Matrix, Mastodon, Peertube, Nextcloud, Restic (an alternative to Borg), Hugo and Publii (a static site generator with a GUI). These applications are only a fraction of all existing applications, but our software is targeted at its users/hosters.
> A distributed system is not necessarily highly available
I will not fight on the wording: we come from an academic background where the term "distributed computing" has a specific meaning that may differ outside. In our field, we define models where we study systems made of processes that can crash. Depending on your algorithms and the properties you want, you can prove that your system will work despite some crashes. We want to build software on these academic foundations. This is also the reason we put "Standing on the shoulders of giants" on our front page and linking to research papers. To put it in a nutshell, one critic we address to other software is that sometimes they lack theoretical/academic foundations that lead to unexpected failures/more work to sysadmins. But on the theoretical point, Basho and Riak were exemplary and a model for us!
[0]: https://docs.riak.com/riak/cs/2.1.1/index.html [1]: https://github.com/exoscale/pithos
- Garage is easier to deploy and to operate: you don't have to manage independent components like the filer, the volume manager, the master, etc. It also seems that a bucket must be pinned to a volume server on SeaweedFS. In Garage, all buckets are spread on the whole cluster. So you do not have to worry that your bucket fills one of your volume server.
- Garage works better in presence of crashes: I would be very interested by a deep analysis of Seaweed "automatic master failover". They use Raft, I suppose either by running an healthcheck every second which lead to data loss on a crash, or sending a request for each transaction, which creates a huge bottleneck in their design.
- Better scalability: because there is no special node, there is no bottlenecks. I suppose that with SeaweedFS, all the requests have to pass through the master. We do not have such limitations.
As a conclusion, we choose a radically different design with Garage. We plan to do a more in-depth comparison in the future, but even today, I can say that if we implement the same API, our radically different designs lead to radically different properties and trade-off.
The goal is to say, as an individual:
- I am not ok anymore that so much sensitive data are collected
- I know data collection had negative impacts on individuals and society
- I can, and we should live without collecting so much data
- Individuals and society should come before companies
And I definitely relate...Using these technologies instead of OP centralized proposition would improve reliability, scalability, security and sustainability : currently hostyoself.com returns a 502 bad gateway.
A Federated Alternative to Facebook Events
[1] https://openfisca.org/en/ [2] https://openfisca.org/doc/index.html
* Seafile (file hosting/synchronization/sharing/history)
* SoGo (a webmail that work with existing IMAP and SMTP servers and exposes an Exchange API)
* Matrix / Riot (matrix is a chat server, Riot is the web client)
* Jenkins 2 (Continuous Integration / Continuous Deployment)
* FreshRSS (RSS aggregator)
Authentication is managed by a single LDAP service (openldap).
I also plan to test/deploy:
* Peertube (video hosting)
* Mastodon (micro-blogging)
My next goal would be to distribute this stack on more than one server, in order to improve availability.
I'm testing Open Xchange for a few weeks now, and I really like it. There are lot of small cool features: remind an email in x minute, meeting organization, OX Guard (a PGP module), the integration with Sieve, etc.
And that's not a secret, such solutions are not easy to deploy. I've spend a certain amount of time writing my ansible script for Open Xchange - that's why I'm not really motivated to try it on my server now.
If I find something in OX which prevents me from switching, SOGO will be the next candidate.