News for hackers. Stuff that matters.
Re:PostgreSQL reconsiders its process-based model by abhibeckert on Monday June 19, 2023 @07:41PM (#36397841)
Reminds me of PHP 6...

For those who don't follow PHP closely - that version was an attempted refactor of the string implementation which essentially shut down nearly all work on PHP for a decade, stagnating the language until it became pretty terrible compared to other options. They finally gave up and started work on PHP 7 which uses the (perfectly good) PHP 5 strings.

Ten years of wasted time by the best internal PHP developers crippled the project - I'm amazed it survived at all.

Re:PostgreSQL reconsiders its process-based model by progmetaldev on Monday June 19, 2023 @08:18PM (#36398126)
I've used PHP in the past (PHP 4 and 5), as well as some simple templated projects in PHP 7. I try to keep up on news with what is happening in the PHP world, and it's difficult because of the hate for the language. Is the solution to Unicode strings still to just use the "mb_*" functions?

I got my real professional start using PHP, and have built even financial systems in the language (since ported to .NET 6 for my ease of maintenance, and better number handling). I'm still very interested in the language itself, in case I ever have the need to freelance or provide a solution to a client that can't afford what I can build in .NET (although to be honest, at this point I'm roughly able to code at the same speed in .NET as in PHP, but with the added type-safety, although I know PHP has really stepped up in providing this).

Re:PostgreSQL reconsiders its process-based model by Hayvok on Tuesday June 20, 2023 @12:52AM (#36399873)
I believe so - most (all?) string functions have an mb_ equivalent, for working on multibyte strings.

Regular PHP strings are actually pretty great, since you can treat them like byte arrays. Fun fact: PHPs streaming API has an “in-memory” option and it’s… just a string under the hood.

Just don’t forget to use multibyte functions when you’re handling things like user input.

Re:PostgreSQL reconsiders its process-based model by robomc on Monday June 19, 2023 @08:53PM (#36398394)
I have the "Professional PHP6" book which I feel like should be a collectors item or something.

Weird book IMO, because it has a lot of content that's just about general software development, rather than anything to do with PHP specifically, or the theoretical PHP6 APIs in particular.

Re:PostgreSQL reconsiders its process-based model by pmontra on Tuesday June 20, 2023 @01:23AM (#36400055)
PHP used to be the first computer language learned by people wanting to create a scripted web page. This was more true in the 90s but maybe it stuck. So it would be OK to add some general guidance about writing software and organizing projects.
Re:PostgreSQL reconsiders its process-based model by SoftTalker on Monday June 19, 2023 @09:35PM (#36398678)
Things You Should Never Do

https://www.joelonsoftware.com/2000/04/06/things-you-should-...

An oldie but a goodie

Re:PostgreSQL reconsiders its process-based model by IgorPartola on Tuesday June 20, 2023 @12:04AM (#36399569)
I think this is a great article that takes a maximalist point and that’s its flaw.

You should rewrite code only when the cost of adding a new feature (one that is actually necessary) to the old codebase becomes comparable to designing your entire system from scratch to allow for that feature to be added easily. That is to say that the cost of the rewrite should become comparable to the cost of continuing development. I have been a part of a couple of rewrites like that, one of them quite complex, and yes they were warranted and yes they worked.

But having said that you should absolutely be conservative with rewriting code. It’s a bad habit to always jump to a rewrite.

Re:PostgreSQL reconsiders its process-based model by devjab on Tuesday June 20, 2023 @01:07AM (#36399958)
I think it’s very dependent on how you use words like “rewrite” or “refactor”. The point the author makes about the two page function, and all the bug-fixes (lessons learned) makes sense only if you “rewrite” from scratch without looking at the history. You can absolutely “rewrite” the function in a manner that is “refactoring”, but will often get called “rewrite” in the real world. This may be because “refactor” is sort of this English CS term that doesn’t have a real translation or usage in many languages and “rewrite” is sort of universal for changing text, but in CS is sort of “rebuilding” things.

I don’t think you necessarily need to be conservative about rewriting things. We do it all the time in fact. We build something to get it out there and see the usage, and then we build it better and then we do it again. Which often involves a lot of “rewriting” but thanks to principles like SOLID’s single responsibility makes this rather easy to both do and maintain (we write a lot of semi-functional code and try to avoid using OOP unless necessary, so we don’t really use all the parts of SOLID religiously).

I do agree that it’s never a good idea to get into things with the mind-set of “we can do this better if we start from scratch” because you can’t.

Re:PostgreSQL reconsiders its process-based model by giraffe_lady on Tuesday June 20, 2023 @05:27AM (#36401600)
There's currently a trend towards shitting on microservices-everything, imo largely justified. But missing from that is that identifying a logical feature and moving it to a microservice is one of the safer ways to begin a gradual rewrite of a critical system. And usually possible to get cross-dept buyin for various not always wholesome reasons. It may not always be the best technical solution but it's often the best political one when a rewrite is necessary.
Re:PostgreSQL reconsiders its process-based model by carlmr on Wednesday June 21, 2023 @02:39AM (#36415164)
>identifying a logical feature and moving it to a microservice is one of the safer ways to begin a gradual rewrite of a critical system

Why not identify that same logical feature and move it into a library. How does a Microservice add value here?

Identifying, extracting and adding tests to logical features has been the sane way to rewrite software for ages. Michael feathers even wrote a book about it [1]. This ship of theseus approach works because it's incremental and allows for a complete rewrite without ever having non-functional software in between.

Adding a REST/networking/orchestration boundary to make it a Microservice just for the sake of extracting it adds a lot of complexity for no gain.

Microservice can be the right architecture, but not if all you want is to extract a library.

[1] https://understandlegacycode.com/blog/key-points-of-working-...

Re:PostgreSQL reconsiders its process-based model by pphysch on Tuesday June 20, 2023 @12:08AM (#36399596)
The issue here isn't "rewriting" per se but "stopping development".

You shouldn't stop development on your important products.

Letting a couple of your talented programmers loose on a greenfield reimplementation is a perfectly sane strategic move.

Stopping development on important products because you are 100% certain that the reimplementation will be successful by $DEADLINE is a foolish gamble.

Re:PostgreSQL reconsiders its process-based model by paulddraper on Tuesday June 20, 2023 @12:32AM (#36399745)
The problem is if it snowballs
Re:PostgreSQL reconsiders its process-based model by pphysch on Tuesday June 20, 2023 @12:53AM (#36399876)
Any project, whether it's a "rewrite" or not, can succumb to scope creep and poor project management.
Re:PostgreSQL reconsiders its process-based model by devjab on Tuesday June 20, 2023 @01:09AM (#36399969)
Isn’t that how you end up with Python 2 and 3 though?
Re:PostgreSQL reconsiders its process-based model by wiz21c on Tuesday June 20, 2023 @02:54AM (#36400591)
Python3 changes were many "little" things; some more fundamental than other (unicode str). So I guess they were able to split the work in tiny pieces and, ultimately, were able to manage the project...
Re:PostgreSQL reconsiders its process-based model by pphysch on Tuesday June 20, 2023 @01:09PM (#36407165)
Yes, that was a rough migration process, but the long-term result is we have an improved language and growing community instead of Python going the way of PHP and Perl.

Between the 3 P's, Python's strategic decisions in 2000s were clearly the most successful.

And it wasn't a total rewrite.

Re:PostgreSQL reconsiders its process-based model by hurril on Tuesday June 20, 2023 @03:40AM (#36400854)
The big problem there is that the people you are letting loose on the alternative, are lost from the original, so O loses steam that A gains. You still have to produce bug fixes and features to _both_ O and A to keep them in sync. So you essentially have a doubled required production rate to be delivered using the same staff.

So in order for there to be a net gain, the gang working on the alternative have to be able to find such big wins as to being neigh impossible.

This is a very very hard problem in our domain. 99% of the time, we have to simply resist the urge to _just rewrite the sucker_. No! Don't do it! (And this is incredibly hard because we all want to.)

Re:PostgreSQL reconsiders its process-based model by pphysch on Tuesday June 20, 2023 @01:13PM (#36407221)
Getting some Mythical Man Month vibes here. Productivity isn't a zero-sum game.
Re:PostgreSQL reconsiders its process-based model by hurril on Friday June 23, 2023 @04:54AM (#36444401)
I'm saying that the productivity gain would have to be incredibly large since it has to encompass a doubled output of features and bug fixes for a net zero change.

Say you have product A with features P, Q, R, and S; writing product B has to reproduce P, Q, R, and S, plus X and Y that is currently produced by the A team. On top of that, it has to fix (conceptual) bugs within P, Q, R, and S. All this is to be done by the new, crack, team that aims to make it so that: cost-of-development(B) < cost-of-development(A).

But the point is that the difference in magnitude of cost-of-development(B) and cost-of-development(A) has to be rather large considering the amount of work needed to have a return on that investment at all.

Re:PostgreSQL reconsiders its process-based model by h0l0cube on Tuesday June 20, 2023 @12:16AM (#36399638)
> Well, yes. They did. They did it by making the single worst strategic mistake that any software company can make:

> They decided to rewrite the code from scratch.

Absolutely not what's being proposed for Postgres.

Re:PostgreSQL reconsiders its process-based model by yxre on Tuesday June 20, 2023 @12:24AM (#36399693)
Process isolation affects so many things in C. The strategy change is going to require changes to so many modules that it will either be a re-write or buggy.

In practical terms, if every line needs to be audited and updated, it is a re-write

Re:PostgreSQL reconsiders its process-based model by anarazel on Tuesday June 20, 2023 @12:44AM (#36399812)
What makes you think that it will require that many changes? There will be some widespread mechanical changes (which can be verified to be complete with a bit of low level work, like a script using objdump/nm to look for non-TLS mutable variables) and some areas changing more heavily (e.g. connection establishment, crash detection, signal handling, minor details of the locking code). But large portions of the code won't need to change. Note that we/postgres already shares a lot of state across processes.
Re:PostgreSQL reconsiders its process-based model by giraffe_lady on Tuesday June 20, 2023 @05:34AM (#36401652)
I'm not the person you asked and I don't have any particular knowledge of postgres internals.

Experience with other systems has taught me that in a system that's been in active use and development for decades, entanglement will be deep, subtle, and pervasive. If this isn't true of postgres then it's an absolute freak anomaly of a codebase. It is in other ways, so it's possible.

But the article mentions there being thousands of global variables. And Tom Lane himself says he considers it untenable for exactly this reason. That's a very good reason to think that it will require that many changes imo.

Re:PostgreSQL reconsiders its process-based model by h0l0cube on Tuesday June 20, 2023 @08:57AM (#36403326)
> that many changes

The 'that many changes' in question is a complete rewrite. Many changes across many files, yes, but nothing even approaching a rewrite.

> I don't have any particular knowledge of postgres

Judging by their bio, the person you're replying to does.

Re:PostgreSQL reconsiders its process-based model by h0l0cube on Tuesday June 20, 2023 @01:23AM (#36400057)
> either be a re-write or buggy

A large refactor at best. It will touch lots of parts of the code base, but the vast majority of the source code would remain intact. Otherwise they could just Rewrite it in Rust™ while they’re at it

> if every line needs to be audited and updated, it is a re-write

I’m not sure why you believe every line needs to be updated. Most code is thread agnostic.

Re:PostgreSQL reconsiders its process-based model by bandrami on Tuesday June 20, 2023 @06:27AM (#36402008)
The lesson that "cruft is problems someone solved before you" is unfortunately entirely lost on most devs today
Re:PostgreSQL reconsiders its process-based model by HeavyStorm on Tuesday June 20, 2023 @07:59AM (#36402812)
Love this article. Completely changed the way I think about certain projects.
Re:PostgreSQL reconsiders its process-based model by kqr on Tuesday June 20, 2023 @02:12AM (#36400355)
On the other hand, there's also the case of the Lunar Module guidance software that was hard-coded to run exactly every two seconds. If the previous subroutine call was still running when the next one was due, the previous one was harshly terminated (with weird side effects).

One of the main programmers suggested making it so that the next guidance routine wouldn't run until the previous one was done. This would make the code less sensitive to race conditions and allow more useful functionality for the pilots (who were the actual users and did seem to want it). However everyone assumed the two-second constant was implicitly embedded everywhere.

It wasn't -- only in a few places -- and with that fixed the code got more general and the proof of concept ran better than ever in about every simulator available. The amount of control it gave pilots was years ahead of the curve. But it never got a chance to fly on a real mission because what was there was "good enough" and nobody bothered to try.

In our combined comments there's a lesson about growing experiments and figuring out how to achieve failure quickly.

Re:PostgreSQL reconsiders its process-based model by zilti on Tuesday June 20, 2023 @11:12AM (#36405159)
Implying it had ever not been terrible compared to other options
Re:PostgreSQL reconsiders its process-based model by orf on Wednesday June 21, 2023 @05:33PM (#36424937)
What was the specifics about the string refactor implementation? I can’t find anything about it online