News for hackers. Stuff that matters.
Re:OpenBSD, C, httpd and SQLite – Web App stack by kuon on Saturday June 09, 2018 @07:00AM (#17272426)
We could argue that modern static analysis tools and the compiler itself are good at catching a lot of errors, and that high level languages have bugs too. C is a dangerous beast, but with good practices it isn't that dangerous. Of course you have to be extra careful when dealing with user content, like JSON payload, but many decoders in other languages are written in C.

Rust is a good solution, but it isn't trivial to learn, and might not suite all situations. I think it's great to have a C solution like this.

Re:OpenBSD, C, httpd and SQLite – Web App stack by unrealhoang on Saturday June 09, 2018 @07:24AM (#17272493)
I don’t think any non-gc language is trivial to learn, C also. Manual memory management is hard, and either you learn from the system (C way) or learn it upfront from the language (Rust way), it’s still the same amount of concept you have to digest to write practical software.
Re:OpenBSD, C, httpd and SQLite – Web App stack by addicted on Saturday June 09, 2018 @08:55AM (#17272788)
It’s not just that C is hard. I can wrangle together pretty much any program in C since I used to program in C about a decade ago. However, having been so long, my greatest fear is that I wouldn’t even be aware of the footguns I have introduced in my code. C is littered with best practices which are absolutely not obvious, and unless you follow them can be massive security risks. And to a slightly lesser extent the same is true of C++ as well (although my friends tell me newer versions of C++ are much better at this. I worked with versions prior to C++11).
Re:OpenBSD, C, httpd and SQLite – Web App stack by tannhaeuser on Saturday June 09, 2018 @07:44AM (#17272549)
That's true, but the original idea of Unix is to have many small program invocations work together, not to erect a monolithical long running daemon that does it all in a single address space. Memory management for one-shot command line apps isn't hard at all and can often get away with static allocations. Even if you're screwing up your heap, process isolation will take care of recycling memory when your program terminates.
Re:OpenBSD, C, httpd and SQLite – Web App stack by dannypgh on Saturday June 09, 2018 @08:14AM (#17272649)
Eh, that's the original idea of the Unix shell, sure. But Unix has many ideas. Long running daemons have been a part of Unix systems since the beginning.
Re:OpenBSD, C, httpd and SQLite – Web App stack by tannhaeuser on Saturday June 09, 2018 @09:27AM (#17272909)
BSD 4.3 introduced inetd along with TCP/IP to mainstream Unix. Quoting from [1]:

> When a TCP packet or UDP packet arrives with a particular destination port number, inetd launches the appropriate server program to handle the connection. For services that are not expected to run with high loads, this method uses memory more efficiently, since the specific servers run only when needed. Furthermore, no network code is required in the service-specific programs, as inetd hooks the sockets directly to stdin, stdout and stderr of the spawned process.

[1]: https://en.wikipedia.org/wiki/Inetd

Re:OpenBSD, C, httpd and SQLite – Web App stack by dannypgh on Saturday June 09, 2018 @10:51AM (#17273296)
I'm familiar with inetd, but I don't understand your point. Unix is more than a dozen years older than 4.3BSD. Claiming that long lived daemons is somehow anti-Unix is absurd.

Just ask init or getty.

Re:OpenBSD, C, httpd and SQLite – Web App stack by tannhaeuser on Saturday June 09, 2018 @10:54AM (#17273313)
My point was that the idea of small, self-contained apps that do one thing, and do it well, was in no way limited to shell programming.

Init and getty are small, self-contained system demons that are part of the O/S rather than application servers inviting long-running, single address-space processes for business logic.

Re:OpenBSD, C, httpd and SQLite – Web App stack by sfifs on Saturday June 09, 2018 @11:27AM (#17273479)
Many small programs simply doesn't scale to the modern web when just the baggage associated with spinning up processes will kill you at any reasonable load. UNIX has nice ideas but recognise they were founded in the shared computer between a few dozen users environment of the 70s, not a web server serving thousands of requests a second of today
Re:OpenBSD, C, httpd and SQLite – Web App stack by bitofhope on Saturday June 09, 2018 @12:31PM (#17273741)
What do you mean by modern web? I don't see how modernity implies more load, at least for fork() bottlenecked programs specifically.

If you're serving thousands of requests per second, you probably need a pretty beefy server anyway, regardless of the stack you're running. Forking makes good use of multiprocessor capabilities if nothing else.

Re:OpenBSD, C, httpd and SQLite – Web App stack by sfifs on Saturday June 09, 2018 @10:08PM (#17276049)
Modern language runtimes used in the server space like Go basically multiplex green threads across processes to minimize context switches between concurrent execution paths. By leveraging the fact that I/O is much slower than processor ops and switching green threads on I/O without necessarily always switching OS process, they easily scale to hundreds of thousands of concurrent requests. I don't really know about Erlang's model but it also has green threads. There's a reason why high traffic sites use languages like these.

https://talks.golang.org/2012/concurrency.slide#1

Re:OpenBSD, C, httpd and SQLite – Web App stack by laumars on Saturday June 09, 2018 @04:02PM (#17274703)
No, it really doesn't. Forking is expensive. Running a multithreaded (even if it's green threads rather than OS threads) http server and web application makes far better use of multi-processor capabilities.

Shell scripting is a handy tool but it is also slow so you wouldn't write hot paths in shell scripts. And for the same reasons you wouldn't write busy web servers in CGI. Both are fork() heavy.

Re:OpenBSD, C, httpd and SQLite – Web App stack by tonyarkles on Saturday June 09, 2018 @11:47AM (#17273555)
Isn't this basically what "serverless" is?
Re:OpenBSD, C, httpd and SQLite – Web App stack by sedachv on Saturday June 09, 2018 @12:23PM (#17273712)
Great observation. And in fact "serverless" is the way that Unix network servers used to work: https://en.wikipedia.org/wiki/Inetd

CGI is also the way the web used to work. Spawning processes has only gotten faster since then. The entire "process spawning doesn't scale to the modern web" argument is completely and totally bogus. Today, spawning a process in Linux is only 10-20 microseconds slower than creating a thread: http://www.bitsnbites.eu/benchmarking-os-primitives/

The performance problems are elsewhere.

Re:OpenBSD, C, httpd and SQLite – Web App stack by sfifs on Saturday June 09, 2018 @10:16PM (#17276069)
What you are missing here is the fact that I/O is orders of magnitude slower than processor. Most of the time in servers is spent waiting for I/O and multiplexing green threads on processes without context switches while some execution paths are waiting on I/O gives you far higher capacity on the same hardware. See the other comment thread on GP comment.
Re:OpenBSD, C, httpd and SQLite – Web App stack by sfifs on Saturday June 09, 2018 @09:56PM (#17276014)
serverless is primarily a decentralisation/small-scale/unpredictable workloads play where you trade off fixed costs of managing a dedicated instance for higher variable per call cost. At any reasonable scale and predictability, running a dedicated server is cheaper.

The folks who started at a small scale in serverless but see traffic growing and kind of are in the middle space before jumping to dedicated servers have an entire art form of keeping their functions "hot" since both Lambda and AppEngine actually keep your functions hot loaded when once it's spun up for some time.

https://www.google.com.sg/search?q=serverless+warm+up&oq=ser...

Re:OpenBSD, C, httpd and SQLite – Web App stack by pjmlp on Saturday June 09, 2018 @11:28AM (#17273483)
The Basics, Pascal, Modula-2 linages are surely relatively easy to learn, while preventing 90% of the typical class of C errors.
Re:OpenBSD, C, httpd and SQLite – Web App stack by mbaeten on Saturday June 09, 2018 @09:58AM (#17273037)
> Rust is a good solution, but it isn't trivial to learn, and might not suite all situations. I think it's great to have a C solution like this.

Isn't the learning curve of C pretty similar or even higher than rust's if you concider all the best practices you'll have to learn and understand and use? Imho in that regard rust pays off very well (and fast), because it's compiler _enforces_ most of these "best practices" to everyone.

Re:OpenBSD, C, httpd and SQLite – Web App stack by chx on Saturday June 09, 2018 @09:15AM (#17272852)
> like JSON payload,

Let SQLite deal with it! The JSON1 extension is quite capable of parsing, extracting, sorting and filtering JSON data and by default it's enabled.

Re:OpenBSD, C, httpd and SQLite – Web App stack by dijit on Saturday June 09, 2018 @09:31AM (#17272921)
What is sqlite written in?
Re:OpenBSD, C, httpd and SQLite – Web App stack by nitrogen on Saturday June 09, 2018 @10:10AM (#17273099)
SQLite is so extensively tested and widely used it may as well be written in stone. It doesn't count as "writing C".
Re:OpenBSD, C, httpd and SQLite – Web App stack by peatmoss on Saturday June 09, 2018 @11:23AM (#17273454)
I think the parent is being a little tongue-in-cheek, but it’s a valid point. C is a dangerously inappropriate language to use... except when it isn’t.

C still has some unique portability characteristics that are very hard to match in other languages. If your requirements lead you down this path, it can be done.

You rightly point out however that this isn’t a path one travels lightly. Maybe the flip side of all this is, “if you’re prepared to invest the kind of rigor into testing that SQLite has, then rock on.”

Re:OpenBSD, C, httpd and SQLite – Web App stack by dangerbird2 on Saturday June 09, 2018 @11:51AM (#17273575)
Yeah, if you run into a security vulnerability caused by SQLite, tens of thousands of websites, desktop apps, and virtually every iOS and Android app will be in the exect same boat.
Re:OpenBSD, C, httpd and SQLite – Web App stack by yellowapple on Sunday June 10, 2018 @05:21PM (#17280199)
Or OpenBSD, for that matter.
Re:OpenBSD, C, httpd and SQLite – Web App stack by j1elo on Saturday June 09, 2018 @07:50AM (#17272563)
Rust is a good solution, but it isn't trivial to learn

Maybe another opportunity to mention Zig again as a possible good alternative in the learning curve front? (I learned about it when it appeared recently in the HN front page)

Re:OpenBSD, C, httpd and SQLite – Web App stack by earenndil on Saturday June 09, 2018 @04:13PM (#17274747)
Zig doesn't have any web frameworks yet, though.
Re:OpenBSD, C, httpd and SQLite – Web App stack by gaius on Saturday June 09, 2018 @07:06AM (#17272444)
C is a dangerous beast, but with good practices it isn't that dangerous. Of course you have to be extra careful when dealing with user content, like JSON payload, but many decoders in other languages are written in C.

It isn't hard to build tools like Valgrind and AFL into your C workflow. I would take that over simply taking for granted that a higher-level language was secure just because it is interpreted. After all it's probably C under the hood anyway.

Re:OpenBSD, C, httpd and SQLite – Web App stack by marmaduke on Saturday June 09, 2018 @07:46AM (#17272552)
This is a good point: you can CI run tests under Valgrind with various sanitizers and -Wall -Wextra. In a small codebase with a few cores, one can compile and run a test suite faster than it takes for a Django app to start.
Re:OpenBSD, C, httpd and SQLite – Web App stack by zeth___ on Saturday June 09, 2018 @07:02AM (#17272435)
When your messaging protocol is Turing complete you have no hope of a secure web server.
Re:OpenBSD, C, httpd and SQLite – Web App stack by peterkelly on Saturday June 09, 2018 @10:08AM (#17273082)
JSON is not Turing complete