News for hackers. Stuff that matters.
Re:Althttpd: Simple webserver in a single C file by ktpsns on Tuesday June 08, 2021 @04:50AM (#27431998)
Here is the actual single C-code file: https://sqlite.org/althttpd/file?name=althttpd.c

Something I absolutely love about text based protocols such as HTTP/1 is how easy you can implement it in any virtually programming language. Sure, the implementation is not top-of-the-notch, but it just damned works, it is portable, it is understandable by humans. That's something what's got lost with HTTP/2 and HTTP/3, respectively.

Re:Althttpd: Simple webserver in a single C file by lmilcin on Tuesday June 08, 2021 @06:07AM (#27432587)
Binary protocols aren't (or at least don't have to be) any more difficult and frequently are even easier to implement.

Text protocols have difficult problems like escaping or detecting the end of particular field that are frequent source of mistakes.

The issue is that many (especially scripting) languages treat binary data as second class.

The only real issue is that inspecting the binary message visually is little bit more difficult. You can usually easily tell if your data structure is broken if it is in text form. What I do is I usually have some simple converter that converts the binary message to text form and this helps me inspect the message easily.

Re:Althttpd: Simple webserver in a single C file by chrismorgan on Tuesday June 08, 2021 @06:39AM (#27433081)
And capitalisation issues, and encodings… give me a binary protocol to parse any time. They’re normally comparatively well-defined, whereas text protocols are seldom properly defined and so have undefined behaviour left, right and centre, which inevitably leads to security bugs—or even if they are well-defined, they’re probably done in a way that makes your language’s string type unsuitable, and makes parsing more complicated.
Re:Althttpd: Simple webserver in a single C file by pjc50 on Tuesday June 08, 2021 @10:01AM (#27435320)
People say this, but then there's ASN.1 which has had critical security bugs in: https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=asn.1

I will agree that exhaustively defining text protocols is extremely hard, starting from character set / encoding and getting worse from there.

Re:Althttpd: Simple webserver in a single C file by lmilcin on Tuesday June 08, 2021 @11:44AM (#27436657)
ASN.1 is not a binary protocol. It is a language to describe messages.

Typically you create message description which is then compiled to code that can serialize/deserialize messages in BER-TLV or PER-TLV.

I know because I wrote a complete parser/serializer for BER-TLV. It is simple protocol and any security issue is in the parser/serializer and not the protocol itself. That for simple reason that the protocol is nothing more than a format to serialize/deserialize the data.

Re:Althttpd: Simple webserver in a single C file by lmz on Tuesday June 08, 2021 @07:33PM (#27442442)
True, but they certainly don't make it too easy either with the indefinite length constructs.
Re:Althttpd: Simple webserver in a single C file by admax88q on Tuesday June 08, 2021 @04:15PM (#27440182)
I guess if you're going to pick the absolute worst example of a binary protocol.
Re:Althttpd: Simple webserver in a single C file by lmilcin on Tuesday June 08, 2021 @05:06PM (#27440794)
That is absolutely not fair.

BER-TLV is really nice protocol. I have worked with it for couple of years when I worked on an EMV application. It uses BER-TLV to communicate with the credit card but it is also very convenient format for all sorts of other uses and I would use it wherever I could. Think of it as Json but in binary form. It is not complicated and I would not even bother parsing the messages -- I could interpret hex dumps of them by sight very easily.

Re:Althttpd: Simple webserver in a single C file by the_arun on Tuesday June 08, 2021 @11:52PM (#27444031)
>>* The author disclaims copyright to this source code. In place of

>>* a legal notice, here is a blessing:

>>*

>>* May you do good and not evil.

>>* May you find forgiveness for yourself and forgive others.

>>* May you share freely, never taking more than you give.

Above all, this note at the beginning of the source code impressed me.

Re:Althttpd: Simple webserver in a single C file by scruple on Tuesday June 08, 2021 @12:59PM (#27437755)
> give me a binary protocol to parse any time

I don't disagree in principle but I've come across a handful of very poorly documentated binary protocols in my years and that is an extremely painful thing to deal with compared to text-based protocols.

Re:Althttpd: Simple webserver in a single C file by jrockway on Tuesday June 08, 2021 @02:23PM (#27438896)
I think it's pretty easy to make poor use of HTTP to the same end. Imagine you're dumping traffic between a client and a server and the exchange is:

   GET /asdfasdfasdfdsaf
   X-Asdf-Asdf: 83e7234
   
   HTTP/1.0 202
   X-83e7233: 1
   X-83f730b: 4
   
It's text, but you still have no idea what's going on.

Overall, I think it's kind of a wash. It's basically equally easy to take a documented text or binary protocol and write a working parser. Neither format solves any intrinsic problems -- malicious input has to be accounted for, you have to write fuzz tests, you have to deal with broken producers. It's a wash.

People like text because they can type it into telnet and see something work, which is kind of cool, but probably not a productive use of time. I can type HTTP messages, but use curl anyway. (SMTP was always my favorite. "HELO there / MAIL FROM foo / RCPT TO bar / email body / ." Felt like a real conversation going on. Still not sure how to send an email that consists of a single dot on its own line though.)

Re:Althttpd: Simple webserver in a single C file by knome on Tuesday June 08, 2021 @04:16PM (#27440189)
>Still not sure how to send an email that consists of a single dot on its own line though.

It seems you prefix any line of data starting with a period with an additional period, to therefore distinguish it from the end of mail period.

https://datatracker.ietf.org/doc/html/rfc5321#section-4.5.2

Re:Althttpd: Simple webserver in a single C file by taejo on Tuesday June 08, 2021 @04:14PM (#27440169)
> Still not sure how to send an email that consists of a single dot on its own line though.

A line starting with a period is escaped by adding an extra period; the receiving side removes the first character of a line if it is a period.

Re:Althttpd: Simple webserver in a single C file by funny_falcon on Wednesday June 09, 2021 @11:09AM (#27448790)
It's the dirties hack I've ever met.

Except I did something like this once in the past (but with zero byte instead of dot).

Re:Althttpd: Simple webserver in a single C file by chrismorgan on Tuesday June 08, 2021 @01:26PM (#27438158)
Ugh, flashbacks. Now that you’ve mentioned that I feel the burning need to change my wording to add the condition “fairly described”. I have also come across very poorly documented binary protocols!
Re:Althttpd: Simple webserver in a single C file by touisteur on Tuesday June 08, 2021 @06:56AM (#27433375)
Just use recordflux for binary parser proved absent of runtime errors... ;-)
Re:Althttpd: Simple webserver in a single C file by einpoklum on Tuesday June 08, 2021 @07:23AM (#27433640)
Can you expand on that a little?
Re:Althttpd: Simple webserver in a single C file by michaelmior on Tuesday June 08, 2021 @09:24AM (#27434830)
RecordFlux[0] is a DSL written in Ada for specifying messages in a binary protocol. Code for parsing these messages is then generated automatically with a number of useful properties automatically proven including that no runtime errors will occur.

[0] https://github.com/Componolit/RecordFlux

Re:Althttpd: Simple webserver in a single C file by jchw on Tuesday June 08, 2021 @02:07PM (#27438725)
They definitely don't have to be. I've echoed these sentiments before.

However, HTTP/2 and HTTP/3 certainly are, though the reasons why they are complicated have nothing to do with choosing to use a binary based format. (They are complicated for good reason, though, and I hope that browsers and servers can continue to support HTTP/1 as the baseline till the sun burns out, just to make life easier.)

Re:Althttpd: Simple webserver in a single C file by mucholove on Tuesday June 08, 2021 @07:37AM (#27433764)
I did implement a webserver (that supports websocket-a binary protocol) and also implemented a `make_binary_string` function along the wah so as to not lose my bearings. Printf is 100% amazing and I generally leave my printf statements inside all my code because I can switch them on and off very easily for any individual function by using macros. Also clutch :)
Re:Althttpd: Simple webserver in a single C file by daitangio on Tuesday June 08, 2021 @08:33AM (#27434275)
Yes and no. Debugging a binary protocol is a bit more difficult then a text one; which is the reason HTTP is ascii based.
Re:Althttpd: Simple webserver in a single C file by zaarn on Tuesday June 08, 2021 @08:44AM (#27434387)
The reason HTTP is ASCII based isn't because it's easier to debug. It's because back then the other end was as likely to be a human as a piece of software. Because HTTP in it's early days barely had headers or even formatting, so people typed "GET /" at the server directly or used the same method to send mail directly.

Nobody does that anymore and debugging is easily solved by converting your binary protocol to a textual form.

Re:Althttpd: Simple webserver in a single C file by throw0101a on Tuesday June 08, 2021 @09:11AM (#27434676)
I question the ascertain that humans were doing "GET /" or "HELO my.fq.dn" on any regular basis. There were mail clients from the very beginning for example:

* https://en.wikipedia.org/wiki/History_of_email

Perusing document-based information was also done via clients, first with Gopher and then with the WWW: either GUIs like Mosaic, or on the CLI via (e.g.) Lynx.

Re:Althttpd: Simple webserver in a single C file by michaelmior on Tuesday June 08, 2021 @09:26AM (#27434842)
I definitely did back in the day :) But I'm only one single human.
Re:Althttpd: Simple webserver in a single C file by lmilcin on Tuesday June 08, 2021 @09:51AM (#27435189)
I did both send mails over SMTP as well as downloading files from FTP using Telnet.
Re:Althttpd: Simple webserver in a single C file by jbverschoor on Tuesday June 08, 2021 @12:13PM (#27437107)
same
Re:Althttpd: Simple webserver in a single C file by zaarn on Tuesday June 08, 2021 @09:16AM (#27434733)
I would question if this weren't the case often enough that people would demand it be compatible with their teleprompter.
Re:Althttpd: Simple webserver in a single C file by choeger on Tuesday June 08, 2021 @10:31AM (#27435673)
I don't think this is about human readability (albeit it might be to some extent during the design phase). Instead, I think this was the sensible thing to do when byte order was way more variable across a network. If you only send single bytes to begin with, you can as well use a textual format, IMO.
Re:Althttpd: Simple webserver in a single C file by squarefoot on Tuesday June 08, 2021 @11:54AM (#27436807)
It depends from the nature of the payload. If I have to send text, I'll use a text based protocol because I can debug it more easily, but if I have to send binary information, I'd rather send it using well defined structures after taking into account word sizes, alignment, endianess etc. on the involved hardware. Back in the day I had to do that with small agents on different achitectures (IBM Power and x86) with different languages (C and Object Pascal), and the correct textbook way to do that was to use xml, but that way everything had to be converted to text, interpreted and then translated back for replies. No way, hardware was slow and we aimed at speed, so we used plain C/Pascal structures and no other manipulation except for adjusting type sizes and endianess if the hardware or language required so to make all modules compatible no matter what they were written in and where they would run. I also built my own tools for testing and debugging which were nothing more than agents that sent predefined packets and echoed at screen the return values, so that any error in the data would be quickly spotted, while a xml translation of a corrupt buffer could have missed any error not comprised in the marked text, hence hiding there was a problem somewhere. A solution like that is highly debatable for sure, but I'm among the ones who want their software to crash loudly at the first problem, not try hiding it until it creates a bigger damage.
Re:Althttpd: Simple webserver in a single C file by Iwan-Zotow on Tuesday June 08, 2021 @09:57AM (#27435277)
> which is the reason HTTP is ascii based.

well, other side of HTTP was telnet in the terminal - this is why it is ASCII to begin with

Re:Althttpd: Simple webserver in a single C file by tdeck on Tuesday June 08, 2021 @01:53PM (#27438538)
This is not true. HTTP was developed for the first (graphical) web browser, WorldWideWeb. It was specifically intended to serve HTML and images, neither of which is particularly well suited for browsing with telnet.
Re:Althttpd: Simple webserver in a single C file by eloop on Wednesday June 09, 2021 @12:14AM (#27444155)
Not sure that is strictly correct. I vaguely remember a command line browser early in the piece. The inline image ability and <img> tag came a bit later in the story when Andreessen put it into the NCSA's Mosaic browser. I've always wondered if it was that single thing that really made the web take off, more so than hyperlinking which had been around for quite a while. People like pretty pictures, especially advertisers.
Re:Althttpd: Simple webserver in a single C file by tdeck on Wednesday June 09, 2021 @12:57PM (#27450257)
You're describing the line mode browser:

https://en.wikipedia.org/wiki/Line_Mode_Browser

But that didn't just spit out the HTTP response - it parsed and rendered the HTML.

Re:Althttpd: Simple webserver in a single C file by arcticbull on Friday June 11, 2021 @12:41PM (#27475533)
Yeah, humans can roughly speaking infer the schema of a text-based protocol but require one for a binary protocol.
Re:Althttpd: Simple webserver in a single C file by geoffdunbar on Tuesday June 08, 2021 @03:18PM (#27439587)
I love C, but it's pretty scary sometime. 5 minutes ago, "I wonder if I can find a potential memory overwrite in 5 minutes?"

Sure enough, the function StrAppend potentially overflows a size_t size (without checking), and then writes into memory could be past the end of the allocated buffer. Given 5 minutes, I didn't look thoroughly if this is actually exploitable, but it's definitely a red-flag for the code. Be careful out there! Hopefully I am missing something, or this is just a simple oversight, but I would carefully audit this code before using it.

Submitted a ticket through the Althttpd website.

static char StrAppend(char zPrior, const char zSep, const char zSrc){ char zDest; size_t size; size_t n0, n1, n2;

  if( zSrc==0 ) return 0;
  if( zPrior==0 ) return StrDup(zSrc);
  n0 = strlen(zPrior);
  n1 = strlen(zSep);
  n2 = strlen(zSrc);
  size = n0+n1+n2+1;
  zDest = (char*)SafeMalloc( size );
  memcpy(zDest, zPrior, n0);
  free(zPrior);
  memcpy(&zDest[n0],zSep,n1);
  memcpy(&zDest[n0+n1],zSrc,n2+1);
  return zDest;
}
Re:Althttpd: Simple webserver in a single C file by spacechild1 on Tuesday June 08, 2021 @03:35PM (#27439752)
> Sure enough, the function StrAppend potentially overflows a size_t size

How should this happen in practice? The three strings would have to be larger than the available address space...

Re:Althttpd: Simple webserver in a single C file by ectospheno on Tuesday June 08, 2021 @03:39PM (#27439797)
Yeah. The function in question is called in only one place. It would seem you’d need to send the web server more than a size_t of data for this to be an issue.
Re:Althttpd: Simple webserver in a single C file by geoffdunbar on Tuesday June 08, 2021 @03:54PM (#27439951)
Yes, absolutely. If the webserver is compiled 32-bit, that is only 4GB of data, which might be feasible? I don't know enough to say. Assuming a hacker kindly won't overflow your buffer is never a good idea.

However, the presence of one piece of code that is not integer-overflow safe definitely makes me nervous. This is just the one I found in 5 minutes, what else is in there?

Re:Althttpd: Simple webserver in a single C file by acqq on Tuesday June 08, 2021 @04:45PM (#27440533)
It's not an integer overflow that would be needed but an unsigned overflow. The way I see it, on 32-bits, that means that the size HTTP request would have to be bigger than what's available to both user application and the OS together. In short, one just can't get the input request that big. Of course, if you manage that, you'll disprove this claim.
Re:Althttpd: Simple webserver in a single C file by ectospheno on Tuesday June 08, 2021 @04:08PM (#27440092)
None that stand out to me, including what you posted. Do you have a real example?
Re:Althttpd: Simple webserver in a single C file by thisgoodlife on Tuesday June 08, 2021 @06:17PM (#27441766)
MAX_CONTENT_LENGTH is 250MB. You won’t be able to send 4GB of data.
Re:Althttpd: Simple webserver in a single C file by gallier2 on Tuesday June 08, 2021 @04:12PM (#27440142)
In most places it uses int for string and buffer sizes lengths. It wouldn't surprize me if 2GiB of data could trigger several overflows.
Re:Althttpd: Simple webserver in a single C file by acqq on Tuesday June 08, 2021 @03:43PM (#27439842)
Exactly. In a single file C nobody can expect to get universal library functions that work in any possible imaginable context. The only relevant context is the code the function is in. And in that context, the function is doing enough.
Re:Althttpd: Simple webserver in a single C file by jackewiehose on Tuesday June 08, 2021 @03:42PM (#27439830)
And there's only one call to StrAppend() which is easily verified as safe.
Re:Althttpd: Simple webserver in a single C file by saba2008 on Tuesday June 08, 2021 @05:07AM (#27432102)
HTTP/1 simplicity comes not from text nature, but from it's clear concept and limited functionality.

At core, it's just request-reply + key-value metadata. Whenever it's text, or binary, it does not matter much. But writing HTTP/2 frame types in letters would not make them any easier to understand.

Re:Althttpd: Simple webserver in a single C file by ahartmetz on Tuesday June 08, 2021 @05:38AM (#27432260)
There are also keep-alive, caching (a big topic), chunked transfer encoding, header parsing peculiarities, and authentication in HTTP. The combination of these creates some nice opportunities for implementation bugs. Source: have worked on a client-side implementation.

Now, HTTP/2 isn't even conceptually simple, I agree about that... it seems ugly.

Re:Althttpd: Simple webserver in a single C file by merb on Tuesday June 08, 2021 @06:36PM (#27441998)
> implementation bugs. Source: have worked on a client-side implementation

well because the hard part is the client-side. caching is a client-side only thing with http, keep-alive is a thing that a server pushes to a client, the same with chunked transfer, which is not as easy to implement for a client like it was with content length.

basically a server does only need to implement certain headers, but a client needs to know all. also most clients even accept bad servers, like content for head requests, etc.. most stateless protocols put a lot of burden into clients.

h2 on the other hand is stateful and keeps the same hard semantics onto the client side and also makes servers more complex, because it's a state machine.

Re:Althttpd: Simple webserver in a single C file by tokamak-teapot on Tuesday June 08, 2021 @07:21AM (#27433621)
One good thing is that you don’t have to support everything if you’re writing a server. It just depends on what kind of use cases you want to support.
Re:Althttpd: Simple webserver in a single C file by jbverschoor on Tuesday June 08, 2021 @12:13PM (#27437116)
That came with http/1.1 ;-)
Re:Althttpd: Simple webserver in a single C file by secondcoming on Tuesday June 08, 2021 @06:21AM (#27432834)
HTTP allows a TimedOut response to be sent by a server to request you haven't yet sent. So it's not strictly request/reply.
Re:Althttpd: Simple webserver in a single C file by saba2008 on Tuesday June 08, 2021 @07:38AM (#27433776)
'100 Continue' would probably be better example of breaking request/reply flow, as it provides useful functionality and requires non-trivial implementation and compatibility measures.

While '408 Request Timeout' is somewhat dubious fig leaf over TCP RST

Re:Althttpd: Simple webserver in a single C file by TZubiri on Tuesday June 08, 2021 @08:26AM (#27434212)
Really simple, I implemented a couple. Remember that the letters after the status code are aesthetic, so make sure to put your style in there:

200 FINE 200 KTHX 404 MISS 403 NO 500 FUCK

Another similar server in one file is busybox httpd command if you are interested https://git.busybox.net/busybox/tree/networking/httpd.c

Re:Althttpd: Simple webserver in a single C file by Sohcahtoa82 on Tuesday June 08, 2021 @01:55PM (#27438581)
I always wanted an HTTP response code for when the server detects a malicious request. Like, 430 DONT BE AN ASSHOLE
Re:Althttpd: Simple webserver in a single C file by zbentley on Wednesday June 09, 2021 @10:15AM (#27448102)
Relatedly, in every C project I've ever worked on, the custom (>255) error code for failures due to malicious looking input that could not possibly have been supplied by mistake is in the const EGETFUCT.
Re:Althttpd: Simple webserver in a single C file by TazeTSchnitzel on Tuesday June 08, 2021 @05:49AM (#27432328)
Once you have to support HTTP/1.1 it's not really a text-based protocol, because support for chunked encoding is mandatory for clients. (Yes the chunk headers are technically text, but it's interspersed with arbitrary binary data. It's not something you can easily read in a text editor.)
Re:Althttpd: Simple webserver in a single C file by textmode on Tuesday June 08, 2021 @07:18AM (#27433595)
I make most HTTP requests using netcat or similar tcp clients so I write filters that read from stdin. Reading text files with the chunk sizes in hex interspersed is generally easy. Sometimes I do not even bother to remove the chunk sizes. Where it becomes an issue is when it breaks URLs. Here is a simple chunked transfer decoder that reads from stdin and removes the chunk sizes.

   flex -8iCrfa <<eof
    int fileno (FILE *);
   xa "\15"|"\12"
   xb "\15\12" 
   %option noyywrap nounput noinput 
   %%
   ^[A-Fa-f0-9]+{xa}
   {xa}+[A-Fa-f0-9]+{xa}
   {xb}[A-Fa-f0-9]+{xb} 
   %%
   int main(){ yylex();exit(0);}
   eof

   cc -std=c89 -Wall -pipe lex.yy.c -static -o yy045
Example

Yahoo! serves chunked pages

   printf 'GET / HTTP/1.1\r\nHost: us.yahoo.com\r\nConnection: close\r\n\r\n'|openssl s_client -connect us.yahoo.com:443 -ign_eof|./yy045
Re:Althttpd: Simple webserver in a single C file by charsi on Wednesday June 09, 2021 @06:26AM (#27446298)
I tried this but ended up with gibberish in my terminal. Also couldn't find an explanation for -a on flex's man page. I've never used the thing before.
Re:Althttpd: Simple webserver in a single C file by textmode on Wednesday June 09, 2021 @11:41AM (#27449208)
The extra "a" is a typo but would have no effect. The "i" is also superfluous but harmless. Without more details on the "gibberish" it is difficult to guess what happened. The space before "int fileno (FILE *);" is required. All the other lines must be left-justified, no leading spaces, except the line with "int main()" which can be indented if desired.
Re:Althttpd: Simple webserver in a single C file by charsi on Wednesday June 09, 2021 @11:13PM (#27456299)
This is the script I am running - https://pastebin.com/65GxJ9i9. The - after << ignores the tabbed indents in heredoc.

This is what it produces for me when I run `lexit.sh us.yahoo.com` - https://stuff-storage.sfo3.digitaloceanspaces.com/ee.txt

Re:Althttpd: Simple webserver in a single C file by textmode on Sunday June 13, 2021 @02:55AM (#27490395)
https://news.ycombinator.com/item?id=27490265 <-- yy054

The "gibberish" is GZIP compressed data. "yy054" is a simple filter I wrote to extract a GZIP file from stdin, i.e., discard leading and trailing garabage. As far as I can tell, the compressed file "ee.txt" is not chunked transfer encoded. If it was chunked we would first extract the GZIP, then decompress and finally process the chunks (e.g., filter out the chunk sizes with the filter submitted in the OP).

In this case all we need to do is extract the GZIP file "ee.txt" from stdin, then decompress it:

    printf "GET /ee.txt\r\nHost: stuff-storage.sfo3.digitaloceanspaces.com\r\nConnection: close\r\n\r\n"|openssl s_client -connect 138.68.34.161:443 -quiet|yy054|gzip -dc > 1.htm
    firefox ./1.htm
   
Hope this helps. Apologies I initially guessed wrong on here doc. I was not sure what was meant by "gibberish". Looks like the here doc is working fine.
Re:Althttpd: Simple webserver in a single C file by charsi on Thursday June 10, 2021 @02:34AM (#27457543)
new pastebin. Had a typo in the old one- https://pastebin.com/4j9Z3eCc
Re:Althttpd: Simple webserver in a single C file by textmode on Saturday June 12, 2021 @06:40AM (#27483448)
Need to get rid of the leading spaces on all lines except the "int fileno" line. Can also forgo the "here doc" and just save the lines between "flex" and "eof" to a file. Run flex on that file. This will create lex.yy.c. Then compile lex.yy.c.

The compiled program is only useful for filtering chunked transfer encoding on stdin. Most "HTTP clients" like wget or curl already take care of processing chunked transfer encoding. It is when working with something like netcat that chunked tranfser encoding becomes "DIY". This is a simple program that attempts to solve that problem. It could be written by hand without using flex.

Re:Althttpd: Simple webserver in a single C file by charsi on Saturday June 12, 2021 @01:36PM (#27486048)
Okay I'll give up for now. There are really no spaces in front of the lines. In pastebin if you check the raw version you'll see they are tabs. Which get stripped out because I added a `-` before eof. Providing the file manually to flex also produces the same gibberish for me.
Re:Althttpd: Simple webserver in a single C file by mbreese on Tuesday June 08, 2021 @06:35AM (#27433035)
HTTP/1 requests for binary files also aren’t text based under that criterion. The protocol itself is text based, but the payload(s) can be binary. That’s doesn’t change between HTTP/1 and /1.1.
Re:Althttpd: Simple webserver in a single C file by kijin on Tuesday June 08, 2021 @04:59AM (#27432066)
HTTP is serious about backward compatibility, so a client that speaks HTTP/1.1 (or HTTP/1.0 + Host header) can still talk to most servers out there.

Similarly, if you write a server that only speaks HTTP/1.1 (or HTTP/1.0 + Host header), you can put it behind a reverse proxy or load balancer that handles higher versions, does connection management, and terminates TLS. It will work perfectly fine, only without some of the latest performance optimizations that you might or might not even need.

Re:Althttpd: Simple webserver in a single C file by chippiewill on Tuesday June 08, 2021 @05:40AM (#27432277)
> you can put it behind a reverse proxy or load balancer that handles higher versions, does connection management, and terminates TLS

This is even standard practice in many production deployments. Typically you want the proxy or load balancer anyway and there's often little benefit (if any) to using HTTP/2 or HTTP/3 over a very low-latency, high reliability local network.

Re:Althttpd: Simple webserver in a single C file by bawolff on Tuesday June 08, 2021 @05:22AM (#27432173)
That seems like a high risk of http desync attacks, if you're only implementing a subset of http/1.1.
Re:Althttpd: Simple webserver in a single C file by unixhero on Tuesday June 08, 2021 @05:41AM (#27432283)
How would one attack a static page?
Re:Althttpd: Simple webserver in a single C file by bawolff on Tuesday June 08, 2021 @11:15AM (#27436290)
Who said anything about static? This webserver supports cgi, which means it supports php, perl, etc.

However even if it didn't, js based client-side apps are probably still attackable in the right set of circumstances.

Re:Althttpd: Simple webserver in a single C file by ric2b on Tuesday June 08, 2021 @06:47AM (#27433233)
Why would you even need a reverse proxy for a static page? This isn't about that.
Re:Althttpd: Simple webserver in a single C file by unixhero on Tuesday June 08, 2021 @07:35AM (#27433746)
I see. Thanks!
Re:Althttpd: Simple webserver in a single C file by andai on Tuesday June 08, 2021 @06:29AM (#27432945)
For people who like text-based protocols (and dislike surveillance and web bloat), I suggest taking a look at Gemini, which was designed so you could write a client for it over the weekend.

https://gemini.circumlunar.space/

Re:Althttpd: Simple webserver in a single C file by peterhil on Tuesday June 08, 2021 @06:01PM (#27441533)
Gemini is quite interesting in that brings nostalgic feeling from when the web was a new thing, but is also modern.

The Lagrange browser seems quite polished.

Re:Althttpd: Simple webserver in a single C file by strangeattractr on Tuesday June 08, 2021 @07:27AM (#27433672)
I prefer binary protocols but I think you make a good point about HTTP/1 especially from a learning perspective. I remember how enlightening it was when I followed a tutorial to create a http server and found that all I needed to do (ignoring TCP/IP) was write 'HTTP/1.1 200 OK' plus a few headers and some raw html in a string and it would actually show up in my browser.
Re:Althttpd: Simple webserver in a single C file by slver on Tuesday June 08, 2021 @10:09AM (#27435404)
It's not a property of text-based protocols, rather it's a property of simple protocols. HTTP/1 is not merely text based, it's ASCII based (technically ISO-8859-1, which includes ASCII). One char, one byte, one encoding. HTTP itself is mostly very simple, text "name: value" pairs separated by newlines, followed by arbitrary content as the body.

I think the solution is to start with a simple protocol and upgrade to more complex protocols after that. While technically you don't need to support HTTP/1 to support 2 and 3, the upgrade over TCP happens mostly in that way.

Re:Althttpd: Simple webserver in a single C file by chrismorgan on Tuesday June 08, 2021 @01:24PM (#27438122)
> While technically you don't need to support HTTP/1 to support 2 and 3, the upgrade over TCP happens mostly in that way.

This is not actually true of HTTP/2 (the upgrade doesn’t use HTTP/1), and slightly misleading of HTTP/3 (there’s nothing to upgrade because it sits beside TCP HTTP; but advertising HTTP/3 support is currently done over TCP HTTP).

Now for the details:

HTTP/2 upgrade is done at the TLS level, via ALPN. Essentially the client TLS handshake says “hello there! BTW do you do HTTP/2?” and the server either responds “hi!” and starts talking HTTP/1, or “hi! Let’s talk h2!” and starts talking HTTP/2.

So it’s perfectly possible (though not a good idea) to have a fully-functioning HTTP/2 server that doesn’t speak a lick of HTTP/1.

(HTTP/2 over cleartext, h2c, does use the old HTTP/1 Upgrade header mechanism, but h2c is more or less just not used by anyone.)

HTTP/3 upgrade, well, “upgrade” is the wrong word. It’s operating over UDP rather than TCP, so you’re not upgrading an existing HTTP thing to HTTP/3, you’re starting a new connection because you’ve learned that the server supports HTTP/3. This bootstraping is currently done by advertising h3 support via the Alt-Svc header (HTTP/1+) or by an ALTSVC frame (HTTP/2+), which the client can then remember so it uses the best protocol next time.

For best results, they’re working on allowing you to advertise HTTP/3 support on DNS, so that after a while it should be genuinely possible (though a very bad idea) to have a fully-functioning HTTP/3 server that doesn’t speak any TCP at all, yet works with sufficiently recent browsers in network environments that don’t break HTTP/3. https://blog.cloudflare.com/speeding-up-https-and-http-3-neg... is good info on this part.

Re:Althttpd: Simple webserver in a single C file by deathanatos on Tuesday June 08, 2021 @10:08PM (#27443440)
> it's ASCII based (technically ISO-8859-1, which includes ASCII)

No, it is (some) ASCII plus "opaque octets". ("A recipient SHOULD treat other octets in field content (obs-text) as opaque data.") If you want to say that historically it was such, that's not correct either; it was that plus "supporting other charsets only through use of [RFC2047] encoding" which is a nightmare of an encoding scheme.

> HTTP itself is mostly very simple, text "name: value" pairs separated by newlines

Except when it isn't: headers can be folded across multiple lines. (Which is also obsolete, and discouraged.)

Add to that 1xx responses, transfer encodings, chunks, chunk extensions, trailers. HTTP is far from "simple"…

Re:Althttpd: Simple webserver in a single C file by anthk on Tuesday June 08, 2021 @10:25AM (#27435591)
Gopher is easier than that.
Re:Althttpd: Simple webserver in a single C file by api on Tuesday June 08, 2021 @08:21AM (#27434157)
Recording state info in static variables makes it only suitable for embedded.

I really hate when people do that. Static should only ever be const, init once, or something intrinsically singleton. There are very few exceptions.

Re:Althttpd: Simple webserver in a single C file by jmaygarden on Tuesday June 08, 2021 @08:40AM (#27434344)
Are you thinking of C++ issues with constructors? I don’t see anything wrong with the use of static variables in this single file C program. This isn’t a library; it’s a standalone web server.
Re:Althttpd: Simple webserver in a single C file by api on Tuesday June 08, 2021 @01:14PM (#27437978)
I am thinking of multithreading. Mutable static variables pretty much destroy any possibility of multithreading without a major refactor. Test harnessing is an issue too.

But if this only ever wants to be an app binary, I guess it's sort of okay.

Re:Althttpd: Simple webserver in a single C file by jschwartzi on Tuesday June 08, 2021 @08:08PM (#27442705)
That class of errors is not an issue here.

This program isn't meant to be multithreaded, it's meant to be multi-/process/. The difference is that each instance of the application has its own independent address space. So that's one address space per instance. And because of that there are no threading issues. Also as a result any communication between different processes has to be explicitly defined.

Making this program multi-threaded would be a big mistake because then you couldn't use any of the HTTP proxies out there to monitor for connections and hand them off to this program.

This is arguably a much better architecture from a safety/security standpoint then trying to spin multiple threads each sharing an address space. It forces you to use OS-provided mechanisms for shared state rather than simply manipulating the memory to share information.

Re:Althttpd: Simple webserver in a single C file by secondcoming on Tuesday June 08, 2021 @06:19AM (#27432802)
Being text based is a huge flaw with HTTP in my opinion (and also elsewhere, like Redis). It leads to parsing bugs and overly verbose communications. Humans are good at reading text, CPUs prefer binary.
Re:Althttpd: Simple webserver in a single C file by mbreese on Tuesday June 08, 2021 @06:45AM (#27433189)
The use of text for popular protocols is for a reason — computers don’t write programs, people do. And while CPUs prefer binary, it’s easier for programmers to read/write/reason about text. This makes it easier to work with a new protocol.

From a practical perspective, with a binary protocol, it can be difficult to use across different languages or add support for a new language. If you use the simplest possible encoding, you’d send raw struct data. But this doesn’t always work across different OS/arch/versions/etc. if the server is in C, but the client is in Python, reading the binary protocol would require a far more complicated parser.

Obviously a more formal encoding (protobuf, etc) would be preferred, but if you already need to use an encoding mechanism, why not wrap it in a text format? It’s easier to write clients that can read/write text protocols in any language. The reason why text protocols are so popular aren’t because they are necessarily “better” but easier to adopt. This is why the most popular protocols are text based.

Re:Althttpd: Simple webserver in a single C file by michaelmior on Tuesday June 08, 2021 @09:37AM (#27435002)
Except it quickly gets messy when you start dealing with real data and making sure encoding and escaping is done correctly.

> with a binary protocol, it can be difficult to use across different languages

This is also true of text protocols that aren't well-designed. I don't think it's necessarily the case that binary protocols are more difficult to deal with. You just have a different set of concerns to address.

> If you use the simplest possible encoding, you’d send raw struct data.

This is the "simplest" in the sense that it's definitely easy to just copy this data on the wire, but I think this is a straw man. I don't think it's really any more difficult to write a simple protocol that uses binary data compared to text.

Re:Althttpd: Simple webserver in a single C file by mbreese on Tuesday June 08, 2021 @10:13AM (#27435462)
> I don't think it's really any more difficult to write a simple protocol that uses binary data compared to text.

I don’t really think so either… I mean, I’ve done both and it’s really not terrible to use binary. I think text is marginally easier to parse, but once to have the routines to read the right endian-ness, the advantage is minor. As you said, the biggest concern (as always) should be the design. A good design can be implemented easily with either mode.

However, it is significantly easier to debug a text protocol. Attaching a monitor or capturing packets is easier with text as the parsers are much easier and more generic.

Re:Althttpd: Simple webserver in a single C file by michaelmior on Tuesday June 08, 2021 @10:26AM (#27435611)
That's fair. Although tools like Wireshark have made this much better.
Re:Althttpd: Simple webserver in a single C file by dnautics on Tuesday June 08, 2021 @09:29AM (#27434883)
Being binary based leads to errors and loss of momentum when you are debugging something that's deployed to prod. It's all about tradeoffs.