News for hackers. Stuff that matters.
Re:The .zip file specification is flawed by niftich on Tuesday December 13, 2016 @04:02PM (#13170974)
Most out-in-the-wild usage of zip files fits the 'Robustness Principle', or, after its author, Postel's Law, found in RFC 1122 [1]:

> Be liberal in what you accept, and conservative in what you send

There are several analyses of this maxim and whether it's a good choice for designing robust, secure systems. This particular Internet Draft doesn't agree [2].

[1] https://tools.ietf.org/html/rfc1122#section-1.2.2 [2] https://tools.ietf.org/html/draft-thomson-postel-was-wrong-0...

Re:The .zip file specification is flawed by user5994461 on Tuesday December 13, 2016 @04:40PM (#13171345)
For user interfaces: Be liberal in what you accept from the user.

For M2M (machine to machine): Be strict in what you accept, and strict in what you send.

That's my take on it now.

For instance, a user will copy/paste URLs to his browser, there will probably be a space too much before or after. It's okay to clean it (and a better experience than a "site not found").

When a machine sends something "weird". Well, it's not possible to know if it's really wrong and it can't be corrected in any meaningful way. Just fail and throw an error so the developer can fix it.

Re:The .zip file specification is flawed by taeric on Tuesday December 13, 2016 @10:11PM (#13173462)
No. You should always be as liberal on what you accept as you can. Primarily because as strict as you try to be in what you send, you are likely to make mistakes. Very very likely.

Obviously, your field of work dictates a lot here. And, if you are accepting something that has severe consequences on acting out, then yes, be more strict. However, the general principal holds. In general.

Re:The .zip file specification is flawed by user5994461 on Wednesday December 14, 2016 @11:53AM (#13177417)
> Primarily because as strict as you try to be in what you send, you are likely to make mistakes. Very very likely.

What you call a human mistake should be called a bug. The only acceptable way to handle a bug is to fix the code.

Not to be dismissive but the way you think is a classic beginner mistake. It's not the responsibility of other software to guess what bugs you'll put in yours. ^^

> Obviously, your field of work dictates a lot here.

large distributed systems, financial exchanges, trading systems, national government projects, aerospace, and even web stuff at times.

Some fields have low standards. That doesn't mean that the good practises don't apply, it just mean that people don't apply them ;)

Re:The .zip file specification is flawed by taeric on Wednesday December 14, 2016 @12:47PM (#13177965)
I conceded it is a bug. I merely claimed you will make some. Because, you will. Period.

You also have an odd misconception. It isn't having lower standards to be liberal in what you accept. It is actually harder. Much harder. And you should do it.

Easy example from finance. You shouldn't just take one currency. You should take in as many currencies as you can understand.

Does this mean to be magical? No. But you should ask, "how many different ways could this be given to me?" And you should instrument this with a marker for "unexpected."

Re:The .zip file specification is flawed by user5994461 on Wednesday December 14, 2016 @01:17PM (#13178247)
> You shouldn't just take one currency. You should take in as many currencies as you can understand.

This is a business feature request.

The "Being liberal about what you accept" is a technical guideline for protocol/format design and input processing/sanitation. Don't apply that rule to feature requests, it's not meant for that :(

Re:The .zip file specification is flawed by taeric on Wednesday December 14, 2016 @01:24PM (#13178312)
I disagree. It is meant in the "if you can understand it, you should act like you do." This is true in life as much as it is in programming.

It is fun to be pedantic with kids about "may I" versus "can I?" However, both work and if something can be understood, it should be understood.

So, should you build in all currencies? Not necessarily. However, you should definitely consider tagged inputs immediately, with the ability to extend the tags at some point. I'm not talking about big bang feature development, but ultimately, useful tools will have a plethora of ways they take in inputs. Because that is useful. And no design constraint should preclude this growth.

Re:The .zip file specification is flawed by manarth on Wednesday December 14, 2016 @04:43PM (#13179931)

  > "if you can understand it, you should act like you do"
For an unanticipated event, it's often less "if you can understand it", than "if you believe you can correctly guess the intention of an ambiguous request".

HTML is a great example of that ambiguity. Take this example markup:

  <h2 title="The <a href="https://en.wikipedia.org/wiki/Fall_of_the_Western_Roman_Empire">Fall of Rome</a>" class="roman_empire">The <a href="https://en.wikipedia.org/wiki/Fall_of_the_Western_Roman_Empire">Fall of Rome</a></h2>
This isn't too naive a markup error, and it's easy to see how a simple error in a CMS template could cause this output. So how should a parser treat it? It's non-conformant to the standard, but should it totally reject it? Recognise that the opening-bracket of the a tag in the title attribute is the start of the incorrect markup, and strip tags within that attribute? How should it appropriately recognise the misuse of double-quotes within a property attribute, that - according to the spec - should close the attribute?

There is some merit in strict behaviour, rejecting non-conformant input. Especially when it comes to computer-to-computer API design, where rejecting non-conformant input may be better than attempting to interpret ambiguous input. Limitations can be an arrow to direct us back on the best path.

Re:The .zip file specification is flawed by taeric on Wednesday December 14, 2016 @06:17PM (#13180690)
I get your point. It is very appealing. I do note that HTML is a definitive example where the strict version lost. Hard.

My argument here would be to do your best to not create ambiguity in what can be fed to you. The currency one is a great example. Never ever make something that takes in untagged integers as a value. It could mean too many things and you would have no method of knowing what was intended. But, once you start accepting strings, "$4" or "4 dollars", or "Four dollars" should all be on the table. (Though, yes, in this case probably best to stick with "4 USD" and complain about "dollars" being ambiguous without another qualifier.)

Re:The .zip file specification is flawed by wolrah on Tuesday December 13, 2016 @08:42PM (#13173105)
I agree with those positions, but what about the middle ground? Browsers for example, taking machine-to-machine data which sometimes is human generated.

I'm a fan of the strict solution as far as that goes, but there's a reason XHTML failed. Somehow asking people who write web sites to do it right or else it doesn't work is a big deal.

Re:The .zip file specification is flawed by taneq on Tuesday December 13, 2016 @10:57PM (#13173701)
The best approach seems to be to be strict in what you accept, and strict in what you emit, and if you want your software to be forgiving to the vagaries of human input, then strictly adhere to a forgiving spec.
Re:The .zip file specification is flawed by chrismorgan on Friday December 16, 2016 @01:50AM (#13191563)
If the web had been strict from the start, we would never have had a problem: people would never have been publishing invalid documents because browsers would have been rejecting them outright. XHTML failed because HTML existed and was good enough.
Re:The .zip file specification is flawed by user5994461 on Wednesday December 14, 2016 @12:05PM (#13177507)
Well. How to say that without being dismissive of the average web hobbyist...

Well, when the job is to accept a hugely complex flexible poorly defined input format from 30 years ago that is written by millions of people who have no clue what they are doing, fixing common errors and getting anything to render is part of the spec ^^

And... oh wait!

I said "Be liberal in what you accept from the user" when it comes to user input. Web pages are user input, so yeah, browsers are no exception to the rules, they're actually a perfect example! :D

Re:The .zip file specification is flawed by Too on Wednesday December 14, 2016 @01:01AM (#13174177)
The difference is a user interface translates the non strict version to the strict version instantly once you click the submit button. A hand written html file is stored persistently, retaining the flaw for all future.
Re:The .zip file specification is flawed by flukus on Tuesday December 13, 2016 @11:30PM (#13173846)
I wonder how many security exploits this principle is responsible for?
Re:The .zip file specification is flawed by vacri on Tuesday December 13, 2016 @07:35PM (#13172749)
As an aside, a couple of months ago I found out that the PHP ISO8601 timestamp format is not 8601 compliant. If you want an ISO8601 timestamp, you have to use a different format.

I'm not sure how this fits in with Postel's law, but your comment jogged my memory of it :)

http://php.net/manual/en/class.datetime.php

Re:The .zip file specification is flawed by Terr_ on Tuesday December 13, 2016 @09:42PM (#13173369)
MySQL utf8 isn't utf8 either :p