News for hackers. Stuff that matters.

ucarion

Re:API pagination design on Saturday December 26, 2020 @09:33PM (#25548341)
From an API evolution standpoint, cursors are a no-brainer over explicit offsets:

1. Nothing stops you from having your cursors be implemented via a SQL offset under the hood.

2. You can have your cursors be base64 data, and embed a cursor version in the ID. You can use this to change how your pagination works without breaking clients during the transition period. (You'll need to document+enforce a maximum validity for pagination tokens to do this successfully.)

3. You can encrypt your pagination tokens, so clients don't get used to making assumptions about how your pagination works under the hood. (This isn't security by obscurity, it's just defending against Hyrum's Law [1].)

It doesn't cost you much to implement cursors early on, and unlike offsets they will grow with you across the evolution of your API.

[1]: https://www.hyrumslaw.com/

Re:Building My Own Chess Engine on Thursday December 24, 2020 @12:50PM (#25529667)
For what it's worth, there is a pretty solid conventional wisdom in this space.

> - A simpler but much faster engine will beat a more complex engine any time because the advantage of another ply easily outweighs the advantage of some clever algo.

Depends a lot on what you call "simple". Stockfish's search and evaluation subroutines are pretty complex and distinctly clever:

https://github.com/official-stockfish/Stockfish/blob/master/...

https://github.com/official-stockfish/Stockfish/blob/master/...

But I agree that early on, simple is the way to go. It's much easier to optimize a simple engine.

> - A 10x10 board with an unreachable edge has some advantages [...]

Maybe in theory, but FWIW I don't know of any modern engine that does this. A more popular variant of the 10x10 or 10x12 idea was, back in the day, the 0x88 layout:

https://www.chessprogramming.org/0x88

If you're going to be writing an engine today from scratch, use the bitboard representation. It's easy to implement and has the most resources out there if you need help.

(Bitboards is just the idea of representing the game state as a bunch of 64-bit ints, with one u64 per white/black type of piece.)

Your example use-case, of knight posisitions, is usually solved with bitboards using an 64-element array mapping squares to a bitboard of all the places a knight on that square can go. Here's where Stockfish does that:

https://github.com/official-stockfish/Stockfish/blob/b06ef36...

Other commenters have noted that you can do a slightly fancier version of this lookup table technique to solve for the sliding pieces (bishops and rooks (queens are a bitwise OR of bishops and rooks)), because the raw data is low entropy, so finding a perfect hash function ("magic bitboard", in the lingo) is fast enough to do on startup:

https://github.com/official-stockfish/Stockfish/blob/b06ef36...

It's existing techniques like this that make bitboards a good choice for beginners; if you get stuck, you can find resources online.

> - Optimization of engine code is much harder than optimization of the yield function.

Big +1 there! The yield function is also much easier to A/B test.

> - It is much better to not generate 'bad' moves than it is to prune them away after a lot of extra work has been done. Colin Wrights' law: 'You can't make computers faster, you can only make them do less work' clearly applies here.

Yes, but it's pretty hard to reliably get your move generator to generate good moves first. The other side of making mixmax search fast is quickly throwing out branches that aren't relevant.

For instance, delta pruning and null-move pruning do this by trying to detect if one of the sides has thrown the game with their last move; since minmax is about assuming optimal play on both sides, branches where someone makes a blunder aren't worth looking into.

> - memoization can bring insane speed gains.

Yup! Also, a lot of engines have optimized routines that let you undo a move quickly, without having to keep a redundant stack of every position's state, which would involve a lot of memcpy-ing.

Have fun! I had to stop doing chess engines, they were becoming too much of a distraction for me. I'm kind of glad ML-AI is starting to beat human-crafted AI in chess, that way the space can go back to being a fun little hacker cottage industry.

Edit: Oh! When making a chess engine, start by implementing "perft":

https://www.chessprogramming.org/Perft

That way you can compare how many legal moves your engine thinks there is in a position, versus how many Stockfish or some other known-good engine thinks there is. It's the closest thing there is to unit tests in the space.

Is it fair to categorize the pitch here as "a simpler AWS Cognito"?
SCIM is a pleasure to implement compared to SAML, no doubt. You might be able to get away with only supporting SCIM, the main thing you'd be missing is "just-in-time" user provisioning.

But given that you'll probably need SCIM at some point anyway, probably a good idea to start with SCIM, and then add SAML only when you need to! It'll also inform what subset of SAML you actually need to implement.

Depending on who your customers are, you might get away with only supporting OIDC. But not supporting SAML is going to be a problem as you move into the big enterprises.

Many big companies run on SAML, and expect to auth with vendors over SAML. That's why russell_h's comment is probably futile; it's the enterprises with the big SaaS budgets that keep SAML relevant, and they don't care if HN doesn't like it.

Maybe in about a decade SAML will be less important to enterprises? SAML 2.0 is only about 15 years old.

Re:Docker's Second Death on Sunday December 06, 2020 @05:14PM (#25327132)
I push for docker-compose v2 over v3 at work for the same reason. Just to spell out explicitly what nikisweeting is hinting at:

For local development use-cases, docker-compose v3 is just v2 minus all the features Docker Swarm doesn't support.

Since you bring up Docker, this little behavior has always baffled me:

    $ docker image ls -h
    Flag shorthand -h has been deprecated, please use --help
But:

    $ docker image -h ls
    Flag shorthand -h has been deprecated, please use --help
    Flag shorthand -h has been deprecated, please use --help
    Flag shorthand -h has been deprecated, please use --help
What's up with that? I'm guessing cobra nonsense, because everyone ends up wrapping cobra's weird API differently...
Re:Production-Oriented Development on Saturday November 21, 2020 @12:39PM (#25171158)
I fully agree with the rest of the article -- production is the only thing that matters, it's our _sine qua non_ -- but this section is too bold by far:

> You’ll never prioritize work on non-prod like you will on prod, because customers don’t directly touch non-prod. Eventually, you’ll be scrambling to keep this popsicle sticks and duct tape environment up and running so you can test changes in it, lying to yourself, pretending it bears any resemblance to production.

Error budgets are good solution to this problem. If your change qualification process (be that a QA team, staging, pre-prod, whatever) plus your release process is very good, then you probably aren't burning through your error budget.

But if it turns out your process isn't good enough, you'll get feedback on this in the form of running out of error budget. So you can spend the rest of your quarter working on your non-production environments, so that in the future you can move fast and break an acceptable amount of things next Q.

Re:Emojis paved the way for UTF-8 everywhere on Tuesday November 17, 2020 @04:36PM (#25130220)
To that point, what are systems supposed to make of UTF-8 strings encoding codepoints in the surrogate pair range? Is that well-defined?

In other words, to what extent are surrogate pairs a UTF-16 thing, rather than a Unicode thing that exists to accommodate for UCS-2 -> UTF-16?

Re:A Case for Properties: Programming Language Design on Sunday November 15, 2020 @02:13PM (#25103820)
One thing that's a bit unclear to me about properties, as described in this article, is how they should behave with respect to serialization.

For instance, in C#, both System.Text.Json and Newtonsoft.Json will get and serialize all properties, regardless of whether they are virtual "projections" (like the "red" property in the article's RgbColor example) or "plain" properties. Similarly, all properties are deserialized and set when doing the reverse.

This has the consequence that you may have redundant information in your serialization format. That's not the end of the world, and in fact there are some use-cases for this. The bigger consequence is that you can have redundant and conflicting information in the serialized data you read in.

For instance, adapting the author's example to C#:

https://try.dot.net/?fromGist=1bb01bd2b78a3212c52b40ff7955c0...

You get a different deserialization result depending on the order of the properties in the JSON, because the last property wins and the properties are mutually inconsistent with respect to the underlying data model.

Is the conclusion from this that objects which you intend to serialize should avoid using this "properties" feature, or else be extremely deliberate about what properties to include in the serialization format?

Re:A web of trust for NPM on Saturday October 03, 2020 @08:59PM (#24675860)
Not saying this is automatically a problem, but you will lose money over this. When a build dependency is unavailable or broken, you cannot create production artifacts. You'll lose money over this because broken builds lead to incidents, longer time-to-recovery during incidents, and slower development. All of these cost money.

Your exposure to defects in your build dependencies will depend on a lot of factors, but the exposure will always be there.

Re:Use long flags when scripting (2013) on Friday September 18, 2020 @08:34PM (#24523483)
Unfortunately, this isn't quite true. At least if by Unix you mean POSIX, i.e. you include macOS in your considerations.

1. Sadly, short options are in fact more portable than long options if you're targeting POSIX. For instance, macOS ships with versions of ls, rm, et cetera that only support short flags. This is because long options don't exist at all in POSIX (they're formalized here: https://pubs.opengroup.org/onlinepubs/009695399/functions/ge...).

2. Auto-completing partial options only happens for long flags, and it's a bit more than "some" parsers; the canonical implementation of "long" options, GNU's getopt_long, has it as a documented feature:

> Long option names may be abbreviated if the abbreviation is unique or is an exact match for some defined option.

https://linux.die.net/man/3/getopt_long

I 100% agree it's a poor feature. You basically entirely preclude yourself from adding features in a backwards-compatible way.

This is pretty widely thrown around, but it should be noted that the central purpose of Haussmannization was public health and beautification: tearing down the slums, building proper infrastructure like sewers and aqueducts, and pretty parks and streets. The article you link to emphasizes this.

The wide boulevards did very little to prevent the Paris Commune, for instance. Napoleon III just wanted to build a second Rome, not some martial panopticon.

Re:How to Write in Plain English on Tuesday August 25, 2020 @01:17PM (#24273633)
As others here have mentioned, the British government maintains similar guidance:

https://www.gov.uk/guidance/content-design/writing-for-gov-u...

As does the US federal government:

https://plainlanguage.gov/

Re:Apple, Epic, and the App Store on Monday August 17, 2020 @05:14PM (#24192191)
Lest your comment suggest the author of this article isn't aware of this point, he has made this argument precisely:

> Whereas China needed to control country-wide Internet access to achieve its censorship goals, Apple and Google have helpfully provided the Indian government with a one-stop shop. This also, for better or worse, gives a roadmap for how the U.S. government could respond to TikTok, if it chose to: there is no need to build a great firewall — simply give the order to Apple and Google. Centralization, at least from a central government’s perspective, has its uses.

https://stratechery.com/2020/india-bans-chinese-apps-the-app...

Re:Conventions for Command Line Options on Saturday August 01, 2020 @04:05PM (#24023460)
Another nice thing about env vars is that they don't appear in the process table, unlike argv. For many applications, that makes them a suitable place to inject secrets, and makes them convenient to inject via wrapper tools.

For instance, the fact that the aws(1) cli tool and the AWS SDKs by default take IAM creds from AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, etc. means that you can write programs like aws-vault (https://github.com/99designs/aws-vault) which can acquire a temporary AWS session and then execute another command with the creds injected in the standard AWS_... env vars. For instance:

    aws-vault exec prod-read -- aws s3 ls
    aws-vault exec stage-admin -- ./my-script-that-uses-the-aws-sdk-internally
Also, passing arguments as env vars is part of the "12-factor app" design (https://12factor.net/config). That page has some good guidance.
Re:Conventions for Command Line Options on Saturday August 01, 2020 @02:39PM (#24022817)
Most cli parsers don't fully support the author's suggested model because it means you can't parse argv without knowing the flags in advance.

For example, the author suggests that a cli parser should be able to understand these two as the same thing:

    program -abco output.txt
    program -abcooutput.txt
That's only doable if by the time you're parsing argv, you already know `-a`, `-b`, and `-c` don't take a value, and `-o` does take a value.

But this is a pain. All it gets you is the ability to save one space character, in exchange for a much complex argv-parsing process. The `-abco output.txt` form can be parsed without such additional context, and is already a pretty nice user interface.

For those of us who aren't working on ls(1), there's no shame in having a less-sexy but easier-to-build-and-debug cli interface.

Re:RBS, Ruby’s new type signature language on Wednesday July 29, 2020 @03:10PM (#23990471)
For other readers: "Sorbet" refers to https://sorbet.org/, Stripe's Ruby type checker.
You can also more easily generate schemas, or generate things from schemas, if they're in JSON.

Sure, you can still do these things when schemas are a non-JSON format, but it increases the barrier of entry to writing tooling on top of schemas.

This article has good discussion of this problem:

https://blog.dyvil.org/syntax/2016/04/19/angle-bracket-gener...

Re:JWT (JSON Web Token) debugger on Tuesday May 26, 2020 @03:00PM (#23315116)
You can pretty easily parse + verify JWTs in Bash using cut + base64 + openssl, although the precise invocations depend on which "alg"s you're using.
Re:Stripe responded to my concerns about user tracking on Thursday April 30, 2020 @08:29PM (#23037901)
Putting secret information in a URI is explicitly called out as a bad idea in the very RFC you reference:

> URI producers should not provide a URI that contains a username or password that is intended to be secret. URIs are frequently displayed by browsers, stored in clear text bookmarks, and logged by user agent history and intermediary applications (proxies).

https://tools.ietf.org/html/rfc3986#section-7.5

As an aside, if you're going to be making any security judgments from an RFC, every RFC since RFC2223 has a Security Considerations section. You usually want to start there.

Re:NHS rejects Apple-Google coronavirus app plan on Monday April 27, 2020 @01:15PM (#22997778)
From the the article:

> Germany had been in line with NHSX, but its government announced on Sunday it had switched tack to a "strongly decentralised approach".

Today's Stratechery (https://stratechery.com/2020/more-on-apple-and-google-german...):

> What is worth noting is why Germany changed their approach:

> > Germany as recently as Friday backed a centralized standard called Pan-European Privacy-Preserving Proximity Tracing (PEPP-PT), which would have needed Apple in particular to change the settings on its iPhones. When Apple refused to budge there was no alternative but to change course, said a senior government source.

> This is exactly what I was driving at in Coronavirus Clarity: tech companies are the ones setting the rules, not governments.

Re:Rust 1.43 on Thursday April 23, 2020 @01:37PM (#22958541)
To clarify for others: "anyhow" and "thiserror" are not placeholder names in that sentence. They refer to:

https://github.com/dtolnay/anyhow https://github.com/dtolnay/thiserror

I'm not sure that's true. The more Florence debases the silver florin while Venice stays honest, the more I'm tempted to melt my old florins into Venetian silver ducats. I only get a better and better deal that way.
Good luck! Seems like previous discussions with the mailing list could have gone better, but PASETO seems like promising work!
What IETF WG are you working through?
Re:Building dark mode on Stack Overflow on Saturday April 04, 2020 @05:45PM (#22781797)
The article mentions they use Bezier curves over the HSB colorspace.

> I used Lyft’s amazing Colorbox to help normalize our colors. Instead of a naive linear scale at 10% increments, I used bezier curves—a vast improvement at the more extreme ends of the scale.

Linear HSB is very naive, but HSB is still quite naive. It's just a cylindrical version of a color space optimized for displays, not human perception.

I don't understand why they wouldn't instead opt for the CIELAB color space, or its cylindrical equivalent. IBM does this, and it gives them effortless support for dark mode, by just flipping the luminance component of colors:

https://www.ibm.com/design/language/color

You can visualize the CIELAB behind IBM's color scheme here:

https://cielab.io/ (go to "IBM Carbon")

You can do this with the Auth0 APIs as well, no? By hitting the `/authorize` endpoint for saml-typed Auth0 connections? In that case Auth0 then acts as a SAML-to-OpenID Connect translation layer.

Older »