News for hackers. Stuff that matters.
Re:Never use a dependency that you could replace with an aft... by axlee on Tuesday August 11, 2020 @03:39PM (#24125010)
Is that really 5 minutes? (For when left-pad was relevant)

  var cache = [
  '',
  ' ',
  '  ',
  '   ',
  '    ',
  '     ',
  '      ',
  '       ',
  '        ',
  '         '
  ];
  
 function leftPad (str, len, ch) {

  // convert `str` to a `string`
  str = str + '';
  // `len` is the `pad`'s length now
  len = len - str.length;
  // doesn't need to pad
  if (len <= 0) return str;
  // `ch` defaults to `' '`
  if (!ch && ch !== 0) ch = ' ';
  // convert `ch` to a `string` cuz it could be a number
  ch = ch + '';
  // cache common use cases
  if (ch === ' ' && len < 10) return cache[len] + str;
  // `pad` starts with an empty string
  var pad = '';
  // loop
  while (true) {
    // add `ch` to `pad` if `len` is odd
    if (len & 1) pad += ch;
    // divide `len` by 2, ditch the remainder
    len >>= 1;
    // "double" the `ch` so this operation count grows logarithmically on `len`
    // each time `ch` is "doubled", the `len` would need to be "doubled" too
    // similar to finding a value in binary search tree, hence O(log(n))
    if (len) ch += ch;
    // `len` is 0, exit the loop
    else break;
  }
  // pad `str`!
  return pad + str;
}
Re:Never use a dependency that you could replace with an aft... by ghj on Tuesday August 11, 2020 @04:03PM (#24125312)
This is so hilariously overengineered: not failing when passed the wrong types, arbitrarily caching padding of len < 10, etc

The most egregious is the bad big O analysis for a pointless binary search. The loop does indeed run O(log(n)) times but `ch += ch` still takes O(ch.length) which is growing exponentially. It ends up being a complicated way of still taking O(n) time while creating a lot of intermediate strings.

It isn't any faster than just creating the padding with a loop or `new Array(len).fill(ch).join('')` or `ch.repeat(len)`

Re:Never use a dependency that you could replace with an aft... by Ajedi32 on Tuesday August 11, 2020 @04:23PM (#24125538)
Did you run benchmarks to validate those assumptions? The developers of left-pad did: https://github.com/left-pad/left-pad/blob/master/perf/perf.j...

It's not overengineered if thousands of downstream projects are relying on it, some of which might see significant benefits from those performance optimizations.

Re:Never use a dependency that you could replace with an aft... by cogman10 on Tuesday August 11, 2020 @04:42PM (#24125746)
Too bad that's a flawed benchmarking methodolgy. JITs are notoriously hard to correctly profile and the benchmark lib isn't even sort of doing the right thing.

For example, it's missing warmup. The results aren't being consumed in a way that wouldn't optimize them away. The framework itself imposes a pretty large amount of overhead (moreso than I'd expect from leftpad).

It is somewhat likely that what they are measuring isn't leftpad performance, but rather how fast the JIT ends up optimizing the benchmark code.

I'd suggest watching this video https://vimeo.com/78900556

It's about microprofiling the JVM, but the principles are the same for other JIT based languages (such as javascript).

Re:Never use a dependency that you could replace with an aft... by Ajedi32 on Tuesday August 11, 2020 @05:01PM (#24125922)
Yeah benchmarks can be much more difficult to get right than it might seem at first glance. Good thing they didn't try to write their own benchmarking code, otherwise they might have fallen into those traps you just mentioned.

Luckily, they didn't, and instead pulled in the the `benchmark` library as a development dependency. The author of said library works on V8, and already considered all those problems and much, much, more[1].

[1]: https://mathiasbynens.be/notes/javascript-benchmarking

Re:Never use a dependency that you could replace with an aft... by cogman10 on Tuesday August 11, 2020 @05:19PM (#24126109)
You are making the assumption that the library doesn't have the flaws I mentioned. It does (You can go read the source yourself https://raw.githubusercontent.com/bestiejs/benchmark.js/2.1.... )

There's no portion of the code that does warmups. There's no portion of code that "blackholes" the results to keep the JIT from optimizing away the code under benchmark. There is a lot of code though... so that's... good?

You make the assumption that just because a lib is popular or widely used it is "correct" or "the best". When it comes to microbenchmarks, that's usually flawed. Very VERY few people actually get them right, benchmark.js is no exception.

That, of course, doesn't mean that benchmark.js can't be useful. For macrobenchmarks it will be roughly right. However, for something as small as leftpad, it's almost certainly not the right way to measure performance.

Re:Never use a dependency that you could replace with an aft... by phpnode on Tuesday August 11, 2020 @08:12PM (#24127391)
> There's no portion of the code that does warmups.

This isn't true - Benchmark.js will repeatedly rerun benchmarks until it gathers statistically meaningful results.

> There's no portion of code that "blackholes" the results to keep the JIT from optimizing away the code under benchmark.

True, and there's actually nothing benchmark.js can do to ensure that doesn't happen in the general case but when this does happen the results are usually pretty obvious - we'd see billions of ops/sec. Incidentally the left-pad benchmarks do not suffer from this issue.

Re:Never use a dependency that you could replace with an aft... by Ajedi32 on Tuesday August 11, 2020 @05:26PM (#24126169)
Okay, you win. I'm not going to read the entire source of that package just to make a point. Though I do find it strange that the author of that library would write an entire blog post on the topic and then not take his own advice in the implementation of the library he wrote.
Re:Never use a dependency that you could replace with an aft... by cogman10 on Tuesday August 11, 2020 @05:37PM (#24126291)
Tell me, where in that blog post does he mention doing warm up cycles or avoiding having the JIT optimize away the method? (Hint, he doesn't mention that... so, no, he didn't actually miss his own advice.)

The article is completely consumed with getting the timing of benchmarking right. Which, to be fair, is a place where microbenchmarks often go wrong. It, however, isn't the ONLY place they go wrong.

Re:Never use a dependency that you could replace with an aft... by cjaybo on Wednesday August 12, 2020 @02:23PM (#24135106)
> Luckily, they didn't, and instead pulled in the the `benchmark` library as a development dependency. The author of said library works on V8, and already considered all those problems and much, much, more[1].

This is the "benchmark lib" that was mentioned in the very first sentece of the comment you replied to.

Re:Never use a dependency that you could replace with an aft... by ghj on Tuesday August 11, 2020 @05:09PM (#24126014)
They failed at writing a correct O(n) left pad in their benchmark:

https://github.com/left-pad/left-pad/blob/master/perf/O(n).j...

They are repeatedly appending a single character to the front of a string. This is actually O(n^2) so of course they are winning against it.

Re:Never use a dependency that you could replace with an aft... by Ajedi32 on Tuesday August 11, 2020 @05:24PM (#24126154)
That would depend on the underlying implementation of the string object, no? (If the string is implemented as a linked list it's O(n))

Anyway, it makes no practical difference in this case, since the one they labeled "O(n)" is the naive implementation that most people would write if they implemented left-pad themselves.

Re:Never use a dependency that you could replace with an aft... by ghj on Tuesday August 11, 2020 @05:28PM (#24126183)
I highly doubt js engines would compile a string down to a linked list. But you're right they might compile it to a circular buffer or deque which can have O(1) prepends.

Quick googling shows that this optimization might exist but only for firefox and only if you use "unshift": https://lannonbr.com/blog/2020-01-27-shift-optimizations https://jandemooij.nl/blog/2017/12/06/some-spidermonkey-opti...

But it's very unlikely that the jit can optimize `str = ch + str;`

Re:Never use a dependency that you could replace with an aft... by cogman10 on Tuesday August 11, 2020 @07:00PM (#24126956)
Depends on the form that `str = ch + str` takes inside the loop. But yeah, the way they wrote the code makes it less likely to work well.

    let padding = '';
    for (let i = 0; i < len; ++i)
      padding += ch;
    return padding + str;
The above would probably get caught by the JIT and would essentially be optimized to what `ch.repeat(len) + str;` would do.
Re:Never use a dependency that you could replace with an aft... by layoutIfNeeded on Tuesday August 11, 2020 @05:46PM (#24126406)
String implemented as a linked list.

You’re a webdev aren’t you?

Re:Never use a dependency that you could replace with an aft... by imtringued on Wednesday August 12, 2020 @06:24AM (#24130377)
Re:Never use a dependency that you could replace with an aft... by Domenic_S on Wednesday August 12, 2020 @04:05PM (#24136230)
Haskell's default String implementation is a linked list; no need for the snark
Re:Never use a dependency that you could replace with an aft... by TheDong on Tuesday August 11, 2020 @03:52PM (#24125164)
One of the points of the article is that when you code a dependency for your purpose, you get smaller code. For example, I know I'll always pass it a string, so I don't need to type-check.

I started with the problem of "I need a function that pads a string with a character out to some length". Coding it up took under 1 minute, easily under 5 minutes.

    function leftPad (str, len, ch) {
      const neededPadding = len - str.length;
      if (neededPadding <= 0) {
        return str;
      }
      return ch.repeat(neededPadding) + str;
    }
It took me longer to write this comment.

The fact that the left-pad code has optimizations (which don't matter for the place I'm using it) and type-checks (which don't matter; my higher level unit tests would catch that mistake) is beside the point.

Re:Never use a dependency that you could replace with an aft... by scarface74 on Tuesday August 11, 2020 @04:45PM (#24125766)
Unless you are doing front end code, why does the size of the code matter?
Re:Never use a dependency that you could replace with an aft... by Ma8ee on Tuesday August 11, 2020 @05:04PM (#24125947)
Less code is (in general, to certain limits) easier to understand and requires less maintenance.
Re:Never use a dependency that you could replace with an aft... by scarface74 on Tuesday August 11, 2020 @05:17PM (#24126082)
If I am using code that someone else writes and maintains it is easier to maintain.

But I can maintain code that uses Entity Framework (or Dapper) much easier than I can maintain code based on a custom ORM that the “architect“ wrote.

Re:Never use a dependency that you could replace with an aft... by Ma8ee on Tuesday August 11, 2020 @07:06PM (#24127007)
As long as it works and is well documented, yes. But the moment there’s a bug somewhere or the docs aren’t adequate for your needs, you are in much worse trouble.
Re:Never use a dependency that you could replace with an aft... by scarface74 on Tuesday August 11, 2020 @07:44PM (#24127240)
Again my general rule - you (generic you) are no special snowflake. What are the chances that you come across a bug in a library that has had 2,176,677 (in the case of Dapper) or Entity Framework (supported by MS) that no one else has come across, found a workaround, and posted the answer somewhere on the Internet compared to your code where you didn’t think about a corner case?
Re:Never use a dependency that you could replace with an aft... by danShumway on Wednesday August 12, 2020 @12:51AM (#24128690)
Literally all the time.

I can count on one hand[0] the number of JS dependencies I've used in enterprise/large projects for extended periods of time where I've never needed to manually debug the library or read the source code. For enterprise software, sometimes the easiest solution is to directly hotpatch a vendored version of the dependency.

This is especially true with dependencies where bugs are fixed in major versions, but where upgrading and dealing with breaking changes would require significant code refactoring.

To drive the point home, I've been bitten by bugs in NPM itself.[1] Fixing that required reading through the source and manually swapping out one NPM's internal dependencies to a newer version.

And it doesn't matter if someone somewhere has had the same problem and posted it on the Internet unless I can find their answer online faster than I can fix the problem myself in my own library. Often this is not the case, filtering through issue trackers and trying to find the one blog post or comment that tells me how to solve the problem can be a big time sink.

[0]: Okay, maybe 2. But the point stands, it's not a rare or exceptional occurance.

[1]: https://github.com/npm/npm/issues/18942

Re:Never use a dependency that you could replace with an aft... by BigJono on Wednesday August 12, 2020 @03:28AM (#24129468)
I've had some serious train wrecks because of shit NPM libraries over the years.

Probably the worst was with a decently popular library someone had brought in, that tried to do a refactor from callbacks to async/await, without understanding at all how async/await worked. They'd leaked an async operation in the library code, so 'await'ing a specific function call in their API that returned a promise, didn't actually await everything the call was doing, ending up in a debugging nightmare. Of course their perfectly manicured suite of 8000 tests with 110% test coverage didn't catch it either, because the number of people who can write good quality tests is shockingly low, and library-writers aren't somehow magically ahead of the pack in that regard.

JS really feels like PHP did back when I was a newbie learning that shit. In other ecosystems, the 95th percentile devs seem to write all the libraries, so everyone comes here and posts repeatedly about how great dependencies are. In JS, it's the average dev writing all the libraries, and the average dev's code is enough to make my brain bleed.

I'm a big proponent of different advice for different ecosystems. If you're doing front-end JS, the pendulum has swung so far to one side that 'NIH syndrome' is treated like it's going to lead to the fourth reich, which makes 'chill out a bit on dependencies' pretty good advice if you're looking to get a leg up in the industry. But I'm sure there's other ecosystems where the same advice will just leave you with a tangled mess while your competitors leapfrog you in productivity with a good 3rd party dependency.

I'd say take any advice in threads like these with a grain of salt unless it's given in a bit of a narrower context. Taking some one liner about software engineering in general and applying it to your specific project is probably just a coin flip as to whether it's going to improve your code or not.

Re:Never use a dependency that you could replace with an aft... by TheDong on Tuesday August 11, 2020 @04:49PM (#24125806)
"smaller" there was short-hand for "more focused, purpose-built to your problem, less generic".

All of those properties make the code easier to reason about and test.

Being easier to reason about, and being a more exact abstraction for my needs, are both incredibly valuable properties.

Re:Never use a dependency that you could replace with an aft... by scarface74 on Tuesday August 11, 2020 @04:52PM (#24125841)
More generic is better. If I go into a codebase that uses standard libraries. Even if I don’t know how to do something about it someone on the Internet does. Your custom framework - not so much.

I don’t have to care about how the underlying libraries work. I can treat them as a black box.

Re:Never use a dependency that you could replace with an aft... by TheDong on Tuesday August 11, 2020 @05:04PM (#24125952)
> I don’t have to care about how the underlying libraries work. I can treat them as a black box.

When I wrote haskell, the majority of the libraries did just work, and I didn't have to dig into their code to find bugs often.

When I wrote javascript, hundreds of the libraries I used did not just work. I usually had to care very much about their details because they were poorly implemented, full of bugs and incorrect abstractions, and often abandoned soon after.

I agree that there's benefits in reusing some well-socialized and well-implemented generic frameworks and abstractions. It's not worth using generic abstractions that are not well understood, buggy, and don't match your needs closely. In that case, write your own.

More generic is not always better. Above, I'm arguing that it's important for code to be easier to reason about. If a generic abstraction helps with that, cool, but it's not always going to be the case.

Re:Never use a dependency that you could replace with an aft... by scarface74 on Tuesday August 11, 2020 @06:14PM (#24126649)
That’s also why I stay away from the clusterf%%% of front end development and JS if possible except for simple AWS Lambda scripts that have one dependency - AWS SDK.

Any other scripting I do with Python. Any more complicated development it’s using a language with an ecosystem with adults - C# or Go.

Re:Never use a dependency that you could replace with an aft... by TheDong on Tuesday August 11, 2020 @06:51PM (#24126917)
Kinda ironic that you say "More generic is better" a few comments above, and then also cite Go in a positive light.

Rob pike espoused "A little copying is better than a little dependency", and go refuses to add suitable abstractions to build generic reusable pieces.

Go has a strong culture of doing exactly the sort of thing I was talking about, and you were arguing against.

Re:Never use a dependency that you could replace with an aft... by toast0 on Tuesday August 11, 2020 @05:27PM (#24126180)
What do you do when the black box doesn't work?

Almost everything I work with has bugs, so chances are I'm going to run into one. It's a lot easier for me to fix bugs when there are fewer layers and more of them are written by me. Of course, I can't write all the layers, but if they run on my service, I have to be prepared to fix them, or suffer from them being broken until a benevolent force fixes them for me. (Sometimes that happens, but usually not for the harder problems)

Re:Never use a dependency that you could replace with an aft... by scarface74 on Tuesday August 11, 2020 @06:15PM (#24126658)
Until I have to come in behind you after you’ve left for greener pastures and I have to figure out your bespoke framework and libraries.
Re:Never use a dependency that you could replace with an aft... by toast0 on Tuesday August 11, 2020 @06:24PM (#24126712)
Don't worry, I would never write a bespoke framework. I'd write a custom one. :P
Re:Never use a dependency that you could replace with an aft... by cogman10 on Tuesday August 11, 2020 @03:53PM (#24125182)
This is actually a great example of why you SHOULDN'T use left pad.

I've not profiled it, but I'm going to guess that now-a-days this will be faster than the current implementation on npm.

    function leftPad (str, len, ch) {
      str += '';
      len = len - str.length;
      if (len <= 0) return str;
      if (!ch && ch !== 0) ch = ' ';
      ch += '';
      return ch.repeat(len) + str;
    }
Why? because the VM is (very likely) going to do exactly what the cache would have done. It can replace `ch.repeat(len) + str;` with a presized string allocation and a memcpy of ch + str characters.
Re:Never use a dependency that you could replace with an aft... by goldenkey on Tuesday August 11, 2020 @03:56PM (#24125216)
This code checks for ch !== 0, yet the bypass branch afterwards calls .repeat. The Number type in JS has no repeat method.

Static typing would actually fix this kind of coding error.

Re:Never use a dependency that you could replace with an aft... by reificator on Tuesday August 11, 2020 @04:00PM (#24125282)
> This code checks for ch !== 0, yet the Number type in JS has no repeat method.

> Static typing would actually fix this kind of coding error.

That's not a coding error, that's addressing an oddity of JS coercion rules that a less experienced developer could easily have missed.

> if (!ch && ch !== 0) ch = ' '

That code says that if `ch` is falsey and not equal to 0, then set it to a space. The only arguable falsey value that should be excluded here is a literal `false`, but that's not a single character and is fairly ambiguous either way. I'd certainly fall on the side that a literal false should not be converted to `'false'` here.

> ch += '';

The next line converts to to a string by adding it to the empty string.

> return ch.repeat(len) + str;

So by the time it gets to this line we know ch is a string.

Static typing is great, but the bug you claim is there is not actually there.

Re:Never use a dependency that you could replace with an aft... by cogman10 on Tuesday August 11, 2020 @04:07PM (#24125363)
FYI, I edited the post after the bug was pointed out to eliminate it and bring this method up to parity with leftpad.
Re:Never use a dependency that you could replace with an aft... by microcolonel on Tuesday August 11, 2020 @03:57PM (#24125233)
What magic would enable that? I don't think the VM is likely to cache a string of 6 spaces, it has no way of knowing that would be a common parameter, and no heuristic to determine that it's likely.

It may specialize or inline, but that's a separate matter.

P.S. I think there's a bug here.

Re:Never use a dependency that you could replace with an aft... by cogman10 on Tuesday August 11, 2020 @04:10PM (#24125395)
JIT is the magic sauce and this a pretty regular optimization.

    Step 1, inline repeat.
    Step 2, remove the intermediate array allocation
    Step 3, allocated a string array sized for the pad + str
    Step 4, Use one of the many CPU instructions to repeatably copy the padding character and then the `str` into the same array of characters.
None of these optimization would be out of the question for the Jit (and I'd expect them). You don't need the cache at all, it's just a waste. The only thing it saves it creating the intermediate string which is HIGHLY likely to be optimized away with the simple code.
Re:Never use a dependency that you could replace with an aft... by Ajedi32 on Tuesday August 11, 2020 @04:35PM (#24125660)
They benchmarked it. Their implementation was faster than ch.repeat: https://github.com/left-pad/left-pad/pull/11#issuecomment-20...

Nowadays there's a native padStart function and the left-pad package is deprecated as a result.

Re:Never use a dependency that you could replace with an aft... by cogman10 on Tuesday August 11, 2020 @04:47PM (#24125784)
They used a flawed benchmarking methodology.

JITs only optimize hot code which means any benchmark without a warmup is going to measure cold + hot code time.

Further, JITs will optimize away unused results. They aren't using the leftpad results in any meaningful way.

What they are likely measuring is how long it takes for the JIT to optimize the benchmarking framework.

watch: https://vimeo.com/78900556 for how to microbenchmark a JITed language.

Re:Never use a dependency that you could replace with an aft... by Ajedi32 on Tuesday August 11, 2020 @05:06PM (#24125974)
Yeah benchmarks can be much more difficult to get right than it might seem at first glance. Good thing they didn't try to write their own benchmarking code, otherwise they might have fallen into those traps you just mentioned.

Luckily, they didn't, and instead pulled in the the `benchmark` library as a development dependency[1]. The author of said library works on V8, and already considered all those problems and much, much, more[2].

[1]: https://github.com/left-pad/left-pad/blob/master/perf/perf.j...

[2]: https://mathiasbynens.be/notes/javascript-benchmarking

Re:Never use a dependency that you could replace with an aft... by cogman10 on Tuesday August 11, 2020 @05:24PM (#24126151)
I know, I read the benchmark suite code. The benchmark suite itself isn't doing the things I said it isn't doing.

Sure, It's popular. It's also wrong.

Re:Never use a dependency that you could replace with an aft... by Ajedi32 on Tuesday August 11, 2020 @05:27PM (#24126175)
Okay, you win. I'm not going to read the entire source of that package just to make a point. Though I do find it strange that the author of that library would write an entire blog post on the topic and then not take his own advice in the implementation of the library he wrote.
Re:Never use a dependency that you could replace with an aft... by tluyben2 on Tuesday August 11, 2020 @04:18PM (#24125477)
Well, my point was more that if a programmer thinks '5 minutes work' it's often 10+ minutes; so when a programmer thinks 'an afternoon', you can possibly lose a week. And then the article really doesn't work.

And yeah, I would and do write leftpad myself it it's not in the stdlib. But if there is a large library full of similar (string) functions that I might need, I would include that library. Not a singular dependency for this type of function.

Re:Never use a dependency that you could replace with an aft... by reificator on Tuesday August 11, 2020 @03:57PM (#24125223)
Yes, particularly when you know the types that will be passed in. True that typechecking in JS can be a little timeconsuming, so it might take 10-15 for me to write a general purpose left pad.

Also I seem to remember that someone benchmarked the cached version and found it to be slower than the naive approach anyway. I could be mistaken there.