db.People.Where(p => p.Name == "Joe")
Since JS doesn't have this, they have to pass in a special placeholder value and try to record what the code is doing to that value.
I wonder why they don't just do `.toString()` on the mapping function and then parse the resulting Javascript into an AST and figure out property accesses from that. At the very least, that'd allow the code to properly throw an error in the event the callback contains any forbidden or unsupported constructs.
Unfortunately, "every object is truthy" and "every object can be coerced to a string even if it doesn't have a meaningful stringifier" are just how JavaScript works and there's not much we can do about it. If not for these deficiencies in JS itself, then your code would be flagged by the TypeScript compiler as having multiple type errors.
On a little less trivial skim over it looks like the intention here isn't to map property-level subsets returned data (e.g., only getting the `FirstName` and `LastName` properties of a larger object); as much as it is to do joins and it's not data entities being provided to the mapping function but RpcPromises so individual property values aren't even available anyway.
So I guess I might argue that map() isn't a good name for the function because it immediately made me think it's for doing a mapping transformation and not for basically just specifying a join (since you can't really transform the data) since that's what map() can do everywhere else in Javascript. But for all I know that's more clear when you're actually using the library, so take what I think with a heaping grain of salt. ;)
That sounds incredibly complicated, and not something we could do in a <10kB library!
The suggestion was to parse _JavaScript_. (That's what `.toString()` on a function does... gives you back the JavaScript.)
It feels like C# has an answer to every problem I’ve ever had with other languages - dynamic loading, ADTs with pattern matching, functional programming, whatever this expression tree is, reflection, etc etc. Yet somehow it’s still a niche language that isn't widely used (outside of particular ecosystems).
I've worked only at startups/small businesses since I graduated university and it's all been in C#.
fucking nice ecosystem
Pi types, existential types and built-in macros to name a few.
You were maybe already getting at it, but as a kitchen sink language the answer is "simplicity". All these diverse language features increase cognitive load when reading code, so it's a complexity/utility tradeoff
If you dare leave the safety of a compiler you'll find that Sublime Merge can still save you when rewriting a whole part of an app. That and manual testing (because automatic testing is also clutter).
If you think it's more professional to have a compiler I'd like to agree but then why did I run into a PHP job when looking for a Typescript one? Not an uncommon unfolding of events.
Granted, I started out on LISP. My version of "easy to read and write" might be slightly masochistic. But I love Perl and Python and Javascript are definitely "you can jump in and get shit done if you have worked in most languages. It might not be idiomatic, but it'll work"...
It does require twice the lines of PHP code to make a Ruby or Python program equivalent, or more if you add phpdoc and static types though, so it is easier to read/write Ruby or Python, but only after learning the details of the language. Ruby's syntax is very expressive but very complex if you don't know it by heart.
Specifically, I'd like to be able to have "inches" as a generic type, where it could be an int, long, float, double. Then I'd also like to have "length" as a generic type where it could be inches as a double, millimeters as a long, ect, ect.
I know they added generic numbers to the language in C# 7, so maybe there is a way to do it?
select(c for c in Customer if sum(c.orders.total_price) > 1000)
Along with https://pypi.org/project/pony-stubs/, you get decent static typing as well. It's really quite something.
It generally unrolls as a `for loop` underneath, or in this case LINQ/SQL.
C# was innovative for doing it first in the scope of SQL. I remember the arrival of LINQ... Good times.
Func<..> is lambda that can only be invoked.
Expression<Func..>> is an AST of a lambda that can be transformed by your code/library.
The only catch is your function needs to have no side effects (other than calling RPC methods). There are a lot of systems out there that have similar restrictions.
For any other function accepting a callback, the function on the server will receive an RPC stub, which, when called, makes an RPC back to the caller, calling the original version of the function.
This is usually what you want, and the semantics are entirely normal.
But for .map(), this would defeat the purpose, as it'd require an additional network round-trip to call the callback.
map() works for cases where you don't need to compute anything in the callback, you just want to pipeline the elements into another RPC, which is actually a common case with map().
If you want to filter server-side, you could still accomplish it by having the server explicitly expose a method that takes an array as input, and performs the desired filter. The server would have to know in advance exactly what filter predicates are needed.
But in the concrete:
* Looking up some additional data for each array element is a particularly common thing to want to do.
* We can support it nicely without having to create a library of operations baked into the protocol.
I really don't want to extend the protocol with a library of operations that you're allowed to perform. It seems like that library would just keep growing and add a lot of bloat and possibly security concerns.
(But note that apps can actually do so themselves. See: https://news.ycombinator.com/item?id=45339577 )
I did a spiritually similar thing in JS and Dart before where we read the text of the function and re-parsed (or used mirrors in Dart) to ensure that it doesn't access any external values.
I'm trying to understand how well this no-side-effects footgun is defended against.
https://github.com/cloudflare/capnweb/blob/main/src/map.ts#L... seems to indicate that if the special pre-results "record mode" call of the callback raises an error, the library silently bails out (but keeps anything already recorded, if this was a nested loop).
That catches a huge number of things like conditionals on `item.foo` in the map, but (a) it's quite conservative and will fail quite often with things like those conditionals, and (b) if I had `count += 1` in my callback, where count was defined outside the scope, now that's been incremented one extra time, and it didn't raise an error.
React Hooks had a similar problem, with a constraint that hooks couldn't be called conditionally. But they solved their DX by having a convention where every hook would start with `use`, so they could then build linters that would enforce their constraint. And if I recall, their rules-of-hooks eslint plugin was available within days of their announcement.
The problem with `map` is that there are millions of codebases that already use a method called `map`. I'd really, really love to see Cap'n Web use a different method name - perhaps something like `smartMap` or `quickMap` or `rpcMap` - that is more linter-friendly. A method name that doesn't require the linter to have access to strong typing information, to understand that you're mapping over the special RpcPromise rather than a low-level array.
Honestly, it's a really cool engineering solve, with the constraint of not having access to the AST like one has in Python. I do think that with wider adoption, people will find footguns, and I'd like this software to get a reputation for being resilient to those!
client.getAll({userIds}).map((user) => user.updatedAt == new Date().toLocaleString() ? client.photosFor(user.id) : {})
client.getAll({userIds}).map((user) => client.photos({userId: user.id, since: new Date(user.updatedAt).toLocaleString()})
You can't perform computation on a promise. The only thing you can do is pipeline on it.
`user.updatedAt == date` is trying to compare a promise against a date. It won't type check.
`new Date(user.updatedAt)` is passing a promise to the Date constructor. It won't type check.
One thing we were surprisingly able to do is trace the js spread operation as that is a rare case of something you can intercept in JS.
Kenton, if you are reading this, could you add a series of fake operators (eq, gt, in etc) to provide the capability to trace and perform them remotely?
But also, apps can already do this themselves. Since the record/replay mechanism already intercepts any RPC calls, the server can simply provide a library of operations as part of its RPC API. And now the mapper callback can take advantage of those.
I think this is the approach I prefer: leave it up to servers to provide these ops if they want to. Don't extend the protocol with a built-in library of ops.
const isHighValueCustomer = (row: { user: User; order: Order }) =>
row.user.active && row.order.amount > 1000
const isHighValueCustomer = (row: { user: User; order: Order }) =>
and(row.user.active, gt(row.order.amount, 1000)) let friendsWithPhotos = friendsPromise.map(friend => {
return {friend, photo: friend.has_photo ? api.getUserPhoto(friend.id) : default_photo};
}
You can't do this in most languages because of if statements, which cannot be analyzed in that way and break the abstraction. You'd either need macro-based function definitions (Lisp, Elixir), bytecode inspection (like in e.g. Pytorch compile), or maybe built-in laziness (Haskell).
Edit: Or full object orientation like in Smalltalk, where if statements are just calls to .ifTrue and .ifFalse on a true/false object, and hence can be simulated.
> .map() is special. It does not send JavaScript code to the server, but it does send something like "code", restricted to a domain-specific, non-Turing-complete language. The "code" is a list of instructions that the server should carry out for each member of the array.
> But the application code just specified a JavaScript method. How on Earth could we convert this into the narrow DSL? The answer is record-replay: On the client side, we execute the callback once, passing in a special placeholder value. The parameter behaves like an RPC promise. However, the callback is required to be synchronous, so it cannot actually await this promise. The only thing it can do is use promise pipelining to make pipelined calls. These calls are intercepted by the implementation and recorded as instructions, which can then be sent to the server, where they can be replayed as needed.