Re:Palantir Is Extending Its Reach Even Further into Government
by lolive on Tuesday August 05, 2025 @05:29AM
(#44795985)
99% of our pipelineBuilders are purely based on the available building blocks [that cover most of PySpark functions: https://www.palantir.com/docs/foundry/pipeline-builder/funct...]. Most notably we do join/filters/explode/group, Dozens of them, realigning data from different parts of the company.
Having most of those PySpark functions exposed visually as building blocks with debug points available at each step simply outperforms a codeWorkbook in term of maintainability and readability.
Re:Palantir Is Extending Its Reach Even Further into Government
by phendrenad2 on Tuesday August 05, 2025 @12:36PM
(#44800337)
Yeah as a general rule, when something requires "no coding" I question if they're too zoomed-in. In reality, you can just hire a programmer instead of a box-clicker.
...Except in this case, there might be a lot of value in letting James Bond or Boris Badenov whip up a quick spy pipeline without putting up a job ad for a Python coder.
These are slogans with marketing grade truth, not engineering truth. The moment you leave the happy path wizards you run straight into Python, SQL, JVM knobs, and cluster limits. The visual Pipeline Builder drops you into a Code node the second you need anything beyond basic joins/filters. Here is an example: https://www.palantir.com/docs/foundry/code-workbook/getting-...
>> nor scalability consideration. Foundry Spark jobs expose tunable variables, you are expected to tweak them for performance and cost: https://www.palantir.com/docs/foundry/optimizing-pipelines/s...