dadbod
Nevertheless a good takeaway on how Ai can relentlessly pursue a target.
My other takeaway: vibing your way to a product can apparently generate a lot of tech debt along the way.
Ai created problem Ai solves said problems
How Many GPUs? A simple LLM inference sizing calculator
jsontap builds a reactive node tree on top of ijson's streaming parser. You await scalar/object values and async for over arrays while the JSON is still being fed in, one chunk at a time. This makes it ideal for LLM token streams, chunked HTTP responses, or any scenario where JSON arrives incrementally.
Jsontap – Reactive access to incrementally parsed JSON for Python
Simulate Apache Spark Workloads Without a Cluster Using FauxSpark
The current version includes:
- DAG scheduling with stages, tasks, and dependencies (but perhaps, designing around "RDD" would've been the right call)
- Modeling the input, output, shuffle partition sizes as probability distributions.
- Automatic retries on executor or shuffle-fetch failures
- Single-job execution
- Simple CLI to tweak cluster configuration, simulate failures, and scaling up executors
This tool might be relevant to the following folks:
- Data & Infrastructure engineers running Apache Spark who want to experiment with cluster configurations
- Anyone curious about Spark internals
I'd appreciate feedback from anyone with experience in discrete event simulation, particularly regarding the planned features as well as from anyone who may find this useful to shape its development.
A walkthrough section in the README demonstrates how it can be used.
Show HN: FauxSpark – An Apache Spark Simulator Using SimPy
https://github.com/fhalde/fauxspark
Hi HN,I've built FauxSpark. A discrete event simulation of Apache Spark, built with SimPy.It's designed to let users experiment with and understand the runtime characteristics of Apache Spark workloads under different cluster configurations, failures & job schedules without spawning a real cluster.In this initial version, FauxSpark implements a simplified version of Apache Spark which...
Customers: 10$
Jensen: 500$ :(
Customers: 20$
Jensen: 250$
Customer: 10$ + your jacket
Jensen: :'((