News for hackers. Stuff that matters.

tspann

not in New Jersey and New York City. There are many high end variante of both
look at sales engineering or make your owen AI startup. lots of seed money especially for MIT grad
Just use Apache NiFi
Re:Dismantling ELT: The Case for Graphs, Not Silos on Thursday November 28, 2024 @11:39AM (#42266627)
reminds me nifi plus kafka
Inkeep has been very solid for us and running in our Discord. https://milvus.io/community

Robert spoke at our meetup and is awesome. https://www.youtube.com/watch?v=35JdjmiDvWI

https://milvus.io/docs/integrate_with_llamaindex.md

Pretty easy to run local and lightweight with Milvus Lite with LlamaIndex

Re:Kafka as an Antipattern on Monday September 25, 2023 @08:38PM (#37652800)
On my laptop even a demo with 5000 records a second seems almost too few for Kafka.
Use Apache NiFi instead
Re:Bell Labs Holmdel Complex on Sunday July 16, 2023 @08:50PM (#36752908)
I host a meetup there sometimes, it feel so Blade Runner run out of electric. https://www.meetup.com/futureofdata-princeton/
That is good to hear. I worked in their ecommerce group for a year and it was interesting. Nice people. Surviving Amazon is amazing.
Re:Let's Monitor All the Things on Thursday October 13, 2022 @02:03PM (#33194395)
My setup for a recent talk at Current Event 22 in Austin.
I often need Dataflow, Spark Engineering, NoSql and Sql datastores, enterprise security and hybrid cloud with my datascience https://docs.cloudera.com/machine-learning/cloud/product/top...
I use Streams Messenging Manager supports schema registry for avro. Also includes alerts and many metrics and stats
Re:Apache Drill's Future on Thursday May 21, 2020 @02:01AM (#23255293)
Just use a Apache NiFi fast and does that on 1 step
Kafka for events Flink for continous sql against topic events Kudu for event storage. Databases for data storage and queries
Re:Apache NiFi on Monday May 11, 2020 @03:29PM (#23145987)
Thanks! I wrote a few more of those. https://github.com/tspannhw/ExecuteClouderaML
Re:Apache NiFi on Monday May 11, 2020 @03:27PM (#23145972)
No more XML. Check out NiFi 1.11.4, it does everything you need for easy ingest. If you are reading some files putting them into Kafka or S3 or a database or MongoDB or Hbase or Hive or Impala or Oracle or Kudu or ..., it's genius.

https://www.datainmotion.dev/

Re:KafkaHQ on Monday December 30, 2019 @08:18AM (#21913132)
SMM is another option and has some really cool features for alerting as well. Runs in any cloud or on premise.

https://blog.cloudera.com/smm-1-2-released-with-powerful-new...