News for hackers. Stuff that matters.

nicornk

Re:AWS Acquires DuckDB on Wednesday August 26, 2026 @11:20AM (#49450737)
I hope AWS doesn’t dilute the DuckDB brand by calling their managed DuckDB offering SageMaker<something>…
Re:AWS Acquires DuckDB on Wednesday August 26, 2026 @09:40AM (#49449012)
This would be my bet and hope as well.
Re:Odoo: Open-Source ERP on Sunday January 04, 2026 @02:18AM (#46485742)
In that sense all popular ERP vendors seems to be the same.
Re:PYX: The next step in Python packaging on Thursday August 14, 2025 @04:20AM (#44898064)
I agree. Pixi solves all of that issues and is fully open source including the packages from conda-forge.

Too bad there is nowadays the confusion with anaconda (the distribution that requires a license) and the FOSS pieces of conda-forge. Explain that to your legacy IT or Procurement -.-

Re:Airport for DuckDB on Friday May 23, 2025 @02:03AM (#44070295)
Fivetran tried to upstream write support but it was not accepted https://github.com/duckdb/duckdb-iceberg/pull/95
If you just want to enable ssh to ec2 instances (through SSM) using ssh i-… you can add the following lines to your ssh config

https://gist.github.com/nicornk/5d2c0cd02179f9b46cc7df459af0...

host i-* IdentityFile ~/.ssh/id_rsa TCPKeepAlive yes ServerAliveInterval 120 User ec2-user ProxyCommand sh -c "aws ec2 start-instances --instance-ids %h ; aws ec2 wait instance-running --instance-ids %h ; aws ec2-instance-connect send-ssh-public-key --instance-id %h --instance-os-user %r --ssh-public-key 'file://~/.ssh/id_rsa.pub' --availability-zone $(aws ec2 describe-instances --instance-ids %h --query 'Reservations[0].Instances[0].Placement.AvailabilityZone') ; aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p'"

This will also allow VSCode remote development.

Re:Should You Ditch Spark for DuckDB or Polars? on Sunday December 15, 2024 @01:59PM (#42425213)
The blog post echos my experience that duckDB just works (due to superior disk spilling capabilities) and polars OOMs a lot.
Re:Snowflake, Cloud Storage Giant, Suffers Breach on Saturday June 01, 2024 @03:18AM (#40543627)
Snowflake usually unloads data to an internal stage bucket in the same region as your snowflake account. If you use an s3 gateway endpoint getting that data is free of egress charges.
Re:DataFusion Comet: Apache Spark Accelerator on Saturday June 01, 2024 @12:39AM (#40542923)
and this is unfortunately the case for Velox.
Re:Conditional Love for AWS Metadata Enumeration on Tuesday February 27, 2024 @09:52AM (#39524789)
Yeah, finally a method to figure out the org id of all those snowflake internal stage buckets that snowflake does not want to share for our VPC Endpoint Policies…
Re:BuildKit in depth: Docker's build engine explained on Tuesday February 06, 2024 @04:19PM (#39280788)
So how did you „further optimized [buildkit] to build Docker images up to 40x faster“
Re:Boring Python: dependency management (2022) on Monday February 05, 2024 @07:07AM (#39260348)
Pixi is great, highly recommended!
Re:The failure of self-checkout technology on Wednesday January 17, 2024 @09:59AM (#39028586)
There are Aldis in Germany with self checkout.
Thanks for sharing your experience. I always think of starting in that space but your post shows me it’s probably better to wait for a few more months.
With S3 you can do something similar my misusing the multiparty upload functionality, e.g.: https://github.com/fsspec/s3fs/blob/fa1c76a3b75c6d0330ed03c4...
Isn’t it a little bit unrealistic that your proxy fleet can keep up with the massive traffic that some s3 buckets experience? I think every solution that Proxies traffic to s3 will at a certain point break.
Re:AWS Outposts on Thursday December 02, 2021 @02:56PM (#29421606)
Uploading data into AWS is free.
Really interesting going through your code, thanks for sharing. Why did you opt for running the parallelization code in a separate python process (s3op)?

You might want to update section ‚Caution: Overwriting data in S3‘ in the docs since S3 offers strong read after write consistency since dec 2020.

https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-rea...

Re:What Comes After Zoom? on Tuesday June 23, 2020 @06:30AM (#23612073)
That‘s basically MS Teams.