News for hackers. Stuff that matters.
Re:How much faster is Redis at storing a blob of JSON compar... by fizx on Wednesday October 02, 2019 @11:53AM (#21137803)
A reasonably configured Postgres is easily sub-millisecond for reads and writes on a small dataset like this. I suspect that the author is doing something quite wrong.

People generally reach for Redis too quickly, IMO. Postgres is about as fast, has better tooling, and will slow down gracefully as the dataset increases, rather than evicting records as the limit is hit.

That said, I guess if you're relatively junior, it's hard to grossly misconfigure Redis, so that's a point in its favor. Somewhat like Java vs Node.js--you can write higher performance network services in Java, but it's a lot easier to make mistakes.

Re:How much faster is Redis at storing a blob of JSON compar... by Benjamin_Dobell on Wednesday October 02, 2019 @12:07PM (#21137988)
> That said, I guess if you're relatively junior, it's hard to grossly misconfigure Redis, so that's a point in its favor.

Redis persistence (or possibly lack-thereof) is probably quite complicated for a junior developer to understand. I'd actually expect a better outcome with PSQL.

Re:How much faster is Redis at storing a blob of JSON compar... by swsieber on Wednesday October 02, 2019 @02:51PM (#21139581)
I disagree. But anytime I've been exposed to redis, it's always been as an ephemeral data store.
Re:How much faster is Redis at storing a blob of JSON compar... by threeseed on Wednesday October 02, 2019 @04:29PM (#21140437)
Not sure what you are getting at here.

Redis is one of the simplest data stores around to use and its persistence is solid.

Re:How much faster is Redis at storing a blob of JSON compar... by erichocean on Wednesday October 02, 2019 @07:36PM (#21141945)
> Redis is one of the simplest data stores around to use and its persistence is solid.

I've lost countless amount of data in production systems running Redis. Haven't ever lost anything in Postgres. (I still run both.)

Re:How much faster is Redis at storing a blob of JSON compar... by Someone on Wednesday October 02, 2019 @05:34PM (#21140987)
There’s

  class AmazonAffiliateLookup(models.Model, TotalCountMixin)
I guess that TotalCountMixin adds some triggers that maintain a count of the records in the table in a separate table (could or could not be a good idea, depending on the table size, relative frequency of ‘give me the number of items’ calls relative to inserts and deletes, etc, but might introduce a bottleneck on updating that count)

There also is:

  song = models.ForeignKey(Song, on_delete=models.CASCADE)
I don’t understand why that foreign key is there, but it can’t speed up inserts and deletes.

Then, we have:

  created = models.DateTimeField(auto_now_add=True, db_index=True)
I think creating an index is done to implement the equivalent of Redis’ auto-expiry.

Again, that may be suboptimal if there are few records in the database. DateTimeField also may be ‘too structured’ for the task at hand (it might needlessly parse year/month/day out of time stamp in nanoseconds, for example, or even do time zone calculations to convert a local time to UTC), but I don’t know. It would be helpful if the text showed the actual SQL DDL.

Re:How much faster is Redis at storing a blob of JSON compar... by tracker1 on Thursday October 03, 2019 @01:52PM (#21149788)
Probably, hard to tell, the biggest difference, is Redis is likely storing the JSON as a string, and Posgres is storing it as binary json (separated out into distinct values/collections). I'd guess if Postgres was using UTF8 text for the JSON it would be closer. Also, Postgres is probably a bit less efficient on persistance requirements in favor of higher stability and preventing data loss.

In any case, Redis is probably a more natural and better fit (remote API result caching). That said, the comparison is awkward at best. If it were to cache localized results, where the dbms (PostgreSQL) is the source of truth, then adding a caching layer above the db may or may not be worth it.

I think the performance comparisons come down to Granny Smith Apples vs Rome Apples... you can bake a pie with either, but they don't taste the same.

Re:How much faster is Redis at storing a blob of JSON compar... by throwawayy98121 on Wednesday October 02, 2019 @12:33PM (#21138306)
Isn’t the fundamental difference also use case? You would have to use Postgres if you need actual durability. AFAIK, one outage and your redis cache is gone... unless you’re running an HA cluster (which I dunno if that’s offered by redis). But even that doesn’t mean you have durability.
Re:How much faster is Redis at storing a blob of JSON compar... by derefr on Wednesday October 02, 2019 @02:22PM (#21139302)
Redis can be configured as `appendonly yes; appendfsync always`, which the Redis docs say is “very very slow”, but really would just make Redis work the way Postgres does with its WAL, with probably equivalent performance (or still slightly better, since there’s no checkpointing going on.)
Re:How much faster is Redis at storing a blob of JSON compar... by iliketoworkhard on Wednesday October 02, 2019 @01:49PM (#21139021)
In general caching systems are volatile and not meant to be used to store data that cannot be afforded to be lost. Of course that doesn't stop clients from storing critical data on there in return for the simple GET/PUT APIs and low latencies.

You can go for more complex caches such as Hazelcast which offers hot restarts by persisting data to disk.

Re:How much faster is Redis at storing a blob of JSON compar... by threeseed on Wednesday October 02, 2019 @04:26PM (#21140411)
You have no evidence that:

a) PostgreSQL has sub-millisecond read/writes for storing JSON blobs than Redis.

b) Java results in buggier code than Javascript.

If you're going to make bold claims the onus is on you to provide evidence to back it up.