News for hackers. Stuff that matters.
Re:Appropriate Uses for SQLite by fauigerzigerk on Monday February 12, 2018 @09:43AM (#16358835)
As I said in my reply to asaph, it doesn't work well with interest rates and exchange rates. Also, it simply adds to the mental load in many places (e.g ad hoc queries)
Re:Appropriate Uses for SQLite by TeMPOraL on Monday February 12, 2018 @09:54AM (#16358919)
And floats do? With all the (implicit) rounding and precision issues, and some base-10 numbers not being representable in an exact form?

Serious question, what's the proper way to handle this case? I'm having hard time to believing it's IEEE-754 floats.

Re:Appropriate Uses for SQLite by aldanor on Monday February 12, 2018 @10:09AM (#16359036)
The proper way to handle this is fixed-point representation (whether the DB natively supports this or not). E.g., you won't find floating point fields in binary protocols for any of the major stock exchanges. A typical scaling factor is 10^8, so if the exchange wants to tell you that the price is now 101.27, they send you an int64 value of 10127000000. You may then choose to take it further and store e.g. in 128-bit decimal.

Of course, there's an issue of how to handle mul/div (i.e., rounding/truncation), but that's a separate question -- at least you'd be making decisions yourself rather than relying on how IEEE-754 floats work.

Some DBs like PG, do, as a matter of fact, support decimals natively which makes things quite a bit simpler.

Re:Appropriate Uses for SQLite by KMag on Monday February 12, 2018 @01:13PM (#16360545)
Certainly for decimalized exchanges, binary floating point is the wrong choice. (On a side note, it's also the wrong choice for spread sheets, at least as the default. The average freshly graduated accountant can have a reasonable grasp on decimal floating point rounding after a 5 minute explanation, whereas most professional software engineers with a decade of experience have a very weak understanding of IEEE 754 binary floating point.)

> E.g., you won't find floating point fields in binary protocols for any of the major stock exchanges.

That's true of NASDAQ's OUCH, but some (many?) other protocols use decimal floating point.

NYSE's UTP Direct[0] (See section 4.15) Is not fixed point, but actually their own decimal floating point system with a 32-bit significand and single ASCII character exponent ("0" to "6", indicating negated base 10 exponent, not allowing "5").

FIX SBE uses an 8 byte significand and a 1 byte (power-of-10) exponent, if I remember correctly.

It's a bit unfortunate that (as far as I know) there aren't any major messaging formats supporting IEEE 754-2008 decimal64 values.

[0]https://www.nyse.com/publicdocs/nyse/markets/nyse/NYSEUTPDir...

Re:Appropriate Uses for SQLite by elteto on Monday February 12, 2018 @10:08AM (#16359028)
OP mentioned a decimal data type, which in postgres can have a very large precision: https://www.postgresql.org/docs/10/static/datatype-numeric.h.... Not the same as a floating point.
Re:Appropriate Uses for SQLite by anarazel on Monday February 12, 2018 @11:00AM (#16359463)
16383 digits ought to be enough for everyone:

"up to 131072 digits before the decimal point; up to 16383 digits after the decimal point"

Non jokingly: I think precision wise that's really plenty enough for pretty much any halfway realistic usecase (no, don't do public key crypto using SQL datatypes). There's some issues with the numeric / decimal type, but lack of potential precision isn't one of them.

Re:Appropriate Uses for SQLite by fauigerzigerk on Monday February 12, 2018 @10:03AM (#16358985)
No, binary floating point doesn't work at all. Decimal floating point can work if the representation is large enough.
Re:Appropriate Uses for SQLite by icebraining on Monday February 12, 2018 @10:07AM (#16359020)
Arbitrary-precision numbers. DBs like Postgres support them, as do most languages (e.g. Ruby has BigDecimal, Python has Decimal).
Re:Appropriate Uses for SQLite by aeorgnoieang on Monday February 12, 2018 @10:34AM (#16359231)
Neither `BigDecimal` in Ruby or `Decimal` in Python are quite what I'd expect for "arbitrary-precision".

From the docs for [`BigDecimal`](http://ruby-doc.org/stdlib-1.9.3/libdoc/bigdecimal/rdoc/BigD...):

> Ruby provides built-in support for arbitrary precision integer arithmetic. For example: > > 4213 -> 1265437718438866624512 > > BigDecimal provides similar support for very large or very accurate floating point numbers.

My takeaway from the above is that, if arbitrary precision is needed, you should still just use integers. Tho, in practice, it may not be significant.

Re:Appropriate Uses for SQLite by fauigerzigerk on Monday February 12, 2018 @11:34AM (#16359735)
Integers are not magic. If you want arbitrary precision on top of integers you're going to need arbitrarily long integers as well.
Re:Appropriate Uses for SQLite by aeorgnoieang on Monday February 12, 2018 @02:32PM (#16361300)
I was referring to Ruby and based on the first sentence I quoted:

> Ruby provides built-in support for arbitrary precision integer arithmetic.

[I'm also assuming that it's really arbitrary precision and not just 'really big but limited' precision.]