News for hackers. Stuff that matters.
Re:What If OpenDocument Used SQLite? (2014) by dathinab on Thursday December 17, 2020 @09:57PM (#25463293)
While I generally agree with the points they make I'm not sure I would want a (stock) sqlite being used for "less" trusted files.

While sqlite has a very comprehensive test suite there had been more than one attack with "manipulated" database files and as far as I know (I might be mistaken) the test suite(s) are not focused on testing that case much.

Through I'm positive that it would be fine with a "hardened" sqlite where certain features are not compiled in at all and in turn the attack surface is reduced.

Re:What If OpenDocument Used SQLite? (2014) by infogulch on Thursday December 17, 2020 @10:13PM (#25463389)
The article describes the current format as a zip of files where the main content of presentation slides is stored in a `content.xml` file within. Yes, sqlite may have had file format vulnerabilities in the past, but I cannot imagine trusting it less than any xml library in existence.
Re:What If OpenDocument Used SQLite? (2014) by fpoling on Friday December 18, 2020 @03:45AM (#25465127)
At work we used a library that relied on Expat XML parser. After 20 years it still gets exploitable bugs. As the software was Windows-only we patched the library to use OS API for XML parsing to delegate to Microsoft the job of fixing security bugs.
Re:What If OpenDocument Used SQLite? (2014) by vlovich123 on Thursday December 17, 2020 @10:47PM (#25463591)
You’d think so from reading [1]. [2] though paints a very different picture.

> The attacker can submit a maliciously crafted database file to the application that the application will then open and query.

I think it’s not impossible to secure SQLite better but it takes some work (like how Chrome sandboxes it in a separate process).

[1] https://www.sqlite.org/security.html [2] https://www.sqlite.org/cves.html

Re:What If OpenDocument Used SQLite? (2014) by moonchild on Thursday December 17, 2020 @11:53PM (#25464001)
> [2] though paints a very different picture.

I don't agree. That document states that the CVEs have historically required one of two preconditions:

> The attacker can submit and run arbitrary SQL statements.

> The attacker can submit a maliciously crafted database file to the application that the application will then open and query.

If you look at the actual list of CVEs, all but one start with ‘Malicious SQL statement’. The single one that doesn't is suffixed with ‘The bug never appeared in any official SQLite release’.

In other words, there has never been an official release of SQLite which was vulnerable when presented with a crafted database file.

Re:What If OpenDocument Used SQLite? (2014) by oever on Friday December 18, 2020 @05:19AM (#25465695)
The proposal to use sqlite database files instead of zip files falls under the precondition that an attacker can submit a maliciously crafted database file. So the security concerns apply.

SQLitte is mostly used in situations where neither the database nor the SQL statements can be provided by the attacker. If SQLite is used to exchange data both of these attack vectors are available.

Re:What If OpenDocument Used SQLite? (2014) by ori_b on Friday December 18, 2020 @12:06AM (#25464082)
> If you look at the actual list of CVEs, all but one start with ‘Malicious SQL statement’. The single one that doesn't is suffixed with ‘The bug never appeared in any official SQLite release’.

The list on the page is partial, and the language on that page is disturbingly dismissive. Reading leaves me more concerned that they're too arrogant to take security researchers seriously.

Re:What If OpenDocument Used SQLite? (2014) by 0zymandiass on Friday December 18, 2020 @01:07AM (#25464353)
I'm sure they aren't perfect, but their test suite and extensive fuzz testing is far better than any XML or JSON parser I've ever seen

https://www.sqlite.org/testing.html

Re:What If OpenDocument Used SQLite? (2014) by fpoling on Friday December 18, 2020 @03:53AM (#25465164)
I do not agree about JSON. As a format it is so trivial one can cover it with few test cases. SQL dialect that SQLite uses is vastly more complex.
Re:What If OpenDocument Used SQLite? (2014) by thristian on Friday December 18, 2020 @08:55AM (#25467070)
You'd think so, but in practice no two JSON parsers have the same bugs, so it's more complex than you might think:

http://seriot.ch/json/parsing.html#all_results

Re:What If OpenDocument Used SQLite? (2014) by dathinab on Friday December 18, 2020 @10:10AM (#25467794)
Think about it if even JSON which is kinda simple has that many problems with parsers how likely is it that something multiple degrees of complexity higher which wasn't developed with untrusted database fiels in mind is secure?

Besides that the large majority of thinks tested in your link are irrelevant for the security in this use-case. They do matter if you need fully stable de- and re-serialization or e.g. have a fully trusted validator. E.g. for certain security token use-cases it matters. Many of the thinks which are marked as failure could even be considered as a "more robust" or "more secure" (through not fully standard conform) parser. E.g. parsing tailing commas (disallowed by JSON), rejecting certain unescaped control/special characters (which JSON allows), rejecting to long field values (which JSON allows) or similar. What matters is that a) it has a protection about to deep recursion and as such no stack overflow can happen, b) it makes sure the text it outputs is correctly encoded (independent of weather or not it was well-formed in the JSON blob) and c) it doesn't crash in a way which leads to security problems (which the test suite you linked doesn't differentiate from "acceptable" crashes).

Re:What If OpenDocument Used SQLite? (2014) by fpoling on Saturday December 19, 2020 @05:23AM (#25476504)
As long as one uses a single parser to read a particular input it matters little if it produces a result that is different from what some other parser generates as long as the parser has no security bugs. And JSON is simple enough to cover a particular interpretation of its spec with test suits to make a security vulnerability extremely unlikely.

Surely if one uses one parser to verify the payload and another to use it, a disaster comes as was with IPhone verification bug.

Re:What If OpenDocument Used SQLite? (2014) by touisteur on Friday December 18, 2020 @02:19AM (#25464645)
I seem to remember really great interaction with lcamtuf (of AFL fame): https://lcamtuf.blogspot.com/2015/04/finding-bugs-in-sqlite-...
Re:What If OpenDocument Used SQLite? (2014) by trevyn on Friday December 18, 2020 @12:18AM (#25464140)
I feel like there should be a way to make a maliciously crafted database file execute arbitrary SQL... SQLite doesn't do stored procedures, but there's gotta be something in that surface area.
Re:What If OpenDocument Used SQLite? (2014) by dathinab on Friday December 18, 2020 @09:59AM (#25467689)
> SQLite doesn't do stored procedures

Link: https://www.sqlite.org/gencol.html

Example:

CREATE TABLE t1(

   a INTEGER PRIMARY KEY,

   b INT,

   c TEXT,

   d INT GENERATED ALWAYS AS (a*abs(b)) VIRTUAL,

   e TEXT GENERATED ALWAYS AS (substr(c,b,b+1)) STORED
);

Note how you can call functions in the columns. Thats one feature which a hardened sqlite for less trusted documents would need to not have.

Re:What If OpenDocument Used SQLite? (2014) by moonchild on Friday December 18, 2020 @12:52AM (#25464292)
I expect the majority of the vulnerabilities were in the command parser. Which attack vector is gone from the database file.
Re:What If OpenDocument Used SQLite? (2014) by yc12340 on Friday December 18, 2020 @03:00AM (#25464882)
AFAIK, SQLite stores database schema as text, so certain subset of parser still has to be invoked on untrusted DDL.
Re:What If OpenDocument Used SQLite? (2014) by gfody on Friday December 18, 2020 @12:41AM (#25464238)
if select is all you need then a view would suffice
Re:What If OpenDocument Used SQLite? (2014) by infogulch on Friday December 18, 2020 @12:07AM (#25464089)
https://www.sqlite.org/cves.html

18 CVEs

http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=xml

> There are 1776 CVE Records that match your search.

Do I win?

Re:What If OpenDocument Used SQLite? (2014) by dathinab on Friday December 18, 2020 @09:55AM (#25467647)
No, the article just replaces the container with SQL but not the components (e.g. a slide). Which without extra changes are still XML, stored as text in the database.

The change from storing the inner document format in something which is not XML is completely independent of changing the outer container to sqlite.

SQL isn't really suited as a markup language so storing styles "flowing" text in it isn't that good of an idea. The article only recommends storing all separate blobs of styled text in tables (it also sidesteps the whole problem about flow text across multiple pages by simple using slides as an example ;=) ).

Lastly I'm against using a "stock"/unhardened sqlite version, not sqlite in general. Ironically on of the features I would disable for now is related to text search but I don't remember the name of it so I can't really use it as an argument :=/.

So if you ask me:

- Change containers to a different format then zipped sql files, consider sqlite but use a hardened feature limited sqlite version.

- Change the inner representation away from XML(1)

(1): Ok, there are ways to safely use XML but this isn't useful for this discussion and they require a bunch of thinks which all reduce usability (of the XML library) and performance and only work for certain use-cases where you e.g. do not need fully stable serialization and de-serialization and might be limited to a subset of XML and don't rely on the verifier for anything "safety" realated and preferably don't use C/C++ to write a parser... so it's easier to just not do it tbh.

Re:What If OpenDocument Used SQLite? (2014) by fastball on Friday December 18, 2020 @12:12AM (#25464114)
Probably better to compare Apples and Apples[1].

> There are 126 CVE Records that match your search.

[1] https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=sqlite

Re:What If OpenDocument Used SQLite? (2014) by infogulch on Friday December 18, 2020 @12:53AM (#25464296)
That's fair. Though the disparity is still an order of magnitude...

More seriously, my point is that SQLite is very unlikely to be the most vulnerable piece of software in your stack. It's probably the most stable and bug free piece of software that you use on a day to day basis that hasn't been formally proven correct. Even openoffice itself has more CVEs: https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=openoffice

And some of them are due to the very issue I brought up above...

> CVE-2012-0037 Redland Raptor (aka libraptor) before 2.0.7, as used by OpenOffice 3.3 and 3.4 Beta, LibreOffice before 3.4.6 and 3.5.x before 3.5.1, and other products, allows user-assisted remote attackers to read arbitrary files via a crafted XML external entity (XXE) declaration and reference in an RDF document.

Re:What If OpenDocument Used SQLite? (2014) by patrec on Friday December 18, 2020 @01:37AM (#25464471)
> My point is that SQLite is very unlikely to be the most vulnerable piece of software in your stack.

Sqlite is very high quality, but I am rather doubtful this is true if you actually care about having a secure system to start with. I can think of three main reasons an office suite would have serious security vulnerabilities:

1. It's written in an unsuitable, memory-unsafe, programming language (as most office suites sadly are).

2. It includes an unsuitable programming language for scripting purposes (Word Macros are of course infamous for this).

3. Vulnerabilities in one of its dependencies (like XML parsers)

If you care to, 1 and 2 are very easy to avoid and 3 will be pretty hard to avoid completely, but you can limit your exposure. You need a GUI toolkit at the minimum, but you can completely avoid parsing as an attack vector by either not using a braindamaged format like XML in the unlikely case you don't need compatibility, or if you must, use an xml (zip etc.) parser written in a memory safe language. If you followed all other steps, but chose sqlite instead, I'd say chances a very good it's now your main vulnerability.

Re:What If OpenDocument Used SQLite? (2014) by infogulch on Friday December 18, 2020 @01:59AM (#25464551)
You have crafted a nigh-impossible tower of requirements that you admit are way beyond what is commonly practiced in the software under question in order to disclaim the statement "SQLite is very unlikely to be the most vulnerable piece of software in your stack". Right now on the front page of HN is an article about how Microsoft was also a victim of what may come to be labeled the most impactful hack ever, a list which includes most (all?) US government agencies. The root vuln? A crap admin password used by an IT monitoring software company which has software deployed to an alarming percentage of the industry.

I dunno man, I feel like my original statement is pretty reasonable. Let's please not move the goalposts to the moon. :)

Re:What If OpenDocument Used SQLite? (2014) by patrec on Friday December 18, 2020 @09:29AM (#25467395)
I don't think your statement is unreasonable at all coming from the common perspective that it is part of the natural order of things that opening a text document would expose your computer to a non-negligible risk of hostile takeover.

Personally I find this a very unreasonable thing to accept. Secure (single user) text processing (etc.) is not a hard problem at all and there is zero reason it should create any security risks.

Crap passwords (and security practices) are not pertinent at all here, unless you also argue that Boeing shouldn't really bother to make airplanes with non-lethal autopilot interactions because being killed by a mis-designed airplane is the least of risks to the typical US citizen's life give how obese he or she is.

The pertinent question is: if you are to design a wordprocessor, should you avoid baking sqlite into the design because of security concerns? I think you should. Sqlite is written and tested very well, but it is written in not formally verified C, and using a memory unsafe language for reading potentially hostile databases poses a completely unnecessary and avoidable security risk.

Re:What If OpenDocument Used SQLite? (2014) by oever on Friday December 18, 2020 @05:24AM (#25465723)
Or run the IO in a sandbox where it is streamed out in a simpler/safer format. Or run the entire suite in a sandbox like we do with parts of web browsers.
Re:What If OpenDocument Used SQLite? (2014) by patrec on Friday December 18, 2020 @09:31AM (#25467413)
How is this better than just avoiding the problem in the first place?
Re:What If OpenDocument Used SQLite? (2014) by fastball on Friday December 18, 2020 @10:29AM (#25467974)
Because SQLite, as a general purpose database system, can be used for loads of things.

A two-piece component (DB + sandbox) that you use for 100 different things is going to end up much more secure (overall) and require less human effort than maintaining 100 different systems for each use case.

It's like "don't roll your own crypto". That is the advice not because it's impossible to roll your own secure crypto, but because it is just much more likely that you will repeat bugs that other people have already figured out, so it is better if everyone is using one library so that when an issue is found and resolved, it applies to everyone. It is still good advice even though that 3rd party crypto lib is probably going to have a lot of unneeded functionality that actually increases the overall attack surface area than if you made your own for whatever your specific purpose is. I think the same logic applies here.

Re:What If OpenDocument Used SQLite? (2014) by TylerE on Thursday December 17, 2020 @11:10PM (#25463759)
I think his point is that if an attacker wants to give you bad data, there is already zero barrier to editing an XML file inside a zip.
Re:What If OpenDocument Used SQLite? (2014) by thayne on Friday December 18, 2020 @12:59AM (#25464321)
I think the GP is more an statement of how bad security is with XML than how good it is with SQLite. See for example https://owasp.org/www-community/vulnerabilities/XML_External....
Re:What If OpenDocument Used SQLite? (2014) by teraflop on Thursday December 17, 2020 @10:09PM (#25463359)
You could always sandbox it. This has actually already been done at least once: years ago, there used to be a "pure Java" JDBC driver for SQLite that was generated by compiling the original C code to a MIPS binary, and then translating the result into Java bytecode.

http://web.archive.org/web/20080703234650/http://www.zentus....

Re:What If OpenDocument Used SQLite? (2014) by ReactiveJelly on Friday December 18, 2020 @12:31AM (#25464196)
https://hacks.mozilla.org/2020/02/securing-firefox-with-weba...

Firefox did the same recently to sandbox Graphite (some font dependency)

C/C++ --> WebAsm --> sandboxed AOT compiled native code

Re:What If OpenDocument Used SQLite? (2014) by remram on Thursday December 17, 2020 @10:13PM (#25463393)
Can you elaborate about this? What are the gotchas when opening an untrusted SQLite3 database?

(I am developing a web application that will allow users to upload application files, which are SQLite3 databases)

Re:What If OpenDocument Used SQLite? (2014) by kibibyte on Thursday December 17, 2020 @10:32PM (#25463501)
Section 1.2 here might be a good start. https://www.sqlite.org/security.html
Re:What If OpenDocument Used SQLite? (2014) by mcint on Thursday December 17, 2020 @10:53PM (#25463625)
The Full Text Search extension (FTS) (FTS4?) has been a repeated source of issues.

If you stick with the core, and disable some of the shiny features, the track record is better. Though still not spotless, iirc.

Re:What If OpenDocument Used SQLite? (2014) by ori_b on Thursday December 17, 2020 @10:40PM (#25463553)
It's a multi-hundred-thousand line project (current count, excluding tests, is about 210,000). Like any project of a similar size, it has a history of security vulnerabilities. OpenBSD removed it because it was too much code to audit. OpenBSD dodged several issues like this RCE (https://research.checkpoint.com/2019/select-code_execution-f...) as a result.

See https://www.cvedetails.com/vulnerability-list/vendor_id-9237...

Hundreds of thousands of lines of code are simply going to have vulnerabilities and bugs. It's unavoidable with that level of complexity. I'm skeptical anything that complicated is a wise choice for a general file format -- especially when smaller, simpler, and more easily debuggable and independently implementable formats exist.

Re:What If OpenDocument Used SQLite? (2014) by hansvm on Thursday December 17, 2020 @10:35PM (#25463520)
I haven't taken the time to verify any of it, but a quick search for "sqlite code execution" doesn't look promising.
Re:What If OpenDocument Used SQLite? (2014) by yencabulator on Friday December 18, 2020 @09:26PM (#25474422)
Further reading: https://research.checkpoint.com/2019/select-code_execution-f...

I've never seen mention of this being fixed in any way. SQLite is not an acceptable format to transfer between computers.

Re:What If OpenDocument Used SQLite? (2014) by SQLite on Saturday December 19, 2020 @09:56AM (#25477740)
That bug was fixed long ago, even before the referenced video was produced.

The described attack is clever. It exploits the fact that an attacker might alter the schema so that it invokes an SQL function with side-effects when an app simply tries to read from a table. And depending on those side-effects, an exploit might be possible. In the video, there was a bug in a built-in SQL function that caused exploitable side-effects. But that bug was fixed long before the video was even produced. The examples in the video were from an older version of SQLite. They did not work for the latest SQLite release on the day that lecture was given.

Since the checkpoint.com attack was described (in the video and elsewhere) new defense-in-depth features have been added to SQLite to make similar exploits increasingly unlikely.

(1) Built-in SQL functions that have side effects cannot be used in the schema. Side-effect functions can only be invoked directly by the application.

(2) When applications register their own custom SQL functions, they can now mark those functions as "direct-only", meaning that they are prohibited in the schema.

(3) Run-time and compile-time options are available to prohibit the use of SQL functions in the schema that are not explicitly declared to be safe for use in the schema - that is, functions without side effects. This is for use in legacy applications that might have been created before the per-function flag that prohibited use within the schema was available. It is also an extra layer of defense for complex applications that might add hundreds or thousands of side-effect SQL functions - to ensure that the "direct-only" flag is not accidentally omitted from one of them.

(4) Run-time options are available to disable triggers and views in applications that do not need them. This is not necessary to avoid an exploit, but it does provide an additional layer of defense.

See https://sqlite.org/security.html and especially paragraph 1.2 item 8 for additional information.

Re:What If OpenDocument Used SQLite? (2014) by Aeolun on Thursday December 17, 2020 @10:32PM (#25463497)
I think the same is true for less trusted MS office files though? E.g. VBA can do some crazy stuff.