For those looking at more extensible options I highly recommend SQLPad: https://github.com/rickbergfalk/sqlpad

A web app for writing and running SQL queries and visualizing the results. Supports Postgres, MySQL, SQL Server, Crate, Vertica, and Presto. Written in Node.js.

Sounds like a summary of this comment should be added to the README / web page as a disclaimer!

SQLite definitely presents some unique challenges, despite it generally being the "simplest" relational database. I've written a web-based SQLite database browser, which you can find here: https://github.com/coleifer/sqlite-web -- importantly, it supports all the `ALTER TABLE` statements you'd expect (add column, drop column, rename column, etc). This is done by renaming the table temporarily, creating a new table with the desired changes, then copying the data into the new table. Kinda clunky, but it works :)

Which is basically how MySQL ALTER TABLE works internally anyway...

doesnt support expressions on DEFAULT keyword of create table syntax. :(

You may be interested in Bedrock, which makes SQLite more performant and clusterable http://bedrockdb.com/

What an interesting project. Until now SQLite at scale has sounded like an oxymoron to me.

I feel like rewriting this on top of SQL Server Express would still be valuable. Setting up and managing linked servers manually, especially with non-SQL Server databases can be a pita especially considering managing their drivers etc.

Also, SQL Server is awful at importing/exporting data directly. Almost all forms are broken in various ways and are a source of endless grief. The only reliable built-in way is to use SSIS which is like using a 200-ton tank for a paperweight.

> I feel like rewriting this on top of SQL Server Express

Why choose a proprietary engine? Why not use PostgreSQL or even MySQL?

I can recommend HeidiSQL, a useful GUI for MySQL, MariaDB, MSSQL and Postgres. https://en.wikipedia.org/wiki/HeidiSQL

They probably meant SQL Server Compact, which is a smaller package and comparable to SQLite. It was meant to power WinFS back in 2006. But it's closed source and now legacy.

https://en.wikipedia.org/wiki/SQL_Server_Compact

There are many business applications that target SQL Server exclusively.

On the one hand, I find that highly annoying. On the other hand, even if you don't like Microsoft, SQL Server is fairly awesome, performance-wise and stability-wise.

But still, there are many applications where you cannot really choose what database engine you want to use.

I suppose PostgreSQL+FDW would be just as good as SQL Server Express+Linked Servers technically. You end up trading users' MSSQL experience for pgsql.

I wonder if it would be easy to support both backends and be able to switch between them depending on the query and which features and integrations you want.

> Also, SQL Server is awful at importing/exporting data directly. Almost all forms are broken in various ways and are a source of endless grief. The only reliable built-in way is to use SSIS which is like using a 200-ton tank for a paperweight.

So much this. BCP is inferior to 5 lines of C# code, the tools for importing and exporting data in SSMS are buggy and cumbersome, SSIS is a gigantic monster. Simple "Export the results of my query into an INSERT statement" is a moronically obvious workflow and is non-existent.

I've read that the tools division of MSSQL was running on a skeleton crew for a long time, and it shows.

Despite everyone saying SSMS is hands down the best database IDE, while this may be true it is also true that MSSQL tooling is a train wreck compared to what it could be. It almost seems Microsoft intends to keep it this way, perhaps because they'd have to finally admit to the decade of neglect? Who knows the exact reason, but it's a sad situation.

What kind of tooling are you speaking of? Honest question.

Sql Server Reporting Services is great but the server website was obviously mid-2000s until very recently, even containing markup that wouldnt render in Chrome.

SSIS is a hellscape of unnecessary designers for an ETL framework, and generates ultra-brittle packages that crash if the slightest schema tweaks happen.

SSDT stores your schema and publishes diffs for easy deployment, but will gobble up all your RAM for half an hour to compile... if it doesn't crash.

And SSMS has the worst tabbed interface ever for managing queries.

I agree with you on all points. Products on the periphery of SQL Server have been severely neglected for years. There is a replacement for some of SSRS in PowerBI and SSAS, which is actually pretty nice. If I'm not mistaken, people serious about version control and deployment in SQL Server use redgate software.

My question is how other databases fare in these areas. I'm not familiar with any solutions for Reporting UI, ETL, and Schema Version Control in MySQL or PostgreSQL. What built-in tools do other databases have that outperform SQL Server's (obviously crumbling) functionality?

The general consensus is usually that this MS tooling, yes the one just described, is very good! Certainly far better than the competition. It boggles the mind.

>Simple "Export the results of my query into an INSERT statement" is a moronically obvious workflow and is non-existent.

I've only used SSIS for a few months now, but wouldn't this just be a simple Data Flow Task for that. Now, I'm not saying I would use SSIS for that (because I frankly hate SSIS) but it is a trivial thing to do.

I've never been able to match bcp's speed when importing large amounts of data, though.

I've never had an import job where run time was a key concern... but the old ADO.Net Bulk Insert thingy has always been good enough when Entity Framework was too slow (which, for inserts, it generally is).

How large a job are you talking? Did you use a transaction? A transaction + being in a nearby location has been good enough for every import I've done (millions of rows).

> T-SQL, the procedural language I chose to implement atop SQLite, is not a great general-purpose programming language

The day every .NET shop on Earth admits this is the day I'll be a happy man. A SQL server is not for writing business code.

You mean MSSQL? Because Postgres has an excellent procedural language built-in, as well as support for lots of others....

Perhaps it does. I've dealt with horrible procedural code in t-sql and pl/SQL and maybe Postgres is better but I'm in no hurry to find out.

I'm a Scala developer by profession and have dealt with horrible Scala code. People can write horribly in any language.

I agree with this but also with the other child comment; it is really really easy to write unreadable plpgsql

Can your software use ODBC connection (specifically System DSN sources)? I'm waiting for my IT dept to install .NET 4.6 so I can't try your software right now.

Do you recommend it for a noob on data analysis? I'd basically set up a MySQL or PostgreSQL somewhere else (like a vps) and load and download data from there.

Is it not possible for you to adapt the code to use a different SQL engine as the backend? Or is it tightly integrated?

> T-SQL, the procedural language I chose to implement atop SQLite, is not a great general-purpose programming language.

T-SQL ( I'm assuming the microsoft variant ) is not a procedural language, it's a declarative language.

> Using C# in LINQpad is a more pleasant experience for experimentally messing around with data.

This is not true. T-SQL is as close to the data as you can get. C#/LINQ/EF/etc are great for business layer/presentation layer. But for messing around with data, you can't really do any better than T-SQL. Trust me on this.

If you are a front-end developer, then LINQpad is great in building .NET code/CRUD libraries/etc for dealing with data, but if you want to work with data, T-SQL is where it is at.

Developing expertise in T-SQL ( it's not as easy as people think ) will open up a new way of understand data/programming/thinking. SQL Server allows importing of .net code/functions which you can run on the sql server via T-SQL so there isn't really much you can't do with T-SQL.

> A local install of SQL Server Express can be linked to remote servers, allowing you to join local tables to remote ones.

SQL Server Developer Edition is free now. That is a great starting RDBMs to learn databases on windows environment. It has almost everything other than enterprise level features ( clustering/replication/etc ) that most people don't need.

> T-SQL ( I'm assuming the microsoft variant ) is not a procedural language, it's a declarative language.

I think he's referring to the terrible terrible terrible procedural components of T-SQL - cursors, local variables, IF/THEN control blocks, etc.

Right Tool For The Job. Sql is good for querying/storing data, so use it for that. Then let the C# handle manipulating the data and applying rules and whatnot.

> I think he's referring to the terrible terrible terrible procedural components of T-SQL - cursors, local variables, IF/THEN control blocks, etc.

Sure. It also has stored procedures/functions/etc. But just because it has elements of imperative->procedural language doesn't mean it is a procedural language. No more than C#'s LINQ implementations mean C# is a declarative language or lambda expressions means it is a functional language.

And though I'm not of a fan of cursors, it is a legacy of a time when it was somewhat needed. And nothing about local variables/controls/etc makes T-SQL "terrible".

All of it is necessary for database maintenance, t-sql programming, etc.

> And nothing about local variables/controls/etc makes T-SQL "terrible".

Compared to a full-featured programming language? It's good for its intent, which is to provide a minimal bit of procedural scaffolding for scripts that alter the database. It is terrible as a general-purpose procedural programming language, and I've worked with too much horrifying T-SQL and PL/SQL that pushes these tools way too far out of their intended workflow.

Yes, T-SQL is predominantely a declarative language. But it includes procedural components for writing procedural scripts. And unless we're talking about maintenance scripts or deployment scripts, don't use them.

You just need to calm down. Everyone here has spent a certain amount of time dealing with shitty code.

That's all you're really claiming here. That some people write shitty code. And I agree. I've spent plenty of time with almost every flavor of SQL code that was incredibly shitty.

But I don't really blame the languages that much. I blame the people who didn't understand when and where to use one language or another.

And frankly, if you're seriously going to complain about T-SQL vs other "more full featured programming languages" and then make a blanket statement about not using any procedural components on SQL flavors . . . ummm, you might be your own worst enemy here.

> You just need to calm down.

You just said "I would shit a brick and beat a developer" upthread.

Since SQL Server 2005, modern alternatives to the procedural components have been added to the product. Even with these features existing it does not make any sense to call SQL a procedural language. Choosing the right tool for the job is certainly important. I notice many people write off SQL almost immediately when it is mentioned here. Often they come from a different language and do not see the value in the relational model. It's unfortunate too because there is real value in being able to go between a relational mindset seamlessly to an object oriented or functional one.

I use SQL Server daily and I love the relational model. The core SQL language I dislike because it's verbose, backwards, and restricts sane forms of reuse. The procedural components of MS SQL are just plain miserable.

I'm constantly frustrated that the two options in the database world appear to be "use SQL and all its anachronistic warts" or "give up on the relational model". NoSQL throws the baby out with the bath-water.

But this is a tangent. The original post came about calling T-SQL a "procedural language". Which is inaccurate... but the author was talking about adding extensions to Sqlite - SQlite is already a SQL dialect. So if you're bolting on T-SQL to Sqlite, what's the main thing it will bring to the table?

It's procedural components. So, in the context of comparing Sqlite to T-Sql, using the term "procedural language" might be a misnomer, but it makes sense here.

Yeah, this is spot on. SQLite provides DDL and DML commands, but no support for composing programs from these commands. I leaned on T-SQL for the syntax and behavior of variables, basic control structures, procedures, error handling, and some common library functions, but that's all that it takes from T-SQL. I didn't intend to make a judgment about T-SQL, but rather to indicate which parts were implemented in SQL Notebook.

>The core SQL language I dislike because it's >verbose, backwards, and restricts sane forms of reuse.

I can appreciate the verbosity and "backwardness" argument you make, but can you cite some examples of its restrictions. (Serious question)

No algebraic types. No good way to query a graph of objects down, only a flat resultset. Infuriating performance problems if you layer views too deep or use scalar functions, which are sane encapsulation. No higher-level concepts of code reuse like templates. If I have a common pattern like "subclass table A by creating table B with a foreign primary key to A and create view vwB that has the combined columns of A and B" there is no SQL construct to express this common pattern.

You just said to use the right tool for the right job, a few comments above.

Everything you are saying here sounds like a job for something other than a relational database.

I would shit a brick and beat any database developer with it that introduced the idea of a class as a primitive. Fuck that happy horse shit.

Also, graphs? Really?

You're looking for an ORM and blaming SQL for not being that.

There is no common database pattern that creates one table as a subset of another table. RDBMS is, by nature and design, not something you modify in that way on the fly. If you want that kind of object oriented functionality, push your data into a cache layer or column store or KV.

You said it yourself. Use the right tool for the right job. Nothing you've said makes even remotely any sense as a criticism of T-SQL or SQL in general. That's not the right tool for what you're talking about.

Not that a class needs to be a primitive, but that patterns like inheritance should be reusable in some way through templating or the like.

And the inability to pull down a graph as a single operation is the perfect example of sql's limitations.

Relational model is great right? Right. So why throw it out the window and give me a glorified excel spreadsheet as a result set? If I want to pull down a thing and it's related subthings in a single operation, I have to join which is wasteful in that case. This actualy hamstrings ORMs.

I don't think you understand relational databases.

I wouldn't hit you with an actual brick, but if I worked with you, I'd certainly nerf dart you on a regular basis.

I understand relational databases just fine. I still have no interest in the Stockholm syndrome of defending a platform where a = a can evaluate to something equivalent to False, regardless of the theoretical underpinnings of three-valued logic.

There are so many pain points in SQL that are defended because theory.

Have you used T-SQL's "CROSS APPLY" or "OUTER APPLY" in conjunction with encapsulated views or table-valued functions?

You might want to have a look at that as a significant enabler of code reuse on SQL Server.

(On PgSQL is LATERAL JOIN)

(Ps: you should try to replace SCALAR functions with SET-BASED logic. These can easily be ringfenced into TVF)

TVFs are intensely verbose to use compared to the corresponding SVF for a single column. Cross Apply will destroy your performance. I'm venting because I've watched clean, legible code get destroyed into soup of a zillion tiny joins and endless repeated boilerplate for performance improvement by this misfeature.

Or use the right tool for the job and use your database for manipulating your data, applying rules, and whatnot.

Use your presentation layer for displaying your data once it's actually properly processed.

Manipulating data and applying rules are also something relational databases are excellent at.

> but if you want to work with data, T-SQL is where it is at.

Depends on your definition of "working with data" I guess.....TSQL is fast, but beyond that almost anything beats it for functionality when messing around.