News for hackers. Stuff that matters.
Re:How I Reduced My DB Server Load by 80% by mamcx on Monday October 09, 2017 @02:20PM (#15436176)
I adopt a idea (not remember the original library where I say it) where ALL the sql strings are in a single .sql file.

Exactly as with sql upgrade scripts.

It look like this:

    --name: post-event
    SELECT post_log(@theId, @ENTITY, @ACTION, @CHANGEBY, @DATA, @VERSION)

    GO

    --name: get-location
    SELECT * FROM "Location"
    WHERE
        id = @id

    GO

    --name: list-location
    SELECT * FROM "Location"
    ORDER BY country, state, city;

    GO
Then I just parse this file (note the names with --) once and use this alike (in F#):

    module Location =
    let ENTITY = "Location"
    let SQL_LIST = SQL_CMDS.["list-location"]

    type LocationRecord = {id:int64 option; address:string; country:string; state:string; city:string; version:int64}

    type LocationQuery =
        | All
        | ById of int64

    let query q =
        use con = openConn()

        let toRec = Db.toRecord<LocationRecord>

        match q with
        | All ->
            Db.select con SQL_LIST []
            |> Seq.map toRec

        | ById(theId) ->
            let id = [P("@id", theId)]
            Db.select con SQL_BYID id
            |> Seq.map toRec
And plug a micro-orm (mainly just a very thin layer over ADO.NET in .NET. I do similar over swift and python).

This take me like a few hours. Let me test easily the sql. I can build the sql exactly as I want.

The only cons is the repetition on the scripts - because SQL is a terrible language that not allow composability, like similar to CSS - . Probably I will later use a template parser (like mustache or similar) but I think this is the closest to the holy grail ;)

Re:How I Reduced My DB Server Load by 80% by ebiester on Monday October 09, 2017 @02:33PM (#15436293)
So, you update a 5,000+ line file every time you need to change a query?

What do you do when you have five similar queries, and each needs to be changed if one changes?

This seems like smaller projects might work, but I'd be concerned about using a strategy on a larger project.

Re:How I Reduced My DB Server Load by 80% by bpicolo on Monday October 09, 2017 @02:54PM (#15436449)
Here's a clojure library designed around it. (https://github.com/layerware/hugsql)

Not a fan of this strategy in particular, but I can see arguments for it.

Re:How I Reduced My DB Server Load by 80% by mamcx on Monday October 09, 2017 @03:50PM (#15436879)
>What do you do when you have five similar queries, and each needs to be changed if one changes?

In short, I truly utilize a DB with full effects.

I don't follow the mantra of using a DB "only" as a dumb datastore. Something that I get validated as when people use NoSQL and have not option that use all of it.

This mean, that I use views and stored procedures/functions.

This cut a lot. Also, proper modeling of tables also help (a lot). I refactor DBs.

I mean, I truly treat a DB as "code".

Obviously, this strategy is not flawless, and it will benefit from some kind of ORM-ish layer to auto-generate the mechanical parts of the querys. I'm building that part myself.

But so far, is have show to be good.