News for hackers. Stuff that matters.
Re:Records: Python library for making raw SQL queries to Pos... by Grue3 on Sunday February 07, 2016 @01:42PM (#11054081)
>Rows are represented as standard Python dictionaries

That seems inefficient. You're always going to have the same columns in each row, so storing column names for each row is a huge waste of memory. Django's "cursor.fetchall" returns a list of tuples, which seems to be a saner way to represent db rows in Python.

Re:Records: Python library for making raw SQL queries to Pos... by glifchits on Sunday February 07, 2016 @02:33PM (#11054370)
Reitz has a public Facebook post on this. Here are his comments:

Harry Percival Looks nice! Y u no namedtuple?

Kenneth Reitz Dictionaries are a much more predictable interface ;)

Kenneth Reitz Just because you can do something, doesn't mean you should!

Re:Records: Python library for making raw SQL queries to Pos... by kenneth_reitz on Sunday February 07, 2016 @02:44PM (#11054438)
Well, that's creepy.
Re:Records: Python library for making raw SQL queries to Pos... by glifchits on Sunday February 07, 2016 @02:57PM (#11054510)
Hahaha... I can only imagine. Sorry to creep you out. Your Facebook profile is just a click away from your personal website, and your discussion about Records is up front and center on your feed for me :)
Re:Records: Python library for making raw SQL queries to Pos... by ldehaan on Sunday February 07, 2016 @03:40PM (#11054697)
%s/'s/ is not/i Fixed it, you're welcome.
Re:Records: Python library for making raw SQL queries to Pos... by mattdeboard on Sunday February 07, 2016 @07:22PM (#11055738)
I learned from a coworker recently that namedtuple is a much lower memory usage than a dictionary, so if you're dealing with lots and lots of data regularly, and lots of processes on a single machine, (i.e. memory is a concern) namedtuple might be a better choice.
Re:Records: Python library for making raw SQL queries to Pos... by nvader on Sunday February 07, 2016 @02:19PM (#11054299)
One argument I've seen for using dictionaries is that namedtuples restrict you to column names that are valid Python identifiers, which is a smaller set than postgres allows for column names (postgres allows the '$' and non-Latin characters).
Re:Records: Python library for making raw SQL queries to Pos... by hobarrera on Sunday February 07, 2016 @02:36PM (#11054390)
So does python:

>>> d = {'ñanáöß$': 1} >>> d {'ñanáöß$': 1} >>> d.keys() dict_keys(['ñanáöß$']) >>>

Re:Records: Python library for making raw SQL queries to Pos... by kilink on Sunday February 07, 2016 @02:44PM (#11054435)
The parent was talking about namedtuple.

    >>> collections.namedtuple("test", "$")
    Traceback (most recent call last):
    ...
    ValueError: Type names and field names can only contain alphanumeric characters and underscores: '$'
Re:Records: Python library for making raw SQL queries to Pos... by hobarrera on Sunday February 07, 2016 @08:02PM (#11055858)
That's because attribute names can't be called "$", so `sometuple.$` would be a syntax violation.
Re:Records: Python library for making raw SQL queries to Pos... by EmanueleAina on Tuesday February 09, 2016 @06:53AM (#11064678)
I guess everyone is aware of that, thanks,

The parent comments were discussing why choosing dicts over namedtuples for db-originated columns, and the fact that field names are restricted on namedtuples but not on dicts is a very compelling point.

Making examples usind dicts and explaining why namedtuples have restrictions is completely missing the point.

Re:Records: Python library for making raw SQL queries to Pos... by pyre on Sunday February 07, 2016 @03:11PM (#11054569)
HN supports "<pre><code>" blocks in a similar manor to Markdown, but using a two-space prefix (rather than Markdown's 4-space prefix):

  >>> d = {'ñanáöß$': 1}
  d {'ñanáöß$': 1}
  >>> d.keys()
  dict_keys(['ñanáöß$'])
Re:Records: Python library for making raw SQL queries to Pos... by r0muald on Sunday February 07, 2016 @01:52PM (#11054152)
Named tuples seem to offer the best of both worlds, as they're immutable.
Re:Records: Python library for making raw SQL queries to Pos... by glifchits on Sunday February 07, 2016 @01:57PM (#11054187)
This memory waste argument is only true if you want to fetch all rows, which you could possibly avoid with different techniques. Otherwise, a dictionary seems like a very simple, convenient, Pythonic, and intuitive representation of a row.

This lib also looks brand new, so there is plenty of time for the community to optimize the hell out of it.

Re:Records: Python library for making raw SQL queries to Pos... by kilink on Sunday February 07, 2016 @02:41PM (#11054424)
If this were a big deal, it could be optimized using something like a key-sharing dict implementation [1]. You can keep the dict interface to accommodate column names that aren't valid identifiers, and avoid most of the memory overhead.

[1] https://www.python.org/dev/peps/pep-0412/

Re:Records: Python library for making raw SQL queries to Pos... by jstimpfle on Sunday February 07, 2016 @02:47PM (#11054456)
python-sqlite3's RowFactory does this as well.
Re:Records: Python library for making raw SQL queries to Pos... by bjt on Sunday February 07, 2016 @02:47PM (#11054457)
I've made a pull request implementing namedtuple support at https://github.com/kennethreitz/records/pull/6.
Re:Records: Python library for making raw SQL queries to Pos... by siddhant on Sunday February 07, 2016 @01:54PM (#11054169)
I'm also a fan of NamedTupleCursor. Not sure how NamedTuples compare to raw tuples in terms of memory consumption, but they make code look a lot nicer.
Re:Records: Python library for making raw SQL queries to Pos... by stuartaxelowen on Sunday February 07, 2016 @02:09PM (#11054245)
They're essentially the same footprint IIRC.
Re:Records: Python library for making raw SQL queries to Pos... by nathancahill on Sunday February 07, 2016 @02:12PM (#11054259)
Re:Records: Python library for making raw SQL queries to Pos... by tcoder on Sunday February 07, 2016 @03:15PM (#11054590)
Namedtuple CPU-speed performance for some common operations is TERRIBLE compared to dictionaries, at least on 2.7. Orders of magnitude difference. So bad that it really matters. It's shocking how bad the implementors got this.
Re:Records: Python library for making raw SQL queries to Pos... by pyre on Sunday February 07, 2016 @03:27PM (#11054645)
namedtuple is just a thin wrapper around tuple. It literally constructs a class definition that sub-classes tuple as a string and then executes it. You can see the string template that it uses here[1]. If you're interested in something like namedtuple there are other[2] things[3] you can use depending on your use-case.

[1] https://github.com/python/cpython/blob/master/Lib/collection...

[2] http://stackoverflow.com/a/2648186

[3] https://pypi.python.org/pypi/frozendict/

Re:Records: Python library for making raw SQL queries to Pos... by Redoubts on Sunday February 07, 2016 @09:21PM (#11056066)
It's really weird that immutable tuples take a performance hit, compared to mutable lists (almost 50% on access!?!).
Re:Records: Python library for making raw SQL queries to Pos... by pyre on Monday February 08, 2016 @01:02AM (#11056570)
That's because Python 2 has a special case for lists[1]. This does not exist in Python 3[2] (and therefore neither does the performance difference).

[1] https://github.com/python/cpython/blob/2.7/Python/ceval.c#L1...

[2] https://github.com/python/cpython/blob/master/Python/ceval.c...

Re:Records: Python library for making raw SQL queries to Pos... by Redoubts on Monday February 08, 2016 @12:57PM (#11059674)
It looks like list access got slower as a result though.
Re:Records: Python library for making raw SQL queries to Pos... by dangoldin on Sunday February 07, 2016 @03:26PM (#11054638)
Curious what the operations you're talking about are. Get?
Re:Records: Python library for making raw SQL queries to Pos... by tveita on Monday February 08, 2016 @08:23AM (#11057897)
Pickling, for one.

  >>> nt = namedtuple('nt', 'a b')
  >>> nt10k = [nt(1, 2) for i in range(10000)]
  >>> dict10k = [{'a':1, 'b':2} for i in range(10000)]
  >>> timeit pickle.dumps(nt10k)
  10 loops, best of 3: 26.3 ms per loop
  >>> timeit pickle.dumps(dict10k)
  100 loops, best of 3: 2.49 ms per loop
Re:Records: Python library for making raw SQL queries to Pos... by pyre on Sunday February 07, 2016 @06:45PM (#11055585)
If you look at the the StackOverflow in my sibling comment to yours, running 'obj.attrname' on a namedtuple takes longer because it needs to translate 'attrname' to an integer index and then run (e.g.) 'obj[0]'. Outside of that, I'm not sure.
Re:Records: Python library for making raw SQL queries to Pos... by cwmma on Sunday February 07, 2016 @01:54PM (#11054164)
probably so you don't have to remember what order they were returned in