Re:Elixir for Humans Who Know Python
by traverseda on Saturday February 25, 2023 @12:23PM
(#34938391)
>I am not familiar with Python Object Model enough but, if your Python example still allocates an object with slots for all 47 other fields besides the three selected, then you may still find yourself under circumstances you would rather have a "slim" version of the model and reduce memory allocation.
Honestly at that point probably don't be using python. ORM objects don't use slots, they use a hashmap to look up everything every time, and probably that hashmap points to a method that points to another hashmap that points to some kind of data structure.
If you're worried about that than django probably is not for you. That being said it scales out very nicely, you probably are not loading dozens of thousands of entries in to memory at once but are instead using a celery distributed task queue. If you did need to load that much in to memory for some reason (presumably some kind of data science?) You can use the `in_bulk("field1","field2","etc")` query option to return raw data as dictionaries/lists instead of ORM objects. You don't get the benefit of ORM methods but you're not instantiating an ORM object for every row and probably for that kind of thing you're better off taking a more functional approach any way.
Honestly at that point probably don't be using python. ORM objects don't use slots, they use a hashmap to look up everything every time, and probably that hashmap points to a method that points to another hashmap that points to some kind of data structure.
If you're worried about that than django probably is not for you. That being said it scales out very nicely, you probably are not loading dozens of thousands of entries in to memory at once but are instead using a celery distributed task queue. If you did need to load that much in to memory for some reason (presumably some kind of data science?) You can use the `in_bulk("field1","field2","etc")` query option to return raw data as dictionaries/lists instead of ORM objects. You don't get the benefit of ORM methods but you're not instantiating an ORM object for every row and probably for that kind of thing you're better off taking a more functional approach any way.