News for hackers. Stuff that matters.
Re:How Python became the language of choice for data science by ma2rten on Friday November 22, 2013 @06:37AM (#6780271)
This is not a major problem for data science. We usually use numpy arrays (or scipy sparse arrays), which are memory efficient. There are also some other efficient data structures for python:

http://kmike.ru/python-data-structures/

For preprocessing you can usually work on a line-by-line basis (e.g. don't keep the entire dataset in memory).

This is a bigger problem for web apps, imo. But there you usually just throw money (e.g. more servers) at the problem.

There are no comments on this comment yet.