>>I'm thinking now either an external hard drive, or on a VPS I can just tunnel into. Consider setting up an ElasticSearch cluster somewhere, like with AWS, which takes plugins for ElasticSearch. Once you've indexed your data with ES, then queries are pretty easy (JSON-based). This would also solve your other problem with data visualization. ElasticSearch has an analytics tool called Kibana. Pretty useful and doesn't require too much effort to set up or use. I'm using this setup for a sentiment analysis project myself.

You didn't mention the libraries in your NLP pipeline (guessing NLTK bc of the Python?), but if you're doing LSM compatibility, I'm guessing you might be interested in clustering or topic-modelling algorithms and such...Mahout integrates easily with ElasticSearch.