News for hackers. Stuff that matters.
Re:Ask HN: What kind of programming do you do and what tools... by siddboots on Saturday August 03, 2013 @08:56PM (#6154084)
I'm a data analyst / data application developer for a large humanitarian NGO.

What: Lots of small data analysis projects, some large web projects. All projects are in git repos or in our division's SVN. My work tends to involve ETL data flows, statistical analysis, web development and data visualisation. I'm not "full stack" as I never really play with hardware or do much systems administration. I do get to control a nice big, enterprisey Oracle data warehouse, with a few hundred users, but I've hardly played with nosql, redis, mapreduce, etc.

Most of my problems can be solved in-memory, and most of my data fits on a single disk.

How: SQL. SQL. More SQL. PL/SQL. Python (Pandas, scipy, numpy). R (these days only when I can't find a python lib). Javascript (just jquery, underscore and D3). Windows 7 while at work, Ubuntu + i3 wm at home. Edit in vim with very few plugins, everything else is done in zsh/powershell. IIS for production, Pyramid/Pylons Project for quick prototyping. A wide variety of RDBMS, but chiefly Oracle. I do lots of my templating/css in-browser using developer tools (FF, Chrome, IE9).

My typical web stack is static html and javascript, with json served up out of purpose-specific database views. No need for an MV* framework, since I'm almost never dealing with transactional data.

I also use Excel for a) doing quick data transforms b) building interactive, distributable data analysis apps in less than an hour. I'm trying to build an excel replacement with vim bindings and LISPy worksheet formulas.

Re:Ask HN: What kind of programming do you do and what tools... by gjreda on Saturday August 03, 2013 @09:07PM (#6154122)
My workflow is very similar. All data analysis, all the time.

Tools: Lots of SQL (MySQL). AWS Elastic MapReduce + Hive for log analysis. Python (pandas). R (ggplot2 + stat modeling - haven't fully grok'd Python's statsmodels package yet). A little bit of D3, and a lot of command line. Maybe some Flask every once in a while to prototype out a reporting web app.

I try to avoid Excel entirely because it always crashes on my MBP, but still find myself in it every so often to simply look at my dataset.

Re:Ask HN: What kind of programming do you do and what tools... by siddboots on Saturday August 03, 2013 @09:10PM (#6154132)
> ... but still find myself in it every so often to simply look at my dataset.

Exactly this. I could never understand why there isn't some awesome, lightweight, terminal-based tool for looking at dataframes, and doing basic manipulation.

Re:Ask HN: What kind of programming do you do and what tools... by late2part on Sunday August 04, 2013 @03:47AM (#6154999)
DevOps for a public company with SaaS cloud offerings.

iTerm

ssh to machines

chef/knife - look at config/status/recipes of servers

vi - edit stuff, esp. recipes or scripts (wish I learned emacs, never have)

scripts in bash/perl/python/ruby

graphite - store data

logstash - store logs, sort logs, quantify logs, send log stats and metrics to graphite

jenkins - continuos integration, take dev code, run through tests, test deployment

postgres - tuning, fighting, debugging, profiling

redis - checking

sensu/nagios - monitoring

lots of good fun!