I just thought of this, and I don't know why it didn't come to me earlier.

Not that I had the foresight to pick Clojure because of this, but after I started working with Clojure I noticed that change became much easier than with the languages I was using before (one of which was C++).

When my thesis advisor would tell me, "you should go do X" (fill in the blank), I'd do what he said, and then a week later he'd have changed his mind and he would say "now go do Y", instead of X. Those changes would kill my enthusiasm and productivity when I was using C++, but once I began using Clojure those changes no longer felt like a bombshell. Change became relatively easy.

Now, I'm not saying that change is trivial, but compared to C++ (in my experience), changes are much easier to deal with using Clojure.

Very interesting. Did you consider using R? Why did you choose Clojure for this? I've been thinking of doing something similar myself and I would love to hear your thoughts considering you've already been-there-done-that.

I'd love to read your thesis and code, if they're available.

I had heard of R, but I didn't really know anything about it. I knew it was a statistical package, and I knew it had its own language, but I didn't really care about learning how to use it; I wanted to learn to use Clojure, Scala, Ocaml, or Haskell.

One of the reasons I picked Clojure over the other languages is that I knew that I could use code from the Incanter project if I needed some statistical functions.

A link to my content-complete thesis is: https://docs.google.com/document/edit?id=1kOKjY265a3F5SbN25d...

The thesis is long and boring to read. Let me save you some time. Here it is in a nutshell: Genetic algorithms (GAs) can be used, somewhat successfully, to identify good parameter-sets for technical trading strategies; however, even the "best" GA-identified parameter-sets in my research failed to outperform the buy and hold strategy when trading, with EOD prices, over the course of a randomly picked 1-year period taken from the interval starting January 1, 1985 and ending May 1, 2010.

My code is not available yet because according to the university it belongs to them (ridiculous, I know). I'll have to ask my advisor whether or not I can make it publicly available.

I understand your university's position ... and yours also. If they do decide to make the code publicly available, I'd appreciate a reply to this comment. I'll check back occasionally to look for updates.

Enjoying you're thesis, btw, especially the section on Bollinger bands. The Turtle Traders (http://bit.ly/by1j2M ) seemed to have used this successfully with commodities. Did you reference any of the Turtle traders stuff when you were designing your system?

I just put my thesis project online at github. The URL is http://github.com/davidkellis/stocktrader_clojure.

Also, I bought and read Way of the Turtle. I really enjoyed it. Thanks for the suggestion.

If my advisor tells me that I can make the code available to the public, I'll put it on github, and post a comment here.

Thanks for the comment about my thesis. I hadn't ever heard of The Turtle Traders, but I read a few pages of the preview that you linked to, and it looked interesting enough to buy a copy. It will be an interesting read. Thanks for the tip.

R does have quite a bit of financial libraries, yet I don't remember R has any good support on parallel computing, hmm?

True, but Matlab has good support for distributed computing. It's even possible to do this on an EC2 cluster. Pretty impressive really: http://www.mathworks.com/programs/techkits/ec2_paper.html

edit: Cleaned up my bad grammar.

There is the multicore package:

http://cran.r-project.org/web/packages/multicore/index.html

My worry with parallel loops in R would be inadvertent race conditions. For example, is the random number generator in R thread safe? I doubt it.

Can you tell us more about your data set?

Yes, in the backtesting trials I ran, I tested three technical trading strategies using EOD adjusted closing prices between January 1, 1985 and May 1, 2010. I only conducted trials against current S&P 500 member companies. I licensed the data from Norgate Investor Services. I am not affiliated with that company. They just had a price that I, as a poor graduate student, could afford.

Gotcha.

How do you think it would hold up to larger, say intraday data?

As far a runtime performance goes, my backtesting framework would run slightly slower using intraday price history than it does with EOD data.

There are two reasons why:

1. There would be more data to load into memory, and that process of loading the price history from a CSV file into memory would take a little longer simply because there is more data to deal with.

2. The data structure I'm using to index the historic price information by timestamp is the Java 6 implementation of TreeMap. The TreeMap class is an implementation of a Red-Black tree, and provides a guarantee that lookup operations (even when the key is not present in the collection) are O(log n). Since the time to find a key in the collection is O(log n), there would be a small performance hit when adding more data to the collection.

I actually began using intraday price history that I had downloaded from TradeStation, but I ran into the problem that I only had intraday price history over a 1-year period. I needed to be able to run 1-year trials over the course of a much larger period of time, so I switched to EOD data. Switching to EOD data had the nice side effect of speeding up my program (mainly due to the fact that loading all that intraday price history into memory took several seconds - up to 30 seconds or so).