Thinking of a genre for this game is actually a good idea. I recall that the last time I saw something like this I think it was a google challenge where they had some kind of spaceship attack game where they pit user submitted AI's vs each other.