News for hackers. Stuff that matters.
Re:Wolfram Alpha’s New Data about Pokémon by _frog on Wednesday October 16, 2013 @10:53PM (#6563423)
It would be extremely cool if they'd expose information such as what moveset each Pokémon has available to them so I could make queries like 'what water type Pokémon that can learn Hydro Pump has the highest Special Attack stat'.
Re:Wolfram Alpha’s New Data about Pokémon by mappu on Thursday October 17, 2013 @12:58AM (#6563749)
Maybe consider scraping bulbapedia into an SQL database? :)

    -- i'm dreaming
    SELECT pokemon.name
    FROM
      pokemon
      JOIN pokemon_learn_moves ON pokemon.id = pokemon_learn_moves.pokemon_id
      JOIN move ON pokemon_learn_moves.move_id = move.id
    WHERE
      move.name_en = "Hydro Pump"
    ORDER BY pokemon.sp_atk DESC LIMIT 1;
Actually, from this perspective there's a lot of boilerplate, no wonder people like key-value stores... Then use python-nltk to make an english wrapper (and say goodbye to your free time for the next month!)
Re:Wolfram Alpha’s New Data about Pokémon by gonnakillme on Thursday October 17, 2013 @02:02AM (#6563891)
The veekun pokedex[1] is on github[2] and they have a crapload of csv[3] files available.

[1]: http://veekun.com/dex [2]: https://github.com/veekun [3]: https://github.com/veekun/pokedex/tree/master/pokedex/data/c...

Re:Wolfram Alpha’s New Data about Pokémon by mappu on Thursday October 17, 2013 @02:44AM (#6563975)
That's excellent, thank you for pointing those out! Renders the whole discussion somewhat moot.
Re:Wolfram Alpha’s New Data about Pokémon by klodolph on Thursday October 17, 2013 @01:22AM (#6563824)
> no wonder people like key-value stores

You're doing a relational query: select, join, project. Translate it to a key-value store and you're going to end up with way more boilerplate. Using a key-value store for a relational query is like trying to use a screwdriver to drive a nail.

Re:Wolfram Alpha’s New Data about Pokémon by mappu on Thursday October 17, 2013 @01:47AM (#6563858)
It definitely suits a relational query.

But assuming you have indexes `pokemon_learn_moves_by_move_id` and `moves_by_name`, you could write

    pokemon_learn_moves_by_move_id[ moves_by_name["Hydro Pump"].id ]
        .map( |id| [pokemon[id].sp_atk, pokemon[id].name] )
        .greatest()[1];
(using a hypothetical system) which i think is a bit less boilerplate. Although, now i've explicitly written an execution plan rather than letting the SQL engine decide (i had `sort()[0]` there instead of `greatest()` for a while, which i think sums up why the SQL approach is generally better).
Re:Wolfram Alpha’s New Data about Pokémon by _frog on Thursday October 17, 2013 @01:10AM (#6563783)
I've thought about doing this before actually, don't know how they'd feel about someone scraping their content though.
Re:Wolfram Alpha’s New Data about Pokémon by mappu on Thursday October 17, 2013 @01:15AM (#6563804)
It's not like bulbapedia own the fundamental content - do they place a public license on the wiki pages?

Ideally bulbapedia would provide mediawiki dumps for this, but they don't, and they've gone on record saying they don't intend to. They did leave the mediawiki API open though if you want to crawl a clean rip of each page's wikitext - the default mediawiki API guidelines are also intact, which say that single-threaded crawls should be acceptable in almost all instances, but you should warn the site owner before initiating a multi-threaded scrape.

Re:Wolfram Alpha’s New Data about Pokémon by _frog on Thursday October 17, 2013 @01:19AM (#6563815)
It looks like all of Bulbapedia's content is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike license. http://bulbapedia.bulbagarden.net/wiki/Bulbapedia:Copyrights
Re:Wolfram Alpha’s New Data about Pokémon by kozlovsky on Thursday October 17, 2013 @06:59AM (#6564562)
Actually, it is not always necessary to write a lot of boilerplate code when using a SQL database. For example, with Pony ORM the same SQL query could be written as following:

    select(p.name for p in pokemon
           if "Hydro Pump" in p.learn_moves.move.en_name
           ).order_by("p.sp_atk").first()
Re:Wolfram Alpha’s New Data about Pokémon by arcatek on Thursday October 17, 2013 @04:40AM (#6564237)
Somewhat related, I made a prolog library to make this kind of query few months ago. I didn't add the moves (yet), but learning Prolog with this kind of project is pretty fun !

https://github.com/arcanis/trivia.pokemon-prolog