based on what you said, and my assumption that Elixir (thanks to Erlang’s VM) is pretty good at concurrency, then yes Elixir would be good for crawling