How do you think it would go learning "backwards" from Elixir (which I'm currently learning w/ Phoenix for another purpose)? I've glimpsed through Reason/Bucklescript but never spent real time in them.

I'm not familiar with Elixir, so I can't speak to that. But I would say that once you are familiar with functional programming, the choice of language comes down to what stack your company already uses, library support, and problem area. For instance, I would love to use OCaml, but we are primarily a dotnet house, so I use F# when I can. For Spark stuff, I use Scala. I'm not a language zealot, I prefer to use functional languages, and encourage their use when I can, but I use the best tool for the job within the constraints of the environment I'm working in.

Not sure that's helpful, but I would just say if you are using Elixir you're probably already in a good place development-wise.

Is there any reason learn OCaml over F# for a new project?

I would look at what libraries you need. If you're a Microsoft house, I would go with F#. If performance is the highest priority, I would go with OCaml.