You might "just" migrate some slower parts to numpy or write a C target and interface with it over Python's CFFI.

+1 numpy could probably handle this

It depends on your use case.

In my experience the reason why you wrote something in python (implement features faster) remain valid reasons later on when you want to add functionality.

By using pypy, cpython, rewriting small parts in C/C++ and/or using the libraries which are written in C (such as numpy) you can normally make the hotspots in your code fast enough, while keeping the advantages of python.

In my experience it is worthwhile to first try and improve the performance in Python. It's easier to play around with different ideas in Python, and things like numpy quite often enable you to get "fast enough". Only if that is not enough should you consider writing a C extension.

Python makes it easy to do lots of things -- including very inefficient ones. Profiling your existing stuff and trying to optimize in pure Python often gets you pretty far.

I've never had to do this, but the common advice I hear is that the right algorithm in a high level language can be faster then the wrong algorithm in a low level language. So even if you do end up going the route of writing it in something lower level, it is likely worth optimizing your code at the higher level first where it is less expensive to try different things.

Same question to you: if you have some code that really has to be fast, is it viable to keep it in C++, or should you ultimately end up rewriting it for GPU?

Is this really a suitable counter-question? GPUs lend themselvse towards specific kinds of programming problems + added latency of GPU communication. On the contrary, the cost between switching from any on-CPU language is programmer time that may result in significant runtime advantages.

Also disadvantages: increased programmer time for implementing new features. Risks: you might optimize the wrong part (i.e. you invest into rewrite, but it will not solve performance problems).

That's why you must quantify advantages and disadvantages, including risks minimization and only then you'll see, whether given course of action is viable.

You should probably first consider seeing if there are critical bits of code that can be rewritten to use MMX/SSE instructions, since your data is in the right cache already, without needing to move it anywhere else.