Using Rust in Mercurial

Initial reaction to headline was it sounded like (a) no more python, and (b) this is a decided future direction.

Instead, it sounds like this is a proof-of-concept for flipping the main 'hg' command from being python + C extensions, to instead being a rust binary with an embedded python interpreter. Part of the rationale appears to be performance, but also smoothing out cross platform experience, especially on Windows.

Pulling out some related snippets:

-----

While Python is still a critical component of Mercurial and will be for the indefinite future, I'd like Mercurial to pivot away from being pictured as a "Python application" and move towards being a "generic/system application." In other words, Python is just an implementation detail.

-----

Desired End State

hg is a Rust binary that embeds and uses a Python interpreter when appropriate (hg is a Python script today). Python code seemlessly calls out to functionality implemented in Rust. Fully self-contained Mercurial distributions are available (Python is an implementation detail / Mercurial sufficiently independent from other Python presence on system)

-----

"Standalone Mercurial" is a generic term given to a distribution of Mercurial that is standalone and has minimal dependencies on the host (typically just the C runtime library). Instead, most of Mercurial's dependencies are included in the distribution. This includes a Python interpreter.

-----

This patch should be considered early alpha and RFC quality.

What are these quotes from? Sounds like it contains some info the original post doesn't.

The submitted article was to the hg wiki.

I also included some quotes from the related code revision in Phabricator:

https://phab.mercurial-scm.org/D1581

... and now looks like the HN headline was improved to be more representative of the content compared to the earlier headline.

what does embed python interpreter mean? are they actually writing a python interpreter using rust so they can write python code and compile to rust?

python has a concept of "extending" and also "embedding". It looks like they are looking at embedding[0], which enables you use the normal CPython interpreter from within another program. (So no, not writing a new Python interpreter in Rust).

Sample snippet from python docs:

-----

So if you are embedding Python, you are providing your own main program. One of the things this main program has to do is initialize the Python interpreter. At the very least, you have to call the function Py_Initialize(). There are optional calls to pass command line arguments to Python. Then later you can call the interpreter from any part of the application.

There are several different ways to call the interpreter: you can pass a string containing Python statements to PyRun_SimpleString(), <...etc..>

-----

[0] https://docs.python.org/3/extending/embedding.html

If interested, you can see their work-in-progress main.rs in the related code revision[0], which includes their Rust code calling down to the C function Py_Initialize() to spin up the now-embedded CPython interpreter that is living "inside" a Rust program:

    unsafe {
        Py_Initialize();
        PySys_SetArgv(args.len() as c_int,
                      argv.as_ptr());
        PyEval_InitThreads();
        let _thread_state = PyEval_SaveThread();
    }

----

[0] https://phab.mercurial-scm.org/D1581#change-t24aVkGEJ5Xh

> what does embed python interpreter mean?

Embedding CPython to script/extend larger applications (much like Lua) is a first-class use case of CPython and well-supported: https://docs.python.org/3/extending/embedding.html.

For instance Civ IV used Python as its scripting language (though I believe Civ V switched to Lua).