PyO3: Python Rust binding

by blacksmythe • 9 years ago

138 15 9 years ago

kevinnk• 9 years ago

Very cool! Good binding libraries between C++ and Python have been super useful for my projects in the past, and I'm glad to see something similar developed for Rust.

For anyone reading the comments first, this is a fork of rust-cpython. The motivation for the fork is linked from the Readme: https://github.com/PyO3/pyo3/issues/55

Steeeve• 9 years ago

It looks like you forked because rust-cpython was abandoned. I see that Dan Grunwald pushed a commit in today after no activity since May. He probably just went on summer vacation.

dgrunwald• 9 years ago

rust-cpython will most likely stay abandoned; I rarely have the time (or interest) to work on it. The commit today was me just finally finishing up a change I started in February...

What happened with the fork: in May @fafhrd91 contacted me about joining the project. When I had time to consider his request (on the following weekend), he had already forked the project and deleted all his original pull requests. He even deleted his bug report about that segfault he keeps mentioning when comparing PyO3 to rust-cpython.

On the technical side, the API changes in PyO3 are interesting. It's kind of a return to my original design (with `&'a PyObject<'p>`), but he makes it work with just a single lifetime by using a "release pool" that temporarily holds ownership over the python references. This does make the API significantly nicer to use, at the cost of leaking any Python objects until the GIL is released / until the extension module returns to the interpreter. For the typical extension module usecase, this doesn't hurt much. But a long-running loop that allocates a temporary Python object each iteration might run out of memory due to this design.

fafhrd915• 9 years ago

sorry about not waiting longer. I didn't want to fork.

as for leaking refs, it maybe be fixed by introducing nested "Pool" objects. objective-c runtime has similar concepts and same problem with long running loops.

saurik• 9 years ago

He even deleted his bug report about that segfault he keeps mentioning when comparing PyO3 to rust-cpython.

That is just sleazy :/.

masklinn• 9 years ago

A part that bothers me about the PyO3 design is this: does it mean you can't keep Python objects around with the GIL released? e.g. looking at this comment [https://github.com/PyO3/pyo3/issues/55#issuecomment-31728794...] they compare

    impl PyList {
        fn new(py: Python) -> PyList {...}
        fn get_item(&self, py: Python, index: isize) -> PyObject {...}
    }

    impl PyList {
       fn new(py: Python) -> &PyList {...}
       fn get_item(&self, index: isize) -> &PyObject {...}
    }

if I don't misunderstand, in rust-cpython (the first snippet) you can have a PyList independent from a Python object (which I understand represents a GIL guard), you just can't operate on it if you don't hold the GIL.

In PyO3 (second snippet) that does not seem to be the case, the object itself has a lifetime dependency on holding the GIL. So you can't e.g. release the GIL to do native processing if you'll still need the native objects afterwards.

fafhrd915• 9 years ago

I also wrote asyncio event loop, based on tokio-rs, using this binding. So it is possible to write complex python extensions in Rust.

https://github.com/PyO3/tokio

welder2• 9 years ago

So this is for running Rust code from Python... Just curious, is there an existing way to do the opposite and run Python code from Rust?

fafhrd915• 9 years ago

this binding can be used both ways. check first example in readme. and https://github.com/PyO3/pyo3/blob/master/src/python.rs#L203

welder2• 9 years ago

Is this an alternative to cx_Freeze or py2exe?

Cieplak• 9 years ago

pybind11 is a similar library, but for interfacing with C++ from Python:

https://github.com/pybind/pybind11

PyO3: Python Rust binding

Related Stories