pybind11学习 在Python中构建编译生成pyd文件

[TOC]

0 准备

项目文件夹结构树如下:

1
2
3
4
5
6
7
pybind11_setup_demo
└─demo # demo包
│ setup.py # 用于编译C++代码,生成C/C++ python扩展
│ test.py # 用于测试生成的拓展

└─src # 源码文件夹
example.cpp

example.cpp

1
2
3
4
5
6
7
8
9
10
11
#include <pybind11/pybind11.h>

namespace py = pybind11;

int square(int x) {
return x * x;
}

PYBIND11_MODULE(example, m) {
m.def("square", &square);
}

test.py

1
2
3
4
5
6
7
import example

help(example)

result = example.square(5)

print(result)

1. setuptools

参考文章:pybind11—python C/C++扩展编译 - 简书 (jianshu.com)

setup.py

1
2
3
4
5
6
7
8
9
10
11
12
from setuptools import setup
from setuptools import Extension

example_module = Extension(name = 'example', # 模块名称
sources = ['src/example.cpp'], # 源码
# 依赖的第三方库的头文件
include_dirs = [r'D:\anaconda3\envs\pybind11\include', # Python头文件路径
r'D:\anaconda3\envs\pybind11\Lib\site-packages\pybind11\include' # pybind11头文件路径
]
)

setup(ext_modules = [example_module])

打开终端,进入到setup.py文件所在目录,运行下面命令:

1
python setup.py build_ext --inplace
1
2
3
4
5
6
7
8
9
10
11
running build_ext
building 'example' extension
creating build
creating build\temp.win-amd64-cpython-38
creating build\temp.win-amd64-cpython-38\Release
creating build\temp.win-amd64-cpython-38\Release\src
...
正在创建库 build\temp.win-amd64-cpython-38\Release\src\example.cp38-win_amd64.lib 和对象 build\temp.win-amd64-cpython-38\Release\src\example.cp38-win_amd64.exp
正在生成代码
已完成代码的生成
copying build\lib.win-amd64-cpython-38\example.cp38-win_amd64.pyd ->

测试拓展模块:

1
python test.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Help on module example:

NAME
example

FUNCTIONS
square(...) method of builtins.PyCapsule instance
square(arg0: int) -> int

FILE
d:\pybind11_setup_demo\demo\example.cp38-win_amd64.pyd


25

2. pybind11.setup_helpers

参考文档网址:Build systems - pybind11 documentation

这种方法和上一种方法的区别在于,后者需要安装在Python环境中pip install pybind11。pybind11提供了setup_helpers用于简化setuptools的构建过程(作用有点类似于pybind11提供的CMake函数pybind11_add_module)。

setup.py

1
2
3
4
5
6
7
8
9
10
11
from setuptools import setup
from pybind11.setup_helpers import Pybind11Extension, build_ext

ext_modules = [
Pybind11Extension(
"example",
["src/example.cpp"],
),
]

setup(cmdclass = {"build_ext": build_ext}, ext_modules = ext_modules)

同样,打开终端,进入到setup.py文件所在目录,运行下面命令:

1
python setup.py build_ext --inplace
1
2
3
4
5
6
7
8
9
10
11
running build_ext
building 'example' extension
creating build
creating build\temp.win-amd64-cpython-38
creating build\temp.win-amd64-cpython-38\Release
creating build\temp.win-amd64-cpython-38\Release\src
...
正在创建库 build\temp.win-amd64-cpython-38\Release\src\example.cp38-win_amd64.lib 和对象 build\temp.win-amd64-cpython-38\Release\src\example.cp38-win_amd64.exp
正在生成代码
已完成代码的生成
copying build\lib.win-amd64-cpython-38\example.cp38-win_amd64.pyd ->

测试拓展模块:

1
python test.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Help on module example:

NAME
example

FUNCTIONS
square(...) method of builtins.PyCapsule instance
square(arg0: int) -> int

FILE
d:\pybind11_setup_demo\demo\example.cp38-win_amd64.pyd


25

效果和第一种没有差别,但在setup.py文件的编写上更加方便。

3. cppimport

参考网址:tbenthompson/cppimport: Import C++ files directly from Python! (github.com)

这种方法更加简单,甚至不需要编写setup.py文件。只需在Python环境中安装cppimport库即可。

修改example.cpp如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
// cppimport
#include <pybind11/pybind11.h>

namespace py = pybind11;

int square(int x) {
return x * x;
}

PYBIND11_MODULE(example, m) {
m.def("square", &square);
}
/*
<%
setup_pybind11(cfg)
%>
*/

打开终端,进入到example.cpp文件所在的src目录下,打开python解释器运行下面代码:

1
2
3
import cppimport.import_hook
import example #This will pause for a moment to compile the module
example.square(5)

运行结果如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
PS D:\pybind11_setup_demo\demo\src> python
Python 3.8.12 (default, Oct 12 2021, 03:01:40) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import cppimport.import_hook
>>> import example
cl: 命令行 warning D9002 :忽略未知选项“-std=c++11”
cl: 命令行 warning D9002 :忽略未知选项“-fvisibility=hidden”
.rendered.example.cpp
正在创建库 ...\pybind11_setup_demo\demo\src\example.cp38-win_amd64.lib 和对象 ...\pybind11_setup_demo\demo\src\example.cp38-win_amd64.exp
正在生成代码
已完成代码的生成
>>> example.square(5)
25

运行成功后,在同目录下会自动生成一个pyd文件。

总结

  • pycharm中运行setup.py文件进行拓展编译报错error: Microsoft Visual C++ 14.0 or greater is required.Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/

    按照这篇文章将python pip error:Microsoft Visual C++ 14.0 or greater is required将Microsoft C++ Build Tools安装成功后,仍然不能在pycharm的终端中运行。只能通过powershell进入文件夹目录下运行setup.py文件。这个问题目前还尚待解决。

  • 对于包含第三方C++库的pybind11项目编译中setup.py文件如何修改的问题,可以参见文章:pybind11—python C/C++扩展编译 - 简书 (jianshu.com)