Introduction to cython: example of GCoptimization
1 like1,243 views
This document discusses using Cython to interface Python with C/C++ code to improve computational performance. It provides two examples: (1) wrapping an entire C++ graph cut library in Cython, resulting in an 18 second runtime; and (2) using Cython to call a C++ graph cut function as a black box, achieving a runtime of 0.37 seconds, nearly 50 times faster. The document emphasizes that Cython can provide large speedups with relatively little code by leveraging existing optimized C/C++ implementations.
![Declare what you need from C++
cdef extern from "graph.h":
cdef cppclass Graph[captype,tcaptype,flowtype]:
Graph( size_t, size_t )
size_t add_node(size_t)
void add_edge(size_t,size_t,captype,captype)
void add_tweights(size_t,tcaptype,tcaptype)
flowtype maxflow()
int what_segment(size_t)
8/21](https://image.slidesharecdn.com/presentation-131101045426-phpapp02/85/Introduction-to-cython-example-of-GCoptimization-8-320.jpg)
![Create your Python class
cdef class PyGraph:
# hold a C++ instance which we’re wrapping
cdef Graph[captype,tcaptype,flowtype] *thisptr
def __cinit__(self, size_t nb_nodes, size_t nb_edges):
self.thisptr = new Graph[captype,
tcaptype, flowtype](nb_nodes,nb_edges)
def __dealloc__(self):
del self.thisptr
9/21](https://image.slidesharecdn.com/presentation-131101045426-phpapp02/85/Introduction-to-cython-example-of-GCoptimization-9-320.jpg)
![Write setup.py
from
from
from
from
distutils.core import setup
distutils.extension import Extension
Cython.Distutils import build_ext
numpy.distutils.misc_util import get_numpy_include_dirs
setup(
cmdclass = {’build_ext’: build_ext},
ext_modules = [
Extension( "graphcut",
[ "graphcut.pyx",
"../maxflow-v3.02.src/graph.cpp",
"../maxflow-v3.02.src/maxflow.cpp" ],
language="c++",
include_dirs=get_numpy_include_dirs()+["../maxflow-v3.02.src"],
)
]
)
And build:
python setup.py build_ext --build-temp tmp
--build-lib lib
--pyrex-c-in-temp
11/21](https://image.slidesharecdn.com/presentation-131101045426-phpapp02/85/Introduction-to-cython-example-of-GCoptimization-11-320.jpg)
![And use it!
from lib import graphcut
G = graphcut.PyGraph(nb_pixels,nb_pixels*(8+2))
G.add_node(nb_pixels)
...
print "building graph..."
for i in range(img.shape[0]):
for j in range(img.shape[1]):
for a,b in neighbourhood:
if ( 0 <= i+a < img.shape[0]
and 0 <= j+b < img.shape[1] ):
dist = np.sqrt( a**2 + b**2 )
if img[i,j] < img[i+a,j+b]:
w = 1.0/dist
else:
w = np.exp(-(img[i,j] - img[i+a,j+b])**2
w /= 2.0 * std**2 * dist
G.add_edge( index(i,j,img),
index(i+a,j+b,img),
w, 0 )
12/21](https://image.slidesharecdn.com/presentation-131101045426-phpapp02/85/Introduction-to-cython-example-of-GCoptimization-12-320.jpg)
![And use it!
def graphcut( np.ndarray[voxel_t, ndim=2, mode="c"] img,
np.ndarray[unsigned char, ndim=2, mode="c"] mask,
double std ):
cdef np.ndarray[unsigned char,
ndim=2,
mode="c"] seg = np.zeros( (img.shape[0],
img.shape[1]),
dtype=’uint8’)
print "starting graphcut..."
_graphcut( <voxel_t*> img.data,
img.shape[0], img.shape[1],
std,
<unsigned char*> mask.data,
<unsigned char*> seg.data )
return seg
16/21](https://image.slidesharecdn.com/presentation-131101045426-phpapp02/85/Introduction-to-cython-example-of-GCoptimization-16-320.jpg)
Introduction to cython: example of GCoptimization
- 1. Example with GCoptimization Kevin Keraudren Imperial College London October 31st , 2013
- 2. Cython overview Syntax between Python and C (keyword cdef) C/C++ code automatically generated, then compiled into a Python module For a speed gain, variables must be declared C++ templates must be instantiated (compiled code) Choose between accessing low-level C++ or a blackbox Documentation: docs.cython.org Learn from examples: scikit-learn, scikit-image, github.com/amueller 2/21
- 3. How to? 1 Organize your C/C++ code 2 Write module.pyx 3 Write setup.py 4 Build 3/21
- 4. Example 1 Interface the whole API 4/21
- 5. Graphcut (Boykov & Kolmogorov) V (p, q ) = 1 p−q 1 p−q 2 2 2 e−(Ip −Iq ) /2σ 2 if Ip ≥ Iq if Ip < Iq σ : noise estimate 5/21
- 6. GCoptimisation (Boykov & Kolmogorov) template <typename captype, typename tcaptype, typename flowtype> class Graph { public: ... Graph( int node_num_max, int edge_num_max, void (*err_function)(const char *) = NULL); void add_edge( node_id i, node_id j, captype cap, captype rev_cap); void add_tweights( node_id i, tcaptype cap_source, tcaptype cap_sink); flowtype maxflow( bool reuse_trees = false, Block<node_id>* changed_list = NULL); termtype what_segment( node_id i, termtype default_segm = SOURCE); ... } 6/21
- 7. Begin your module.pyx import numpy as np cimport numpy as np np.import_array() ctypedef double captype ctypedef double tcaptype ctypedef double flowtype 7/21
- 8. Declare what you need from C++ cdef extern from "graph.h": cdef cppclass Graph[captype,tcaptype,flowtype]: Graph( size_t, size_t ) size_t add_node(size_t) void add_edge(size_t,size_t,captype,captype) void add_tweights(size_t,tcaptype,tcaptype) flowtype maxflow() int what_segment(size_t) 8/21
- 9. Create your Python class cdef class PyGraph: # hold a C++ instance which we’re wrapping cdef Graph[captype,tcaptype,flowtype] *thisptr def __cinit__(self, size_t nb_nodes, size_t nb_edges): self.thisptr = new Graph[captype, tcaptype, flowtype](nb_nodes,nb_edges) def __dealloc__(self): del self.thisptr 9/21
- 10. Create your Python class def add_node(self, size_t nb_nodes=1): self.thisptr.add_node(nb_nodes) def add_edge(self, size_t i, size_t j, captype cap, captype rev_cap): self.thisptr.add_edge(i,j,cap,rev_cap) def add_tweights(self, size_t i, tcaptype cap_source, tcaptype cap_sink): self.thisptr.add_tweights(i,cap_source,cap_sink) def maxflow(self): return self.thisptr.maxflow() def what_segment(self, size_t i): return self.thisptr.what_segment(i) 10/21
- 11. Write setup.py from from from from distutils.core import setup distutils.extension import Extension Cython.Distutils import build_ext numpy.distutils.misc_util import get_numpy_include_dirs setup( cmdclass = {’build_ext’: build_ext}, ext_modules = [ Extension( "graphcut", [ "graphcut.pyx", "../maxflow-v3.02.src/graph.cpp", "../maxflow-v3.02.src/maxflow.cpp" ], language="c++", include_dirs=get_numpy_include_dirs()+["../maxflow-v3.02.src"], ) ] ) And build: python setup.py build_ext --build-temp tmp --build-lib lib --pyrex-c-in-temp 11/21
- 12. And use it! from lib import graphcut G = graphcut.PyGraph(nb_pixels,nb_pixels*(8+2)) G.add_node(nb_pixels) ... print "building graph..." for i in range(img.shape[0]): for j in range(img.shape[1]): for a,b in neighbourhood: if ( 0 <= i+a < img.shape[0] and 0 <= j+b < img.shape[1] ): dist = np.sqrt( a**2 + b**2 ) if img[i,j] < img[i+a,j+b]: w = 1.0/dist else: w = np.exp(-(img[i,j] - img[i+a,j+b])**2 w /= 2.0 * std**2 * dist G.add_edge( index(i,j,img), index(i+a,j+b,img), w, 0 ) 12/21
- 13. Result 13/21
- 14. Example 2 Use C++ as a blackbox 14/21
- 15. Declare C++ function cdef extern from "_graphcut.h": void _graphcut( voxel_t*, int, int, double, unsigned char*, unsigned char* ) 15/21
- 16. And use it! def graphcut( np.ndarray[voxel_t, ndim=2, mode="c"] img, np.ndarray[unsigned char, ndim=2, mode="c"] mask, double std ): cdef np.ndarray[unsigned char, ndim=2, mode="c"] seg = np.zeros( (img.shape[0], img.shape[1]), dtype=’uint8’) print "starting graphcut..." _graphcut( <voxel_t*> img.data, img.shape[0], img.shape[1], std, <unsigned char*> mask.data, <unsigned char*> seg.data ) return seg 16/21
- 17. Result 17/21
- 18. Timing Example 1: 18.01s Example 2: 0.37s Nearly 50 times faster... 18/21
- 20. Conclusion Huge speedup for a low amount of code Perfect if C++ code already exists Make sure your Python code is optimised (good use of numpy) before using cython Slides and code are on github github.com/kevin-keraudren/talk-cython 20/21
- 21. Thanks! 21/21