Taipei.py 2018 - Control device via ioctl from Python

用 Python 和硬體溝通一下 !
Taipei.py Sep. 2018

About the speaker
kaif@kaif.io
● (infra|storage) 打雜工
● mogilefs-moji member
● Py@home, Java@workspace
● ...

Outline in the bottom-up perspective
Device
Device driver
ioctl
Your python app here!

Operating tape device
讀寫檔案步驟
● 將磁帶放入tape library
● 以機械手臂將指定磁帶放到tape drive
● 驅動磁帶轉至要讀寫的位置 <= 今天要示範控制的部分!
● 開始讀寫

Tape drive actions
● status - Print status information about the tape unit.
● rewind - Rewind the tape.

Drivers (device file) in linux

Drivers for tape devices
● sg - Linux SCSI generic (sg) driver
● st - Linux SCSI tape (st) device driver

ioctl overview
● first appeared in Unix v7 (1979)
● a system call for input/output control
○ user space <== system call ==> kernel space
○ user code <== ioctl ==> driver
● carrys message which contains:
○ file descriptor
○ type (8 bits, a~z)
○ number (8 bits, 1~255)
○ argument size (14 bits, max 16KB)
○ direction (2 bits, R/W/RW/NONE)
○ argument (C struct in typical)

It’s just like your RESTful API call in the old school style!
Concept mapping between client-server and user space -kernel
space

It’s just like your RESTful API call in the old school style!
Concept mapping between ioctl message and restful API
PUT /myBucket/my-object.jpg HTTP/1.1
Host: s3.amazonaws.com
Date: Wed, 12 Oct 2009 17:50:00 GMT
Authorization: authorization string
Content-Type: text/plain
Content-Length: 11434
x-amz-meta-author: Janet
Expect: 100-continue
[11434 bytes of object data]
file descriptor
direction type number
argument size
arguments

ioctl() in standard C library
#include <sys/ioctl.h>
int ioctl(int fd, unsigned long request, ...);
file descriptor
arguments
IOC (direction, type,
number, argument size)

We are Pythonista!
There are at least three kinds of ways to do ioctl() from python:
● cpython extension
● ctypes — A foreign function library for Python
● fcntl — The fcntl and ioctl system call
● ...

We are Pythonista!
Assume we will like to call the python functions which are look like:
● mt.rewind(/dev/nst0)
● mt.status(/dev/nst0)

1. Define functions
1a. Parse input from PyObject to C
types
1b. Write down C procedure as usual
way
1c. Convert result back to a PyObject
static PyObject * do_status(PyObject *self, PyObject *args) {
// parse input
const char *device;
if (!PyArg_ParseTuple(args, "s", &device))
return NULL;
// open device file
int fd;
if ((fd = open(device, O_RDONLY)) < 0) {
PyErr_SetFromErrno(PyExc_OSError);
return NULL;
}
// execute ioctl command
struct mtget status;
if (ioctl(fd, MTIOCGET, (char *)&status) < 0) {
PyErr_SetFromErrno(PyExc_OSError);
return NULL;
}
if (status.mt_type != MT_ISSCSI2) {
PyErr_SetString(PyExc_NotImplementedError, "Unsupported tape
type");
return NULL;
}
close(fd);
// return status info in dict
return Py_BuildValue("{s:i,s:i,s:i}",
"file number", status.mt_fileno,
"block number", status.mt_blkno,
"partition", (status.mt_resid & 0xff)
);
}

2. Module metadata
2a. init function
2b. module definition
2c. function/method lookup table
PyMODINIT_FUNC PyInit_mt(void) {
PyObject* m = PyModule_Create(&mtmodule);
if (m == NULL) {
return NULL;
}
return m;
}
static struct PyModuleDef mtmodule = {
PyModuleDef_HEAD_INIT,
"mt", /* name of module */
NULL, /* module documentation, may be NULL */
-1,
MtMethods
};
static PyMethodDef MtMethods[] = {
{"rewind", do_rewind, METH_VARARGS,
"Rewind the tape."},
{"status", do_status, METH_VARARGS,
"Print status information about the tape unit."},
{NULL, NULL, 0, NULL} /* Sentinel */
};

1. Load C library
Load glibc by ctypes. We will call ioctl()
from this later
def _get_ioctl_fn():
global _ioctl_fn
if _ioctl_fn is not None:
return _ioctl_fn
libc_name = ctypes.util.find_library('c')
if not libc_name:
raise Exception('Unable to find c library')
libc = ctypes.CDLL(libc_name, use_errno=True)
_ioctl_fn = libc.ioctl
return _ioctl_fn

2. Define data
structure
Define input/output/buffer data
structure by extending
ctypes.Structure
class mtop(ctypes.Structure):
_fields_ = [
("mt_op", ctypes.c_short),
("mt_count", ctypes.c_int)
]
class mtget(ctypes.Structure):
_fields_ = [
("mt_type", ctypes.c_long),
("mt_resid", ctypes.c_long),
("mt_dsreg", ctypes.c_long),
("mt_gstat", ctypes.c_long),
("mt_erreg", ctypes.c_long),
("mt_fileno", ctypes.c_int),
("mt_blkno", ctypes.c_int),
]
def rewind(device):
MTIOCTOP = ioctl.linux.IOW('m', 1, ctypes.sizeof(mtop))
MTREW = 6
mt_com = mtop(MTREW, 1)
with open(device, 'r') as fd:
ioctl.ioctl(fd.fileno(), MTIOCTOP,
ctypes.byref(mt_com))
def status(device):
// plz ref it in github

1. Porting
You needs to do porting job from C to
python since fcntl does not provide any
help on this...orz
_IOC_NRBITS = 8
_IOC_TYPEBITS = 8
_IOC_SIZEBITS = 14
_IOC_DIRBITS = 2
_IOC_NRMASK = (1 << _IOC_NRBITS) - 1
_IOC_TYPEMASK = (1 << _IOC_TYPEBITS) - 1
_IOC_SIZEMASK = (1 << _IOC_SIZEBITS) - 1
_IOC_DIRMASK = (1 << _IOC_DIRBITS) - 1
_IOC_NRSHIFT = 0
_IOC_TYPESHIFT = _IOC_NRSHIFT + _IOC_NRBITS
_IOC_SIZESHIFT = _IOC_TYPESHIFT + _IOC_TYPEBITS
_IOC_DIRSHIFT = _IOC_SIZESHIFT + _IOC_SIZEBITS
IOC_NONE = 0
IOC_WRITE = 1
IOC_READ = 2
def IOC(dir, type, nr, size):
assert dir <= _IOC_DIRMASK, dir
assert type <= _IOC_TYPEMASK, type
assert nr <= _IOC_NRMASK, nr
assert size <= _IOC_SIZEMASK, size
return (dir << _IOC_DIRSHIFT) | (type << _IOC_TYPESHIFT) | (nr << _IOC_NRSHIFT) |
(size << _IOC_SIZESHIFT)
def IOC_SIZECHECK(t):
result = t
assert result <= _IOC_SIZEMASK, result
return result
def IOW(type, nr, size):
return IOC(IOC_WRITE, type, nr, IOC_SIZECHECK(size))
def IOR(type, nr, size):
return IOC(IOC_READ, type, nr, IOC_SIZECHECK(size))

2. input/output
conversion
Convert function input/output back and
forth. struct.pack() and
struct.unpack() are your friends
here.
def rewind(device):
MTREW = 6
mt_com = struct.pack('hi', MTREW, 1)
MTIOCTOP = IOW(ord('m'), 1, len(mt_com))
fcntl.ioctl(fd, MTIOCTOP, mt_com)
def status(device):
long_size = 8
int_size = 4
status = bytearray(long_size * 5 + int_size * 2)
MTIOCGET = IOR(ord('m'), 2, len(status))
fcntl.ioctl(fd, MTIOCGET, status)
status = struct.unpack('lllllii', status)
return {
"file number": status[-2],
"block number": status[-1],
"partition": status[1] & 0xff
}

3. Iusse ioctl()
via fcntl.ioctl(), this is actually the
only thing fcntl can do....
def rewind(device):
MTREW = 6
mt_com = struct.pack('hi', MTREW, 1)
MTIOCTOP = IOW(ord('m'), 1, len(mt_com))
fcntl.ioctl(fd, MTIOCTOP, mt_com)
def status(device):
long_size = 8
int_size = 4
status = bytearray(long_size * 5 + int_size * 2)
MTIOCGET = IOR(ord('m'), 2, len(status))
fcntl.ioctl(fd, MTIOCGET, status)
status = struct.unpack('lllllii', status)
return {
"file number": status[-2],
"block number": status[-1],
"partition": status[1] & 0xff
}

Comparision
Pros Cons
cpython extension 最大的控制權
CPython only，要會一
點C
ctypes 功能強大、跨平台
複雜，擁有三者中篇幅
最大的文件
fcntl
毫無反應，就只是提供
ioctl()
毫無反應，就只是提供
ioctl()

● Understand your device
● Read driver’s document
● Drive you driver via ioctl()
● Binding it in Python
Example code is available on
https://github.com/hrchu/playioctl
Speaker is available!? on
https://medium.com/@petertc

Taipei.py 2018 - Control device via ioctl from Python

More Related Content

What's hot (20)

Similar to Taipei.py 2018 - Control device via ioctl from Python (20)

More from Hua Chu (8)

Recently uploaded (20)

Taipei.py 2018 - Control device via ioctl from Python