EuroPython 2020 - Speak python with devices

Speak Python with Devices
Petertc Chu @ EuroPython 2020

We will see:
● How Python can be used in IoT/infrastructure automation tasks
You might:
● wish to know how Python can be used beyond data analysis and web dev
● a Pythonista who interested in craft some touchable things 🚂🚂🚂
● want to acquire something new into your Python skillset 💪💪💪
Why this session?

Outline
● Devices in Linux/UNIX-like system
● How to manipulate a device
● Manipulate a device in Python, a mini example
● A more attractive example

Computer oganization, brieﬂy
User space
Hardware LED panel, camera and sensors on bluetooth/USB/serial/parallel ports...
Kernel space Driver of LED panel, camera, sensors and other devices...
device ﬁles in the /dev/ directory
Your Python interpreter, packages and code here

Everything is a ﬁle, so is a
device

Manipulate a device
with common ﬁle operations:
● open()
● write()
● read()
● close()
and a more interesting one...
Example: blink an LED on Raspberry Pi

Input Output ConTroL
Read/write is not enough for a device which is more complex than an LED
Example: a modem
● READ - reveive data
● WRITE - send data
● IOCTL - talk to the modem itself, e.g., set bitrate, get conﬁg
IOCTL - What? Why?

IOCTL - Decoration
#include <sys/ioctl.h>
int ioctl(int fd, unsigned long request, ...);
ﬁle descriptor arguments
request (direction, type,
number, argument size)

● ﬁle descriptor
● request
○ a.k.a. (device-dependent) request code or command
○ composed of:
■ type (8 bits, a~z)
■ number (8 bits, 1~255)
■ argument size (14 bits, max 16KB)
■ direction (2 bits, R/W/RW/NONE)
● argument (string, a C struct or anything)
IOCTL - Parameters

An analogy
PUT /myBucket/my-object.jpg HTTP/1.1
Host: s3.amazonaws.com
Date: Fri, 24 Jul 2020 06:00:00 GMT
Authorization: authorization string
Content-Type: text/plain
Content-Length: 11434
x-amz-meta-author: Janet
Expect: 100-continue
[11434 bytes of object data]
ﬁle descriptor
direction type number
argument size
arguments
request (direction, type,
number, argument size)

just like RESTful APIs
we use every day!
“DON’T PANIC!”

Let’s start from a mini example:
Get the name of input devices

Do IOCTL() from Python
Things to do:
● Create an IOCTL request (header)
● C<->Py type convertion (body)
● Do IOCTL system call
(At least) two approaches:
● C extension module
● Pure Python solution

Approach 1: C extension module
Step 1: IOCTL call
Create IOCTL request (header) by macros

Step 2: C<->Py type convertion (req/resp body)

Step 3: packaging

Install and use it as usual

Approach 2: Pure Python solution
Step 1: Create an IOCTL request (header)
● porting IOC* macros from asm-generic/ioctl.h => Someone has already done it!
○ olavmrk/python-ioctl
○ vpelletier/python-ioctl-opt
● porting driver speciﬁc macros
Courtesy of
vpelletier/python-ioctl-opt/blob/master/README.rst

Step 2: ioctl call and C<-> data type convertion
Use build-in module
byte array <-> str
macro we implemented in
the first step

Same result we saw before

Question: any use case? 🤔
OK now I know how these things work but...

Raspberry Pi + ?
Hmm… good for demonstration, boring
to be a use case.
https://zh.wikipedia.org/wiki/%E6%A0%91%E8%8E%93%E6%B4
%BE#/media/File:Raspberry_Pi_4_Model_B_-_Side.jpg

The CERN Advanced STORage system (CASTOR)
http://storageconference.us/2010/Presentations/MSST/15.Bah
yl.pdf
http://castor.web.cern.ch/castor/
https://youtu.be/IDgXa0ioVTs

Explore the universe 🚀
with what we learn today!

Quick start
● Device: mhVTL simulated
● Driver: Linux SCSI tape (st) device driver

Quick start
Typical tape write procedure:
1. Find the cartridge by barcode scanner
2. Load the cartridge by a robot arm
3. Check the cartridge status is ready
4. Rewind the cartridge by a tape drive
5. Write data on the cartridge
6. Unload the cartridge
👈 👈 👈 What we're gonna do today
👇 👇 👇

Snippet 1:
Get tape status
by C extension
// open device file
int fd;
if ((fd = open(device, O_RDONLY)) < 0) {
PyErr_SetFromErrno(PyExc_OSError);
return NULL;
}
// execute ioctl command
struct mtget status;
if (ioctl(fd, MTIOCGET, (char *)&status) < 0) {
PyErr_SetFromErrno(PyExc_OSError);
return NULL;
}
if (status.mt_type != MT_ISSCSI2) {
PyErr_SetString(PyExc_NotImplementedError, "Unsupported tape
type");
return NULL;
}
close(fd);
// return status info in dict
return Py_BuildValue("{s:i,s:i,s:i}",
"file number", status.mt_fileno,
"block number", status.mt_blkno,
"partition", (status.mt_resid & 0xff)
);
}

Snippet 2:
use struct
Convert function arguments back and
forth. struct.pack() and
struct.unpack() are your friends
here.
def rewind(device):
MTREW = 6
mt_com = struct.pack('hi', MTREW, 1)
MTIOCTOP = IOW(ord('m'), 1, len(mt_com))
with open(device, 'r') as fd:
fcntl.ioctl(fd, MTIOCTOP, mt_com)
def status(device):
long_size = 8
int_size = 4
status = bytearray(long_size * 5 + int_size * 2)
MTIOCGET = IOR(ord('m'), 2, len(status))
fcntl.ioctl(fd, MTIOCGET, status)
status = struct.unpack('lllllii', status)
return {
"file number": status[-2],
"block number": status[-1],
"partition": status[1] & 0xff
}

Bonus:
rewind cartridge
by ctypes
Deﬁne input/output/buﬀer data
structure by extending
ctypes.Structure
class mtop(ctypes.Structure):
_fields_ = [
("mt_op", ctypes.c_short),
("mt_count", ctypes.c_int)
]
def rewind(device):
MTIOCTOP = ioctl.linux.IOW('m', 1, ctypes.sizeof(mtop))
MTREW = 6
mt_com = mtop(MTREW, 1)
ioctl.ioctl(fd.fileno(), MTIOCTOP,
ctypes.byref(mt_com))

Example code is available on https://github.com/hrchu/playioctl
Takeaway
● You can manipulate a device like a ﬁle
● IOCTL is just like RESTful APIs we use every day
● Yes, we can speak Python while working on IoT and
infra automation tasks

Thank you! 🙏🙏🙏
@petertc_chu

EuroPython 2020 - Speak python with devices

More Related Content

What's hot (20)

Similar to EuroPython 2020 - Speak python with devices (20)

More from Hua Chu (6)

Recently uploaded (20)

EuroPython 2020 - Speak python with devices