MeCab in docker action(OpenWhisk)

MECAB IN DOCKERACTION!!!
USING DOCKERACTION OF IBM
CLOUD FUNCTIONS ( OPENWHISK )
2017.10.19 Bluemix User Group Tsukiji,Japan#1

ABOUT MYSELF
 (KUNITO Atsunori)
 Atsunori_kunito@niandc.co.jp
 I’m a Java programmer at NI and C Soft.inc.
 We are Nippon Information and Communication
Corporation Group.

MY PRIVATE PROJECT
 I was creating a personal tool which monitoring google
search ranking everyday. It works on Bluemix -
OpenWhisk.
 Next , I’d like to extract frequently appeared keyword
from the top 50 web pages.
 I wish I used Watson - Natural Language
Understanding(NLU). But the NLU doesn’t support
Japanese!!!

SOLUTION – WHAT IS MECAB?
 MeCab: Yet Another Part-of-Speech and Morphological
Analyzer(http://taku910.github.io/mecab/)
 MeCab is OSS which is created by Taku Kudo.
 Japanese has no separator(space) between word each
other.
 So I can count frequency of the words with the MeCab.

DOCKER ACTION
 MeCab is written in C++. How do I run this tool on
Cloud?
Docker Action !!!
wsk action update
wsk action invoke
/init Init process
Run MeCab/run

Local
Bluemix - OpenWhisk
OVERVIEW
MeCab
MeCab
MeCabMeCab
exec
exec.zip
exec
STEP 1) Make Dockerfile
STEP 2) Push Docker image
STEP 3) Write entry point
STEP 4) Deploy exec
invoke
Docker Pull

STEP 1) MAKE DOCKERFILE
Key Learnings:
 Use ‘openwhisk/dockerskeleton’ as base image.
 Inject MeCab , dependencys and other tools(if needed) to
your docker image.
 Your docker image will be uploaded to
DockerHub(https://hub.docker.com/) and will be
PUBLIC. So You SHOULD NOT include private
informations. Be aware to password , API-key etc…
 Lunch actionproxy.py. It’s contained in
‘openwhisk/dockerskeleton’ .

DOCKERFILE
FROM openwhisk/dockerskeleton
RUN apk add --no-cache gcc libc-dev g++
RUN apk add --no-cache --virtual .build-deps git make ¥
&& apk add --no-cache libxml2-dev libxslt-dev
# install mecab
RUN cd /tmp && ¥
git clone https://github.com/taku910/mecab.git
RUN cd /tmp/mecab/mecab && ¥
./configure --enable-utf8-only && ¥
make && ¥
make install
RUN echo 'export LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH"' >> ~/.bashrc && ¥
. ~/.bashrc && ¥
cd /tmp/mecab/mecab-ipadic && ¥
./configure --with-charset=utf8 && ¥
make && ¥
make install
RUN pip install --no-cache-dir lxml
RUN pip install --no-cache-dir requests
RUN pip install --no-cache-dir beautifulsoup4
RUN apk del .build-deps
CMD ["/bin/bash", "-c", "cd actionProxy && python -u actionproxy.py"]

STEP 2) PUSH YOUR DOCKER IMAGE AND
CREATE ACTION
 Create an account in DockerHub(if you didn’t have it).
docker build -t openwhiskdatest/mecabda .
docker login
docker push openwhiskdatest/mecabda
bx wsk action create wskMeCabDH --docker openwhiskdatest/mecabda

STEP 3) WRITE AN ENTRY POINT OF YOUR
ACTION
 I wrote web crawler with Beautiful Soup and lxml in
python.
 Execute the MeCab.
 Make MeCab’s outputs into single-lined-JSON format. I
wrote this process in python too.
 Finally , I wrote BASH script which organize those
programs described above. This script must be named
‘exec’.

STEP 4) CREATE ZIP FILE AND DEPLOY IT
 Package programs described in step 3 into a zip file.
 Upload it to your action.
 Actionproxy.py receive the zip file. Then It extracts and
deploys contained programs.
bx wsk action update wskMeCabDH exec.zip --docker openwhiskdatest/mecabda

NOW WE CAN INVOKE DOCKER ACTION !!!
bx wsk action invoke wskMeCabDH --blocking --result -p "url" https://bmxug.connpass.com/event/66961/

IT WORKS ON MY PROJECT
 Google search word is ‘OpenWhisk’.
 Crawling top 50 web pages.
 Extract frequently appeared keyword.
 It’s represented with wordcloud2 in R.
 Can you read ‘Lambda’?
It shows many sites about Openwhisk
compare with Lambda , I guess.

CONCLUSION
Key Learnings:
 Docker Action is great way to execute some binaries on
Bluemix.
 Every Openwhisk actions are implemented on Docker
Action. In other words , Action Runtime is a pre-defined
Docker Action for specific programming language.

USECASE OF DOCKER ACTION
 Execute some binaries or shell scripts on Bluemix, of
course.
 You can create your runtime which supports your
favorite programing language.
 If you wanted to use some dependencies which are
unsupported in runtime , You can create a dockerfile to
inject those dependencies.

REFERENCES
 https://hub.docker.com/r/openwhisk/spellcheck/
 https://www.ibm.com/blogs/bluemix/2017/01/dock
er-bluemix-openwhisk/
 http://jamesthom.as/blog/2017/01/16/openwhisk-
docker-actions/
 DockerHub
 https://hub.docker.com/r/openwhisk/dockerskeleton/
 https://www.niandc.co.jp/sol/tech/date20170626_6
98.php

MeCab in docker action(OpenWhisk)

More Related Content

What's hot (20)

Similar to MeCab in docker action(OpenWhisk) (20)

Recently uploaded (20)

MeCab in docker action(OpenWhisk)