20130530-PEGjs

PEG.js
- Javascript Parser Generator -
Hidetomo Suzuki

2013 / 5 / 30
Scope
Page 2
This MTG Target
Understand What’s PEG.js
Know How to Use PEG.js
Make New DSL and Parser

2013 / 5 / 30
Agenda
Page 3
Table of Contents
1.How to make parser
2. What’s PEG.js
3. Try to used to be PEG
4. Let’s make DSL
5.Applications

2013 / 5 / 30
Agenda
Page 4
Table of Contents
2. What’s PEG.js
4. Let’s make DSL
5.Applications

2013 / 5 / 30 Page 5
1. How to make parser
1. Lexical Analysis(字句解析)
2. Syntactic Parsing(構文解析)
3. Semantic Analysis(意味解析)
4. Intermidiate Code Generation(中間コード生成)
5. Code Optimization(コード最適化)
6. Code Generation(コード生成）
General Step of Processing with Compiler
Parser

2013 / 5 / 30 Page 6
1. Lexical Analysis(字句解析)
2. Syntactic Parsing(構文解析)
3. Semantic Analysis(意味解析)
Analysis for Make Parser
Make strings(minimum unit of string has semantic) from characters
Make tree structure from strings which result of Lexical Analysis
Type Check
※ A string of Lexical Analysis is called “Token”.

2013 / 5 / 30 Page 7
1. Lexical Analysis
position_x = position_x + 2.0 * time
 Identifier : position_x
 Substitution Symbol : =
 Identifier : position_x
 Addition Symbol : +
 Number : 2.0
 Multiple Symbol : *
 Identifier : time

2013 / 5 / 30 Page 8
2. Syntactic Parsing
 Identifier : position_x Substitution Symbol : =
 Identifier : position_x Addition Symbol : +
 Number : 2.0 Multiple Symbol : *
 Identifier : time
Substitution Symbol
Identifier
position_x
Expression
ExpressionExpression
Expression Expression
Identifier
position_x
Number
2.0
Identifier
time
=
+
*

2013 / 5 / 30 Page 9
3. Semantic Analysis
Substitution Symbol
Identifier
position_x
Expression
ExpressionExpression
Expression Expression
Identifier
position_x
Number Identifier
=
+
 Correct Case : Real Number * Integer Number
 Wrong Case : Real Number * Function Pointer
2.0 time
*

2013 / 5 / 30
Agenda
Page 10
Table of Contents
2. What’s PEG.js
4. Let’s make DSL
5.Applications

2013 / 5 / 30 Here comes
your footer 
2. What‘s PEG.js?
 Parser Generator for JavaScript
 Use PEG
(Parsing Expression Grammar)
 Easy to try with the web page
Outline of PEG.js
http://pegjs.majda.cz/

2013 / 5 / 30 Here comes
your footer 
2. What‘s PEG.js?
PEG(Parsing Expression Grammar) is one of grammar for artificial language.
Focus point is how input will be analyzed.
What‘s PEG
Ex) Simple Expression Grammar which Has Four Arithmetic Operations
expression <- addexp
addexp <- multiexp (“+” multiexp / “-” multiexp)*
multiexp <- number (“*” number / “/” number)*
number <- [0-9]+
Accept : 2*3+6/2-5*3 -> multiexp + multiexp – multiexp -> ・・・
Decline : (1+1)*3+6/2-(7-2)*3 -> ?*3+6/2-?*3 -> can’t recognize…

2013 / 5 / 30 Here comes
your footer 
2. What‘s PEG.js?
• Nothing Confusion
• Different : “a b / a”, “a / a b”
• Easy to use compared with CFG
• Don’t need to implement
scanner (Lexical Analyzer)
• Don’t Express Left Recursive
PEG vs CFG(Context Free Grammar)
• Confusion Existing
• Same : “a b | a”, “a | a b”
• Need to implement scanner
• Can Express Left Recursive
PEG CFG

2013 / 5 / 30 Page 14
2. What‘s PEG.js?
1. Make Grammar with PEG
2. Generate Parser with “pegjs” Command
3. Use the Parser Loaded as a Library
Step of Make Parser with PEG.js
vim filename.pegjs
Pegjs filename.pegjs parser.js
<script src=“./parser.js” type=“text/javascript”></script>
<script type=“text/javascript”>
parser.parse(“something”);

2013 / 5 / 30 Page 15
2. What‘s PEG.js?
Install PEG.js
# Install Pythonbrew
$ curl -kL http://xrl.us/pythonbrewinstall | bash
$ vim ~/.bash_profile
[[ -s $HOME/.pythonbrew/etc/bashrc ]] && source $HOME/.pythonbrew/etc/bashrc
$ source ~/.bashrc
$ pythonbrew install 2.7.2
$ pythonbrew switch 2.7.2
# Install Node
$ git clone git://github.com/creationix/nvm.git ~/.nvm
$ nvm install v0.8.7
$ source ~/.nvm/nvm.sh
$ nvm use v0.8.7
$ vim ~/.npmrc
$ registry = http://registry.npmjs.org/

2013 / 5 / 30 Page 16
2. What‘s PEG.js?
1. Make Grammar with PEG
2. Generate Parser with “pegjs” Command
3. Use the Parser Loaded as a Library
Let’s use PEG.js a little bit
$ vim ffirst.pegjs
start = addexp
addexp = integer ("+" integer / "-" integer)*
integer = [0-9]+
$ ./node_modules/pegjs/bin/pegjs first.pegjs parser.js
$ vim parser.js
1st line: module.export = … -> var parser = …
$ vim index.html
<script type=“text/javascript” src=“./parser.js”></script>
<script type=“text/javascript”>
alert(parser.parse(“1+2+3-5”));
</script>

2013 / 5 / 30
Agenda
Page 17
Table of Contents
2. What’s PEG.js
4. Let’s make DSL
5.Applications

2013 / 5 / 30 Page 18
3.Try to used to be PEG
PEG.js Online Version (http://pegjs.majda.cz/online)

2013 / 5 / 30 Page 19
Very Simple PEG.js Grammar(calc : integer)
Grammar
Test Input
Test Input Result
start = integer
integer = digits:[0-9]+ { return parseInt(digits.join(“”), 10); }
123456789
123456789

2013 / 5 / 30 Page 20
Very Simple PEG.js Grammar(calc : add two integer)
Grammar
Test Input
Test Input Result
start = target1:integer "+" target2:integer { return target1+target2; }
integer = digits:[0-9]+ { return parseInt(digits.join(""), 10); }
81+19
100

2013 / 5 / 30 Page 21
Very Simple PEG.js Grammar(calc : add and minus two integer)
Grammar
Test Input
Test Input Result
start
= target1:integer "+" target2:integer { return target1+target2; }
/ target1:integer "-" target2:integer { return target1-target2; }
100-19
81

2013 / 5 / 30 Page 22
Very Simple PEG.js Grammar(calc : add and minus integers)
Grammar
Test Input
Test Input Result
start = expression
expression
= ope1:integer "+" ope2:expression { return ope1+ope2; }
/ ope1:integer "-" ope2:expression { return ope1-ope2; }
/ ope:integer { return ope; }
1+2+3+4+5-6+7-8+9-10
9

2013 / 5 / 30 Page 23
Very Simple PEG.js Grammar(calc : four arithmetic)
Grammar
Test Input
Test Input Result
start = expression
expression
= ope1:integer "+" ope2:expression { return ope1+ope2; }
/ ope1:integer "-" ope2:expression { return ope1-ope2; }
/ ope1:integer "*" ope2:expression { return ope1*ope2; }
/ ope1:integer "/" ope2:expression { return ope1/ope2; }
2*3/2
3

2013 / 5 / 30 Page 24
Very Simple PEG.js Grammar(calc : four arithmetic with priority)
Grammar
Test Input
Test Input Result
start = expression
expression = ope1:multiple "+" ope2:expression { return ope1+ope2; }
/ ope1:multiple "-" ope2:expression { return ope1-ope2; }
/ ope:multiple { return ope; }
multiple = ope1:integer "*" ope2:multiple { return ope1*ope2; }
/ ope1:integer "/" ope2:multiple { return ope1/ope2; }
2*3/2+8/4
5

2013 / 5 / 30 Page 25
Very Simple PEG.js Grammar(calc : four arithmetic with priority)
Grammar
Test Input
Test Input Result
start = expression
expression = ope1:multiple "+" ope2:expression { return ope1+ope2; }
/ ope1:multiple "-" ope2:expression { return ope1-ope2; }
/ ope:multiple { return ope; }
multiple = ope1:bracket "*" ope2:multiple { return ope1*ope2; }
/ ope1:bracket "/" ope2:multiple { return ope1/ope2; }
/ ope:bracket { return ope; }
bracket = "(" exp:expression ")" {return exp; }
/ ope:integer {return ope; }
(7+3)*3/2+8/(6-2)
17

2013 / 5 / 30
Agenda
Page 26
Table of Contents
2. What’s PEG.js
4. Let’s make DSL
5.Applications

2013 / 5 / 30 Page 27
4. Let‘s make DSL
Config File Format
Parser Result
Web Page Image
Target
<LogicalExpression?>Y---[function param]
N---[function param]
if (isLogicalExpression) function(param);
else function(param);
Ex)
<Hydea?>Y---[show ‘hydea’]
N---[show ‘hydeb’]
Ex)
if (isHydea) show(‘hydea’);
else show(‘hydeb’);
URL: ex.com/?p=hydea URL: ex.com/?p=hydeb

2013 / 5 / 30 Page 28
4. Let‘s make DSL
Just write over specification
Make Grammar
start = "<" logic "?>Y---[" function " " param
"]nN---[" function " " param "]"
logic = [a-zA-Z]+
function = [a-zA-Z]+
param = "'" [a-zA-Z]+ "'"

2013 / 5 / 30 Page 29
4. Let‘s make DSL
Refactoring a little bit
Make Grammar
start = "<" identifier "?>Y---[" identifier " " param
"]nN---[" identifier " " param "]"
param = "'" identifier "'"
identifier = [a-zA-Z]+

2013 / 5 / 30 Page 30
4. Let‘s make DSL
Finish to make param and identifier part
Make Grammar
start = "<" identifier "?>Y---[" identifier " " param
"]nN---[" identifier " " param "]"
param
= "'" id:identifier "'"
{return "'" + id + "'"; }
identifier
= chars:[a-zA-Z]+
{return chars.join(""); }

2013 / 5 / 30 Page 31
4. Let‘s make DSL
Organize Function Part
Make Grammar
start = "<" identifier "?>Y---[" function
"]nN---[" function "]"
function
= id:identifier " " param:param
{return id + "(" + param + ")"; }
param
{return "'" + id + "'"; }
identifier
= chars:[a-zA-Z]+

2013 / 5 / 30 Page 32
4. Let‘s make DSL
Finish to make
Make Grammar
start
= "<" id:identifier "?>Y---[" f1:function
"]nN---[" f2:function "]"
{return "if (is" + id + ") " + f1 + ";nelse " + f2 + ";"; }
function
= id:identifier " " param:param
{return id + "(" + param + ")"; }
param
{return "'" + id + "'"; }
identifier
= chars:[a-zA-Z]+

2013 / 5 / 30
Agenda
Page 33
Table of Contents
2. What’s PEG.js
4. Let’s make DSL
5.Applications

2013 / 5 / 30 Page 34
5. Application
Show My Demo(Demo’s grammar)
start = tree
tree = stat:statement? { return stat; }
statement = stat:if_statement { return stat.toString(); }
/ stat:proc_statement { return stat.toString(); }
if_statement = fac:if_factor 'Y' edge branch1:proc_statement 'n'
'N' 'n' edge branch2:statement
{ return branch2.toString().match(/if/) ?
"if (" + fac + ")nt" + branch1 + 'nelse ' + branch2
: 'if (' + fac + ')nt' + branch1 + 'nelsent' + branch2;}
proc_statement = fac1:proc_factor [-|]+ fac2:proc_statement { return fac1 + '.then(' + fac2.slice(0,fac2.length-1) + ');'; }
/ factor:proc_factor { return factor + ';'; }
proc_factor = '[' procexp:expression ']' { return procexp; }
if_factor = '<' ifexp:expression '>' { return ifexp; }
expression = elem:element ' ' cdr:arguments { return elem.toString() + '(' +cdr + ')'; }
/ elem:element { return elem.toString(); }
arguments = elem:element ',' arg:arguments { return elem.toString() + ',' + arg; }
/ elem:element { return elem.toString(); }
element
= characters:[a-zA-Z0-9-_."]+ { return characters.join('') }
edge
= symbols:[-|n]+ { return symbols.join('') }

2013 / 5 / 30 Page 35
5. Application
Recommendation
Conf1 Conf2 Conf3 ・・・
Recommendation
Engine
Choose a config file

2013 / 5 / 30
Reference
Page 36
Famous Book for Compiler

2013 / 5 / 30
Thank you
Page 37
Thank you for your listening

20130530-PEGjs

More Related Content

What's hot (20)

Viewers also liked (17)

Similar to 20130530-PEGjs (20)

Recently uploaded (20)

20130530-PEGjs