First Class Variables as AST Annotations

First Class Variables

as

AST Annotations
Marcus Denker

Inria RMoD

Part I: The AST
• AST = Abstract Syntax Tree

• Tree Representation of the Method

• Based on the RB AST

• Used by all tools (refactoring, syntax-highlighting,…)
Smalltalk compiler parse: 'test ^(1+2)'

AST
• RBMethodNode Root

• RBVariableNode Variable (read and write)

• RBAssignmentNode Assignment

• RBMessageNode A Message (most of them)

• RBReturnNode Return

Inspect a simple AST
• A very simple Example
Smalltalk compiler parse: 'test ^(1+2)'

User: Tools
• Refactoring

• Breakpoints / Watchers

• Syntax Highlight / Code Completion

• AST based Menu in the Code Browser

User: The Compiler
Source AST
Annotated
AST
IR
Annotated
AST
Bytecode
RBParser OCSemanticAnalyzer
OCASTTranslator/
IRBuilder
IRBytecodeGenerator

Variables in the AST
• Example: (Point>>#x)

•

Problem: Kind of Variable?
• Example: SHRBTextStyler

• Syntax highlighting needs to know which kind

• Every de
fi
nition, read and write gets one new instance
of RBVariableNode (as we have to encode the parent
for each di
ff
erently)

• We just know the name

• SYNTAX, but no SEMANTICs

• Kind? (temp or ivar)

• Variables with same name can be di
ff
erent
variables

To the Rescue: Name
Analysis
• We have to annotate the AST with information about
Variables

• Block/Method: de
fi
ned Variables are put in a Scope

• Scopes know the parent Scope

• When we see a use, we loop up the variable in the Scope

Semantic Variables
• Every RBVariableNode gets a semantic variable
annotation

• Both the de
fi
nition and all uses

• There is one instance for each variable that models

• name

• scope it was de
fi
ned

• Example Again: (Point>>#x)

•

Variables and Compilation
• Compiler just delegates to the Variable, e.g for instance
Variables:

• emitStore/emitValue: de
fi
ned for each kind of Variables
(global/temp/ivar)
emitStore: methodBuilder
"generate store bytecode"
methodBuilder storeInstVar: index

Repeat:The AST
• AST = Abstract Syntax Tree

• Tree Representation of the Method

• Produced by the Parser (part of the Compiler)

• Used by all tools and the Compiler

• We need to model Variables semantically to make it useful

Look at it from Re
fl
ective
Point of View

PartII

First Class Variables

Instance Variables
• De
fi
ned by the Class (list of variable names)

• Can be read via the object:

• instVarNamed:(put:), #instVarAt:(put:)
• Instance Variables have an o
ff
set in the Object

• De
fi
ned by the order of the de
fi
ned vars in the Hierarchy
1@2 instVarNamed: 'x'

Temporary Variable
• De
fi
ned by a method or Block

• Arguments are temps, too

• Can be read via the context

• #tempNamed:, tempNamed:put:
• With Closures this is more complex than you ever want to
know!
[| temp | temp := 1. thisContext tempNamed: 'temp' ] value

Globals
• Entries in the “Smalltalk globals” Dictionary

• Contain the value

• Can be read via the global Dictionary

• Access via #value / value: on the Association

• Class Vars and Pool Vars are just Associations from other
Dictionaries
Smalltalk globals at: #Object.
Object binding value.

Globals/Class Vars
• Here we have at least the Association (#binding):

• But there is no “GlobalVariable” class

• No API other than #value:/#value

• Classes de
fi
ne just names of variables
Object binding

Instance Variables
• The class just knows the names

• There is no Object representing instance variables

• Classes de
fi
ne just names of variables

• Bytecode accesses by o
ff
set
Point allInstVarNames

Temporary Variables
• The methods know nothing. Even to know the variable
name we need the compiler (and the source)

• There is no object representing temp Variables

• Re
fl
ective read and write is *hard* -> compiler needs to
create extensive meta-data

Why Not Do Better?
• Every de
fi
ned Variable is described a meta object

• Class Hierarchy: Variable

The Hierarchy
• Variable

• LiteralVariable

• ClassVariable

• GlobalVariable

• UndeclaredVariable

• WorkspaceVariable
• LocalVariable

• ArgumentVariable

• TemporaryVariable

• ReservedVariable

• SelfVariable

• SuperVariable

• ThisContextVariable

• Slot

Example: vars of a class
• Get all Variables of a class

• Inspect it

• #usingMethods
Point instanceVariables

Instance Variable
• Read x in a Point

• Write

• read/write without sending a message to the object!
(Point instanceVariables
fi
rst) read: (5@4)
point := 5@4.
(Point instanceVariables
fi
rst) write: 100 to: point.

Globals
• Object binding class

• Object binding read

• We keep the Association API so the Global Variables can
play the role of associations in the global dictionary.
Object binding usingMethods

Temporary Variables
• There are too many to allocate them all

• They are created on demand (with the AST)
((LinkedList>>#do:) temporaryVariableNamed: 'aLink')

#lookupVar:
• Every variable knows the scope is was de
fi
ned in

• Every scope know the outer scope

• #lookupVar: looks up names along the scope
[ | temp |thisContext lookupVar: 'temp' ] value.
[ | temp |thisContext lookupVar: ‘Object' ] value
(Point slotNamed: #x ) scope outerScope

Debugger: Read Vars
• In the Debugger we to be able to read Variables from a
DoIt.

• lookupVar, then readInContext works for all Variables!

• If you know the context, you can read any variable

• DoItVariable: Nice names in DoIts (—> Show Us)
[ | temp | temp :=1 . (thisContext lookupVar: 'temp')
readInContext: thisContext] value

Part III: Putting it Together
• We have seen how Semantic Variables are needed to
make the AST useful

• We have seen First Class Variables as part of the
Re
fl
ective Model

• Do we really need the two?

Solution: Scope
• What is needed? Add the concept of Scope

• Scope of a global is Smalltalk globals

• Scope of an instance variable is the class

• Scope of temp: method and block scope

Example: Point x
(Point slotNamed: #x) scope == Point
(Point lookupVar: #x) == (Point slotNamed: #x)
(Point>>#x) ast variableNodes
fi
rst variable == (Point slotNamed: #x)

What do we get?
• Simpli
fi
ed Name Analysis in the Compiler

• Open Compiler: De
fi
ne your own kinds of Variables

• While fully integrated in the Re
fl
ective Model

• Re
fl
ective Reading/Writing

• All tools work for you own kinds of Variables

What we did not see…
• De
fi
ne your own kinds of Variables (e.g. subclasses of
Slot / ClassVariable)

• Fluid Class De
fi
nitions: How to create classes that use
these variables

• How this enables DoIts with nice variable names

• Re
fl
ection: MetaLinks on Variables

Thanks…
• This is the work on *many* contributors from the Pharo
Community

• Thanks for lots of interesting discussions, ideas, and
code!

Questions?
• We have seen how the AST needs semantic variables to
be useful

• We have seen First Class Variables as part of the
Re
fl
ective model

• First Class Variables, with just adding the concept of a
Scope, can serve as semantic annotations on the AST

First Class Variables as AST Annotations

More Related Content

Similar to First Class Variables as AST Annotations (13)

More from ESUG (20)

Recently uploaded (20)

First Class Variables as AST Annotations