Code Generation in Perl
2 likes901 views
Ian Kluft's presentation on code generation in Perl discusses the concept of programs that write programs, highlighting various methods such as templates, translators, and compilers. While code generation allows programmers to work at a higher level, the necessity of human-written code remains, and the limitations of artificial intelligence in this area are emphasized. The presentation also touches on Unified Modeling Language (UML) as a tool for generating code and the ongoing research in this field.
Code Generation in Perl
- 1. Code Generation in Perl Presentation by Ian Kluft for Silicon Valley Perl September 7, 2017 Santa Clara, California
- 2. What is Code Generation? ● Programs that write programs ● There are many ways to approach this ● Advantage: lets the coder work at a higher level ● Disadvantage: someone still has to write program
- 3. The simplest example ● A program hard-coded to print another program ● Even the simple example has non-trivial quoting #!/usr/bin/perl use strict; use warnings; print qq(#!/usr/bin/perln); print qq(use strict;n); print qq(print "hello worldn";n);
- 4. Other approaches ● Templates – fill in the blanks ● Translators – parse, analyze and re-generate ● Compilers – parse high level, generate lower level ● SQL schemas & queries – generate SQL code ● XML parsing – generate programs from XML data ● UML tools – draw diagrams, generate code
- 5. CPAN Modules which generate code ● At a low level it’s widespread – Many modules & scripts generate code for eval() ● Perl build tools ● SQL generation for queries and schemas ● Perl application frameworks – Easier to get started with tools that generate examples ● Compilers
- 6. CPAN Modules Perl build tools ● CPAN Modules which generate code – Dist::Zilla – ExtUtils::MakeMaker – Module::Build ● These generate build scripts from module configuration data
- 7. CPAN Modules SQL code generation ● DBIx::Class – object relational mapper ● Fey – SQL schema and query generator ● …CPAN is huge - the list goes on and on... ● Many examples – which do you like?
- 8. CPAN Modules XML & data code generators ● Data::Dumper – dumps data as Perl code ● Faker – extensible fake data generator ● Not as much code generation from XML on CPAN – Obvious category for ad hoc tools – Any XML parser can be used to generate code ● Writing HTML is not technically writing code – But it also has similarities
- 9. CPAN Modules Compilers ● B::* - access to Perl compiler ● Compiler generation tools – Parse::Yapp/Parse::Eyapp – compiler parser generator ● Similar to Unix “yacc” but generates Perl code – Parse::RecDescent – generate recursive descent compiler – Parse::Lex – generate lexical analyzer for compiler ● A lot of CPAN modules claiming to “compile” actually “translate”
- 10. How Compilers Work ● High level view of a compiler Preprocessor Scanner/ Tokenizer Parser Semantics ● Preprocessor – macros & text manipulation ● Scanner – generates stream of tokens – Removes comments ● Parser – recognizes language structures ● Semantics – generate code – Some compilers like Perl make intermediate code for interpretation
- 11. Using Parser Generator Tools ● Have you noticed that language compilers do things that are way above the capabilities of regular expressions? – RegEx’es are ideal for tokenizers/scanners – There’s another level of language parsing above that ● One example is to grab a yacc grammar and translate it to Parse::Yapp or Parse::Eyapp – I did that once for a Perl script that parsed /etc/sudoers files for automated security audits ● Language definitions in Backus-Naur Form (BNF) can be converted to language parser defintions
- 12. UML Unified Modeling Language ● XML format representing standard diagrams – Parts of a program or system design – Analysis of code can generate UML for documentation – Active research into ways to generate code from UML ● Code generation can be done ● But diagramming is far more tedious than typing code ● You can experiment with this on your own projects ● Idea is to use UML as design/documentation & generate code
- 13. UML – Unified Modeling Language types of diagrams Diagram Class Diagram Component Diagram Object Diagram Activity Diagram Use Case Diagram Profile Diagram Composite Structure Diagram Deployment Diagram Package Diagram Interaction Diagram State Machine Diagram Sequence Diagram Communication Diagram Interaction Overview Diagram Timing Diagram Notation: UML Behaviour Diagram Structure Diagram UML diagram types – as a class diagram (from Wikipedia)
- 14. UML – Unified Modeling Language example class: Pizza UML code generation makes class source code starter templates for ● Pizza ● CrustEnum ● ToppingList ● ToppingEnum ● SizeEnum How would you go from here to write the code for the classes in UML? Good ways to do that are still open to suggestion.
- 15. UML – Unified Modeling Language Tools for Code Generation from UML ● Open source “umbrello” from KDE generated previous slide’s graphic ● Eclipse IDE has UML generation tools ● CPAN has many UML modules, mostly for generating UML from Perl for analysis ● Many other examples...
- 16. Setting expectations What a non-engineer manager wants… This one doesn’t grasp the current limitations. He isn’t alone. What people want is often a hint where research should push the limits for the future.
- 17. Setting expectations ”Artificial Intelligence” ● Artificial Intelligence has a broken promise built right into its name ● It has always been advanced algorithms – It’s always non-trivial – Not all uses of the term AI are as sophisticated as claimed ● Its results usually appear clever ● It can do lots of things people don’t do well ● But it still can’t think ● It isn’t real or artificial intelligence… yet
- 18. Science Fiction examples Code generation and AI ● One early SciFi story: “The Adolescence of P1” by Thomas J Ryan, 1977 ● Based on modem-connected IBM 360 mainframes ● Program to break OS “supervisor” – Needed decision trees, code generator, threat analysis – When told to shut down it played dead, kept running – AI evolved organically – Installed itself like a worm, broke OS like a virus ● Few SciFi AI’s attempted such depth
- 19. Code Generation Setting expectations back to reality ● SciFi myths need to be debunked ● Code generation just adds a level of abstraction ● Someone still has to write the code ● Short of an actual artificial intelligence, programs won’t be writing programs any time soon ● Maybe during 21st Century? – 20th Century showed how much advancement can happen – But AI is a fundamentally difficult computing challenge – Don’t count on it
- 20. Code Generation Q & A