Porting 100k Lines of Code to TypeScript
1 like1,933 views
The document discusses Tiny's project of converting 100,000 lines of code for their rich text editor, TinyMCE, from JavaScript to TypeScript, highlighting the motivations, benefits, and the conversion process. Key points include the advantages of type safety, gradual typing, and the challenges faced during the transition. The author shares lessons learned and ongoing efforts to improve the codebase as they further incorporate TypeScript features.
![Module Conversion Example
define(
'Example Module',
[ ‘example.Dependency' ],
function (Dependency) {
var bar = 0;
var foo = function () { … };
return {
foo,
bar
};
}
);
import { Dependency } from 'example';
var bar = 0;
var foo = function () { … };
export default <any> {
foo,
bar
};](https://image.slidesharecdn.com/porting100klinesofcodetotypescript-forwardjsjan2019-milliemacdonald-190126161351/85/Porting-100k-Lines-of-Code-to-TypeScript-16-320.jpg)
Porting 100k Lines of Code to TypeScript
- 1. tiny.cloud PORTING 100K LINES OF CODE TO TYPESCRIPT MILLIE MACDONALD SOFTWARE ENGINEER @ TINY
- 2. The Talk ● About Tiny and me ● The Conversion Project: ○ Motivation ○ Why TypeScript? ○ How we converted our JavaScript to TypeScript ○ Benefits, lessons learned and ongoing work TL;DR: Types help. Gradual typing really helps.
- 3. About Tiny ● TinyMCE is a popular open source project ● A rich text editing platform that helped launch and scale the adoption of Medium, WordPress, Shopify, Tumblr, Atlassian, EventBrite & more ● In 2015, we launched a startup with offices in Palo Alto, Australia and Sweden By the numbers: ● 1 million+ users on Cloud ● 125,000 downloads/week ● 1,000 paying customers ● 232 contributors
- 4. About Me ● From Brisbane, Australia ● Bachelor of Engineering (Honours), Software Engineering ● Intern then QA Engineer then Software Engineer at Tiny ● Admin of the Australian gaming community for Splatoon
- 5. “100k Lines of Code”? TinyMCE is: ● Core is ~30k LoC ● User Interface is ~30k LoC ● 44 core plugins + several premium plugins with ~40k LoC Plus various other projects and libraries…The point of conversion!
- 7. Before ● Over the years, we tried and rejected a lot of frameworks ● We also use a functional programming style ● So we wrote our own libraries and frameworks ○ Avoided upgrade problems and framework churn ○ Easier for us to fix problems ○ FP + JS means defensive code, which resulted in code size and runtime performance issues ● Then ES2015 appeared...
- 8. Motivation for the Conversion Project ● ES2015 features - especially a standard module system - were very enticing, so in 2017 we decided to convert our code. ● We had also been considering switching to an alternative JS language. Our wish list was: ○ static typing ○ pattern matching
- 9. Why TypeScript? ● We looked at ReasonML, PureScript and TypeScript ● We considered: ○ how much of the functional programming feature set we needed ○ the learning curve for each language ○ the upfront development cost to switch
- 10. Why TypeScript? ● ReasonML ○ Rewrote our PowerPaste plugin for TinyMCE, replacing complicated regex with strong types and pattern matching ○ Syntax is similar to JS so easy to learn ● PureScript: ○ Used for some of our internal prototypes ○ Side-effect tracking speeds up code reviews and training, and optics improve runtime efficiency ○ Lots of FP features we don’t need on the client side ● Neither is an easy drop-in replacement for the editor code
- 11. Why TypeScript? ● Unfortunately... ○ No pattern matching ● But... ○ Vanilla JS is valid TypeScript ○ Due to already using a module system with strict syntax, converting our code could be mostly done with a script ○ Types! ○ Gradual types!
- 12. What is Gradual Typing? ● Process of starting with minimal types and adding more over time ● Requires a mix of static and dynamic types - like TypeScript ● Requires minimal initial work ● Can invest as however much time and developers as you like in increasing types each week/month/etc. ● Can start with simple types then increase complexity as you learn and adjust to the type system ● Low initial cost of adding types, flexible ongoing cost
- 13. The How
- 14. Conversion Process for each Library 1. Run script to convert modules to ES2015 and TypeScript 2. Manually check the script converted each file correctly 3. Update with changes from already-converted dependencies 4. Fix problems found by TypeScript 5. Run tests and fix any problems 6. Update build process
- 15. The Conversion Script 1. Transpile all modules using an Abstract Syntax Tree (AST) a. Change import syntax b. Pull code from inside the wrapping function to the root level c. Change export syntax and apply export default <any> 2. Move files and rename to .ts 3. Copy templates for configuration files 4. Generate Main.ts from api folder 5. Update package.json and .gitignore
- 16. Module Conversion Example define( 'Example Module', [ ‘example.Dependency' ], function (Dependency) { var bar = 0; var foo = function () { … }; return { foo, bar }; } ); import { Dependency } from 'example'; var bar = 0; var foo = function () { … }; export default <any> { foo, bar };
- 17. Adding Types ● The script doesn’t add types ● We started adding types to libraries, starting with the libraries most commonly depended on ● We add types with each PR ○ New code must be at least partially typed ○ “If you touch a file, type it” was the rule for a while ● Now, many of our libraries are at least partially typed
- 18. Benefits of TypeScript and Gradual Typing
- 19. Types ● Gained type-based tools e.g. find all usages and automated refactoring ● Compile time type checking ● Even just adding simple types caught some serious bugs ● Typing our most common dependencies caught many more
- 20. Example Bugs Incorrect function argument list length Missing comment resulted in accidental global
- 21. Gradual Typing ● Most of the team was able to go back to normal development work quickly ○ Sometimes devs take a couple days to type a library ○ We add some types with most PRs ● Conversion script used export default <any> so minimal errors and developer work upfront ● Gradually removing any from modules allowed us to type modules slowly and methodically
- 22. Increased Readability ● Types make it easier to follow code ○ Especially in our giant projects where variables sometimes get passed around a lot ● Types make it easier to debug code ○ Type errors are generally nicely worded ○ Sometimes easier to match weird values to types, then track down which variable or function they come from ● Types make it easier for new developers to get up to speed and learn the code
- 23. Lessons Learned
- 24. Convert Common Libraries First ● Because of our modular architecture we have a lot of libraries, and many depend on other libraries ● We mostly converted the most common dependencies first, so we could cascade their types and bug fixes through other libraries ● But some other libraries were done at the same time, and they have duplicate type definitions, clashing types, etc.
- 25. Gradually Introduce Compiler Features A couple times we enabled compiler features without testing them properly on every library...
- 26. Write a Script to Test Everything ● Write a script that does every kind of check it can - types, linting, build, tests, etc. ● Run it when you make a big change ● Ours is called the Really Cool ScriptTM ○ Contains a list of all our libraries ○ Clones each one, installs its dependencies, runs tsc and npm test
- 27. Developer Adjustments ● Discuss changes in coding style, tools and processes ● Communicate changes ● Script pre-commit or pre-build type checks to help developers remember what to check
- 28. Continuing Work
- 29. Continuing Work ● More types! ● Enabling more compiler features ● Building a type definition file for TinyMCE’s editor API
- 30. Check out TinyMCE ● TinyMCE is open source ○ https://github.com/tinymce/tinymce ○ Contains TS files and most config and build files ○ Master branch is TinyMCE 4 ○ 5.x branch for TinyMCE 5 ● Found a bug? Win some swag! https://go.tiny.cloud/blog/tinymce-5-developer-challenge/
- 31. Thank you!
- 32. Questions?