Code review on the Web - Google
1 like645 views
The document discusses Google's code review process and the development of Mondrian, a web-based tool for facilitating code reviews. Key points: - Google uses a command-line based code review process where developers email patches and comments back and forth. - Mondrian was created to address issues with this process like VPN access and tracking changes across revisions. It uses a web interface to show diffs and collect inline comments. - Mondrian stores code snapshots, comments, and metadata in Google's Bigtable database. It integrates with the company's Perforce source control and email systems to automate the code review workflow.
Code review on the Web - Google
- 1. Mondrian: Code Review on the Web Nov 30, 2006 Open Source Speakers Series Guido van Rossum guido@google.com
- 2. 11/30/06 2 What is Code Review? • When one developer writes code, another developer is asked to review that code • A careful line-by-line critique • Happens in a non-threatening context • Goal is cooperation, not fault-finding • Often an integral part of coding process • Involuntary code review happens when debugging someone else's broken code – Not so good; emotions may flare
- 3. 11/30/06 3 Benefits of Code Review • Two pairs of eyes catch more bugs (duh) – Catch 'em early to save hours of debugging • Enforce coding standards, style guides – Keep overall readability & code quality high • Mentoring of new developers – Learn from mistakes without breaking stuff • Establish trust relationships – Prepare for more delegation • A good alternative to pair programming
- 4. 11/30/06 4 Code Review in Open Source • Author & reviewer on separate computers • Author invokes "diff -u" to create patch file • Author mails patch file to reviewer – or uploads to e.g. SourceForge patch manager • Reviewer uses "patch" to recreate the files • They email back-and-forth a few times • Finally, reviewer submits into svn/cvs/etc – patch author often has no privileges (yet) • process helps "vetting" new developers
- 5. 11/30/06 5 Google's Development Process • (In theory :-) • Single company-wide Perforce (p4) depot – No developer branches • Company-wide NFS – Developers workspaces accessibly by others • All code reviewed before submission – All code-review email logged for auditors • Wrapper "g4" implements superset of "p4" – Originally a shell script; now a Python program
- 6. 11/30/06 6 Google's Code Review Process • All command-line and email based: 1. Author edits changes in workspace, tests etc. 2. Author send email to reviewer (a tool helps) 3. Reviewer views the diff (another tool helps) 4. Reviewer sends mail back (regular mail reply) 5. Rinse and repeat (using regular mail replies) 6. When reviewer replies "lgtm", author submits • lgtm = looks good to me
- 7. 11/30/06 7 Original, Minimal Tools • "g4 change" invoked by author defines set of files involved, adding comments • "g4 mail" sends form email to reviewer – (also integrated with g4 change call) • "g4 diff" invoked by reviewer diffs files straight out of author's workspace – "tkdiff" X11-based side-by-side differ • "g4 reply" synthesizes reply mail – optional; not used very much
- 8. 11/30/06 8 Problems With Old Process • VPN usage (reviewers working from home) – can’t use tkdiff, or it is too slow • Line numbers change as code evolves – line-oriented comments get out of sync • Can't diff between pre-submit revisions – reviewer must keep manual track of evolution • Email can get lost in inbox; no work flow – reviewer: what do I still need to review? – author: is my code reviewed?
- 9. 11/30/06 9 Enter Mondrian • Web-server based – Addresses VPN issues – Snapshots all previously reviewed versions • Shows side-by-side diff (like tkdiff) • Lets you add in-line comments – Collect comments into a single message • Also receives and organizes email – Not all parties need to use Mondrian
- 10. 11/30/06 10 Live Demo • Show dashboard – explain categories; hover over CL, flags, users, description • Show someone else’s dashboard • Show a group’s TO DO list (build-grouplet) • Show a CL view – browse some comments (expand/collapse) – follow links to Sourcerer, p4web – show Request code review form; Upload snapshot form; Approve form • Show a file view – use n/p to navigate diff chunks – navigate between files – show intra-region diffs – add, edit, discard, reply to in-line comments • Show Review and publish form
- 11. 11/30/06 11 How It Started • As a Noogler, I needed a starter project • Someone proposed a code review web app • Decided to freshen up my web knowledge • Learned some Django on the way • Adopted WSGI (PEP 333) while I was at it • Learned about Google's Bigtable and p4lib • Early prototype was immediately hot!
- 12. 11/30/06 12 How It Works (Overview) • Bigtable (a Googly db) used to store: – Change metadata (description, list of files, …) – Comments (entered on web or received email) – File snapshots taken from user workspaces – Per-user data (active changes, last view dates) • Web server talks to user, Bigtable, p4 – Also to NFS, SSH when fetching snapshots • Mail server processes incoming email – Storing them as comments
- 13. 11/30/06 13 Snapshots • Copies of files taken from user's directory – Updated whenever change view visited – Solve two problems with the old process/tools • What if the files aren't on NFS? – Linux: use SSH/SCP to user's workstation – Others: author uploads plain old patch • Snapshots and comments are kept forever – To satisfy auditors – To track extended history of changes
- 14. 11/30/06 14 Web User Interface • Mostly pretty traditional HTML with CSS – Django templates, wsgiref server (all Python) • Some JavaScript for UI niceties – Expand, collapse comments – Show unified diffs in-line in CL view (AJAX) • Full file view uses more AJAX: – Keyboard navigation – Inline draft comment editing • Avoids full-page refresh
- 15. 11/30/06 15 Editing Inline Draft Comments • Each line in the diff is two table rows: – <tr><td> 123 textA</td><td> 127 textB</td></tr> – <tr><td>…</td> <td>…</td> </tr> – First row displays left & right diff text – Next row (hidden) used for inline comments – Each <td> has a unique id {old,new}+lineno • Double click on first row calls JavaScript – Inserts form into next row – “Submit” sends form data to server – Server sends back HTML to replace form
- 16. 11/30/06 16 Mail Server • Receives every email sent to review logs • Ignores messages sent by Mondrian itself • Parses subject looking for revision # • Inserts message as comment into Bigtable • Also updates reviewers’ TO DO queues • Implemented using standard smtpd.py
- 17. 11/30/06 17 Performance • Mostly fast enough, on just one server • Most of the work is done elsewhere: – Bigtable+GFS server infrastructure – Perforce server – In the browser (rendering reams of HTML) • What's slow: – Contention for Perforce – Generating HTML for loaded dashboard – Diffing huge files (uses Python's difflib)