Wanted: Best Practices for Collaborative Translation
- 1. Wanted: Best Practices for Collaborative Translation Alain Désilets National Research Council of Canada alain.desilets@nrc-cnrc.gc.ca With support from:
- 2. A (Very) Brief History of Collaborative Translation Circa 2005: “Wikis, what's that?” Circa 2006: “I know about Wikipedia, but I hear it’s garbage because anyone can write anything on it.” Circa 2007: “You know, I have been to Wikipedia a couple of times and was pleasantly surprised by the quality of what I found there.” Circa 2008: “Actually, this wiki stuff is really interesting. Now, I routinely use Wikipedia in my work, and although I am cautious with it, I find it useful. I have a sense that this wiki/collaborative stuff will have a wider impact for translation, but I’m not quite sure how.”
- 3. A (Very) Brief History of Collaborative Translation (2) Circa 2009: “Whoa! Translation Crowdsourcing is going to put me out of a job!” Circa 2010: “Well, I guess that was a storm in a teacup. Crowdsourcing will be used in some specific and limited contexts, but it won't take over. Maybe this is more an opportunity for us than a threat…” Circa 2011: “Hum… getting this collaborative translation stuff to work right is hard and confusing.”
- 4. Talk Outline The different “flavors” of Collaborative Translation Common issues and Challenges in Collaborative Translation Capturing Collaborative Translation best-practices in the form of Design Patterns
- 5. The different flavors of Collaborative Translation
- 7. Definition Collaborative Translation is the use of any open online collaboration technology or process, in order to help with translation tasks, or tasks related to translation (ex: terminology).
- 8. Available in the following flavors… • Translation crowdsourcing • Collaborative terminology resources • Translation memory sharing • Online marketplaces for translators • Agile translation teamware • Post-editing by the crowd
- 9. Translation Crowdsourcing Mechanical-Turk-like systems to support the translation of content by large crowds of mostly amateurs, through an open-call process. By far the most talked about collaborative translation approach • Software user interface (Facebook, Adobe, Symantec, Firefox) • Technical documentation (Adobe, Symantec) • Transcripts of videos of an “inspirational” nature (TED Talks, Adobe TV) • Humanitarian aid content (Translators without Borders, Kiva.org, Haiti Earthquake Mission) • Large scale collection of linguistic data for research purposes or machine translation training (NAACL Workshop on Crowdsourcing).
- 10. Collaborative terminology resources Wikipedia-like platforms for the creation and maintenance of large terminology resources by a crowd of translators, terminologists, domain experts, and even general members of the public. • Wikipedia • Wiktionary • ProZ’s Kudoz forum • Urban Dictionary • TermWiki.com • TikiWiki • Reverso dictionary
- 11. Translation memory sharing Platforms for large scale pooling and sharing of multilingual parallel corpora between organizations and individuals. • TAUS Data Association • MyMemory • Google Translator Toolkit • WeBiText Often, collaboration is “implicit”, for example, in the case of WeBiText.
- 12. Online marketplaces for translators eBay-like disintermediated environments for connecting customers and translators directly, with minimal intervention by a middle man. • ProZ.com • TranslatorsCafe • Translated.net Collaborative aspects comes from things like “open call sourcing” and reputation management based on community assessment.
- 13. Agile translation teamware Wiki-like systems and processes that allow multidisciplinary teams of professionals (translators, terminologists, domain experts, revisers, managers) to collaborate on large translation projects, using an agile, grassroots, parallelized process instead of the more top-down, assembly-line approach found in most translation workflow systems. No specific software or site, but many case studies describing how to implement this approach, using general purpose collaboration tools like wikis, BaseCamp. • Beninatto & De Palma, 2008, • Calvert, 2008 • Yahaya, 2008 Some translation workflow systems starting to market themselves as being “collaborative”
- 14. Post-editing by the crowd Systems allowing a large crowd of mostly amateurs to correct the output of machine translations systems, often with the aim of improving the system’s accuracy. • Asia Online’s Wikipedia translation project • Google Translate allows anonymous users to correct the outputs produced by the systems • Likewise for Microsoft’s Bing Translator
- 15. Is this REALLY New? Weren’t Terminology Databases, Translation Memories and Translation Workflow Systems already collaborative? • Yes, but… • … Collaborative Translation is about using these kinds of groupware technologies in the context of much larger groups or communities, where people have fewer reasons to trust each other a-priori. It’s one thing to open yourself to collaboration with colleagues and customers. It’s quite another thing to open yourself to the whole world.
- 16. Common issues and Challenges in Collaborative Translation
- 17. This is NOT easy Choosing a flavor and tailoring it to your needs is still somewhat of a black art, guided by trial and error. There are lots of important and poorly understood issues that arise, many of which are common to most flavors: • Alignment with business goals • Quality control • Crowd motivation • Proper role of professionals
- 18. Alignment with business goals Why are you doing this in the first place? Which flavor can deliver what you want? The actual benefit you get from a given flavor is not necessarily what you think! Translation crowdsourcing • Reduce cost? – Yes, but not the biggest benefit. • Decrease lead time? – Definitely. • Translation more in-tune with target audience’s idiosyncrasies? -- Also • Most importantly: Increase brand loyalty by engaging end-users as co-creators of products, instead of passive consumers.
- 19. Quality Control How to control quality when you open yourself to contributions from a potential large group of “outsiders”? Many ways: • Screen contributors before letting them in (ex: Translators Without Border, Kiva.org). • Have members of the community vote on the quality of each other’s work (ex: Facebook, Translated.net). • Have in-house professionals revise the work done by the community (ex: Facebook).
- 20. Quality Control (2) Do not assume that quality of community-produced content will be lower. For instance, Wikipedia provably measures up to professionally produced encyclopedia like Britannica (English) and Brockhaus (German). Quality issues tend to iron themselves out provided that you attract a sufficient large number of the right people Wisdom of crowd effects works surprisingly well when the following conditions are met: • Diversity • Independence • Aggregation
- 21. Crowd motivation If you are to attract and retain enough of the “right people” you need to understand why thy might contribute. • Mandated by management (ex: Agile Translation teamware) • Emotional bond with the content (ex: Facebook, and surprisingly, Adobe) • Prestige of the content (ex: TEDTalks) • Wanting to do good (ex: Translators Without Border, Kiva, Haiti Earthquake Mission, Data collection for scientific research) • Pride in one’s native language (ex: Data collection for R&D in MT for small density languages) • Trying to perfect second language skills • Trying to make a go at professional translation career (ex: Kiva.org) • And in some cases, $$$ – Will this be the dominant scenario? – How to set compensation high enough to attract good contributors, but not so high that it interferes with more intrinsic motivations, or attracts people out to game the system.
- 22. Role of professionals Some flavors of CT are designed specifically for professionals (ex: Agile translation teamware, Online translator marketplaces). But some (e.g. Translation crowdsourcing), tend to de-emphasize their role. When should professionals be involved, and what should be their role? • Revise work done by amateurs? – Focus on more challenging aspects of translation like terminology, style, fluidity? • Manage and coach the crowd? • Focus on more mission-critical and hard to translate content? Translation Crowdsourcing may actually increase the size of the pie, by making it possible to tackle content and/or small languages that would otherwise not have been dealt with anyway.
- 23. Capturing Collaborative Translation best-practices in the form of Design Patterns
- 24. Wanted: Best- Practices Collaborative Translations presents practitioners with a varied and complex envelopes of different approaches and technologies. Selecting a flavor and tuning it to meet your needs is complex. We need some sort of concise, easy-to-consult repository of best- practices for that field. We propose a way to collaboratively create such a repository a community, in the form of a design patterns language.
- 26. About Design Patterns A format for describing a common solution to a common problem in a given field Originally used in Architecture, but since then adopted in other fields such as Software Engineering, Education, etc.
- 27. Design Patterns Example Publish Contributions Rapidly Context This pattern is useful for motivating contributors in any collaborative translation context, but it is particularly useful in translation crowdsourcing scenarios. Problem Contributors are often motivated by a desire to have a positive impact on the community they are participating in. However, they cannot achieve this sense of being useful, if their contributions do not become available to the rest of the community in a reasonable amount of time. Solution Therefore, minimize the delay between the moment when a member of the community contributes to the site, and the moment where it becomes publicly available to the rest of the community. Ideally, the contribution should become visible to the rest of the community as soon as the user clicks on the Save button.
- 28. Design Patterns Example (2) Related patterns – Point System is another way for a contributor to get a sense of how useful he has been to the community. – Campaign Progress Gauge is another practice which allows members of the community to see the positive impact of their actions. The main difference is that it operates more at a community/project level rather than at a individual/contribution level. Real-life examples – At Facebook, translations become available in a matter of hours. – In the context of software localization by the crowd, Adobe makes a conscious effort to wrap the community's translations into every new releases of the product.
- 29. TAUS Roundtable on Collaborative Translation Wiki “Barn Raising” workshop held on October 12th, 2011 at Localization World in Santa Clara. 12 practitioners • One third with hands on experience of CT (NRC, Adobe, Symantec, Kiva, World Wide Lexicon) • Two thirds with no experience, but a strong interest in trying it (In Every Language, MemSource, Firma 8, SPIL Games) Talks by the experienced users about what worked and didn’t. Followed by brainstorming of what the recurring best-practices seem to be.
- 30. The Best-Practices End result: => 50+ best practices organized into 6 themes Planning and Scoping Translation as User Engagement, Align Stakeholder Expectations, Early and Continuous Clarification of Translator Expectations, Backup Plan, Project, Check Points, Appoint Initial Community Manager, Clear Objectives, Identify Compatible Content Community Motivation Campaign Progress Gauge, Contributor Recognition, Leader Board, Official Certificate, Point System, Offer Double Points, Hand-Out Unique Branded, Products, Contributor of the Month, Grant Special Access Rights, Playful Casual Translation, Campaign, Publish Contributions Rapidly, Playful Competition Among Contributors
- 31. The Best-Practices (2) Quality Content-Specific Testing, Entry Exam, Peer Review, Automatic Reputation Management, Random Spot-Checking, Revision Crowdsourcing, Users as Translators, Voting, Transparent Quality Level, Publish then Revise Contributor Career Path Flexible Contributor Career Path, Lurker to Contributor Transition, Anonymous Translation, Find the Leaders, Support Variable Levels of Involvement, Community Manager, Content Prioritizer Right Sizing Appropriate Chunk Size, Community-Appropriate Project Size, Break Up Crowd Into Teams, Require Minimal Involvement Level, Keep the Crowd Small, Volunteer Team Leaders
- 32. The Best-Practices (3) Tools and Processes Hint at Content Priority, First In, First Out, Task Self Selection, Layered Fallbacks, Official Linguistic Resources, Automatic Suggestions, Provide, Context, In-Place Translation, Community Forum, Analytics for Content Prioritization, Simplicity First, Good Examples of Contributions, Encourage Self, Set Deadlines
- 33. Some Observations The bulk of practices relate to Translation Crowdsourcing. => We need to spend more time capturing practices for other flavors of Collaborative Translation The bulk of the practices so far are not specific to translation. • They would be useful in the context of crowdsourcing efforts in any domain. • Maybe all we need is to codify and/or learn about the best practices for crowdsourcing in general? The more similar two organizations are, the more similar their practices will be (ex: Kiva and TWB, versus Kiva and Adobe).
- 34. Conclusion
- 35. Conclusion Collaborative Translation presents practitioners with a very large and varied set of tools and processes. Choosing a particular flavor of CT and tailoring to meet one’s needs can be a daunting task. We need a concise, easy to consult, modular compendium of current best practices in that area. We have started building such a compendium in the form of a wiki site (www.collaborative-translation-patterns.com) which captures best practices in the form of design patterns. We invite every one in this room to contribute to it if they can.
- 36. Thank You!