Technology of the New News Workflow

Editor's Notes

• #3: This site is built on WordPress MU, with about 300 contributors, each with their own blog (“site”). If you’re familiar with WordPress MU, you’ll understand that out of the box, no plugins, each blog/site stands apart. You can’t search across blogs, you can’t query posts across blogs. It’s great for each blog – nice fast pageloads. But to integrate the network installation into one presence, you need to build on this great WordPress platform. I’m going to walk you through some of the things True/Slant can do, including editorial workflows – not everything you see is automated. And then I’ll get into techniques and methods to combine multiple blogs into something bigger than the sum of its parts.
• #4: This is one blog. This particular one by Matt Taibbi who also writes for Rolling Stone. The functionality here is pretty simple, just a few sidebar pieces to show content from the rest of the site, and what we call an Activity Feed showing what the contributor has done in the past few weeks. We started this before BuddyPress was really available, and haven’t had a chance to check it out – in case you were wondering if there’s any BPress in True/Slant. Not yet…
• #5: Here’s another contributor. This is the main page – very similar, and a common theme.
• #6: And here’s another. They are all the same theme, just different content and writers. So why not stop here? Because it’s more than just one writer, just like any newspaper is more than just one column.
• #7: Here are some of the bits we put on the home page. With 300 contributors, there are a lot of viewpoints. We have basically one full-time editor picking content to feature. And to keep him productive, we’ve created a few tools. But some things are automated.
• #8: Many users arrive on our site at a specific post. Our chance to engage them starts on the single post page. These sidebar modules will attract users to other posts on the site.
• #9: Here’s our footer. The most popular posts, recent contributors, and the last few posts and comments wind up here. This is all automatic.
• #10: Rather than just posts from this contributor, we feature related posts from all blogs. We also have what we call topic streams. They aren’t exactly categories or tags; a post can be assigned to multiple topics; this exposes them to the site-wide topic streams.
• #12: We call these Live Streams, but they’re more like topics. Posts can be assigned to multiple topics, based on - which category they are in in their home blog, or - an editor’s manual selection per post (either forcing a post into a topic, or possibly excluding it) Inside the streams, we can have posts and various status updates – such as this bit that lists recently recommended posts in the stream. In the sidebar, we can feature a post. All the editor needs to do is hover over the story, and click a link.
• #13: This is how an editor manually overrides the automatic topic assignment. Here, the contributor’s Category placed this post into the World topic, but the editor manually added the post to the Haiti Earthquake topic as well.
• #14: A user can follow contributors and topics, once you follow them, you can receive email alerts, and this handy drop-down shows you the latest updates.
• #15: We modularized our code, so that we can hook in additional features later. When we added Facebook Share, and Email alerts, we did not have to modify the Follow code or change the Follow workflow for the user. If a user is not connected to Facebook, the Facebook part doesn’t appear.
• #16: Everything a user does on the site can be displayed to other users.
• #17: We implemented this with a special master table, listing all events. We indexed it and made some keys unique, so that there’s only one record, with the latest timestamp.
• #18: We created several editorial tools, which add to the automation. The idea is to get the best content exposed, We have data to help us – eg post views, trends, comments. But don’t make it hard for the editor, And certainly don’t make anyone jump through hoops We use a lot of AJAX, which is only exposed to privileged users
• #19: This is one of the first, and simplest customizations we did.
• #20: The only thing you need to do here is increment comment_karma in the comment table by 1, and use that to set a class on the comment. Also, alerts hooks into the comment being called out action, so we can send a note to the original commenter.
• #21: We also enabled an interface on the backend to curate comments from the comment list; green here indicates a called-out comment.
• #22: Our home page needed a bit fancier set of tools, to create and arrange posts in panels.
• #23: This network header editor allows editors to prioritize panels, hide them, change or re-crop the thumbnails and headlines, etc.
• #24: To make search results fast across all blogs, we Added a new column to our shadow posts and comments table with just the searchable text of the post Made a fulltext index of that Intercepted the standard search query and queried our shadow table instead We also derive topics and contributors to provide refinement tools and links in the sidebar
• #25: We also give contributors some tools – this is my dashboard Blog stats, top posts, referers all via internal stats (which we use in many other ways)
• #26: Needs work
• #29: One big issue we faced early was combining the multiple blogs into one. Does it make sense to use RSS feeds if the database is right there? What other information do we need? And – we’d still need to pull 300 feeds! Forget that!
• #30: This is a normal, single “site” or blog.
• #31: When you have two blog sites, you have to use a JOIN to query both tables at the same time (eg to find the latest post)
• #32: When you have 300 blogs/sites, a normalized database is hard to deal with for reads. It would kill performance.
• #33: Solution: de-normalize the data; we hooked into the post update and comment update actions and copied the data into this shadow table. Adding a blog_id column.
• #34: No point in using this for each blog; WordPress handles things fine.