Caching, Indexes, and Laravel: Surviving Scale with a WordPress Core over 8 years

In 2017, I launched a tiny WordPress-powered podcast service for my local church. Eight years and tens of thousands of users later, it’s still alive—and still evolving. Here’s the messy, honest journey of scaling it without rewriting everything from scratch.

Fun fact: The podcast shows hosted on this platform can be found on both iTunes and Spotify.

Since we needed something up and running quickly, we put together a few WordPress plugins to get things up and running. I even wrote a couple myself to enable some things that the plugin did not provide. The main plugin (developed by Castos) can be found here. I will share how I set things up in a later post.

Initial Setup with WordPress

The service was first hosted on shared hosting at TMD. It was very responsive since there were just a few people using it until the numbers grew and things started slowing down. I moved the service to cloud hosting at TMD. This was still shared hosting but with a bit more resources. As expected, it scaled for a very short time and things started to slow down once again.

From Shared Hosting to VPS

I moved the application over to a Virtual Private Server with a bit more resources. The web server (Apache) and database server (MySQL) were all running on the same server at the time. This gave the system quite a boost but with rapid user growth, it did not stay fast for long. Checking the bandwidth usage on the server made me realize streaming the media directly from the VPS was also becoming a performance bottleneck. It was time to host the media files somewhere else.

Offloading Media to Digital Ocean Spaces

There were a lot of media files to be moved (audios and images) at the time but a plugin helped reduce this stress. The media files were successfully moved to Digital Ocean spaces through a GUI interface right in my WordPress admin backend. After this a lot of the load left the server and it stayed performant for quite some time. But again, you will have to always deal with a growing user base. I noticed from checking CPU usage that it was easily reaching 100%. At this point I was not sure what to do because there were no observability tools to really tell what was going on.

Bottlenecks and Database Splits

I provisioned a new server to host just the database to remove this load from the server. They were kept in the same zone and used internal private addresses to ensure communication between them was fast and easy. This did not seem to do much but gave a bit of a speed boost. I was using an analytics plugin to track some metrics on the various podcast shows. It's pages started to get slower and I realized this plugin had a bad query which was really slowing things down a lot. I used a plugin to track the queries that were being made and found the time it took to complete and found the bad query. I created a composite index and this resolved the issue. Wasn't a big win but a bottleneck was removed.

Real Gains from Caching

All podcast episodes when being viewed on the site had to be generating all the time. These pages were not changing often so it was a good idea to cache them to reduce the expensive processing. This is when I saw real performance gains. The site became extremely response once again. Very similar to what it was like when we first started it in 2017. The caching plugin could go through all the pages and cache them in advance. This was perfect at this point and it did not feel like we will ever have to optimize again. Guess what, things started to slow down once again.

Dealing with Plugin Bloat

At this point, I had used quite a number of plugins to add some features which seemed like nice to haves. As it is known in the WordPress world, plugins are useful but can be a disaster sometimes. At this point I had integrated DataDog with my WordPress application and I could see the high numbers of queries per request. Every plugin was just bombarding my database with a lot of queries. I removed all the plugins that were not really needed. Again, not a major win but it was a necessary step.

User Behavior Surprises

The most visited page of every podcast is the podcast feed link. Almost every swipe to refresh the podcast client on mobile phones hits the endpoint. Web crawlers keep hitting this page to get the current episodes. Some podcast shows show just a few current episodes of the podcast and prefers that the listener goes to the podcast site to find the other episodes. I tried this and this drastically reduced the load on my server. There were over hundreds of episodes so generating the feed was quite expensive and was one of the things that was leading to extra load on the server. A few hours after doing this, I got a lot of complaints about users searching for episodes and know longer finding them. I had inadvertently trained users to rely solely on the podcast apps instead of visiting the website. I had to revert to showing all the episodes in the podcast feed.

I tried caching the feed which was really tough to do. The caching plugin did not seem to be doing a very good job at it. Caching the feed did not work very well and therefore I had to ignore this option.

Laravel to the rescue 🚀

At this point I felt like I had done all I could and the podcast service was still struggling to serve the growing number of users. I thought about rewriting the entire podcast platform but this seemed like a lot of work which I was not ready for. So I decided to use "microservice-ish" approach to deal with the situation.

I wrote a thin layer in Laravel which was going to be serve the podcast feeds (the most visited parts of the podcast service) while I left the WordPress application to continue serving as the podcast website and admin portal. The laravel layer works in a very simple way. It occasionally sends a request to the podcast service (WordPress) to get the podcast feeds and it caches them locally. This way, the WordPress podcast service only has to perform this expensive operation just a few times in a day. This significantly improved the service and has continued to scale very well till date.

The thin layer can be found here on GitHub: https://github.com/brytey2k/podcast-speedster

Hope this helps someone and if you also need to build a podcast application, I hope this helps you skip some of the issues I have had to deal with over this past 8 years.

TL;DR: Key Lessons from Scaling a Podcast Platform

• WordPress is fast to start, but scales poorly without planning.
• Moving media off the server made a big impact.
• Caching is critical—but doesn't fix everything.
• Laravel can play nicely with WordPress as a backend accelerator.