For those of you who have had difficulty accessing LeanBlog.org for the past two weeks or so, I apologize. It's been frustrating to me and I've gotten a number of emails from people who couldn't get the site to load.
I think I finally got to the root cause, with the help of tech support from GoDaddy.com. Things should be back to a stable, “site loads every time” state. Update: there are still some problems today, so maybe we don't have the root cause ID-ed.
I'll spare you all of the technical details, but will share a bit to help other WordPress bloggers who might run into the same problems. I'll also tie it to a core “Toyota Way” principle that I violated.
I switched from Blogger to WordPress late last year. WordPress offers many more features in an open-source platform. The number of features is both a strength and a weakness (as people can sometimes overload their site with tons of “plug ins” that add little value and slow down page loads dramatically).
I've tried to keep my site “lean” without a ton of extraneous plug-ins. But one extra “feature” that I turned on turned out to be the problem… we think.
I have been using one plug in called “TwitterTools” that, among other things, automatically generates a new ‘Tweet' when I publish a new blog post. After a while, I decided to try another feature that's shown below. The fact that it was labeled ” – Experimental -” was a red flag that I missed. Lesson learned.
The plug in was supposed to generated a weekly blog post that summarized my tweets for the week. I thought this might be helpful (be “value”?) as:
- Allows me to archive Tweets and links I sent out over time
- Allows readers to catch up on links that I sent out that weren't blog worthy (lots of good reading)
This plug in apparently caused a huge load on the database server as it ran throughout the week. Things would be worse in the early morning, right after my new posts are auto-published. The site often wouldn't load, showing database errors or not loading unless you hit “reload” many times – and sometimes not at all.
After GoDaddy thought the problem was their servers for the first week, that through us off the trail of the real root cause. I was basically blaming GoDaddy (since their said it was their problem). Then, when they decided it was NOT their problem, better root cause problem solved happened.
GoDaddy helped me track down that my first call to tech support was on April 5. They kept better logs of all of this than I did. Thanks to them for that. They helped me did into the server files to see what I turned on or change on or around April 5. With a little trial and error (and recognizing an “experimental” feature shouldn't have been used), we turned off that auto-tweet post. In hindsight, I was about to turn that feature off anyway, since the value seemed minimal.
I violated the Toyota Way principle #8:
Use only reliable, thoroughly tested technology that serves your people and processes.
Lesson learned. Read more about The Toyota Way in this great book: The Toyota Way by Jeff Liker.
In the course of the troubleshooting, GoDaddy recommended a site caching plug in that should make the site load faster than before. Basically, the cache means that the site doesn't dynamically re-create each blog page each time a person loads it (that constantly re-creation could be considered the waste of “overprocessing”?). Now, after a page is rendered the first time, the second visitors and beyond get served a static version of the page, meaning it loads faster (until it's updated with a new comment, for example).
So, for all of that hassle, my site is faster than before… and now that it's unbroken, it should load consistently like it did before April 5. I will be more cautious in the future about not breaking things. Organizations and people do this all the time – we are wowed by some new technology or feature and we rush it into place. Sometimes bad things happen… mine is just a silly blog… the consequences in healthcare can be far more serious.
Update: things were still slow and bad this morning… I disabled the entire Twitter Tools plug-in, not just the experimental feature. Although the plug-in was there and working fine before I turned on the experimental feature, I'll see how things work with the whole thing turned off.
In honor of fixing my blog, I'm sharing a favorite site: There, I Fixed It!: http://thereifixedit.com/
What do you think? Please scroll down (or click) to post a comment. Or please share the post with your thoughts on LinkedIn – and follow me or connect with me there.
Did you like this post? Make sure you don't miss a post or podcast — Subscribe to get notified about posts via email daily or weekly.
Check out my latest book, The Mistakes That Make Us: Cultivating a Culture of Learning and Innovation:
Those experimental bells and whistles are always tempting – proven or not. I think their potential and our curiousity gets the best of all of us, sometimes.
I’ve seen Principle #8 violated a number of times with electronic systems that have wreacked havoc and generated waste in the form of searching, errors, and waiting for YEARS – mostly because up-front cost was low and tempting. Thing is, as you point out – do we learn anything, and do we have the self-discipline to not make the same mistake again?
Good post, Mark, and I respect your humility – a sign of lean leadership.
Hi Mark,
Good luck with your research, things like this are really frustrating. I try not to use wordpress for anything other than it’s core function as a CMS. We use Ping.fm to push updates to Twitter, LinkedIn, and Facebook.
Regards,
Colby
.-= Colby Voorhees ´s last blog ..Riesling Rules at Pacific Rim =-.
You can use Loudtwitter to have your tweets sent to your blog en masse on a schedule…