Optimize Optimize Optimize!

One of the problems many websites run into as they grow up is that they were not built very well to scale.  Classic example, twitter. Over the past year they’ve gotten much better but for several years fail whales were flying left and right.  While there are no whales here at Social Blade being thrown at you I have been constantly working behind the scenes changing the way things are done to try to make things go faster.

I knew right from the start that since I am not a DBA the database structure I was building to store all the user info was not the best.  I mean it worked, and worked pretty well for a small user base but as I mentioned in yesterday’s blog Social Blade now tracks over 100k YouTube channels and from a data structure perspective that’s no small task.  One of the problems with the structure I created was I stored everything in one massive table and updating that table to get the ranks sorted for people has taken longer and longer as the database has grown.  What once was a minute long task had grown to several hours.  If you ever visited the site in the morning from around 6am-9am ET and noticed long load times this is why.

    Graph showing the systerm load on the server. The higher up the graph goes the slower the site gets. Major improvement is shown after Monday. Wed night still has a small spike which shows my work here is not done, but its much better!
Graph showing the systerm load on the server. The higher up the graph goes the slower the site gets. Major improvement is shown after Monday. Wed night still has a small spike which shows my work here is not done, but its much better!

Today I’m happy to announce that at least that problem has been resolved! The past few days I’ve updated the way that data is stored and we’re now back to a really quick update which doesn’t slow the system down!  Are there still many things I need to improve? Yes.  As more and more of you keep visiting its a constant learning experience for me which is great.  So in short the reason I wanted to share all of this with you was to let you know that in between the times of adding new features there still is work going on in the background to improve the stability of the site and keep SocialBlade as the best place to go to track your youtube stats!