New Atlanta / MySpace Keynote at CFUNITED-05
At the Friday keynote session at CFUNITED-05, Peter Amiri, Director of Technical Operations for MySpace.com, gave an informative and entertaining presentation about the history and growth of MySpace, and the reasons why the site is being migrated to BlueDragon.NET. For those of you who couldn't attend, here's a brief summary of the events that led to their decision to use BlueDragon.NET to power the world's fifth largest (and growing!) web site:
1. MySpace.com went live in September 2003 running on Windows 2000 Server (IIS 5.0), Microsoft SQL Server 2000, and ColdFusion 5 Professional. As early as December 2003 they realized that this infrastructure was not going to scale as they needed, and identified their use of ODBC for database access as the critical bottleneck.
2. They decided to upgrade the site to ColdFusion MX, but immediately ran into severe problems with performance and reliability of the JDBC database drivers for Microsoft SQL Server that are bundled with CFMX. Ultimately, CFMX was eliminated as an upgrade path.
3. With the elimination of CFMX as a viable upgrade alternative, as Peter says, "everything was on the table." They considered the following: replacing Microsoft SQL Server with Oracle; moving the web front-end from Windows/IIS to Linux/Apache; rewriting the entire web site in something other than CFML; using alternate JDBC drivers for CFMX; and, switching to BlueDragon Server JX.
4. After examining each alternative in turn, by the late summer of 2004, MySpace had concluded that switching to BlueDragon Server JX was the best choice. (BlueDragon Server JX is a Java/J2EE-based version of BlueDragon. It includes a JDBC driver for Microsoft SQL Server--JTurbo--that is developed and supported by New Atlanta, unlike CFMX, which bundles a third-party JDBC driver. Therefore, BlueDragon Server JX doesn't exhibit the same JDBC-related performance and reliability problems as CFMX).
5. However, by the time the decision was made to use BlueDragon Server JX, several things had happened: MySpace realized they were a Microsoft shop and were likely to remain one; they began to understand the performance and reliability enhancements available by upgrading to Windows Server 2003 and IIS 6.0; they realized that improvements in performance and reliability could be achieved by using native ADO.NET database drivers instead of JDBC drivers; and, some of their CFML developers had begun experimenting with ASP.NET and they were evolving into a hybrid CFML/ASP.NET development shop.
6. All of the items above pointed towards using BlueDragon.NET instead of BlueDragon Server JX; so in the fall of 2004, MySpace began a technical evaluation of BlueDragon.NET, which was then in beta testing. There were three technical "proof points" that needed to be met before MySpace would make the decision to migrate to BlueDragon.NET: (a) CFML Compatibility; (b) Performance; and, (c) Reliability.
7. CFML Compatibility was the first, and in many ways the easiest technical hurdle to cross. Most of MySpace's CFML applications simply ran unmodified on BlueDragon.NET. Some needed modifications that are similar to changes that would be needed in an upgrade from CF5 to CFMX, and others needed minor modifications that are unique to BlueDragon.NET; all of the latter are documented in the BlueDragon CFML Compatibility Guide, which allowed them to be made quickly and with minimal effort.
8. Performance and Reliability of BlueDragon.NET were evaluated during live production testing during February and March 2005. That is, we took one of the servers from MySpace's production server farm, converted it to BlueDragon.NET, and measured its performance and reliability versus the existing CF5 servers while serving live production traffic. The results were outstanding. (The final release of BlueDragon.NET was actually delayed from December 2004 until March 2005 while we completed this testing with MySpace).
The end result of all of this is that the MySpace server farm is currently in the process of being upgraded from CF5 to BlueDragon.NET. Click here to download Peter's entire presentation, along with my introduction (for those of you interested in more details about the BlueDragon Profiler presented by Charlie, I've asked him to post a separate entry on his blog about that).
Peter never stated how they develop currently, and many left with the impression that they still develop in this haphazard way. I've read a few blog posts that spoke ill of the keynote (and of New Atlanta / BlueDragon ) because of this.
While, I wouldn't recommend the "Build and fix" model of software development, I can't blame NA for such development practices from MySpace.com. I think a key component these people are forgetting is that MySpace took the same code base and it was running 50% better on BlueDragon.NET than on CF5.
Is there any elaboration that NA can provide on this topic?
I think that as important as the simple speed increase (which could have happened with an MX 6.1 or 7 upgrade) is that how well it has consistently performed with the BD upgrade, despite the increased loads. That speaks volumes for Blue Dragon - I wonder how much better the site would have performed with a *nix/apache and the J2EE version of BD.
regards,
larry
I think Peter would agree with you that "build and fix" isn't the right model for development, and that it doesn't reflect what they're doing today. His comments referred to the "early days" and "lessons learned." My experience working with them over the past few months is that they have a fairly well-structured and disciplined development process.
Any criticism of MySpace on this point is unwarranted, and is being made by people who simply misunderstood the presentation.