"Linux Gazette...making Linux just a little more fun!"

Web Portals Bank on Open-Source Infrastructure

By Ned Lilly

Wireless Developer Network (WDN) and GeoCommunity say sayonara to the database big boys.

When a single company formed to operate two web portals for online communities, they turned to the biggest names in the business to build their technology infrastructure. Wireless Developer Network (WDN) for wireless communications professionals, and its sister site, GeoCommunity for geographic information specialists, used hosting services to license the use of Microsoft and Oracle respectively. The two sites were up and running in no time, offering virtual homes to thousands of professionals who needed industry news, software downloads, product reviews and live chats. But while the portals seemed a success on the surface, disaster lurked not far below.

Within months, WDN ran into performance problems with SQL Server and security issues with MS IIS. Oracle 8i worked like a dream for the GeoCommunity, but the licensing fees threatened to crush the small company. As the portal expanded capacity and users, it knew Oracle's aggressive pricing structures would cut even deeper into its slim profit margin. That's when the company's technical staff began to push a radical concept: ditch the big boys in favor of a single open-source technology platform for both web portals.

The web portals had to be able to serve web pages up 24/7 without any crashes or service interruptions. The developers wanted to go with Linux because their experiences told them it was a stable and reliable platform for the web. They also knew firsthand that Apache web servers were a superior product, faster, more scalable and easier to configure, and optimized for the Linux platform.

The technical staff had worked in both open-source and proprietary environments, and had come to believe open source was the more secure choice for web-based applications. In their view, open-source technologies grew up on the web, while most proprietary applications were later adapted to it. Throughout the open-source development process, developers drill down on security and performance issues for web applications. With hundreds of developers and users testing and tweaking the programs, security holes are often caught and corrected at an early stage of the development process.

WDN's leadership quickly bought into the idea. However, many managers believe that "nothing good can be free" and subscribe to the common myth that open-source products lack professional technical support. But this company's senior management team was quickly sold on their developers' enthusiasm and positive experiences with open source.

In fact, managers skittish about defying convention need only look around for evidence of the proliferation of open-source applications and tools in business. Linux powers an estimated 36 percent of Internet-connected servers today, while Apache web servers are on about 61 percent of public web sites, according to the Internet research firm, Netcraft. Industry analysts at Forrester Research recently identified open source as a powerful growing trend in business with potential to radically reshape the software industry by 2004.

Yet the database market, the core of web-based businesses today, still remains firmly in the grip of proprietary vendors such as Oracle, Microsoft and IBM. But in recent years, open-source databases such as PostgreSQL and MySQL have evolved to the point where they're beginning to compete with the propriety giants in performance and functionality. They're attracting skilled development and user communities, as well as enterprise business users across a wide range of industries, and are rising up to challenge the proprietary status quo in the competitive database market.

The heart of the portals

For WDN and GeoCommunity, the most difficult decision related to its technology infrastructure was its choice of the right database management system. The portals needed a system that was scalable and functional enough to handle thousands of visitors each month and power scores of dynamic applications, including e-commerce.

The portals tested two of the most widely used open source databases, MySQL and PostgreSQL. While MySQL was simple to configure and use, it lacked the transaction support and scalability that the company needed to run their highly interactive sites. MySQL has attracted a large user base, but the staff thought it seemed more suitable for lower-traffic web sites. The staff also ran rigorous tests on PostgreSQL, a heavy-duty object-relational database. It withstood the barrage of tests without flinching, supporting advanced features during heavy simulated transactions very well.

After downloading their selection of open-source applications, including the Red Hat Linux operating system, Apache web server and PostgreSQL database, the technical staff configured the system in less than a half hour. The portals run 12 servers, with PostgreSQL powering dynamic applications such as book sales, message boards and mailing lists. The new system was up and running in minutes, with no interruptions, and neither web portal has since crashed or lost data.

Confronting the FUD (fear, uncertainty and doubt) factor

Yet there are good reasons why open-source technologies were once the exclusive domain of skilled hackers and expert users. In the past, these applications purposefully lacked the bells and whistles of their proprietary competition and were difficult for the less advanced user even to install. Open-source programs have become much more user-friendly over time because independent developers have begun to pay greater attention to improving tools, additional features and perhaps most importantly, documentation.

For WDN and GeoCommunity, the decision to migrate from a proprietary to an open-source system was less difficult than for most traditional businesses. The web portals employ technical staff with experience in both environments. At every level, the company embraced the idea of adopting a more flexible, and less financially draining, open-source alternative. They understood the open-source development model and bought into its underlying philosophy. Just as importantly, they possessed the technical skills to confront many of the issues that could arise in an open-source platform. In fact, with access to their new system's code, they could now even modify their software's features to better fit the company's needs.

Many e-businesses like these web portals, along with brick and mortar retailers that are moving into e-commerce, have similar needs, but lack the background and technical expertise to easily integrate open-source technologies or migrate to a fully open-source platform. These businesses simply want web sites that their customers and vendors can use without difficulty. They need database-enabled web applications with 24/7 availability that won't crash or lose data--even with thousands of daily transactions. They want a site that always works, convenient ways for customers to buy their products, and secure methods through which to bring in their money. Because database applications are so critical to their mission, many businesses adopt well-known proprietary systems, feeling confident these companies will deliver quality and reliability. Yet the rising costs, the uncertain economy, and in some cases, the surprisingly unpredictable performance of commercial applications, all have begun to spark greater interest in open-source technologies today.

Still, these businesses are understandably skeptical about open source. They're used to the proprietary business model, and can't quite fathom why good software applications would be available to download for free from the Internet. The fact that these applications are not owned by a corporation causes suspicion and concern; if no single vendor owns it, people assume the software is not secure, powerful or reliable, and that it lacks accessible support and services. And the idea of thousands of independent developers around the world collaborating to create free software strikes many business managers as chaotic, which makes them even more reluctant to trust the results.

Slowly, these businesses are becoming educated about the open-source development model, which evolved not to make money, but to produce functional software efficiently. They're finding that while the development process varies for each open-source application today, the best projects most often attract a global community of highly skilled developers. And it's becoming clearer that these systematic meritocracies encourage rigorous testing and rapid development rates, and result in fewer bugs and security holes and more frequent releases of new and improved features.

A growing number of e-businesses such as WDN and the GeoCommunity are building their businesses on open-source platforms. These web portals have found that the software's fast-evolving development cycles, its lower costs, and its customizability make it ideal in their high-growth, quick-changing industries. The lower overall cost of open-source software is attractive to small and mid-sized businesses like these, who often have to spend thousands--even hundreds of thousands--of dollars on purchasing or licensing proprietary applications alone.

Another important advantage is that its code is open and modifiable. The open-source model rests on the belief that software develops faster and better when its source code is accessible to all skilled developers. Mature open-source technologies such as Apache, Linux, and PostgreSQL have thrived under the principles of open collaboration. Similarly, businesses that employ open source technologies can benefit both from open source's accelerated development model and free access to its internal code, which enables them to modify the code as needed. Open-source technology is highly conducive to innovation, and ensures that most of the applications it produces improve continuously and quickly. Companies that use it usually find that their software programs evolve as quickly as their businesses do.

Why isn't everyone using open source?

The perceived lack of professional support services for open-source software remains the stumbling block to its widespread use in business and industry. Business managers want to be able to call a service center when problems arise. In the past, those experiencing problems with open-source applications could send out an e-mail and usually within hours receive the right solution from the developers themselves. These informal networks of technical support can provide the highest possible levels of support, but they are not always immediately available, nor can they scale to meet the growing demands.

The issue of technical support was important to WDN and GeoCommunity because they knew they would occasionally need a technical safety net. They purposely chose applications with strong development communities in order to get the help they need directly from their web sites. Their technical staff concedes that companies without their own in-house technical staff need more comprehensive support. It's the one issue that continues to scare managers away from open source.

Fortunately, entrepreneurs always rush to fill a vacuum. Red Hat was one of the first companies to provide support for the Linux operating system, and now a slew of other companies are springing up to provide support, training and consulting services for some of the best open source database applications. The open-source support gap is shrinking fast, which is good news for emerging companies in need of an affordable platform for their growing business.

In the meantime, the WDN and GeoCommunity remain satisfied with their open-source decision, and the reliability, error reporting, community support, clean designs and strict adherence to industry standards that came with it. It's also refreshing that with open source, there can be no effort to force "lock-in" or add proprietary hooks that will prevent transition to other products in the future. As one senior staff member said, "It's truly been a liberating experience to use good products that were designed simply to meet a need--not to further a corporate agenda."