Computing co-ops

Will distributed computing increase network bottlenecks or help eliminate them?

Napster and Gnutella are the early signs of a computing epidemic. These music and mutual interest communities, which allow users to swap files and share resources, have fueled the public's curiosity in peer-to-peer, or P2P, networking. Now consumers - and corporations - are imagining the unlimited power that results when large contingents combine forces for a common interest.

This interest in P2P technology is nurturing the growth of P2P's first cousin, distributed computing. Using P2P's resource sharing concept, distributed computing brings together the idle processing power of PCs scattered throughout the globe. By downloading an agent to their PCs, individuals can participate in studies that analyze the human gene, research a cure for influenza and seek out intelligent life on other planets.

One of the more widely-known distributed computing projects is SETI@home (Search for Extraterrestrial Intelligence) from the University of California at Berkeley. More than 2.5 million people have participated in this project since May 1999. SETI@home uses the combined power of these individual PCs to crunch telescope data compiled by UC Berkeley in its search for extraterrestrials. "They needed a supercomputer to analyze the huge backlog of data," said Jikku Venkat, vice president of engineering for United Devices. "By distributing the data across multiple PCs via the Internet, they are able to get supercomputer results without the price."

David Anderson, SETI@home's director, is now fine-tuning the technology as chief technology officer of United Devices. His SETI@home project, as well as other distributed computing applications, goes to work when a participating computer is idle. The workstation contacts the server, and the server dispatches a client to the PC. The PC then begins working on the data, as instructed by the server. By sending portions of data across millions of computers, these projects get the benefits of a supercomputer.

Although the initial projects are non-profit, the start-ups behind distributed computing expect to move into the for-profit arena. They expect support from pharmaceutical companies, space research, financial houses and environmental groups. "We're still working on the 1.0 commercial release," said Nelson Minar, chief technology officer and founder of Popular Power. "When it is available, we want to balance our commercial pursuits with our attempts to help mankind."

For now, Popular Power and United Devices claim people sign on for these projects at breakneck speeds because the public wants to support and contribute to politically correct, feel-good causes. However, the companies also are adding incentives to increase their numbers.

These companies and their competitors provide statistics to each participant that show just how much work the individual's computer did and how their workload compares with other participants' efforts. In addition to these rewards, participants can win frequent flyer miles or cash returns of $5 to $15 per month. Beyond these individual incentives, Venkat expects that cash may appeal to corporations' IT departments. "A lot of resources sit idle at night," Venkat said. "We would be willing to give companies a return on these investments for the use of their resources."

But before United Devices or any other company brings this proposal to corporations, the start-ups must prove that the technology is 100% reliable and secure. Popular Power and United Devices expect that these large-scale, non-profit projects will provide the necessary evidence to move distributed computing into business applications and onto corporate networks.

To prove reliability, the companies plan to continue to increase their number of users without affecting the system negatively. As for security, Venkat claims the application is as secure as any other application downloaded from the Internet.

"The pipe between the server infrastructure and client machine is completely encrypted. And the application has stringent requirements regarding what it can do on the client workstation," Venkat said. "We have a certification and testing process for all the applications and we honor all constraints that users may have on their operating systems."

Privacy is another concern start-ups must address. Neither company collects data about the individuals in the projects; instead, they gather information about the PC such as the operating system, memory and connection speed.

But if distributed computing gains a foothold among individual users and corporations, it could affect service providers' network performance. Networks with large distributed computing populations could experience increased congestion. Although not a problem now, Minar's company created safeguards for this potential challenge.

"We look at the ratio of bandwidth and CPU that each application needs, and we focus on jobs that require less bandwidth. We can see which networks are getting a lot of traffic, and when one is getting hit particularly hard, we lay off them."

United Devices has a similar strategy. "We can constrain our traffic, and we can schedule data transfers between client and server for certain times of day. We can also limit the amounts of traffic according to a service provider's request," Venkat said.

Rather than cause headaches for service providers, the companies expect that distributed computing applications could give carriers relief. "A carrier could buy our resources to understand how their networks behave. We could run traffic simulations," Minar said. "Or, we could run diagnostics on the Internet and create maps that look at delays and outages."

Exodus Communications already works with United Devices to do Web site performance testing on its server farms. The hosting and co-location facility uses a distributed computing application to run load and stress tests on its servers to locate performance breakdowns.

While Exodus applies distributed computing for performance testing, other carriers are looking at P2P for content distribution, especially video. Israeli-based vTrails has developed full duplex packet cascading (FDPC) for providers that want to multicast audio and video to millions of subscribers.

Instead of sending the video to each individual viewer, the FDPC server, which sits between the content server and the Internet, creates a hierarchical structure that arranges viewers in tiers, with the fastest connections at the top. Only the top tier gets content from the original site. Everyone else gets replicated content from the viewer on the tier above. FDPC uses the individual workstation's CPU power to replicate the content and pass it on to the next viewer.

"Our technology streams packets from multiple users and routes it along different routers. Most important, it saves service providers' bandwidth," said Nezer Zaidenberg, chief technology officer and founder of vTrails. "It looks and smells like an IP multicast, and users believe that's what they are getting, but we're really using a P2P network."

To move P2P and distributed computing forward, Intel formed a working group in late August. In October, more than 300 people attended the first meeting where they rolled out the working group structure, decided voting rights and created sub-working groups focused on different technology areas. These included architecture, security, manageability and interoperability.

"We see this as similar to when Mosaic sparked the Internet and created something that the community gathered around. The working group will help teach why this technology is important and define standards and infrastructure," said Patrick Bohart, marketing manager for Intel's P2P Initiative.