Solutions for the Information Project Team • www.dbta.com
Volume 15, Number 12 • December 2002
Make Room for the Monster Databases

Some of the world’s biggest databases are growing by a factor of 20.
How do you begin to manage such growth?

By Joe McKendrick
They're big, and they're getting bigger. New statistics out of Winter Corp. - a consulting firm that tracks database size on an annual basis - finds that databases are poised to hit the 100-terabyte mark within the next two years. Remarkable, considering that just a year ago, the largest databases on record were between five and ten terabytes in size.
...Some of the biggest databases are growing by a factor of 20, Richard Winter, CEO of Winter Corp., told DBTA. "They're projecting growth in databases by 2004 that are in the range of 100 terabytes of data," he said. Even more remarkable is the fact that Winter is only measuring actual data stored in his annual survey - not system overhead. A hundred terabytes of data "easily equates to more than 500 TBs of disk," he said.
...Much of this growth is around decision support functions, according to the latest Winter survey. "We found that the average growth projection for decision support databases over three years was 169 percent. For transaction processing systems, the rate was 124 percent," Winter said. "On the average, decision support databases were a half a terabyte larger than operational systems," he added. The Winter survey also calculates that such databases will grow to more than two terabytes larger. "Decision support databases are growing much faster than transaction processing databases - they're almost tripling in size over the next two years."
...At the core of this decision support, of course, is customer data. CRM was the principle application among the very large databases measured by Winter, followed by e-commerce. Interestingly, most of these applications for very large databases were custom developed by the end-user company. For example, the survey found that 41 percent of CRM solutions were custom developed, versus 7 percent acquired through a DBMS vendor, and 7 percent through another third-party application. Another 45 percent were not sure of the source of their CRM applications.
...Of course, most companies don't have multi-terabyte behemoths on site. In fact, most are still working on reaching their first terabyte. In the mainstream, typical large databases range anywhere from 100 GBs to 500 GBs in size, Winter reported. Such database power is in evidence at Foxwoods Resort Casino in Connecticut, the nation's largest casino complex. The resort casino, owned and operated by the Mashantucket Pequot Tribal Nation, relies on a 300 GB database for both marketing and casino financial information. The system, which runs on a Progress database, currently supports about 20 million transactions a day, according to Todd Williams, MIS database and DSS manager for Foxwoods, in an exclusive interview with DBTA.
...The casino's player management system is a "customer tracking and loyalty program the business users to track activity, better understand profitability, and better manage those key customer relationships," said Williams. "Much of the core application database consists of millions of patrons, and hundreds of millions of player rating transactions. These ratings help track patrons' gaming activity which provides insight into the marketing initiatives. Those transactions also feed into a loyalty program that allows patrons to earn comp points. So a certain amount of gaming play gets rewarded with 'points' that can be used at retail shops, restaurants, and for many other casino offerings." The other half of the database supports many other casino operations, including financials, Williams added. The Progress database system runs on IBM pSeries hardware running AIX.
...While there are many technical challenges to address in deploying large databases, the most daunting obstacles are political ones, the Winter survey found. Almost a third of the group say organizational issues hampered the progress of their projects. "The most widespread obstacle was getting the buy-in from all levels of the organizations," said Kathy Auerbach, vice president of Winter Corporation. "You have to have the users buying in to saying, 'yes, these are the requirements we need, and when you give us this system, we're going to use it, and we're not going to hold back and do things the old way.'"

For more information visit www.wintercorp.com.