Over the Wall

You need your Web analytics to be scalable; how can you be sure they'll grow enough?

By Richard Winter

Web analytics has delivered enormous business value in practice and has even greater promise as it and related technologies mature. Web analytics is the process of inferring the answers to important business questions from the behavior of visitors to a Web site — and delivering the results in a time frame and form that enables valuable business actions.

Without Web analytics, all visitors to a Web site are treated pretty much the same: Without detailed knowledge of who visitors are, what they want, and how they tend to buy, there isn't much opportunity to personalize the customers' experience. That's true whether the customers are visiting the Web site, the call center, the store, or any other channel. (See the sidebar, "Web Analytics: A Multichannel Strategy," below right.)

But there's a problem: Web activity for many companies generates too much of a good thing. That is, successful Web sites produce such enormous clickstreams that their managers soon encounter the brick wall of insufficient scalability. After making a big investment in CRM, many companies can lose it all when they outgrow their analytic capability or otherwise simply can't get to useful business results.

The Route to Business Value

Before looking into scalability, let's first define what you need in a product for clickstream analysis. The type of Web analysis product that actually helps executives make business decisions is one that's all of the following:

Complete. You need a solution to your entire Web analysis problem: data extraction, extensive preprocessing, integration, loading, aggregation, analysis, and reporting.
Results-oriented. You need a solution that ultimately will answer your critical business questions; one that's planned from the ground up to deliver insights into key business questions, rather than just count page hits.
Timely. Answers are based on sufficiently fresh data. The knowledge has to come quickly after the behavior is exhibited. Only then can you get the powerful, positive feedback loop that results in effective action.
Flexible. The solution has to be adaptable to accommodate change, because opportunities come and go, customer appetites and market conditions change, Web sites are fluid, and most of the profits go to those who time their moves well.
Scalable. And, of course, the solution must obey the golden rule of modern commerce: Don't run out of headroom.

Scalable Web Analytics

But what really is scalability in Web analytics? In Web analytics, scalability means all of the following:

The entire solution grows as your business grows. You need to get all the way from 100 million page hits a day to meaningful insights about your business and your customers, as shown in Figure 1. These stages include:

Data gathering
Preprocessing and integration
Storage and aggregation
Analysis
Query and reporting
Action.

Figure 1: From Web logs to business actions with Web analytics.

The ability to answer the hard (analytical) questions grows. Your solution needs to handle increasingly larger and more complex demands for data access and data analysis as your program progresses.

Handling growth in both database size and workload. The growth rate of both database size and workload tends to be rapid and more than linear, as shown in Figure 2.

Figure 2: Rapid growth in database size and workload in Web analytics.

Performance remains satisfactory. A scalable solution for Web analytics will continue to deliver satisfactory performance, even as the difficulty of queries, the number of queries, the number of users, and the size of the database continue to increase.

Throughput continues to meet requirements. For example, there may be a requirement to incorporate a day's worth of Web site activity into the database in four hours. In a scalable solution, throughput requirements will continue to be met, even as Web site activity, business focus, data volumes, and analytical activity grow, as illustrated in Table 1. In fact, as Table 1 shows, this combination of circumstances often causes the processing window to shrink, which induces tremendous pressure on the architecture's scalability.

Table 1: Throughput requirements often intensify as scale increases.

Price/performance remains stable. Price/performance is the cost of a system per unit of work. A scalable solution for Web analytics will ideally exhibit stable price/performance as the system grows.

The system never becomes unmanageable. Manageability refers to the ease with which you can perform the routine day-to-day and week-to-week tasks associated with operating the system, monitoring its performance and availability, and so on.

What to Do

If you think you may need a solution — or a better solution than you have — for Web analytics, here's what I recommend:

Start with your business case and needs. Then define your requirements quantitatively. Next, define quantitatively how your requirements are likely to grow. Only then are you ready to ask your prospective suppliers probing questions about performance and scalability.

Be sure to ask the following questions in terms of your most demanding likely requirements two to three years after implementation. Ask how the vendor's proposed solution will:

Handle capacity requirements over time. What is the maximum capacity of the database that the analytics engine can handle?
Manage capacity requirements over time. What capacity management capabilities are inherent in the product? Can they be tailored to fit your analysis requirements? For example, do they have the flexibility to age out different types of data? Can those parameters be changed?
Allow you to determine space and hardware requirements for the Web analytics environment. Can the vendor forecast those needs over a two- to three-year time period?
Handle varied data resources for analysis. Does the data-import framework support multiple formats including log files, network sniffers, page dots or tags, and Web server plug-ins, including Netscape Application Programming Interface (better known as NSAPI) modules? Does it match your operational requirements?
Allow sampling of data based on the notion of a "user" that's defined by a given request rather than a sample that is just a percentage of the overall data stream.
Get your data into the database, including indexes and aggregates, on time.
Produce the standard reports, including the metrics you need to run your business, on time.
Respond to your typical queries in seconds.
Ensure that your data will be available when you need it, even in the presence of disk failures and human errors.

Once you've selected your vendor, insist on proof of performance and scalability sufficient to meet your needs. To have reasonable assurance that your requirements will be met, you need more than good answers. You need proof: You want references from companies that are doing today what you need to do in two to three years. If at least five such references, including some in your industry, aren't available, you must do a full-scale proof-of concept or competitive benchmark.

Make sure you're getting a solution with the right function, architecture, and scalability. Don't forget that you also need the right capabilities; you need a quality product and vendor; you need a vendor that has staying power, commitment, and vision.

Longevity Pays

Web analytics offers one of the most compelling business opportunities of the coming decade by offering timely, cost-effective insights into customer behavior. Most important, Web analytics aren't just about what happens at your Web site. They often provide the clues needed to implement customer and market strategies across all market channels.

And here's the most overlooked point of all: Web analytics without scalability is a dead end. Success in understanding and leveraging customer behavior leads swiftly and intensively to a remarkable growth in requirements. A Web analytics system that isn't scalable dies in a matter of months. So, in acquiring these systems, you absolutely must minimize your risk of running out of headroom.

But if you implement a scalable Web analytics solution that can address your full set of needs, you have an excellent chance of realizing a big return on your investment.

Traditional Methods Failed

This costly promotion produced almost nothing in sales. Initially, the product manager feared that there was something wrong with the product or the timing or manner of its introduction. But she believed in this product line and kept asking questions.

Almost ready to give up, she discovered that online sales, while small in volume, were showing a definite upward trend. She asked for a profile of those who were buying the product online and learned that most of them were men who hadn't previously bought any investment product from the company other than mutual funds and had an income of $60,000 to $100,000 per year. Further statistical analysis revealed that a remarkably high percentage had an oldest child who was four or five years old. A modest survey of a sample of the buyers showed that these men were worrying about paying for college then because the oldest child was about to enter kindergarten. Somehow, the realization that the child was beginning school triggered worries about paying for college. Analysis also revealed 80 percent of those men had also checked out term life insurance while online.

The product manager was then able to win approval for a multichannel promotion built around family security and educational financing for relatives of children entering kindergarten. Brokers and call-center agents were trained to ask callers who fit the profile certain questions about security and educational goals. If the customer expressed interest, the agents and brokers were trained to present an offer bundling the new educational product with life insurance. The Web site was programmed to ask the same questions of those who fit the profile and, if they said yes, make a similar online presentation. A mailing was sent to just those male customers who fit the income profile and had an oldest child who was four or five. All these steps produced above-average response rates. Sales of both the new educational financing product and life insurance quickly increased. The return on investment (ROI) of these more targeted promotions was more than 70 percent, annualized. The product manager earned a hefty raise and a promotion.

Why Failure Turned to Success

The product manager was able to overcome the initial failure for one reason: Her company had an electronic customer relationship management (eCRM) program founded on a strong capability for Web analytics that was capable of dealing with very large volumes of data.

From millions of customers, the product manager was able to identify just 50,000 who were especially likely to be interested in the specific product she had to offer. It was the customer behavior at the Web site that tipped her off: she learned who her first set of buyers were, why they were buying, and how to appeal to them. And she learned all that inexpensively and quickly, once she understood that the company's standard approach wasn't going to work well for this new product. Most important, she leveraged her findings from the Web site across all channels in a coordinated strategy. That's how she realized the 70 percent annualized ROI.

Customer Insight

Almost all enterprises are in some stage of planning, implementing, or using CRM. But CRM programs don't automatically tell you what will make the customers more satisfied, more profitable, or more frequent purchasers. To realize the business value of CRM, you need insight into customer behavior and preferences. Comprehensive, intimate knowledge of the customer is available rapidly and economically only by tracking Web activity.

When customers visit a Web site, every keystroke and mouse click is recorded, along with the time and date it occurred. If you analyze the resulting "clickstream," you can learn a great deal. For example, you can learn not only what purchases the customers made but also what they looked at during their visits. The resulting knowledge can be used not only on the Web, but across all channels.