|
|
Intelligent Enterprise
March 2000
Think Systematically
Richard Winter
When you're implementing a large database, few things are more critical than a full-scale proof-of-concept. The full-scale measurement and experimentation inherent in a proof-of-concept lower risk and reduce the chance of poor decisions.
The value of a proof-of-concept is often intuitively obvious to architects and engineers with a background in large systems implementation. Unfortunately, to many senior executives . both in IT and in the business functions . it. s not so clear that taking three to six months to examine database performance and scalability issues is worth the time and money required.
Many executives will nix any major measurement or experimentation because they cannot concretely connect it to the issues for which they are directly accountable: schedule, return on investment, cost, and risk.
In my experience, if the subject of a major proof-of-concept is pursued at all, the parties to it . some with a business background and some with a technical one . have difficulty communicating about the alternatives and their implications. Uncertainty and complexity tend to cloud the issues.
In addition, most development methodologies . even those specific to the data warehouse . assume a single plan, not some continually evolving set of alternatives and trade-offs.
Why Decision Trees?
Decision trees help you visualize and communicate strategies, options, and trade-offs. A decision tree, as I use the term in this article, is a structure depicting alternatives, decisions, and outcomes in relation to time, value, and probability. There are two principle types of nodes in such a decision tree: decision nodes, as shown in Figure 1, and event nodes, as shown in Figure 2.FIGURE 1 A decision node with two alternative actions.
A rectangle represents a decision node, a situation in which a choice must be made. The expected date of the decision appears inside the rectangle. The options expected to be available at the time of the decision are represented by lines connecting the decision to subsequent nodes of the tree. Thus, Figure 1 (page 64) represents a decision to be made January 1, at which time the decision maker will determine whether: (a) a full-scale proof-of-concept will be conducted or (b) work will proceed immediately to the implementation of Release 1 of the data warehouse. Note that a decision node always describes a point in time at which the decision-maker has the option to exercise some control over outcomes.
In contrast, an event node, represented as an oval, describes an occurrence over which the decision-maker does not exercise control. In Figure 2 (page 64), the event is the user acceptance test for Release 1 of the data warehouse, which has two possible outcomes: Either the resulting solution . Meets all critical business requirements. (upper branch in Figure 2) or . Does not meet all critical business requirements. (lower branch in Figure 2). A probability is associated with the likelihood that each outcome will occur. The two outcomes in Figure 2 are considered equally likely, so each is assigned a probability of 0.5.
FIGURE 2 An event node with two possible outcomes.
Let. s assume that implementation of Release 1 of this large-scale data warehouse costs $10 million and is expected to yield $100 million in business benefits in the first three years after its implementation. Then, if the implementation of Release 1 is successful, the net benefit to the organization from the project is ($100m - $10m = $90m). If the implementation is unsuccessful (meaning, not successful enough to yield the expected benefits), the net benefit to the organization is -$10m.
The value of the outcome appears at the termination of the line representing it. Thus, the upper line in Figure 2, representing . success,. yields $90 million of net business benefits from the implementation over three years. The outcome of the lower line, representing . failure,. is a loss of $10 million to the organization.
Equipped with a value and probability for each possible outcome of the event, you can calculate an expected value for the event. Expected value appears beneath the oval representing the event and is calculated as the sum of the expected value of the outcomes. Thus, in Figure 2, the calculation is [($90m * 0.5) + (-$10m * 0.5) = $40m].
Decision Tree in Practice
Using these notational concepts, taken from Management Decision Making, A Formal/Intuitive Approach by Jerome D. Braverman (Amacom, 1980), you can readily construct a decision tree for a proof-of-concept and the role it plays in a data warehouse implementation. An example appears in Figure 3.Let's assume a very large-scale data warehouse is to be developed in a project starting January 1. There are two choices. The upper branch of the tree represents the choice of forging ahead immediately and completing implementation by June 30. The lower branch represents the choice of doing a full-scale proof of concept, also completing by June 30.
FIGURE 3 A decision tree for a data warehouse implementation with full-scale proof-of-concept.
If the proof-of-concept is done and shows that the system as originally conceived meets all requirements, then the actual implementation is launched July 31. The implementation can probably be done more rapidly after a proof-of-concept because of learning and progress made during the process, so let's say the estimated completion is December 31, five months from launch.
If the proof-of-concept shows that the system as originally conceived does not meet all critical requirements - a common outcome with systems that are at, near, or beyond the frontier - a two-week evaluation of the data will be conducted to determine whether the approach can be modified or whether the effort should be postponed or cancelled. The go/no-go decision is made on July 15 and, if it's a "go," is followed on July 31 by a decision to modify either the business requirements or the technical solution.
For purposes of illustration, the following assumptions underlie this decision tree:
A very large-scale data warehouse implementation is risky and, without a proof-of-concept, is as likely to fail as to succeed (certainly many such projects do fail in practice); there's a 50/50 chance of success if you just go ahead and build it.
A proof-of-concept will indicate either that the solution as initially conceived will meet all critical business requirements or that it will not; the two outcomes are equally likely
If the proof-of-concept indicates the solution will not meet all critical business requirements as originally conceived, two possible outcomes follow: It is feasible to modify the approach to yield success or it is not feasible and the project should be postponed or cancelled.
If modifying the approach is feasible, it can be done by modifying either the business or technical solution. Business-solution modification reduces the benefit from $100 to 80 million (the famous "80 percent" solution). Technical-solution modification increases the cost of implementation from $10 to 13 million. (These are deliberately pessimistic assumptions.)
If the project goes ahead after the proof-of-concept step, the probability of success will be 0.9 because approaches have been tested, unsuccessful tactics have been discarded, limits have been measured, and so on.
If the system is implemented without a proof-of-concept and cannot meet requirements, it is a failure and the $10 million cost of implementation is viewed as a loss.
Proof-of-concept costs $2 million to conduct.
If the project is postponed or cancelled immediately after proof-of-concept and before implementation, the $2 million expenditure is viewed as an investment, which was approved so that the company could avoid much greater risks or problems.
Expected-value calculations are done by starting from the ending nodes and working back to the starting node. To carry the expected value calculation all the way back, you must make an assumption concerning how decisions will be made. For the purpose of this exercise, I assume that the July 31 decision would be made to maximize the expected value.
Presented with this decision tree, a decision-making body can readily see the trade-offs. A proof-of-concept increases cost by $2 million and could postpone the completion of Release 1 by six months. However, it increases the expected value of the result by nearly 50 percent, from $40 to 57 million. In many companies, it could thus be shown to produce a more solid technical approach and to make economic sense.
It's interesting that our example, not contrived to support a particular outcome, illustrates that a proof-of-concept can increase the expected value of a data warehouse project. A review board can be walked through the assumptions, the calculations, and the decision tree so they can understand the options and tradeoffs much more clearly than they ordinarily would.
An important point about decision trees is that they provide a means, in the context of a professional methodology, to point out that undesirable outcomes may occur - and to show why, even with these undesirable possibilities, the project still makes business sense. By acknowledging the risk and thus sharing it to a degree with the approving authority, you can then talk sensibly about using time and resources to reduce risk. The result can be a stronger plan that ultimately benefits all parties involved.
Richard Winter (Richard.Winter@wintercorp.com, fax: 617-338-4499) is a specialist in large database technology and implementation, and president of Waltham, Mass.-based Winter Corp. (www.wintercorp.com).