|
|
||||||||
University of Kentucky, Lexington 40546-0215
| Abstract |
|---|
|
|
|---|
Key Words: Replication Statistical Power Swine Nutrition
| Introduction |
|---|
|
|
|---|
| Some Basics |
|---|
|
|
|---|
Replication refers to the assignment of more than one experimental unit to the same treatment. Each replication is an independent observation; thus, each replication involves a different experimental unit. Correct definition of the experimental unit determines the entity to be replicated. Consider the following example. An experiment is conducted to compare effects of four different diets on the performance of growing-finishing pigs. Four pens of the same size are available and each will house eight pigs of the desired age and weight. The investigator randomly assigns eight pigs to each pen and then randomly assigns diets to pens. The investigator believes "pig" is the experimental unit and that there are eight replications. However, because diets were assigned to pens, and all pigs in the same pen receive the same diet, "pen" constitutes the experimental unit. As a result, the experiment has no replication, and further assumptions are needed before valid conclusions can be drawn.
The basic function of replication is to provide an estimate of experimental error. In general, increasing the number of replications results in a more accurate estimate of experimental error and more precise estimates of treatment effects. For a particular experiment, the number of replications needed depends on the magnitude of the difference or effect to be detected at a specified significance level, the power, and the variability of the experimental material.
Power analyses, as discussed in this article, revolve around the investigators interest in establishing validity of the alternate, or research, hypothesis. Therefore, a review of classical hypothesis testing is useful. In classical hypothesis testing, a decision is made to either accept or reject the null hypothesis. Rejection of the null hypothesis points to validity of the alternate, or research, hypothesis. There are two possibilities for the underlying reality (Cousens and Marshall, 1987): either the null hypothesis is true or it is false. If the null hypothesis is accepted when it is true or, conversely, rejected when it is false, no error will have been made. This leaves two possibilities for error: a Type I error is made when a true null hypothesis is rejected; a Type II error is made when a false null hypothesis is accepted. Possible outcomes of the hypothesis test are often summarized in a contingency table (Table 1
).
|
Step 1: Formulate hypotheses. The null hypothesis is one of no difference, no effect, no change. It is the hypothesis to be tested and is the opposite of the alternate, or research, hypothesis, which represents what the investigator truly believes and seeks to verify.
Step 2: Decide on Type I error rate,
. This rate is referred to as the significance level of the test. Because classical hypothesis testing is intended to reduce the risk of false claims,
is set as low as possible. Typical yet arbitrary values are 0.01, 0.05, and 0.10.
Step 3: Collect and summarize data; calculate the appropriate test statistic (t, F, etc.).
Step 4: Determine the critical value of the test statistic given the value of
and size of the experiment, identify the rejection region, and state the decision rule. If the test statistic falls within the rejection region, the null hypothesis will be rejected in favor of the alternate.
Step 5: Make decision and draw conclusions.
Step 6: Compute P-value and interpret. The P-value is the observed significance level of the test. It is the probability (assuming the null hypothesis is true) of observing a test statistic as large or larger than that observed from the experimental, or sample, data. It describes the strength of evidence against the null hypothesis; the smaller the P-value, the stronger the evidence against the null hypothesis.
As noted previously, classical hypothesis testing is intended to reduce the risk of false claims. Thus,
is set to as low a value as possible to guard against Type I errors. Conversely, establishing a value for ß, the probability of a Type II error, is rare, and nonsignificance is often used to indicate a true lack of effect. As a result, Type II errors often occur (Table 1
). As noted by Cousens and Marshall (1987), investigators must realize that a nonsignificant difference indicates that if there is a real difference it is smaller than could be detected as significant in that particular experiment. It does not mean that treatments do not differ. Classical hypothesis testing is inappropriate if the intent is to prove that some treatment has no effect. In such cases, equivalence testing should be used (Dixon, 1998).
"Power" in this context refers to the ability of the experiment to detect real differences, if they exist, at the desired significance level. In other words, it is the probability (1 ß; Table 1
) of rejecting the null hypothesis when it is false. Given two experiments, the one that can detect the smaller treatment effect has the greater power. Size of the experiment, as measured by the number of replications, is highly dependent on the desired power. In turn, power depends on the magnitude of the difference to be detected, the significance level, and the size of experimental error. As
is reduced, the probability of rejecting false, as well as true, null hypotheses is lowered. A reduced Type I error rate is accompanied by reduced power. The chance of detecting smaller differences is increased as power is increased, and this occurs when experimental error is decreased.
| The Use of Power Analyses in Swine Nutrition Research |
|---|
|
|
|---|
Planning
The a priori approach (prospective power analysis) provides an estimate of the number of replications necessary for an experiment of known power and sensitivity, where "sensitivity" refers to the minimal treatment response that will be detectable (Berndtson, 1991). It helps ensure that an investigator does not waste time and resources on an experiment that has little chance of finding a significant effect, if one truly exists, and ensures that resources are not wasted by including more replications than necessary to detect an effect. This is especially important with livestock, such as swine, for which the cost per replicate is high, both in terms of capital required to purchase and house the animals and the labor needed to care for them and collect data (Morris, 1999). Ethical considerations may also come into play (Erb, 1990; Van Wilgenburg et al., 2003). For example, in experiments involving surgical modifications, either using so few animals that the smallest important effect cannot be observed or using more animals than are needed may be considered an improper use of the animals and a waste of research funds. In reality, using more replications than are needed is rarely a problem in studies involving livestock.
An a priori power analysis is a critical part of a good experimental design; however, Morris (1999) observed that it is frequently the most neglected part of a preexperimental protocol. In reality, investigators seem to choose the number of replicates arbitrarily on the basis of cost or availability of animals, housing considerations, convenience, or tradition. Nevertheless, the answer to the question of how many pigs is a statistical one. Reasons given by investigators for not conducting a prospective power analysis include "Ive always used this many," "I dont know what variance to use," "I dont know how big a difference to expect," "These are all the pigs (pens) available," and "I dont know how." The overriding reason, however, is that they know that power calculations will likely yield much larger numbers than are desirable, feasible, or even possible, given the resources available.
These excuses point to six hurdles that must be overcome if an a priori power analysis is to be conducted. First, the number of treatments must be determined. The choice of treatments that are to be compared is one of the most difficult and important decisions made during the planning stage. Ambiguous results can generally be avoided if the choice of treatments is based on the specific objectives of the experiment. Expense, time, and level of complexity are also important. Second, hypotheses must be specified. As with the choice of treatments, this is a function of the experiments objectives. Third, the level of significance must be chosen. Typically, but arbitrarily, levels of 0.01, 0.05, and 0.10 are used, but whatever level of significance, the choice should be based on the relative consequences of rejecting the null hypothesis when it is true (Type I error) and accepting the null hypothesis when it is false (Type II error). The other three hurdles include deciding on the smallest difference that is worth detecting in a particular experiment, selecting the appropriate power, and estimating the variability in response. These are more difficult to overcome and, unlike the first three, are not usually addressed in introductory statistics courses.
In deciding on the minimal treatment response to be detected, an investigator must consider what might be biologically or economically meaningful. This is often difficult to ascertain and depends on the specific situation; however, the minimal treatment effect must be large enough to make the experiment worthwhile biologically and/or economically. As the size of the difference to be detected decreases, the number of replications required for an experiment of known power will increase. For example, if a difference in a rate of gain of 40 g/d can be detected with 4 replicates, an experiment of approximately 16 replicates will be needed to detect half of this difference, 20 g/d, because the standard errors are in the ratio 2:1 (assuming the same variance). However, if this difference is halved again, to 10 g/d, approximately 64 replicates will be necessary to detect this difference. An investigator will have to determine whether the smaller difference is worth the cost of increased replication.
In swine nutrition experiments, guidelines for determining expected differences may be obtained from previous work. For example, in sow reproduction studies, estimated average litter size is 12 to 14 pigs at birth and 10 to 12 pigs at weaning (personal communication, G. L. Cromwell, University of Kentucky). Expected differences are typically in the range of 5 to 10% of the mean, which would be 0.6 to 1.2 pigs per litter for an average litter size of 12 pigs. In growing-finishing pig experiments, normal ADG is approximately 0.80 kg/d (personal communication, G. L. Cromwell). For practical diets, with no deficiencies, an expected difference of 5 to 10% would be 0.04 to 0.08 kg/d. A difference of 10% is generally assumed to be meaningful. If one of the treatment groups consists of a negative control, in which pigs are deficient in some nutrient, the expected difference may be as large as 50%. In these examples, the minimum difference to be detected has been expressed as a percentage of the mean response.
In determining the appropriate power, the idea is to have a reasonable chance of detecting the stated minimum difference. A target power of 80% is common and can be used as a minimal value. Some statisticians argue for higher powers, such as 85, 90, or even 95%. As power increases, however, the required number of replications increases. Therefore, it is rare with animal experiments to set power at values larger than 80%.
The last, and most difficult, hurdle to overcome involves the question of how much variability. An estimate of the amount of variability in the response variable may be obtained by making an educated guess, by conducting a pilot study, or by reviewing previous experiments conducted by the investigator or reported in the scientific literature. The reliability of estimates obtained from previous experiments depends on the following: the similarity of the populations; the number of observations upon which estimates are based; the experimental protocol (instrumentation and how, when, and who administers treatments and collects data, feeding management, type of experimental unit, time at which measurements are taken, etc.); the design structure (blocking); sources of variation (such animal attributes as gender, age, and breed); and whether or not covariates are included in the analysis. Treatments do not have to be the same or even similar because the error mean square provides an estimate of the variation among replicates within treatments. Furthermore, the analysis of variance requires the assumption of a homogeneous variance; that is, the variance among replicates within treatments is assumed to be the same for all treatments (Steel and Torrie, 1980). Finally, previous experiments do not have to follow the same design structure as the planned experiment, but the investigator must be careful that the correct variance is being estimated (Lenth, 2001a).
The absence of a previous estimate of variability is never a valid reason for not calculating the number of replicates required. Guidelines for estimating the coefficient of variation of future experimental material are presented by Morris (1999). Among mammals, the coefficient for growth rate is approximately 12%; for reproductive traits, 20 to 40%; for characteristics such as milk yield, 20 to 25%; and for linear measures, such as bone length or height, 6%. Also, as noted by Morris (1999) the coefficient of variation of a trait is inversely related to its heritability. Thus, for traits that are highly heritable (e.g., carcass traits), the coefficient of variation would be lower than for traits that are lowly heritable (e.g., reproductive traits). These guidelines provide investigators with a general idea of the amount of variability for different types of traits across species.
More specifically, estimates of variability for response traits measured on a particular species can be obtained by averaging values calculated from previous experiments. Based on data collected at the University of Kentucky (our unpublished results), as well as a review of experiments reported in the scientific literature, estimates of variability were obtained for some of the response variables typically measured in swine nutrition experiments. For the most part, these estimates are compatible with guidelines presented by Morris (1999). For sow reproduction experiments, a completely randomized design is assumed because it is common to assign treatments completely at random to individually fed sows. The average coefficients of variation for total number of pigs born; number of live pigs at birth; birth weight; number of live pigs at 21 d; and number of pigs weaned are 28, 29, 18, 36, and 37%; respectively (Table 2
). These values are based on 29 estimates involving in excess of 16,500 litters. For growing-finishing pig experiments, a randomized complete block design is assumed because experimental units are commonly blocked on factors such as initial age, weight, and gender. Pen is considered the experimental unit. The average coefficients of variation for rate of gain and feed efficiency range from 5 to 8 and 4 to 7%, respectively (Table 3
). For very young pigs, these values may range from 7 to 13% (our unpublished results).
|
|
|
|
To illustrate a priori power calculations, consider a proposed sow reproduction experiment. Sows will be individually fed two diets. The response variable is litter size as measured by number of pigs born alive. A coefficient of variation of 29% (Table 2
) is assumed, and the investigator would like to be able to detect a difference of at least 10% with 80% power and a significance level of 5%. Using Table 4
, the estimated number of sows per treatment is found to be approximately 142. This yields an experiment of size 284 (142 sows for each of two treatments). Thus, for a population with a coefficient of variation of 29%, 142 sows will be needed per treatment to provide an 80% chance that a 10% treatment response will be detected at the 5% significance level. If the power were increased to 90%, 190 sows per treatment would be required. If it were increased to 95%, 234 sows per treatment would be required. As the power is increased from 80 to 90% and from 90 to 95%, the number of replications required increases by 34 and 65%, respectively. This illustrates how rapidly the number of replications increases as power increases.
Using G*Power is only slightly more complex. It requires an estimate of the standardized effect size (Cohen, 1988). The standardized effect size (f) is calculated as the ratio of the variance of the treatment population means to the population error variance (Myers and Well, 2003) and is an adjunct to the analysis of variance. For a completely randomized design, the effect size can be calculated from a previous experiment as follows:
![]() | [1] |
where t = number of treatments, r = number of replications, and MST and MSE refer to the treatment and error mean squares, respectively. Expressed more simply,
![]() | [2] |
where FT is the ratio of MST to MSE. Cohen (1988) has suggested that f = 0.10, 0.25, and 0.40 correspond to small, medium, and large effect sizes, respectively. This convention is somewhat controversial. Proponents claim two advantages: 1) pilot or previous data are not required to estimate the variance and 2) the standards for small, medium, and large provide realistic norms (at least in the social sciences). Opponents claim that a standardized effect size, such as f, has no relevance as a criterion for determining sample size (Elashoff, 2000). And a standardized effect size is harder to relate to the actual units of measure for a particular response variable. For example, given the proposed sow reproduction experiment, what does a small effect size mean as it relates to a difference in number of live pigs born per litter? a medium effect size? a large effect size?
To find the required number of replications for the sow reproduction experiment using G*Power, the following steps are taken
The result will be the total sample size. This will not be exact because G*Power requires the difference to be expressed as the standardized effect size, and the criteria for the proposed sow reproduction experiment specified the minimum detectable difference as 10% of the mean. Thus, a trial-and-error approach is taken. If a large effect size (f = 0.40) is used, the total sample size, n, is 52, or r = 26 sows per treatment. If a medium effect size (f = 0.25) is used, n = 128 and r = 64 sows per treatment. If a small effect size (f = 0.10) is used, n = 788 and r = 394 sows per treatment. As discussed previously, the smaller the size of the difference to be detected, expressed here as the standardized effect size, the more replications that will be required. Using Table 4
, the number of replications for the proposed experiment was approximated to be 142 sows per treatment. Comparing this value with the number approximated for a small effect size (394) and a medium effect size (64), it could be concluded that a 10% difference corresponds to an effect size somewhere between small and medium.
As a second example, consider a growing-finishing pig experiment being designed to compare the ADG of pigs fed four diets in a randomized complete block design. Pigs will be group-fed (4 to 8 pigs per pen), with pens being blocked according to weight and gender (a chunk-type blocking factor as described by Ott [1975]). The investigator would like to have an 80% chance of detecting a minimum difference of 10% at a 5% significance level. A coefficient of 8% (Table 3
) is assumed. Because each block is a replication, the question actually is, how many blocks are required? Assuming sphericity, power calculations are conducted as with a completely randomized design. Using Table 4
, it is determined that approximately 11 replications (blocks), or a total of 44 pens, will be required to provide an 80% chance that a 10% treatment response would be detected at the 5% level of significance.
The number of replications applet will yield the same estimate. Using the applet, the following values are input:
A direct answer cannot be obtained for randomized complete block experiments using G*Power. Instead, a trial-and-error approach (select "Other F tests") must be employed.
The number of replications required is often quite large, and in reality, resources are likely to be limited. If the number of replicates calculated is prohibitively large, an investigator has several options. First, ask whether the difference is too small to be of biological or economical importance. If the minimum detectable difference is smaller than necessary, the planned study may be more powerful than necessary. Second, can the Type I error rate be increased from 0.01 to 0.05? Can it be increased from 0.05 to 0.10? In other words, how serious would the consequences be if the null hypothesis is rejected when it is actually true? Third, is the power set too high? Typically, 80% is used as a minimal value. Finally, is the variability of the response variable overestimated? Could it be reduced?
If it is determined that all input values are appropriate, there are three possibilities. First, find a satisfactory way to reduce the amount of variability. This might be accomplished by choosing a more precise measuring device, using more homogeneous experimental material, choosing a design structure that will provide more error control (i.e., blocking), or by using one or more covariates to account for extraneous variability. Second, amend treatments so that a larger difference can be expected. Third, go back to the drawing board and narrow the scope of the experiment or argue for a bigger budget.
In many cases, the investigator may ask, what good is a power analysis if I have only a set number of pigs available? When the maximum sample size is set by spatial, physical, or economical constraints, a power analysis may be useful to determine whether sufficient power exists for specified values of the level of significance, sensitivity required, and anticipated variation. Ultimately, the investigator may have to determine whether the experiment is worth pursuing. The goal should be to establish balance in what is statistically desirable and what is practically possible.
Other Considerations. In conducting an a priori power analysis, there are other important considerations. The number of treatments must be decided on. Additional treatments increase total sample size, but they also increase error degrees of freedom and, as a result, may decrease the size of the error variance. This, in turn will increase the power of the experiment. However, according to Berndtson (1991), the effect on the number of replicates is minimal.
The type of means comparison procedure may also affect power calculations. Gill (1989) estimates that replication requirements may be reduced by as much as 20 to 30% if orthogonal contrasts are used instead of pairwise comparisons. Ultimately, this decision should be made based on the nature of the treatments (Aaron and Hays, 2001). One-tailed vs. two-tailed tests could be considered. Using a one-tailed test will reduce replication requirements within limits, but, given that the outcome of the experiment is unknown in advance, a two-tailed test is often more appropriate.
The choice of experimental unit is another factor that affects power in an experiment. As discussed earlier, the experimental unit can be an individual or a group. It is the smallest unit receiving a single treatment provided two such units could receive different treatments. In some experiments, pen is correctly recognized as the experimental unit for response variables like ADFI or gain:feed, which are measured on the pen of pigs, but not for traits such as longissimus dorsi area or plasma urea nitrogen, which are measured on the individual pig. Pen is both the experimental unit and the sampling unit for ADFI and G:F, but for traits such as loin muscle area or plasma urea nitrogen, pen is the experimental unit and pig the sampling unit. In either case, replication involves increasing the number of pigs per treatment. Increasing the number of pigs per pen (i.e., within-replicate sampling) will increase power up to a point, but, if a given number of pigs can be fed individually, power can be increased more by using the individual pig as the experimental unit and having more replications than by using the same number of pigs in fewer experimental units. Treating pigs individually maximizes the use of animals, but maximizing the efficiency of total resources may necessitate the grouping of pigs. In addition, pigs treated individually may not respond to treatment in the same manner as pigs treated in groups. Specifically, variability in response may be greater among individually fed pigs than among group-fed pigs.
Special considerations are also required for more-complex designs, factorial treatment structures, experiments involving repeated measures, use of covariates, and experiments involving multiple response variables. The last is more often than not the case with animal experiments. Multiple response variables will exhibit different degrees of variation, and thus will require different numbers of replications. To further complicate things, different minimum detectable differences may be of interest. The simplest approach is to take the response variable expected to express the greatest degree of variability and having the smallest minimum detectable difference and calculate the number of replications required for known power and sensitivity for that variable. For example, in a sow reproduction experiment with two dietary treatments, both number of live pigs at birth and litter birth weight may be of interest. The number of live pigs at birth is expected to have a coefficient of variation of approximately 29%, whereas the corresponding value for litter birth weight is only 18%. For number of live pigs at birth, it was herein determined that approximately 142 sows per treatment would be required to provide an 80% chance of detecting a 10% treatment response at a 5% significance level. For litter birth weight, however, only 52 sows per treatment would have been required to detect a 10% treatment response. With 142 sows per treatment, the power for detecting a 10% difference in litter birth weight would be over 95%. Whereas power is overestimated for the trait with the smaller coefficient of variation, a minimum power is ensured for the trait with the larger coefficient of variation.
Interpreting Results
A second kind of power analysis may also be useful. If no significant effects are found in an experiment, the investigator can assess post hoc the observed (actual) power of the experiment or determine the size of treatment effect that could have been detected using the standard deviation and number of replicates in the experiment. If used appropriately, this a posteriori, or retrospective, power analysis can be very useful in interpreting results.
Interpretation of results is not difficult when all treatments being compared are significantly different from each other. However, when nonsignificant results are observed, care must be exercised in interpreting the results. A common mistake is to say the treatment effect was nonsignificant and, therefore, does not exist. A nonsignificant statistical result does not mean a difference does not exist.
To calculate the observed power of an experiment, the existing analysis is used to estimate both the effect size and population variance. Consider the following hypothetical sow reproduction experiment. Two dietary treatments (0 and 200 ppb Cr) were randomly allotted to individually fed sows such that there were 10 sows per treatment. The response variable was litter size as measured by number of pigs born alive. Upon completion of the experiment, an F-ratio of 1.92 with a corresponding P-value of 0.1818 was obtained (Table 6
). Clearly, this was not significant at a 5% level of significance. The observed difference between means was 0.8 pigs per litter (10.8 vs. 11.6). Using Eq. [2]
, the estimated effect size is calculated to be 0.17. Using the Cohen (1988) guidelines, this is judged to be of small size.
|
The results are Critical F = 4.4139 and Power = 0.1112. Thus, the observed power is approximately 11%. This might be interpreted (Myers and Well, 2003) as follows: given the sample size used in the present experiment, and assuming the mean square error is a reasonable estimate of the population variance, there is a probability of about 0.11 that the null hypothesis will be rejected if treatment effects in the population are of the order of magnitude estimated from the present data. In other words, if the treatment population means are about as different as the sample means suggest, there is still a 0.89 (1 0.11) probability of making a Type II error.
Calculation of the observed power is controversial. Myers and Well (2003) present post hoc power calculations as a useful supplement to hypothesis testing in cases where significant results are not detected. If the observed power to detect an effect of the size found in the experiment is high, it can be safely concluded that the treatment has no effect. If the observed power is low, results will not be sufficient to say there is no effect. Although this sounds reasonable, there is one major problem. As the P-value increases, the observed power decreases (Thomas, 1997; Hoenig and Heisey, 2001; Lenth, 2001a; Quinn and Keough, 2002). In fact, Hoenig and Heisey (2001) demonstrated that the observed power has a 1:1 relationship with the P-value. Therefore, when the P-value is high, the observed power will always be low, and vice versa. It can be shown that when the observed P-value is equal to the preselected value of
, the observed power is approximately 0.50. A good analogy was made by Lenth (2001b): "If my car made it to the top of a hill, then it is powerful enough to climb that hill; if it didnt make it to the top of that hill, then it obviously isnt powerful enough." Post hoc power calculations are like that; they are not necessarily wrongthey just do not contribute new information and may be somewhat misleading.
A more appropriate use of an a posteriori power analysis is for calculation of the minimum detectable difference for a given power. This can be done using results of the last sow reproduction experiment and interpolating from the tabular data presented in the first three tables of Berndtson (1991). Assuming the coefficient of variation in the population was approximately 12%,([
11.2 x 100), at least a 17% treatment response would have been needed for 80% certainty of statistical significance at the 5% level with 10 sows per treatment. Corresponding differences of 20 and 22% would have been needed for experiments of 90 and 95% power, respectively. From these calculations, it is conceivable that rather large treatment differences (two or more pigs per litter) might go undetected in an experiment of this size with this amount of variability. Thus, the nonsignificant result may be due to the true absence of a treatment response or may be a reflection of inadequate power and sensitivity. Results of the experiment should be regarded as inconclusive and the nonsignificant result interpreted cautiously. Rather than claiming no difference between the two dietary treatments, it should simply be concluded that, given the size of the experiment, there is insufficient evidence for declaring a treatment effect.
| Conclusion |
|---|
|
|
|---|
and ß are arbitrary. These should be based on the relative consequences of Type I and Type II errors; more often than not, they are simply conventional values. The minimum detectable difference may also be arbitrary. Ideally, it is the smallest difference that would be biologically or economically meaningful. In practice, this value may be hard to define. Although these admonishments may seem to imply that the number of replicates generated by the power analysis is little more than a guess, the bottom line is that it is the best guess possible. And, the more care that is exercised in obtaining the necessary input valueswhether to be used with reference tables or computer softwarethe better that guess will be. Finally, the discussion herein has assumed that the design and treatment structures appropriate for a given experiment are known and that the only remaining question is the number of replications. As part of the preexperimental protocol, careful consideration should be given to the treatments to be included and the experimental units to which the treatments will be applied. These experimental units are the replicates for conducting the hypothesis test. Also, the design structure should be selected such that adequate error control is provided, given the nature of the experimental material. Investigators should recognize the wisdom of Cousens and Marshall (1987): "Good experimentation depends as much on an understanding of statistics as it does on an unraveling of the biology."
| Implications |
|---|
|
|
|---|
| Footnotes |
|---|
2 Correspondence: 208 W. P. Garrigus Bldg. (phone: 859-257-7553; fax: 859-257-5318; e-mail: daaron{at}uky.edu).
Received for publication July 10, 2003. Accepted for publication September 24, 2003.
| Literature Cited |
|---|
|
|
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |