Comparison of Methods

COMPARISON OF METHODS

In this chapter we have concentrated on the use of the BCPNN, partly because it is the most examined system used at present. As mentioned above, in vari-ous centres, different measures are used to quantify the extent to which a certain adverse drug reaction (ADR) is reported in a disproportionate relationship to a certain drug compared to the generality of the database that is standing out from the background of all reports.

There have been a few studies (Kubota, Koide and Hirai, 2004; van Puijenbroek et al., 2002) comparing the BCPNN with other methods. In the van Puijen-broek comparative study, the level of concordance was measured of the various estimates to the measures produced from the BCPNN. The investigation was performed on the data set of the Netherlands Pharma-covigilance Foundation (Lareb), which maintains the spontaneous adverse drug reaction reporting system in the Netherlands on behalf of the Dutch Medicines Evaluation Board. In essence all the other methods highlighted the same combinations as the BCPNN, and indeed more with lower numbers of cases. When the ‘disproportionality’ was based on relationships with four or more reports (about 11% of the Lareb database), all the methods were comparable. It was only at low count values where any difference could be detected.

The above finding is significant. The precise method used for data mining should be based upon the benefits and drawbacks of each. Crucial to the Bayesian method is the initial setting of the a priori probability. How this is set determines the perfor-mance of the BCPNN at low counter values. At the UMC we chose an a priori probability of indepen-dence which is consistent with the WHO definition of a signal and the previous publication (Edwards et al., 1990), suggesting that normally more than one report would be needed to trigger an expert to think that they had found a signal, unless there was some-thing exceptional qualitatively about a report (such as a case with proven, true re-challenge). Moreover the WHO database has many more incident reports than the Lareb database so that as greater numbers of reports are submitted, little time will be lost in find-ing the signal even though the BCPNN requires about three or more reports to trigger.

It is clear that the other methods may be just as suitable as the BCPNN for routine use to identify cases on a continuous basis which deserve follow-up for more information. The trade-off between sensi-tivity and specificity of the other methods, however, needs to be investigated further for predictive value at a practical signal detection level. Table 21.2, taken from the comparisons paper, gives a very good idea of some of the comparative benefits of the methods.