Identifying Safety Issues From the Spontaneous Reporting System

IDENTIFYING SAFETY ISSUES FROM THE SPONTANEOUS REPORTING SYSTEM

TREND AND SIGNAL DETECTION TOOL

Because of the large number of reports, it is almost impossible for the responsible assessors to have an overview about the incoming information based on their experience and memory alone. Different terms are out for the use of computer-based tools in that field: signal generation as well as signal detection. We prefer the term ‘signal detection’ to ‘signal gener-ation’, because the issue is not to generate things that are not there. The issue is to recognise signals when they are present and can be derived from the exist-ing data but which have not yet been detected by individual report review.

The BfArM does not consider a signal detection tool to be a replacement of staff skills and intellec-tual work in that field. It should rather function as a measure of security – a second safety net. It must be emphasised that the output of such electronic tools needs careful review and assessment by medically qualified staff, because these systems also bear the risk to detect false positive ‘signals’. Certainly, these need to be distinguished from the real ones. To build up the above-mentioned ‘second safety net’ effectively, it is necessary to integrate such tools into the user environment of the ADR database. Ideally, different approaches supportive of daily routine work should be installed.

In BfArM’s new system, three strategies have been implemented that are now in the process of evaluation and refinement for future versions:

1. A tool that detects new substance–ADR combina-tions reported within a specified period, i.e. those that have not yet been reported at all or very seldom. The term ‘seldom’ is defined by the number of specified substance–ADR associations reported in the whole database. For example a substance–ADR combinaion that has been reported within the last month, but which has also been reported 200 times in the period before, would not be considered ‘new’ and would therefore be ignored by this tool. Given that the same association has been reported only once or twice before, the system would generate an output. The time covered and the number of reports neces-sary to trigger an output are flexible, i.e. can be tested and adapted to find the right balance between results worth being elaborated on further, and noise.

2. A tool enabling the detection of trends. This approach compares substance–ADR associations of equal sequential time. The output shows whether clusters have emerged or whether a continuous increase of report numbers describing a specified association can be observed. Again, the length of time and limits for total numbers are flexible and are currently in the process of being tested.

3. The proportional reporting ratio (PRR) approach that enables the comparison of a specified substance–ADR association compared with the whole database or with substance–ADR associa-tions concerning drugs of the same ATC group. This approach is quantitative and does not focus on trends or new associations (Evans, Waller and Davis, 1998). This tool would generate an output if a specified substance–ADR association in the database occurs more frequently – beyond a pre-defined threshold – compared with the whole database or ATC group. Again there is flexibility in the determination of thresholds, and tests are necessary to determine suitability and thresholds.

These basic tools will be used for screening. The substance–ADR associations existing in the database for a specified period, e.g. the last quarter, are gener-ated by the computer and are checked against the parameters applied. An output is provided if the crite-ria are met regardless of a specific user question. They may also be used for specific searches, i.e. the user has the possibility of launching a request for a specific substance–ADR combination applying one or all methods described.

Future versions envisaged will focus not only on specific ADR terms but also on term group-ing developed in the CIOMS/ICH Working Group on Standardised MedDRA Queries (SMQs). The SMQs are groupings of terms from one or more MedDRA System Organ Classes (SOCs) that relate to a defined medical condition or area of interest. They are intended to aid in case identification. We expect that the combination of the tools already established together with the methodology of SMQs would enable regulators as well as companies to detect signals with greater sensitivity and hopefully earlier compared with the use of single ADR terms alone.

SINGLE CASE ASSESSMENT

Over many years while BfArM and PEI received spontaneous reports on paper, medical assessors made quality checks of the reports and also a causality assessment of each individual reported case. The cases were categorised according to the WHO scheme used in causality assessment (certain, probable, possible, unlikely, unassessable, unclassified, etc.) (Meyboom et al., 1997). The outcome of the assessment was documented in the case files.

Medical assessors are responsible for and have special expertise in assessing ADRs caused by drugs that belong to one (main) plus, in some cases, to one or more ATC subgroups. The guiding concept behind this is that assessors are generally medical special-ists, e.g. in cardiology, neurology, infectiology, etc., and have the best and complete insight in the related diseases and therapeutic options in that field. This enables assessors to extend their risk-benefit assess-ment from the suspected drug to therapeutic alterna-tives in a comparative way. This is relevant in the risk-to-benefit assessment and in the decision-making process.

In the past, individual case report assessment consti-tuted an enormous workload. In Germany, the revi-sion of national reporting requirements, apart from the very extensive ones laid down in the former Directive 75/319/EC towards the new rules of the EU Direc-tive 2001/83/EC as amended, took place very recently (April 2005). Consequently, BfArM and PEI received a large number of reports on serious ADRs over the past 11 years from EU Member States as well as from third countries.

Following the implementation of Directive 2001/83/EC as amended, into national law that led to a reduced total number of reports and now being able to receive case reports electronically from a rele-vant proportion of large companies, a tremendous change in the character of work of the assessors and of the workload is expected to occur. It is envisaged that the effectiveness of case assessment is clearly increased and resources are much better used for more and complex risk assessments. Identifying signals of safety problems could take place earlier and quicker that would be to the benefit of the consumers.