Skip to content

Advertisement

  • Research
  • Open Access

Evaluation of malaria rapid diagnostic test (RDT) use by community health workers: a longitudinal study in western Kenya

  • 1Email author,
  • 2,
  • 1, 3,
  • 4 and
  • 1, 2, 5
Malaria Journal201817:206

https://doi.org/10.1186/s12936-018-2358-6

  • Received: 2 January 2018
  • Accepted: 15 May 2018
  • Published:

Abstract

Background

Malaria rapid diagnostic tests (RDTs) are a simple, point-of-care technology that can improve the diagnosis and subsequent treatment of malaria. They are an increasingly common diagnostic tool, but concerns remain about their use by community health workers (CHWs). These concerns regard the long-term trends relating to infection prevention measures, the interpretation of test results and adherence to treatment protocols. This study assessed whether CHWs maintained their competency at conducting RDTs over a 12-month timeframe, and if this competency varied with specific CHW characteristics.

Methods

From June to September, 2015, CHWs (n = 271) were trained to conduct RDTs using a 3-day validated curriculum and a baseline assessment was completed. Between June and August, 2016, CHWs (n = 105) were randomly selected and recruited for follow-up assessments using a 20-step checklist that classified steps as relating to safety, accuracy, and treatment; 103 CHWs participated in follow-up assessments. Poisson regressions were used to test for associations between error count data at follow-up and Poisson regression models fit using generalized estimating equations were used to compare data across time-points.

Results

At both baseline and follow-up observations, at least 80% of CHWs correctly completed 17 of the 20 steps. CHWs being 50 years of age or older was associated with increased total errors and safety errors at baseline and follow-up. At follow-up, prior experience conducting RDTs was associated with fewer errors. Performance, as it related to the correct completion of all checklist steps and safety steps, did not decline over the 12 months and performance of accuracy steps improved (mean error ratio: 0.51; 95% CI 0.40–0.63). Visual interpretation of RDT results yielded a CHW sensitivity of 92.0% and a specificity of 97.3% when compared to interpretation by the research team. None of the characteristics investigated was found to be significantly associated with RDT interpretation.

Conclusions

With training, most CHWs performing RDTs maintain diagnostic testing competency over at least 12 months. CHWs generally perform RDTs safely and accurately interpret results. Younger age and prior experiences with RDTs were associated with better testing performance. Future research should investigate the mode by which CHW characteristics impact RDT procedures.

Keywords

  • Community health worker
  • Malaria
  • Rapid diagnostic test
  • RDT
  • Kenya
  • Sub-Saharan Africa
  • Longitudinal

Background

Malaria remains a leading public health problem, causing high levels of morbidity and mortality worldwide, particularly in sub-Saharan Africa, where an estimated 90% of all malaria deaths occur [1]. Previously, malaria case management relied on the presumptive treatment of febrile illness with anti-malarial drugs [2]. However, malaria treatment policy has changed and presumptive treatment no longer represents the recommended approach for malaria case management. In 2010, the World Health Organization revised their recommendations to require parasitological confirmation of malaria infection prior to treatment, also known as the ‘test-and-treat’ strategy [3]. This change was precipitated by a declining prevalence of malaria in sub-Saharan Africa [46], evidence suggesting that malaria only causes a proportion of all febrile illness in malaria-endemic regions [79], concerns surrounding anti-malarial drug resistance [10, 11], and improvements in diagnostic technologies [12, 13]. The confirmation of parasitological infection is also important for the management of non-malaria febrile illnesses [2].

Malaria rapid diagnostic tests (RDTs) are immuno-chromatographic tests that detect the presence of malaria antigens released from parasitized red blood cells. RDTs require that healthcare workers obtain a blood sample from a patient using a lancet, place the sample and a buffer solution in a test cassette, and interpret the results. The tests do not require electricity or specialized equipment and return results within 30 min [14, 15]. Systematic reviews have shown that RDTs are accurate [13] and cost effective [16, 17]. The simplicity of these tests eliminates the need for high levels of technical expertise and allows for them to be used by a wide range of personnel, which may dramatically reduce the presumptive use of anti-malarial drugs [18, 19].

Community-based health interventions are becoming increasingly prevalent, especially in resource-constrained settings [20]. They can improve access to health services in medically underserved areas and reduce the burden felt by health systems by task shifting [21]. In Kenya, these interventions are most often implemented by community health workers (CHWs), a volunteer, non-salaried workforce involved in health campaigns, health promotion and referral. CHWs are an integral part of the 2006 Kenya Community Health Strategy and are linked to local health facilities through a government employed community health extension worker (CHEW) [22]. The current strategy for community case management of malaria pairs RDTs and artemisinin-combination therapy (ACT) in the event of a positive test result [23].

Previous work has demonstrated that CHWs can safely and accurately use RDTs after receiving training and instruction [17]. However, concerns remain pertaining to the long-term use of RDTs by CHWs, specifically how to monitor them over time to ensure the quality of testing is maintained. Concerns that have been most commonly noted in literature relate to infection prevention and control measures (e.g., blood safety), the ability to correctly interpret test results, and adherence to protocols [17, 2428]. Furthermore, some speculate that overall performance and skill retention may be lower among those with little or no formal education [27]. Despite these reservations, large programmes relying on accurate malaria diagnosis by lay health persons continue to be implemented in many countries [2931].

To address these concerns, an observational study was conducted amongst CHWs participating in a large malaria RDT programme in western Kenya [32]. The objective of the study was to determine whether CHWs maintain their competency at conducting RDTs over time and if performance differs across sociodemographic characteristics of the CHWs. Specifically, the study sought to (i) evaluate if the ability of CHWs to safely and correctly administer RDTs changes over time; (ii) examine which, if any, steps are performed incorrectly; (iii) determine if performance of the RDT procedure varies with selected characteristics; and, (iv) evaluate if CHWs demonstrate high levels of competence in interpreting malaria RDT results 12 months post-training.

Methods

Study context and site

This study was part of an ongoing, cluster-randomized, controlled trial to evaluate an intervention to improve targeting of anti-malarials [32]. The study ran from June 2015 to July 2016 in 16 communities in Bungoma East sub-county and Kiminini sub-county in western Kenya. Study areas are located in western Kenya roughly 50 km east of the Ugandan border. The sub-counties have a similar malaria burden, predominantly characterized by Plasmodium falciparum with perennial transmission patterns.

Study procedures

Community health worker training

Community health workers affiliated with the Kenyan government, who were part of an ongoing, cluster-randomized, controlled trial, were eligible for the current study [32]; 32 clusters of communities were enrolled, and 16 were randomly selected for inclusion in the trial. CHWs from the selected communities, who had previously received training from the Kenyan Ministry of Health (MOH), received additional RDT training between June and September, 2015 using a validated 3-day curriculum derived from materials from the Kenyan MOH in conjunction with skills-oriented training sessions. Training personnel included researchers from the study team, nurses, public health officers, members of the sub-county health management team, and peer mentors with extensive experience administering RDTs. A total of 271 CHWs successfully completed the training by demonstrating competence in completing the RDT procedure and interpretation of test results, and were equipped with a supply of RDTs (CareStart Malaria HRP2 (Pf); AccessBio Inc, USA), including all the supplies required to perform the tests. The supplies provided included a thermometer, placemat, pen, gloves, lancet, pipette, alcohol swab, cotton, buffer solution, RDTs, sharps box, non-sharps disposal bag, and a water-proof transport bag, allowing them to perform RDTs independently in their own community.

Community health workers offered free RDT testing to febrile individuals or those presenting with malaria-like symptoms. Conditional on a positive test result, CHWs were trained to provide individuals with the positive test with a voucher with the testing details allowing for the purchase of quality-assured ACT medicines at a reduced, fixed price at a participating pharmacy. Pharmacies would only dispense ACT to individuals presenting both the positive test and accompanying voucher with matching details. Those with negative test results or severe illness (of any origin) were referred to a health facility for further testing. The sub-county health team and study team met with groups of CHWs once per month to discuss challenges, replenish supplies, and to spot check RDT storage. RDT procedures were not practiced or observed at monthly supervision meetings.

A sample of 100 randomly-selected CHWs was selected into groups of 10 and received additional training on how to use a small, automated, battery-operated device known as a Deki reader (DR) (Fio Corporation, Toronto, Canada) [33]. These devices were developed to improve the interpretation of RDTs through image analysis software [34]. Briefly, CHWs were instructed to use DRs on 10 successive patients over the span of about 1 month between July 2015 and April 2016. For a DR-evaluated RDT, CHWs recorded a unique study identifier on the RDT cassette, placed it in the DR for a photograph, removed the cassette and performed the RDT. After performing the RDT, CHWs inserted the cassette into the DR once again and used the machine to record his or her interpretation. The DR took a second photograph of the final RDT for automated interpretation of the results. The CHWs interpretation of RDT results was compared with the Deki reader’s automatic interpretation. When all 10 CHWs in the group had conducted at least 10 DR-RDT tests, or the time limit with the DR had been reached, the DRs were rotated to the next group of 10 randomly selected CHWs. This study is described in detail elsewhere [33]. Financial constraints prevented all CHWs from receiving Deki reader training.

Baseline data collection

Baseline data were collected from all CHWs who attended the 2015 RDT training. Data were collected using a standardized RDT observation checklist at the training that was based on one validated through previous work [25]. The checklist divided the RDT procedure into 20 steps in three domains: safety (five steps), accuracy (seven steps), and procedural (eight steps) (Table 1). A member of the training team watched CHWs conduct one RDT procedure on the final day of training, noting whether the CHW performed each step correctly, incorrectly, or not at all. These tests were performed on volunteers of unknown malaria infection status and diagnostic results were confirmed by the training team member. Anti-malarials were provided to any individual testing positive for malaria. Prior experiences with RDTs, time employed as a CHW, prior training in malaria case management, as well as sociodemographic characteristics (gender, age, education level, marital status, formal employment status) were recorded using standardized survey forms.
Table 1

Observation checklist and step-by-step performance of community health workers

No.

Task category

Task

% of CHWs completing each step correctly

Baseline (n = 90)

12 months (n = 103)

1.

P

Assembles necessary materials

92.2

89.3

2.

A

Read RDT expiration date

53.3

52.4

3.

P

Remove contents of test packet

98.9

99.0

4.

P

Write patient’s name on cassette

92.2

94.2

5.

P

Identify patient’s details and date on the RDT cassette

87.8

92.2

6.

P

Explain procedure to patient

66.7

73.8

7.

S

Wear gloves

97.8

95.2

8.

P

Select 4th finger from the thumb of the left hand for blood collection

93.3

86.4

9.

S

Clean finger with alcohol swab and allow it to dry

95.6

89.3

10.

S

Prick finger firmly with sterile lancet

96.7

94.2

11.

S

Discard lancet in sharps bin immediately after pricking finger

95.6

95.2

12.

P

Do not squeeze finger excessively

84.1

88.3

13.

A

Collect an adequate volume of blood with pipette

78.9

87.4

14.

A

Dispense blood in correct well

97.8

98.1

15.

S

Discards the pipette in the sharps box

98.8

92.2

16.

P

Dispose of gloves and cotton wool in non-sharps container

94.4

70.9

17.

A

Dispense correct volume of buffer

97.8

97.1

18.

A

Wait for 20 min

96.6

96.1

19.

A

Read results correctly

90.4

99.0

20.

A

Verify internal test control

97.6

88.4

Time-point

Summary statistics (range)

Mean (SD)

Median (IQR)

Baseline

Total steps correct (0–20)

17.80 (1.65)

18 (17–19)

Safety steps correct (0–5)

4.80 (0.48)

5 (5)

Accuracy steps correct (0–7)

5.39 (0.98)

6 (5–6)

12 months

Total steps correct (0–20)

17.78 (1.89)

18 (17–20)

Safety steps correct (0–5)

4.66 (0.60)

5 (4–5)

Accuracy steps correct (0–7)

6.18 (0.81)

6 (6–7)

P steps relate to procedural aspects, S steps relate to test safety and A steps relate to test accuracy. Of the 103 participating CHWs, baseline checklists for 13 CHWs were unable to be located and were not included in baseline data analysis; 90 baseline observations exist, although some checklists (n = 14) had incomplete observation data resulting missing scores for no more than two steps. For additional information please see Additional file 1

Recruitment of participants for follow-up observations

In June 2016, CHWs who had completed RDT training were recruited for a follow-up assessment. Test quality, defined as a relative per cent change in the completion of a step, may be expected to change from between 4 and 20% over a 12-month period [25]. Assuming that at the completion of their training CHWs performed all 20 steps correctly, the study team was interested in detecting reduction to 90% in the proportion of CHWs performing all 20 steps correctly at follow-up. To do so, a minimum of 90 CHWs were required to estimate a 95% confidence interval for that proportion with a margin of error of 6%. Based on this sample size calculation, a minimum of 90 CHWs were targeted at follow-up, and a total of 105 CHWs were invited to allow for some loss to follow-up. The sample was randomly selected from a complete list of CHWs using Microsoft Excel (Redmon, WA, USA). All CHWs were randomly assigned a number, sorted from lowest to highest, and the first 105 CHWs were invited to participate in the study. At follow-up, 103 CHWs were observed. One CHW who did not participate because she was on maternity leave; another CHW had recently deceased. Baseline data RDT checklist data could not be located for 13 participants (Additional file 1). Sensitivity analyses were conducted to investigate whether these populations differed significantly.

Follow-up data collection

Initial observations and follow-up observations were designed to be as similar as possible to allow for the comparison of observations. Two of the training personnel acted as observers for follow-up collection and received 1 day of training including instruction on how to minimize observer-induced bias.

In an effort to assess CHW performance as close as possible to real-life conditions, CHWs were instructed to report to local health facilities for a 1-day malaria testing exercise. The study team communicated with health facility staff to request that febrile patients be sent to the CHWs for malaria testing, rather than to the clinical laboratory as usual. Using the same checklist as baseline, an observer would watch CHWs conduct one RDT procedure, noting whether the CHW performed each step correctly, incorrectly, or not at all. Observers did not interrupt the CHW except if one of the five safety steps (steps 7, 9, 10, 11, 15) or two accuracy steps (steps 18 or 19) were performed incorrectly. These steps represented an ethical dilemma, as failure to correctly complete them could compromise patient safety or accurate test results.

Following testing, CHWs referred patients back to the health facility with an encounter form where the date, patient’s name, age, temperature at the time of testing, and malaria RDT result were recorded to allow the patient to receive appropriate follow-up treatment. Patients were excluded from the study and referred back to the health facility staff if they were under 1 year old, pregnant, or displaying severe illness. On six occasions, no febrile patient was available for testing. In these instances, volunteers from the research team were recruited, asked to provide informed verbal consent, and used to assess CHW RDT performance.

Community health workers were also requested to participate in an interpretation assessment at follow-up. This assessment involved the interpretation of ten RDTs. The RDTs used in the assessment had been previously conducted and collected by the study team prior to follow-up assessments. The study team selected RDTs to represent positive, negative and invalid test results. To assess CHW test result interpretation, each CHW was asked to interpret each of the tests as positive, negative or invalid. The gold standard for this assessment was the interpretation of the research team.

Data analysis

Checklists and interpretation assessments were collected on paper forms and input to Microsoft Excel. Data were reviewed for consistency and validity to ensure data accuracy and completeness prior to beginning analysis. Data were analysed using STATA version 14.0 (College Station, TX, USA). The percentage of steps performed correctly at baseline and 12 months, as well as counts of errors in total steps, safety steps, and accuracy steps were calculated for both time-points. Univariable and multivariable Poisson regressions were fit to test for associations between error count data at the 12-month time-point and pre-specified covariates (gender, age, education level, marital status, formal employment status, prior RDT experiences, prior CHW work experience, prior training in malaria case management, DR experiences).

The longitudinal analysis included Poisson regression models fit using generalized estimating equations (GEE) with an exchangeable correlation matrix to account for correlation between test scores from the same CHW collected at 2 time-points [35]. Selected pre-specified covariates (gender, age, education level, prior RDT experiences, prior CHW work experience, prior training in malaria case management, DR experiences) were included in the analysis based on their hypothesized relationship to the outcome. This analysis was used to identify factors associated with RDT performance at both time-points (baseline or 12-months).

Sensitivity and specificity of test interpretation were calculated for all tests. Multivariable logistic regression analysis was used to examine the relationship between committing at least one error in the RDT interpretation assessment and selected covariates. Statistical significance for all analyses was defined as P < 0.05.

Ethical approval

Verbal informed consent was obtained from both CHWs and patients, or caregivers in the event of a patient under the age of 18 years. The study protocol was approved by the Duke University Institutional Review Board (Durham, NC) and the Moi University Institutional Research and Ethics Committee (Eldoret, Kenya).

Results

Participant demographics

Sociodemographic characteristics of the 103 participating CHWs are presented in Table 2. They were 43.6 years old on average and most were women (68.9%), were married (82.5%), and a majority (58.3%) had completed secondary school or higher education. Half (51.5%) had prior experience working as a CHW, one-third (32.2%) had received prior training in malaria case management and several (17.5%) had prior experiences with malaria RDTs. Sensitivity analyses showed that no statistically significant differences existed between the populations.
Table 2

Characteristics of participating community health workers (n = 103)

Characteristic

n (% or range)

Gender

 Male

32 (31.1)

 Female

71 (68.9)

Mean age (years)

43.6 (20–69)

 ≤ 39

36 (35.0)

 40–49

42 (40.8)

 ≥ 50

25 (24.3)

Education (highest level attained)

 Primary or less

43 (41.7)

 Secondary or greater

60 (58.3)

Married

85 (82.5)

Formally employed

16 (15.5)

Had prior CHW work experience

53 (51.5)

 Mean work experience (years)

4.1 (0–6)

Had prior malaria treatment experience

33 (32.2)

Had prior malaria RDT experience

18 (17.5)

Characteristics were recorded at baseline observations

Baseline RDT performance

At baseline, each step was carried out correctly by greater than 80% of participants, except for steps 2, 6, and 13 (Table 1). None of these steps were safety steps, although steps 2 and 13 were accuracy steps. The median number of total steps completed correctly was 18 (IQR: 17–19), the median number of safety steps correctly completed correctly was 5 (IQR: 5), and the median number of accuracy steps correctly completed was 6 (IQR: 5–6). The number of errors observed ranged from 0 to 4 errors per CHW.

All safety steps were correctly completed by at least 95% of CHWs at baseline observations. At baseline observations, greater than 80% of CHWs completed 5 of the 7 accuracy steps. The steps most commonly performed incorrectly by CHWs were steps 2 and 13.

Follow-up RDT performance

At follow-up, each step was carried out correctly by > 80% of CHWs except for steps 2, 6 and 16 (Table 1). The median number of total steps completed correctly was 18 (IQR: 17–20), the median number of safety steps completed correctly was 5 (IQR: 4–5), and the median number of accuracy steps correctly completed was 6 (IQR: 6–7). The number of errors observed at follow-up observations ranged from 0 to 11 errors per CHW. An additional file details the errors observed (Additional file 2).

At follow-up observations, greater than 90% of CHWs completed 4 of 5 safety steps. Only step 9, which entailed the cleaning of a patient’s finger, was completed incorrectly by greater than 10% of CHWs. The most common error relating to properly cleaning a patient’s finger with alcohol (step 9) was excessive wiping or scrubbing. One CHW attempted to conduct the RDT procedure without cleaning the patient’s finger with alcohol. On four occasions at follow-up observations, CHWs opened the lancet and placed it down before pricking the patient’s finger, compromising the sterility of the tool (step 10). The most common error regarding the disposal of used lancets was placing it near but not in the sharps box. Only once did a CHW dispose of a used lancet in the non-sharps container.

At follow-up observations, greater than 90% of CHWs completed 4 of the 7 accuracy steps, and greater than 80% of CHWs completed 6 of the 7 steps. The most frequent errors were for step 2, which was completed correctly in 52.4% of the observations. Other errors related to collecting an adequate volume of blood, which was completed correctly by 87.4% of CHWs, and verifying the internal test control which was completed correctly by 88.4% of CHWs.

Poisson regressions modelling the outcome of total error count at 12 months showed that CHW gender, prior RDT training, and age were associated with overall test performance (Table 3). The mean error ratio (MER) for gender was 0.70 (95% CI 0.52–0.96), which corresponds to the mean number of total errors for men being 30% (95% CI 4–48) lower when compared to women. The MER for CHWs 50 years of age or older compared to CHWs younger than 40 years of age was 1.69 (95% CI 1.23–2.31), corresponding to the mean number of total errors being 69% (95% CI 23–131) higher for CHWs aged 50 or older. The MER for prior RDT experience was 0.50 (95% CI 0.31–0.80), which corresponds to the number of total errors being 50% (95% CI 20–69) lower for those with RDT training prior this training program compared to those without prior training.
Table 3

Covariates associated with total error counts at follow-up observations using multivariable Poisson regression (n = 103)

Variable

MER (95% CI)

P value

Male

0.70 (0.52–0.96)

0.029*

Age (years)

 < 40

0 (referent)

 40–49

0.78 (0.57–1.07)

0.181

 ≥ 50 

1.69 (1.23–2.31)

0.001*

Education

 Primary or less

0 (referent)

 Secondary or greater

0.91 (0.72–1.14)

0.395

Married

0.93 (0.67–1.30)

0.683

Formal employment

1.13 (0.78–1.64)

0.519

Had prior CHW experience

0.84 (0.65–1.08)

0.169

Had prior malaria experience

1.37 (0.98–1.91)

0.062

Had prior RDT experience

0.50 (0.31–0.80)

0.004*

Had Deki reader experience

0.90 (0.68–1.18)

0.435

Data are presented for 103 CHWs who were observed at follow-up observations. Mean error ratios (MER) were used to compare CHWs that belonged to categorically different covariate groups (e.g., formally employed vs no formal employment)

* Denotes statistically significant results (P < 0.05)

RDT performance comparison

When compared to baseline observations, a higher percentage of CHWs were observed to correctly complete steps 5, 6, 13, and 19, but fewer CHWs correctly completed steps 8, 9, 15, 16, and 20 (Table 1).

Results from GEE modelling demonstrated that, on average, CHW performance did not differ significantly between baseline and 12 months post-training for all steps and safety steps. GEE modelling (Table 4) showed that the error rates for accuracy steps were significantly reduced by half at 12 months (MER = 0.51, 95% CI 0.40–0.63). Characteristics related to age were associated error rates at a statistically significant level in the GEE model that included both baseline and follow-up observations. Being 50 years of age or older was observed to be associated with a higher number of total errors (MER = 1.39, 95% CI 1.04–1.87) and safety errors (MER = 3.58, 95% CI 1.64–7.81).
Table 4

Covariates associated with error count data at two time-points using generalized estimating equation models (n = 103)

Variable

Total errors

Safety errors

Accuracy errors

MER (95% CI)

P value

MER (95% CI)

P value

MER (95% CI)

P value

Time

 Baseline

1.00 (referent)

 

1.00 (referent)

 

1.00 (referent)

 

 Follow-up

0.88 (0.72–1.07)

0.208

1.65 (0.96–2.82)

0.066

0.51 (0.40–0.63)

0.000*

Gender

 Female

1.00 (referent)

 

1.00 (referent)

 

1.00 (referent)

 

 Male

0.91 (0.70–1.18)

0.497

1.16 (0.64–2.10)

0.626

1.06 (0.82–1.36)

0.656

Age (years)

 ≤ 39

1.00 (referent)

 

1.00 (referent)

 

1.00 (referent)

 

 40–49

0.91 (0.71–1.18)

0.486

1.37 (0.61–3.07)

0.440

0.87 (0.71–1.12)

0.308

 ≥ 50

1.39 (1.04–1.87)

0.025*

3.58 (1.64–7.81)

0.001*

1.24 (0.93–1.65)

0.144

Education

 Primary or less

1.00 (referent)

 

1.00 (referent)

 

1.00 (referent)

 

 Secondary or greater

1.09 (0.86–1.37)

0.475

1.17 (0.64–2.17)

0.599

1.21 (0.97–1.52)

0.097

CHW experience

 None

1.00 (referent)

 

1.00 (referent)

 

1.00 (referent)

 

 Experience

0.94 (0.75–1.17)

0.584

0.75 (0.41–1.37)

0.349

1.01 (0.82–1.26)

0.910

Malaria experience

 None

1.00 (referent)

 

1.00 (referent)

 

1.00 (referent)

 

 Experience

1.10 (0.87–1.40)

0.424

1.30 (0.66–2.53)

0.442

1.21 (0.95–1.53)

0.121

RDT experience

 None

1.00 (referent)

 

1.00 (referent)

 

1.00 (referent)

 

 Experience

0.95 (0.67–1.36)

0.787

1.21 (0.51–2.87)

0.660

1.14 (0.79–1.63)

0.488

Deki experience

 None

1.00 (referent)

 

1.00 (referent)

 

1.00 (referent)

 

 Experience

0.91 (0.71–1.16)

0.450

1.36 (0.75–2.46)

0.318

1.07 (0.85–1.36)

0.537

Mean error ratios (MER) were used to compare CHWs that belonged to categorically different covariate groups

n = 90 for baseline observations; n = 103 for follow-up observations

* Denotes statistically significant result (P < 0.05)

Interpretation of RDT results

Community health worker interpretation of RDTs yielded high sensitivities and specificities when compared to research team interpretation. The overall sensitivity at follow-up observations was 92.0% and the overall specificity was 97.3% (Table 5). The interpretation test also included one invalid test cassette, which all CHWs (n = 103) correctly interpreted. The cassettes most frequently misinterpreted were 2 with faint-positive results. Most CHWs (n = 92) correctly interpreted at least one of these cassettes correctly, and 73.8% (n = 76) correctly interpreted both faint-positive cassettes; 30 CHWs committed at least one error and the number of errors committed ranged from 0 to 4.
Table 5

Correct interpretation of RDT results 12 months post-training for 103 CHWs each evaluating 10 tests

 

True positive

True negative

CHW positive

474

11

CHW negative

41

401

 

Sensitivity

Specificity

 

92.04

97.33

Excludes one invalid test cassette, which 100% of CHWs (n = 103) correctly interpreted

Multivariable logistic regression (Table 6) showed that none of the covariates had a statistically significant impact on RDT interpretation. Use of the Deki reader appeared to reduce the odds of committing an interpretation error by nearly 65%, although this did not reach statistical significance at the 0.05-level (Adjusted OR 0.36, 95% CI 0.13–1.04; P = 0.06).
Table 6

Covariates associated with committing at least one RDT interpretation error using multivariable logistic regression (n = 103)

Variable

Odds ratio (95% CI)

P value

Male

1.00 (0.33–2.99)

0.998

Age (years)

 < 40

0 (referent)

 40–49

0.90 (0.31–2.65)

0.851

 ≥ 50

2.16 (0.65–7.23)

0.207

Education

 Primary or less

0 (referent)

 Secondary or greater

0.61 (0.27–1.38)

0.237

Married

0.64 (0.18–2.23)

0.488

Formal employment

1.92 (0.50–7.35)

0.340

Had prior CHW experience

0.62 (0.27–1.77)

0.443

Had prior malaria experience

1.78 (0.50–6.30)

0.370

Had prior RDT experience

1.15 (0.244–5.48)

0.854

Had Deki reader experience

0.36 (0.13–1.04)

0.060

Discussion

Results of this study demonstrate that CHWs generally adhere to testing procedures, can safely and accurately perform RDTs, and interpret test results correctly 12 months after they receive training. This adds to the growing body of evidence that CHWs perform RDTs at an acceptable level [13], and that these skills are maintained over time [25]. Although CHWs correctly performed most steps in the testing procedure, this study revealed several specific aspects, explaining the procedure and results to the patient, interpreting faint-positive test results, and other steps relating to accuracy, that hold implications for monitoring and improving future RDT use by CHWs.

Regarding safety, training and supervision should emphasize the importance of properly cleaning the patient’s finger, keeping lancets clean, and correctly disposing of used lancets. On some occasions in this study, CHWs were observed to excessively scrub a patient’s finger, place the lancet down after opening it, and placing used lancets and pipettes down after using them in the test procedure. Cleaning a patient’s finger is a necessary step in the procedure, but excessive scrubbing can result in residual quantities of alcohol on a patient’s finger that can impact test results. Once a lancet is opened, it should be held by the healthcare worker until it is properly disposed of in a sharps box. Placing the lancet down prior to use can jeopardize the sterility of the tool, potentially exposing the patient to infectious agents in the environment. Placing a lancet down following testing can result in a potentially unsafe situation where a sharp object that may be contaminated with blood could pose a risk to the CHW or other individuals in the testing vicinity. These actions have not been noted in other CHW studies but could compromise the safety and accuracy of the tests.

The accuracy of the test can be affected by incorrect blood volumes and timing of the test interpretation. In this study, blood collection techniques had improved at follow-up observations. However, observers noted that CHWs collected too little or too much blood on several occasions. Inadequate blood volumes can reduce test sensitivity, while excessive blood volumes can result in staining and obscuring test lines [36]. As opposed to other studies that have noted difficulties with blood collection apparatus [25, 28], the most frequently observed error in this study was inadequate amounts of blood. Researchers have also raised concerns about puncturing technique affecting blood volumes, noting that CHWs may set the lancet on the patient’s fingertip and attempt to push it in, rather than using a stabbing motion [28]. These actions were not observed in the assessments conducted in this study. Another error that was observed, although rarely, was reading RDT results early. This can compromise the accuracy of the test results. Other studies have also noted that reading RDT results early is a frequently observed error [27, 28] and have suggested time-saving motivations as one explanation for this practice [27].

The results showed that experience using RDTs were found to be significantly associated with lower error rates at follow-up observations. Other work has demonstrated that practicing RDT procedures is associated with improved performance [28]. The current study may have also reinforced previous trainings by building on existing skills.

Follow-up cross-sectional analysis and longitudinal analysis of errors rates at both time-points demonstrated that total errors and safety errors were higher for CHWs 50 years of age or greater compared to those younger than 40 years. Taken together, these results support the assertion that, on average, younger CHWs perform RDTs better than those over 50 years of age. The reasons for this are unknown, but potential explanations may be related to physical differences between these populations (e.g., vision), older CHWs attending a greater variety of training and using practices learned elsewhere that do not concur with the practices endorsed in this study, or differences in comfort and facility with new technologies. Other work has shown similar results. Counihan and colleagues [25] found that CHWs 50 years of age and older were less likely to perform as well as CHWs of 40 years or under for some indicators.

CHWs performed excellently when asked to interpret test results, achieving at least a sensitivity of 92% and a specificity of 97%. These results are similar to those reported in most studies that use microscopy as a gold standard for diagnosis [13]. Previous studies have also noted difficulties in the interpretation of faint-positive test lines by CHWs [24, 25, 28, 37], which was thought to be due to age-related vision degradation [24, 37]. While sensitivity was lower for cassettes with faint-positive lines, there was no observed effect of age on RDT test interpretation in this study. An important association was observed between having used a Deki reader and better interpretation, suggesting that real-time feedback on interpretation may improve their ability to read RDTs. Although this finding did not reach statistical significance in this study, future work with larger sample sizes should continue to investigate the potential of this technology to improve RDT techniques.

It is also worth noting that adherence to RDT test results is crucial for achieving positive health outcomes. Ultimately, even if CHWs perform RDTs at high or acceptable levels over time, the expected reductions in malaria morbidity and mortality will only be realized should patients follow treatment protocols. Gathering information regarding treatment adherence following RDT testing by CHWs would be required to fully endorse a community case management policy. Although outside the aims of this study, this could represent one avenue for future research.

This study had several notable strengths including a relatively large sample size for a study involving CHWs, a longitudinal study design, and employing the same observers at both time-points to standardize scoring. However, limitations do exist that may have influenced study results. Perhaps most significant of these concerns is missing baseline data for 13 of the CHWs. These missing data resulted from irretrievable baseline checklists. Although the sensitivity analyses conducted to compare the various populations found the sub-set of CHWs with missing data did not differ significantly from the rest of the study population, these data may have influenced the study findings or introduced biases based on unmeasured characteristics. Second, the presence of observers may have impacted CHW performance. Researchers have discussed how an awareness of being observed may affect the behaviour of study participants. Some suggest that formal observation can lead to a state of anxiety that holds negative consequences for behaviour [38]. Other works note increases in performance during supervision and evaluation [39]. In this study, the presence of observers may have impacted CHW performance in either fashion leading to an anxious state adversely affecting performance, or an increased adherence to preferred testing practices due to an awareness of the evaluation. It is not possible to predict what impact observers may have had on the results of this study [39]. Finally, previous work has noted that the maintenance of testing skills may also be related to regular supervision [25, 27]. CHWs met with study staff once per month, although the RDT procedure was not practiced or observed at these supervision meetings. Although meetings may have influenced morale and performance in a general manner, they likely did not impact the basic skills. This study cannot definitively distinguish the influence of training from supervision conducted by study staff.

Conclusions

This work demonstrates that with proper training, most CHWs maintain their competency at conducting RDTs and interpreting subsequent results over a 12-month timeframe. Error rates are associated with some characteristics such as gender, age and prior experience with RDTs. Future programmes that engage CHWs to conduct RDTs should focus additional supervision and support on older CHWs, should encourage that testing be performed in environments conducive to accurately interpreting faint-positive test results, and emphasize the importance of explaining test results to patients. Additional research should be conducted to investigate the method by which CHW characteristics influence error rates in the RDT procedure.

Abbreviations

ACT: 

artemisinin-combination therapy

CHEW: 

community health extension worker

CHW: 

community health worker

DR: 

Deki reader

GEE: 

generalized estimating equations

MER: 

mean error ratio

MOH: 

Ministry of Health

RDT: 

malaria rapid diagnostic test

Declarations

Authors’ contributions

All authors contributed to the study conceptualization and methodology. MRB contributed to the investigation, data curation, formal analysis, project administration, and drafting of the manuscript; JL and DM contributed to funding acquisition, and project administration; WPO contributed to project administration, supervision, and drafting of the manuscript; LT contributed to the formal analysis and drafting of the manuscript. All authors have edited final version of the manuscript. All authors read and approved the final manuscript.

Acknowledgements

We are grateful for the close support and dedication of the community health workers and clinic staff of Bungoma East sub-county, Kenya, and Kiminini sub-county, Kenya. The authors also thank Edith Nafula, Lillian Nukewa, and Jojes Ambani for their contributions to data collection; Lucy Cayth, Joseph Kipkoech, and Adriane Lesser for their assistance in project logistics; and Alyssa Platt and Larry Park for their knowledge and assistance in data analysis.

Competing interests

The authors declare that they have no completing interests.

Availability of data and materials

The datasets used and/or analyzed during the current study are included in this published article and its Additional files, or are available from the corresponding author on reasonable request.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Moi University School of Medicine Institutional Research and Ethics Committee (No. 0001403) and the Duke University Medical Center Institutional Review Board (Pro00063384). Verbal informed consent was obtained from participating CHWs and patients, or caregivers in the event of a patient under the age of eighteen.

Funding

This work was supported in part by the National Institutes of Health–National Institute of Allergy and Infectious Diseases (NIH–NIAID) Grant Number R01AI110478. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Allergy and Infectious Diseases or the National Institutes of Health. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Duke Global Health Institute, Duke University, Durham, NC, USA
(2)
Department of Epidemiology and Biostatistics, Moi University School of Public Health, College of Health Sciences, Eldoret, Kenya
(3)
Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
(4)
Department of Family Medicine, Moi University School of Medicine, Eldoret, Kenya
(5)
Duke University Medical Center, Durham, NC, USA

References

  1. WHO. World malaria report. Geneva: World Health Organization; 2015.Google Scholar
  2. Yukich JO, Bennett A, Albertini A, Incardona S, Moonga H, Chisha Z, et al. Reductions in artemisinin-based combination therapy after the nationwide scale up of routine malaria rapid diagnostic testing in Zambia. Am J Trop Med Hyg. 2012;87:437–46.View ArticlePubMedPubMed CentralGoogle Scholar
  3. WHO. Guidelines for the treatment of malaria. 2nd ed. Geneva: World Health Organization; 2010.Google Scholar
  4. O’Meara WP, Mangeni JN, Steketee R, Greenwood B. Changes in the burden of malaria in sub-Saharan Africa. Lancet Infect Dis. 2010;10:545.View ArticlePubMedGoogle Scholar
  5. Bhattarai A, Ali AS, Kachur P, Mårtensson A, Abbas AK, Khatib R, et al. Impact of artemisinin-based combination therapy and insecticide treated nets on malaria burden in Zanzibar. PLoS Med. 2007;4:e309.View ArticlePubMedPubMed CentralGoogle Scholar
  6. Sharp BL, Kleinschmidt I, Streat E, Maharaj R, Barnes KI, Durrheim DN, et al. Seven years of regional malaria control collaboration—Mozambique, South Africa, and Swaziland. Am J Trop Med Hyg. 2007;76:42–7.PubMedPubMed CentralGoogle Scholar
  7. Reyburn H, Mbatia R, Drakeley C, Carneiro I, Mwakasungula E, Mwerinde O, et al. Overdiagnosis of malaria in patients with severe febrile illness in Tanzania: a prospective study. BMJ. 2004;329:1212.View ArticlePubMedPubMed CentralGoogle Scholar
  8. D’Acremont V, Kilowoko M, Kyungu E, Philipina S, Sangu W, Kahama-Maro J, et al. Beyond malaria—causes of fever in outpatient Tanzanian children. New Engl J Med. 2014;370:809–17.View ArticlePubMedGoogle Scholar
  9. Okiro EA, Hay SI, Gikandi PW, Sharif SK, Noor AM, Peshu N, et al. The decline in paediatric malaria admissions on the coast of Kenya. Malar J. 2007;6:151.View ArticlePubMedPubMed CentralGoogle Scholar
  10. Wellems TE, Plowe CV. Chloroquine-resistant malaria. J Infect Dis. 2001;184:770–6.View ArticlePubMedGoogle Scholar
  11. Parija SC, Praharaj I. Drug resistance in malaria. Indian J Med Microbiol. 2011;29:243.View ArticlePubMedGoogle Scholar
  12. WHO. Universal access to malaria diagnostic testing: an operational manual. Geneva: World Health Organization; 2011.Google Scholar
  13. Abba K, Kirkham AJ, Olliaro PL, Deeks JJ, Donegan S, Garner P, et al. Rapid diagnostic tests for diagnosing uncomplicated P. falciparum malaria in endemic countries. Cochrane Database Syst Rev. 2011;7:CD008122.Google Scholar
  14. Gerstl S, Dunkley S, Mukhtar A, De Smet M, Baker S, Maikere J. Assessment of two malaria rapid diagnostic tests in children under five years of age, with follow-up of false-positive pLDH test results, in a hyperendemic falciparum malaria area, Sierra Leone. Malar J. 2010;9:28.View ArticlePubMedPubMed CentralGoogle Scholar
  15. Maltha J, Gillet P, Jacobs J. Malaria rapid diagnostic tests in endemic settings. Clin Microbiol Inf. 2013;19:399–407.View ArticleGoogle Scholar
  16. Ruizendaal E, Dierickx S, Grietens KP, Schallig HD, Pagnoni F, Mens PF. Success or failure of critical steps in community case management of malaria with rapid diagnostic tests: a systematic review. Malar J. 2014;13:229.View ArticlePubMedPubMed CentralGoogle Scholar
  17. White MT, Conteh L, Cibulskis R, Ghani AC. Costs and cost-effectiveness of malaria control interventions—a systematic review. Malar J. 2011;10:337.View ArticlePubMedPubMed CentralGoogle Scholar
  18. Bisoffi Z, Sirima BS, Angheben A, Lodesani C, Gobbi F, Tinto H, et al. Rapid malaria diagnostic tests vs. clinical management of malaria in rural Burkina Faso: safety and effect on clinical decisions. Trop Med Int Health. 2010;14:491–8.View ArticleGoogle Scholar
  19. O’Brien C, Henrich PP, Passi N, Fidock DA. Recent clinical and molecular insights into emerging artemisinin resistance in Plasmodium falciparum. Curr Opin Infect Dis. 2011;24:570.View ArticlePubMedPubMed CentralGoogle Scholar
  20. Lewin SA, Dick J, Pond P, Zwarenstein M, Aja G, van Wyk B, et al. Lay health workers in primary and community health care: a systematic review of trials. Cochrane Database Syst Rev. 2005;1:CD004015.Google Scholar
  21. USAID. Community health worker programs: a review of recent literature. Washington: USAID; 2010.Google Scholar
  22. McCollum R, Otiso L, Mireku M, Theobald S, de Koning K, Hussein S, et al. Exploring perceptions of community health policy in Kenya and identifying implications for policy change. Health Policy Plan. 2016;31:10–20.View ArticlePubMedGoogle Scholar
  23. Chanda P, Hamainza B, Moonga HB, Chalwe V, Pagnoni F. Community case management of malaria using ACT and RDT in two districts in Zambia: achieving high adherence to test results using community health workers. Malar J. 2011;10:158.View ArticlePubMedPubMed CentralGoogle Scholar
  24. Mukanga D, Babirye R, Peterson S, Pariyo GW, Ojiambo G, Tibenderana JK, et al. Can lay community health workers be trained to use diagnostics to distinguish and treat malaria and pneumonia in children? Lessons from rural Uganda. Trop Med Int Health. 2011;16:1234–42.View ArticlePubMedGoogle Scholar
  25. Counihan H, Harvey SA, Sekeseke-Chinyama M, Hamainza B, Banda R, Malambo T, et al. Community health workers use malaria rapid diagnostic tests (RDTs) safely and accurately: results of a longitudinal study in Zambia. Am J Trop Med Hyg. 2012;87:57–63.View ArticlePubMedPubMed CentralGoogle Scholar
  26. Hawkes M, Katsuva JP, Masumbuko CK. Use and limitations of malaria rapid diagnostic testing by community health workers in war-torn Democratic Republic of Congo. Malar J. 2009;8:308.View ArticlePubMedPubMed CentralGoogle Scholar
  27. Beisel U, Umlauf R, Hutchinson E, Chandler CI. The complexities of simple technologies: re-imagining the role of rapid diagnostic tests in malaria control efforts. Malar J. 2016;15:64.View ArticlePubMedPubMed CentralGoogle Scholar
  28. Harvey SA, Jennings L, Chinyama M, Masaninga F, Mulholland K, Bell DR. Improving community health worker use of malaria rapid diagnostic tests in Zambia: package instructions, job aid and job aid-plus-training. Malar J. 2008;7:160.View ArticlePubMedPubMed CentralGoogle Scholar
  29. Chuma J, Abuya T, Memusi D, Juma E, Akhwale W, Ntwiga J, et al. Reviewing the literature on access to prompt and effective malaria treatment in Kenya: implications for meeting the Abuja targets. Malar J. 2009;8:243.View ArticlePubMedPubMed CentralGoogle Scholar
  30. Malik EM, Hanafi K, Ali SH, Ahmed ES, Mohamed KA. Treatment-seeking behaviour for malaria in children under five years of age: implication for home management in rural areas with high seasonal transmission in Sudan. Malar J. 2006;5:60.View ArticlePubMedPubMed CentralGoogle Scholar
  31. Littrell M, Gatakaa H, Evance I, Poyer S, Njogu J, Solomon T, et al. Monitoring fever treatment behaviour and equitable access to effective medicines in the context of initiatives to improve ACT access: baseline results and implications for programming in six African countries. Malar J. 2011;10:327.View ArticlePubMedPubMed CentralGoogle Scholar
  32. Laktabai J, Lesser A, Platt A, Maffioli E, Mohanan M, Menya D, et al. Innovative public–private partnership to target subsidised antimalarials: a study protocol for a cluster randomised controlled trial to evaluate a community intervention in Western Kenya. BMJ Open. 2017;7:e013972.View ArticlePubMedPubMed CentralGoogle Scholar
  33. Laktabai J, Platt A, Menya D, Turner EL, Aswa D, Kinoti S, et al. A mobile health technology platform for quality assurance and quality improvement of malaria diagnosis by community health workers. PLoS ONE. 2018;13:e0191968.View ArticlePubMedPubMed CentralGoogle Scholar
  34. Shekalaghe S, Cancino M, Mavere C, Juma O, Mohammed A, Abdulla S, et al. Clinical performance of an automated reader in interpreting malaria rapid diagnostic tests in Tanzania. Malar J. 2013;12:141.View ArticlePubMedPubMed CentralGoogle Scholar
  35. Zeger SL, Liang KY, Albert PS. Models for longitudinal data: a generalized estimating equation approach. Biometrics. 1988;44:1049–60.View ArticlePubMedGoogle Scholar
  36. Luchavez J, Lintag ME, Coll-Black M, Baik F, Bell D. An assessment of various blood collection and transfer methods used for malaria rapid diagnostic tests. Malar J. 2007;6:149.View ArticlePubMedPubMed CentralGoogle Scholar
  37. Rennie W, Phetsouvanh R, Lupisan S, Vanisaveth V, Hongvanthong B, Phompida S, et al. Minimising human error in malaria rapid diagnosis: clarity of written instructions and health worker performance. Trans R Soc Trop Med Hyg. 2007;101:9–18.View ArticlePubMedGoogle Scholar
  38. Donaldson SI, Gooler LE, Scriven M. Evaluation strategies for managing evaluation anxiety: toward a psychology of program. Am J Eval. 2002;23:261.View ArticleGoogle Scholar
  39. McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epi. 2014;67:267–77.View ArticleGoogle Scholar

Copyright

© The Author(s) 2018

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.

Advertisement