Feasibility of an innovative electronic mobile system to assist health workers to collect accurate, complete and timely data in a malaria control programme in a remote setting in Kenya
© Soti et al. 2015
Received: 30 July 2015
Accepted: 24 October 2015
Published: 4 November 2015
The cornerstone of decision making aimed at improving health services is accurate and timely health information. The Ministry of Public Health and Sanitation in Kenya decided to pilot feasibility of Fionet, an innovation that integrates diagnostics, data capture and cloud services, in its malaria control programme to demonstrate usability and feasibility by primary level workers in a remote setting in Kenya.
Eleven sites comprising one sub-district hospital, ten health centres and dispensaries were selected in three districts of Kisumu County to participate. Two health workers per site were selected, trained over a two-day period in the use of the Deki Reader™ to undertake rapid diagnostic testing (RDT) for malaria and data capture of patients’ records. Health managers in the three districts were trained in the use of Fionet™ portal (web portal to cloud based information) to access the data uploaded by the Deki Readers. Field Support was provided by the Fio Corporation representative in Kenya.
A total of 5812 malaria RDTs were run and uploaded to the cloud database during this implementation research study. Uploaded data were automatically aggregated into predetermined reports for use by service managers and supervisors. The Deki Reader enhanced the performance of the health workers by not only guiding them through processing of a malaria RDT test, but also by doing the automated analysis of the RDT, capturing the image, determining whether the RDT was processed according to guidelines, and capturing full patient data for each patient encounter. Supervisors were able to perform remote Quality assurance/Quality control (QA/QC) activities almost in real time.
Quality, complete and timely data collection by health workers in a remote setting in Kenya is feasible. This paperless innovation brought unprecedented quality control and quality assurance in diagnosis, care and data capture, all in the hands of the health worker at point of care in an integrated way.
Health information infrastructure in developing countries continues to be one of the weakest components among the six building blocks considered by the World Health Organization to be necessary in order to achieve strengthening of health systems . Fortunately most health leaders agree that accurate and timely health information is the cornerstone for decision-making, serving the key functions: data generation, compilation, analysis and synthesis, communication, and use .
A well-functioning and efficient information system is able to track the performance of health systems in general. As noted by Mutale et al. , weak health information systems (HIS) are a critical challenge to reaching the health-related Millennium Development Goals because health system performance cannot be adequately assessed or monitored where HIS data are incomplete, inaccurate or untimely. Making data more complete, more accurate and timelier are imperatives to strengthen health systems as all the other building blocks depend on it. In order to achieve this, capacity of front-line workers at point of care in rural areas of developing countries must be strengthened. This must include provision of tools to assist in data capture and transmission.
Recent efforts towards strengthening information infrastructure have led to the development of the District Health Information System (DHIS). This is aimed at collection of health data and aggregation for analysis purposes to support decision-making. The latest version, DHIS 2, has been adopted by 30 countries in Africa, Asia, Latin America, and the South Pacific. Countries that have adopted DHIS 2 as their nationwide HIS software include Kenya, Tanzania, Uganda, Rwanda, Ghana, Liberia, and Bangladesh. Other mHealth or e-Health systems were reviewed in detail in the mHealth Compendium published by African Strategies for Health (ASH) Project in 2013 .
The quality of data in the DHIS, as well as in other similar systems, is as good as the data that feeds into it from facilities below District. Data capture at point-of-care (POC) in the community, rural dispensaries, health centres, and sub-district hospitals is mostly paper-based and depends on few, often overworked, health workers. Such workers have very little supervision and have no tools to digitally collect data and transmit it to their managers in a timely manner. This dilemma has been recognized and is being addressed by review of various technologies  that are being introduced, as discussed by African Strategies for Health (ASH) in the mHealth Compendium 2013.
Quality and complete data needs to be transmitted, aggregated, analysed, and transformed into information that can be communicated and used for decision making. Digitally generated data makes it easier to transmit, aggregate and analyse. Fionet (Fio Corporation, Toronto, Canada) is an integrated digital health collection and reporting system using mobile devices to help health workers deliver quality healthcare services, automatically capturing and disseminating primary data to a central web portal. Devices currently compatible with Fionet include: android smartphones, tablets and the Deki Reader™. The Deki Reader is a mobile, in vitro diagnostic device that interprets commercially available rapid diagnostic tests (RDTs) for infectious diseases such as malaria, HIV, syphilis, hepatitis B, and dengue, among others. Deki mobile software applications integrate clinical workflow guidance and digital data capture at POC, then transmit geo-tagged records of patient encounters to a secure cloud database. The Fionet portal allows health programme stakeholders to remotely monitor diagnostic performance, adherence to case management protocols, healthcare workers’ activity levels, and other health performance indicators. The performance of the Deki Reader as an in vitro diagnostic tool has been validated [5, 6] in field conditions.
Many health information technologies (HIT), despite being promising, have failed when implementation attempts were made in low-resource settings. Therefore, it is warranted to conduct implementation research studies to evaluate the feasibility and usability of new HIT in developing countries [7–9].
This paper presents results of implementation research to evaluate the feasibility and usability of Fionet system new technology by healthcare workers in remote locations in rural Kenya and to collect feedback on the system from intended Deki users using Fionet.
Selection of field sites and Deki Reader users
The data-capture forms to be used were defined by the investigators, reviewed and approved by the malaria control unit and subsequently programmed into the Deki Readers. Fio Corporation established a secure Portal to which the data from the Deki Readers were uploaded for easy access by programme managers. Malaria Control Unit (MCU) also designed preconfigured aggregation reports and data mining views that were developed by Fio Corporation and installed in the MCU Portal prior to the start of the feasibility implementation study.
Field support to Deki users and programme managers
Field support was provided by an IT specialist of Viva Afya, Fio’s local partner, and by MCU’s IT specialist. Software and hardware-related assistance was provided remotely by Fio’s software engineers in Toronto, Canada.
The feasibility evaluation was undertaken over an eight-week period in May and June 2013.
Feasibility and usability measures
Malaria Control Unit and the investigators agreed on the following feasibility and usability measures and targets: timeliness of data capture, transmission and aggregation of more than 95 % in the first week; completeness of individual datasets of greater than 96 %; accuracy of individual records greater than 96 %; and, proportion of time information services available through Fionet portal (portal to cloud services). Because Fionet is automated, the results of these measures were available from the cloud service database throughout the study period. These target thresholds were considered to be stringent criteria to be able to make a final assessment of the feasibility and usability of the Fionet system.
At the end of the pilot, all 23 users completed a qualitative usability survey on their experience with the Deki Reader. Users were asked to provide feedback regarding three main features of the reader: the user interface, RDT workflow, and test results. The Additional file 1: Appendix provides the survey questions for each of these features. Each response was ranked on a qualitative scale consistent of four options: ‘Very good’, ‘Somewhat’, ‘A little’, and ‘Not at all’.
Training for the pilot evaluation
Fionet training was conducted over a two-day period. The training was divided into three sessions: classroom training, clinical practice and Fionet portal training. Trainees included 22 practitioners and 19 administrators. All trainees attended the classroom and clinical training; in addition, programme administrators attended Fionet portal web portal training.
Classroom training session familiarized trainees with the Deki Reader and its basic operations. Trainees learned how to log patient and test information using electronic survey forms and RDTs. Additionally, trainees were provided with a refresher on RDT processing and the Deki Reader’s RDT processing quality control features. While the Deki Readers initially detected a lot of improperly run RDTs due to too much blood being added, as training progressed, particularly in the clinical practice session, the quality of RDT processing increased and errors were reduced.
Clinical practical training
Clinical practice session divided trainees into two teams and consisted of a full-day clinical practice with the Deki Readers at Ahero Sub-District Hospital and Nyangoma Health Centre. The trainees processed RDTs on real patients and became familiar with the clinical workflow. Additionally, trainees used the Deki Readers to process multiple patients simultaneously in order to accelerate their performance. By the end of the session, all the users were processing multiple patients easily with very few quality errors reported by the Deki Reader.
Fionet portal training
Fionet portal training session familiarized administrators with how Fionet portal stores and reports patient records. Fionet portal trainees learned how to monitor the daily, weekly and monthly test results, run reports, and query and filter data. In addition, trainees were taught how to manage quality issues, including how to identify and intervene when users experienced challenges in processing RDTs.
Descriptive statistics were used to summarize the data collected. Information was analysed both collectively and by participating study site. General demographic data were summarized using common statistical tests, such as average and mean. Deki users were asked to rank how usable the Deki Readers were for three key aspects: user interface, workflow and test results. For each of the items evaluated, users were asked to use a predefined scale designed to capture the level of agreement of the user with each of the presented statements (see Additional file 1). The ‘User Interface’ domain consisted of 14 sentences, the ‘Workflow’ domain had seven questions and the ‘Test Results’ three. For each sentence presented the average response was calculated by assigning a numeral value to the qualitative options as follows: ‘Very good’ 4 points, ‘Somewhat’ 3 points, ‘A little’ 2 points, and ‘Not at all’ 1 point.
Data capture, transmission and aggregation
Summary of demographics of patient population
5 to 15
16 to 45
Since all tests processed by the Deki led to capture of the data and transmission in real time, this reflects the capability of the Deki Reader to assure complete data capture even from remote rural areas.
Timeliness of data capture and transmission
Overall performance in respect to feasibility measures
Summary of feasibility of quality and timely data collection measures
Timeliness of data aggregation as measured by average time difference between completion of dataset at point of care and its arrival in database barring unforeseen mobile network interruptions or connection problems
>95 % within 1 week
96 % within 1 week
Completeness of individual datasets as measured by average number of data fields containing response (includes Ministry of Health official malaria case report forms, RDT results and image when applicable, user and facility information, and meta information such as date, time. Note: GPS location is excluded due to dependencies on weather patterns and building location)
Accuracy of individual records as measured by percentage of data fields containing responses within pre-defined allowable ranges (includes mandatory forms, RDT results and image, patient demographic information, user and facility information, and meta information such as date, time)
Data can be exported in XML, XLS, CSV formats
Proportion of time information services available through Fionet portal
Proportion of Deki unit support queries responded to within 72 h of initiation
Proportion of Deki unit service requests responded to within 72 h of initiation
Usability of Deki devices
The main objective of the present implementation study was to test an innovative system, based on modern communications technology, to assist the Malaria Control Unit activities under real conditions in the public health sector in Kisumu County, rural Kenya. The conditions of the location selected for the implementation study are comparable to conditions in many other malaria-endemic countries, and are characterized by being low-income communities, economically active in small agricultural activities, with scarce means of transportation, and very limited housing conditions, in many cases without basic services (portable water, electricity, sewage system, etc.) but with reasonably good mobile phone network coverage. The results obtained in the present experience can be generalized to other locations in malaria-endemic areas of Africa.
The sound decision-making at all levels of the health system requires reliable, timely and quality information that are presented by location, sex, age, socio-economic status, and other characteristics. At policy level, decisions informed by evidence contribute to more efficient resource allocation and, at delivery level, information about quality and effectiveness of services can contribute to better outcomes. Information systems, particularly at POC, need to be simple and sustainable and not overburden health delivery staff or be too costly to run. Considering the results observed in the present implementation, the Fionet system was demonstrated to be a useful tool to accomplish all of the above mentioned tasks.
In traditional, paper-based, data recording systems, still present in most rural settings in developing countries of Africa, because of the many forms and registers to be filled from as many data sources, data are often inaccurate, illegible and incomplete. It has been very difficult to know whether what is reported is really what the health worker does. Use of Fionet system in the current deployment eliminated the need for paper records by having all the required indicators preprogrammed into electronic forms and pushed into the Deki devices. The health worker used menus that ensured standard protocols are followed and no missed fields in data capture. This in turn helped to obtain completeness of data, including capture of invalid processes of care or testing, which provided an opportunity for remote quality control, support and corrective measures to be taken.
The Deki Reader enhanced the performance of the health workers by not only guiding them through processing of a test, but also doing an automated digital analysis of the RDT, capturing the image, performing on-site quality control of RDT process, giving immediate feedback to the health worker and capturing full patient data for each patient encounter. The Deki Readers transmitted all this data to the supervisors who accessed it through the Fionet portal. Upon noticing care management-related problems, the supervisors gave prompt feedback to the health workers. The supervisors periodically, through log-into the Fionet portal, were able to readily access data collected by each healthcare facility, prepare a report and share it with each of the participating centres. Data captured at point of care was automatically available for review on the Fionet portal through individual records and aggregated reports. These reports could be shared with personnel from all health facilities participating in the implementation study, as well as with higher levels of the Malaria Control Unit management team, Minister of Health officers, etc.
In the implementation effort reported here, Fionet showed many attractive features and was highly regarded by healthcare workers in the field as a useful tool to increase their efficiency, and by supervisors as a great resource for quality control of both diagnostics and patient case management. The cost advantages of the implementation of a system like this can be anticipated since it strengthens the overall healthcare system. A recent cost-benefit investigation considering improvements in diagnostic quality and case management showed that deploying Fionet to strengthen malaria RDT-based case management, results in cost savings per Disability Adjusted Life Years (DALY) averted, in other words, reduced disease burden is observed while spending less per patient . Additional cost savings were found when including improvements in data collection and appropriate treatment. Furthermore, after a successful pilot conducted recently in Meru County, Kenya, the local government is funding a countywide implementation of Fionet covering all level 2 and level 3 health facilities. This endorsement points to the affordability, scalability and sustainability of Fionet (William Muraah, pers. comm.).
Staff need frequent feedback on how routine practices are being performed (mainly diagnostics and case management); but also on utility of data they collect to understand the importance of good quality data for improving health. Capacity building is required to ensure policymakers at all levels have the ability to use and interpret health data. It is also important that health system staff understand the significance of local data for local programme management, and that their needs for strengthened capacity for critical health statistical analysis are met. Local use of data collected at lower levels of the health system is a key step for improving overall data quality. This paperless, mHealth innovation brought about unprecedented quality control and quality assurance in diagnosis of malaria, patient care and data capture, all in the hands of the health worker at point of care in an integrated way. This finding is in agreement with what has previously been demonstrated . The malaria programme in the Kisumu area successfully tested the Fionet system for its ability to enable remote quality assurance and control in areas where otherwise it would have been more costly and cumbersome to perform following current recommendations [12, 13].
The outcome measurements of the study met the expectations in terms of accuracy, completeness, timeliness, transmission, and use of real-time data for feedback and support of health workers in a remote setting in Kenya, as shown in Table 2. Implementation of a system such as Fionet on a larger scale, such as a whole district level, is therefore warranted and being implemented in Kenya. This is a reliable indicator of the feasibility of the system in the hands of intended users, and the ultimate test to determine scalability of the innovation. Because this tool is based on mobile technology, it is expected that its capability will expand as the mobile technology penetration increases in developing countries, especially in Africa .
Innovative technologies can greatly improve the delivery of quality health care in remote locations of low- and middle-income countries. Quality, complete and timely data collection by health workers in a remote setting in Kenya is feasible. Such data can be transmitted, aggregated and used in real time to make service-related decisions and to remotely support healthcare workers to improve the services they provide.
DS, AO, SK, JL, TK, and SF conceived and designed the study. AO, DS, SK, and SF wrote the manuscript. ZH collected and analysed data, and provided comments on the manuscript. All authors read and approved the final manuscript.
This feasibility study was made possible by the support provided by the leadership of the Ministries of Medical Services and Public Health and Sanitation which have since been merged into one Ministry of Health. Many thanks go to the then Director of Public Health and Sanitation Dr S K Sharif who approved the implementation of the feasibility study. The authors thank the team at the then Malaria Control Unit for their participation in the design of study tools, their role in training of health workers and taking charge of the actual field implementation. Special thanks go to Mr John Nyamuni of MCU, and Mr Peter Sirima, the Kisumu County Manager of the project and for approvals from Dr Ojwang Lusi. The heads of the participating facilities and the Deki users in those facilities did an excellent job in demonstrating that Fionet did improve their performance in care for malaria patients in Kisumu County.
The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- WHO. Everybody’s business: strengthening health systems to improve health outcomes, framework for action. Geneva: World Health Organization; 2007.Google Scholar
- WHO. Monitoring the building blocks of health systems: Handbook of indicators and their measurement strategies. Geneva: World Health Organization; 2010.Google Scholar
- Mutale W, Chintu N, Amoroso Ch, Awoonor-Williams K, Phillips J, Baynes C, et al. Improving health information systems for decision making across five sub-Saharan African countries: implementation strategies from the African Health Initiative—Africa Health Initiative Data Collaborative. BMC Health Serv Res. 2013;13(Suppl2):S9.PubMed CentralView ArticlePubMedGoogle Scholar
- mHealth Compendium: African strategies for health project. Technical Report 2 May 6, 2013.Google Scholar
- Shekalaghe S, Cancino M, Mavere C, Juma O, Mohammed A, Abdulla S, et al. Clinical performance of an automated reader in interpreting malaria rapid diagnostic tests in Tanzania. Malar J. 2013;12:141.PubMed CentralView ArticlePubMedGoogle Scholar
- Herrera S, Vallejo A, Quintero JP, Arévalo-Herrera M, Cancino M, Ferro S. Field evaluation of an automated RDT reader and data management device for Plasmodium falciparum/Plasmodium vivax malaria in endemic areas of Colombia. Malar J. 2014;13:87.PubMed CentralView ArticlePubMedGoogle Scholar
- Azubuike MC, Ehiri J. Health information systems in developing countries: benefits, problems, and prospects. J R Soc Promot Health. 1999;119:180.View ArticlePubMedGoogle Scholar
- Rahimi B, Moberg A, Timpka T, Vimarlund V. Implementing an integrated computerized patient record system: Towards an evidence-based information system implementation practice in healthcare. In: American Medical Informatics Association (AMIA) 2008 Symposium Proceedings; 2008. p. 616–20.Google Scholar
- Shirima K, Mukasa O, Schellenberg J, Manzi F, John D, Mushi A, et al. The use of personal digital assistants for data entry at the point of collection in a large household survey in southern Tanzania. Emerg Themes Epidemiol. 2007;4:5. doi:10.1186/1742-7622-4-5.PubMed CentralView ArticlePubMedGoogle Scholar
- Pitts D, Hayford K, Ferro S, Liederman A, Lipson D, Wasser M. Cost-effectiveness of an innovation for strengthening RDT-based malaria control programs in low-resource settings. In: Presented at ASTMH 63rd Annual Meeting, New Orleans, LA, USA, abstract LB-3453 2014.Google Scholar
- Hall CS, Fottrell E, Wilkinson S, Byass P. Assessing the impact of mHealth interventions in low- and middle-income countries—what has been shown to work? Glob Health Action. 2014;7:25606.PubMedGoogle Scholar
- WHO. Universal access to malaria diagnostic testing. An operational manual. Geneva: World Health Organization; 2011.Google Scholar
- Hill Z, Dumbaugh M, Benton L, Källander K, Strachan D, ten Asbroek A, et al. Supervising community health workers in low-income countries—a review of impact and implementation issues. Glob Health Action. 2014;7:24085.PubMedGoogle Scholar