The use of text-mining and machine learning algorithms in systematic reviews: reducing workload in preclinical biomedical sciences and reducing human screening error

摘要: Background: In this paper we outline a method of applying machine learning (ML) algorithms to aid citation screening in an on-going broad and shallow systematic review, with the aim achieving high performing algorithm comparable human screening. Methods: We tested range algorithms. applied ML incremental numbers training records recorded performance on sensitivity specificity unseen validation set papers. The these was assessed measures recall, specificity, accuracy. classification results best taken forward remaining dataset will be next stage review. used identify potential errors during by analysing datasets against machine-ranked score. Results: found that perform at desirable level. Classifiers reached 98.7% based from 5749 records, inclusion prevalence 13.2%. highest level 86%. Human were successfully identified using scores highlight discrepancies. Training corrected improved without compromising sensitivity. Error analysis sees 3% increase or change which increases precision accuracy algorithm. Conclusions: technique error needs investigated more depth, however pilot shows promising approach integrating decisions automation review methodology.