Abstract Background Systems providing computer-based analysis of the resting electrocardiogram (ECG) seek to improve the quality of health care by providing accurate and timely automatic diagnosis of, for example, cardiac rhythm to clinicians. The accuracy of these diagnoses, however, remains questionable. Objectives We tested the hypothesis that ( a ) 2 independent automated ECG systems have better accuracy in rhythm diagnosis than nonexpert clinicians and ( b ) both systems provide correct diagnostic suggestions in a large percentage of cases where the diagnosis of nonexpert clinicians is incorrect. Methods Five hundred ECGs were manually analyzed by 2 senior experts, 3 nonexpert clinicians, and automatically by 2 automated systems. The accuracy of the nonexpert rhythm statements was compared with the accuracy of each system statement. The proportion of rhythm statements when the clinician's diagnoses were incorrect and the systems instead provided correct diagnosis was assessed. Results A total of 420 sinus rhythms and 156 rhythm disturbances were recognized by expert reading. Significance of the difference in accuracy between nonexperts and systems was P = .45 for system A and P = .11 for system B. The percentage of correct automated diagnoses in cases when the clinician was incorrect was 28% ± 10% for system A and 25% ± 11% for system B ( P = .09). Conclusion The rhythm diagnoses of automated systems did not reach better average accuracy than those of nonexpert readings. The computer diagnosis of rhythm can be incorrect in cases where the clinicians fail in reaching the correct ECG diagnosis.