Developmental dyslexia cannot currently be diagnosed until a child has failed to learn to read as expected. Researchers have sought to find neural measures that may help predict a child’s later reading ability. One of these measures is the mismatch negativity (MMN), an event-related potential (ERP) component elicited by an oddball within a stream of standard stimuli. The MMN is thought to reflect automatic auditory change detection, and has been shown to predict later reading. For the MMN to be clinically useful, its psychometric properties must be further evaluated. In a sample of 147 kindergarten children oversampled for risk for dyslexia, we calculated reliability measures for early and late MMN time windows in terms of different electrodes of interest, stimuli, and response mean amplitude. Subjects were presented with 2 blocks of 1200 trials; one using /ba/ as the standard and one using /da/ as the standard. The early and late MMN were reliable (r = .54 and r = .62 respectively) when measured using alternating trials, but decreased to nearly zero correlation when comparing the first half of each run to the second half of each run. This indicates that the MMN changes considerably over time for children. Reliability did not differ for individuals who were at risk for dyslexia (t(8) = -1.37, p = .21). This change over time may affect the strength of the conclusions we can draw from models incorporating these predictors. We discuss future directions for using this component as a predictor of future reading abilities.