Background: While atrial fibrillation (AF) is an important risk factor for ischemic strokes and mild cognitive impairment (MCI) in Alzheimer’s disease, the association between AF and post-stroke cognitive impairment (PSCI), and the factors mediating this association, is unclear. Objective: To investigate the role of AF in PSCI, especially in relation to other markers of cerebrovascular disease. Methods: 445 subjects with mild ischemic stroke without pre-stroke cognitive decline were assessed 3–6 months post-stroke for cognitive deficits. MRIs were reviewed by trained raters for acute infarct characteristics, global cortical atrophy, white matter hyperintensities, cerebral microbleeds, and intracranial stenosis. Logistic regression analysis was used to identify factors independently associated with PSCI. Subjects were also categorized according to paroxysmal (pAF) or persistent/chronic AF (p/cAF), and presence or absence of AF or large cortical infarcts (LCI) to study cognitive trends. Results: 80 (18.0%) subjects had AF. 76.3% of AF subjects and 42.7% of subjects without AF had PSCI. The odds ratio (OR) of AF in developing PSCI was 2.31 (95% CI: 1.12–4.75; p = 0.035), after correcting for other risk factors. pAF subjects and AF subjects with LCIs had higher ORs for PSCI. AF subjects performed worse in neuropsychological tasks associated with global cognition, episodic memory, and executive function. Conclusion: AF is a significant risk factor for PSCI, even after correcting for AF-related infarcts. Other mechanisms, such as hypoperfusion, microhemorrhages, and neuroinflammation, may be at play. All stroke patients with AF, regardless of the type of infarction, should be closely monitored for PSCI.