The most prescribed antidiabetic drugs in Indonesian primary health care are metformin or a combination of metformin and sulfonylurea. Studies on metformin have shown various impacts on cognitive decline in patients with type 2 diabetes mellitus, whereas sulfonylurea has been shown to reduce this impact. This study aimed to compare the impacts of metformin and metformin-sulfonylurea on cognitive function and determine what factors affected it. This cross-sectional study was conducted at Pasar Minggu Primary Health Care involving 142 type 2 diabetes mellitus patients taking metformin or metformin-sulfonylurea for > 6 months and aged > 36 years. Cognitive function was assessed using the validated Montreal Cognitive Assessment Indonesian version. The effects of metformin and metformin-sulfonylurea on cognitive decline showed no significant difference, even after controlling for covariates (aOR = 1.096; 95% CI = 0.523–2.297; p-value = 0.808). Multivariate analysis showed age (OR = 4.131; 95% CI = 1.271–13.428; p-value = 0.018) and education (OR = 2.746; 95% CI = 1.196–6.305; p-value = 0.017) affected cognitive function. Since a lower education and older age are likely to cause cognitive decline, health professionals are encouraged to work with public health experts to address these risk factors for cognitive function.