Snoring at night doesn’t just drive your partner crazy—it might signal a serious problem for your brain.

People with sleep-breathing disorders tend to have poorer memory and trouble paying attention, new research in the Annals of the American Thoracic Society suggests.

In the study, researchers tested 1,700 older adults with at-home polysomnopgraphy, a tool that measures oxygen level and other sleep factors during shuteye, and surveyed them on their sleep habits. Then, the participants took some tests to measure their cognitive function.

The researchers discovered that people with overnight oxygen saturation levels of less than 90 percent—a marker of sleep apnea, a condition that causes you to stop breathing in your sleep—scored significantly worse on tests of memory and attention that those who took in more oxygen as they slept. People who ranked higher on a sleep apnea score also performed worse on those two measures, too.

What’s more, people who reported higher levels of sleepiness during the day were also more likely to show problems with attention and concentration.

The effects of these associations were even greater in participants who carried a certain genetic trait called APOE-ε4—present in about 20 percent of the general population—which raises your risk of developing Alzheimer’s disease.

Sleep apnea—which is often marked by snoring, morning headache, and daytime sleepiness—has previously been linked to poorer brain function. One reason? The lack of oxygen while you sleep may lead to damage in the blood vessels in your brain, the researchers write. And the changing oxygen levels may spark inflammation, possibly resulting in nerve cell loss in certain areas of your brain, which can lead to cognitive deficits.

It’s possible that people with the genetic trait are more susceptible to the brain-damaging effects of loss of oxygen seen in sleep apnea, which may be why the sleep-disordered breathing condition can be more harmful for people who carry it, the researchers believe.

The study didn’t look at whether treating sleep apnea—which is often done with a CPAP machine, which helps force your airway open and allows oxygen in—could reduce the cognitive issues. But the researchers do say that the findings suggest that disorders like sleep apnea may be a “modifiable risk factor” to reduce the progression of typical aging to mild cognitive impairment, and mild cognitive impairment to dementia.

So if you notice any of the sleep apnea symptoms mentioned above—or any of these 7 common signs you actually stop breathing when you sleep—loop in your primary care doctor. The next step would be a sleep study, so your doctor can determine if you have sleep apnea, and if so, how severe it is.