Where AI gets it wrong in hypothalamic amenorrhea related questions

I've come across four examples in recent days where AI answers to questions related to my work in supporting people in hypothalamic amenorrhea (HA) recovery have been inaccurate enough to be concerning. It's a shame because it makes it even harder to know who you can trust.

1. If using AI as a tool, double check what it's telling you by going to actual websites that have been vetted, and checking sources.

2. Know that some AIs will make up citations (and what the citations say!) - go to PUBMED to see if you can find the actual articles.

3. Chat bots at companies are using AI now. They can also make stuff up.

Example 1 - Incorrect math!

I asked google for the conversion factor for estradiol from pmol/L to pg/mL. The correct answer is that you multiply pg/mL by 3.671 to get to pmol/L. Google AI said to do the reverse, and multiply pmol/L by 3.671. This is just flat out wrong!!!

Example 2 - Hypothalamic amenorrhea likely does NOT cause early menopause

A former client, who discovered that her amenorrhea is due to primary ovarian insufficiency (POI, also called "early menopause") rather than hypothalamic amenorrhea has been trying to understand if her HA-behaviors could have caused the POI (and thus, if it could be reversed). There is a small amount of evidence that having been "underweight" (BMI <17.5 at age 18 or <18.5 at age 35) may lead to a slightly higher risk of early menopause (before age 45). This is analysis of the Nurses Health Study, not specifically hypothalamic amenorrhea, and there may be other causes of being underweight besides HA. Smoking also had an impact. This is the only evidence I have been able to find that might be somewhat relevant.

If you're a data nerd, like me, here are some interesting factoids I pulled out of the paper, "Adult adiposity and risk of early menopause."

Group	Number with early menopause	Total number	Percentage with early menopause	Increased risk (95% confidence interval)
All study participants	2,804	78,759	3.56%	Not applicable
BMI <17.5, age 18	103	1,992	5.2%	54% (24 - 90%)
BMI 17.5 - 18.4, age 18	102	3,198	3.2%	No increased risk
BMI <18.5, age 35	61	1,111	5.5%	59% (21 - 109%)
smoker, BMI < 18.5, age 18	50	777	6.4%	Not calculated

There has not been enough investigation into long term effects of having had HA to be able to say more than that.

In my experience so far, many of the folks from the original HA board "Fertile Thoughts" that answered the surveys for No Period. Now What? are reaching the age of starting to exprience perimenopause now but there has not been any significant number going through menopause in their 40's. So my answer to her was no, there likely isn't a link, and there's also nothing in the medical literature to answer that question. ChatGPT thinks there is a link. A truncated version says (I've put the parts I disagree with in italics, with {number} to indicate my correcponding comments below):

Can Hypothalamic Amenorrhea Become Early Menopause?

-----------------------------------------------------

{1} Yes, hypothalamic amenorrhea (HA) can potentially lead to early menopause, but it doesn't always result in that.

Can HA Lead to Early Menopause?

- Temporary vs. Long-Term Effects: In many cases, HA is reversible, and once the underlying cause is addressed, a woman's periods can return. However, if the condition persists for a prolonged period, {2}it can affect ovarian function and might increase the risk of early menopause.

- Impact on Ovarian Reserve: {3}Chronic hormonal disruptions due to HA can negatively affect the ovarian reserve, potentially leading to earlier depletion of eggs and earlier onset of menopause.

- Premature Ovarian Insufficiency (POI): In severe cases, {4}long-term HA may contribute to a

condition called premature ovarian insufficiency (POI), where the ovaries stop functioning normally before age 40, leading to early menopause.

{5}Preventing Early Menopause:

It's important to treat the underlying causes of hypothalamic amenorrhea, such as reducing stress, adjusting exercise routines, improving nutrition, and maintaining a healthy weight.

{6}Citations from ChatGPT:

1. "Hypothalamic Amenorrhea: Diagnosis and Treatment," The Journal of Clinical Endocrinology & Metabolism (2015).

2. "Premature ovarian insufficiency: pathogenesis and management," Journal of Clinical

Endocrinology & Metabolism (2014).

3. "Impact of hypothalamic amenorrhea on ovarian function and reproductive lifespan," Fertility and

Sterility (2012).

4. "Hypothalamic amenorrhea and premature ovarian insufficiency: A review," The Journal of

Reproductive Medicine (2013).

My responses:

{1,2} We do not have evidence of HA affecting ovarian function. When you have HA your ovaries are not going through the normal follicular waves, where follicles grow to dominance and either ovulate or die back, but the ovarian reserve does not seem to be impacted (in fact, many people with HA are misdiagnosed with PCOS because of HIGH ovarian reserve!) {3} What chronic hormonal disruptions are affecting ovarian reserve? It is a completely open question as to what in fact leads to primary ovarian insufficiency, so the idea that we can point to amenorrhea related "disruption" and say it has an effect is bullshit.

{4} We have no data on long term consequences from HA, so we cannot possibly know that a long term consequence is POI. (Working on getting those data, but certainly we do not have them yet!) (If you currently have HA, please join the REVEAL registry study so that we can get these data!)

{5} Finally, while it is absolutely true that reversing HA using the methods described (nourishing yourself well, cutting high intensity exercise for the time being, reducing stress where you can) is beneficial for short and long term health, we cannot say that any of this will "prevent early menopause".

{6} NONE of these references actually exists.

Example 3 - Pregnancy dating

I was feeling lazy, so asked Google (today, June 9) "If I ovulated on February 19, how many weeks pregnant am I now?" The response?

"...you are currently 35 weeks pregnant. Pregnancy is typically counted from the first day of the last menstrual period (LMP) which is usually two weeks before conception. Since ovulation occurs around the middle of a menstrual cycle, the period after ovulation is the period of pregnancy. Therefore, if you ovulated on February 19th, you would be 38 weeks pregnant, or approximately 35 weeks after conception".

The correct answer is 17 weeks 5 days pregnant, or 15w5d post conception. And what does "the period after ovulation is the period of pregnancy" even mean?

Example 4 - Supplement information

Someone in my support group asked me whether a supplement, "montmorency cherry" might help increase the length of her luteal phase. On the company website it says that it increases melatonin naturally, so she emailed the founder to ask about whether it might improve luteal phase. This was the response:

She shared this with me, and I asked her to email back and ask for the references. Because when you tell someone "research shows..." you should be able to provide that research.

This is what came back:

Cool cool. I'm a little annoyed, because when you provide someone with references, you should provide the actual links so they don't have to do the work of finding them, but fine. So I start with #3 because that seems most interesting and relevant... and what do you know, but I can't find any of these either.

I do find some references that suggest that Montmorency cherries contain actual melatonin, not some substance that magically gets our body to increase melatonin "more physiologically", like this and this.

So I ask her again, for the PUBMED references, and I get back these three articles:

1. Garrido et al from 2013 in the Journal of Nutrition, Health, and Aging, that talks about melatonin in cherries, but NOT about follicular development and luteal function

2. Srinivasan et al from 2011 in Therapeutic Advances in Neurological Disorders that discusses use of melatonin in Parkinson's

3. Losso et al from 2019 in American Journal of Therapeutics, on the topic of improvements in sleep and the components of cherries that might be relevant.

   
   

So NONE of these is actually the article mentioned in the original email, and they don't address any of the points about luteal phase OR "increasing melatonin in a more physiological way".

It seems clear to me that the correspondent at this company was using AI to respond with something that sounds good but is actually complete bullshit.

This all means that you have to be even more careful than before about who you trust.