How to separate “fake news” from useful information.

Somewhere around 1980 the internet was born, and the digital revolution began. One result of this technological advancement, for all the positives, is that we are drowning in data. But having a ton of data does not equate to a ton of information. In fact, most data has precious little information. If you google data vs. information, you get the following: “Information is a processed, organized data presented in a given context and is useful to humans. Data is an individual unit that contains raw material which does not carry any specific meaning.” https://byjus.com/biology/difference-between-data-and-information . In other words, data is only the unedited and raw state on the way to becoming information you can trust. Further confusing the picture is the fact that a considerable portion of the studies that people rely upon are poorly done. Physicians and patients rely upon randomized controlled trials to help formulate decisions about how to treat their infertility. The final source of confusion derives from the abundance of data which allows a person using that data to pick and choose which data to use to make the point they want to make. It’s called selection bias or confirmation bias and is rampant in both the media and the medical world.

Unquestionably, everyone is swamped with information today. The various forms of media provide an endless amount of data. Does that help or hinder attempts to achieve a pregnancy? The answer will depend upon how a person views predicting the future. Obviously, no one can predict the future and just when you think you’ve got it figured out, someone changes the rules and you’re lost again. So how can you use all of this information that is at your fingertips and actually help yourself? You need to believe in and understand the basic principles of predicting the future. In a sense, you already know all that you need to about statistics and predicting the future. For example, did you watch the weather report last night? Then you have a basic understanding of statistics and predictions. Consider a weather report that says today will be cloudy with a chance of severe thunderstorms. Further suppose that they said there was a 70% chance of severe thunderstorms. Note that they did not say there would be thunderstorms, just that there was a 70% chance of thunderstorms. So, suppose you go through your day and there are no thunderstorms- clouds, yes- thunderstorm-no. Are you unhappy? No, who want severe thunderstorms? You would probably comment on how the weather was “wrong.” But was the weather person wrong? Not really- remember, they did not say there would be thunderstorms; just that there might be thunderstorms- a 70% chance that there would be thunderstorms. So, now you are at work talking with a co-worker and you both comment that the weather person never gets it right. As a matter of fact, you are so sure that they never get it right that you are willing to bet $20 that if the weatherman makes the same prediction on tonight’s weather report that tomorrow there is a 70% chance of thunderstorm, there will in fact not be thunderstorms tomorrow. Your co-worker, having their PhD in statistics, pulls out $20 and takes your bet. Who has a better chance of winning that $20…you or your co-worker? Your co-worker. Will they in fact make $20? Don’t know…only time will tell. But common sense says taking the bet was a smart choice. And that is the major principle of statistics; making smart choices.

The Flip of a Coin 

So how does statistics work? The most common misperception about statistics is that they will tell a person what will happen. Actually, they tell a person what might happen and how likely the event is to happen. Patients frequently ask whether a prediction applies to them. Yes, it does, but that is not what a person wants to know. Most people want to know what will happen to them and statistics cannot answer what will happen to an individual. If a physician tells a person that the chance of having an infection or a bleeding complication from an egg retrieval is about 3 in a 1000, that is reassuring. Unless of course you are one of the three who had the complication and then for you the chance of 3/1000 does not seem so reassuring. Furthermore, many people equate infrequent with not severe. Complications from IVF are rare, but if they occur, they can be quite severe even to the extent that hospitalization and surgery are necessary.  What statistics allows a person to do is to make a risk-benefit determination. What is my chance of being pregnant from the cycle of IVF versus what is the chance of significant harm from IVF? For example, there is actually a death rate for doing egg retrievals for IVF. It is quite low and perhaps incalculable but some estimates are 5/1,000,000. What most people don’t consider is that there is also a death rate from being pregnant. On any given year in the USA, the death rate varies around 16/100,000 live births. In 2008, the motor vehicle fatalities were 18/ 100,000 registered drivers. Interesting, but let’s stop being macabre and get back to getting pregnant. How do you make a smart choice about getting pregnant?

Making smart choices in medicine rests upon the use of predictions. A prediction makes a statement, usually in the form of a percent or odds such as one in ten, in the form of: ‘if you do this, then that is the chance of getting pregnant’. Notice that both types of predictions require two numbers. A percentage is nothing more than one number (numerator) divided by another number (denominator) expressed as a percentage. So, let’s say that you were just born and had never seen a coin nor bet on a coin toss. A somewhat nefarious fellow shows you only one side of a penny and all you see is Lincoln. He says he’ll flip the coin in the air and he bets you a buck that when it lands, Lincoln won’t be on the coin. Again, you were just born, so being somewhat gullible, you think this is a good idea and you bet him the buck. He tosses it and ‘voila!’ it’s Lincoln- you win. Bolstered by your success, when he says he’ll make the same bet if you toss it again, you eagerly take the bet. But wait- what is this? A picture of some stupid memorial? Where did that come from? Now he wins and you learn an important principle. The coin has two faces and a smart choice would have been to have inspected the coin to begin with and determined what was on the other side. Furthermore, tossing the coin does not always guarantee that it will land on Lincoln. So, what about a third toss? Should you take that bet?  Here is where statistics can help you make a smart choice.

Statistics as a game changer

Anyone who understands a coin toss can understand the important points from using statistics to help make a decision. The points to remember are that rare events are actually predicted by statistics, so rather than being wrong they are, in fact, correct. Unless you know both the numerator AND the denominator, a statistic is meaningless. The problem with mass communication and overwhelming data is trying to determine the accuracy of a prediction. For example, suppose you are browsing the web and a 42 year old lady says she got pregnant on her fifth try at IVF. You are 42 and trying to decide if you should do IVF at all. The first problem with a personal story on the web is that it is impossible to tell if the story is accurate or even if it is true. Just because someone put it on the web does not make it factual. But let’s suppose the above is true. What is the chance of a 42 year old person getting pregnant and how fast should it happen? That information is available and a woman 42 years old, just starting to try to get pregnant has less than a 50:50 chance of being successful. Furthermore, if the woman can get pregnant, it happens quickly, usually within the first 6 months of trying. The longer a woman tries to get pregnant and is unsuccessful, the less the chance that she can conceive. So, assume that you have been trying for six months and you are not pregnant. You consult an REI and that person suggests moving straight to IVF. Seems a bit aggressive, but the question is what would be the chance of achieving a successful pregnancy on the first try for a woman 42 years old with a normal evaluation (semen analysis for partner OK, uterine cavity OK, ovarian reserve [hormone testing] OK)? National data suggest that it would be somewhere around 10%. What would be the chance if you did nothing? Again, an estimate is somewhere around 1-3%. Other choices like clomid or FSH- probably not better than IVF and most likely somewhat less successful. Ok, so you do your IVF and don’t get pregnant. That story about five times looms in your mind. Doing a cycle of IVF was not as easy as you thought it was going to be, and you really want to be pregnant now. Your response was not great and the quality of the two embryos that formed was only average. So, what about a second cycle of IVF? The chance is still about 10%. Ok, so you do a second cycle and still no pregnancy. Again, not a great response, only one embryo which look OK but did not result in a pregnancy.  The thought of a third try is almost overwhelming. It is worth it? Use statistics! 

An unappreciated issue with the use of statistics in medicine and especially in infertility is that the rules of statistics require that the chance of having something happen is uniform for the group being studied. The frequently used example is the roll of the dice. With legitimate dice, each time you roll two dice you have an equal chance of get any one of the 12 numbers. Loaded dice are nothing more than dice that have been altered to favor one result over the other. So if you want to roll a 12 and you have rolled the dice 11 times without getting a 12, most of us would assume that last roll would be 12. That is how Vegas makes money on us. The chance of rolling a 12 is the same on the 12th roll as it was on the first 11.  Applying this to a 42-year-old female considering doing IVF, the national data for 2018 was a 10.4% chance of having a live birth with one cycle of IVF. But there are really two different groups of 42 year old women. One group has a few normal eggs remaining and they can create a child. The other group is structurally abnormal and cannot result in the birth of a child. The problem for making a decision is that medicine cannot distinguish between the one with some good eggs and those with no good eggs, and thus no chance of conceiving. If the 42 year old has an embryo that has the correct number of chromosomes, then that embryo has a 50-60% chance of that embryo resulting in the birth of a child. The woman who has embryos that have the wrong number of chromosomes has no chance of having a child. Ultimately, women will reach a point where all of the remaining eggs are damaged. For women who do IVF and do not test the embryos, the more unsuccessful cycles they have, the higher the chance that they will never have a child that is genetically theirs. For women who test the embryos and repeatedly have embryos that are proven to have the wrong number of chromosomes, they too will likely not have a child that is genetically theirs. Currently, there is no accurate way to distinguish the 42 year old who has a chance and the 42 year old who does not have a chance. The bottom line is that the use of statistics to make predictions in medicine has severe limits, due the fact that many medical problems have a number of groups of people with different diseases who display the same symptoms.

Figuring out your choices…and your odds

What are your options? Do nothing, do another cycle of IVF, do a cycle of IVF using donor eggs, adopt. You already know that the chance of getting pregnant on your own is exceeding small, not zero, but really, are you feeling that lucky? Donor oocyte cycles are somewhat successful with a delivery rate around 50% for the first try. Adoption is an option but it is expensive and neither you nor your partner has any genetic input into the child and you do not have the chance to carry and deliver the baby. On the other hand, when done properly, adoption has a very good “success” rate. How about that third cycle of IVF? Estimates will vary, and national data does not present the success rates by number of attempts. The literature that does exist suggests that after two tries for women between the ages of 41 and 43, the success for the third try at IVF is very low and perhaps not different from doing nothing. Knowing the statistics puts the options in perspective. It does not help make the decision because a lot of factors go into a decision about what form of treatment to use, but at least statistics allows a person to rank the options by the likelihood of success.

Critical points to remember about statistics are:

  1. Is the information true?
  2. How much information is available? Was the prediction base upon one event, 10 events, 1000 events?
  3. Statistics cannot tell you what will happen to you, they can only help put options in perspective concerning chance for success.
  4. The least predictive data comes from case reports (i.e. a single person’s story) or opinions. The best predictive data is derived from randomized controlled scientific trials or from combined data from a number of randomized controlled trials.
  5. For many problems, there is no clear-cut answer and thus expert opinion will vary.

In the end, the best you can do is to gather reputable information, combine the outcomes, add the things that are important to you and go for it. The use of statistics can give you the best chance of getting what you want. All statistics comes down to the concept of the coin toss, so don’t let the numbers scare you. Use statistics to achieve success.

Leave a reply