I once had a series of posts I put on here that I offered as corrective to the multitude of “How to Be Scientifically Literate” posts, articles, and infographics that flood my feed every so often. I ended up pulling them because I found them to be clunky and overlong–the exact opposite thing they needed to be to achieve what I had hoped for them.
So, if I am to offer a primer on how to be scientifically literate, it would start here:
- All individual scientific experiments have flaws that limit the size, scope, or generalizability of their finding. This is true regardless of the statistical or probabalistic model used to demonstrate the relationship or its strength, magnitude, and direction. This is even more true in our current scientific climate that rewards findings that reject the null hypothesis and other issues in the “publication bias” world. If you cannot accurately describe the limitations of the study you are citing, then you should not be confident in its findings.
- All findings are subject to statistical noise. That is, even if we know and understand the limitations (and maybe flaws) of the study in question, there is always the chance that this finding is an anomaly. And if the study you are using is the only such one with the finding you need to support your argument (or in the general minority), you probably should be cautious in deploying it.
Knowing these limitations, this is how you should approach a scientific article.
- Ask yourself, “What do I already know about this subject?” and “What do I think is the answer to the question the authors are attempting to answer?” And ask yourself, how confident you are in this conclusion. Don’t allow yourself to give yourself a 100% confident score.
- Ask yourself next, “What is the most direct way to answer that question; what data would I need?” Go ahead and think big, act like you have magic powers that can compel individuals to join your study for free and forever. Act like you have all the money in the world and can buy whatever equipment or hire whatever staff you need to answer the question. (Note: Your powers are limited, you cannot just wish the answer into existence.)
- Then compare your dream study to the real one. How do they differ?
- Finally, are these differences so vast, or the experiment so indirect that they do not even answer the question in any real way? (e.g., are they only really measuring proxy variables?)
If, at this point, you agree (with yourself) that the study in question, though limited, does point a finger toward the answer they seek, compare their findings to what you already thought the answer might be. Does it agree with what you thought they’d find? Disagree?
The study should move your prior opinion one way or the other–make you more confident it’s true (but not entirely) or less confident that it is. How much and in which direction depends on, among other things, your assessment of the relative strengths of this study (compared to the ones that helped you form your original opinion) and its flaws relative to the scientific gold standard.
Note: If the findings agree with what you already believe, you should re-assess the article with the specific intent of trying to see its limitations through the eyes of someone who disagrees with you. You are more likely to reduce the importance of limitations of an argument whose conclusions reinforce previously held beliefs.
The other day, I got into a somewhat protracted battle on Facebook with a fellow who I charitably describe as “absolutely fine with confirmation bias.” The nature of that conversation inspired me to look a bit deeper into this story when someone else posted it as a link in another subthread of the same conversation. You don’t have to read it. The gist of it is this: there’s a Utah neurologist who thinks that living at altitude effects the mood altering/controlling neurotransmitters serotonin and dopamine, and that this is why Utah (and the other mountain west states) have such high rates of suicide. How good is this theory?
There is no questions that these states have high rates of suicide. Pictured to the left is 2012 data from the CDC. You can see that the “mountain west” states are 7 of the top 10, and all mountain west states are in the top 12. If you assume that Oregon’s and Alaska’s high rates are connected to those that live in or near the mountains those states have, then altitude explains 11 of the top 12 highest rates.
It’s starting off as a pretty strong case. So I set off in search of some data. I’ll skip what I found first. I ultimately decided on getting county-level suicide data from here, and county-level elevation data from the here.
So far, so good. Both datasets required a bit of cleaning (I used R) and then I joined the two datasets using a combined State FIPS + County FIPS variable. I ran a simple linear regression using “Average Elevation of the County” as the input and “Crude Suicide Rates per 100,000” as the output. And here it is.
And the theory looks pretty good. There’s seems to be a strong correlation between elevation and suicide. In fact, this model shows this correlation is statistically significant at p<.001. And I can say I used two good, trustworthy data sources: the CDC for mortality data and USGIS for elevation data. I didn’t do anything sneaky statistics-wise. the Q-Q residuals look good and confirm that no major assumptions of the linear model were violated. The only thing you might want for a more robust finding is to control for other variables (like poverty).
And that’s why I thought this was worthy of a post. It isn’t that this data analysis is “bad” per se, but it’s woefully incomplete. Stopping here would be a bad thing to do, not just because I didn’t control for other variables (that are likely more important than elevation, R-squared for this model is only 0.1663).
No, the real problem with this data shows up in the data I found first, from the WISQARS interactive database the CDC recently launched. On there I was able to make a query and generate a map..and theoretically…download the data that generated the map. But alas, this functionality seems to be broken at the moment. So if I wanted to run my own regression (and I did) I had to go get my data elsewhere. Here is the map I generated.
Notice that big white band running north – south in the middle of the country? And all those giant white islands in the sea of brown further west? Those are really important. That’s missing data. And that data is “missing” because the CDC considered it “unreliable” and “suppressed” it. That data is “unreliable” primarily because those counties are either really sparsely populated (so a single suicide would generate an incredibly high rate/100,000 or those counties had too few suicides in the five-year time span to calculate a genuine rate).
That’s really important because that means that the missing data is not random and the reason it’s missing is directly related to the hypothesis under investigation.
What that means is, for the most part, these counties had very few suicides. And (and this is important) most of those counties exist at elevations higher than the 0ft – 1000ft area where the suicides cluster on the left-hand side of the scatterplot.
When I got rid of counties with no data, I went from the +3000 counties the US has down to 483, a loss of around 85% of total counties. In the remaining dataset the lowest suicide rate was 4.7 suicides per 100,000 people. So I ran a simulation where all missing rates were replaced with this number. And what happened? The correlation vanished.
Now this is not an entirely fair way to test this data. But it’s not entirely unfair either, 4.7 suicides per 100,000 is a pretty low rate, but it’s also a much higher rate than any of these counties actually experience (which is why the data is missing). Consistent variations within these ~2500 counties might still lead to a detectable correlation with altitude. But I doubt it.
And I doubt it because what appears to a Utah-based neurologist to be an issue with elevation is probably much more strongly correlated with other features that also correlate with elevation: poverty, rurality, machismo, gun culture, high levels of drug and alcohol abuse–and all of these things, in turn, correlate with suicide in general and can help explain the rise in rates that these states have seen in the last few years. That is, elevation may effect dopamine and serotonin levels, but they were doing that in 2005, 2000, 1995, … and on and on. So we can’t use elevation to explain the rise in rates even if it helped explain the high base rate (which it probably doesn’t.)
One of the recurring sentiments in the discussion of US politics is the notion of the “Republicrat,” this idea that there is no difference between Republicans and Democrats. I’ve hated this idea for a very, very long time. We can score this thing and every time we do, we find pretty stark differences between Republican and Democrat public statements and Republican and Democrat votes.
Trump proposes a new challenge. He is not a Republican. He’s a conservative nationalist. For the most part, those with conservative nationalist tendencies have been in the Republican party, have voted Republican, run as Republicans etc. But they are not particularly well-aligned with the party’s leadership. So it remains an open question where Trump’s coalition will be found.
538 tackles this question with some quick back of the envelope scoring. Good article. Simple methodology. It’s got issues. One major problem is that one thing is pretty clear, even from this read and that’s that it matters which issue is under discussion to really determine the for- and against coalitions. Silver is aware of this issue. He makes it clear he’s looking for aggregate affinities. This just gives us a general picture of the possible coalition-building space.
I want to focus on the partisan divide. Even with this index score, it becomes clear that most Republican senators are closer to Trump than almost all of the Democrats. There is a space where 5 Republicans and 6 Democrats share a space near the middle. The remaining 89% of Senators fall on different sides of the spectrum. And Trump’s coalition is squarely among Republicans.
In traditional measure of partisanship using votes, Collins (ME) and McCain (AZ) regularly fall to the left of the median Republican. And I don’t think McCaskill (MO), Warner (VA), and Donnelly (IN) will surprise anyone as being to the right of the median Democrat. The others? Maybe?
I’ve already assumed that McCain will be a loud voice against Trump and this confirms that his discontent with the president elect exists in more areas than just torture and Russia. I also thought Paul since he’s already been pretty outspoken on some of Trump’s picks (Bolton). So it seems clear to me that those two represent the definite end of his coalition space and it could extend deeper into the Republican party.
So There is a question of which Democrats might jump ship to join him. If Paul-McCain is the actual boundary (4.8), that only leaves Campbell as a potential frequent cross-party-lines Trump voter. (538 thinks Campbell will lose to his run-off challenger Kennedy anyway–both Kennedy and Campbell were scored by 538). Depending on the issue, Warner, Donnelly and Manchin are possible, but unlikely supporters that will need extra convincing–especially Donnelly since he won’t be doing himself any favors in Indiana voting with in-state rival ex-governor and now VP-elect Mike Pence. Collins is running pretty deep into blue team territory (the northeast is weird).
Another confused political reading of the Star Wars universe, this time trying to prove that the Star Wars universe is a “neoconservative” one. The author has three main points of comparison:
- A believe in stark contrasts between Good and Evil
- Only force can be used to defeat Evil (compromise is disaster)
- Mixed feelings about democracy
The author is an expert on neoconservatism, so I tread here cautiously. The question I would have is, provided these are three characteristics of neoconservatism, are they the three most critical? That is, are there *other* -isms to which any or all of these three apply?
Take # 1 and #2 for example.Naive binary thinking (is an essential starting point of all analytical thinking–only after recognizing there are at least two groups can we begin to think there might be more than two. Only in recognizing there are two groups can we begin both contrast –and comparison.) The idea of Ego vs Alter is a flawed but standard way of discussing hypothetical first civilizations). So the idea that there is a stark contrast between good and evil predates neoconservatism. The idea that only force can defeat Evil (and that force is the only true source of morality at all) goes back at least as far as Plato (an certainly predates him). The “neo-” part of neoconservative is anachronistic at best. Philosophies or ideologies that share these characteristics are not all neoconservatism. In fact, philosophies/ideologies that do not share some form of this view are the minority.
On #3 in all cases where total war is present, democracy can be said to be treated ambivalently. This is true even during such times when the democraticness of warring states is unquestioned. Militaries are not democratic. The process of war is not democratic. When countries are fighting wars they are, by definition, not practicing democracy. They have, by definition, set democracy aside for the moment –in their dealings with the enemy. However, it sounds like the Republic (previously the Rebellion and prior to the that The Republic) made it a priority to re-establish democracy in all reclaimed territory after the fall of the Empire. It also seems that the decision to continue to support the Resistance is being made democratically in the newly re-established Republican Senate. It may be true that in the Star Wars universe compromise has been disastrous but that also was true in the real world. The famous example of course is the famous appeasement of Hitler in the run-up to WWII–which happened well before “neoconservatism” was a thing. Point being, the war here is just further evidence that democracies do not fight each other. Had the First Order been a democratic state, it’s possible that the idea of diplomacy would have been appealing to them. They were not and it was not. War was inevitable.
To say that “The Star Wars universe is a neoconservative one” is to claim that the laws that govern (political) action and reaction are the laws that are derived from a neoconservative interpretation of history. The point in restating that is this: “neoconservatism” isn’t derived from nothing. The appeasement of Hitler really did happen. World War II really did happen. So it is possible to derive neoconservative principles from the events in Star Wars–because it isn’t what happens, but how one interprets what happens.
Let me see if I can say it a different way. Neocons would have predicted that the First Order would back out of the compromise and use the Republic’s compassion against it. And then that’s what happened! Neoconservatism!
There are already plenty of real world examples where compromise did not lead to the rise of a Hitler like creature. And yet, neoconservatism came into existence and persists. That’s because neoconservatism espouses general laws, not absolute ones.
It is also true that if Star Wars were built on neoconservative principles, the Republic would never have offered a compromise–much as real neocons are always warning policymakers of the dangers of appeasement.
It’s gotten so bad, that the superficial reading of Star Wars is the contrarian view. For the love of the Force, people, stop trying to reread a fairy tale. It’s Good vs Evil; and, the Good is Good and the Evil is Evil. That’s how it works and that’s why it works.
The goal is raise TWO-HUNDRED AMERICAN DOLLARS between now and December 31. If I do, I will make a mess of my beard and likely mine or someone else’s house by traipsing around on New Years Day with a Glitter Beard.
I can hear you asking, “How can I make a shiny mess of some random Denver bar/One of Jim’s Friend’s Houses?” And the answer is “It’s EASY!!!”
- Go to THIS LINK and hit the Donate Now button on the right.
- Pick an amount.
- In the Comments section, please reference “glitter” or “glitterbeard” in some way. I plan on having several fundraising things happening at once and to make sure your funds go to the appropriate thing, I’ll need *some* way of knowing where you want your funds to go.
- Funds raised between today (11-23) and December 30th and labeled with any reference to glitter, will be applied to the $200 goal.
First to donate $50 or more gets to pick the color. (After that, larger donations can outbid for color. For example, first $50 bidder calls “gold,” a later bidder at $51 can choose “red” instead and, if not outbid, red it stays.) Put the color in the comments as well.
To get a taste of what this entail, check out the video below.
Can I share something with you? I am bad swimmer. I mean really bad. Full disclosure: for someone who works out as much as I do, rides my bike and hikes as much as I do, and runs as much as I do, I am a miserable athlete altogether. But I am the most miserable of all the miserable swimmers out there.
Let me prove it to you.
Some of you have heard this story, but it’s a good one and everyone likes a story that makes a pompous, intellectual dilettante like myself look foolish. So heal yourself with laughter at my expense.
My very first triathlon—in fact my very first “competitive” event of any sort was in 2010. I had just recently started dating the wonderful human who is now my wife. She is an anxious sort who likes running for its therapeutic effects (sort of, I’ll leave her to define her relationship with running). I don’t remember what possessed her to do a triathlon, but I do know that I like to think of myself as the kind of person that runs triathlons. I was 35 years old.
Thirty-five, for those of you who are math deficient, is one year younger than 36…which is the beginning of the second age bracket in the Mighty Mississinewa Sprint Triathlon. So when I showed up on race day I was pleased to find out I would be a member of the first wave: 18-35 year olds and those competing at the “elite” level.
Oh. I should mention that I worked out extremely hard so they I would not qualify for “Clydesdale” status. What is a “Clydesdale?” you ask. That’s the name for somebody who is racing at over 200 pounds. You can put “Clydesdale” on your race status so that those who see your abysmal ranking will know to convert their snorts of derision into sympathetic sighs. I weighed 198 on race day.
The way the waves work is that one cadre of, in my case, extremely fit young men and seasoned racers (and me) enter the Mississinewa Reservoir at the sound of a starting pistol. Every few minutes, the starting pistol cracks again and a new wave of progressively less competitive swimmers enters the reservoir. A sprint triathlon is not very long. The swim is 500 yards (for MMS), or .4k if you’re metric. Point Four K. The average swimmer should be able to complete POINT EIGHT K in less than 14 minutes. So that means, worse case scenario, I should have heard my own wave’s pistol, the pistol for the wave behind, and the wave behind them just as I was about to exit the lake. Which makes that silver capped matron who swam over my sinking body an 20 minutes later…disconcerting.
I don’t know how many waves there were. But I know there were at least 2 waves of men behind me and at least 3 waves of women. I know that the last wave of women were women over 50, denoted by (and I am not making this up) silver swim caps.
That’s right, readers. The average swimmer…not the elite crowd with which my nearly horse-sized body entered the lake …should swim that length in, at most, 14 minutes. But I was nearly drowned by the forceful strokes of a woman who entered the lake no less than 10 minutes later and who caught up with me, presumably no earlier than 9 minutes after that. Indeed, later records show that I finished the race in a staggering 21:54. Which is a score so miserable that I came in last, not only in my own group of the elite and young, but also in the group of all males 30-44. In fact, dear readers, in a field of 197 racers, men and women of all ages, I was 192nd. The only people too come in behind me were a 43 year old, a 49 year old, two 53 year olds, and a 55 year old. The guy (or gal) in 191st place was 60.
I also want to take a moment to reiterate. Some woman in a silver cap swam over my body. I had gone into a backstroke because, basically, after 21 minutes in the pool at 490 yards (give or take) my body just couldn’t take it anymore. I needed to rest and another human being swam over me like I was …jeez…I don’t really know…you don’t really swim on top of anything the way she swam on top of me.
I went under. I literally thought that this was how I was going to die: 10 yards from shore, in a reservoir in northern Indiana, during a race I clearly had no business in.
I’m not sure if I fully appreciate the distance between “humbling” and “humiliating,” but I got closer to understanding that day.
Ladies and Gentlemen, dear readers, I need what Team-in-Training is offering. No joke. And Team-in-Training needs your support. How about following this here hyperlink and helping me raise money to end blood cancers (and also not ruin an otherwise beautiful bay with my drowned body in April).
I have been following the science of determining (or disproving) the “left wing bias” of the media for a long time. I have seen lots and lots articles on the subject with many many charts and graphs in them. And they all basically look like the one here [PDF]. I’m linking to this one because it’s new and…if you’re following this whole “Lying Liar Political Scientist Published an Article with Lies” story, it has the benefit of being relevant to that discussion as well.
If you are following the whole LaCour fiaso read the whole thing. Basically, LaCour probably faked data on a 2nd article as well (and this essay is written by the guy that LaCour jocked the data from). If you’re only interested in the “media’s left wing bias” angle, then direct your gaze to page 5. What that chart is telling you is that, while there does seem to be an apparent “leftish slant” this is far from certain and could actually be flat wrong.
- Most of the studied channels appear to be inbetween the moderate left hump and non-partisan 0.
- There are two channels to the left of moderate left and no channels to the right of moderate right.
However, and this is critical, with the exception of Lou Dobbs’s show, all the confidence intervals (Bayesian credible intervals) overlap 0…which means they aren’t statistically different from “centrist”…and some of them could be “conservative.” Those CIs are huuuuuugggeeee. They basically run the entire gamut of the base dataset. I’ll also mention that most of the CIs (all but three) run most of their difference to the right. I think that’s difficult to interpret, but the consistency of this right skew across channels implies meaningfulness.
Some of this is a limitation to the types of methodologies available to this kind of research. But some of this is likely due to the fact that…well….most channels aren’t interested in 100% alienating 50% of their potential audience and thus…at the very least…attempt to appear neutral if not being, in fact, neutral. In the words of the author:
In the replication, only one show, Lou Dobbs Tonight, has a credible interval that does not overlap zero. Applying LaCour’s criterion, Lou Dobbs Tonight would be classified as “conservative news,” while all the remaining shows would be classified as “centrist news.
One of the stories that got a lot of media time during the fight to make recreational marijuana a legal reality was the one of Haleigh–a young girl who, prior to adopting a CBD regimen to treat her seizures, was having 200 episodes a day and now is closer to 10. That is a huge success. We read a lot out here about all the families that are moving to Colorado to obtain marijuana for their children’s seizure treatment. I would say that, aside from treating chronic pain…especially chronic pain in association with chemotherapy…marijuana’s miraculous power in treating seizures in children is the driving narrative of why marijuana should be reschedule, why every state without a medical marijuana law should get one, and why recreational marijuana should be far more widespread.
However, that same study found that 44% of children taking CBD were suffering negative health outcomes from the treatment including (and this is important) increased seizures.
In a more objective measure of seizure-related brain health, only 3 of the children in the study showed any actual improvement (which indicates that some portion of those 33% of children who saw their seizure decrease by half, may have have been over-reporting the benefit, or may be operating under some placebo effect).
In any case, those last two points are crucial and they will no doubt be completely, and tragically, overlooked.
Medicine is weird; it works for whom it works. The way we determine if a medicine works is to get two groups. Individuals are randomly selected into one of the two groups. One group receives the treatment, the other receives a placebo. If possible, even the administrators don’t know which individuals are receiving which medication and which the placebo. Then we look for improvements in both groups. For various reasons, some people in both groups will get worse and some people will get better and some people may see no change in their condition at all. Ideally we will see more people get better in the medication group and less people getting worse. In either case, the differences observed within each group will be compared to the differences observed in the other group to determine if those change are “statistically significant.” That’s a real rough description of the “double blind, randomized, placebo-controlled, trial.” Basically if the improvements in the medication group are better than the natural improvements seen in the control group, researchers conclude that the medicine “worked.”
But something peculiar happened up there. Something that researchers know about but doesn’t always get translated out. The medicine didn’t seem to have any effect at all on some people. And worse, some got worse.
Some of those differences might be chalked up to an inability to precisely measure changes in the condition. We tend to think of illness as a thing you either have or you don’t, rather than thinking of it in terms of how much of it you have. So it might be that very minor improvements in condition were beneath a threshold where those improvements were observed and reported to researchers. It’s also possible that declining condition in the medicine group would have been more pronounced had the medicine not been present.
But it’s also entirely possible that the medicine helped some people in the medicine group and harmed other people in the medicine group. Humans are all different. Diseases manifest differently in different people, medicines react differently to different people. So when we claim that a medicine “worked” it’s not always entirely clear what is meant. So the medicine works for whom it works and not on anyone else. Ideally, we would give medicine that works only to people that it works on…and maybe even prescribe medicine that “doesn’t work” to the people it works on as well even if it doesn’t “work” any better than chance at the group level.
I’m not saying that the story linked here proves that marijuana is a crappy medicine for children with seizures. It’s entirely possible that marijuana is very good for some children, and maybe not quite as good and possibly harmful for others. The problem is that the rhetoric of these “miraculous” cures is problematic. Parents desperate to find any cure for their children may overlook problems with marijuana in their child, they may see signs of improvements that don’t exist. And worse, they may forego other, effective, treatments while hoping that the marijuana miracle works out.
I think this is an area where we definitely need more research. And marijuana is cheap and, compared to many traditional treatments, safe. So I’m glad that it’s available for parents to try. But I also wish that the conversation we were having didn’t involve the word “miracle” quite so often. And that the potential for no- or bad outcomes was more appreciated.