Courtney Plumley (00:00):

We asked teachers how much science, professional development, they’ve had in the last three years, and nearly half of elementary teachers said none.

Eric Cross (00:10):

Welcome to Science Connections. I’m your host, Eric Cross. I am super-excited to be kicking off the third season with the show. This entire season will be exploring the theme of science as the underdog. And we’re gonna make the case for science, by showing how and why it can be used more effectively. In the coming episodes, we’re gonna talk about how science can be better integrated into other content areas like literacy and math, and explore some of the benefits that you might not be thinking about good science instruction. But first, science as the underdog. I bet some of you out there feel like science is the underdog in your community at school. I know I have at times. To kick off this season, I’m gonna talk to two people who really studied this question by looking at the state of science instruction across the US. Eric Banilower is Vice President of Horizon Research and Courtney Plumley is Senior Researcher at Horizon Research. Eric was the principal investigator and Courtney an author of the latest in a series of studies called “The National Survey of Science and Mathematics Education.” We’re gonna dive into the findings of their most recent report to see what the data’s showing us. Please enjoy my discussion with Eric Banilower and Courtney Plumley. Courtney, hello. And thank you so much for joining us.

Courtney Plumley (01:25):

Hi Eric. It’s nice to be here.

Eric Cross (01:26):

And Eric, welcome.

Eric R. Banilower (01:27):

We’re thrilled to be here, so thank you for having us.

Eric Cross (01:30):

I was reading through the report. Four hundred…a very thorough report, 471 pages, I think, as I got it?

Eric R. Banilower (01:37):

And that’s only one of the many reports from that study.

Eric Cross (01:40):

Yeah. You all have done your work, so I’m really excited to to talk to you about this. And on this season of the show, we’re exploring the theme of science as the underdog. And I think a lot of our listeners, we feel like science is an underdog either in their school or in their district. But you’ve actually done some research on this, in a 2018 study, “The National Survey of Science and Mathematics Education.” So I wanna talk about this report. But first I was hoping you can kind of set the stage. How did you come to work on this report, and then, big picture, what were you hoping to find out?

Eric R. Banilower (02:10):

So the 2018 study that you just mentioned was actually the sixth iteration of a series of studies dating back to 1977. And we collect data every decade or so—you know, plus or minus a few years. And really, what we’re trying to do is get a snapshot of what the science and math education system looks like in in the nation. So my role grew. I started working at Horizon in about 1998, after teaching high school for five years in California. And then going to graduate school. And right about that time, the company was doing the 2000 iteration of the survey. And I worked on it with the team here at Horizon. And then we did it again in 2012. And I had a much more prominent role in that study, and became the kind of leader of the study. And in 2018, the most recent version, we just did it again. So the goal of this study is really to kind of examine key aspects of the K–12 STEM education system. And the main audience of the work has traditionally been policy makers, researchers, and practitioners who work at the federal, state, and district level.

Eric Cross (03:30):

So this study, you took kind of a sample size, but it’s reflective of trends that we tend to see across the nation as a whole. Would that be fair to say?

Eric R. Banilower (03:38):

Yes, definitely it is. It is a random sample of schools in the country. So we start with a list of all the public and private schools in the nation, and then do a random sample of those schools, and then work really, really hard to recruit schools to agree to be in the study. And that has gotten harder every time we’ve done the study, for many understandable reasons. And then once we have schools on board, we sample teachers within schools. So we don’t even survey every teacher in a school. It’s really a sub-sample. So that we can make inferences about the nation as a whole.

Eric Cross (04:14):

Makes sense. And so Courtney, what did you find out about the time spent on science instruction in US schools?

Courtney Plumley (04:22):

So, I’m gonna talk about elementary teachers to begin with.

Eric Cross (04:26):

Because that was your past life, right?

Courtney Plumley (04:28):

I am a former elementary teacher, yeah. So that’s kind of where my head is. And that’s relatable for me. Right? So we asked teachers, like, how many days of the week or weeks of the year that they teach elementary school. And fewer than 20% teach science every day of the school year. They kind of do one or two things, for the most part. They teach a couple days a week or they teach every day of the week, but only for, like, maybe six weeks, and then they swap with social studies and they kind of do that across the school year. Which is really different from, like, math, right? We also asked elementary teachers, how often do they teach math, and it’s every day of the year. Then we also asked them how many minutes they teach when they’re teaching, and we kind of did the math to figure out, all right, if they taught science every day of the school year, how many minutes would it be in a single day, so that we could make a more comparable comparison with math and ELA. If you were to work it out, how many minutes of science an elementary teacher teaches across the year, and break it down to per day, it’s like 18 minutes for the lower elementary grades, 27 for the upper elementary grades. Which is not a lot. But it’s pretty much an hour a day in math, and 80 plus minutes in ELA. So, a lot less. And then, you know, when I was teaching, the first thing to go was always science, right? If there was an assembly, if there was early release or whatever, that was the first thing to go. So those numbers might even be higher. Just because they aren’t factoring that kind of thing in, too.

Eric Cross (06:05):

So, now I’m curious. That is something that I’ve seen just anecdotally, science being the first thing to go. I feel like I’ve seen that almost…it’s almost become a meme, that I’ve heard that so often. Just in your experience, why do you think that is that huge disparity between the two?

Courtney Plumley (06:26):

Well, I mean, when I was teaching, I was teaching third grade. I had an end-of-grade test in math and ELA for my kids. I didn’t have one in science. So the administration said, “Hey, if you’re gonna drop something, drop something that’s not tested.”

Eric Cross (06:41):

Simple as that. And Eric, you, past life: physics teacher. High school. What did you see? ‘Cause our listeners run the gamut from elementary all the way up to high school. What did you see, as far as relative science instruction in the secondary level?

Eric R. Banilower (07:00):

Sure. You know, secondary is just a whole different situation than elementary. Rght? Because you have departmentalization. I taught science. I didn’t have to teach other subjects. And students had periods, and they still do, sorry, they still have periods, even though it’s been a long time since I taught. And you know, they rotate from one class to another. So all the classes were essentially the same length. So, you know, when I was teaching, it was about 50-minute periods. So in terms of minutes of a class or minutes on a subject, it’s not really different. But what is different is what students are required to take in order to graduate high school. One of the things we asked schools about in this study was how many years of a subject do students have to take in order to graduate? And what we saw was in mathematics, over half the schools in the nation require students to take four years of mathematics to graduate. OK? And the vast majority of the rest, about 44%, require three years in science. Most schools require three years. Very few require four years. And many, or a fair number, still only require two years to graduate. So the expectation of what students are taking is lower in science than it is in mathematics.

Eric Cross (08:20):

So you were seeing the same trend in secondary, essentially.

Eric R. Banilower (08:24):

Yes.

Eric Cross (08:24):

The amount of time devoted to the instruction of science…we’re kind of seeing it mirrored just across K–12 across the board.

Eric R. Banilower (08:33):

That’s correct.

Eric Cross (08:34):

And that’s across the country. ‘Cause the sample size represents teachers from Alaska, Hawaii, the South, SoCal, everywhere. So what’s been the reaction to that number? Like 18 to 20 minutes is…I mean, it’s, it’s half of my lunch at our school. What’s been the reaction to that number since this data has been published?

Eric R. Banilower (08:58):

I don’t know, Courtney, if you want to take that…

Courtney Plumley (09:00):

It’s a lot of what you just did. Like, what??? Like, how is it possible to teach all the things you need to teach in such a little amount of time?

Eric R. Banilower (09:08):

What’s really kind of surprising to me, though — though now that I’ve worked on three iterations of the study, it no longer surprises me, but it did at first — is that these numbers really aren’t changing since we’ve started doing this study. You know, people thought maybe with No Child Left Behind and the increase in accountability, time on science might actually go down, because there was more testing in math and English Language Arts. It didn’t happen. It was pretty much constant, that this has been kind of the state of science education for a long time.

Eric Cross (09:44):

So Eric, if I’m hearing you right: The past studies, we’re not seeing an increase or a decline. This has been this way for how many years, roughly, would you say? Since it’s been studied?

Eric R. Banilower (09:54):

You know, I’d have to go back to the 1977 report to get the numbers, but I’m gonna say since then, it has not changed much, if at all.

Eric Cross (10:03):

So this has kind of been entrenched. This has been the norm for almost for the career of a teacher, almost generationally. We’re looking at anyone who’s been in the highest levels of leadership to someone just entering the classroom, this has been the way it’s always been. This is kind of for many people what they’ve only known.

Eric R. Banilower (10:20):

Right.

Eric Cross (10:21):

Kind of become the norm.

Courtney Plumley (10:21):

We didn’t even have science when I was in elementary school. We had science on a cart that came by, you know, every other week.

Eric Cross (10:28):

Was that like a food truck, but like the science version of it? It shows up and does quick science and takes off?

Courtney Plumley (10:35):

And New York was, I mean — we always watched Voyage of the Mimi. I don’t know if you ever watched that. But that’s what we watched every single time the Science on the Cart came. So it’s like a marine biology show. Ben Affleck was on it when he was a kid.

Eric Cross (10:48):

<laugh> Really? For me it was, Mr. Wizard. For some of my students, even now, Bill Nye. You know, the Bill Nye show or something would come on. So what happens when you look at less wealthy districts? Is there a relationship between community resources and science instruction, or is it pretty much equal no matter what the district resources are, the school’s resources are? Did you see any data there?

Eric R. Banilower (11:12):

Yes. We actually did a lot of disaggregating the data by community type, student demographics in the schools, to look to see whether there were areas of inequities across the country. And, you know, one of the factors we looked at was kind of a measure of socioeconomic status. You know, wealth in the community. By looking at percentage of students eligible for free or reduced-price lunch. And interestingly, in terms of time on science instruction, there is actually not a relationship between income level and how much time is spent at the elementary level on science, which actually surprised us.

Eric Cross (11:54):

Because you might have expected it to be the other way now. And granted, it’s 18 to 20 minutes, there isn’t much more to shave off off of that. But were there other differences, like when you compared those communities? Maybe it wasn’t the amount of science instruction, but was there anything else, like teacher preparedness, resources? Were there anything else that you did see discrepancies in? Or was it equal across the board?

Eric R. Banilower (12:13):

No, unfortunately there, there have been, and still are, a number of areas where community resources are related to pretty substantial differences in educational opportunities that students have. So, you know, we’re talking about the high school science requirements. One of the things that we saw was that high schools in less wealthy communities tend to offer less rigorous science courses than high schools in better-off-financially communities. So they may not be AP courses or second year advanced courses to the same extent that there are in the wealthier communities. That’s one big difference that we saw. Another one was what you were just saying about, sort of, the teachers who teach in these communities. You know, I think that for many years people have had a feeling that the best teachers go to the better off schools because it’s easier to teach there. Well, we see that the schools with the most poverty, they tend to have the newer teachers, who are just starting their career. They tend to have teachers who are less well prepared to teach their subject. And there’s a host of other differences we found. And you know, you mentioned the report being 400 pages. This other report that looks at these differences is also quite long, and, you know, identified a number of areas where there are these disparities in the system.

Eric Cross (13:43):

Well, we appreciate you synthesizing this for us, because this is super-important. And you’ve fleshed out a lot of things. And the fact that it’s driven by data, we as science teachers, we as scientists, being objective, really, really value that. Because this is actually validating a lot of the things that our listeners and myself, we experience anecdotally. But you don’t have a lot of things to network you. And sometimes, when you see this, you wonder if it’s just you, or is are other people experiencing this? And so as you start talking about this data, realizing, oh wow, this is not something in isolation. This is systemic. This is something that’s impacted. And then Eric, what you said about schools that were lower-income, that were under-resourced, and didn’t offer those advanced classes, what are some of the impacts of that, maybe downstream, of doing that? Not having those AP classes? I just kind of wanted to put that out there and ask you.

Eric R. Banilower (14:31):

You know, this is a really…this is a current debate right now, about what the goals of schooling K–12 should be. You know, are all kids meant to go to college? Should there be alternative paths? And you know, I know when I was teaching, I would have students say, “Why do I need to know this? I’m not gonna go into science. I’m not gonna study physics. Why do I need to take this?” And, you know, the answer I used to give them was, “You never know where your life is gonna end up and what opportunities you’ll have. And by having these educational experiences, you have more opportunities available to you. Whether or not you choose to go down those paths, you have opportunities. And when you don’t take this kind of coursework, you know, even if you don’t want to go to college, you limit your potential careers. Because so many careers nowadays require some technical knowledge, some knowledge of science, even if it’s not explicitly a science job. It is embedded in our society now. We are a technological and science-based society.”

Eric Cross (15:37):

It reminds me of something that I’ve told my students, that if you become a scientist, that’s awesome. I love that. But if you don’t, and you want to be a dancer or an actor or a lawyer or anything that may not be directly related to STEM, I want you to choose it because it was a choice, and not a lack of options. So as long as you’re choosing not to go in STEM, and you don’t make that decision because you can’t, or because you weren’t given the opportunity. So that’s how I’ve always had this mindset as a teacher. And I’ve explained it to my students. So if you say, “Cross, you know what I want to do, I wanna be an awesome chef,” which, you know, low-key that’s science, right? <laugh> Molecular gastronomy, we know that. But like, you be the best chef. But as long as you’re being a chef because you choose that, and you’re like, “I love science, but I don’t wanna go that direction,” we’re good.

Eric R. Banilower (16:26):

Right. And if you think about, a lot of social justice issues with pollution and climate change, and you look at which communities are more affected by some of these larger environmental problems and challenges, it tends to be the lower socioeconomic communities, the more poverty-stricken communities have worse water, have worse air quality. And so if, if people from these communities are going to make informed decisions about who they’re gonna vote for, about what policies they’re gonna support, those are science topics that you have to have some understanding in order to make informed decisions in your life.

Eric Cross (17:09):

Courtney, you were one of the Swiss Army Knife teachers. This is how I perceive it for elementary. You had to teach everything. And shout out to all of my elementary school teachers that have to be mathematicians and grammar whizzes and scientists and PE instructors and social emotional, all of those different things. you also looked at teacher preparedness. How did teachers feel about teaching science compared to other subjects like language arts and math? Did you see anything there?

Courtney Plumley (17:39):

We did, we did. And I’m glad you said, “How did they feel about it?” Because one thing that, you know, in a survey you can’t really do is capture how someone actually…how good someone actually…the quality of someone’s instruction. But you can ask them how prepared they feel. And you can even ask them like stats, like, “What did you major in in college?” You know. But you really are going on based on what what they say. So we ask them how prepared they feel to teach all the core subjects. And two-thirds of elementary teachers felt very well prepared to teach reading. They felt very well prepared to teach math. But when it comes to science, it’s less than a third felt very well prepared. And you know, like you said, when you’re teaching elementary school, you’re teaching all the subjects. But also in science, there’s usually four main instructional units in a school year. And they’re all from different science disciplines. So not only are you going on, like, “Maybe in college took a lot of bio classes, but I didn’t take any physics classes, and now I have to teach physics to my kids and I have no experience there.” So, you know, we also ask them how well-prepared they felt in these different disciplines. And the numbers are even smaller, you know. Fewer than a quarter felt very well-prepared in life science. And like 13% felt very well-prepared in physical science. So there’s definitely a big difference between how much teachers feel prepared for ELA and math versus science.

Eric Cross (19:08):

And just from a human perspective, when we don’t feel prepared for something, we’re not really gonna probably lean into it as much as we are into our strengths. Like, that’s just kind of how we are across the board.

Courtney Plumley (19:18):

Yeah.

Eric Cross (19:18):

I’m even like that with my own chores in the house. Or when I have things I need to get done, and I might not be as good at doing those things—it’s gonna be a heavy cognitive load; I’m gonna have to do some background research—I tend to find other areas to excel in. Like, I’m gonna be productive in this other area. I’m gonna really crush it here. But this other thing gets put to the back burner.

Courtney Plumley (19:36):

Totally. And the same reason I might skip science today, <laugh> ’cause it’s scary.

Eric Cross (19:41):

Yeah, exactly. But I love this book. <Laugh> Or we could do this math, and let’s really, really dive deep into it. Now, did you also look at professional development and instructional resources that are being provided?

Courtney Plumley (19:53):

We did.

Eric Cross (19:54):

And on the whole, how was the amount—and I’m seeing a trend here, so I’m kind of feeling like I know where this might go—but I wanted to ask it, did the amount of professional development and resources for science, was there much of a difference between that and other subjects?

Eric R. Banilower (20:10):

Well, I’ll start on this, and Courtney, feel free to jump in. You know, one of the things that we asked was how much kind of discretionary funding do schools devote to science and how much to mathematics? So, for consumables or equipment and supplies or computer software for teachers to use in the classroom. And it’s hard to compare, I think, across subjects because the demands for this kind of supplies, et cetera, is very different, I think, in science than it is in mathematics. Right? We have a lot of, you know, equipment for doing investigations, consumable supplies in science. And those things need to be replenished on a regular basis. It turns out, when we look at the data for school discretionary spending on this kind of stuff, the median school spends less than $2 per student at the elementary level on science, compared to over $6 for mathematics. At the high school level, it’s kind of reversed. Schools spend more money on high school science than they do on high school math. but even still, at the high school, it’s less than $7 per student. Which is not a lot of money being devoted to thinking about all the materials, supplies, chemicals, et cetera, that you need to teach science well, at the high school level. More disturbing is the fact that, you know, we were talking about inequities before, schools that serve less well-off communities spend less than schools that serve wealthier communities, by quite a big amount.

Eric Cross (21:46):

So essentially the per-student thing just kind of popped out to me: So, like, an expensive Starbucks drink is what we’re spending on science per student.

Eric R. Banilower (21:57):

At the high school level. Yes.

Eric Cross (21:58):

At the high school level. And I get those catalogs in the mail, from all of those big science companies. You can’t get much for seven bucks. At least, nothing high-level. And I know I do a lot of 99-cent store science. I go down the street, go to the 99-cent store. Thankfully we could do a lot of awesome science with just, you know, cheap things. But a lot of the higher level experiences, they’re pricey. But the experiences are so rich! And $7 at the high school level is nothing. It’s not much at all.

Eric R. Banilower (22:28):

Yeah. It is definitely, you know, kind of shocking to think about what we’re investing in our children’s future.

Eric Cross (22:37):

Now, just to put you both on the spot, ’cause I feel like that we’ve identified some…we’re seeing a trend here, we’re seeing a pattern. We’re talking about, you know, being science teachers. There’s a pattern going on here. Do you think it’s fair to characterize science as the underdog?

Courtney Plumley (22:52):

I think in elementary school, it is a fair statement. Because, like we said before, I mean they’re gonna preference math and ELA almost all the time. I mean, the other thing you’d asked a little bit ago was about professional development, too. And we do have some data on that. And we ask teachers, you know, how much science professional development they’ve had in the last three years. And nearly half of elementary teachers said none. And I know I didn’t have any science professional development. If I was gonna pick from among the catalog, I was picking one that I needed more, like math. Math and ELA. I keep making that statement, but just over and over, it’s the truth.

Eric Cross (23:31):

And going back to what you said earlier, because that’s where the accountability was, right? And that kind of came top-down.

Courtney Plumley (23:38):

Yes.

Eric Cross (23:38):

And influenced everything else.

Eric R. Banilower (23:40):

Yeah. Now, really interesting thing that we did, a year or so ago, ’cause someone asked us, you know, “Hey, could you look at this?” is we compared elementary science instructional time among states where science counted towards accountability versus states where science doesn’t count towards accountability. And at the upper elementary grades, more time was spent on science in schools in states where they had science accountability. Now I’m not arguing for adding science to accountability systems. But that’s a pretty telling piece of data.

Eric Cross (24:19):

What gets measured gets done.

Eric R. Banilower (24:20):

Yeah.

Eric Cross (24:20):

Or what was getting evaluated was getting done. And that raises, that opens up a myriad of other questions about testing, and what that reveals, and all of those different things. But at the end of the day, what you’re finding is that the things that were getting tested were the things that were getting the priority.

Eric R. Banilower (24:36):

That’s right.

Eric Cross (24:37):

How did we get to this point? And Eric, you said it goes back at least to ’77, but we look at society and we’re…I wanna say we’re post-pandemic, but we’re we’re not. but we’re trying to, we’re trying to get past that. But we’re looking at…we had innovations in biology, we have innovations right now in green energy and electric cars and all of these things that are STEM-based. We know that these are things that have moved humanity forward. And we look at the pipeline of people who are in STEM and we, we see the disparities and things like that. Why was science given less of a priority? I’m just curious. Maybe, Courtney, we could start with you, if you have any ideas. Or Eric. Either one. But how did we get here?

Eric R. Banilower (25:22):

<laugh> I think Courtney wants me to take that one. I’m older so I’ve seen more <laugh>. So, you know, I have the gray hair. She doesn’t. I think it’s complicated. And I know this sounds cliche, but but schools are a reflection of society, right? And, and so science education, you know, if you think back when Sputnik was launched, there became this great demand in America to improve and produce more scientists and engineers in response to this Cold War threat. Right? And then in the ’80s there was rising, oh, the gathering storm was an economic argument that we needed to increase science and math, you know, education and people going into those fields in order to compete economically against the global competitors. And I think that America has always produced a fair number, a large number, of high-quality scientists and engineers, you know. And we still lead the world in many ways. But where we’ve identified as a problem is who has those opportunities to go into those fields. You know, it used to be a very select, a very male-dominated, white male-dominated field. Right? And other people didn’t have the opportunity, or they were shown the way out pretty early. And we, I think, have come to realize as a country that, you know, the, the greater the diversity of thought that we can get into these discussions, the more innovative we can be and the more productive as a society we can be. And so I think we’ve had this shift in the country to, instead of thinking about just the quality for the select few, but to be thinking about the quality for everyone. And so that makes it seem like some of these challenges are greater than they used to be. And I think they’re different challenges, right? We’ve evolved as a society and I think schools have evolved.

Eric Cross (27:40):

There is a conversation I was in on a plane with a person who was a materials manager for a company that made the adhesive for sandpaper. And we were flying…I was flying to Denmark and he was flying to some other Scandinavian country. And we were just talking about it. And he came from another industry, and somehow the conversation led to science. I don’t know how that happened. But somehow I just started talking about science and I asked him about, Eric, kind of what you said about the US kind of leading the way in science innovation versus the rest of the world. And I asked him why. And he said one of the reasons why is because the heterogeneous thought. The different groups of people that are coming to a problem actually create more innovative and novel solutions. Versus when it’s more homogeneous. And everyone’s either culturally or just for whatever reason, kind of thinks a certain way. While they might have a more efficient way, the variety of solutions are not as varied and not as novel. I was reminded of that story based on what you just said. So it’s really interesting. So it seems to be that it benefits if we have more heterogeneous groups, more folks who are contributing to STEM, because that’s gonna be solving the next problem more efficiently. Or I guess maybe in my head it seems like the next we need…we do really well when we have a dragon to slay. I mean, it seems like we come together when that’s the case, right? Like, I dunno.

Eric R. Banilower (29:06):

No, I think that’s…I think that’s accurate.

Eric Cross (29:09):

Later on the season of the podcast, we’re gonna explore ways to better integrate science with other subjects like literacy and math. Were you able to study at all any more integrated approaches to science instruction? Does any of your research support that approach?

Courtney Plumley (29:25):

Not on the national survey, we didn’t study that. And it’s something that we’ve talked about before, because it’s difficult to get teachers to…we were talking about instructional time. It’s hard for teachers to put a number on it when they’re integrating, because, you know, it’s not like I have my science block from 3 to 3:30 anymore. Now it’s kind of scattered about. But it’s something that has been in the ether. We’ve been looking at it in a couple of projects. So there’s some evidence that it can be effective, especially for getting more, you know…the idea is you can get more time for science if you are integrating with other subjects. But one thing to kind of caution is like, students need to have opportunities to learn each discipline when they’re doing integrated instruction. So you don’t wanna just have, like, math in your science. Kids already know to just, like, support it. Then it’s hard to take time from math to put it into science when they’re not actually learning anything new. That’s the easy thing to do, though, is say, “Oh, my kids already know how to measure. We did that in a previous unit. So now we’ll we’ll do it as part of our science instruction.” So it’s a lot of work to make it so they’re learning something new, mathematics and science, at the same time. And it’s not really something that we think that teachers should be having to do on their own, with all the other things that teachers have to do. The last thing they need to do is be creating their own, you know, curriculum. Something that’s already…you know, it’s not straightforward. So we’ve been talking about it, we think it’s really something that instructional materials maybe need to be focusing on instead of teachers having to do that on their own,

Eric Cross (31:01):

Teachers would implement it, but asking them to create it is a whole different thing, and it’s a huge ask.

Courtney Plumley (31:08):

Yes.

Eric Cross (31:08):

Yeah. And, did I hear you right? So the ideal situation would’ve been the students learning a newer math concept, but embedded in a science kind of context? Or was that the better way? Versus, “I’m gonna take a math concept they already know and then just put it into the science setting?”

Courtney Plumley (31:26):

Well, if the idea is that you can get more science time if you’re, you know, integrating things, so you can maybe take time away from a specific math block by putting it with science, or whatever, then if the math is something that the kids already know, now you’re just taking away. I think that that has to be new in both cases, in order to justify having more time.

Eric Cross (31:49):

Right. Eric, in the secondary level, any thoughts on that? On integrating these disciplines together?

Eric R. Banilower (31:56):

I think, you know, just like at the elementary level, it can be challenging to do it well. When I taught, I taught my last couple years in a kind of school-within-a-school kind of situation, where our goal was to try to integrate science, mathematics, and language arts. And it’s hard to do that in a meaningful way. And we did not have curriculum materials given to us to help us do this. We were trying to figure out how to do this on our own, while we were teaching 200 kids a day in our subjects. Right? And five preparations. And you know, it’s a big ask of any teacher. And there are teachers who thrive on this and are great at this. And, you know, that’s one thing I wanna, make clear: our data is about the system, and we are former teachers. Almost everyone who works at Horizon is a former teacher. We have the greatest respect for teachers and what they do. And what our data is showing is are kind of like areas where the system isn’t providing teachers and their students the opportunities to do great things. I think at the high school level, there has been this idea of project-based learning where students are bringing together different skills, different ideas from across disciplines. And I think there’s, again, a lot of potential in doing that. But trying to develop those experiences so that they are doing service to the different subjects, so students are learning what they’re supposed to learn in English Language Arts, that they’re learning, important mathematics, and that this is in a science context, where they are getting to do and understand what science is and how science, as a discipline, operates…that’s just a really hard thing to develop.

Eric Cross (33:53):

So what I’m hearing—and I really appreciate the nuance in this, because it’s not a simple “Yes. Integrated is better,”—I’m hearing “Yes. Quality control.” “Yes. It needs to be written not by teachers; they’re the practitioners.” It’s “Yes. And,” not just simply binary. Which…it’s so easy to wanna chunk things and say yes or no on things. But this one seems a much more nuanced approach. And in a future episode, you mentioned project-based learning, we’re gonna try and talk to people who have thoughts on this. And I really appreciate that you talked about project-based learning, because also, how do you evaluate that? How do you evaluate whether or not it is high quality? Is this is something I see? You know, high-quality standards, highest quality science teaching, highly qualified teachers. It’s something that I see often. Now, based on all your research, this is kind of the 30,000-foot view. What advice might you have for people who are thinking about changing the way science is taught in this country? Which hasn’t changed since 1977, at least since we’ve been measuring it. Any advice for people who do want to act? Another way to ask, it might be, if you were given a magic wand, <laugh>, you have all power, what might you do if you can control the entire vertical system?

Eric R. Banilower (35:07):

Yeah, so a clarification, I do think science instruction has changed. It has evolved. I think there’s a lot of really good things going on in different pockets of the country. One of the challenges is bringing those good ideas and good practices to scale. Right? There are approximately 1.2 million teachers of science K–12 in this country. That’s a lot of people. And about 80% of those are elementary teachers who are responsible for teaching other subjects as well. So my thinking is often about, “How do we take what we know and that we’ve learned through decades of research is effective, and impact a large number of teachers, and therefore a large number of students?” And you know, Courtney I think has hinted at this already. And you’ve mentioned it too, Eric, is that teaching is a profession, right? And it’s a craft. But in no other profession do practitioners have the expectation that they’re developing their own tools and methods for their work. I know when I was in my teacher preparation program, and it’s still extremely common, one of the assignments perspective teachers are given is to develop a unit and develop a lesson, right? You don’t have doctors being asked to develop new treatments and new tests to use. Their job is to get to know their patient, assess what’s going on, and then using research-based methods to develop a plan of action, right? And I think that analogy works really well in education and is a way that we could have a scalable approach for kind of raising the floor across the country for the quality of science education. Giving teachers research-based materials, high-quality instructional materials, that they can then use and adapt to meet the needs of their students, would allow them to focus on getting to know their students, seeing what their strengths are, seeing where they have room for growth, and using the materials they’re given to help those students progress. And I think that is definitely a way where we could have a big impact at a large scale.

Eric Cross (37:39):

Courtney, same question: Magic wand, all power. You can change systems from the elementary perspective. What would you do? I’m assuming part of it’s gonna be changing that 18 to 20 minute time. But even for that to happen, what would you do? What would you change?

Courtney Plumley (37:57):

Well, I don’t know. Like, for it to change, I don’t know the answer to that. But yes, increasing the time would be great. And like Eric was saying, giving teachers— ’cause again, I’m coming in, not enough probably background in science—and then, you know, when I was, when I was teaching, we had one set of textbooks for the entire grade. Six classes, right? Like, share them. But third graders aren’t gonna read textbooks anyway, right? So instead I’m going to the teacher store. I’m pulling things off the shelf. And like, “OK, yeah, sure, I’ll use this.” And nowadays, teachers are going to Teachers Pay Teachers or whatever. Because I didn’t have anything good to use. So like Eric is saying, if I had instructional materials that were good instructional materials that were gonna teach my kids, that they were gonna be engaged, that they weren’t sitting and listening to science, but they were doing science, you know, and I had professional development to actually help me do it? That’s what I think we need to have. And I mean, I know there are some people out there that are working on that, but it’s not a lot. I mean, if you look at Ed Reports, they rate how well-aligned science curriculum are to standards. And there are two right now that have Ed Reports green lights. There’s Amplify and there’s OpenSciEd. You know, so there’s not much out there for teachers to use. And, so it’s hard. It’s hard. Where am I gonna go and get this stuff if it doesn’t exist? And so I’m making it up by myself. Which we already said is not the best use of teachers’ time, when they’ve got so many other demands on their time.

Eric Cross (39:27):

Eric and Courtney, listening to both of your responses, it created a visual in my mind. And Eric, I loved your analogy of…I started thinking of a chef, a welder, and a farmer. And I thought about the chef saying like, “You’re a great chef! Now, can you go farm, and make your own food, so that you can cook it?” Or the welder who has to make his own welding tools and go smelting. You know, making the different rods. I’m not a welder. But you know, all those different parts. Or the farmer who has to build his own tractor and innovate all that stuff. You’re absolutely right, the way you articulated that. And then Courtney, you essentially said, “Give them the tools and then teach them how to use it so they can go and actually be effective with it, because you’re in front of kids doing so many different things.” There’s only so much time in the day, and teachers want to do these things; they want to, but you end up having to triage when you’re asked to. Going back to Eric’s analogy, if you’re in the ER, but you’re also creating the vaccines and you’re also doing the research on which types of vaccines are gonna be the most effective, that’s, that’s a lot to ask. And so, I appreciate both your responses on that. Now, last question, what are you both working on now? This report came out in 2018. What’s, what’s next on the horizon? Actually literally, that’s no pun intended. <laugh> What’s next? <laugh> What’s next for, for you both? What are you working on?

Eric R. Banilower (40:42):

Well, you know, we would love to do another national survey, in a few years. We have to get funding to do it. And you know, that’s always something that takes effort and isn’t a guarantee. We’ve written grants to do these studies in the past, and there’s also the dealing with the reality of the situation. I think a lot of schools, still coming off the tail end of dealing with Covid, are overwhelmed. And we’ve had a hard time, I mentioned before, recruiting schools, and it gets harder every time, just ’cause they have so much on their plate. And I couldn’t see going to a school now and saying, “Hey, one more thing. Do you mind?” So I think we have to kind of wait a little bit for things to settle down before we can do another one of these studies. It just doesn’t seem feasible right now. But we’d love to in the not-too-distant future. Other than that, Courtney and I actually work on some projects together and some projects not together. One of the things that we’re working on together is a study of a fifth grade science curriculum that was developed by Okhee Lee at NYU and her colleagues, that is both aligned with the NGSS and purposely designed to support multilingual learners in developing both their science knowledge and skills as well as their language skills. And we’ve been working with the crew at NYU to study this curriculum and try to figure out, how well it’s working and under what circumstances. So that’s been a really interesting project that’s going on right now.

Courtney Plumley (42:26):

I recently worked on a report with the Carnegie Corporation in New York that actually I think, compliments what we’ve been talking about a lot. It’s about the status of K–12 education in the US—or science education in the US! <Laugh>—and so as part of that report we interviewed like 50 science education experts across the country. We surveyed teachers, people in the university settings, researchers, and everything to kind of get a little bit more update of the state of science education right now. And so a lot of the things we’ve been talking about, we still are talking about with the people in this report four years later. So, work in progress. <Laugh>

Eric Cross (43:09):

And again, going back to 1977, based on what Eric was saying earlier, we’re looking at these large systems, these systemic changes don’t happen overnight.

Eric R. Banilower (43:20):

That’s right.

Eric Cross (43:21):

It’s very slow-moving.

Eric R. Banilower (43:22):

That’s right. I would say there is progress. I think we’ve learned a lot. We are getting better. Are we there yet? No, we’re not happy with where we are. But I think, you know, I think it’s important to be hopeful about the direction things are going in.

Eric Cross (43:37):

Well-said. I agree. Courtney. Eric, thank you so much for unpacking that report that speaks to, that validates what so many teachers across the country are experiencing. And thank you for your advocacy for high-quality science education and your passion for supporting teachers and being that voice from a data-driven perspective of what teachers experience and then advocating for solutions for them. It’s super-encouraging for me, and I know it’s gonna be really encouraging for a lot of our listeners. So thank you.

Eric R. Banilower (44:10):

Thank you for having us.

Courtney Plumley (44:12):

Yeah. Thank you, Eric.

Eric Cross (44:15):

Thanks so much for listening to my conversation with Eric Banilower, Vice President of Horizon Research, and Courtney Plumley, Senior Researcher at Horizon Research. For much more, check out the show notes for a link to the 2018 National Survey of Science and Mathematics Education. And please remember to subscribe to Science Connections wherever you get podcasts, so that you’re not missing any of the upcoming episodes in Season three. Next time on the show, we’re gonna start laying out the road map for using science more effectively. And we’ll start by looking at the how and the why of integrating literacy instruction.

Susan Gomez Zwiep (44:49):

When we look at Science First and build language development around it, the experience tends to be more authentic and organic.

Eric Cross (44:58):

That’s next time on Science Connections: The Podcast. Thanks so much for listening.

Melanie Trecek-King (00:00):

We say knowledge is power, but it’s not enough to know things. And there’s too much to know. So being able to think and not fall for someone’s bunk is my goal for my students.

Eric Cross (00:12):

Welcome to Science Connections. I’m your host, Eric Cross. On this third season, we’ve been talking about science’s underdog status. And just this past March at the NSTA conference in Atlanta, I had the chance to speak with science educators from around the country about this very topic.

Hermia Simanu (00:28):

Right now, there’s only two teachers in our high school teaching science.

Shane Dongilli (00:32):

I have 45 minutes once a week with each class. The focus is reading and math.

Alexis Tharpe (00:38):

Oftentimes science gets put by the wayside. And you know, I love math and I love my language arts, but I also think science needs to place be placed on that high pedestal as well.

Askia Little (00:46):

In fifth grade, oh, they teach science, because that’s the only grade that it’s tested.

Eric Cross (00:50):

That was Hermia Simanu from American Samoa. Her team flew for three days to make it to the conference. You also heard from Shane Dongilli from North Carolina, Alexis Tharpe from Virginia, and Askia Little from Texas. All of these teachers were excited to be at the conference and had a lot to say about the state of science education in their local schools. Throughout this season, we’ve been trying to make the case for science, showing how science can be utilized more effectively in the classroom. We’ve explored the evidence showing that science supports literacy instruction. We’ve talked about science and the responsible use of technology like AI. My hope is that all of you listeners out there can use some of this evidence to feel empowered to make the case for science in your own communities. And on this episode, we’re going to examine how science can help develop what might be the most important skill that we try to develop in our students: Good thinking. On this episode, I’m joined by a biologist who actually advocated for eliminating the Intro to Bio course at her college. Instead, Professor Trecek-King created a new course focused on critical thinking, information literacy, and science literacy skills. In this conversation, we discuss why the science classroom is such a good environment for helping students become better thinkers. Now, I don’t think that you can make a much stronger argument for science than using it to develop the skills that Melanie describes in this conversation. So, without further ado, I’m thrilled to bring you this conversation with Melanie Trecek-King, Associate Professor of Biology at Massasoit Community College, and creator of Thinking Is Power. Here’s Melanie.

Eric Cross (02:29):

Well, Melanie, thank you for joining us on the show. It’s so good to have you.

Melanie Trecek-King (02:34):

I am so happy to be here.

Eric Cross (02:35):

Now, I went to your session at NSTA in Chicago … I think it was two years ago. A couple years ago. And I was listening to your session, and as I was listening to you, I started Reverse Engineering in my mind what you were doing with your college students. I started reverse engineering the K–8. I was like, “This is amazing.” Where has what you’ve been doing been hiding? We need this not just in the college, higher ed. We need this all the way up and down. Because I hadn’t seen it before. So I think a good place for us to start is gonna be like the story of how and why you as a biologist wound up making the case to actually eliminate the Intro to Biology course at your college. So can you start off and tell us a little bit about that story?

Melanie Trecek-King (03:20):

Sure. So I started teaching at a community college in Massachusetts. And I absolutely love teaching at a community college. And I was teaching the courses that people who don’t wanna be scientists when they grow up have to take to fulfill their science requirement. And that course was Intro Bio. And I tried every way I could figure out to make that class be useful,] relevant to students. I mean, the thing is, our world is based on science and you have to understand science to be a good consumer of information, to make good decisions. And I’m a biologist, so it pains me to say this, but you know, somewhere in the middle of teaching students about the stages of mitosis and protein synthesis, I thought, “Is this really — like, if I have one semester that’s gonna be the last chance that someone’s gonna get a science education, is this really what they need?” And I just decided, “No.” So, to my college’s credit, they were very supportive. I went to them and said, “You know, I think we should assess the non-majors courses. Like, why do we teach non-majors science?” And we all agreed, well, it was for science literacy. OK, great. Do our existing non-majors courses do that? And so we evaluated each of the courses. I made a case that Intro Bio was not doing it. And so we actually replaced it with a course that I call Science for Life. And the whole course is designed to teach science literacy, critical thinking, and information literacy skills.

Eric Cross (04:48):

And so you did this while you were looking at mitosis. And you’re looking at students who may or may not be science majors. And then kind of asking that question. I know every educator asks this, and whether or not it’s welcomed or supported is a different question: “Is what I’m teaching actually gonna be relevant and useful later on down the road for this group of students?” And you actually got to run with it and then create this course, this new course. So, what were the skills that you were hoping to achieve with the new course you developed, and and why were those skills so important?

Melanie Trecek-King (05:21):

Well, if I just go back for a second to what you said, ’cause it, really hit me: I remember the actual moment — it had been building up to that point, but the actual moment that it hit me — I was teaching students the stages of mitosis. And I was applying it to cancer, because the thought is that if we use issues that are relevant to students to teach concepts, that it will be more meaningful to them. They’ll learn it better; they’ll be able to apply it. And they just looked absolutely deflated. They didn’t wanna be there. And I had this moment where I thought, “You know, if, if these students ever have cancer somewhere in their lives, is what I taught them going to be something that they remember? Is it going to be useful to them?” And quite frankly, like, no. <Laugh> They’re not gonna remember proto-oncogenes. And quite frankly, is that really what they need to know at that moment? What they need to know is, “What does this mean? Who is a reliable source of information here? If these treatments are recommended, what is the evidence for them? What are the cost-benefit analyses? Where do I go to find reliable information?” And in that space, cancer in particular, we have this whole field of — I wanna say charlatans, ’cause they may not actually be lying, but they’re pedaling false cures, false hopes. And people need that kind of hope, and so in their time of need, they’re more likely to fall for that kind of thing. Which leads me to the skills that I teach students. I call them this tree of skills. And the order is important. I start — and there’s a lot of overlap to be fair — but critical thinking, and then information literacy, and science literacy. The idea is that students carry in their pockets access to basically all of human’s knowledge at this moment in time. And if they needed to access it, they could. The question is, do they know what they’re looking for? Are they aware of their own biases that are leading them to certain sources, or certain false hopes? Are there certain things that are making them more vulnerable to the people that might prey on them? Are they able to use that information to make good decisions? There’s a great Carl Sagan quote, and it’s something like, “If we teach people only the findings of science, no matter how useful or even inspiring they may be, without communicating the method, then how is anyone to be able to tell the difference between science and pseudoscience?” So yes, the process of science is a process of critical thinking. However, we do tend to present science most of the time. Like, here’s what science has learned. And to be fair, those things that we’ve learned from science are really useful and inspiring. But if we don’t teach the process, so you’ve got somebody now who let’s say has been diagnosed with cancer and is on their phone and they’re scrolling through social media and everything looks the same. And of course the algorithms learn who you are. Next thing you know, there’s all of these like pseudo-treatments popping up. It all looks the same. Somebody who says that acupuncture can be used to cure cancer can feel the same, from someone who doesn’t understand the process of science, as a medical fact. And so the process is the process of critical thinking. My class everything is open note. The quizzes are open note. The exams — and I say open note, they’re also open online, because I know for the rest of their life they’re gonna have resources available to them; I want them to be good consumers with that information, which to me requires metacognition and critical thinking and information literacy and all those skills that I’m trying to teach them.

Eric Cross (08:58):

You’re basically taking what … we’ve taught science for so long. And more recently, it’s changed to more focusing on skills. At least in K through 12. But a lot of it was just memorization of a ton of different things that now we can pull up our phone, go on the internet. You can pull up a lot of those facts. But those facts don’t necessarily translate to actual real-world skills. When I listen to… I kind of make this analogy sometimes: students say … it’s funny, I have 12-year-olds that say this. They go, “How come they don’t teach us how to do our taxes?” And you know they’re regurgitating what they hear from adults, right? “Teach us real-world skills!” And I was like, really, if we taught you right now how to do your taxes, how many of you would really be like, “Oh, this is an awesome lesson! We’re really engaged!” But their point is that “I wanna learn something that I could actually use later on, that’s that I’m gonna carry on.” And in your course, you’re talking about these skills that actually can apply. Like you said, if I had cancer and I’m looking at different types of medical procedures, do I have the skills to really be able to evaluate and make informed decisions on that? And that’s, that’s not something that I’ve seen explicitly taught really anywhere. And I hadn’t heard anybody talk about it, really, until I heard your session, where you’ve kind of unpacked this, and over the last couple of years, have created some programs or resources for educators, where they can take this into their classroom. So what were some of those skills, again? What were were some of the skills that you thought, “I wanna make sure that my students can walk out and they know how to do this and apply it to maybe several different fields”?

Melanie Trecek-King (10:35):

Oh, that’s a really good question. Because the whole thing was a process for me. Like, when I finally let go of Intro Bio, I was so glad to see that class go, by the way. ‘Cause I just felt like I was beating a dead horse. So when I let go of it, I thought, “What do they need instead?” And for me, what I realized was I was trying to make the class I would’ve wanted to take. I realized the things that I personally didn’t know, that my own education maybe let me down a bit. But things that I thought were important. So then I took all of those, synthesized them, tried to figure out the best order. The class is currently in its third iteration. And I hope every iteration is an improvement. But I’m thinking about the students that I taught before the pandemic. It was Intro Bio. Up to just maybe the couple years before the pandemic, and during the pandemic, we had a new virus and we had a new vaccine and we had new treatments. There was hydroxychloroquine and there was ivermectin and then there’s masks. Are masks effective? Well, you know, in what circumstances? What kind of mask? There are all of these questions. And that whole thing was we saw science playing out in real time.

Eric Cross (11:50):

Absolutely.

Melanie Trecek-King (11:51):

And so were my students able to follow that? And then what happened in that process is that science became politicized. And in a time where things are uncertain and we need answers, ’cause it’s scary, people want certainty and science doesn’t tend to provide that. Especially when it’s just starting out. And then when it becomes politicized, people decide that they’re going to — it’s not necessarily a conscious decision — but they retreat into what people in their camps are saying or their groups are saying. Which actually leads me to one of the more important parts of information literacy skills in there, which is most of our knowledge is shared. We tend to have overinflated senses of what we individually know. And studies actually show that with Google, if you have access to Google, you think you’re smarter than if you don’t have access to Google. But we all have access to knowledge in our communities, and that’s one of the reasons humans are so successful, is that we can each specialize in different things and share our expertise and become greater than the sum of our parts. The problem with that, of course, is that we forget what we don’t know, and we assume that we know what the community knows. And so recognizing the limits of your own knowledge and how different communities produce knowledge, like the different epistemic processes that communities use to come to knowledge. When it comes down to it, an important part of knowing is knowing who to trust, right? Knowing where the source of knowledge lives. And in order to do that, you have to understand the processes that they’re using to come to that knowledge and the limits of your own knowledge. And then how to find who has that knowledge so that you can use that to make better decisions.

Eric Cross (13:38):

So, when I hear what you’re doing with your college students, and I think about what I’m doing in the classroom, in the middle school, we are really focusing on literacy as skills. Reading, writing, speaking, listening. And then when I think of the next step of the journey, your information literacy and the literacy you’re teaching is really the application of those things in the real world. And the examples that you gave are very critical examples. Evaluating claims about Covid. Making informed decisions about a medical procedure that you might need. And we all get that applied to us. We’re scrolling through social media and somehow social media is listening. It’s figuring out exactly what I’m doing, because all of a sudden the ads are telling me … how did you know I was alking about KitchenAid mixers now? I just said KitchenAid mixers and it’s gonna show up in my feed! But <laugh> I take that in the same way from the same place that I take in maybe an oncologist. So it’s it’s coming through the same channels. So now I kind of wanna pivot. So we’ve talked about what you’re doing, why you’re doing it, the connection between “am I really teaching the skills that my students need in the science class? Is it really critical thinking explicitly or is it just kind of implied?” Now I wanna ask you how you do it. What’s the annotated, abbreviated kind of syllabus of your course?

Melanie Trecek-King (15:03):

So the course is called Science for Life. And the premise behind it is the kinds of skills and understanding of the process of science that they would need to make good decisions to be empowered in a world based on science. And so the very first lecture, I say, “OK, I’m gonna tell you a story and I just want you to listen to the story. And at the end I’m gonna ask you why I told the story.” And the story that I tell them is some of the history of the witchcraft trials in Europe. And I start with the Malleus Maleficarum, or the Hammer of Witches, from the Pope, and about how people would accuse witches of causing birth defects or storms or crops dying. And, the best evidence that they had to absolutely know somebody was a witch was if somebody accused them, and then if they were accused, if they confessed. OK? But the problem is, to get them to confess, they would torture them. Roasting over coals, or splitting until somebody broke. And so I tell my students, “OK, this was absolute proof that someone was guilty of witchcraft. I don’t know about you; I would confess to anything, right? Make it stop!” So this is where I get to ask students, “Why would I ask you this? Why would I tell you this story? And traumatize you on the very first day of lecture?” And they see the reasoning, right? They thought they had evidence. The question was, is that good evidence? And so, you know, I’m getting students to have a basic understanding of epistemology, right? Without calling it that, or without going into all of the philosophical background of epistemology. Apply this to your own reasoning. What are you wrong about? Well, you probably wouldn’t know. OK, how would you know if you were wrong? Like what kinds of things do you feel that you’re so right about? How good is your evidence for that? So what I want them to do is internalize the thinking about thinking, and analyzing how they come to conclusions, and proportioning how strongly they believe. Their confidence in how right they are. So I think starting with that kind of misinformation, and getting students to internalize that process is important. But I think the example is really useful, because most of my students don’t believe in witchcraft. Right? So it’s not an issue that would immediately threaten them in some way. So when, when a belief is tied to identity or how we see ourselves or is really important to us, then it’s very difficult to be objective about that belief. And so by starting with witchcraft, it’s not triggering. I get them to think about thinking and practice that muscle so that when we get to those more important issues, they have the skills they need to evaluate them.

Eric Cross (17:55):

So would it be fair to say that your Science for Life class is really applied scientific thinking for the real world?

Melanie Trecek-King (18:01):

Absolutely. That’s the idea. I mean, science is too good to keep to ourselves, right? And it’s everywhere. So how can you understand the world through a scientific lens?

Eric Cross (18:10):

What are the nuts and bolts of how you teach your students these strategies? What do you do? What are some strategies and techniques that we can maybe share with listeners? And then where I want to go after that is I wanna ask you, how early do you think this can be started? So lemme start off first with, what do you do?

Melanie Trecek-King (18:28):

So I use three different strategies. One is, I provide students with a toolkit. And the toolkit is one that I created and it is like my one toolkit to rule them all. It is trying to apply critical thinking and science reasoning all together in one place. So that if students are met with a claim, they’ve got the toolkit with an acronym. They can now start and have somewhere to go. In that if I gave you a claim and said, “Just critically think through this claim,” I mean, that’s a mighty task. But if you have a structured toolkit, then it’s hopefully a systemic way that’s helpful. The toolkit is summarized by FLOATER. I have published it on Skeptical Inquirer. It’s free. So it’s Falsifiability, Logical, Objectivity, Alternative Explanations, Tentative Conclusions, Evidence, and Reproducibility. So I provide students with a toolkit. The next thing I do is I use a lot of misinformation in class. Back to what Carl Sagan says: What I heard was we should use pseudoscience to teach students the difference between a pseudo-scientific process and a scientific process. So, I use science denial, conspiracy theories, and give my students a lot of opportunities to practice evaluating claims with the toolkit. And the other thing I do is, I use inoculation activities. So inoculation theory is based on William McGuire’s original research in the ’60s, which is basically like a vaccine analogy. Where you can inject a small amount of a virus or bacterium into the body, so that it creates an immune response, so that it can learn the real thing. And so in the real world, it can fight it off. Inoculation theory does the same thing, but with misinformation. So, what we can do is, in controlled environments, expose students to little bits of misinformation so that they can recognize it in the real world. There’s different kinds of inoculation, but I’m a big fan of what’s called active and technique-based inoculation. So technique-based means that students are learning not the facts of misinformation, not factually why this thing is wrong, but about the technique used to deceive. So maybe the use of fake experts. Or maybe the use of anecdotes. Or the use of logical fallacies. The other part of that is active, which is where students create the misinformation. So for example, my students, just now, we finished covering pseudoscience. And I teach students the characteristics of pseudoscience. And basically we have fun with it. Where they pretend to be grifters and they sell a pseudoscience product. And so they have to make an ad like they’d see on social media, using the different techniques. And the point there is that it’s supposed to be funny, right? And lighthearted. But in a real way, by using the techniques used to sell something like pseudoscience, it’s opening their eyes. You can’t unsee how every alternative product has, “it’s an all-natural and used for centuries and millions use it and look at this person who says, ‘Wow, it worked for me!’ And it’s certified by some society that doesn’t exist, but this doctor behind it says that it’s really great!” I mean, it’s all the same stuff. So they create the misinformation using their own techniques.

Eric Cross (22:02):

That’s one of my favorite things that you’ve talked about, and I want to dive in that a little bit more. But when you’re teaching the toolkit, FLOATER, what does that look like in the classroom, when you’re actually breaking all of those things down? What does it look like as you’re walking your students through this, and you’re kind of coaching them on all of those different things? ‘Cause I feel like some things might be like, “Oh yeah, I got that.” And then some of them might be, “Oh, what is that?”

Melanie Trecek-King (22:24):

Yeah, it takes me probably a good solid lecture to get through the basis of the toolkit. But then over the rest of the semester, I’ll spend more time going into different parts, different rules, a bit more in-depth. So, for example, logical fallacies and objectivity. So the rule of objectivity basically states that you need to be honest with yourself. I’m gonna quote Feynman here, so: “The first principle is that you must not fool yourself — and you are the easiest person to fool.” We don’t tend to think that we can be fooled. But of course we can. So actually, if you wanna talk about it, I start class by fooling my students.

Eric Cross (23:03):

Wait, what do you do? What do you do for that?

Melanie Trecek-King (23:05):

Oh, so this is really fun. Day 1 of class, after the syllabus, I tell my … so you’re in my class now, Eric. “So I have a friend, and she’s a psychic. She’s an astrologer and she’s pretty good at what she does. I mean, she’s got books and she’s been on TV and stuff. She knows I teach this course about skepticism. And so she’s agreed to test how effective she is by providing personality assessments to students in class. So if you wanna participate, what I need from you is your birthday, your full name, answer a few questions. Like, if your house was on fire and you could take one thing, what would it be? Or if you could get paid for anything to do anything for a living, what would it be? Um, there’s a third one. Oh! If you could have any superpower, what would it be?” So the next class, it’s usually over a weekend. The next class I say, “OK, I’ve got your personality assessments back, but remember, we wanna test how effective she is. So in order to do that, I need you to read your profile as quietly as possible. And then I’m gonna have you rate her accuracy on a scale of 1 to 5. OK? So close your eyes; rate her.” Over the years doing this, it’s about a 4.3 to 4.5 out of 5. They think she’s pretty accurate. OK? “So now, if you feel comfortable, get with a person next to you. And I want you to talk about what parts of the personality assessment really spoke to you and, and why, and why you thought she was accurate or not.” And it takes them 5, 10 minutes before they realize they all got the same one. So, this is not my original experiment. It was first done by Bertram Forer in … I think it was the ’50s. And it’s done in psychology classrooms. James Randi made it famous. But the personality assessment itself is full of what are called Barnum statements. So, named after P.T. Barnum. These are statements that are very generic. So, “You have a need to be liked and admired by people. You are often quiet and reserved, but there are times where you can be the life of the party.”

Eric Cross (25:13):

How do you know this about me, by the way? This is a — I feel like you know me right now.

Melanie Trecek-King (25:17):

“There are times where you’ve wondered whether you’ve done the right thing.”

Eric Cross (25:19):

This is getting weird.

Melanie Trecek-King (25:21):

I’m just on fire, right? So these are Barnum statements. They’re the basis of personality assessment.

Eric Cross (25:29):

Mel, can I pause you right there? You said Barnum. Is that the same Barnum, like Barnum & Bailey Circus?

Melanie Trecek-King (25:34):

Yeah. P.T. Barnum, who didn’t actually say “There’s a sucker born every minute,” but we attribute him with that kind of ethos. These statements though, if you read a horoscope or even like personality indicators, like the MBTI, it is basically pseudo-scientific. And it ends up with lots of these Barnum statements. They produce what’s called the Barnum Effect, which is, “Wow, that’s so me! How did you know me?” I could even do more. Like, you have a box of photos in your house that need to be sorted. Or unused prescriptions. And these can apply to nearly everyone, but they produce this effect where we go, “Wow, that is so me!” Right? So by fooling them this way, I get to … well, so the next thing is, “Yes, I lied to you. And I’d like to tell you I won’t do that again. But I’m not going to, ’cause I might. So be on your guard.” But I did it for free. And why did I do it? “I did it because I could tell you ‘I could fool you,’ but you wouldn’t necessarily believe me. So I fooled you, so that you would learn what it feels like to be fooled.” It’s not fun. But we’re gonna make a joke outta this. And students are almost never upset about this ’cause it’s a fun process and they’re all fooled. And again, the point is, I didn’t disprove psychic powers. I didn’t just disprove psychics with this exercise. But I did show you how easy it was to fake. So if somebody is gonna tell you that they can know these things about you through some way, hopefully the evidence they provide should be stronger than something that’s easily faked. Right? Extraordinary claims require extraordinary evidence. If you claim to be able to read my personality based on my birthdate, then I need more than something that you can be taught to do in 15 minutes. So, I fool them to convince them that they could be fooled.

Eric Cross (27:27):

You’re giving them a practice scenario for thinking. And I was thinking about basketball. I grew up playing basketball. And my coach would have our own team be the defenders of the next team we were gonna play, so that we can be prepared for the defense. We were gonna see. Now, when I’m thinking about education, and what you just said reminded me of this, it’s like we’re often just teaching offense. We’re always teaching the plays. We’re always teaching what to do. But we rarely teach defense. What happens when someone comes towards you and, and they challenge you or they come at you with claims? How do we evaluate this? And I think in pockets we do it. We do claim-evidence-reasoning. We present claims and evidence and reasoning. But we don’t always have practice defending them. And I think there’s great resources. There’s Argumentation Toolkit and there’s all these awesome resources that do this. But does that fit? You’re kind of having them practice defense?

Melanie Trecek-King (28:26):

Yeah. You know, that’s brilliant. I never considered that analogy. But, yeah, in the real world, you don’t just get to always try to score all the time. Someone’s gonna challenge you and give you a claim that maybe you haven’t heard before. So how do you think through it?

Eric Cross (28:41):

Yeah. And you become better. So now I’m thinking about how early could we start doing this? For one, I love the idea of lying to your students, because I do that. And it’s just such a fun scenario. How early could we start implementing these strategies or these ideas or these toolkits? In your mind, what do you imagine? How early could we start this with young people?

Melanie Trecek-King (29:07):

Yeah. I’m so glad you asked that question, ’cause honestly, by the time they get to me, it’s almost too late. And I don’t wanna say it’s too late, ’cause it’s never too late. But, oh, we need to start so much earlier! That example that I gave about the selling pseudoscience argument? I have a wonderful colleague, Bertha Vasquez, who’s a middle school teacher in Miami and the director of TIES at CFI. She did this with her middle school students. And quite frankly, their examples were just as good, or in some cases better, than my college students. And they had so much fun with it, too. And she just said that, you know, <laugh>, they actually are more savvy with the kinds of things that they see online than we — I don’t wanna say give them credit for. But almost that we want to believe. My students give me examples of things that are from corners of the internet that I didn’t know existed. And quite frankly, that’s probably a good thing for my own mental health. But students are on there too, like middle school students, and we need to prepare them for the kinds of things that they see in the wild.

Eric Cross (30:13):

So in middle school, definitely. Now, you’ve also done some work in high school as well, right? In Oklahoma? Did you do some. …?

Melanie Trecek-King (30:17):

Yeah.

Eric Cross (30:18):

…some work with high schoolers? What was that like? Did you see any impact there?

Melanie Trecek-King (30:21):

So I didn’t actually do it in Oklahoma. I have taught the course … actually, you were talking about younger kids. I’ve taught the course to high schoolers in my area that are parts of dual enrollment. And they absolutely ate up the curriculum. And they were wonderful, wonderful students. And it was completely appropriate for … they were juniors, actually. But the course has also been taught in Oklahoma, through a dual enrollment program as well. And it was a small sample size. But we have pre-post testing that showed that it improved their critical thinking, their acceptance of science. But anecdotally the head of the program there said that in his years doing this, he’d never seen a course that helped them improve in their other courses so well. So, I felt very rewarded by hearing this. But apparently their critical thinking skills and information literacy skills helped them succeed in their other courses that they were taking. And I love that the students were transferring those skills to other classes. That’s the whole point.

Eric Cross (31:23):

And that’s a big … I think that what you just said is really the core, especially of what we’ve been talking about this season: What you’re talking about and what you’re teaching can transfer and supports literacy. And this is an example of science doing that across all other content areas. So I think that that’s huge, that that was said. What do people say about this course? I know I went on your website, and I looked at some of the comments that some folks were saying, and I know it’s just a snippet, but what do you hear from the education world about this? Because I don’t see it in many places. I see it kind of embedded, sprinkled into different content areas. But you’re actually teaching it explicitly. Do you tend to find positive feedback, overwhelmingly? Or do you get pushback on on some of this? What’s it been like for you?

Melanie Trecek-King (32:16):

I think the biggest pushback — and it’s good pushback, and I would agree entirely — is with inoculation activities, you do need to be careful to, when you debrief students, you wanna tell them why you did what you did and to use their powers for good and not for fooling other people. And I think importantly, for not putting misinformation out into the wild without having context around it. So if you do these kinds of inoculation activities, like if you have your students create pseudoscience ads, don’t just let them put them on social media. Obviously, you can’t control everything that they’re doing. But explain to them why you wouldn’t wanna do that. As far as everything else, I’ve heard really great feedback. You’re referencing my website. So, when I put together the course, I was trying to find resources for students to read. Textbooks are ridiculously expensive and I couldn’t find anything that I really wanted students to buy. So I just started writing, and I put it on my site. I have a site that’s basically the core of the curriculum. More in progress. And then I’ve got some of the topics that we explore and those are all assigned readings. My students are captive, in that I know they want a grade, and for four months they have to sit with me for the entire semester, in that I’ve specifically ordered the content in a way that would be most conducive to them learning these things. On the internet, though, and on social media, ’cause I post on there as well, people come in from all kinds of entry points, and so the goal would be to have them start at the beginning and go to the end. But people … I’m pleasantly surprised that there is an audience for critical thinking and science literacy content out there. And so that really warms my heart. But I am doing more and more for educators. And so I have a section for educators. I put content on there. I put assignments, the assignments that we’ve talked about and more, are on there. And the educators that I’ve had use it have just been really wonderful. Like, I hear great things. If I might, the biggest issue that I’m having is actually reaching educators. I’ve gone to — I met you at NSCA, actually, that was only last summer.

Eric Cross (34:30):

Oh, wow. Wow.

Melanie Trecek-King (34:32):

Right?

Eric Cross (34:32):

Yeah, you’re right. It wasn’t even a year.

Melanie Trecek-King (34:35):

Yeah, I think it was like July last year. So, um, you’ve been to the conferences. And I just went to the last one as well. But I have yet to figure out a way to really get in front of enough educators to share the content. So if anybody’s listening and is interested in learning more, please let me know! <Laugh>

Eric Cross (34:52):

Yes. And we talked about your website, but I didn’t say what the website was. So it’s ThinkingIsPower.com.

Melanie Trecek-King (34:57):

Yes.

Eric Cross (34:58):

And on there, there’s tons of resources. There is the toolkit. And it’s all free.

Melanie Trecek-King (35:06):

Yes.

Eric Cross (35:07):

And there’s a dope t-shirt on there that I just bought today, that Melanie’s actually wearing right now. It says, “Be curious, be skeptical, and be humble.” And I love that. Because I think one of the things that we can’t forget about teaching people how to think and critically evaluating information, sometimes those conversations can become very dehumanizing. And what I mean by that is it sometimes can become, like, intellectual sport, where we forget that there’s a human being on the other other side. And we lose that empathy and compassion. We can kind of see that. It just becomes this intellectual jousting and arguing. And one of the things I know about you, and when you talk about this or you talk about the work that you do, and even the shirt that you’re wearing, there’s this, “be humble.” There’s this human that is never lost in this. And you said it, too: When you’re teaching your students and you’re equipping them with all of these intellectual skills and all of these tools, to use it for good. So to maintain your humanity, to maintain your character, and then to use it to edify and lift people up, not to go out and do harm. That balance, I think, is so, so important. So it’s something that I really appreciate about you and how you teach.

Melanie Trecek-King (36:19):

I appreciate those kind words. Actually—

Eric Cross (36:21):

Oh, of course!

Melanie Trecek-King (36:22):

—and if I might, I sometimes see people using critical thinking like a weapon. It’s like, “I have learned fallacies and I’m just gonna use the tools of critical thinking to tell you why you’re stupid, or why you’re wrong, and why my position is right!” But real critical thinking involves applying those same standards to your own thought processes. And even something like argumentation: the goal of our argumentation is not to BE right; it’s to GET it right. And so we’re on the same team. If we’re arguing about something, if the idea is in scientific argumentation we’re trying to find the truth, which one of us is making a better argument based on the evidence? Can your perspective help me see my own blind spots and vice versa? And the more different perspectives that we have, the more able we are to find whatever reality is. But we are in this together. And so, yeah, I think … I’m glad to hear that that’s coming through. But if you don’t have the kind of humility that says, “You know, I could be wrong,” then you’re never gonna change your mind anyway. So having the humility to say, I’m wrong. <Laugh>

Eric Cross (37:33):

Yeah. You end up just seeing people just defend turf, as opposed to support “look for truth.” And I know for me, my own education journey, I end up with more questions than answers anyways. So I go in trying to find an answer for something and I end up with 10 more questions. And I go, “OK, this is kind of how it is.” You go down this rabbit hole and you just end up with all these different questions. And it forces the humility, because you’re like, “I don’t know! I think this is what it could be, but it could also be these other answers or explanations. So this is just where I’m at, based on what we know right now, at this present time, which might shift.”

Melanie Trecek-King (38:07):

And that sounds reasonable. Yes. Which might shift. Yes.

Eric Cross (38:11):

And especially for us as life-science biology teachers, our content is something that definitely shifts. I know some of the things I teach now are not things that I learned when I was even in middle school. Just because things evolve. They change. We learn, we get new data. That’s just the way it is.

Melanie Trecek-King (38:24):

<Sighs> And Pluto is no longer a planet.

Eric Cross (38:26):

I know. Rest in — well, no, Pluto’s still there. Yeah. It’s no longer a planet. But that was one part of my kindergarten memorizations <laugh> is Pluto being in there.

Melanie Trecek-King (38:36):

Gotta change your mind.

Eric Cross (38:38):

I know. Any words of advice for science educators out there who want to focus more on honing these critical thinking skills and strategies with their own students, but they don’t know where to start? Where would you point them? Or what advice would you give them?

Melanie Trecek-King (38:52):

I think start with what you want the students to know. And not necessarily the FACTS that you want students to know, but start with the skills that you want them to know. And then really be honest with your process. When I designed Science for Life, I started with, “these are the skills that I want students to know.” And everything was in service of that. So this sort of backwards design, I think, helped me follow a path that was more likely to be useful, if that makes any sense. But it really required doing it all over again. So don’t be afraid to question the things that you’re currently doing, even if that’s all you’ve been taught or all you know.

Eric Cross (39:41):

What I’m hearing is, don’t be afraid to question your own assumptions about what you’re doing. And don’t be afraid to adapt or change or modify. Kinda, pivot. Be flexible.

Melanie Trecek-King (39:51):

Yes, be flexible and pivot. And this is where I’m in a different position than middle school and high school educators. Because I have complete freedom over what I teach in my class.

Eric Cross (40:01):

Sure.

Melanie Trecek-King (40:01):

At the end of the semester, I always joke with non-majors that there’s nothing they have to know, which actually gives me a lot of flexibility, because I could teach ’em a lot of different things. So if there are things that you have to teach students, obviously that’s one thing. But I personally think that the way that we’ve been teaching science needs a refresher. A rethinking. And so I would say, “If you want your students to learn science literacy, honestly ask, what does that mean to you? And what would that look like to get to that point?” For me, though, it was also keeping in mind that maybe I didn’t already know the best way to do that.

Eric Cross (40:43):

One of the things you mentioned earlier is trying to reach out to educators. And I know that when we work together, it’s a force multiplier. And what you’re doing is developing skills. And there’s these skills that are happening right now in academia that you’re doing. And then how do we transfer that into middle and high school. Or, I’m sorry, middle and elementary school, high school. We need to get more people into this conversation to kind of brainstorm and figure that out. We have a Facebook group, Science Connections: The Community, where we have educators that gather. That can be one place we start the conversation. And again, I know on your website you’ve been super active on social media; you’ve grown your presence on Twitter and all these different places, engaging with folks. Which is awesome. ‘Cause I know I see your posts and I’m saving the things that you’re posting and I’m thinking of ways that I can do it in my classroom. I’m gonna take that product. By the way, is that on your website, the lesson that you do with the product?

Melanie Trecek-King (41:43):

No, actually. So the article, “How to Sell Pseudoscience” is … I know Bertha Vasquez wrote up a version of it.

Eric Cross (41:50):

Maybe we can grab that. ‘Cause we might be able to put that into the show notes for folks, because she’s a middle school educator. If there’s already something that’s been done for teachers like us, we’re like, “Yeah, let me get that and let me remix it and make it my own!” if there’s already a exemplar out there.

Melanie Trecek-King (42:04):

Yeah, she’s done it. And so I will absolutely share that with you.

Eric Cross (42:08):

So, all season long, we’ve been talking about science as the underdog. We kind of framed it, you know, science oftentimes takes a back seat to math and English. It’s kinda the first thing to go. Or the first area where time can get cut. Because of what gets tested gets focused on, oftentimes. And then in addition to that, when you’re a multi-subject teacher, elementary science isn’t just one thing — it’s every field. You know, you’re a biologist, which is different than a geologist. And when you’re teaching every subject, that’s a lot. And you might not have had a science class for years. And the realities that we’re seeing over and over with different researchers and practitioners is that science could actually enhance literacy, and building those skills. And I think you really talked about it with the critical thinking skills. Those can transfer. Or the administrator that said, “This is one of the only courses I’ve seen where it transfers to other areas.” Could you share maybe with our listeners, just any advice for advocating for science in their own world?

Melanie Trecek-King (43:13):

Wow, I’m not sure I’m qualified to answer that question! One of the things that comes to mind though — because I was listening to your last episode and educators … I honestly didn’t realize how little time they had for science. And how often science was then the first to go, to allow room for other subjects. But science overlaps with a lot of other issues. And so I feel like there could be a way to bring in science when teaching these other subjects. So, for example, argumentation and logical fallacies are easy to apply to reading and writing. Information literacy, and being able to find good information online, teaching students how to laterally read, to be able to check a source, or how to use Google effectively, to put in neutral search terms to find sources, or teaching students how to recognize the characteristics of conspiratorial thinking: All of these things can overlap with so many other subjects. So the scientist in me is a little biased towards science being important enough to do this. But try to bring it into the other subjects. It doesn’t have to be completely separate.

Eric Cross (44:43):

So integrating science into other things. And I … big believer. And a hundred percent agree with you. Now I’m gonna ask a question that kinda like takes us backwards. You shared an app with me when we first met that I thought was really cool. And I know it’s a friend or colleague of yours. But as a middle school teacher, I thought it was great, because it was something that my students could download and practice some of the skills that you’re talking about. Would you talk a little bit about the cranky uncle? Is it the Cranky Uncle app?

Melanie Trecek-King (45:17):

Cranky Uncle.

Eric Cross (45:18):

Could you share a little bit about that?

Melanie Trecek-King (45:20):

Yeah. Cranky Uncle is awesome. So, Cranky Uncle is the brainchild of John Cook, who is the founder of Skeptical Science and the author of the 97% Consensus study on climate change. Cranky Uncle … so he’s also a cartoonist. And Cranky Uncle is a cartoon game where … I don’t even have to explain who Cranky Uncle is to my students. Everybody inherently gets the, the character, right? So he’s like the guy at Thanksgiving that you don’t wanna talk to because he denies climate change and he’s just really cranky. But Cranky Uncle uses the techniques of science denial, which are summarized by the acronym FLICC: So it’s Fake experts, Logical fallacies, Impossible expectations, Cherry-picking, and Conspiratorial thinking. So he uses those techniques. Again, this is technique-based inoculation. So they recognize the techniques in the game, and you earn cranky points. And as you make Cranky crankier and crankier because you’re recognizing his techniques, you learn the techniques of science denial, and level up and open up other techniques. This is another one of those examples where climate change has a lot of science behind it, right? And if you wanted to get to the science behind climate change for any particular issue … so let’s say it’s cold today, so I’m gonna say there’s no climate change. OK? If I’m gonna unpack that at a factual level, and with science, we could be here for a while. But if I told you, “That’s like saying, ‘I just ate a sandwich so there’s no global hunger.’” OK? So that’s a parallel argument. Humorous. Love to use this kind of argumentation, ’cause it makes for some … I mean, it’s funny, but you get the point. It’s an anecdote. And anecdotes aren’t good evidence. So just like that, you could teach the technique of using an anecdotal fallacy for climate-change denial. So, I have my students play this game. You could do it when you’re studying argumentation. You could do it for science denial. I use an inoculation extension with that, where I have my students pretend that … um, actually, back up for a second. So I teach a class on critical thinking. And at the end of semesters I would get emails from students on, well, they’re failing the class, but they really shouldn’t, for all of these reasons. And reading these emails, I’m like, “If you think that’s a good argument, you clearly didn’t learn what I was hoping you would learn.” So I now have my students, early in the semester, after they play Cranky, pretend that it is the end of the semester and you’re failing the class and you’re failing because you didn’t do the work. Use at least four of the fallacies from class to argue for why you should pass. So they have to put it on a discussion forum, and they’ll say things like, “Well, if you fail me, then I won’t get into graduate school and then people will die and it will all be your fault.” Or, “My dog died, and so I was really sad.” Or, um, “You’re just a terrible teacher. And you’re short. So I don’t like you.” Or that kind of thing. So, oh, they love to attack my character. It’s really funny. But it’s supposed to be funny. And the point is, the students are using those arguments, they’re using the fallacies, to argue for something. And so by creating that misinformation themselves, they learn how those fallacies work. But taken together, I mean, everything that we just talked about there, Cranky Uncle, and the fallacy assignment, or whatever iteration you want that to be in, that doesn’t have to be in a purely science unit. Right? That could be sociology. It could be argumentation. It could be English.

Eric Cross (49:01):

Absolutely. That could be totally a prompt in an English class. And practiced in there. And then this could be an interdisciplinary thing, going back and forth between English and and science. Just having these discussions and looking at it from different angles. And you’re practicing the skills in two different contexts. So you get into argumentation. And then that app, I know I had fun with it. And the questions on there definitely resonate with people in my own family. I’m like, “I feel like I’m talking to exactly somebody that I’m related to right now.” <Laugh> Melanie, anything else that you wanna share, or discuss or highlight, before we wrap up?

Melanie Trecek-King (49:39):

So we could talk about lateral reading, if you like. ‘Cause I know a lot of educators use the crap test.

Eric Cross (49:45):

Please, please, please talk about that.

Melanie Trecek-King (49:47):

So, when evaluating sources, a lot of educators teach what’s called the CRAP test. And I wish I remembered what it stood for. But basically what you do, a lot of us have been taught when you go to a website, to figure out if it’s reliable, you wanna go to the about page. Read the mission; see who they are; maybe read some of the content; evaluate the language. So is it inflammatory? Are they making logical arguments? Are the links to reputable sources as well? And the problem is that if a site wants to mislead you, they’re not going to tell you that it’s a bunk site, right? They’re just gonna do a good job of misleading you. And so, what you wanna do instead … the CRAP test basically is an evaluation of a site. And that’s what’s called vertical reading. So you’re looking through a site to determine if it’s reliable. Uh, I think his name’s Sam Wineberg at Stanford, proposed something called lateral reading. Where, instead of on the site, what you wanna do is literally open a new tab and into the search engine type the source. You could do the claim, too. And then something like Reliability or FactCheck or whatever it’s that you’re checking, and then see what other reputable sites have to say about it. So, in their study, actually, they did a really interesting study where they compared professional fact checkers to PhD historians to Stanford undergrads. And they evaluated — I wish you could … um, there’s two pediatrician organizations. One’s like the American Association of Pediatrics and the American Academy of Pediatricians, something like that. They’re very similar sounding. So you give them to students. I do this with my students as well, the same study. So I give my students those two websites. And I say, “Which one of these is more reliable?” And they do exactly what most of us do, which is spend time on the site looking around. And most of the time, if not nearly all the time, they come to the wrong conclusion. And so then I tell them what lateral reading is: “OK, instead of looking through the site, open a new tab, search the organization and reliability.” Something like that. And it takes probably 30 seconds before they realize one of them has been dubbed by the Southern Poverty Law Center as a hate group. As opposed to the other one, which is like a hundred year old huge pediatrician organization that produces their own journals and so on. But nearly all my students are fooled. And in the study, none of the fact checkers were fooled. I’m gonna get the number right. It’s something like 50% of the historians and 20% of the Stanford undergraduates got the correct answer. And they spent a lot more time on it. So it’s a great way to teach students how to use the power of the internet to evaluate sources much more quickly and, effectively. And yes, use Wikipedia, right? Wikipedia is not a final answer, but Wikipedia is actually pretty accurate. So if Wikipedia is the first place you stop, then yes, go there, see what Wikipedia says, and then follow some of their sources.

Eric Cross (52:47):

What popped in my head was like, Yelp reviews for websites. That almost sounds like what it was. It’s like when I search for a product, I don’t go and read the product description marketing. ‘Cause that’s all designed to sell me on something. But I’ll go and look in Reliability, if it’s like a car, or just other sites to cross-reference. And that sounds like what you were talking about is like cross-referencing. Seeing what FactChecker [sic] said about this site, versus seeing what a site says about itself.

Melanie Trecek-King (53:14):

Well, that’s a great analogy. Because if I wanted to know if a product was effective, what the manufacturer says about the product, clearly there’s a strong chance of bias. Right? They’re going to be on their best, um, put their best foot forward. Versus, what do independent reviewers say about this product?

Eric Cross (53:35):

Yep. And I am known to research something to death. And I get something called “paralysis by analysis.”

Melanie Trecek-King (53:42):

Ohhhh, yeah.

Eric Cross (53:44):

And it’s so bad that even if I’m trying to buy, like, towels, I need to find the best-bang-for-the-buck towel. I have to defer some of these decisions out, because I’m on the internet for three hours now. I’ll be a pseudo-expert in towels, and thread count, and all of that stuff. But yeah, that maybe that’s just the science person.

Melanie Trecek-King (54:03):

I mean, I feel your pain. I do the same thing. <Laugh> It’s annoying. Like, it’s just towels. What does it really matter? But yeah.

Eric Cross (54:10):

Coffee! It doesn’t matter what it is. I just need to go, “OK, I have to use these powers for good. Otherwise I’m gonna be researching forever.”

Melanie Trecek-King (54:16):

I wanna say one other thing. So, again, this is a college class and I have a lot of freedom. But one of the driving philosophies behind the class is a wonderful quote in a book, Schick and Vaughn, How to Think about Weird Things. And they said, “The quality of your life is determined by the quality of your decisions, and the quality of your decisions is determined by the quality of your thinking.” And I know my students want a grade. But I’m really trying to teach them how to be empowered through better thinking. That’s where the name “Thinking is Power” came from. I mean, we say “Knowledge is Power,” but it’s not enough to know things. And there’s too much to know. So being able to think and be empowered to have your own agency and not fall for someone’s bunk is my goal for my students.

Eric Cross (55:07):

And doing that is gonna help them through the rest of their lives. Not be swindled, not be taken advantage of, be able to make better decisions. There’s so many benefits to building that skill. And I know your students have definitely grown and benefited. I’m sure you’ve heard, long after you’ve taught them, heard back from them and how they’ve applied that course to their lives. Melanie, thank you so much for being here. For a few things. One, for providing and filling this space where there’s such a need. Again, the critical thinking resources, the tools that you used, are so, so important. If we ever lived in a time where they were critical, it was really what we experienced during the pandemic in the last few years. We watched people’s information literacy and science literacy play out in real time. And we literally saw life-and-death decisions being made based off those skills. That highlighted, I think how important this is. And then, taking the time to generate resources for educators like myself, that we can take and adapt and put into our classroom and start teaching our students. ‘Cause like you said, by the time they get to you, they’re, they’re so far downstream or so far in a system that, depending on the teachers that they’ve had and the education system they’ve been in, may or may not have even touched on these things. They might have learned a lot of facts, but they may not have built their muscle to be able to critically analyze and interpret the world around them. And you’ve just — even the last year, it hasn’t even been a year since we talked the first time — I’ve watched your resources continue to grow, and you share them. And so I, on behalf of those of us in K–12, thank you. And thank you for being here.

Melanie Trecek-King (56:49):

Oh, well, thank you so much for this opportunity. Thank you for everything that you do, reaching out to other educators and for giving me a platform to hopefully reach other educators.

Eric Cross (57:00):

Thanks so much for listening to my conversation with Melanie Trecek-King, Associate Professor of Biology at Massasoit Community College and creator of Thinking Is Power. Make sure you don’t miss any new episodes of Science Connections by subscribing to the show, wherever you get podcasts. And while you’re there, we’d really appreciate it if you can leave us a review. It’ll help more listeners to find the show. You can find more information on all of Amplify shows at our podcast hub, Amplify.com/Hub. Thanks again for listening.