S4E11: Revisiting Preregistration and Registered Reports

It’s no mystery that research papers are essential for academic work. What IS a mystery is how to figure out who should be an author on those academic papers and how to have conversions about it. In this episode, Jess and Sara go over some authorship roles, guidelines and rules that govern authorship in the field, and talk about different ideas of determining order of authorship. This episode was recorded live at the Emerging Perspectives on Early STEM Learning, Development, and Education Workshop hosted by Purdue's Center for Early Learning.   In this episode we mention:  APA Standards for authorship. The Committee on Publication Ethics (COPE),  The 14 CRediT authorship roles:  https://credit.niso.org  The Authorship Project: https://repository.charlotte.edu/islandora/object/work%3A921  Connect with the podcast on twitter @within_between, or email us letters about developmental science at [email protected]. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. This episode was recorded October 19, 2023.

In this episode, Jess and Sara reckon with what the preregistration revolution means in developmental science. We first visited preregistration and registered reports in Season 3. Since then, we’ve been a part of many preregistrations and registered reports, and have reviewed and handled them as editors. Through that process, we’ve noticed that questions often come up about whether something is really a preregistration and whether some studies or analyses can or should be preregistered.  We mention:  Several definitions: Open Science Framework:  “Preregistration is the practice of documenting your research plan at the beginning of your study and storing that plan in a read-only public repository”. APA: “Preregistration allows researchers to specify and share the details of their research in a public registry before conducting the study.” IES - no definition, just that you should do it; it’s part of the SEER standards.  REES similarly just says “REES is designed to increase transparency of and access to information about both ongoing and completed efficacy and effectiveness studies.” Even the preregistration revolution article doesn’t include a definition. The closest it gets (about ⅓ of the way through the article is “Preregistration of an analysis plan is committing to analytic steps without advance knowledge of the research outcomes.”  Paper about preregistering exploratory analyses: Compares it to a funded sea voyage:  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7098547/  Connect with the podcast on twitter @within_between, or email us letters about developmental science at [email protected]. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. This episode was recorded 9/10/2023.

What can we say about the often used and often confused statistical approach that is moderation? Quite a lot actually. In this episode Jess and Sara introduce the basics of moderation (aka statistical interactions) using absolutely no math (or very nearly absolutely no math). Listen to get a primer on the kinds of questions you can ask with moderation, and to get answers to common moderation questions like: Why is my interaction negative if my main effect is positive? Can I interpret the man effect if the interaction is significant? Why is my interaction significant if my main effect isn’t? And how do I graph the result when I have continuous variables?  (see the figure Sara is talking through on our website: http://www.withinandbetweenpod.com/) Connect with the podcast on twitter @within_between, or email us letters about developmental science at [email protected]. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. Recorded August 5, 2023.

Related to this podcast's interests in data sharing, this week Jess and Sara talk about common data elements initiatives. These are movements that are hoping to get all of the researchers in the same field or subfield to agree to a given set of assessments, measures, procedures, and/or reporting metrics (Think: Everyone who measures mother’s education would ask the same stem question with the same eight category responses, which would be coded and reported in the same way).  We’ll talk about different types of initiatives, the reasons why proponents think it’s a great idea, and what major concerns might come up. Sara also gives us a mini lecture introducing genome-wide association studies. Trust us, it’s related!  In this episode, we talk about:  The NIH Common Data Elements Program: https://heal.nih.gov/data/common-data-elements  NIH Common Measures website: https://www.phenxtoolkit.org/  A paper about the NIMH common data elements: Barch, D. M., Gotlib, I. H., Bilder, R. M., Pine, D. S., Smoller, J. W., Brown, C. H., ... & Farber, G. K. (2016). Common measures for National Institute of Mental Health funded research. Biological Psychiatry, 79(12), e91-e96. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4968690/ What’s the difference between common measures and common metrics:  de Beurs, E., Boehnke, J. R., & Fried, E. I. (2022). Common measures or common metrics? A plea to harmonize measurement results. Clinical Psychology & Psychotherapy, 29(5), 1755-1767. Paper describing the openly available Project KIDS data: Van Dijk, W., Norris, C. U., Al Otaiba, S., Schatschneider, C., & Hart, S. A. (2022). Exploring individual differences in response to reading intervention: Data from Project KIDS (Kids and Individual Differences in Schools). Journal of Open Psychology Data, 10(1). Educational attainment GWAS paper data harmonization appendix: https://static-content.springer.com/esm/art%3A10.1038%2Fs41588-022-01016-z/MediaObjects/41588_2022_1016_MOESM1_ESM.pdf   Connect with the podcast on twitter @within_between, or email us letters about developmental science at [email protected]. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. Recorded July 9, 2023.

Measuring change is our first ever LIVE EPISODE! Recorded at the Association for Psychological Science conference with a live conference audience! Over here in the developmental sciences, we are often trying to measure or predict how much people grow and change over time. To do that, we’ll sometimes measure a skill or ability twice (for kids, maybe that’s once near the beginning of the school year and once at the end of the school year). In this episode, Jess and Sara talk about options for statistically modeling and predicting change between two timepoints. We talk about two basic models: the simple difference scores and residualized gain scores, explain why we can’t just measure people at post test, and conclude that the question is important in making your decision. Have a listen as we get to take some related questions from the audience.  Things we mentioned:  If you are working in the latent space and have a measurement model at each time point, you may need to establish measurement invariance across time. To do so, you can follow steps in Brown (2015; pp. 259–265).  Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford Publications. Connect with the podcast on twitter @within_between, or email us letters about developmental science at [email protected]. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. Recorded May 27th, 2023.

Mediation models! At their core, mediation models explore the relations among three variables: A predictor, an outcome, and a secret third thing they call a Mediator. You’ve probably seen mediation models out there in the published literature, with folks testing paths, seeing if a relation is still there if a mediator is involved, or testing for an “indirect effect”. Jess and Sara talk about all of this in this episode. What is mediation, what kinds of questions can you ask with it, and why are people so mad about it? Jess and Sara go over the basics of this method, including a few examples, and talk a bit about the controversies surrounding it.  A link to Barron and Kenny’s Mediation paper  A link to MacKinnon’s paper on Mediation analysis  Some information on the PROCESS macros we mentioned.  A link to the paper Jess mentioned on reading and math language.  A link to the paper Sara mentioned on the home math environment.   Connect with the podcast on twitter @within_between, or email us letters about developmental science at [email protected]. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer. Recorded May 13th, 2023.

In the second part of our series on academic conferences, Jess and Sara talk through the unique vocabulary of conferences (the word plenary, by the way, means “fully attended by all entitled to be present”), and walk through a conference day. Conferences are set up for those of us who love being around big groups of people, so we also talk about how those who are more introverted or shy can get the most out of the experience. Happy conferencing! Connect with the podcast on twitter @within_between, or email us letters about developmental science at [email protected]. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer.

One of the ways the research machine operates is through research conferences. Scientists present new work, hear about others’ work, and (maybe most importantly) network with one another. In the first of a two-part episode about conferencing, Jess and Sara talk about what research conferences are and why you might want to go. We talk through how we choose which conferences to attend, how to plan your travel, and the importance of a comfortable pair of shoes. Connect with the podcast on twitter @within_between, or email us letters about developmental science at [email protected]. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer.

If a scientist learns something new about the world but doesn’t tell anyone, have they really learned something new? Telling people is precisely the point of a dissemination plan or broader impact statement that the Institute of Education Sciences or National Science Foundation asks for as part of their research grant applications. Federal agencies want you to share what you learn with the world. You might be used to thinking about this as publishing scientific papers or presenting your work at academic conferences, but that’s only one potential audience. These agencies also want you to share your work with the public. In this episode, Jess and Sara talk about creative and engaging ways to share your work with the public, including playing video games for science! We talked about:  Dr. Jenny Root plays Mortal Kombat and talks about her research  Connect with the podcast on twitter @within_between, or email us letters about developmental science at [email protected]. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer.

Developing research questions is one of the most challenging aspects of the research process. This one little statement reflects so much about the rest of your project or paper. Jess and Sara reflect on how they develop their own research questions, the challenges with research questions and secondary data, and how they help teach students to develop research questions in their own work. Spoiler alert: Part of the answer is to read. A lot. And then read more. Connect with the podcast on twitter @within_between, or email us letters about developmental science at [email protected]. More episodes and podcast information at WithinandBetweenPod.com. Follow Dr. Hart on twitter @Saraannhart Follow Dr. Logan on twitter @Jarlogan. Our theme music was composed by Jason Flowers. Our logo was created by Nathan Archer.