Class Reflection (8/18) EDU 6982

Wednesday, August 18                                    Educational Research

Vocabulary

No reading due for today

Reflection

The final class day in Educational Inquiry provided an opportunity for all of us to share an article critique. Students in our class researched a variety of topics from fitness and academic achievement to self-concept to art and negative emotions. Hearing my classmates share their opinions of the strengths and weaknesses of each article made me  realize how much we learned in this course. I found a number of flaws in the study I shared with the class, despite the research problem itself being very interesting and significant to the future of physical education courses.

As I move forward and begin teaching again this year (I go back on Monday!), I wonder how I will utilize this information in my daily teaching life. I do think that I will use action research at times in my classrooms in order to improve the behavior or academic standing of a particular student. I also believe that I might someday be involved in writing an article for a journal using quantitative or qualitative research. I am not sure how it will happen or when, but I definitely have the interest.

Class Reflection (8/17) EDU 6982

Tuesday, August 17                        Chapter 13                          Educational Research

Vocabulary

1. Population validity: (text pg. 353) generalizability to other individuals.

When researchers look for the population validity of their research, the often must generalize based on characterisitics like race and gender.

2. Ecological validity: (text pg. 355) generalizability to other settings, times, treatments, and measures.

Ecological validity is strong when the results can be generalized to different settings.

3. Hawthorne effect: (text pg. 356) individuals’ realization that they are subjects in a study.

The Hawthorne effect can limit results because the subjects know they are in a study and thus their answers change.

Reflection

After today’s class and completing the article critiques, I gained a new perspective on evaluating research studies. Our class discussed the phrase “research says” or “research believes” a lot over the course of the last four weeks. I agreed with many who said that they don’t really buy into that phrase, but I didn’t really know why. Now I understand after examining the discussion and results section of a few journal articles. In one of my articles, the authors essentially excused away all limitations and conflicting evidence. It made me very skeptical. Yet another article explained very clearly and with supporting evidence why a relationship existed. The difference between the two was astounding. I feel that one of my “take-aways” from this class is that I can help administrators and other teachers interpret data and best practices. When we are making a decision at our school on a policy or something that involves educational research, I feel that I can help make informed decisions and interpret the data.

Class Reflection (8/16) EDU 6982

Monday, August 16                        Chapter 9 (241-243)                        Educational Research

Vocabulary

1. Single-subject design: (text pg. 241) individual behavior recorded before and after an intervention.

Researchers use individuals to study an intervention on behavior utilizing a single-subject design.

2. Baseline: (text pg. 241) measurement of behavior before the intervention.

The baseline refers to a period of time in which the target behavior is observed and recorded as it occurs without a special or new program or procedure.

3. Multiple-baseline design: (text pg. 243) more than one subject, behavior or setting.

In a single-subject multiple-baseline design, observations are made on several subjects, different target behaviors of one or more subjects, or different situations.

Reflection

Single-subject designs seem the most applicable as a classroom teacher when it comes to educational research. I don’t envision myself running or being a part of a quantitative study right now. Even a qualitative study seems quite daunting, but the single-subject design actually seems plausible. In some ways, I do this all the time in my class. When a student faces some difficulty, whether behaviorally or academically, I brainstorm ways to help that student. When a good idea comes up, we implement that “intervention” in class. The central differences are that I don’t record the data and I usually don’t go back to the baseline after the intervention. With the information provided from this class and the text, I have a better understanding of how to implement an intervention and ensure that it is successful.

Class Reflection (8/11) EDU 6982

Wednesday, August 11            Chapter 11 & 12                        Educational Research

Vocabulary

1. Ethnography: (text pg. 276) in-depth involvement in a culture to describe naturally occurring behavior.

Anthropologists have used ethnography to investigate primitive cultures for years.

2. Emic: (text pg. 283) participant wording.

Emic data contain information provided by the participants in their own words.

3. Phenomenological: (text pg. 291) understanding the essence of experiences.

A phenomenological study describes and interprets the experiences of participants in order to understand how it is perceived by those individuals.

4. Grounded theory: (text pg. 293) theory generated from qualitative data.

Grounded theory studies look at qualitative data, analyze it, and then create a theory based upon the information.

5. Triangulation: (text pg. 296) compares the findings of different techniques.

Triangulation enhances the credibility of a study by collect data in a variety of manners including observation, research and interviews.

6. Reconnaissance: (text pg. 333) self-reflection of focus.

Reconnaissance occurs as you take time to self-reflects on the area from the perspective of your own beliefs and from the context of the classroom or school.

Reflection

After listening to the lecture on what defines qualitative studies, much of the interest in the class, and subsequent discussion, turned to the debate between qualitative and quantitative studies. Some members of the class advocated for quantitative studies because the numbers or facts present a more compelling argument. Yet others pleaded on behalf of qualitative for the story they tell including the span of human emotions. It seems to me that a mixed-method approach, or something close to it, would get the most attention from a reader. A study based mostly on numbers, without a story highlighting the “why” or significance of those numbers, lacks punch. Yet a story without much empirical evidence seems to be too “fluffy”. Imagine a study that looked at the impact of inquiry-based learning in a social studies classroom. If the researchers could include not only the numerical impact (using a test like the Stanford Achievement) to show improvement, but also provide quotes from students on why they like the method, then there exists a more compelling argument. The mix of human experience and numerical proof of improvement would seem to me to get the most attention from educators, administrators, teachers, and parents. With their attention in hand, the chance of implementation increases as well.

Class Reflection (8/10) EDU 6982

Tuesday, August 10                        Chapter Ten            Educational Research

Vocabulary

1. Level of significance: (text pg. 254) probability of being wrong in rejecting the null hypothesis.

Due to the probability being .20, the level of significance proved to be only 20% that the independent variable achieved the result the researchers wanted.

2. Type 1 error: (text pg. 254) rejecting the null hypothesis when it is true.

The researchers attributed the lack of correlation to type 1 error after the researchers found no difference between the group that received a reading intervention and those that didn’t.

3. Confidence interval: (text pg. 255) interval in which the true value of a trait lies.

When comparing the two groups, the data showed only a small overlap thus providing a high confidence interval to the researchers.

4. Parametric: (text pg. 258) statistical procedures based on certain assumptions.

When a researcher used interval-level measures and has a population that is normally distributed, they are looking at results that are parametric statistics.

5. Analysis of variance (ANOVA): (text pg. 259) compares two or more means.

The ANOVA test compares two or more means to find the probability of being wrong in rejecting the null hypothesis.

6. Univariate: (text pg. 264) one dependent variable analyzed.

A univariate study analyzes only one dependent variable.

Reflection

After comparing the various types of experimental and non-experimental designs from Chapter Nine during class, my group held an interesting discussion regarding our hypothesis. As the group studying the effects of fitness on academic achievement, we felt that our question and hypothesis led us to the simple design of single-group pretest-posttest. Yet after some discussion on our population, we realized this wouldn’t yield great results for a comparative study. We knew that we are selecting essentially “couch potatoes” or students not involved in organized sports. Thus, despite allowing any students to participate in our fitness program, we thought that single-group would be the best design. Yet most research shows that by comparing a control group and an experimental group, researchers can find more relevant and important data. So we switched to nonequivalent-groups pretest-posttest design. But there is more! After discussing this as a class, it turns out that even though our population is selected, our sample can still be random. Thus we finalized the design of randomized-to-groups pretest-posttest. With this we can give ourselves the best chance for a reliable and valid study.

Class Reflection (8/9) EDU 6982

Monday, August 9                        Chapter Nine                        Educational Research

Vocabulary

1. Internal validity: (text pg. 220) control of extraneous variables.

Due to the fact that the plausible extraneous variables were controlled, the study had high internal validity.

2. Statistical regression: (text pg. 223) threat from change of extreme scores to those closer to the mean.

Due to statistical regression, the outliers scoring very high on the first test came back to the mean in the second test, thus the internal validity wasn’t as strong as the researchers hoped.

3. Diffusion of treatment: (text pg. 224) threat from treatment effect on one group affecting other groups.

The diffusion of treatment became obvious as the control group began getting resentful toward the group receiving the intervention.

4. Experimenter effects: (text pg. 224) threat from characteristics or expectations of the experimenter.

The age and gender of the experimenter seemed to have detrimental experimenter effects on the subjects.

5. External validity: (text pg. 225) generalizability of results.

The study proved to have strong external validity as it applied to most classrooms in the United States regardless of differences in characteristics.

6. Factorial designs: (text pg. 234) containing two or more independent variables.

By studying more than one independent variable and their interactions, the researchers employed the factorial design experiment.

7. Intervention fidelity: (text pg. 239) extent to which intervention occurred as intended.

When the subjects filled out the daily logs as the experiments wished, they knew they had strong intervention fidelity in their study.

Reflection

I consider myself a pretty strong visual learner. There are times that someone will begin reading a passage from a book to me and I simply ask to stop and read it myself. I can be auditory, but it is a lot more work for my brain to process and begin analyzing information (especially when something is read out loud). With this information in mind, one would think I would use more graphic representations in my classroom. Yet I don’t think I do it enough.

After today’s class in which each group had to visually present the threats to internal validity, I am determined to have my students complete more non-linguistic representations. Not only does it vary the class activities and provide a nice break, it creates meaning in the mind for many students. I know I went from a peripheral understanding of the threats to a very solid comprehension after the activity in class. I look forward to adapting these activities to my history classroom this year.

Class Reflection (8/5) EDU 6982

Thursday, August 5

Vocabulary

No reading due today

Reflection

Today’s class provided a nice summary of validity and reliability for me, as well as an explanation of the different types of non-experimental research. While yesterday I felt a bit confused about validity, I know can boil it down to a simple statement: does the test or experiment do what it is supposed to do? This makes it much easier to remember what validity means.

When discussing non-experimental research, it became very clear that education could be the recipient of many of these studies. After brainstorming a list of topics that cannot be used in an experiment, it becomes pretty obvious that education sits right at the top of the list. Thus many of our studies in the educational field must be ex post facto or causal-correlational studies. Yet these have great value and I would argue could even be more valuable than experimental studies. In an experiment, it really is impossible for the researchers to prevent the subjects from knowing they are part of an experiment. Seems obvious right? Yet if we look back at demographics and achievement or teaching style or class size, we get information that is less contaminated in many ways. Thus I see great value in these types of non-experimental studies.

Class Reflection (8/4) EDU 6982

Wednesday, August 4                        Chapter Eight                        Educational Research

Vocabulary

1. Predictor variable: (text pg. 196) predicts the criterion variable.

Since high school GPA precedes college GPA, it is called the predictor variable.

2. Criterion variable: (text pg. 196) the predicted dependent variable.

College GPA would be the criterion variable when using high school GPA as a predictor for college success.

3. Multiple regression analysis: (text pg. 197) combines several predictor variables.

When trying to predict college success, researchers used multiple regression analysis be combining the predictor variables of SAT scores, high school GPA and extra curricular activities.

4. Logistic regression: (text pg. 197) combines several variables to predict a dichotomous outcome.

When a researcher uses pass/fail as the dependent variable combined with various predictor variables, they use logistic regression to explore the relationship.

5. Attenuation: (text pg. 200) lowering of correlation because of unreliable measures.

As the measures used in a study become unreliable, the correlation lowers and is called attenuation.

6. Causal-comparative: (text pg. 202) nonexperimental studies designed to determine cause and effect.

When the researchers cannot control the program or study, they use a causal-comparative design such as different ways the principals evaluate teachers.

7. Ex post facto: (text pg. 202) presumed cause that occurred in the past.

Sometimes researchers cannot have any involvement in a study, thus instead they look into the past to see if for instance class size impacts achievement.

Reflection

While we catch up to the reading in class, I find myself struggling a bit with reliability and validity. Maybe if I write it out here I can gain some more understanding. When I look at whether a instrument is reliable or not, I want to see if it has been used over and over in studies. Then I want to make sure that there are as few chances for errors as possible. When I am looking for validity, I want to make sure that variable chosen in valid for that study. Sometimes an independent variable can be valid for one dependent variable, but not for another. I think I still need some examples to fully understand this. Hopefully, I will get them in class Thursday.

Class Reflection (8/3) EDU 6982

Tuesday, August 3                        Chapters Six & Seven                        Educational Research

Vocabulary

1. Nominal scale: (text pg. 133) numbers assigned to categories.

The researchers used a nominal scale in assigning “1” to males and “2” to females.

2. Ordinal scale (text pg. 133) numbers rank ordered.

The participants in the study were lined up by height with the tallest person being 1^st and so on down the line.

3. Interval scale: (text pg. 134) equal intervals between numbers.

When John scores 90 and Hank scores 80 on an interval scale, we know they are the same difference as June who scores 20 and Jill who scores 30.

4. Ratio scale: (text pg. 134) numbers expressed as ratios.

When starting a scale based upon height, if you start at 0 it can be a ratio scale as there are equal intervals between inches.

5. Histogram: (text pg. 135) bar graph.

The researchers used a histogram to display the data in graph form.

6. Positively skewed: (text pg. 137) large numbers of low scores; few very high scores.

In a study of administrators’ salaries, the distribution would be positively skewed if most of the salaries were relatively low and a few were very high.

7. Standard deviation: (text pg. 139) average distance of the scores from the mean.

A standard deviation of 35 shows the reader that the scores in the study were very widespread across the board.

8. Correlation coefficient: (text pg. 142) number between -1 and +1 that indicates the direction and strength of the relationship.

When the correlation coefficient is closer to +1, the relationship between two variables being studied is strong.

9. Validity: (text pg. 144) the extent to which inferences are appropriate and meaningful.

A test for beginning teacher competency may be valid for how much those teachers know about classroom management.

10. Reliability: (text pg. 149) consistency of scores.

The more ambiguous questions are and the more tired the participants are when taking the test, the less reliable the instrument for that test proves to be.

11. Stability (test-retest reliability): (text p. 150) measured by giving the same instrument twice.

Researchers found high stability in their test of academic achievement by testing the group in June, intervening with a tutoring program, and retesting the group in December using the same instrument.

12. Equivalence: (text pg. 151) correlation of two forms of the same test.

Rather than giving the same test, researchers can find equivalence by administering gives alternate but equal tests to the same group.

13. Norm-referenced: (text. 160) interpretations that compare subjects with others.

A reference group, called the norm group, is the group that scores on the test are compared to when using a norm-referenced instrument.

14. Criterion-referenced: (text. 161) interpretations that compare subjects with a standard of performance.

The group being tested is compared to an established level of performance or skill when using a criterion-referenced instrument.

15. Semantic differential: (text pg. 169) a 7-point scale with adjective pairs as end points.

By placing adjectives on opposite ends of the scale as anchors, the researchers checked on attitudes of participants using a semantic differential.

Reflection

The last two chapters (six and seven) of the Educational Research text were full of information regarding using instruments in research and whether they are reliable and valid. While it seemed to be quite overwhelming (evidenced by my list of 15 vocab words!), I feel that I am getting the hang of it. Some of the information is easy to digest such as the different types of scales. I think this is due to the activities we performed in Educational Inquiry class. By getting up and visually showing the class what a nominal, ordinal and interval scale looked like using height, the information sticks in our brains a bit more. I find this helpful in my classroom as well and it was a good reminder to try this more often. It especially helps to use something so visual like height, when we see the difference between Andrew and David as compared to Molly and Janelle.

I also liked seeing the visual on the PowerPoint of positive and negative correlation. It helped to see a program where the graph can move and we can see the scatter points. This shows very clearly how scores from participants in a study can be very close together or quite far apart, and then the impact this has on correlation of variables. I feel that it also helped us to have to “act out” correlation. It triggers a more creative part of the brain and provides a snapshot memory of correlation for me. I can now think of no correlation as people randomly doing things in the room (no relationship or very little between two variables) and positive correlation as cartwheels and clapping.

Class Reflection (8/2) EDU 6982

Monday, August 2                        Chapters Three & Four                        Educational Research

Vocabulary

1. Meta-analysis: (text pg. 67) a quantitative review of previous studies.

Utilizing meta-analysis, Fred Folvary completed his study on genetics and obesity through examining a number of previous studies and analyzing the data himself.

2. Refereed: (text pg. 69) sent to reviewers for an evaluation.

The Journal of Internal Awesomeness is refereed because they utilize the most awesome people in reviewing studies sent to them.

3. Retrieval algorithms: (text pg. 92) determine both how many pages each search retrieves as well as how the results of each search are ordered.

When searing “Karate Kid” on Google, the retrieval algorithm produced the IMDB website first after going through all possible websites.

Reflection

The first day of the second week in Educational Inquiry felt jam-packed full of information and resources. While we spent the first full week discussing different aspects to research and research studies, Monday became a “hands-on” day. After meeting in the computer lab in the SPU library, we began searching ERIC and PsychINFO for articles pertaining to our hypotheses and research problem. While I felt confident in beginning my search, I appreciated the tips provided by Librarian Cindy Strong in finding full text articles. So often I run into abstracts of what I deem to be “perfect” articles. With Mrs. Strong’s help, I found a variety of ways to access full-text. Sounds simple and somewhat trivial, but in research it is essential.

I also found the chapter entitled “Introduction to the Internet” to be really useful in a different manner. As a high school history teacher, I teach my students how to research, especially for their National History Day paper in junior year. This massive, five month process in many ways typifies and identifies their junior year for them. Thus learning what is “good” research and what is “poor” research is very important. I liked the way the text explained the pros and cons of internet research and provided some valuable links to websites that can help. I plan on using this text as a guide for my teaching this year when it comes to research.