Correlational research

Purpose: To describe the relationship between two variables (but does not attempt to explain the relationship)

Data: Measures two variables (two scores) for each individual in one group

Features: May have a randomly selected sample of participants, Does not have different groups, Does not include a manipulation or intervention

Experimental research

Purpose: To explain cause and effect relationships between (usually two) variables

Data: Create two treatment conditions by changing the level of the independent variable and measure the 2nd (dependent) variable for participants in each condition

Features: Random assignment to treatment conditions, (Random selection is ideal but not required)

Quasi-Experimental research

Purpose: Attempts to explain a relationship (cause-and effect), but frequently has limitations

Data: Measure before/after scores for the treatment group and a different group that does not receive treatment

Features: Does not control assignment to group (e.g., no random assignment)

Nonexperimental research

Purpose: To describe a relationship between two variables (but not to explain the relationship)

Data: Measure scores for two different groups of participants (or one group at two different times)

Features: May have a randomly selected sample of participants, The research does not include a manipulation or intervention

Generalization and threats to validity

A well-designed study with good external & internal validity can generalize from a sample to the population, from one research study to another, from sample to real world

Threats to validity:

• Can be anticipated
• Can be minimized or eliminated generally (one can never eliminate all possibilities completely)
• Can be weighed for importance to choose the most crucial to eliminate

Research and validity

Variable

• Some studies have strong internal and external validity
• Others have moderate—or little to no validity

Awareness allows assessment

• Understanding threats to validity allows researchers to assess validity
• Does a study assess its validity limitations within the Discussion section?

Validity

The degree to which a research study accurately answers the question it was intended to answer

Note: this is similar to validity of measurement instruments [Unit 3], but now the focus is on the study as a WHOLE

Threats to validity: Any component of a study that introduces questions or doubts about the quality of the research process or the accuracy of results

External validity

The extent to which research results can be generalized to UTOS (units, treatments, observations, & settings) – in other words, people, settings, times, measures, and characteristics other than those used in the study

Threat to External Validity: Any characteristic of a study that limits the ability to generalize the study’s results

- Category 1: Generalizing across Participants or Subjects

- Category 2: Generalizing across Features of a Study

- Category 3: Generalizing across Features of the Measures

Category 1: Generalizing Across Participants

Selection Bias: Sampling procedure favors selecting some individuals over others, can result in the sample not being representative of target population

College students: Easily accessible (and often inexpensive) population, difficult to generalize from college students to adults as a whole

Volunteer bias: Volunteers are not necessarily representative of a population, certain variables predict the likelihood of participants volunteering for research studies (e.g., education, motivation, socioeconomic status, etc.)

Participant characteristics: A study within a specific demographic (gender, age, race/ethnicity, socioeconomic status) can limit the possibility of generalization, study results may only be applicable to a certain demographic

Cross-species generalization: Cannot presume that all non-human research applies directly to humans, similarities and differences between species must be noted, especially with regard to the mechanism or process of interest

Category 2: Generalizing Across Features of a Study

Novelty effect: Participation in the novel activity of a research study may alter participant responses from their typical, everyday responses, for example, participation may cause participants excitement or anxiety

Multiple treatment interference: In a series of treatment conditions, participation in one condition may affect participation in the next

• Fatigue effects: Getting tired after participating in one condition or treatment may cause scores to deteriorate in the next condition
• Practice effects: Participants may gain experience due to participating in one condition or treatment that may cause performance (scores) to improve in the next condition

Experimenter characteristics: Study effects could be specific to the experimenter and her/his characteristics

• Demographics (e.g., age, sex, ethnicity, SES) and personality (e.g., friendliness, prestige, anxiety, hostility) are characteristics that could be factors in the study outcomes
• Questions about validity related to experimenter characteristics are related to the extent to which the study results can be generalized to other experimenters (e.g., if you did this study, would you get the same results?)

Category 3: Generalizing Across Features of a Measure

Sensitization: Measurement may cause participants to respond differently to treatment, commonly occurs when a measure is given before the treatment (also called pre-test sensitization), especially relevant for measures that involve self-monitoring

Generality across response measure: When a variable is defined in a specific way for purposes of measurement, results may only be generalizable for that specific form of measurement, example: In the treatment of phobias, using heart rate to measure fear

Time of measurement: Scores are measured at a specific time (e.g., before or after treatment), effects of treatment may increase or decrease with time, and may be different from what was visible at the chosen time of measurement