##### Research Methods exam 2

mixed method research

combination of quantitative and qualitative approaches to data collection and analysis

quantitative data collection examples

primary data collection such as surveys with numerical scales

qualitative data collection examples

interviews

focus groups

observations

document review

explanatory sequential design

quantitative data-qualitative data-interpret results

exploratory sequential design

qualitative-quantitative-interpret

scout

gain direct insight into setting's dynamics through observing, reading and interacting with people on sight

can use focused interviews

reality check

assess the validity of quantitative measures

interpreter

understand the meaning of the patterns uncovered through quantitative methods

can help understand why an intervention has certain outcomes

incubator of theory

recast basic assumptions

reframe definitions

conceptualize the evaluation problem in a new way

refine intervention concepts

generate theoretical ideas

primary qualitative methods

review of relevant documents

direct observation

focused interviewing

purposive sampling

individuals are selected who meet a specific criteria

quota sampling

respondents with particular qualities are targeted

chunk sampling

individuals are selected based on their availability

snoball sampling

individual meeting specific criteria are identified who in turn identify other potential respondents of similiar criteria

structured interview with closed, fixed responses

all interviewees are asked the same questions and choose answers from among the same set of alternatives

structured with open ended responses

same open ended questions are asked to all

semi-structured interview

same general areas of info are collected but allows for adaptability

unstructured

informal and conversation like, no predetermined questions

population

the broader group of people who you would like to make generalizations about

sample

the group of subjects that actually participate in your study

non-random sample selection

potential threat to external validity

over sampling of minorities (good and bad)

convenience sampling (ie all college students) may need to compare to population

categorical/discrete variables

sex, occupation, race

some categorical variables are ordinal when possible values are ordered ie education: less than high school, high school, some college, etc

continuous variables

age, height, weight

can be divided into groups ie bmi, underweight, normal, overweight, obese

intake of fat-tertiles, quartiles, quintiles

categorical variables are described using...

frequency distributions

continous variables are described using

probability distribution-the range of values your variable can take

skew of the distribution is based on the....

tail: a longer tail on the left= a left skew=negative skew

longer tail on the right= a right skew= positive skew

descriptive stats: categorical variables

% of the total population in each category

descriptive stats: continuous variables

measures of central tendency, normal distribution=mean, non-normal distribution= median mode

continuous variables-measures of variability

normal distribution use standard deviation

non-normal- use range or interquartile range

range

high score minus low score

interquartile range

difference between the 1st and 3rd quartile 25%-75%

standard deviation

numerical indicator of the spread of the data values within a sample

1 SD=68%

2 SD= 95%

3 SD = 99.7%

standard error of the mean

numerical indicator of the expected difference between the sample and the population mean

what is the difference between SD and SEM

SD= how scattered a sample is

SE= how precise your estimate is when compared to the population/"true" value

SE formula

SE= SD/square root of the sample size

when SD is small....

when sample is larger...

it is easier for your mean estimate to get close to the population value

your estimate of the mean gets closer to the population value

when you goal is to describe the sample, report...

standard deviation

when your goal is to indicate how precise your measurement is in relation to the pop use,

standard error

standard error is most useful when we calculate....

confidence interval

confidence interval

calculated from sample data nd gives an estimated range of values which is likely to include an unknown population parameter

what does a confidence interval mean

if the same population is sampled numerous times and interval estimates are made on each occasion, the resulting intervals would bracket the true population parameter 95% of the time

confidence interval formula

mean +- 1.96 * standard error

3 steps of hypothesis testing

make an assumption

collect data

rejct or don't reject the initial assumption based on the data

null hypothesis

there is no difference among groups or correlation between variables

alternative hypothesis

there is a difference among groups

when you reject the null hypothesis

you are never 100% sure, there is always a chance that we made an error

type 1 error=false positive

the null hypothesis is rejected when it is true (alpha)

telling a male he is pregnant

type 2 error

the null hypothesis is not rejected when it is false (beta)

telling a 9month pregnant woman she is not pregnant

alpha is

the level of significance ie p-value

power is

the sensitivity of your analysis to correctly detect if there is an association

p-value is

the probability of finding the observed or more extreme results when the null hypothesis of the study question is true

steps of hypothesis testing using p-value

specify null and alt

use sample data to calculate the value of the test stat

use the known distribution of the test stat, calculate p-value

compare p-value with the pre-set sig level-if p-value is less than alpha (0.05) reject the null in favor of the alt hypothesis.

if the p-value is greater than the alpha, do not reject the null

p-value interpretation

p value is greater than 0.05= reject the null, there is an association

p value less than 0.05-do not reject the null, there is no association

what you need to calculate an acceptable sample size

acceptable power

acceptable probability for type 1 error

expected effect size

variability of the measurement

describe relationships-qualitative

is there a relationship?

positive or negative?

linear/non-linear

outliers?

describe relationships-quantitative

1 unit change in x is associated with how mnay units of change in y?

scatter plot

graphic rep of 2 variables in which independent variable is on the x-axis and the depend variable is on the y axis

cluster

smaller groupings of data

may show an effect modifier in those subjects

correlation

if entities are correlated they should change together in a predictable fashion

positive correlation

when variables increase or decrease together /<what the line looks like

negative correlation

when variable increases while the other decreases \<example of the line

pearson correlation assumptions

when variables are continuous

there is a linear relationship

normal distribution

no outliers

use spearman if assumptions are violated

interpretting pearson

r ranges from -1 to +1

negative r= negative association

larger the absolute value= a stronger correlation-ie plot looks less scattered

rule of thumb for inerpreting the size of a correlation

.9-1 very high positive

.7-.9-high positive

.5-.7 moderate positive

.3-.5 low positive

0.0-0.3= negligible correlation

r has nothing to do with...

slope

pearson r degrees of freedom equals...

n-2

coefficient of determination r2

the amount of common variance shared by the two variables

what percent fo the variability of one variable is explained by the variability of the other variable

weak r does not mean no relationship

it means there is no LINEAR relationship

r and r2 can be greatly affected by....

OUTLIERS

regression used to...

quantify the relationship and predict how much change we expect to see in one variable when other variables change

simple linear regression

used to predict one dependent variable suing one explanatory variable and a constant

SLR formula

Y=B0 +B1* X

B0=intercept

B1 = slope

SLR makes a...

line of best fit

least squares method is the most commonly used method

least squares regression line is a line that....

makes up the sum of the squares of the vertical distances of the data points from the line as small as posble

residuals

the vertical distance between the actual and predicted values of y

when to use SLR-assumptions

two continuous variables

linearity

stat independence of errors

constant variance of errors

normality of error distribution

interpreting SLR

usually pay attention to the slope (B1)

hypothesis testing for SLR

want to determine if B1 is significantly different from 0

p-value with SLR

calculated using a t-test, is it smaller than your preset alpha?

confidence intervals in SLR

calculated using the estimated B1 and the standard error of the estimated B1, does the interval include 0?

if confidence interval does not include zero than you can reject the null

multiple regression

involves one dependent variable and two or more independent variables

multiple regression formula

T= B0+B1*X1+B2*X2+B3*X3.....ETC

one unit change in x predicts b1 unit change in y when all other xs are held constant

why use multiple regression?

better predictor

explore relationship between y and multiple Xs simultaneously

control for confounding

explore interaction

t-test is used...

to study the difference in 2 groups

students t-test

study sample vs population

study bmi among local students and want to know if they are more/less obese than typical us children

two dependent study samplesgroups t-test

two independent study samples

2 exercise programs and want to know if their fasting glucose levels are different after completing the programs

here you compare the groups to each other

two dependent study samples t-test

samples are related in some manner

you want to know whether exercise has an impact on fasting glucose levels and you compare glucose levels before and after exercise in 1 group of children

here you compare one group to 2 factors

t-test formula

t= (sample mean-population mean)/ standard error of the means |standard deviation/square root of n|

degrees of freedom for the students t-test

n-1

one tail t-test

when you expect the difference only goes in one direction

example- trial drug is cheaper, all you care about is if it is worse, not if it is better than an already existing drug

two tail t-test

when you expect the difference to go in both directions

interpret independent t-test

t score must be larger than the critical value from the chart in order to reject your null

independent t-test (groups)

assign subjects to 2 diff ex programs and want to know if their fasting glucose levels differ after completing the program

what influences independent t-test stat?

sample mean of the two groups, larger difference= larger t-value

standard deviation of the two groups-smaller Sd=larger t-value

sample size of the two groups-larger sample size=larger t-value

effect size-cohen's D

how large the effect of the intervention is

D stands for distance

0.2 or less= small difference

0.2-0.8-moderate difference

0.8 or more= large effect size

assumptions of independent t-test

random sampling

samples are independent

normal distribution

equal variance of the two samples

degrees of freedom for independent t-test

N1+N2-2

dependent t-test

when 2 samples are related

ie repeated measures-before and after treatment

want to study whether exercise affects blood glucose before and after an exercise program

degrees of freedom for dependent t-test

n-1, n= the number of pairs!!!

simple anova example

example-studying the effects of 3+ exercise programs on body fat

factorial anova example

studying 2+ dufferent components of health programs (ie diet and pa) and want to know how each of them affect body weight as well as their interaction effects

anova for repeated measures example

multiple ex programes and measure their body fat before and after program, which program is the best

anova assumptions

independence

normality

equal variance

how does anova work

compares between group variance with within group variance

steps for anova

1. calculate between group variance and divide it by its degrees of freedom (number of groups-1) = MSb

2. calculate within group variance and divide it by its degree of freedom (sample n- number of groups)= MSw

3. use f-statistic- MSB/MSW

if between group variance exceeds within group variance we reject the null

using the ctiritcal f values

top row- degree of freedom for the numerator (between group=number of groups)

far left column- degree of freedom of the denominator (sample size minus the number of groups)

factorial anova is used when...

you are interested in more than one independent variable

ie duration and intensity

mvpa and sedentary behaviour

interaction effect

does the effect of one independent variable on the dependent variable differ by the level of the other independent variable?

post hoc test-anova only tests...

whether there is a difference among groups, it does not tell you which groups are different

post hoc tests are used....

to follw up a significant anova

they take care of multiple comparison problem and retain the original alpha level

there are multiple test, choose appropriately

analysis of covariance (ANCOVA)

used to adjust for covariate

when you suspect the groups are different in certain characteristics that may influence your results, you use ancova to control for external factors ie confounders

operationalization

process of specifying how a concept or phenomenon will be defined/measured

quant easy

qual harder

measurement validity

the extent to which a test instrument measures what it is supposed to measure

a test is not universally valid, depends on goals of the testing and the subjects being tested

valildity is not....

dichotomous-we ask how valid a test is, not whether it is or isn't

types of validity

logical or face

content

criterion based-concurrent or predictive

construct

logical validity

the extent to which a measurement method appears on its face to measure the construct of interest

ie using a scale to measure weight

weakest evidence of validity

based on human intuition

usually assessed informally with no external standard

content validity

the degree to which a test serves as a representative sampling of content

often used in education (how good a standard test is)

criterion based validity

the degree to which the measurement of one test correlates with the other tests

ie a gold standard

concurrent validity

how well one measurement compares to the criterion standard

predictive validity

how well one measurement method predicts future events

ie a functional test to predict falls in the elder.

a weight scale is not predictive

criiterion validity with multiple measurements

you can use multiple regression to evaluate the collective validity of multple measurements to assess which measurements have the highest predictive power

ie skinflod measurements from different areas of the body

construct validity

used when you attempt to measure a theoretical construct that is not directly measurable ie intelligence, anxiety, trust

usually incorporate other validity measures with construct validity

reliability

how consistent or repeatable measurements are

reliability (shot group analogy)

high reliability is like a close shot grouping, whetheer it is on the bullseye or it is on the 1 circle

validity (9shot grouping analogy)

validity is not like accuracy. you can hit all shots in the 7 group but be spread out (low validity)

measurements of reliability

test-retest

equivalence or alternate forms

internal consistency

intertester reliability (objectivity)

test-retest

redo the test exactly the same on different days

used to asses temporal stability

assumption is that quantity being measure does not change over time

not suitable for volatile variables like mood

alternate forms reliability

refers to how similarly different versions of a test or questionnaire perform in measuring same entity

important for standardized tests that exist in multiple versions

internal consistency

how well the items within a test that are supposed to measure the same construct are correlated

uses the same day test-retest method

intertester reliability (objectivity)

the degree to which two independent testers can provide the same scores on the same subject

how well the tester evaluates the subject

comparing measures

sometimes you need to compare measurements from 2 different tests and direct comparison may not be appropriate

if results are normally distributed use the standard z scores

what variables require nonparametric methods?

categorical variables

count

rank data

skewed distribution

examples- bmi gorups, breast cancer incident, pain scale, vigorous pa in elderly adults (right skew)

chi square test

applied to frequency data (categorical variables)

evaluates the number of subjects in each category is different from what would be expected

one way and two way

one way chi-square

when you are interested in the freq distribution of only one variable

two way chi square

when you are interested in the joint freq distribtuion of two variables

chi square df

number of categories minus 1

what to know about nonparametric tests

no assumptions about the distribution of data

usually use ranking instead of the actual value of the variable (robust for outliers)

often less powerful than parametric tests (results in a larger p-values and are more conservative

things to consider when selecting stat tests

how many variables?

types of variables (continuous or categorical)

distribution of variables

purpose of the study

one variable analysis

categorical variable:frequency table

continuous variable measure of central tendency-

normal distribution-mean

non normal distribution-median/mode

measures of variability

normal dist-SD

non-normal dist- range/interquartile range

2 variable analysis if both are continuous...

normal distribution: pearson

non-normal-spearman

2 variable analysis if both are categorical...

chi-square

2 variable analysis if one is continuous and one is categorical (2 groups)

w/normal- t-tests

non-normal-mann-whitney or wilcoxon

2 variable analysis if one is continuous and the others are categorical (more than two groups

normal distribution-anova, repeated anova

non-normal-kruskal-wallis test, friedman test

more than 2 variable analysis when one dependent variable and multiple independent variables

to control for confounding-ANCOVA or multiple regresion

to evaluate effect modification-factorial ancova or multiple regression

when you have multiple dependent variables use..

MANOVA

MANCOVA

MULTIVARIATE REGRESSION