Data Collection

Concept

A good questionnaire is short, clear, logically ordered (easy and non-sensitive items first) and free from jargon, ambiguity, leading, loaded and double-barrelled questions. Closed-ended items (dichotomous, multiple-choice, rating, ranking) are easy to score; open-ended items give rich detail but are harder to code. Contingency or filter questions direct only some respondents to follow-up items. Pre-testing and a cover letter explaining the purpose improve quality and response. Online surveys are fast and cheap but suffer from self-selection, coverage bias (people without internet are excluded) and low control over who answers.

Concept

Test–retest reliability correlates scores from the same people on two occasions; practice and memory effects can distort it. Parallel-forms reliability correlates two equivalent versions. Split-half reliability correlates two halves of one test (often odd and even items); because the halves are shorter, the coefficient is stepped up with the Spearman–Brown formula, r(full) = 2r / (1 + r). KR-20 suits dichotomous items and Cronbach’s alpha (Cronbach, 1951) suits multi-point items such as Likert scales. Inter-rater agreement is shown by percentage agreement or Cohen’s kappa. Longer tests, clearer items and uniform administration raise reliability.

Concept

Content validity is judged by experts who check that items represent the whole domain, often guided by a table of specifications; face validity is only the surface impression. Criterion-related validity compares scores with an external criterion, either measured at the same time (concurrent) or later (predictive). Construct validity is shown by convergent evidence (correlation with measures of the same construct), discriminant evidence (low correlation with unrelated constructs), known-groups comparison and factor analysis; Campbell and Fiske (1959) proposed the multitrait–multimethod matrix. A test’s validity cannot exceed the square root of its reliability.

Concept

Observation may be participant or non-participant, structured or unstructured, overt or covert, and uses tools such as checklists, rating scales, anecdotal records, time sampling and event sampling. Rating errors include the halo effect (a general impression colours all ratings), leniency or severity error, central tendency error and the logical error. Interviewer bias can also distort answers. Projective techniques present ambiguous stimuli so that people project hidden feelings: the Rorschach inkblot test (1921), the Thematic Apperception Test (Murray and Morgan, 1935), word association, sentence completion and draw-a-person tests. They are used mainly for personality assessment and are open to subjective scoring.

Concept

A standardised test has fixed content, uniform administration and scoring procedures, and norms obtained from a representative sample, so that an individual’s score can be interpreted against others. Norm-referenced tests compare a person with the norm group using percentiles, stanines or T-scores; criterion-referenced tests compare performance with a set standard. Examples are intelligence tests (Binet–Simon 1905, Stanford–Binet, Wechsler scales), aptitude and achievement tests, personality inventories (MMPI, Cattell’s 16PF) and interest inventories (Strong, Kuder). Speed tests stress time, whereas power tests stress difficulty.