I. Test-taker characteristics
Physical/ Physiological
Short-term aliments
Longer-term disabilities
Age
Gender
Psychological
Personality
Memory
Cognitive Style
Affective Schemata
Concentration
Motivation
Emotional state
Experiential
Education
Examination preparedness
Examination Experience
Communication experience
Target language country residence
II. Cognitive Validity
Factorial approach to RC
various components
non-observable mental competencies of
Reading process
Types of Reading
careful
expeditious
skimming
scanning
level of cognitive demand
Reading Subskills approach 80s
Grellet, Munby, Nuttall
Break reading down ino constituent competencies
Generating Reading EXECISES
than RCQ
Coursebook with practices
Disadvantages
more organisationally than theoretically driven
unclear distinction b/w
R skills
R strategies
Cognitive processing approach
Reading types
cognitive processes
text complexity
Mismatch
Theory
R process
Bottom-up
Top-down
combining, but diverged
Practices
A Model of Reading p43
Goal Setting
Careful R
Local
understanding sentence
Global
main ideas
overall text
Expeditious R
Local
scanning
Global
skimming
Word recognition
lexicon form
orthography
phonology
morphology
Lexical Access
lexicon lemma
meaning
word class
Syntactic Parsing
syntactic Kw
Establish propositional meaning
Inferencing
Building a Mental model
Creating a text level representation
Creating an intertextual representation
Monitoring Comprehension
GK (topic Kw)
Text Structure Kw (Genre, Rhetorical Tasks)
III. Context Validity
Exam should aim
Contextual
situational
Cognitive
Interactional
Task Setting
Response method
1. Selected Response format
MCQ
T/F
Right/ Wrong/ Doesn't say
Matching
Multiple Matching
Gapped Text
2. Constructed response format
SAQ (short Answer Questions)
information transfer
Random deletion cloze & Selective deletion gap filling
Reading into Writing
weighting
points assigned to a test item
score value
with Rational
knowledge of criteria
clear idea of a task purpose
order of items
expeditions types of reading
focus for test items
Search reading
Scanning
channel of presentation
Comp by Non-verbal information
Interaction b/w Text and
Diagrams
charts
graphs
tables
pictures
notices
illustrations
text length
Short
long
time constraints
speed
Setting Administration
physical conditions
uniformity of administration
security
Linguistic demands: task input & output
overall text purpose
Referential
to inform
conative
to persuade or convince
emotive
to convey feelings or emotions
poetic
to entertain, delight, please
phatic
to keep in touch
writer - reader relationship
Audience Addressed
real, intended readership
Audience invoked
fictitious readership for a rhetorical purpose
discourse mode
Text Organization/ type
functional resources
a common scale
levels
argument
disagree
inform
Advocate
suasion
Recommend
Threaten
rational enquiry and exposition
Justification
propositional
demonstration
illustration
emotional relations
acknowledgement
judgement and evaluation
Assess
personal emotions
grammatical resources
K of Syntactic Structure
Lexical resources
Nature of information
Abstract
concrete
Content knowledge
Text Knw
Background Knw
IV. Scoring Validity
Item difficulty
CTT (classical test Theory)
IRT (Item response theory)
IF (Item Facility)
Item discrimination
Point Biserial Correlations (rbpi)
Item discrimination
Internal consistency
3 categories of instrument reliability
1. Alternate-form Coefficients
2. Test-retest coefficients
3. Internal consistency coefficients
Error of Measurement
no test is likely to be totally accurate
useful measure of test consistency
also in variation in test scores
Marker reliability
Grading and Awarding
V. Consequential Validity
Social Consequences of test use
appropriateness
meaningfulness
usefulness
Washback on individuals in classroom/ workplace
Macro level
socio-cultural level
educational systems
society in general
Micro level
local and personal level
Impact on institutions and society
Avoidance of text bias
Standardized tests
4 potential sources
1. Cultural background
2. Background knw
3. Cognitive Characteristics
4. Native lang/ ethnicity/age and gender
VI. Criterion-Related Validity
Cross-test comparability
Equivalence with different versions of the same test
Comparability with external standards