HLM Analysis and Modeling of Large-scale assessment data for science and mathematics literacy - continuation

The logic to this interest lies in what may be termed the path of policy influence (Kennedy, 1999) – those elements of public schooling that are accessible to policy makers (such as funding, certification qualifications of professional staff, curriculum, the nature and extent of instructional support and supervision, the provision of opportunities for professional development, and school organizational structure) should show some influence over the key consequences of schooling (such as student learning, graduation rates and ease of entry into the labour market). The empirical investigations associated with this field often take the form of studies of the correlation between educational indicators such as expenditures (school resource inputs) and test scores (student learning achievement outcomes). The field has a history of equivocal findings, and even the interpretation of this history is not without controversy with some claiming there are clear patterns of association among various educational indicators whereas others claim the opposite (Greenwald, Hedges, & Laine1996a&b; Hanushek, 1996). Even from the analytic perspective there is equivocation. For example, there is evidence that the level at which the system traits are aggregated influence the predictivity of results. Yair (1997) pointed out that within-school variation in student achievement is generally greater than between-school variation – the consequence being that system traits should be aggregated at the classroom level rather than the school or district level in studies of system input/outcome relationships. Work of this nature can fall in the domain of educational indicators systems. Educational indicator systems have several potential uses that range from a simple description of the educational system to the development of cause-effect models to inform policy decisions (Camilli & Firestone, 1999).

Empirical research of these the complex and varied datasets of student assessment information can inform policy issues of language, mathematics and science education, and lead to better understandings of the performance and quality of schools in relation to the literacies of language, mathematics and science. The future research agenda of the project involves exploring the theoretical associations of expository language arts, mathematics operations and science understanding at the selected item level to better define the potential roles language and mathematics play in science and technology literacy. Given the nature of extant datasets most HLM projects are forced to use data aggregated at the sub-test level that consolidate language, mathematics and science performances over a broad-ranges of cognitive and conceptual levels. It is believed that some of performances in narrative and lower-level cognitive tasks may be masking the actual relationships amongst language arts, mathematics and science literacies. These relationships may be better explored through item-level data analysis – which we will work on in this project.

References

• Camilli, G. & Firestone, W.A. (1999). Values and state ratings: An examination of the state-by-state education indicators in Quality Counts. Education Measurement: Issues and Policies, 18(4), 17-25.
• Greenwald, R., Hedges, L.V. & Laine, R.D. (1996a). The effect of school resources on student achievement. Review of Educational Research. 66(3), 361-396.
• Greenwald, R., Hedges, L.V. & Laine, R.D. (1996b). Interpreting research on school resources on student achievement: A rejoinder to Hanushek. Review of Educational Research. 66(3), 411-415.
• Hanushek, E.A. (1996). A more complete picture of school resource policies. Review of Educational Research, 66(3), 397-410.
• Kennedy, M. M. (1999). Approximations to indicators of student outcomes. Educational Evaluation and Policy Analysis, 21(4), 345-363.
• Yair, G. (1997). When classrooms matter: Implications of between classroom variability for educational policy in Israel. Assessment in Education, 4(2), 225-249.