guide to the proficiency level specifications - Lake County Schools

Florida Hard-to-Measure Content Area: World Languages Test Item Specifications Proficiency Levels: Novice-Low/Novice-Mid to Advanced-Low [pic] Florida Department of Education
www.fldoe.org
Contents Introduction 5
Origin and Purpose of the Specifications 5
Table 1: Proficere Leadership and Specifications Developers 6 Scope of This Document 8 Overall Considerations 8 Item Contexts (Scenarios) 9 Construct Assessed: Language Proficiency 9 2011 Florida World Languages Next Generation Sunshine State Standards 10 Three Modes of Communication Described by ACTFL 10 Proficiency Levels 13
Benchmark Levels Described by CASLS 14
Table 2: Correspondences to Year of Study 15 Key Committee Decisions 16
Proficiency Levels 16
American Sign Language (ASL) 16
Criteria for World Languages Item Bank and Test Platform (WL IBTP) Items
18
Item Style and Format 18 Criteria for Selecting Stimulus Materials 18 Item Response Types 19
Table 3: Item Response Types 19 Texts 19
Table 4: Types of Texts 20 Media 20 Sources 20 Characteristics of Texts and Media 21 Content (See Appendix B) 21 Modifications 21 Texts and Media Features 22 Diversity 22 Reading Level 22
Degree of Challenge of WL IBTP Items 23
Item Difficulty 23 Cognitive Complexity 23 Categories of Complexity 24
Table 5: Cognitive Complexity Levels 25
Table 6: WL IBTP Percentaage of Points by Cognitive Complexity Level 26
Guidelines for Item Writers 27
Format 27 Sources 27 Correct Response 27 Item Difficulty 27 Cognitive Complexity 27 Electronic Submission of Items 27
Guide to the Proficiency Level Specifications 28
Benchmark Classification Scheme 28 Definitions of Benchmark Specifications 28 Recommended Depth of Knowledge (DOK) Level 29 Text/Stimulus Attributes 29 Distractor Attributes 29 Item Types 29 Sample Items 29
Review Procedures for World Languages IBTP 30
Review of Texts and Media 30 Review for Potential Bias and Sensitivity Issues 30 Review of Assessment Items 30 Review for Universal Design 30
Test Item Specifications for WL-NGSSS Benchmarks 32
Standard 1: Interpretive Listening 33 Standard 2: Interpretive Reading 84 Standard 3: Interpersonal Communication 110 Standard 4: Presentational Speaking 215 Standard 5: Presentational Writing 257 Standard 6: Culture 316 Standard 7: Connections 342 Standard 8: Comparisons 358 Standard 9: Communities 385
Appendix A: Glossary of Terms 400 Appendix B: Internationally Accepted Topics Lists for Proficiency Levels
404 Appendix C: Guide to the Proficiency Level Descriptors 405 Appendix D: Checklist for Reviewing Test Items 406
Introduction ORIGIN AND PURPOSE OF THE SPECIFICATIONS
FROM THE AMERICAN RECOVERY AND REINVESTMENT ACT OF 2009, A RACE TO THE TOP
FUND WAS PROVIDED FOR 11 STATES AND THE DISTRICT OF COLUMBIA FOR THE
PURPOSE OF ENGAGING IN RACE TO THE TOP REFORM GRANTS THAT ARE GROUNDED IN
COMPREHENSIVE REFORM. IN SEPTEMBER 2010, THE U.S. DEPARTMENT OF EDUCATION
ANNOUNCED THAT FLORIDA WAS A WINNER OF THE FEDERAL RACE TO THE TOP PHASE 2
COMPETITION. AN IMPORTANT COMPONENT OF FLORIDA'S WINNING APPLICATION
FOCUSED ON THE STANDARDS AND ASSESSMENTS ASSURANCE AREA THAT "[ADOPTS]
RIGOROUS STANDARDS AND ASSESSMENTS THAT PREPARE STUDENTS FOR SUCCESS IN
COLLEGE AND THE WORKPLACE" AS WELL AS "[BUILDS] DATA SYSTEMS THAT MEASURE
STUDENT SUCCESS AND INFORM TEACHERS AND PRINCIPALS HOW THEY CAN IMPROVE
THEIR PRACTICES" (RACE TO THE TOP, FLORIDA REPORT, YEAR 1: SCHOOL YEAR 2010-
2011, P.2). As an awardee of the Race to the Top reform grants, Florida, through its
Office of Race to the Top Assessments, began to manage seven projects
awarded to Local Education Agencies for the purpose of developing high-
quality assessments for hard-to-measure content areas. The Florida
Department of Education (FDOE) sought and included recommendations from
educators across the state, including the Florida Organization of
Instructional Leaders (FOIL) and a State Advisory Committee on District-
Developed Student Assessments for Instructional Effectiveness (DDSAIE),
comprised of parents, teachers, and district-level administrators, to
determine the appropriate scope of work for assessment development for the
hard-to-measure content areas. Provided these recommendations, teacher
assignment and student enrollment data were analyzed to determine the
courses that reach the greatest number of students and teachers in the hard-
to-measure content areas. As a result of the analysis, World Languages was
identified as a content area with courses having the greatest need for
assessment development. To develop high-quality assessments for the hard-to-measure content area of
World Languages (WL), Duval County Public Schools was awarded the WL Race
to the Top grant identified henceforth as Project Proficere. Proficere
engaged Florida teachers to develop high-quality assessments for the
following five language areas: American Sign Language, French, German,
Italian, and Latin. As such, the assessment items will become accessible to
Florida public school districts and teachers through the State's electronic
Interim Assessment Item Bank and Test Platform (IBTP). When the IBTP is
fully operational, educators will have the ability to search the extensive
bank of test items that have been vetted by Florida teachers, to export
items, and to generate customized assessments to meet district and
classroom needs. Additionally, a pool of practice items will be made
available to students and parents, which independent schools may access as
well. Via the Project Proficere website
(http://www.duvalschools.org/reseval/proficere/), the Project invited
teachers from Florida's 67 districts to submit applications for
participation. Furthermore, Proficere developed partnerships with Lake,
Polk, and Seminole Counties as well as the following universities and
associations: the University of South Florida (College of Education, David
C. Anchin Center), the University of North Florida (World Languages,
College of Arts and Sciences), the Florida Foreign Languages Association
(FFLA), and the Foreign Languages Association for Managers in Education
(FLAME). These partners help provide expertise throughout the process of
developing the Specifications. In October of 2011 Proficere was launched at conferences for FFLA and
FLAME. Through Proficere's website, the Project recruited a geographically
diverse group of teachers from across the State to work alongside Duval
teachers who applied for participation as Specifications developers.
Subsequent to the application process, 25 teachers were selected to
participate in Phase One of the Project. These Original 25 participants
comprised five distinct linguistic groups, with each group consisting of
five teachers who possessed distinguished WL expertise and/or participated
actively in their WL associations. Phase One leadership and teacher-
participants are listed in the table below. Table 1
Proficere Leadership and Specifications Developers |Phase One | |
|Primary Leadership | |
|Director of Project Proficere: |Duval County |
|Melanie Bolt, Ph. D. | |
|Executive Director of Instructional|Duval County |
|Research | |
|and Accountability: |Duval County |
|Timothy Ballentine | |
|Supervisor of World Languages: |Duval County |
|Joanne Davis, Ph. D. | |
|Specialist for the Deaf/HH Program |Duval County |
|Brenda N. Dale, M.Ed. | |
|Supervisor of Test Development: |Leon County |
|Gwyn Seltzer | |
|FDOE Proficere Liaison: | |
|Annamarie Cairo-Tijerino, M.Ed. | |
|Teachers of American Sign Language | |
|Jennifer Allinson |Duval County |
|*Craig Leavitt |Duval County |
|FFLA 2011 Associated Leagues |Seminole County |
|Teacher of the Year, Florida | |
|American Sign Language Teacher | |
|Association (FASLTA) | |
|Shawn Olmstead | |
|*Lesley Silvestris |Duval County |
|Dimarly Suarez |Clay County |
|Teachers of French | |
|President of the American |Duval County |
|Association of Teachers of French | |
|(AATF) | |
|*Lauren Doyle-McCombs, Ph. D. | |
|*Louise Hunley, Ph. D. |Duval County |
|*Grace Kellermeier, Ed. D.