PRIME Process for Publishers

The purpose of WIDA PRIME is to offer tools to assist publishers and educators in determining a degree of alignment between a given set of instructional materials and the WIDA English Language Development Standards Framework, 2020 Edition (Henceforth referred to as the Framework) based on the PRIME rubric’s criteria. PRIME stands for Protocol for Review of Instructional Materials with the English Language Development Standards Framework, 2020 Edition (PRIME 2020). All publishers and educators are encouraged to use the PRIME 2020 Rubric and Portfolio Workbook to inform local instructional and curricular decisions and build knowledge related to the Framework. When a publisher would like to request the WIDA PRIME 2020 Seal for a set of instructional materials, they must submit those instructional materials and a portfolio of evidence for evaluation by WIDA-trained reviewers.

The process for submitting materials, the scoring criteria, and requirements for gaining the PRIME 2020 Seal have changed: 

• Publishers now compile and submit an evidence-based portfolio to demonstrate alignment between a given set of instructional materials and the Framework according to the PRIME rubric (more info on the submission and review process below).
• WIDA-trained reviewers use updated criteria to score each of the four components of the Framework (ELD Standards Statements, Key Language Uses, Language Expectations, and Proficiency Level Descriptors) according to a 4-point scale.
• The instructional materials submitted by a publisher for review will earn the WIDA PRIME 2020 Seal if they score at least a 3 (present) or higher for alignment to three components of the Framework (e.g., ELD Standards Statements, Key Language Uses, Language Expectations) and if the lowest criterion score is at least 2 (present but insufficient) for alignment to one Framework component (e.g., Proficiency Level Descriptors).

If publishers score a 2 (present but insufficient) in two or more Framework components (e.g., Language Expectations and Proficiency Level Descriptors), they are not yet eligible to earn the Seal. See page 10 in the Workbook for more information.

 In the WIDA PRIME 2020 protocol, alignment refers to the degree to which instructional materials are in agreement and serve in conjunction with the Framework to support multilingual learners in developing language in the context of grade-level disciplinary learning (outlined on page 9 of the Rubric and Portfolio Workbook).

It is expected that users first develop a thorough understanding of the Framework and curriculum design before using WIDA PRIME 2020. WIDA offers several ways to support learning about the Framework, including through the WIDA ELD Standards page and a suite of professional learning offerings.

Publishers may submit the following instructional materials for external review of alignment by a WIDA-trained team of reviewers in an application for a WIDA PRIME 2020 Seal (found on page 7 in the Rubric and Portfolio Workbook):

• Materials for one full year’s course of study in the core academic disciplines (language arts, mathematics, science, and social studies or interdisciplinary materials) that are designed to align with the Framework.
• Materials for one full year’s course of study dedicated to ELD instruction that clearly and concretely connect to grade-level academic content standards.
• Supplemental materials for either of the above submissions can be submitted as a directly connected extension of the yearlong set of materials for either content or ELD. Supplemental materials for that year’s course of study do not need a separate request form.
• A second year-long set of materials for an adjacent grade within a grade cluster may be submitted through the Request for Extended Review within a Grade-Level Cluster Google Form, but only if clearly and concretely connected to grade-level core instructional materials. (See Appendix A of the Workbook on page 50).
  • For the purpose of identifying acceptable adjacent grade submissions, WIDA PRIME 2020 defines the Grade Clusters as the following. For example, these clusters mean that there is no adjacent grade for materials related to Kindergarten or Grade 1.
    
    
    • Kindergarten
    • Grade 1
    • Grades 2-3
    • Grades 4-5
    • Grades 6-8
    • Grades 9-12

For example, these clusters mean that there is no adjacent grade for materials related to Kindergarten or Grade 1.

The submission process is meant to facilitate a publisher’s thoughtful reflection on the alignment of their instructional materials with the WIDA English Language Development Standards Framework. By identifying the specific ways in which instructional materials fulfill various rubric criteria, the aim is that publishers will see areas of strength as well as opportunities to strengthen alignment in future editions of the instructional materials.

The WIDA ELD Standards Framework, 2020 Edition represents an evolution of WIDA's Language Proficiency Standards. As such, the process for PRIME has also evolved. It now considers the multifaceted approach to language development which involves a different and more comprehensive process for determining alignment to the 2020 Edition. 

Price for a new WIDA PRIME 2020 alignment review and seal: $20,000.00

Here is the price per grade cluster breakdown for any given program:
Kindergarten = $20,000
Grade 1 = $20,000
Grades 2-3 = $30,000
Grades 4-5 = $30,000
Grades 6-8 (assuming you start with 7th grade) = $40,000
Grades 9-12 (assuming you start with 10th grade, 9th and 11th are adjacent and 12th is new) = $60,000
Full program K-12 = $200,000
Full program K-5 = $100,000

WIDA and WCEPS have priced the product to cover the cost of delivery to publishers. 

The WIDA PRIME 2020 review and Seal offers publishers the opportunity to:

• Prompt productive conversations about how instructional materials are serving multilingual learners 
• Guide self-reflection, self-analysis, self-assessment, and self-determination of a degree of alignment between a given set of instructional materials and the Framework via the criteria specified in the PRIME rubric 
• Collect evidence and information about instructional materials for potential improvements and revisions to strengthen alignment with the Framework 
• Support communication with stakeholders (e.g., parents, program directors, school boards, teachers, program reviewers, adoption committees) about instructional materials under consideration for adoption

Yes, but any independent analyses that are shared/posted must be accompanied with the following statement:

“The WIDA PRIME 2020 Portfolio Workbook is available at no cost to publishers and local users to encourage informal independent analysis of curricular materials for alignment with the WIDA ELD Standards Framework, 2020 Edition. These independent analyses are not certified by WIDA and therefore do not qualify these instructional materials for a WIDA PRIME 2020 Seal.”

There is no guarantee that a portfolio will earn the Seal – Seals will be awarded according to the WIDA PRIME reviewers’ evidence-based determination of a degree of alignment to the Framework.

The WIDA PRIME 2020 Seal indicates that WIDA-trained reviewers believe the publisher has provided sufficient evidence to determine a degree of alignment between a given set of instructional materials and the Framework, based on the PRIME rubric. Publisher’s materials that earn the Seal may be posted, along with final reports, on the WIDA PRIME Instructional Materials Published Reviews page, which then serves as one data point to inform district and school choices in materials adoption. 

Only WIDA-trained reviewers are sanctioned to participate in the formal review process that decides whether materials may earn the WIDA PRIME 2020 Seal of alignment. Review teams are comprised of at least two reviewers, with a third reviewer in case of a scoring disagreement. The PRIME Coordinator will serve as the publisher’s primary contact and will consult the team’s lead reviewer when necessary.

The WIDA ELD Standards Framework, 2020 edition represents an evolution of WIDA's Language Proficiency Standards. As such, the process for alignment to the Framework has also evolved. It now considers the multifaceted approach to language proficiency which involves a different and more comprehensive process for conducting correlations. Therefore, all instructional materials will have to go through the review process outlined in the Rubric and Portfolio Workbook to be considered for the WIDA PRIME 2020 Seal.

Although 41 U.S. states and territories currently use the Framework, published instructional materials are used across all states. Therefore, the design of PRIME intentionally does not require reference to the WIDA Framework in the instructional materials submitted for review. 

To illustrate, the PRIME rubric points to types of evidence showing that materials explicitly address the WIDA Key Language Uses (KLUs) or prominent genres of schooling, the WIDA Language Expectations or content-driven language goals. Either way, a thorough understanding of the components of the Framework is a prerequisite for applying the rubric criteria to the materials to determine alignment. If a publisher team member is examining the materials for alignment to the Key Language Uses, they would not be able to determine alignment without understanding what the Key Language Uses are within the context of content learning. The Workbook provides prompts and questions related to each component of the Framework but responding to those prompts/questions without knowledge of the Framework would be insufficient for completing the Workbook. 

Although not required, direct mention of the Framework does make it easier to understand how materials are aligned, and in terms of equity and accountability, some kind of ELD standards – WIDA or otherwise – ought to be mentioned with parity for multilingual learners.

Yes, all reviews include a debrief session. The lead reviewer will schedule a meeting with the publisher and send a draft report ahead of the meeting. During the meeting, findings will be discussed, and next steps will be determined. The publisher can decide to:

1. Accept the findings of the report as is.
2. Revise materials based upon findings and resubmit for a follow-up review (e.g., the publisher overlooked a couple of instances of better evidence of alignment). If the publisher selects this option, the report will be held until the revisions have been made. Beyond some instances where the publisher may have overlooked instances of better evidence in currently existing materials, when the publisher needs to revise materials (not just their selection of evidence), that will trigger a new application review process (including additional fees).

Publishers should follow these steps to prepare their reflections from the WIDA PRIME Rubric and Portfolio Workbook for review (described on page 13 of the Workbook):

1. Use the website form to receive an email with instructions and the Rubric and Portfolio Workbook download.
2. Review the Rubric and Portfolio Workbook to understand what is required for submission (e.g., eligible materials, submissions process) and how that information will be organized and scored (e.g., portfolio format). Become familiar with the PRIME rubric’s criteria, indicators, and key questions, and the review and scoring process. All these factors will help your submissions team determine the degree to which you believe instructional materials are aligned to the Framework.
3. If your instructional materials are eligible for a review and you are ready to move forward with the alignment, use the “Contact Us” form on the PRIME website before you begin, to set up a call for planning your submission and scheduling a review. It is imperative that we match your timeline with reviewer availability.
4. Create a submissions team: assemble a small, qualified team with a depth of expertise in curriculum design and the Framework. (A small team is preferable to a single person.)
5. Begin self-analysis of alignment: use the Rubric and Portfolio Workbook to guide self-reflection and self-analysis. Take notes and begin planning the pieces of evidence you might collect to make evidence-based claims of alignment between instructional materials and the Framework.
6. Develop a portfolio workbook as outlined in the Rubric and Portfolio Workbook.
7. Complete the portfolio checklist: use the checklist found in the Rubric and Portfolio Workbook to determine if your portfolio is ready for submission.
8. Submit your materials and self-reflections via the Submission Forms shared by the PRIME Coordinator.
9. Complete the contract and approve the quote for the PRIME Review and Seal to allow the PRIME team to begin their review.

Once a publisher’s portfolio has been accepted for review, a WIDA-trained team of reviewers begins analysis. Review teams are comprised of at least two reviewers, with a third reviewer in case of scoring disagreement. The review process is described below in six steps (and found on page 44 of the WIDA PRIME Rubric and Portfolio Workbook):

1. Eligibility and completeness of materials: Reviewers check that materials are eligible, and that all required components are present prior to beginning review. If any pieces of the portfolio are missing, reviewers will send it back to the primary contact. The lead reviewer specifies which piece(s) of the submission are missing and provides the primary contact with a date by which to submit the missing pieces. Upon this initial look at the portfolio, the team also determines the length of the review window.
2. Selection of unit sampling: The review team selects a common set of units and/or sections of instructional materials to analyze. For example, the team may determine that all members will review the common set of units, such as units 2 and 6, for example. Together with the portfolio, these common units of analysis serve as the basis for the team’s collaborative report. Additionally, each reviewer will select at least one additional unit and/or section of the materials to review as a dipstick. These individual "dipstick units" are used to support the team’s collaborative analysis as needed.
3. Individual reviewer analysis and rating: Reviewers first work individually on all aspects of the publisher’s self-assessment portfolio: descriptions, narratives, alignment claims, justifications, and supporting evidence. Moreover, reviewers substantiate and triangulate evidence in the materials, examining both the review team’s common units or sections (e.g., 2 and 6) and the individually assigned "dipstick units" or sections (e.g., units 4, 10, and supplement C). Reviewers rate the evidence submitted for each criterion using the 4-point scoring system.
4. Team triangulation and determination of final score and eligibility for the WIDA PRIME 2020 Seal: Reviewers reconvene as a team to share findings. Reviewers then triangulate data and calibrate responses to reach consensus about final ratings and the degree of alignment they find.
5. Debrief session: The PRIME Coordinator will schedule a meeting with you and the lead reviewer. A draft report will be sent ahead of the meeting. At the meeting, the lead reviewer and your team discuss findings and determine next steps. You can decide to: 
   1. Accept the findings of the report. 
   2. Revise the portfolio based upon draft findings and re-submit for a one-time follow-up review (e.g., the publisher overlooked a couple of instances of better evidence of alignment). If you select this option, the report will be held until the revisions have been made. The lead reviewer will determine the length of the allowable re-submission window. When revisions to the materials are needed to provide the necessary evidence (instead of selecting better examples), then a new application review process would be required (including additional fees). 
6. Final report available for publication: The lead reviewer shares the final report with the publisher. If the materials are approved for a PRIME 2020 Seal, the PRIME team will post the final report on the widaprime.org website and send Seal files and guidelines to the main contact(s). The publisher will then be invoiced for the work upon completion of the PRIME 2020 review.