Since most ELA high school classes use a dynamic set of units rather than textbooks with a fixed sequence of lessons, this review focused on unit level resources. Reviewers evaluated twenty English Language Arts (ELA) units for the 9th and/or 10th grades.
* units pre dating the Common Core State Standards
These OER were reviewed with the specific goal of looking at how well they address CCSS shifts, not evaluating their quality against existing Washington state grade level expectations. The CCSS in ELA are very different from previous K–12 state learning standards. In particular, there are several key shifts in instruction:
- Content knowledge built through content-rich nonfiction
- Reading, writing, and speaking grounded in evidence from text, both literary and informational
- Regular practice with complex text and its academic language
Though many of the units reviewed were crafted to specifically address the new standards, six of the units pre-date the CCSS. Thus, the review process compared these materials against target standards that developers were not originally aiming for at material creation. In those instances, we noticed much higher variation in reviewer scores. Though still within acceptable ranges of inter-rater reliability (see Data Analysis), interpretation of how well the legacy resources aligned with the new standards was a bit more challenging and open to user interpretation of the resource intent.
Overall, the findings indicated many strong choices from among the available OER materials for educators seeking ELA units with alignment to the CCSS. Twelve units received an overall average score of 2 or higher (on a 0–3 point scale) across all rubrics. They were:
- Literary Technique (Odell Education)
- Speeches (New York City Department of Education/University of Pittsburgh)
- Work of Art (Engage NY/Public Consulting Group)
- Wisely and Slow (Engage NY/Public Consulting Group)
- EB Arguments (Odell Education)
- EBC Speeches (Odell Education)
- Romeo and Juliet (New York City Department of Education)
- Gettysburg (Achieve the Core)
- Things Fall Apart (EDSITEment!)
- Metacognition (CC Better Lesson)
- Enlightened Thinking (CC Better Lesson)
- Edgar Allan Poe (The Big Read)
An additional six units had average total scores at or above the midpoint of the scale.
As with the mathematics review, this review process was not intended to rank or endorse the materials. As such, there are few comparative graphs in this report. It is also important to note that the materials reviewed are not the only ELA OER resources available – many others exist and new resources emerge regularly. We were limited in scope and solely examined ELA thematic units that extended instruction over multiple weeks and met the criteria outlined in the Selection Criteria.
This review should be viewed as an opportunity to provide input on the changes necessary to bring the OER resource into closer alignment with the CCSS. The reviews represent a point in time. More so than print materials, digital resources with an open license can be freely modified, so all the products that were reviewed can be and are frequently updated.
The EQuIP rubric is designed to be used at the unit, rather than full-course, level to get a more detailed picture of the quality of alignment to the CCSS for a resource. Reviewers considered four areas described below:
- Alignment to the Rigors of the CCSS: the unit targets a set of grade CCSS ELA/Literacy standards; includes a clear and explicit purpose for instruction; selects texts of sufficient quality and scope that measure within the grade-level text complexity band; integrates reading, writing, speaking and listening
- Key Shifts in the CCSS: the unit addresses reading text closely; capturing text-based evidence; writing from sources; using academic vocabulary; increasing text complexity; building disciplinary knowledge; providing a balance of texts and writing
- Instructional Supports: the unit is responsive to varied student learning needs
- Assessment: the unit regularly assesses whether students are mastering standards-based content and skills through direct, observable evidence, via accessible and unbiased method using varied modes of assessment
Figure 17. Average EQuIP ratings for all 20 resources – 79 total reviews. Overall Rating 8.1 (Exemplar if Improved)
Each dimension had a number of criteria that were considered. The number of criteria for each dimension that were met was rated on a scale from 0–3 (None – 0, Few – 1, Many – 2, All – 3). The rubric also provides an Overall rating for the resource based upon the sum of each of four dimensions. Scores from 11–12 are considered Exemplar, 8–10 are Exemplar if Improved, 3–7 are in the Revision Needed category, and scores 2 and below are Not Ready to Review.
- Average Overall rating for the 20 units was Exemplar if Improved. Individual reviewers gave the following evaluations
Exemplar 19 reviews Exemplar if Improved 26 reviews Needs Revision 34 reviews Not Recommended 0 reviews
- Average Alignment for all resources combined was 2.5, indicating that Many to All of the CCSS criteria were met. Nineteen out of the 20 resources fell into this category.
- On average, the reviewed resources met Many to All of the Key Shifts:
All 32 reviews Many 27 reviews Some 20 reviews
- The Assessment scale showed a lower average score than others. Reviewer comments indicated that some of the products under review had few assessment components.
Achieve OER Rubrics
The Achieve OER rubrics are specifically designed to be used with digital resources, as opposed to print media. They also examine other aspects of OER quality, may be used with any standards, and are designed to evaluate resources that may be smaller in grain size than units or lessons.
The Achieve instrument has eight different smaller rubrics, several of which significantly overlap the EQuIP instrument. Since the EQuIP instrument was developed specifically to consider alignment to the CCSS, it was used in this review in lieu of the overlapping Achieve OER rubrics in order to minimize duplicative measurement scales. The three Achieve rubrics used for the ELA review process are:
- Rubric II. Quality of Explanation of the Subject Matter
- Rubric VI. Quality of Instructional Tasks and Practice Exercises
- Rubric VII. Opportunities for Deeper Learning
Figure 18. Average Achieve OER ratings for all resources.
The Quality of Interactivity (Rubric V) used in the math review was not used in the ELA review. The intent of the rubric was to measure interactive modules, like assessments, that provide live feedback or widgets that could be manipulated to view variable outcomes. These types of objects were not present in the ELA resources we examined.
Resources scored well in these categories, with overall averages tending to fall in the Strong or Superior category.
Reviewers were asked to write a short narrative providing an evaluation of each of the resources the reviewed. They were instructed to cite evidence from the resource that supported their comments about are as needing adaptation. Additionally, they provided suggestions for changes that would help improve alignment.
Figure 19. Number of times out of 79 reviews that each potential use was cited.
As part of their professional assessments, reviewers clarified the ideal use scenario for each reviewed resource and estimated the amount of work that would be required for a small group to make adaptations to bring the resource into CCSS alignment. Finally, reviewers selected all the ways they would use the resource in both its current and adapted form. Below are some of the highlights, but for an in-depth look at comments for each resource, please visit the OER Project reviewed materials library.
- Out of 79 reviews, 24 stated they would use a resource as a unit replacement in its current state. That number jumped to 35 if suggested adaptations were made.
Current unit replacement (number of reviewers) Better Lesson Metacognition 2 Big Read Edgar Allan Poe 1 Engage NY Work of Art 3 Engage NY Wisely and Slow 4 NYC Dept. of Ed. Romeo and Juliet 2 NYC Dept. of Ed Speeches 3 Odell EB Arguments 3 Odell EBC Speeches 1 Odell Literary Techniques 4 Saylor Unit 5 Slavery 1
- Only 7 reviews out of 79 stated that they would not use a resource in some capacity in its current state.
The overall results shown in Figure 19 indicate the overall strength of OER ELA material currently available.
While the intent of this report is not to rank the products based upon their overall average scores, comparing the performance of the resources on certain scales or items provides meaningful information. The charts below show how the resources compared with each other based upon selected scales or items.
Alignment to the Depth of the CCSS
Figure 20. EQuIP. This scale looks at the overall alignment of the resource to the CCSS.
Key Shifts in the CCSS
Figure 21. This scale measures how the unit addresses key shifts in the CCSS.
Figure 22. EQuIP. Examines whether a unit is responsive to varied student learning needs.
Figure 23. EQuIP. Unit regularly assesses whether students are mastering standards-based content and skills.
Figure 24. Achieve OER. Measures the unit’s ability to engage learners in one or deeper learning skills, including think critically and solve complex problems, reason abstractly, construct viable arguments and apply discrete knowledge and skills to real-world situations.
Quality of Explanation of Subject Matter
Figure 25. Achieve OER. Rates how thoroughly the subject matter is explained or otherwise revealed in the object.
For detailed information on each reviewed ELA resource, including scores on all rubrics, extensive reviewer comments, and supplemental metadata, visit the OSPI OER Project Materials Review website.
This report is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License.