ELA Findings

General Observations

Since most ELA high school classes use a dynamic set of units rather than textbooks with a fixed sequence of lessons, this review focused on unit level resources. Reviewers evaluated twenty English Language Arts (ELA) units for the 9th and/or 10th grades.

DeveloperFull TitleShort Title
BetterLesson/Christopher ArnettEnlightened Thinking: The Argument and the Research PaperEnlightened Thinking
BetterLesson/Tim PappageorgeMetacognition and The Absolutely-True Diary of a Part-Time IndianMetacognition
EDSITEMENT! National Endowment for the HumanitiesFrom Courage to Freedom: Frederick Douglass's 1845 Autobiography *Courage to Freedom
EDSITEMENT! National Endowment for the HumanitiesA "New English" in Chinua Achebe's "Things Fall Apart": A Common Core ExemplarThings Fall Apart
EngageNY/Public Consulting GroupNew York State Common Core ELA & Literacy Curriculum 9.1.3:“Wisely and slow; they stumble that run fast.”Wisely and Slow
EngageNY/Public Consulting GroupNew York State Common Core ELA & Literacy Curriculum 9.1.2: “A work of art is good if it has arisen out of necessity.”Work of Art
Georgia Virtual LearningNinth Literature and Composition Unit 4: Short Stories *Short Stories
Library of Congress/Kathleen Prody & Nicolet WheartyTo Kill a Mockingbird: A Historical Perspective *Mockingbird Historical
Library of Congress/Linda and David LackeyThe Grapes of Wrath: Scrapbooks and Artifacts *Grapes of Wrath
NYC Dept. of EducationGrade 9 Literacy in English Language Arts: Who Is to Blame for Romeo and Juliet's Death?Romeo and Juliet
NYC Dept. of Education/University of PittsburghGrade 9–10 English Language Arts: Speeches—Argument and MethodsSpeeches
Odell EducationBuilding Evidence-Based Arguments—Grade 9: “What is the virtue of a proportional response?”EB Arguments
Odell EducationMaking Evidence-Based Claims Unit Grade 10: MLK, Barak ObamaEBC Speeches
Odell EducationMaking EBC About Literary Technique Grade 10: Emily Dickinson, Robert FrostLiterary Technique
Achieve the Core – Student Achievement PartnersThe Gettysburg Address by Abraham LincolnGettysburg
Achieve the Core – Student Achievement Partners/LASDWeapons of the SpiritWeapons of Spirit
The Big Read: National Endowment for the ArtsThe Stories and Poems of Edgar Allan Poe *Edgar Allan Poe
The Big Read: National Endowment for the ArtsTo Kill a Mockingbird *Mockingbird
The Saylor Foundation/Karen BreazealeSaylor ELA 10 Unit 3: AnthemUnit 3 Anthem
The Saylor Foundation/Karen BreazealeSaylor ELA 10 Unit 5: Up from SlaveryUnit 5 Slavery

* units pre dating the Common Core State Standards

These OER were reviewed with the specific goal of looking at how well they address CCSS shifts, not evaluating their quality against existing Washington state grade level expectations. The CCSS in ELA are very different from previous K–12 state learning standards. In particular, there are several key shifts in instruction:

  1. Content knowledge built through content-rich nonfiction
  2. Reading, writing, and speaking grounded in evidence from text, both literary and informational
  3. Regular practice with complex text and its academic language

Though many of the units reviewed were crafted to specifically address the new standards, six of the units pre-date the CCSS. Thus, the review process compared these materials against target standards that developers were not originally aiming for at material creation. In those instances, we noticed much higher variation in reviewer scores. Though still within acceptable ranges of inter-rater reliability (see Data Analysis), interpretation of how well the legacy resources aligned with the new standards was a bit more challenging and open to user interpretation of the resource intent.

Overall, the findings indicated many strong choices from among the available OER materials for educators seeking ELA units with alignment to the CCSS. Twelve units received an overall average score of 2 or higher (on a 0–3 point scale) across all rubrics. They were:

  • Literary Technique (Odell Education)
  • Speeches (New York City Department of Education/University of Pittsburgh)
  • Work of Art (Engage NY/Public Consulting Group)
  • Wisely and Slow (Engage NY/Public Consulting Group)
  • EB Arguments (Odell Education)
  • EBC Speeches (Odell Education)
  • Romeo and Juliet (New York City Department of Education)
  • Gettysburg (Achieve the Core)
  • Things Fall Apart (EDSITEment!)
  • Metacognition (CC Better Lesson)
  • Enlightened Thinking (CC Better Lesson)
  • Edgar Allan Poe (The Big Read)

An additional six units had average total scores at or above the midpoint of the scale.

As with the mathematics review, this review process was not intended to rank or endorse the materials. As such, there are few comparative graphs in this report. It is also important to note that the materials reviewed are not the only ELA OER resources available – many others exist and new resources emerge regularly. We were limited in scope and solely examined ELA thematic units that extended instruction over multiple weeks and met the criteria outlined in the Selection Criteria.

This review should be viewed as an opportunity to provide input on the changes necessary to bring the OER resource into closer alignment with the CCSS. The reviews represent a point in time. More so than print materials, digital resources with an open license can be freely modified, so all the products that were reviewed can be and are frequently updated.

EQuIP Rubric

The EQuIP rubric is designed to be used at the unit, rather than full-course, level to get a more detailed picture of the quality of alignment to the CCSS for a resource. Reviewers considered four areas described below:

  • Alignment to the Rigors of the CCSS: the unit targets a set of grade CCSS ELA/Literacy standards; includes a clear and explicit purpose for instruction; selects texts of sufficient quality and scope that measure within the grade-level text complexity band; integrates reading, writing, speaking and listening
  • Key Shifts in the CCSS: the unit addresses reading text closely; capturing text-based evidence; writing from sources; using academic vocabulary; increasing text complexity; building disciplinary knowledge; providing a balance of texts and writing
  • Instructional Supports: the unit is responsive to varied student learning needs
  • Assessment: the unit regularly assesses whether students are mastering standards-based content and skills through direct, observable evidence, via accessible and unbiased method using varied modes of assessment

Figure 17. Average EQuIP ratings for all 20 resources – 79 total reviews. Overall Rating 8.1 (Exemplar if Improved)

Each dimension had a number of criteria that were considered. The number of criteria for each dimension that were met was rated on a scale from 0–3 (None – 0, Few – 1, Many – 2, All – 3). The rubric also provides an Overall rating for the resource based upon the sum of each of four dimensions. Scores from 11–12 are considered Exemplar, 8–10 are Exemplar if Improved, 3–7 are in the Revision Needed category, and scores 2 and below are Not Ready to Review.

  • Average Overall rating for the 20 units was Exemplar if Improved. Individual reviewers gave the following evaluations
    Exemplar19 reviews
    Exemplar if Improved26 reviews
    Needs Revision34 reviews
    Not Recommended0 reviews
  • Average Alignment for all resources combined was 2.5, indicating that Many to All of the CCSS criteria were met. Nineteen out of the 20 resources fell into this category.
  • On average, the reviewed resources met Many to All of the Key Shifts:
    All32 reviews
    Many27 reviews
    Some20 reviews
  • The Assessment scale showed a lower average score than others. Reviewer comments indicated that some of the products under review had few assessment components.

Achieve OER Rubrics

The Achieve OER rubrics are specifically designed to be used with digital resources, as opposed to print media. They also examine other aspects of OER quality, may be used with any standards, and are designed to evaluate resources that may be smaller in grain size than units or lessons.

The Achieve instrument has eight different smaller rubrics, several of which significantly overlap the EQuIP instrument. Since the EQuIP instrument was developed specifically to consider alignment to the CCSS, it was used in this review in lieu of the overlapping Achieve OER rubrics in order to minimize duplicative measurement scales. The three Achieve rubrics used for the ELA review process are:

  • Rubric II. Quality of Explanation of the Subject Matter
  • Rubric VI. Quality of Instructional Tasks and Practice Exercises
  • Rubric VII. Opportunities for Deeper Learning

Achieve OER
Figure 18. Average Achieve OER ratings for all resources.

The Quality of Interactivity (Rubric V) used in the math review was not used in the ELA review. The intent of the rubric was to measure interactive modules, like assessments, that provide live feedback or widgets that could be manipulated to view variable outcomes. These types of objects were not present in the ELA resources we examined.

Resources scored well in these categories, with overall averages tending to fall in the Strong or Superior category.

Reviewer Comments

Reviewers were asked to write a short narrative providing an evaluation of each of the resources the reviewed. They were instructed to cite evidence from the resource that supported their comments about are as needing adaptation. Additionally, they provided suggestions for changes that would help improve alignment.

Reviewer Comments
Figure 19. Number of times out of 79 reviews that each potential use was cited.

As part of their professional assessments, reviewers clarified the ideal use scenario for each reviewed resource and estimated the amount of work that would be required for a small group to make adaptations to bring the resource into CCSS alignment. Finally, reviewers selected all the ways they would use the resource in both its current and adapted form. Below are some of the highlights, but for an in-depth look at comments for each resource, please visit the OER Project reviewed materials library.

  • Out of 79 reviews, 24 stated they would use a resource as a unit replacement in its current state. That number jumped to 35 if suggested adaptations were made.
    Current unit replacement(number of reviewers)
    Better Lesson Metacognition2
    Big Read Edgar Allan Poe1
    Engage NY Work of Art3
    Engage NY Wisely and Slow4
    NYC Dept. of Ed. Romeo and Juliet2
    NYC Dept. of Ed Speeches3
    Odell EB Arguments3
    Odell EBC Speeches1
    Odell Literary Techniques4
    Saylor Unit 5 Slavery1
  • Only 7 reviews out of 79 stated that they would not use a resource in some capacity in its current state.

The overall results shown in Figure 19 indicate the overall strength of OER ELA material currently available.

While the intent of this report is not to rank the products based upon their overall average scores, comparing the performance of the resources on certain scales or items provides meaningful information. The charts below show how the resources compared with each other based upon selected scales or items.

Alignment to the Depth of the CCSS
Figure 20. EQuIP. This scale looks at the overall alignment of the resource to the CCSS.

Key Shifts in the CCSS
Figure 21. This scale measures how the unit addresses key shifts in the CCSS.

Instructional Supports
Figure 22. EQuIP. Examines whether a unit is responsive to varied student learning needs.

Figure 23. EQuIP. Unit regularly assesses whether students are mastering standards-based content and skills.

Deeper Learning
Figure 24. Achieve OER. Measures the unit’s ability to engage learners in one or deeper learning skills, including think critically and solve complex problems, reason abstractly, construct viable arguments and apply discrete knowledge and skills to real-world situations.

Quality of Explanation of Subject Matter
Figure 25. Achieve OER. Rates how thoroughly the subject matter is explained or otherwise revealed in the object.

Detailed Findings

For detailed information on each reviewed ELA resource, including scores on all rubrics, extensive reviewer comments, and supplemental metadata, visit the OSPI OER Project Materials Review website.