ELA Findings

General Observations

Since most ELA middle school classes use a dynamic set of units rather than textbooks with a fixed sequence of lessons, this review focused on unit level resources. Reviewers evaluated twenty English Language Arts (ELA) units for 6th–8th grades.

DeveloperFull TitleShort Title
Better Lesson/Devon O'BrianNature, Naturalism, and The Call of the WildNature, Naturalism, Call
Better Lesson/Nicholas GearingFahrenheit 451: Novel StudyFahrenheit 451
EDSITEMENT! National Endowment for the HumanitiesUsing Textual Clues to Understand "A Christmas Carol"A Christmas Carol
EngageNY/Expeditionary LearningModule 1 Unit 1 - Percy Jackson and the Hero’s JourneyHero’s Journey
EngageNY/Expeditionary LearningModule 4A Unit 1 - Development of the Adolescent BrainAdolescent Brain
EngageNY/Expeditionary LearningModule 1 Unit 1 - War Coming Close to HomeWar Close to Home
Georgia Virtual LearningMiddle School Language Arts - 6th Grade: Friendship Unit*Friendship
Library of Congress/Alison Westfall and Laura MitchellFound Poetry with Primary Sources: The Great Depression*Found Poetry
Library of Congress/Patricia Solfest & Kimberly WardeanNatural Disasters: Nature's Fury*Nature’s Fury
National Endowment for the Arts: The Big Read/Erika KossThe Call of the Wild*Call of the Wild
NYC Dept. of Education/misc. authorsLiteracy: Can Animals Think?Can Animals Think
NYC Dept. of Education/misc. authorsLiteracy in ELA: Economics and the EnvironmentEcon and Environment
Odell EducationReading Closely for Textual Details: At the PoleAt the Pole
Odell EducationResearching to Deepen Understanding: Water ReferencesResearching Water
Odell EducationBuilding Evidence Based Arguments: E pluribus UnumE Pluribus Unum
Saylor.org AcademyUnit 1: Under the SeaUnder the Sea Unit
Saylor.org AcademyUnit 5: Figurative LanguageFigurative Language
Stanford Graduate School of EducationPersuasion Across Time and SpacePersuasion Across Time
Student Achievement Partners/Westside DistrictThe First EmperorFirst Emperor
Student Achievement Partners/Westside DistrictZlateh the GoatZlateh the Goat

* units pre-dating the Common Core State Standards

These OER were reviewed with the specific goal of looking at how well they address CCSS shifts, not evaluating their quality against existing Washington State grade level expectations. The CCSS in ELA are very different from previous K–12 state learning standards. In particular, there are several key shifts in instruction:

  1. Content knowledge built through content-rich nonfiction
  2. Reading, writing, and speaking grounded in evidence from text, both literary and informational
  3. Regular practice with complex text and its academic language

Though many of the units reviewed were crafted to specifically address the new standards, four of the units pre-date the CCSS. Thus, the review process compared these materials against target standards that developers were not originally aiming for at material creation. In those instances, we noticed much higher variation in reviewer scores. Though still within acceptable ranges of inter-rater reliability (see Data Analysis), interpretation of how well the legacy resources aligned with the new standards was a bit more challenging and open to user interpretation of the resource intent.

Most ELA middle school classes use a flexible set of units through the course of a quarter or semester, rather than textbooks with a fixed sequence of units and materials. Educators can reliably consider many of the OER ELA units that were reviewed for use in their classroom and be confident that the units can be reasonably adapted to meet the CCSS by a teacher well versed in the standards.

Overall, the findings indicated many strong choices from among the available OER materials for educators seeking ELA units with alignment to the CCSS. Seven units received an overall average score of 2 or higher (on a 0–3 point scale) across all rubrics. They were:

  • Hero’s Journey (EngageNY)
  • Adolescent Brain (EngageNY)
  • War Coming Close to Home (EngageNY)
  • E Pluribus Unum (Odell Education)
  • Persuasion Across Time and Space (Stanford Graduate Education)
  • Economics and the Environment (New York City Department of Education)
  • At the Pole (Odell Education)

An additional three units had average total scores at or above the midpoint of the scale.

  • Can Animals Think? (New York City Department of Education)
  • Researching to Deepen Understanding: Water (Odell Education)
  • Fahrenheit 451 (Better Lesson)

As with the mathematics review, this review process was not intended to rank or endorse the materials. As such, there are few comparative graphs in this report. It is also important to note that the materials reviewed are not the only ELA OER resources available – many others exist and new resources emerge regularly. We were limited in scope and solely examined ELA thematic units that extended instruction over multiple weeks and met the criteria outlined in the Selection Criteria.

This review should be viewed as an opportunity to provide input on the changes necessary to bring the OER resource into closer alignment with the CCSS. The reviews represent a point in time. More so than print materials, digital resources with an open license can be freely modified, so all the products that were reviewed can be and are frequently updated.

EQuIP Rubric

The EQuIP rubric is designed to be used at the unit, rather than full-course, level to get a more detailed picture of the quality of alignment to the CCSS for a resource. Reviewers considered four areas described below:

  • Alignment to the Rigors of the CCSS: the unit targets a set of grade CCSS ELA/Literacy standards; includes a clear and explicit purpose for instruction; selects texts of sufficient quality and scope that measure within the grade-level text complexity band; integrates reading, writing, speaking and listening
  • Key Shifts in the CCSS: the unit addresses reading text closely; capturing text-based evidence; writing from sources; using academic vocabulary; increasing text complexity; building disciplinary knowledge; providing a balance of texts and writing
  • Instructional Supports: the unit is responsive to varied student learning needs
  • Assessment: the unit regularly assesses whether students are mastering standards-based content and skills through direct, observable evidence, via accessible and unbiased method using varied modes of assessment

Each dimension had a number of criteria that were considered. The number of criteria for each dimension that were met was rated on a scale from 0–3 (None – 0, Few – 1, Many – 2, All – 3). The rubric also provides an Overall rating for the resource based upon the sum of each of four dimensions. Scores from 11–12 are considered Exemplar, 8–10 are Exemplar if Improved, 3–7 are in the Revision Needed category, and scores 2 and below are Not Ready to Review.

  • Reviewers gave the following Overall evaluations:
    Exemplar4 resources
    Exemplar if Improved4 resources
    Needs Revision10 resources
    Not Ready to Review2 resources
  • Average Alignment for all resources combined was 2.1, indicating that Many to All of the CCSS criteria were met. Eleven out of the 20 resources fell into this category (fig 20).
  • When reviewer scores for each resource were averaged, 10 of the resources met Many to All of the Key Shifts (fig 26).

EQuIP
Figure20. Average EQuIP ratings for all 20 resources – 80 total reviews.

When averaged, all resources were near or above the midpoint for each of the scales; however, ten resources consistently scored in the Many to All range for most categories – Adolescent Brain, At the Pole, Can Animals Think, Economics and Environment, E Pluribus Unum, Fahrenheit 451, Hero’s Journey, Persuasion Across Time, Researching Water, and War Close to Home.

The chart below shows EQuIP averages when the ten resources listed above are split out. We suggest that there are may be several reasons for the differences in scores. Two of the developers we examined, Georgia Virtual Learning and Saylor.org Academy, were specifically designing online courses. These are best used as self-directed practice for students or as supplemental material by teachers in a traditional classroom. They may also work well in a homeschool or alternative learning environment. As such, they do not neatly fall into the categories targeted by the EQuIP rubric. Additionally, some resources were created pre-Common Core and adapted to increase alignment to the new standards. Such adaptation is often less effective than resources created from scratch with the CCSS as guidance. Finally, some resources were of a smaller 1-2 week grain size and were not able to address all the EQuIP criteria in that instructional time period.

EQuIP – Breakdown Scores

Adolescent Brain; At the Pole; Can Animals Think; Economics and Environment; E Pluribus Unum; Fahrenheit 451; Hero’s Journey; Persuasion Across Time; Researching Water; War Close to Home

A Christmas Carol; Figurative Language; Found Poetry; Nature’s Fury; Call of the Wild; First Emperor; Friendship Unit; Nature, Naturalism, Call of the Wild; Under the Sea; Zlateh the Goat
Figure 21. Comparison of EQuIP averages when broken down between into two scoring tiers

Achieve OER Rubrics

The Achieve OER rubrics are specifically designed to be used with digital resources, as opposed to print media. They also examine other aspects of OER quality, may be used with any standards, and are designed to evaluate resources that may be smaller in grain size than units or lessons.

Achieve OER – Breakdown Scores

Figure 22. Average Achieve OER ratings for all resources.

The Achieve instrument has eight different smaller rubrics, several of which significantly overlap the EQuIP instrument. Since the EQuIP instrument was developed specifically to consider alignment to the CCSS, it was used in this review in lieu of the overlapping Achieve OER rubrics in order to minimize duplicative measurement scales. The three Achieve rubrics used for the ELA review process are:

  • Rubric II. Quality of Explanation of the Subject Matter
  • Rubric VI. Quality of Instructional Tasks and Practice Exercises
  • Rubric VII. Opportunities for Deeper Learning

The Quality of Interactivity (Rubric V) used in the math review was not used in the ELA review. The intent of the rubric was to measure interactive modules, like assessments, that provide live feedback or widgets that could be manipulated to view variable outcomes. These types of objects were not present in the ELA resources we examined.

In the breakout chart below, you can see that resources in the first group fell into the Strong or Superior category for each of the rubrics.

Achieve OER – Breakdown Scores

Adolescent Brain; At the Pole; Can Animals Think; Economics and Environment; E Pluribus Unum; Fahrenheit 451; Hero’s Journey; Persuasion Across Time; Researching Water; War Close to Home

A Christmas Carol; Figurative Language; Found Poetry; Nature’s Fury; Call of the Wild; First Emperor; Friendship Unit; Nature, Naturalism, Call of the Wild; Under the Sea; Zlateh the Goat
Figure 23. Comparison of Achieve OER averages when broken down between into two scoring tiers

Reviewer Comments

Reviewers were asked to write a short narrative providing an evaluation of each of the resources they reviewed. They were instructed to cite evidence from the resource that supported their comments about areas needing adaptation. Additionally, they provided suggestions for changes that would help improve alignment.

As part of their professional assessments, reviewers clarified the ideal use scenario for each reviewed resource and estimated the amount of work that would be required for a small group to make adaptations to bring the resource into CCSS alignment. Finally, reviewers selected all the ways they would use the resource in both its current and adapted form. Below are some of the highlights, but for an in-depth look at comments for each resource, please visit the OER Project reviewed materials library.

The overall results shown in Figure 24 indicate the overall strength of OER ELA material currently available.


Figure 24. Number of times out of 80 reviews that each potential use was cited.

Out of 80 reviews, 24 stated they would use a resource as a unit replacement in its current state. That number jumped to 35 if suggested adaptations were made.

Current unit replacement (number of reviewers out of four total reviewers per resource)

Better Lesson: Fahrenheit 4511
Better Lesson: Nature, Naturalism, Call of the Wild1
EngageNY: Hero’s Journey3
EngageNY: Adolescent Brain2
EngageNY: War Close to Home3
NEA Big Read: Call of the Wild1
NYC Dept. of Ed.: Can Animals Think1
NYC Dept. of Ed.: Economics and the Environment2
Odell Education: At the Pole2
Odell Education: E Pluribus Unum3
Odell Education: Researching: Water1
Stanford Education: Persuasion Across Time and Space4

While the intent of this report is not to rank the products based upon their overall average scores, comparing the performance of the resources on certain scales or items provides meaningful information. The charts below show how the resources compared with each other based upon selected scales or items.

Alignment to the Depth of the CCSS
Figure 25. EQuIP. This scale looks at the overall alignment of the resource to the CCSS.

Key Shifts in the CCSS
Figure 26. This scale measures how the unit addresses key shifts in the CCSS.

Instructional Supports
Figure 27. EQuIP. Examines whether a unit is responsive to varied student learning needs.

Assessment
Figure 28. EQuIP. Unit regularly assesses whether students are mastering standards-based content and skills.

Deeper Learning
Figure 29. Achieve OER. Measures the unit’s ability to engage learners in one or deeper learning skills, including think critically and solve complex problems, reason abstractly, construct viable arguments and apply discrete knowledge and skills to real-world situations.

Quality of Explanation
Figure 30. Achieve OER. Rates how thoroughly the subject matter is explained or otherwise revealed in the object.

Detailed Findings

For detailed information on each reviewed ELA resource, including scores on all rubrics, extensive reviewer comments, and supplemental metadata, visit the OSPI OER Project Materials Review website.