ELA Findings

General Observations

Since most ELA high school classes use a dynamic set of units rather than textbooks with a fixed sequence of lessons, this review focused on unit level resources. Reviewers evaluated twenty English Language Arts (ELA) units for the 11th - 12th grade. Many more OER units exist, and new resources emerge regularly.

DeveloperFull TitleShort Title
Georgia Virtual AcademyAmerican Literature - Colonial LiteratureColonial Literature
Learn NCPicturing America at the Turn of the Twentieth CenturyPicturing America
Library of Congress1900 America: Primary Sources and Epic PoetryEpic Poetry
Library of CongressThe US Constitution: Continuity and ChangeUS Constitution
National Endowment for the ArtsThe Great GatsbyGreat Gatsby
National Endowment for the ArtsAdventures of Tom SawyerTom Sawyer
National Endowment for the HumanitiesCritical Ways of Seeing The Adventures of Huckleberry FinnHuck Finn
NYC Department of EducationOn Behalf of OthersOn Behalf of Others
NYC Department of EducationCreating a Research Paper: Literacy in Social StudiesCreating Research
NYC Department of EducationAre Humans Good or Evil?Good/Evil
EngageNY (Odell Education)Promised LandPromised Land
EngageNY (Odell Education)Lay Down All My JoysLay Down All My
EngageNY (Odell Education)Life Steps Almost StraightLife Steps
ReadWriteThinkAn Exploration of Romanticism through Art and PoetryRomanticism
Saylor.orgAmerican Literature - Unit 1 America's Religious HeritageRelig Heritage
Saylor.orgAmerican Literature - Unit 2 Reason and Enlightenment in AmericaReason & Enlight
Utah Education NetworkAmerican Dream and the Great GatsbyAm Dream/Gatsby
Utah Education NetworkCharacter Analysis and the CrucibleCrucible
Mountain Heights Academy (formerly Utah Open High School)English 11 Q1Eng 11 Q1
Mountain Heights Academy (formerly Utah Open High School)English 11 Q2Eng 11 Q2

The CCSS in ELA are very different than previous K-12 state learning standards. In particular, there are several key shifts in instruction necessary:

  1. Content knowledge built through content-rich nonfiction
  2. Reading, writing, and speaking grounded in evidence from text, both literary and informational
  3. Regular practice with complex text and its academic language

The OER materials were reviewed with a specific goal of not evaluating their quality against existing standards but by looking at how well they address CCSS shifts. Like most of the currently available commercial textbooks, some of these OER materials were not designed specifically with the CCSS in mind. Thus, the review process compared some materials against target standards that developers were not originally aiming for at material creation. However, some of the materials reviewed were crafted to specifically address the new standards.

Overall, the findings indicated that there are many choices from among the available OER materials for educators seeking ELA units with some alignment to the CCSS that could be used as part of an English Language Arts high school class. For 10 of the 20 units reviewed, reviewers on average indicated Agree or Strongly Agree to the statement, “I would use this material in my classroom.” An additional 6 of the 20 units received an average rating of close to Agree. Educators can reliably consider many of the OER ELA units that were reviewed for use in their classroom and be confident that the units can be reasonably adapted to meet the CCSS by an experienced teacher well versed in the CCSS. Six units received an overall average score above 2.0 (on a 3.0 scale) across all rubrics. They were:

  • Promised Land
  • Good/Evil
  • Lay Down All My Joys
  • Life Steps
  • On Behalf of Others
  • Great Gatsby

As with the mathematics review, this review process was not intended to rank or endorse the materials. As such, there are few comparative graphs in this report. It is also important to note that the materials reviewed are not the only ELA OER resources available – others exist. We were limited in scope and solely examined ELA thematic units that extended instruction over multiple weeks. OSPI staff then pre-reviewed many of the units to select ones that had CCSS components.

This review should be viewed as an opportunity to provide input on the changes necessary to bring the OER resource into closer alignment with the CCSS. The reviews represent a point in time. More so than print materials, digital resources with an open license can be freely modified, so all the products that were reviewed can be and are frequently updated.

Publishers’ Criteria

The Publishers’ Criteria rubric examined three scales:

  • Quality of Text
  • Quality of Questions and Tasks
  • Writing

The Publishers’ Criteria rubric measures broad alignment to the intent of the CCSS. It addresses the range, complexity and focus on the text, inclusion of text dependent questions, and writing to sources.

Publishers' Criteria
Figure 13. Average Publishers' Criteria ratings for all resources combined.

Average responses for the items in the three scales trended toward Agree, indicating a good overall baseline quality measure.

EQuIP Rubric

The EQuIP rubric measures overall quality of alignment to the CCSS at a unit level. It looks at four areas including Alignment, Key Shifts, Instructional Supports, and Assessment. The rubric also provides an Overall assessment rating for the resource based upon the four domains listed above. Note that in the EQuIP graph, two measurement scales are used. The bottom axis shows the scale values for the four areas of focus, and the top axis shows the ratings for the Overall recommendation for the resources. EQuIP is designed to be used at the resource unit level, rather than the full course, to get a more detailed picture of the quality of alignment to the CCSS for a resource. Note the Alignment bar shows that for all resources combined, the average score indicates that many of the CCSS criteria are met. As in mathematics, Assessment is the weakest component of the units on average, but the rating is still reasonably positive.

EQuIP
Figure 14. Average EQuIP ratings for all resources combined.

The Overall rating for all of the products ranged from Not Recommended to Exemplar, with a majority of the products receiving an Overall rating of Needs Revision (37 of 80 total responses) or Exemplar if Improved (28 of 80). There were only six instances where reviewers rated a product Not Recommended, and in nine instances reviewers rated a product Exemplar.

Achieve OER Rubrics

The Achieve OER rubrics are specifically designed to be used with digital resources, as opposed to print media. They also examine other aspects of OER quality, may be used with any standards, and are designed to evaluate resources that may be smaller in grain size than units or lessons.

The Achieve instrument has eight different smaller rubrics, several of which significantly overlap the EQuIP instrument. Since the EQuIP instrument was developed specifically to consider alignment to the CCSS, it was used in this review in lieu of the overlapping Achieve OER rubrics in order to minimize duplicative measurement scales. The four Achieve rubrics used for this review process are:

  • Rubric II. Quality of Explanation of the Subject Matter
  • Rubric V. Quality of Technological Interactivity
  • Rubric VI. Quality of Instructional Tasks and Practice Exercises
  • Rubric VII. Opportunities for Deeper Learning

Achieve OER
Figure 15. Average Achieve OER ratings for all resources.

The Quality of Interactivity rubric proved problematic for the ELA review. The intent of the rubric was to measure interactive modules, like assessments which provide live feedback or widgets that could be manipulated to view variable outcomes. Fifty of the eighty responses for this scale indicated that the item was not applicable to the unit being reviewed, because there were no interactive components. The remaining responses had high variability due to conflicting interpretations of what constituted an interactive component, and ultimately, the scale was dropped from subsequent data analysis because so few of the units reviewed were designed to have interactive components.

Reviewer Comments

Reviewers were asked to write a short narrative providing an assessment of each of the resources they reviewed. As part of their professional assessments, they were asked to identify the amount of work they felt was necessary to bring the product into alignment with the CCSS. They were also asked to identify their level of agreement with the statement, “I would use this resource in my classroom.” The overall results are shown in Figure 16. Of note here is that the average rating for all ELA units for the item, “I would use this in my classroom,” is close to Agree, which indicates the overall strength of OER ELA material currently available.

Reviewer Comments
Figure 16. Overall results for Reviewer Comments, showing averages for all resources combined.

While the intent of this report is not to compare or rank the products based upon their overall average scores, comparing the performance of the resources on certain scales or items provides meaningful information. The charts below show how the resources compared with each other based upon selected scales or items.

I would use this in my classrooms
Figure 17. Reviewer Comments. Note that the average reviewer score exceeds or approaches Agree in 16 of the 20 cases.

Amount of Alignment Work
Figure 18. Reviewer Comments. This item measures the amount of work necessary to complete the alignment of the resource to the CCSS. It does not reflect the degree of alignment, which may be different. This is a reverse scale item. A longer bar is better, in that less work is necessary to bring the unit into alignment with CCSS.

Alignment
Figure 19. EQuIP. This scale looks at the overall alignment of the resource to the CCSS. Fourteen of the twenty products reviewed meet many or all of the criteria for alignment to the CCSS.

Key Shifts
Figure 20. This scale measures how the unit addresses key shifts in the CCSS, including requiring the student to read the text closely, cite text-based evidence, write from sources and build academic vocabulary.

Instructional Supports
Figure 21. EQuIP. While a few of the units reviewed met many of the criteria for instructional supports, a majority of the units reviewed scored at the midpoint or below on average for this scale.

Assessment
Figure 22. EQuIP. This chart shows which units have strong assessment tools that meet many or all of the criteria identified in the Assessment scale of the EQuIP rubric.

Quality of Text
Figure 23. Publishers' Criteria. The Quality of Text scale measures the range and complexity of text, along with sufficient practice in reading complex texts.

Quality of Questions and Tasks
Figure 24. Publishers’ Criteria. The Quality of Questions and Tasks scale measures both the focus on the text and the inclusion of text dependent questions.

Writing
Figure 25. Publishers' Criteria. The Quality of Writing scale measures how the written and oral tasks require the student to draw on textual evidence, using a balance of argumentative and explanatory writing.

Deeper Learning
Figure 26. Achieve OER. Twelve of the twenty units reviewed showed strong or superior opportunities for deeper learning.

Explanation
Figure 27. Achieve OER. This scale measures the quality of explanation of the subject matter, for a target audience of either the teacher or students.

Detailed Findings

For detailed information on each reviewed ELA resource, including scores on all rubrics, extensive reviewer comments, and supplemental metadata, visit the OSPI OER Project Materials Review website.