Applicants will be scored on each of the 54 criteria listed below. Each reviewer will mark an item as "Evident", "Partially Evident", or "Not Evident" based on the evidence provided by the applicant. Course demos will be considered as the primary evidence for the marked (*) criteria.
Note that there have been changes in the criteria over time. This document includes the current version of the criteria with those changes incorporated.
Proposed criteria changes for Spring 2015
Proposed changes for the spring 2015 review can be found here (pdf).
A. Course Content and Instructional Design
Courses and instruction incorporate the following to ensure a quality academic experience:
- Course goals and outcomes
- A1.* Stated course goals and objectives: Clearly stated and measurable objectives and course goals describe the student's knowledge at the end of the course.
Course goals and objectives are present, explicitly stated, and can be easily found by students. After reading the list of goals and objectives, students will understand what they will be learning throughout the course.
- A2.* Relevant course assignments: Course assignments are clearly stated, reflect course goals, and are representative of the scope of the course. The scope and sequence of the course is appropriately designed for the subject area and grade level. Concepts and skills are accurately presented, built on one another logically, and connections between concepts and subjects are explicit and relevant.
- A3.* Cognitive complexity: The course is structured to engage students in higher-order thinking, critical-reasoning activities, and thinking in increasingly complex ways; the learning environment stretches the individual learner to move beyond his/her comfort zone and grow as an independent learner.
Assignments, activities, and assessments require students to elevate their thinking beyond memorization into the realm of analyzing situations, synthesizing information, or evaluating an argument. Activities may include open-ended questions, and encourage students to categorize and classify information. Group work, decision-making, and finding patterns may also be included in the course activities.
- A4.* Assignment expectations: Students are provided guiding materials that include rationale, desired characteristics and clear expectations for graded assignments.
- Course content and organization
- A5.* Easily accessed course materials: Instructional materials, including supporting materials—such as instructional text, textbooks, manuals, and videos—are easily accessed by students.
Instructional materials are found online with few, if any, additional logins or technology requirements. Offline materials are sent to the student by the provider in a timely fashion at the beginning of the course or are obtained locally with little effort.
- A6. Readability of course content and materials: Readability levels, written language assignments, and mathematical requirements are appropriate for the course content.
Evidence shows that in the course design process, all course content – including instructional content and explanations – is written at appropriate readability levels for the grade level of the student audience, and readability formulas are used to identify the readability level. For math courses, the evidence shows that mathematical language is written at the appropriate level for the intended audience. District programs should disclose the manner by which their district curriculum adoption process addressed the determination of readability of any curriculum or course content outsourced from another provider.
- A7.* Course organization: Course content is organized in a consistent format (i.e., units and lessons), which include overviews describing central objectives, activities, and resources.
- A8. Consistent work flow: Course work, including assignments, reading, assessments, and activities are structured to promote consistent effort throughout the term.
Evidence may include documentation which describes an estimate of the expected amount of time spent by the student, per unit or activity, within each course.
- A9. Unbiased content and instruction: The course content is accurate and free of any bias.
Evidence shows that in the course design process, a procedure is used to eliminate bias. District programs should disclose the manner by which their district curriculum adoption process addressed the determination of bias in any curriculum or course content outsourced from another provider.
- Student engagement
- A10.* Variety of instructional methods: A variety of instructional methods are used within the class structure and course content to promote a student-centered learning environment that addresses different learning styles.
Evidence shows how the course offers multiple and varied kinds of learning experiences for acquiring knowledge or skill.
- A11. Collaborative instructional activities: The course includes appropriate teacher-student and student-student interaction to foster mastery and application of the material.
Learning activities foster appropriate teacher-student and student-student interaction. Within the course grading policy, guidelines defining student participation and expectations are identified.
- A12.* Participatory learning: The course includes activities that engage students in active participation and use of newly learned concepts.
Students are given the means for an increased level of participation through discovering, processing, and applying information they learn throughout the course. Less emphasis is placed on the teacher giving information and more on the student discussing, listening, writing, reading, and/or reflecting.
- A13. Interactive feedback: The course uses appropriate learning activities which foster teacher-student interaction (synchronously and/or asynchronously) and timely and frequent feedback about student progress.
Students receive timely and frequent feedback on their progress that emphasizes the intended learner outcomes. The feedback is highly individualized, detailed, and recommends specific, individualized improvement and strategies to encourage continued progress toward mastery.
- A14.* Modes of classroom interaction: The teacher promotes classroom interaction using various modes of communication.
The teacher drives teacher-student and student-student exchanges through various modes which may include email, discussions, synchronous chats, simulations, lab activities, and other group projects. Evidence shows that the communication features are present and that they are integrated into the activities.
- » Possible sources of supporting evidence for section A
B. Student Assessment
Courses and teachers use the following to ensure effective assessment of student performance:
- B1.* Appropriate assessment: Adequate and appropriate assessment methods and procedures for the subject matter are used throughout the course to assess students' mastery of content, course goals, and standards.
- B2.* Variety of assessments: The course includes assessment materials which assess students in a variety of ways.
The course uses a wide variety of assessment techniques to measure ongoing student progress on clearly identified learner outcomes. A variety of assessments reaches a broad and deep array of skill sets and learning models and allows students to demonstrate their understanding in different contexts. Assessment types may include pre-tests, post-tests, objective and subjective questioning, self-assessments, group projects, peer review, and portfolios and may incorporate teacher and/or computer-graded assessments and activities.
- B3.* Frequency of assessment: Frequent and ongoing assessments ensure each student has mastered the content and is prepared to move forward in the coursework.
- B4.* Effective use of technology in assessments: Selected assessments and the methods used for submitting assessments are an effective use of the technology provided in the courseware platform.
- B5.* Assessment rubrics: Assessment rubrics, answers and/or explanations are provided to the student.
Assessment rubrics are disclosed in preparation materials for assessments; correct answers and/or explanations are available at the end of assessments.
- » Possible sources of supporting evidence for section B
C. Classroom Management
Courses and instruction include the following to ensure an effective academic experience:
- C1.* Grading policies: Grading policies and practices are explicitly stated and presented to the student.
The policies include a grading scale that defines letter grades and/or weights, if applicable. As part of the grading policy, student participation – if required – is clearly defined. Any penalties that may be assessed to grades and/or extra credit opportunities are also identified within the policy. If regular contact with the teacher is required as part of the course, clear expectations for meeting this requirement are posted within the course.
- C2. Academic integrity expectations: Clearly stated academic integrity (plagiarism and netiquette) expectations regarding participation in lesson activities, discussions, and communications.
- C3. Ensuring academic integrity: The teacher ensures that authentic work has been produced by the student via measures which may include: tests administered by proctors, password-protected tests, limited test-time windows, synchronous contact with the teacher, and/or plagiarism checks.
- C4. Acceptable Use Policy: A clearly stated technology Acceptable Use Policy (AUP) is posted on the course provider's website, in the learning management system, or in the student handbook.
- C5. Discipline policies: A clearly defined discipline policy and procedure is posted on the course provider's website, in the learning management system, or in the student handbook to identify and deal with inappropriate student behavior and issues of discipline.
- » Possible sources of supporting evidence for section C
D. Course Evaluation and Management
The provider uses the following to ensure the delivery of a high quality program:
- D1. Course development quality assurance: The provider uses an internal review process verifying course reliability, completeness, and effectiveness.
Courses are developed and assessed by a standardized quality assurance protocol which addresses aspects such as course reliability, completeness, and effectiveness, prior to student participation; ongoing review and revision are managed by staff who process feedback about courses. For any curriculum or course content outsourced from another provider, district programs should disclose the manner by which their district curriculum adoption process addressed the provider’s quality assurance procedures.
- D2. Course maintenance: Planned, scheduled course maintenance (updates, revisions, etc) is conducted to ensure course content timeliness and accuracy, as well as functionality.
For any curriculum or course content outsourced from another provider, district programs should disclose the manner by which their district curriculum adoption process addressed the provider’s practice of ongoing maintenance.
- D3. Feedback gathering and implementation: Feedback is solicited from a variety of sources about various aspects of the program, including the quality of the course design, content, instruction, support systems, and/or infrastructure; findings are used as basis for improvement.
The provider exhibits regularly conducted course evaluations which gather input from students and other various constituents who may include teachers, staff, content experts, instructional designers, and outside reviewers. For any curriculum or course content outsourced from another provider, district programs should disclose the manner by which their district curriculum adoption process addressed the practice of the provider’s ongoing course evaluations. Additionally, district programs should disclose how local procedures provide for ongoing evaluation of the general program.
- D4. Provider transparency: Results of regularly conducted evaluations of courses are posted or otherwise noted as available upon request.
District programs outsourcing curriculum or course content may refer to their provider’s results of regularly conducted evaluations and/or to their own program evaluations.
- » Possible sources of supporting evidence for section D
E. Student Support
The provider uses the following to enhance student experience and success:
- E1. Technology skills: Expected technology skills are disclosed prior to enrollment in a course.
Technology skills include any skills necessary to be successful in any online course (keyboarding, web navigation, etc.) and any skills (coding, photo editing, etc.) necessary for the specific class.
- E2. Student-facing course demonstrations: The provider offers demonstrations of its courses and programs that are designed to give prospective students an understanding of the student experience and participation expectations.
- E3. Online student orientations: Required training and online support is provided to students to aid them in navigating the online environment.
Evidence should include demo access to the online student orientation. The orientation leaves students with an understanding of how to efficiently use the courseware system and complete all the technical aspects of participating in the online course/program (i.e., communicating with the online teacher, submitting assignments and assessments, participating in group work and/or discussions, etc). Student participation requirements may be exhibited by a policy that does not allow the student to progress into the academic portion of the course, by omission of points from the student’s overall course grade, etc.
- E4. Communicating with the teacher: Students are provided information about protocols for communicating with the teacher and course/program provider.
Teacher information, including contact, availability, and biographical information is provided. Information that clarifies how to contact the teacher via phone, email, and/or online messaging tools is provided within the contact information.
- E5. Monitoring and pacing: The provider uses a protocol for monitoring student progress and helping students keep up with the pace of their course.
Monitoring occurs throughout the student’s participation in the course, is conducted by program or course provider staff and may be facilitated by system monitoring tools. Pacing assistance may occur via course calendars, schedule-driven syllabi, system-driven reminders, and/or prompts.
- E6. Issue resolution: The provider uses policies and systems to resolve student, school, and parent questions, complaints, and appeals.
Complaint mechanisms and protocols are clearly explained and easily accessible to students and schools in advance of need. For school program providers, when coordination with the partnering district is required to implement the policy, evidence may include a model policy.
- » Possible sources of supporting evidence for section E
F. School-based Support
The provider uses the following to facilitate support of student success:
- F1. School-based support role: A school-based support person is recognized within the provider's systems and frameworks as the local/online adult point of contact who is not the course teacher but is available to the student and teacher and as a responsible agent of support to the student's success.
The provider's program, instructional practices, and student support policies clearly identify and engage the school-based online/local support person who may be recognized as a mentor, advisor, advocate, counselor, proctor, coordinator, or other school-based support.
- F2. School-based support systems: The school-based support person is provided various means to support student success which may include: the ability to view course content from the student or teacher point of view; technology troubleshooting information; online participation and communication tracking and grading systems; staff online handbook and policies; and teacher contact information.
- F3. School-based support training: The provider delivers training opportunities to school-based support people on the support role and on the available student support mechanisms and resources.
Training may include some exposure to the student and/or staff versions of the courseware, effective student support techniques, and technology troubleshooting guidelines.
- » Possible sources of supporting evidence for section F
The provider uses the following to facilitate successful use of its online systems:
- G1.* Ease of navigation: The navigation of courses and supporting systems is presented in a logical order allowing students to efficiently get from one place to another.
- G2. System technology requirements: Disclosure of program- and course-specific hardware, web browser, and software requirements is made prior to enrollment.
- G3. Technology support: Technology support is offered via various disclosed means including phone, email, and/or online help pages.
- G4. Technical issue resolution: The provider uses monitoring protocols and mechanisms which assure the user is contacted within 24 hours to resolve technical problems in a timely manner.
- G5. Platform flexibility: Course architecture permits the addition of content, activities, and assessments to extend learning opportunities, as needed.
The teacher for the course has the ability to make additions to the content within the learning management system.
- » Possible sources of supporting evidence for section G
H. Staff Development and Support
The provider uses the following to ensure the online teachers' ability to challenge and meet the needs of online students:
- H1. Teacher training – LMS: Teachers are trained in the online course delivery system on which they teach in order to effectively use the courseware and various instructional media and to assist students.
- H2. Teacher training – social aspects of online learning: Teacher training addresses the emotional and social aspects of online learning, particularly with regard to communicating with students.
Online teachers are trained to identify and mitigate the ways in which the online environment can enhance or hinder the learning experience and to be sensitive to the perception of written language.
- H3. Teacher training – student perspective: Teacher training includes the experience of online learning from the perspective of a student.
Teacher training includes exposure to the student version of the courseware system in order to understand the technical and logistical requirements of, and to be equipped to help the student function effectively in, the online learning environment.
- H4. New teacher support: New teachers are given extra support in their first year of instruction.
Supports and resources may include formal connections to other teachers, new teacher meetings, and the exchange of best practices to ensure their effectiveness and success.
- H5. Teacher performance reviews: Teacher performance reviews are conducted on a regularly scheduled basis.
- » Possible sources of supporting evidence for section H
I. Program Management
The provider uses the following to ensure effective program management:
- I1. Monitoring of teacher communications: A program administrator (ie, principal, department head, lead teacher, or instructional lead) uses established protocols to monitor and intervene as necessary in the quality and timeliness of teachers’ responses to students.
- I2. Monitoring of student performance: A program administrator uses established protocols to monitor student records to ensure students are progressing through their courses at an acceptable rate, to identify any problems, and to intervene when necessary.
- I3. Student outcomes: The provider uses student performance and outcome data to directly inform program improvements and modifications.
- I4. Academic calendar: The provider presents an academic calendar which clearly identifies course/program start and end dates and enrollment deadlines.
Information is disclosed regarding the expected duration of the course/coursework. For providers who enroll students in a cohort model, expectations for adherence to a course calendar are communicated to the student.
- I5. System-driven non-instructional tasks: Non-instructional tasks – enrollments, login information dissemination, and course materials delivery, etc. – are system-driven and performed via established protocols.
- I6. Financial procedures: The provider uses established protocols for handling enrollment fees and payments.
District providers should show evidence of proper procedures for handling enrollments, fees, and payments. This may include the claiming of state funding, ALE, and choice transfers. Non-district providers should show evidence of procedures and mechanisms by which fees are gathered for students enrolling in online courses.
- I7. Special Services: Provider has an identified procedure for confidentially receiving and implementing IEP and 504 plans.
- » Possible sources of supporting evidence for section I
* Course demos as evidence for asterisked criteria
Course demos will be considered as part of the submitted evidence for the criteria marked with an asterisk. Additional documentation may be provided. Use the comment box to specify where exactly reviewers should expect to find evidence within multiple course demos (i.e., Math 1, Unit 2, Lesson 3, Activity 4 and English 2, Unit 3, Lesson 4, Activity 5) or within the learning management system.