Accountability Indicators and School Performance Grades Business

Accountability Indicators and School Performance Grades Business Rules 2017–18

NCDPI| Division of Accountability Services| Analysis and Reporting Section—8/28/2018 Page 1

This document contains the business rules applied to data used in North Carolina school accountability indicators reporting

and the calculations of the School Performance Grades (SPG). It provides information on how an indicator is calculated

(determining the numerator and denominator). The following sections are found in this document:

Indicators in Every Student Succeeds Act (ESSA) Accountability Model (School Performance Grades)

Calculations for School Performance Grades

Definitions Used in Accountability Measures

Additional Measures Reported

As noted, some indicators and calculation processes have distinct business rules separate from the rules noted in this

document.

The following notes and summary business rules apply to all indicators and calculations as defined.

Students must meet partial enrollment for each accountability indicator except Cohort Graduation Rate to be

included in the School Performance Grades. Please refer to Partial Enrollment business rules for more information.

Partial Enrollment business rules can be found on the Accountability Services website.

Subgroup Rules

• Students categorized as a Student with Disabilities (SWD) or an English Learner (EL) at any time during their

designated 4-year graduation cohort count in these associated subgroups for calculations of the Cohort

Graduation Rate.

• Students who met the defined criteria for exiting English Learners (EL) status at any point in the previous four

years are included in the denominator of the EL subgroup for all indicators, except the English Learners

Progress indicator.

• The following subgroups receive a school performance grades:

Race/Ethnicity groups (American Indian, Asian, Black, Hispanic, Two or More Races, and White)

Students with Disabilities

English Learners

Economically Disadvantaged

Indicators in ESSA Accountability Model (School Performance Grades)

Reading and Mathematics End-of-Grade (EOG) and End-of-Course (EOC) Assessments

Science End-of-Grade Assessments

English Learners Progress (ELP)

4-Year Cohort Graduation Rate (CGR)

School Growth

Elementary/Middle School Growth (Reading, Math and Science)

High School Growth (Reading and Math)

Biology End-of-Course Assessments

ACT/WorkKeys Assessments (AWA)

Passing NC Math 3 (Math Course Rigor)

NCDPI| Division of Accountability Services| Analysis and Reporting Section—8/28/2018 Page 2

I. Reading and Mathematics End-of Grade (EOG) and End-of Course (EOC)

Annual assessments are administered as follows: EOG English Language Arts (ELA)/Reading and Mathematics at

grades 3–8, EOC assessments in NC Math 1, NC Math 3 (starting in 2018–19), English II and the related alternate

assessments (NCEXTEND1) for students receiving instruction on the extended content standards.

The assessments are reported as five academic achievement levels (Levels 1–5) with Levels 3–5 demonstrating North

Carolina grade-level proficiency and Levels 4–5 demonstrating on track for college and career. SPG calculations use

scores that meet grade-level proficiency.

1. The denominator includes all current year assessment scores for eligible students in membership (i.e., enrolled in a

school) at grades 3 through 8 and in high school courses in which an EOC assessment is required.

a. The participation expectation for all assessments is 95%. If a school fails to meet the 95% participation

rule for ELA/reading or mathematics, an adjustment is made to the denominator to ensure the denominator

accounts for 95% of students expected to test. For more information regarding participation rules please

go to the Accountability Services website.

b. Students who are in their first or second year in a U.S. school are not included in proficiency calculations;

c. For schools starting with grade 9 and for 2017–18 ONLY, NC Math 1 scores for students who took the

course prior to high school are included in grade 9.

d. NC Math 1 scores for students taking the course prior to their grade 8 year are counted when the student is

in grade 8 – Please refer to NC Math Exemption Business Rules on Accountability Services website.

e. Beginning in 2018–19 the mathematics high school accountability assessment for students who took NC

Math 1 for the first time prior to grade 9 is NC Math 3. Again, please refer to the NC Math Exemption

Business Rules.

f. For this indicator to be included in SPG calculations, the school must have a total of 30 reading and

mathematics scores combined.

g. Students with a North Carolina Department of Public Instruction (NCDPI) approved medical exemption

are excluded from all relevant proficiency calculations.

2. The numerator is based on the number of students scoring grade level proficiency. Annual assessments for SPGs

are calculated by determining the percent of students who score at Achievement Level 3 or higher.

a. If two valid scores are received in the same accountability year for a student enrolled in an EOC course, the

higher score is used for proficiency calculations.

b. Students earning credit through the Credit by Demonstrated Mastery (CDM) program in a course that

administers an EOC assessment are counted as proficient.

c. Students in Grade 3 who are not proficient on the EOG but receive achievement Level 3 or higher on the

Beginning of Grade 3 ELA/Reading Test (BOG) count as proficient.

d. Students who score below Achievement Level 3 on an EOG or EOC assessment (or their alternates) in the

current school year and obtain a higher score while enrolled in a summer school program, that is completed

and test scores are submitted before the published end of the accountability year, have the higher score

replace the lower score in the performance calculations for all. (Testing for this purpose is available for

grade 3 mathematics, grades 4–8 ELA/reading and mathematics, NC Math 1, and English II.).

e. All students with disabilities who take the NCEXTEND1 alternate assessment are included in performance

calculations.

II. Science End-of Grade Assessments

Annual assessments administered are as follows: EOG Science at grades 5 and 8 and the related alternate assessments

(NCEXTEND1).

The assessments have are reported as five academic achievement levels (Levels 1–5) with Levels 3–5 demonstrating North

Carolina grade-level proficiency and Levels 4–5 demonstrating on track for college and career. SPG calculations use scores

that meet grade-level proficiency.

NCDPI| Division of Accountability Services| Analysis and Reporting Section—8/28/2018 Page 3

1. The denominator includes all current year assessments for eligible students in membership (i.e., enrolled in a school)

at grades 5 and 8.

a. Students who are in their first or second year in a U.S. school are not included in proficiency calculations.

b. Students with a North Carolina Department of Public Instruction (NCDPI) approved medical exemption

are excluded from all relevant proficiency calculations.

2. The numerator is based on the number of students scoring grade level proficiency. Annual assessments for SPGs

are calculated by determining the percent of students who score at Achievement Level 3 or higher.

a. Students who score below Achievement Level 3 on an EOG assessment (or their alternates) in the current

school year and obtain a higher score while enrolled in a summer school program, that is completed, and

test scores are submitted before the published end of the accountability year, have the higher score replace

the lower score in their performance calculations for SPG. (Testing for this purpose is available for grades

5 and 8 science.)

b. All students with disabilities who take the NCEXTEND1 alternate assessment are included in performance

calculations.

III. English Learners Progress

1. The denominator is the number of ELs in grades 3–8 or 10.

2. The numerator is the number of ELs who met North Carolina’s definition of progress toward English language

attainment as demonstrated on the language proficiency test.

Additional business rules for the EL Progress Indicator can be found on the Accountability Services website.

IV. 4-Year Cohort Graduation Rate

The Cohort Graduation Rate denominator is the number of students that are part of a designated cohort as defined using

the 4-year Cohort Graduation Rate business rules found on the Accountability Services website.

1. The denominator is the total number of students expected to graduate within four years in the current year’s cohort.

2. The numerator is the number of students who graduate (earned a high school diploma) on time as defined by the

designated cohort.

V. School Growth

North Carolina uses the EVAAS growth index for the specified subjects and converts the index to a 100-point value

prior to inclusion in the SPG.

Rules applied to growth are as follows:

a. Elementary and middle schools use ELA/Reading EOG, Mathematics EOG/EOC and Science EOG

assessments for growth calculations.

b. High schools use ELA/Reading and Mathematics EOC assessments for growth calculations.

c. Subgroups receive a growth score for inclusion in the subgroup school performance grade.

d. A growth index is only used if there are 30 current year scores that are used for the growth calculation for

the school/subject or subgroup.

NCDPI| Division of Accountability Services| Analysis and Reporting Section—8/28/2018 Page 4

Growth for high schools is defined as part of the academic achievement indicator, but the weight of growth, as required

by § 115C-83.15, is 20% of the model.

For elementary and secondary schools that are not high schools, the School Quality or Student Success Indicator for all

grade spans and all schools is growth which accounts for 20% of the model as required by §115C-83.15.

North Carolina school growth is measured by EVAAS, a value-added growth model that includes student performance

on all applicable subject assessments for that school. The growth model calculations result in a composite growth value.

The composite growth value spans a range from -10.0 to 10.0 (it is possible to achieve values greater than 10.0 or below

-10.0 but are transformed to 10.0 and -10.0 for use in the accountability model). These composite growth values are

transformed to a 100-pt score rounded to 10

and this score is used as 20% of the overall school/subgroup grade. North

Carolina reports the growth values that correspond to EVAAS designations for all schools. The distribution of the

growth designations (did not meet, met, and exceeded) allows for additional meaningful differentiation between schools

and subgroups.

• If a school or subgroup achieves a composite growth value below -2.0, the growth expectation was not met.

• If a school or subgroup achieves a composite growth value between -2.0 and 2.0, the growth expectation was met.

• If a school or subgroup achieves a composite growth value equal to or above 2.0, the growth expectation was

exceeded.

VI. Biology End-of Course Assessments

Annual assessments administered are as follows: Biology and the alternate assessment (NCEXTEND1).

The assessments have are reported as five academic achievement levels (Levels 1–5) with Levels 3–5 demonstrating North

Carolina grade-level proficiency and Levels 4–5 demonstrating on track for college and career. SPG calculations use scores

that meet grade-level proficiency.

1. The denominator includes all current year assessments for eligible students in membership (i.e., enrolled in a school)

in high school courses in which an End-of-Course (EOC) assessment is required.

a. Students who are in their first or second year in a U.S. school are not included in proficiency calculations.

b. For schools starting with grade 9, biology assessments taken prior to grade 9 are banked to grade 9 for

calculations. Banked biology scores are not included in the school where the student took the assessment

at grade 8 unless it is the same school in grade 9

c. Students with a North Carolina Department of Public Instruction (NCDPI) approved medical exemption

are excluded from all relevant proficiency calculations.

2. The numerator is based on the number of students scoring at grade level proficient. Annual assessments for SPGs

are calculated by determining the percent of students who score at achievement Level 3 or higher.

a. If two valid scores are received in the same accountability year for a student enrolled in an EOC course, the

higher score is used for proficiency calculations.

b. Students earning credit through the Credit by Demonstrated Mastery (CDM) program in a course that

administers an EOC assessment is counted as proficient.

c. Students who score below Achievement Level 3 on an EOC assessment (or their alternates) in the current

school year and obtain a higher score while enrolled in a summer school program, that is completed, and

test scores are submitted before the published end of the accountability year, have the higher score replace

the lower score in their performance calculations for SPG. (Testing for this purpose is available for biology).

d. All students with disabilities who take the NCEXTEND1 alternate assessment are included in performance

calculations.

NCDPI| Division of Accountability Services| Analysis and Reporting Section—8/28/2018 Page 5

VII. ACT/ACT WorkKeys Assessments (AWA Indicator)

1. The denominator is the number of grade 12 students who have either a valid ACT (from grade 11) or a valid

WorkKeys assessment score (i.e. student must also be a Career and Technical Education (CTE) Concentrator).

a. Students with an approved ACT or ACT WorkKeys Exemption score are included in the denominator

b. Grade 12 students are students who are in grade 12 at the First Day Spring testing date or mid-year graduates

who are in grade 12 at the First Day Fall testing date.

c. English Learners who take both assessments during their first or second year in a U.S. school are not

included.

d. Students who do not have a valid ACT or ACT WorkKeys assessment are not included in the denominator,

this includes:

i. Students who are instructed on the Extended Content Standards and participated in the

NCEXTEND1 at grade 11

ii. Students with an NCDPI approved Medical Exemption

iii. Students who take the College and Career Readiness Achievement Assessment (CCRAA) alternate

assessment.

iv. English Learners who earned the score in their first or second year in a U.S. school are excluded

from the denominator for SPG purposes but are included in other state reporting documents.

v. For 2017 and prior ACT scores, the EL exemption rule only applies to students who were in their

first year at a U.S. school at the time of testing and applies to SPG calculations and all other state

reports.

2. The numerator is the number of students who either achieved a composite score of 17 or higher on the ACT or

achieved a Silver, Gold or Platinum on the ACT WorkKeys assessment.

a. Students with an approved ACT or ACT WorkKeys exemption are included in the numerator.

b. When an eligible English Learner takes the ACT WorkKeys assessment in their third year in a U.S. school

the score is included. If the ACT WorkKeys assessment is not proficient (Silver or higher) the ACT score

is reviewed and if proficient (Composite 17 or higher), the student is included in the numerator.

A student’s ACT WorkKeys score is only considered for inclusion in the numerator if they are also a CTE

concentrator.

VIII. Passing NC Math 3 (Math Course Rigor)

1. The denominator is based on all grade 12 students. Grade 12 students are determined by grade level at each school’s

First Day of Spring (FDS) data collection or mid-year graduates who were also in grade 12 at First Day of Fall

(FDF) data collection.

a. Students who transfer into a school are included in the denominator.

b. Students who are instructed in the Extended Content standards or on the Occupational Course of study are

included in the denominator.

2. The numerator is the number of grade 12 students who have earned credit in the designated math course of NC

Math 3 or equivalent. The course codes used to determine course credit are: 2024, 2300, 2301, 2302, or 2309.

NCDPI| Division of Accountability Services| Analysis and Reporting Section—8/28/2018 Page 6

IX. Calculations for School Performance Grades

Please note the following changes effective with the 2017-18 school year:

1. If growth lowers the SPG, the school receives the lower grade

2. Biology growth has been removed from the high school growth measure

3. A

+NG

is not a designation

4. Subgroup grades at the school-level are calculated and reported for each subgroup within a school that has enough

data to make a computation (i.e., at least one indicator with 30 data points)

For all indicators, the denominator must meet the minimum number of data points (30) after all business rules are

applied to be included in the SPG or subgroup grade calculation. All non-growth indicators account for 80 percent of

the SPG. The indicators’ denominators are added together to create the composite denominator. The corresponding

numerators are added to make the composite numerator. The numerator is divided by the denominator and multiplied

by 100 to get a total achievement score rounded to 10

. This score is multiplied by .8 (80%). This result is added to

the converted 100-point growth score after it is multiplied by .2 (20%) to achieve a final SPG or subgroup grade.

Sample high school Calculation, assuming all Indicators had at least 30 scores after all business rules are applied:

Total Score = 68.0(.8) + 75.2(.2) = 54.4 + 15.0 = 69.4

Measure

Numerator

Denominator

Achievement Score used in final

calculations

Academic Achievement

Assessments (AAA)

Combines ELA/Reading

and Math

117 (Math) +

135 (English II)

269 (Math) +

274 (English II)

4-year Cohort Graduation

Rate

284

330

EL Progress

EOC Biology

124

240

The ACT/WorkKeys

Assessments

197

238

Math Course Rigor

261

273

Total

1127

(sum of numerators)

1658

(sum of denominators)

1127/1658 = 68.0

Composite Index

Growth Score used in final calculations

Accountability Growth

Score (Reading and Math

Composite)

-0.95

75.2

NCDPI| Division of Accountability Services| Analysis and Reporting Section—8/28/2018 Page 7

A similar calculation is conducted to determine a separate Reading and a separate Mathematics SPG for schools serving

grades 3-8. Separate Reading and Mathematics SPGs are for whole school only, not subgroups.

When calculating the SPGs and subgroup grades, the achievement score is rounded to the 10

before being combined

with the growth score. Prior to assigning all letter grades the final score is rounded to the whole.

As per the ESSA, School Performance grades are used to meaningfully differentiate schools for the identification of:

1. Comprehensive Support and Improvement (CSI) schools and;

2. Targeted Support and Improvement (TSI) schools

The flowing rules may apply to schools without enough data for calculations:

a. K-2 schools receive the same SPG as the school in the LEA receiving the highest percentage of their students

for third grade.

b. When a school does not have any indicators with enough data to compute a designation using current year data

only, the school’s SPG is determined from a combined 3-year calculation.

c. If a school who does not have enough data to compute a designation is also a NC approved alternative school,

the school has the option to use a 3-year calculation or have all scores returned to other schools within the

district and the alternative school receives the SPG of the school to which most of the scores were returned.

Alternative charter schools will default to a 3-year calculation option if needed.

Alternate Calculation for Special Weighting

To ensure School Quality or Student Success (SQSS) measures do not have significantly more weight in the High School

accountability model, NCDPI conducts a relative percent analysis to ensure the growth (20%) plus the Achievement Relative

Percent (HS Math, HS Reading, 4-year cohort graduation rate and EL Progress measures) is greater than the relative percent

of the SQSS measures (Biology, AWA Indicator and Math Course Rigor).

The following calculations is used for this analysis:

Achievement denominator (AD) = HS Math denominator + English II Denominator + 4-year Cohort Graduation Rate

denominator + English Learners Progress denominator

SQSS denominator (SD) = Biology denominator + AWA Indicator + Math Course Rigor denominator

Growth Percent (GP) = 20 (or 0 if not enough data to calculate growth)

Achievement Relative Percent (ARP) = [AD/(AD+SD)] x 0.8

SQSS Relative Percent (SRP) = [SD/(AD+SD)] x 0.8

If (GP + ARP) < SRP then SQSS has more weight than the Achievement Indicators. When this occurs, these

schools’ measure of annual differentiation is adjusted in the following manner:

The English EOC, Math EOC, 4-year Cohort Graduation Rate and EL Progress academic achievement indicators:

account for 31 percent of the schools’ overall score. The growth academic achievement indicator accounts for 20

percent of the overall score. All of the academic achievement indicators combined account for 51 percent of the

model. Thus, the high school SQSS measures (Biology, AWA Indicator and Math Course Rigor) account for 49

NCDPI| Division of Accountability Services| Analysis and Reporting Section—8/28/2018 Page 8

percent of the schools’ overall score. This ensures that the academic indicators result in a greater weight than the

SQSS indicators.

Additional details regarding the calculations of SPGs can be found in the Accountability Brief: School Performance

Grades on the Accountability Services website.

X. Additional ACT and WorkKeys Notes

To define valid ACT and WorkKeys assessments and for reporting purposes outside the accountability system, the

following rules are applied to ACT and WorkKeys assessments in the year the assessments are administered:

ACT Assessment

1. Denominator is based on 11

grade students based on the March data collection

a. Students whose assessments are declared misadministrations or are vacated by ACT are included in

denominator.

b. Students who are repeating the 11

grade, who have a previous score on the ACT assessment, and therefore are

not eligible to test, do not count against participation and are removed from both the numerator and the

denominator.

c. Students who take the NCEXTEND1 Alternate Assessment at Grade 11 are excluded from the denominator.

d. Students who meet the eligibility requirements and take the College- and- Career Readiness Alternate

Assessment are excluded from the denominator.

e. Students with an NCDPI-approved medical exception do not count in the denominator.

f. Students who are identified as EL in their first or second year in a U.S. school are not included in the

denominator.

2. Numerator includes students who meet the UNC System admissions minimum composite score of 17.

a. Students who have an approved ACT exception request, as noted below submitted by the LEA/charter school

test coordinator are included in the performance measure as meeting the UNC System’s minimum composite

score of 17.

ACT WorkKeys Assessment

1. The denominator is based on 12

grade students who are CTE concentrators

a. 12 grade students include those who are in grade 12 at First Day Spring (FDS) or who are in grade 12 at First

Day Fall (FDF) who are mid-year graduates.

b. Students with an NCDPI-approved medical exception do not count in the denominator.

2. The numerator is the number of CTE concentrators who earn a silver certificate or better on the assessment.

a. Students who have an approved ACT WorkKeys exception request, as noted below submitted by the

LEA/charter school test coordinator are included in the numerator.

ACT Exception—Students who submit a request to the LEA/charter school to not be required to participate in the ACT.

These students must have previously taken the ACT or SAT prior to the state administration window and met the

eligibility criteria to be exempted from taking the ACT during the state administration.

To meet the eligibility criteria, students must have either SAT or ACT test scores that meet the following college

readiness benchmarks:

NCDPI| Division of Accountability Services| Analysis and Reporting Section—8/28/2018 Page 9

SAT college readiness benchmark scores include the following subtests:

Before March 2016

After March 2016

Subtest

Score

Subtest

Score

Critical

Reading

500

Evidence-Based

Reading and Writing

480

Mathematics

500

Mathematics

530

Writing

500

ACT college readiness benchmark scores include the following subtests:

Subtest

ACT Score

English

Reading

Mathematics

Science

Writing

Prior to and after 2015–16 = 7 2015–16 = 18

ACT WorkKeys Exception—Students who submit a request to the LEA/charter school not be required to participate

in the ACT WorkKeys because they previously took the WorkKeys assessment prior to the state administration

window(s) and met the eligibility criteria to be exempt from taking the ACT WorkKeys during the state administration.

must provide documentation of having received a Silver, Gold, or Platinum WorkKeys certificate.