relative to an average performance on a neutral field against an average opponent"

We don't know the formula but it is assessing how good the defense is by determining what the average offense is likely to against them on a neutral field and then comparing that to how you actually performed. Then all games during the season are graded to determine the averages.