Accuracy at a Given Cutoff.

Find the accuracy at a given cutoff. Actuals should be binary, where 1 = present and 0 = absent.

Usage

accuracyAtCutoff(
  predicted,
  actual,
  cutoff,
  UH = NULL,
  UM = NULL,
  UCR = NULL,
  UFA = NULL
)

Arguments

predicted

vector of continuous predicted values.

actual

vector of binary actual values (1 = present and 0 = absent).

cutoff

numeric value at or above which the target condition is considered present.

UH

(optional) utility of hits (true positives), specified as a value from 0-1, where 1 is the most highly valued and 0 is the least valued.

UM

(optional) utility of misses (false negatives), specified as a value from 0-1, where 1 is the most highly valued and 0 is the least valued.

UCR

(optional) utility of correct rejections (true negatives), specified as a value from 0-1, where 1 is the most highly valued and 0 is the least valued.

UFA

(optional) utility of false positives (false positives), specified as a value from 0-1, where 1 is the most highly valued and 0 is the least valued.

Value

• cutoff = the cutoff specified

• TP = true positives

• TN = true negatives

• FP = false positives

• FN = false negatives

• SR = selection ratio

• BR = base rate

• percentAccuracy = percent accuracy

• percentAccuracyByChance = percent accuracy by chance

• percentAccuracyPredictingFromBaseRate = percent accuracy from predicting from the base rate

• RIOC = relative improvement over chance

• relativeImprovementOverPredictingFromBaseRate = relative improvement over predicting from the base rate

• SN = sensitivty

• SP = specificity

• TPrate = true positive rate

• TNrate = true negative rate

• FNrate = false negative rate

• FPrate = false positive rate

• HR = hit rate

• FAR = false alarm rate

• PPV = positive predictive value

• NPV = negative predictive value

• FDR = false discovery rate

• FOR = false omission rate

• youdenJ = Youden's J statistic

• balancedAccuracy = balanced accuracy

• f1Score = F1-score

• mcc = Matthews correlation coefficient

• diagnosticOddsRatio = diagnostic odds ratio

• positiveLikelihoodRatio = positive likelihood ratio

• negativeLikelhoodRatio = negative likelihood ratio

• dPrimeSDT = d-Prime index from signal detection theory

• betaSDT = beta index from signal detection theory

• cSDT = c index from signal detection theory

• aSDT = a index from signal detection theory

• bSDT = b index from signal detection theory

• differenceBetweenPredictedAndObserved = difference between predicted and observed values

• informationGain = information gain

• overallUtility = overall utility (if utilities were specified)

Details

Compute accuracy indices of predicted values in relation to actual values at a given cutoff by specifying the predicted values, actual values, and cutoff value. The target condition is considered present at or above the cutoff value. Optionally, you can also specify the utility of hits, misses, correct rejections, and false alarms to calculate the overall utility of the cutoff. To compute accuracy at each possible cutoff, see accuracyAtEachCutoff.

See also

Other accuracy: accuracyAtEachCutoff(), accuracyOverall(), nomogrammer(), optimalCutoff(), posttestOdds()

Examples

# Prepare Data
data("USArrests")
USArrests$highMurderState <- NA
USArrests$highMurderState[which(USArrests$Murder >= 10)] <- 1
USArrests$highMurderState[which(USArrests$Murder < 10)] <- 0

# Calculate Accuracy
accuracyAtCutoff(predicted = USArrests$Assault,
  actual = USArrests$highMurderState, cutoff = 200)
#>   cutoff TP TN FP FN   SR   BR percentAccuracy percentAccuracyByChance
#> 1    200 15 30  4  1 0.38 0.32              90                   54.32
#>   percentAccuracyPredictingFromBaseRate      RIOC
#> 1                                    68 0.8991935
#>   relativeImprovementOverPredictingFromBaseRate     SN        SP TPrate
#> 1                                       0.34375 0.9375 0.8823529 0.9375
#>      TNrate FNrate    FPrate     HR       FAR       PPV       NPV       FDR
#> 1 0.8823529 0.0625 0.1176471 0.9375 0.1176471 0.7894737 0.9677419 0.2105263
#>          FOR   youdenJ balancedAccuracy   f1Score       mcc diagnosticOddsRatio
#> 1 0.03225806 0.8198529        0.9099265 0.8571429 0.7879121               112.5
#>   positiveLikelihoodRatio negativeLikelihoodRatio dPrimeSDT   betaSDT
#> 1                 7.96875              0.07083333  2.720952 0.6234551
#>         cSDT      aSDT bSDT differenceBetweenPredictedAndObserved
#> 1 -0.1736446 0.9476103 0.85                                 207.8
#>   informationGain
#> 1       0.4947688
accuracyAtCutoff(predicted = USArrests$Assault,
  actual = USArrests$highMurderState, cutoff = 200,
  UH = 1, UM = 0, UCR = .9, UFA = 0)
#>   cutoff TP TN FP FN   SR   BR percentAccuracy percentAccuracyByChance
#> 1    200 15 30  4  1 0.38 0.32              90                   54.32
#>   percentAccuracyPredictingFromBaseRate      RIOC
#> 1                                    68 0.8991935
#>   relativeImprovementOverPredictingFromBaseRate     SN        SP TPrate
#> 1                                       0.34375 0.9375 0.8823529 0.9375
#>      TNrate FNrate    FPrate     HR       FAR       PPV       NPV       FDR
#> 1 0.8823529 0.0625 0.1176471 0.9375 0.1176471 0.7894737 0.9677419 0.2105263
#>          FOR   youdenJ balancedAccuracy   f1Score       mcc diagnosticOddsRatio
#> 1 0.03225806 0.8198529        0.9099265 0.8571429 0.7879121               112.5
#>   positiveLikelihoodRatio negativeLikelihoodRatio dPrimeSDT   betaSDT
#> 1                 7.96875              0.07083333  2.720952 0.6234551
#>         cSDT      aSDT bSDT differenceBetweenPredictedAndObserved
#> 1 -0.1736446 0.9476103 0.85                                 207.8
#>   informationGain overallUtility
#> 1       0.4947688           0.84