Prediction

A common post-processing task in survey analysis is the “imputation” of opinions or behaviors to respondents who have not provided that information. Here I want to differentiate between “imputation” which I will restrict to an auxiliary task for the estimation of parameters of interest and “prediction” which I will use for all tasks that fill fill-in gaps of direct interest. The differentiation is slightly arbitrary but it is useful for our purposes.

Let’s start with prediction. In this case, we are going to take a study from the CIS and we are going to do something fairly straightforward. We will use all the information we can from the questionnaire in order to assign a vote choice to all the respondents in our data. For instance, we can take study 3242 from the CIS (the pre-electoral study for the April 2019 elections) and we can fill-in question P10 about vote intention. In this case, we are going to assign a party choice to all respondents, i.e., we are going to work under the assumption of full turnout. In addition, we are not going to edit the intended behavior of anyone.

We have two possible strategies ahead of us. One is to follow a theoretical model of voting behavior that can be used to make reasonable, theoretically-grounded predictions. The other is the one we will use here: we will use a flexible, data-driven approach to make predictions. In this case, the we have to be careful to not overfit the data.

Our main model will be a XGBoost, which is very common in the machine learning literature. We could use the caret package to test different approaches (never a bad idea!), but in this case we will go straight to the xgboost package.

library(caret)

Loading required package: lattice

Loading required package: ggplot2

library(xgboost)

First of all, we need to load and clean up the data a little bit.

CLEAN_DATA <- file.path(DATA_DIR, "cis-clean-data.RDS")
dta <- readRDS(CLEAN_DATA)

As we said above, we will start by reallocating those who are undecided and also those who have decided to not vote. These are our unknown observations.

dta$voteintention[dta$voteintention == "undecided" |
                  dta$voteintention == "wont"] <- NA
dta$voteintention <- as.factor(as.character(dta$voteintention))

unknown <- dta[is.na(dta$voteintention), ]
known <- dta[!is.na(dta$voteintention), ]

Our goal now is to make inferences about the unknown sample using whatever we can learn about the known one. The key concept here is that of bias-variance trade-off. We will start by splitting our known dataset into a training and a test set. In this case, we will set aside 20% of the observations as test.

training <- sample(1:nrow(known), nrow(known)*.8)
testing <- setdiff(1:nrow(known), training)
training <- known[training, ]
testing <- known[testing, ]

Now the idea is the following. We are going to take our training set and we are going to fit models of different complexity to it. In the case of XGBoost there are several hyperparameters that we can tune in order to select the one with the best performance. The strategy we will use here is the simplest one available: we will simply test different combinations of hyperparameters in a regular grid and we will see how each of those combinations work. For the sake of illustration, I am going to be using here a very small grid:

param_grid <- expand.grid(max.depth=c(1, 2, 3),
                          eta=c(.1)) 

In doing so, there is the risk that our models will learn too much about the training data (Why?). Thus, we will want to fit the model and to evaluate the model using different datasets. That’s where the xgb.cv function will help us. In this case, we will be using 5-fold cross-validation over the training set (How many cases are we using in each step?).

best <- 1e6
best_index <- NA
best_params <- NA

for (i in 1:nrow(param_grid)) {
    msg <- sprintf("Computation with (%s)",
                   paste(param_grid[i, ], collapse=","))
    cat(msg, "\n")
    mod_ <- xgb.cv(data=as.matrix(dv),
                  label=as.matrix(as.numeric(training$voteintention) - 1),
                  nfold=5,
                  params=param_grid[i, ],
                  nthread=4,
                  nrounds=500,
                  early_stopping_rounds=5,
                  objective="multi:softmax",
                  num_class=12)

    merror_mean_index_ <- mod_$best_iteration
    min_merror_ <- mod_$evaluation_log[merror_mean_index_]$test_merror_mean

    if (min_merror_ < best) {
        best <- min_merror_
        best_index <- merror_mean_index_
        best_param <- param_grid[i, ]
    }
}

Computation with (1,0.1) 
[1] train-merror:0.167117+0.008064  test-merror:0.172744+0.012918 
Multiple eval metrics are present. Will use test_merror for early stopping.
Will train until test_merror hasn't improved in 5 rounds.

[2] train-merror:0.149047+0.001238  test-merror:0.149120+0.006875 
[3] train-merror:0.148229+0.001384  test-merror:0.148265+0.007988 
[4] train-merror:0.147979+0.001379  test-merror:0.148265+0.007424 
[5] train-merror:0.146806+0.000668  test-merror:0.148265+0.008929 
[6] train-merror:0.146770+0.002784  test-merror:0.147980+0.007860 
[7] train-merror:0.145810+0.001770  test-merror:0.146272+0.007510 
[8] train-merror:0.146058+0.001928  test-merror:0.146699+0.007414 
[9] train-merror:0.141434+0.001964  test-merror:0.143142+0.007983 
[10]    train-merror:0.139300+0.003083  test-merror:0.142146+0.008121 
[11]    train-merror:0.134640+0.003550  test-merror:0.136597+0.009489 
[12]    train-merror:0.133324+0.002871  test-merror:0.136028+0.008723 
[13]    train-merror:0.130656+0.002331  test-merror:0.131189+0.010494 
[14]    train-merror:0.129055+0.002587  test-merror:0.129909+0.008606 
[15]    train-merror:0.127526+0.002662  test-merror:0.128344+0.008009 
[16]    train-merror:0.126530+0.001893  test-merror:0.127205+0.009089 
[17]    train-merror:0.125000+0.002293  test-merror:0.125640+0.008771 
[18]    train-merror:0.124751+0.002313  test-merror:0.125640+0.007755 
[19]    train-merror:0.124004+0.002099  test-merror:0.125355+0.007293 
[20]    train-merror:0.122546+0.002568  test-merror:0.124644+0.008648 
[21]    train-merror:0.121727+0.001827  test-merror:0.122795+0.008520 
[22]    train-merror:0.120838+0.001369  test-merror:0.122225+0.009147 
[23]    train-merror:0.120518+0.001757  test-merror:0.121941+0.008940 
[24]    train-merror:0.119593+0.002486  test-merror:0.120803+0.007126 
[25]    train-merror:0.118597+0.002441  test-merror:0.119807+0.007211 
[26]    train-merror:0.118277+0.002455  test-merror:0.119095+0.006807 
[27]    train-merror:0.118064+0.002591  test-merror:0.120233+0.006672 
[28]    train-merror:0.117708+0.002626  test-merror:0.119522+0.006366 
[29]    train-merror:0.117459+0.002215  test-merror:0.119095+0.006623 
[30]    train-merror:0.116712+0.002051  test-merror:0.118811+0.006241 
[31]    train-merror:0.116498+0.002085  test-merror:0.118384+0.006349 
[32]    train-merror:0.115645+0.002296  test-merror:0.117673+0.006337 
[33]    train-merror:0.115538+0.001804  test-merror:0.117815+0.006154 
[34]    train-merror:0.114755+0.001973  test-merror:0.117103+0.006068 
[35]    train-merror:0.114756+0.002331  test-merror:0.117673+0.006093 
[36]    train-merror:0.114186+0.002295  test-merror:0.117673+0.005708 
[37]    train-merror:0.113866+0.001654  test-merror:0.117104+0.005971 
[38]    train-merror:0.113297+0.001919  test-merror:0.116534+0.005903 
[39]    train-merror:0.112763+0.001254  test-merror:0.117103+0.005577 
[40]    train-merror:0.112052+0.001791  test-merror:0.116107+0.005544 
[41]    train-merror:0.111447+0.002052  test-merror:0.116391+0.005921 
[42]    train-merror:0.110842+0.002069  test-merror:0.115680+0.005886 
[43]    train-merror:0.110700+0.001976  test-merror:0.115537+0.005866 
[44]    train-merror:0.110238+0.001519  test-merror:0.115680+0.006362 
[45]    train-merror:0.109882+0.001853  test-merror:0.115395+0.005850 
[46]    train-merror:0.109740+0.001686  test-merror:0.115253+0.006589 
[47]    train-merror:0.109206+0.001724  test-merror:0.115538+0.006209 
[48]    train-merror:0.108779+0.001586  test-merror:0.114826+0.006322 
[49]    train-merror:0.108423+0.001719  test-merror:0.114399+0.006792 
[50]    train-merror:0.108103+0.001571  test-merror:0.113688+0.006340 
[51]    train-merror:0.107712+0.001279  test-merror:0.113404+0.006313 
[52]    train-merror:0.107427+0.001451  test-merror:0.112977+0.006133 
[53]    train-merror:0.107499+0.001435  test-merror:0.113404+0.006522 
[54]    train-merror:0.107107+0.001404  test-merror:0.112835+0.005970 
[55]    train-merror:0.106609+0.001377  test-merror:0.112124+0.006606 
[56]    train-merror:0.106680+0.001395  test-merror:0.112266+0.006824 
[57]    train-merror:0.106325+0.001355  test-merror:0.111697+0.006869 
[58]    train-merror:0.105898+0.001421  test-merror:0.111270+0.006425 
[59]    train-merror:0.105578+0.001455  test-merror:0.110416+0.005948 
[60]    train-merror:0.105222+0.001362  test-merror:0.109989+0.005942 
[61]    train-merror:0.104653+0.001412  test-merror:0.110131+0.006691 
[62]    train-merror:0.103835+0.001512  test-merror:0.108850+0.006434 
[63]    train-merror:0.104048+0.001202  test-merror:0.108850+0.006434 
[64]    train-merror:0.103977+0.001088  test-merror:0.108850+0.006095 
[65]    train-merror:0.103835+0.000999  test-merror:0.108423+0.006465 
[66]    train-merror:0.103194+0.001303  test-merror:0.107996+0.006089 
[67]    train-merror:0.102696+0.001076  test-merror:0.107854+0.005925 
[68]    train-merror:0.102768+0.000876  test-merror:0.108281+0.006412 
[69]    train-merror:0.102412+0.000998  test-merror:0.108139+0.006726 
[70]    train-merror:0.102021+0.000897  test-merror:0.107854+0.006926 
[71]    train-merror:0.101665+0.001073  test-merror:0.107142+0.006597 
[72]    train-merror:0.101558+0.001250  test-merror:0.107142+0.006398 
[73]    train-merror:0.101451+0.001209  test-merror:0.107284+0.006550 
[74]    train-merror:0.101131+0.001506  test-merror:0.106858+0.006857 
[75]    train-merror:0.101024+0.001640  test-merror:0.106431+0.006107 
[76]    train-merror:0.100811+0.001735  test-merror:0.105578+0.006127 
[77]    train-merror:0.100740+0.001638  test-merror:0.105862+0.006911 
[78]    train-merror:0.100242+0.001472  test-merror:0.105151+0.006781 
[79]    train-merror:0.100064+0.001510  test-merror:0.105293+0.006465 
[80]    train-merror:0.100064+0.001523  test-merror:0.105151+0.006642 
[81]    train-merror:0.099744+0.001600  test-merror:0.105435+0.006701 
[82]    train-merror:0.099779+0.001527  test-merror:0.105293+0.007009 
[83]    train-merror:0.099530+0.001581  test-merror:0.105293+0.006860 
[84]    train-merror:0.099424+0.001849  test-merror:0.105293+0.006980 
[85]    train-merror:0.099495+0.002071  test-merror:0.105008+0.006736 
[86]    train-merror:0.099281+0.002044  test-merror:0.104866+0.007100 
[87]    train-merror:0.099317+0.002478  test-merror:0.104581+0.007287 
[88]    train-merror:0.098961+0.002286  test-merror:0.105009+0.006873 
[89]    train-merror:0.098748+0.002248  test-merror:0.104439+0.006805 
[90]    train-merror:0.098641+0.002356  test-merror:0.104724+0.007095 
[91]    train-merror:0.098428+0.002246  test-merror:0.104297+0.007431 
[92]    train-merror:0.098285+0.002478  test-merror:0.104581+0.007144 
[93]    train-merror:0.098108+0.002303  test-merror:0.104154+0.007252 
[94]    train-merror:0.097788+0.002349  test-merror:0.104155+0.007057 
[95]    train-merror:0.097716+0.002250  test-merror:0.103727+0.007096 
[96]    train-merror:0.097467+0.002484  test-merror:0.104296+0.007471 
[97]    train-merror:0.097503+0.002396  test-merror:0.103870+0.007449 
[98]    train-merror:0.097539+0.002264  test-merror:0.103585+0.007417 
[99]    train-merror:0.097218+0.002362  test-merror:0.103727+0.007589 
[100]   train-merror:0.097325+0.002377  test-merror:0.103158+0.007231 
[101]   train-merror:0.097112+0.002252  test-merror:0.103301+0.007109 
[102]   train-merror:0.096863+0.002177  test-merror:0.103443+0.007072 
[103]   train-merror:0.096613+0.002139  test-merror:0.103159+0.007359 
[104]   train-merror:0.096578+0.002049  test-merror:0.103159+0.007498 
[105]   train-merror:0.096436+0.002166  test-merror:0.103016+0.007514 
[106]   train-merror:0.096293+0.002007  test-merror:0.102874+0.007677 
[107]   train-merror:0.096116+0.001863  test-merror:0.102732+0.007998 
[108]   train-merror:0.096080+0.001979  test-merror:0.102162+0.007975 
[109]   train-merror:0.095831+0.002064  test-merror:0.102732+0.007688 
[110]   train-merror:0.095511+0.002194  test-merror:0.102447+0.007979 
[111]   train-merror:0.095440+0.002161  test-merror:0.102163+0.007811 
[112]   train-merror:0.095440+0.002095  test-merror:0.102305+0.007979 
[113]   train-merror:0.095226+0.002003  test-merror:0.102305+0.007979 
Stopping. Best iteration:
[108]   train-merror:0.096080+0.001979  test-merror:0.102162+0.007975

Computation with (2,0.1) 
[1] train-merror:0.134284+0.002734  test-merror:0.140439+0.002010 
Multiple eval metrics are present. Will use test_merror for early stopping.
Will train until test_merror hasn't improved in 5 rounds.

[2] train-merror:0.129269+0.003594  test-merror:0.137023+0.003976 
[3] train-merror:0.127063+0.003038  test-merror:0.133041+0.003862 
[4] train-merror:0.125747+0.002409  test-merror:0.133040+0.003653 
[5] train-merror:0.123933+0.002829  test-merror:0.129339+0.002428 
[6] train-merror:0.121550+0.002980  test-merror:0.127348+0.001944 
[7] train-merror:0.120874+0.003168  test-merror:0.127205+0.002360 
[8] train-merror:0.119131+0.002574  test-merror:0.125924+0.001435 
[9] train-merror:0.118135+0.002344  test-merror:0.124645+0.002473 
[10]    train-merror:0.116819+0.002106  test-merror:0.124218+0.002437 
[11]    train-merror:0.115645+0.001308  test-merror:0.122369+0.002994 
[12]    train-merror:0.114222+0.001604  test-merror:0.121373+0.002378 
[13]    train-merror:0.113084+0.001431  test-merror:0.119096+0.002467 
[14]    train-merror:0.111767+0.001231  test-merror:0.117105+0.002970 
[15]    train-merror:0.111198+0.001418  test-merror:0.117247+0.002897 
[16]    train-merror:0.110522+0.001274  test-merror:0.116109+0.003062 
[17]    train-merror:0.109242+0.001588  test-merror:0.116251+0.002382 
[18]    train-merror:0.108246+0.001485  test-merror:0.115540+0.003235 
[19]    train-merror:0.107570+0.001780  test-merror:0.114686+0.003597 
[20]    train-merror:0.107001+0.002156  test-merror:0.113121+0.002656 
[21]    train-merror:0.106040+0.001834  test-merror:0.112979+0.003328 
[22]    train-merror:0.105649+0.001840  test-merror:0.111982+0.002860 
[23]    train-merror:0.104475+0.001914  test-merror:0.111840+0.002517 
[24]    train-merror:0.103408+0.001680  test-merror:0.110276+0.003993 
[25]    train-merror:0.102946+0.001788  test-merror:0.110561+0.004400 
[26]    train-merror:0.102092+0.001917  test-merror:0.108853+0.003909 
[27]    train-merror:0.100740+0.001801  test-merror:0.108853+0.003529 
[28]    train-merror:0.100135+0.002024  test-merror:0.108853+0.003642 
[29]    train-merror:0.099317+0.002259  test-merror:0.106861+0.004009 
[30]    train-merror:0.098464+0.002607  test-merror:0.106577+0.004826 
[31]    train-merror:0.097752+0.002795  test-merror:0.106435+0.004543 
[32]    train-merror:0.097610+0.002609  test-merror:0.105865+0.004157 
[33]    train-merror:0.097005+0.002730  test-merror:0.105438+0.004201 
[34]    train-merror:0.096614+0.002704  test-merror:0.105011+0.004422 
[35]    train-merror:0.095796+0.002517  test-merror:0.104301+0.005461 
[36]    train-merror:0.095582+0.002720  test-merror:0.104443+0.005677 
[37]    train-merror:0.095191+0.002616  test-merror:0.105154+0.005803 
[38]    train-merror:0.094587+0.002387  test-merror:0.104728+0.006029 
[39]    train-merror:0.094053+0.002489  test-merror:0.104016+0.005715 
[40]    train-merror:0.093554+0.002307  test-merror:0.103874+0.005797 
[41]    train-merror:0.093234+0.002270  test-merror:0.103447+0.005131 
[42]    train-merror:0.093021+0.002232  test-merror:0.103589+0.005210 
[43]    train-merror:0.092737+0.002244  test-merror:0.102736+0.005010 
[44]    train-merror:0.092381+0.002146  test-merror:0.103163+0.005636 
[45]    train-merror:0.091705+0.002314  test-merror:0.103020+0.005305 
[46]    train-merror:0.091421+0.002533  test-merror:0.103305+0.005660 
[47]    train-merror:0.091065+0.002538  test-merror:0.103021+0.006245 
[48]    train-merror:0.090318+0.002322  test-merror:0.103163+0.006619 
Stopping. Best iteration:
[43]    train-merror:0.092737+0.002244  test-merror:0.102736+0.005010

Computation with (3,0.1) 
[1] train-merror:0.113653+0.002873  test-merror:0.122657+0.006746 
Multiple eval metrics are present. Will use test_merror for early stopping.
Will train until test_merror hasn't improved in 5 rounds.

[2] train-merror:0.110380+0.001876  test-merror:0.122233+0.006802 
[3] train-merror:0.107926+0.003337  test-merror:0.119387+0.007224 
[4] train-merror:0.106147+0.003352  test-merror:0.117396+0.007623 
[5] train-merror:0.104831+0.002683  test-merror:0.117539+0.009204 
[6] train-merror:0.103195+0.002551  test-merror:0.115686+0.006939 
[7] train-merror:0.101914+0.001950  test-merror:0.114690+0.006737 
[8] train-merror:0.101238+0.001576  test-merror:0.113552+0.006563 
[9] train-merror:0.100313+0.001618  test-merror:0.112840+0.005875 
[10]    train-merror:0.099317+0.001420  test-merror:0.113123+0.005723 
[11]    train-merror:0.098037+0.001591  test-merror:0.113409+0.006318 
[12]    train-merror:0.097396+0.001937  test-merror:0.112698+0.006653 
[13]    train-merror:0.096827+0.001632  test-merror:0.112271+0.006681 
[14]    train-merror:0.095938+0.001810  test-merror:0.111844+0.006160 
[15]    train-merror:0.094870+0.001952  test-merror:0.110989+0.007585 
[16]    train-merror:0.094550+0.002255  test-merror:0.110563+0.007012 
[17]    train-merror:0.093483+0.002441  test-merror:0.109141+0.007765 
[18]    train-merror:0.092701+0.002175  test-merror:0.108999+0.007595 
[19]    train-merror:0.091669+0.002297  test-merror:0.108572+0.007368 
[20]    train-merror:0.090851+0.002512  test-merror:0.108288+0.007577 
[21]    train-merror:0.089428+0.002168  test-merror:0.107577+0.007025 
[22]    train-merror:0.088539+0.002121  test-merror:0.106864+0.005835 
[23]    train-merror:0.087792+0.001834  test-merror:0.106580+0.005866 
[24]    train-merror:0.087080+0.001226  test-merror:0.106011+0.005600 
[25]    train-merror:0.086369+0.001263  test-merror:0.105014+0.005163 
[26]    train-merror:0.085622+0.001354  test-merror:0.105156+0.005304 
[27]    train-merror:0.084911+0.001786  test-merror:0.104729+0.005522 
[28]    train-merror:0.084270+0.001563  test-merror:0.104445+0.006052 
[29]    train-merror:0.083665+0.001620  test-merror:0.103591+0.006312 
[30]    train-merror:0.083167+0.001667  test-merror:0.103307+0.006315 
[31]    train-merror:0.082420+0.001589  test-merror:0.102737+0.005812 
[32]    train-merror:0.081673+0.001842  test-merror:0.102168+0.005849 
[33]    train-merror:0.081567+0.001609  test-merror:0.102454+0.006729 
[34]    train-merror:0.080642+0.001573  test-merror:0.102169+0.005765 
[35]    train-merror:0.079930+0.001658  test-merror:0.102026+0.006090 
[36]    train-merror:0.079326+0.001662  test-merror:0.101458+0.007262 
[37]    train-merror:0.078899+0.001472  test-merror:0.101031+0.007001 
[38]    train-merror:0.078116+0.001390  test-merror:0.101600+0.007130 
[39]    train-merror:0.077440+0.001507  test-merror:0.100890+0.007661 
[40]    train-merror:0.076658+0.001423  test-merror:0.101174+0.007799 
[41]    train-merror:0.076373+0.001691  test-merror:0.101173+0.007751 
[42]    train-merror:0.075911+0.001744  test-merror:0.100604+0.007704 
[43]    train-merror:0.075377+0.001838  test-merror:0.100747+0.007600 
[44]    train-merror:0.075128+0.001752  test-merror:0.100319+0.007324 
[45]    train-merror:0.074665+0.001922  test-merror:0.100319+0.007378 
[46]    train-merror:0.074061+0.001898  test-merror:0.100461+0.007266 
[47]    train-merror:0.072994+0.001667  test-merror:0.099892+0.007087 
[48]    train-merror:0.072602+0.001564  test-merror:0.100318+0.006458 
[49]    train-merror:0.071749+0.001572  test-merror:0.100176+0.007213 
[50]    train-merror:0.070895+0.001671  test-merror:0.099750+0.007406 
[51]    train-merror:0.070362+0.001591  test-merror:0.099465+0.006961 
[52]    train-merror:0.069864+0.001428  test-merror:0.099465+0.007258 
[53]    train-merror:0.069508+0.001615  test-merror:0.099180+0.006812 
[54]    train-merror:0.068690+0.001645  test-merror:0.098754+0.007258 
[55]    train-merror:0.068227+0.001682  test-merror:0.099464+0.006508 
[56]    train-merror:0.067765+0.001779  test-merror:0.099750+0.006595 
[57]    train-merror:0.067196+0.001672  test-merror:0.099607+0.006649 
[58]    train-merror:0.066769+0.001538  test-merror:0.099323+0.006913 
[59]    train-merror:0.066342+0.001481  test-merror:0.099038+0.007349 
Stopping. Best iteration:
[54]    train-merror:0.068690+0.001645  test-merror:0.098754+0.007258

Now that we have our best performing combination of hyperparameters, we can now apply it to the full training set.

base <- xgboost(data=as.matrix(dv),
                label=as.matrix(as.numeric(training$voteintention) - 1),
                nthread=4,
                params=best_param,
                nrounds=best_index,
                objective="multi:softmax",
                num_class=12,
                verbose=FALSE)

From this model we could try to say something about performance and how well it works. However, that’s not a good idea. Instead we are going to evaluate the performance of the model using the data that we set aside.

The easiest way to make this evaluation is through the confusion matrix: a comparison between the predictions we made and the known responses. We can characterize the confusion matrix in different ways, and caret::confusionMatrix offers many of the common statistics.

cm <- confusionMatrix(data=factor(predict(base, as.matrix(dv)), labels=lev),
                      reference=testing$voteintention)
print(cm)

Confusion Matrix and Statistics

            Reference
Prediction   bildu ciudadanos erc  iu other pdecat pnv podemos  pp psoe
  bildu         30          0   0   0     0      0   0       0   0    0
  ciudadanos     0        224   0   0     3      0   0       1   5    2
  erc            0          0  60   0     3      3   0       0   0    1
  iu             0          0   0  26     1      0   0       5   0    0
  other          0          9   0   5   111      1   1       0   3    8
  pdecat         0          0   1   0     2     16   0       0   0    0
  pnv            0          0   0   0     1      0  25       1   0    1
  podemos        2          0   1   8     2      0   0     107   0    9
  pp             0          8   0   0     8      0   0       0 307    2
  psoe           0          1   6   6    11      1   1       9   1  576
  vox            0          3   0   0     3      0   0       1   3    1
            Reference
Prediction   vox
  bildu        0
  ciudadanos   4
  erc          0
  iu           0
  other        3
  pdecat       0
  pnv          0
  podemos      0
  pp           9
  psoe         1
  vox        115

Overall Statistics
                                        
               Accuracy : 0.908         
                 95% CI : (0.894, 0.921)
    No Information Rate : 0.341         
    P-Value [Acc > NIR] : <2e-16        
                                        
                  Kappa : 0.887         
                                        
 Mcnemar's Test P-Value : NA            

Statistics by Class:

                     Class: bildu Class: ciudadanos Class: erc Class: iu
Sensitivity                0.9375             0.914     0.8824    0.5778
Specificity                1.0000             0.990     0.9959    0.9965
Pos Pred Value             1.0000             0.937     0.8955    0.8125
Neg Pred Value             0.9988             0.986     0.9953    0.9890
Prevalence                 0.0182             0.139     0.0387    0.0256
Detection Rate             0.0171             0.127     0.0341    0.0148
Detection Prevalence       0.0171             0.136     0.0381    0.0182
Balanced Accuracy          0.9688             0.952     0.9391    0.7871
                     Class: other Class: pdecat Class: pnv Class: podemos
Sensitivity                0.7655        0.7619     0.9259         0.8629
Specificity                0.9814        0.9983     0.9983         0.9865
Pos Pred Value             0.7872        0.8421     0.8929         0.8295
Neg Pred Value             0.9790        0.9971     0.9988         0.9896
Prevalence                 0.0825        0.0119     0.0154         0.0705
Detection Rate             0.0631        0.0091     0.0142         0.0609
Detection Prevalence       0.0802        0.0108     0.0159         0.0734
Balanced Accuracy          0.8735        0.8801     0.9621         0.9247
                     Class: pp Class: psoe Class: vox
Sensitivity              0.962       0.960     0.8712
Specificity              0.981       0.968     0.9932
Pos Pred Value           0.919       0.940     0.9127
Neg Pred Value           0.992       0.979     0.9896
Prevalence               0.181       0.341     0.0751
Detection Rate           0.175       0.328     0.0654
Detection Prevalence     0.190       0.349     0.0717
Balanced Accuracy        0.972       0.964     0.9322