Pfthb

I have several prediction models which are created using the same trainControl. These models have to be created beforehand (i.e. I can't use caretList to train multiple models simultaneously).

Below is my minimal example. When I manually combine multiple (already created) models and pass them to caretStack,

library("kernlab")
library("rpart")
library("caret")
library("caretEnsemble")

trainingControl <- trainControl(method='cv', number=10, savePredictions = "final", classProbs=TRUE)
data(spam)
ds <- spam
tr <- ds[sample(nrow(ds),3221),]
te <- ds[!(rownames(ds) %in% rownames(tr)),]
model <- train(tr[,-58], tr$type, 'svmRadial', trControl = trainingControl)
model2 <- train(tr[,-58], tr$type, 'rpart', trControl = trainingControl)
multimodel <- list(svm = model, nb = model2)
class(multimodel) <- "caretList"
stack <- caretStack(multimodel, method = "rf", metric = "ROC", trControl = trainingControl)

the library throws the error:

Component models do not have the same re-sampling strategies.

Why is that since I'm using the same strategy to generate the base models?

I found the "casting" to caretList class in the github discussion zachmayer/caretEnsemble/issues/104.

You are almost there. One of the things to remember is that when you want to use caretEnsemble is that in trainControl you have to set the resample index via the 'index' option in trainControl. If you run caretList it tends to set this itself, but it is better to do this yourself. This is especially true when you run different models outside of caretList. You need to make sure the resampling is the same. You can also see this in the example on github you refer to.

trainingControl <- trainControl(method='cv', 
 number=10, 
 savePredictions = "final", 
 classProbs=TRUE, 
 index=createResample(tr$type)) # this needs to be set.

This will make sure that your code will run.

Note that in the example code you have given, it will return with errors.