1

I have a Regression Model with Train MAPE as 6% and Test MAPE as 15%. This appears to me as a clear case of over fitting. But can I still use this model assuming 15% Error is not a bad number after-all. Is this there a flaw in this thinking?

1

I have a Regression Model with Train MAPE as 6% and Test MAPE as 15%. This appears to me as a clear case of over fitting. But can I still use this model assuming 15% Error is not a bad number after-all. Is this there a flaw in this thinking?

0

Yes, assuming you **haven't overfitted on the test set** (which may happen after extensive hyperparameter optimization), you can assume that your model has a MAPE of 15%.

However, if you limit the overfitting, the test performance would probably go down!

it is overfitting, best would be to use k-fold cross-validation to test how much it overfits and decide – Nikos M. – 2021-01-05T14:43:40.217

What is the Baseline i.e. if humans can simply guess with a 20% error, so that would not be a great model? You must not simply accept it i.e. do detail causal of overfitting. If Train/Test is split on Time, then this might become 25% with new data – 10xAI – 2021-01-05T15:52:28.083