Выбор автоматического выключателя — правила выбора автоматического выключателя по мощности

Правила выбора автоматических выключателей

Автоматические выключатели предназначены для защиты электропроводки от перегрузок и короткого замыкания. Ошибочно полагать, что при выборе электроприбора нужно руководствоваться показателями нагрузки на сеть. Автомат защищает именно кабели и провода, а не подключенную бытовую технику.

При повышении нагрузки на электрическую сеть возрастает сила тока, из-за которой начинают греться провода, и происходит оплавление изоляции. В этот момент срабатывает автоматический выключатель. Ток перестает поступать на данный участок цепи, т.к. электроприбор ее размыкает. Автоматические выключатели ставят на вводе.

Типы автоматов

Типы автоматических выключателей различают по расцепителям. Расцепитель – это конструктивный элемент автомата, на который возложена основная функция по разрыву электросети в случае увеличения напряжения.

• Электромагнитные расцепители – моментальное реагирование и срабатывание автомата. Принцип работы: при увеличении силы тока сердечник в сотые доли секунды втягивается, тем самым напрягая пружину, которая заставляет срабатывать расцепители
• Тепловые биметаллические расцепители – разрыв сети происходит, только если нарушаются предельные значения параметров кабеля. Принцип действия заключается в изгибе пластины при ее нагреве. Она толкает рычаг в автомате, и он отключается
• Полупроводниковые расцепители – используют на сети переменного/постоянного тока на вводе. Работу по разрыву линии осуществляет блок реле трансформатора

Характеристики чувствительности к перегрузкам

Для начала нужно обратить внимание на основные характеристики срабатывания:

• Характеристика А – для электропроводки с особо чувствительным оборудованием. Расчет на мгновенную реакцию автомата на перегрузку
• Характеристика В – для защиты электропроводки (розетки и освещение) от нагрузки в жилых домах. Небольшая задержка в срабатывании автомата при увеличении силы тока в 3-5 раз от номинального значения
• Характеристика С – для защиты электропроводки от нагрузки в жилых домах и для сетей с большим пусковым током. Наиболее распространенная характеристика. Автомат не реагирует на небольшие скачки напряжения, а срабатывает только при серьезных перегрузках – увеличении силы тока в 5-10 раз от номинального значения
• Характеристика D – для защиты электропроводки от нагрузки с большим пусковым током. Устанавливают на вводе для контроля электрической сети всего здания. Отключает сеть при увеличении тока в 10-50 раз от номинального значения

Выбор автомата по количеству полюсов

В зависимости от цели применения автомата выбирают количество полюсов автомата:

• Однополюсный – для защиты освещения и розеток
• Двухполюсный – для защиты мощной бытовой техники (стиральная машина, электрическая плита и т.д.)
• Трехполюсный – для защиты генераторов, скважинных насосов и т.д.
• Четырехполюсный – для защиты четырехпроводной сети

Выбор автомата по мощности

Выбор автоматического выключателя осуществляется по номинальному току. Для его расчета нужно использовать общепринятую формулу:

I = P / U

Где: I – это величина тока

P – мощность всех электроприборов в Вт

U – напряжение в сети в В (обычно 220В)

Чтобы рассчитать мощность электроприборов, показатель кВт нужно перевести в Вт.

Помимо выбора автоматического выключателя по мощности необходимо учитывать расчет максимального рабочего тока. Номинальный ток должен быть больше или равен максимальному. Для расчета нужно суммировать мощность всех приборов и разделить ее на напряжение в сети, умноженное на понижающий коэффициент.

В зависимости от типа проводки расчет предельных значений:

• Для алюминиевых проводов – до 6А на 1 квадратный миллиметр
• Для медных проводов – до 10А на 1 квадратный миллиметр

При установке автоматического выключателя нужно еще учитывать и повышающие коэффициенты. Они рассчитываются от количества потребителей электроэнергии:

• Количество потребителей 2 -0,8
• Количество потребителей 3 – 0,75
• Больше 5 потребителей – 0,7

Помимо повышающих, для расчета используют и понижающие коэффициенты: отличие суммарной и потребляемой мощности. Значение 1 – для одновременного подключения нескольких бытовых приборов и 0,75 – если бытовые приборы есть, но из-за отсутствия розеток одновременно их включить нельзя.

После расчета нужно сверить по таблице максимально допустимое значение тока для проводника:

Сечение жилы, мм2	Для меди	Для алюминия
0,75	11	8
1	15	11
1,5	17	13
2,5	25	19
4	35	28
6	42	32
10	60	47
16	80	60

 

Основные правила выбора автоматов

Есть ряд рекомендаций, которые помогут сделать выбор автоматического выключателя.

• Покупать автомат нужно в специализированных магазинах
• При выборе производителя отдавать предпочтение наиболее известному и надежному
• Нельзя приобретать автоматы с поврежденным корпусом
• Выбор автомата должен соответствовать параметрам электропроводки после расчета мощности
• Для старой электропроводки, в которой были использованы алюминиевые провода, можно использовать автомат не больше 16А, либо два по 16А при наличии двух отходящих проводов. Включать одновременно несколько видов бытовой техники нельзя

90000 Feature Selection Techniques in Machine Learning with Python | by Raheel Shaikh 90001 90002 90003 With the new day comes new strength and new thoughts 90004 — 90005 Eleanor Roosevelt 90006 90007 90002 We all may have faced this problem of identifying the related features from a set of data and removing the irrelevant or less important features with do not contribute much to our target variable in order to achieve better accuracy for our model. 90007 90002 90005 Feature Selection is one of the core concepts in machine learning which hugely impacts the performance of your model.90006 The data features that you use to train your machine learning models have a huge influence on the performance you can achieve. 90007 90002 Irrelevant or partially relevant features can negatively impact model performance. 90007 90002 Feature selection and Data cleaning should be the first and most important step of your model designing. 90007 90002 In this post, you will discover feature selection techniques that you can use in Machine Learning. 90007 90002 Feature Selection is the process where you automatically or manually select those features which contribute most to your prediction variable or output in which you are interested in.90007 90002 Having irrelevant features in your data can decrease the accuracy of the models and make your model learn based on irrelevant features. 90007 90002 90005 How to select features and what are Benefits of performing feature selection before modeling your data? 90006 90007 90002 · 90005 Reduces Overfitting 90006: Less redundant data means less opportunity to make decisions based on noise. 90007 90002 · 90005 Improves Accuracy 90006: Less misleading data means modeling accuracy improves.90007 90002 · 90005 Reduces Training Time 90006: fewer data points reduce algorithm complexity and algorithms train faster. 90007 90002 90005 I want to share my personal experience with this. 90006 90007 90002 I prepared a model by selecting all the features and I got an accuracy of around 65% which is not pretty good for a predictive model and after doing some feature selection and feature engineering without doing any logical changes in my model code my accuracy jumped to 81% which is quite impressive 90007 90002 Now you know why I say feature selection should be the first and most important step of your model design.90007 90002 90005 Feature Selection Methods: 90006 90007 90002 I will share 3 Feature selection techniques that are easy to use and also gives good results. 90007 90002 1. Univariate Selection 90007 90002 2. Feature Importance 90007 90002 3.Correlation Matrix with Heatmap 90007 90002 Let’s have a look at these techniques one by one with an example 90007 90002 90005 You can download the dataset from here 90006 90005 https: / /www.kaggle.com/iabhishekofficial/mobile-price-classification#train.csv 90006 90007 90002 90005 Description of variables in the above file 90006 90007 90002 battery_power: Total energy a battery can store in one time measured in mAh 90007 90002 blue: Has Bluetooth or not 90007 90002 clock_speed: the speed at which microprocessor executes instructions 90007 90002 dual_sim: Has dual sim support or not 90007 90002 fc: Front Camera megapixels 90007 90002 four_g: Has 4G or not 90007 90002 int_memory: Internal Memory in Gigabytes 90007 90002 m_dep: Mobile Depth in cm 90007 90002 mobile_wt: Weight of mobile phone 90007 90002 n_cores : Number of cores of the processor 90007 90002 pc: Primary Camera megapixels 90007 90002 px_height 90007 90002 Pixel Resolution Height 90007 90002 px_width: Pixel Resolution Width 90007 90002 ram: Random Access Memory in MegaBytes 90007 90002 sc_h: Screen Height of mobile in cm 90007 90002 sc_w: Screen Width of mobile in cm 90007 90002 talk_time: the longest time that a single battery charge will last when you are 90007 90002 three_g: Has 3G or not 90007 90002 touch_screen: Has touch screen or not 90007 90002 wifi: Has wifi or not 90007 90002 price_range: This is the target variable with a value of 0 (low cost), 1 (medium cost) , 2 (high cost) and 3 (very high cost).90007 90002 90005 1. Univariate Selection 90006 90007 90002 Statistical tests can be used to select those features that have the strongest relationship with the output variable. 90007 90002 The scikit-learn library provides the SelectKBest class that can be used with a suite of different statistical tests to select a specific number of features. 90007 90002 The example below uses the chi-squared (chi²) statistical test for non-negative features to select 10 of the best features from the Mobile Price Range Prediction Dataset.90007 90126 import pandas as pd 90127 import numpy as np 90127 from sklearn.feature_selection import SelectKBest 90127 from sklearn.feature_selection import chi2data = pd.read_csv ( «D: //Blogs//train.csv») 90127 X = data.iloc [ : 0: 20] #independent columns 90127 y = data.iloc [:, — 1] #target column ie price range # apply SelectKBest class to extract top 10 best features 90127 bestfeatures = SelectKBest (score_func = chi2, k = 10) 90127 fit = bestfeatures.fit (X, y) 90127 dfscores = pd.DataFrame (fit.scores_) 90127 dfcolumns = pd.DataFrame (X.columns) 90127 #concat two dataframes for better visualization 90127 featureScores = pd.concat ([dfcolumns, dfscores], axis = 1) 90127 featureScores.columns = [ ‘Specs’, ‘Score’] #naming the dataframe columns 90127 print (featureScores.nlargest (10, ‘Score’)) #print 10 best features 90140 Top 10 Best Features using SelectKBest class 90141 2. Feature Importance 90142 90002 You can get the feature importance of each feature of your dataset by using the feature importance property of the model.90007 90002 Feature importance gives you a score for each feature of your data, the higher the score more important or relevant is the feature towards your output variable. 90007 90002 Feature importance is an inbuilt class that comes with Tree Based Classifiers, we will be using Extra Tree Classifier for extracting the top 10 features for the dataset. 90007 90126 import pandas as pd 90127 import numpy as np 90127 data = pd.read_csv ( «D: //Blogs//train.csv») 90127 X = data.iloc [:, 0:20] #independent columns 90127 y = data.iloc [:, — 1] #target column ie price range 90127 from sklearn.ensemble import ExtraTreesClassifier 90127 import matplotlib.pyplot as plt 90127 model = ExtraTreesClassifier () 90127 model.fit (X, y) 90127 print (model.feature_importances_) # use inbuilt class feature_importances of tree based classifiers 90127 #plot graph of feature importances for better visualization 90127 feat_importances = pd.Series (model.feature_importances_, index = X.columns) 90127 feat_importances.nlargest (10) .plot (kind = ‘barh’ ) 90127 plt.show () 90140 top 10 most important features in data 90141 90005 3.Correlation Matrix with Heatmap 90006 90142 90002 Correlation states how the features are related to each other or the target variable. 90007 90002 Correlation can be positive (increase in one value of feature increases the value of the target variable) or negative (increase in one value of feature decreases the value of the target variable) 90007 90002 Heatmap makes it easy to identify which features are most related to the target variable, we will plot heatmap of correlated features using the seaborn library.90007 90126 import pandas as pd 90127 import numpy as np 90127 import seaborn as snsdata = pd.read_csv ( «D: //Blogs//train.csv») 90127 X = data.iloc [:, 0:20] #independent columns 90127 y = data.iloc [:, — 1] #target column ie price range 90127 #get correlations of each features in dataset 90127 corrmat = data.corr () 90127 top_corr_features = corrmat.index 90127 plt.figure (figsize = (20 , 20)) 90127 #plot heat map 90127 g = sns.heatmap (data [top_corr_features] .corr (), annot = True, cmap = «RdYlGn») 90140.90000 What Are Feature Selection Techniques In Machine Learning? 90001 90002 Feature selection is the method of reducing data dimension while doing predictive analysis. One major reason is that machine learning follows the rule of «garbage in-garbage out» and that is why one needs to be very concerned about the data that is being fed to the model. 90003 90002 In this article, we will discuss various kinds of feature selection techniques in machine learning and why they play an important role in machine learning tasks.90003 90006 Filter Method 90007 90002 90009 90003 90011 90012 90011 90002 This method uses the variable ranking technique in order to select the variables for ordering and here, the selection of features is independent of the classifiers used. By ranking, it means how much useful and important each feature is expected to be for classification. It basically selects the subsets of variables as a pre-processing step independently of the chosen predictor. In filtering, the ranking method can be applied before classification for filtering the less relevant features.It carries out the feature selection task as a pre-processing step which contains no induction algorithm. 90003 90002 Some examples of filter methods are mentioned below: 90003 90011 90019 90011 90021 90022 Chi-Square Test: In general term, this method is used to test the independence of two events. If a dataset is given for two events, we can get the observed count and the expected count and this test measures how much both the counts are derivate from each other. 90023 90022 Variance Threshold: This approach of feature selection removes all features whose variance does not meet some threshold.Generally, it removes all the zero-variance features which means all the features that have the same value in all samples. 90023 90022 Information Gain: Information gain or IG measures how much information a feature gives about the class. Thus, we can determine which attribute in a given set of training feature is the most meaningful for discriminating between the classes to be learned. 90023 90028 90006 Wrapper Method 90007 90002 90032 90003 90034 90035 Fig: Wrapper Approach to feature subset selection 90036 90037 90002 The Wrapper Methodology was made famous by researchers Ron Kohavi and George H.John in the year 1997. This method utilises the learning machine of interest as a black box to score subsets of variables according to their predictive power. In the above figure, in a supervised machine learning, the induction algorithm is depicted with a set of training instances, where each instance is described by a vector of feature values ​​and a class label. The induction algorithm which is also considered as the black box is used to induce a classifier which is useful in classifying. In the wrapper approach, the feature subset selection algorithm exists as a wrapper around the induction algorithm.One of the main drawbacks of this technique is the mass of computations required to obtain the feature subset. 90003 90002 Some examples of Wrapper Methods are mentioned below: 90003 90021 90022 Genetic Algorithms: This algorithm can be used to find a subset of features. CHCGA is the modified version of this algorithm which converges faster and renders a more effective search by maintaining the diversity and evade the stagnation of the population. 90023 90022 Recursive Feature Elimination: RFE is a feature selection method which fits a model and removes the weakest feature until the specified number of features is satisfied.Here, the features are ranked by the model’s coefficient or feature importances attributes. 90023 90022 Sequential Feature Selection: This naive algorithm starts with a null set and then add one feature to the first step which depicts the highest value for the objective function and from the second step onwards the remaining features are added individually to the current subset and thus the new subset is evaluated. This process is repeated until the required number of features are added. 90023 90028 90006 Embedded Method 90007 90002 This method tries to combine the efficiency of both the previous methods and performs the selection of variables in the process of training and is usually specific to given learning machines.This method basically learns which feature provides the utmost to the accuracy of the model. 90003 90002 Some examples of Embedded Methods are mentioned below: 90003 90021 90022 L1 Regularisation Technique such as LASSO: Least Absolute Shrinkage and Selection Operator (LASSO) is a linear model which estimates sparse coefficients and is useful in some contexts due to its tendency to prefer solutions with fewer parameter values. 90023 90022 Ridge Regression (L2 Regularisation): The L2 Regularisation is also known as Ridge Regression or Tikhonov Regularisation which solves a regression model where the loss function is the linear least squares function and regularisation.90023 90022 Elastic Net: This linear regression model is trained with L1 and L2 as regulariser which allows for learning a sparse model where few of the weights are non-zero like Lasso and on the other hand maintaining the regularisation properties of Ridge. 90023 90028 90006 Importance 90007 90002 The feature selection techniques simplify the machine learning models in order to make it easier to interpret by the researchers. IT mainly eliminates the effects of the curse of dimensionality.Besides, this technique reduces the problem of overfitting by enhancing the generalisation in the model. Thus it helps in better understanding of data, improves prediction performance, reducing the computational time as well as space which is required to run the algorithm. 90003 90006 Provide your comments below 90007 90002 comments 90003 90072 90073 Ambika Choudhury 90002 A Technical Journalist who loves writing about Machine Learning and Artificial Intelligence.A lover of music, writing and learning something out of the box. Contact: [email protected] 90003 .90000 A Feature Selection Tool for Machine Learning in Python | by Will Koehrsen 90001 90002 On the left we have the 90003 plot_n 90004 most important features (plotted in terms of normalized importance where the total sums to 1). On the right we have the cumulative importance versus the number of features. The vertical line is drawn at 90003 threshold 90004 of the cumulative importance, in this case 99%. 90007 90002 Two notes are good to remember for the importance-based methods: 90007 90010 90011 Training the gradient boosting machine is stochastic meaning the 90012 feature importances will change every time the model is run 90013 90014 90015 90002 This should not have a major impact ( the most important features will not suddenly become the least) but it will change the ordering of some of the features.It also can affect the number of zero importance features identified. Do not be surprised if the feature importances change every time! 90007 90010 90011 To train the machine learning model, the features are first 90012 one-hot encoded 90013. This means some of the features identified as having 0 importance might be one-hot encoded features added during modeling. 90014 90015 90002 When we get to the feature removal stage, there is an option to remove any added one-hot encoded features. However, if we are doing machine learning after feature selection, we will have to one-hot encode the features anyway! 90007 90002 The next method builds on zero importance function, using the feature importances from the model for further selection.The function 90003 identify_low_importance 90004 finds the lowest importance features that do not contribute to a specified total importance. 90007 90002 For example, the call below finds the least important features that are not required for achieving 99% of the total importance: 90007 90032 fs.identify_low_importance (cumulative_importance = 0.99) 90033 123 features required for cumulative importance of 0.99 after one hot encoding. 90034 116 features do not contribute to cumulative importance of 0.99. 90035 90036 90002 Based on the plot of cumulative importance and this information, the gradient boosting machine considers many of the features to be irrelevant for learning. Again, the results of this method will change on each training run. 90007 90002 To view all the feature importances in a dataframe: 90007 90032 fs.feature_importances.head (10) 90036 90002 The 90003 low_importance 90004 method borrows from one of the methods of using Principal Components Analysis (PCA) where it is common to keep only the PC needed to retain a certain percentage of the variance (such as 95%).The percentage of total importance accounted for is based on the same idea. 90007 90002 The feature importance based methods are really only applicable if we are going to use a tree-based model for making predictions. Besides being stochastic, the importance-based methods are a black-box approach in that we do not really know why the model considers the features to be irrelevant. If using these methods, run them several times to see how the results change, and perhaps create multiple datasets with different parameters to test! 90007 90002 The final method is fairly basic: find any columns that have a single unique value.A feature with only one unique value can not be useful for machine learning because this feature has zero variance. For example, a tree-based model can never make a split on a feature with only one value (since there are no groups to divide the observations into). 90007 90002 There are no parameters here to select, unlike the other methods: 90007 90032 fs.identify_single_unique () 90033 4 features with a single unique value. 90035 90036 90002 We can plot a histogram of the number of unique values ​​in each category: 90007 90032 fs.plot_unique () 90036 90002 One point to remember is 90003 NaNs 90004 are dropped before calculating unique values ​​in Pandas by default. 90007 90002 Once we’ve identified the features to discard, we have two options for removing them. All of the features to remove are stored in the 90003 ops 90004 dict of the 90003 FeatureSelector 90004 and we can use the lists to remove features manually. Another option is to use the 90003 remove 90004 built-in function. 90007 90002 For this method, we pass in the 90003 methods 90004 to use to remove features.If we want to use all the methods implemented, we just pass in 90003 methods = ‘all’ 90004. 90007 90032 # Remove the features from all methods (returns a df) 90034 train_removed = fs.remove (methods = ‘all’) 90033 [ ‘missing’, ‘single_unique’, ‘collinear’, ‘zero_importance’, ‘low_importance’] methods have been run 90002 Removed 140 features. 90007 90035 90036 90002 This method returns a dataframe with the features removed. To also remove the one-hot encoded features that are created during machine learning: 90007 90032 train_removed_all = fs.remove (methods = ‘all’, keep_one_hot = False) 90033 Removed 187 features including one-hot features. 90035 90036 90002 It might be a good idea to check the features that will be removed before going ahead with the operation! The original dataset is stored in the 90003 data 90004 attribute of the 90003 FeatureSelector 90004 as a back-up! 90007 90002 Rather than using the methods individually, we can use all of them with 90003 identify_all 90004. This takes a dictionary of the parameters for each method: 90007 90032 fs.identify_all (selection_params = { ‘missing_threshold’: 0.6, 90034 ‘correlation_threshold’: 0.98, 90034 ‘task’: ‘classification’, 90034 ‘eval_metric’: ‘auc’, 90034 ‘cumulative_importance’: 0.99}) 90033 151 total features out of 255 identified for removal after one-hot encoding. 90035 90036 90002 Notice that the number of total features will change because we re-ran the model. The 90003 remove 90004 function can then be called to discard these features. 90007 90002 The Feature Selector class implements several common operations for removing features before training a machine learning model.It offers functions for identifying features for removal as well as visualizations. Methods can be run individually or all at once for efficient workflows. 90007 90002 The 90003 missing 90004, 90003 collinear 90004, and 90003 single_unique 90004 methods are deterministic while the feature importance-based methods will change with each run. Feature selection, much like the field of machine learning, is largely empirical and requires testing multiple combinations to find the optimal answer.It’s best practice to try several configurations in a pipeline, and the Feature Selector offers a way to rapidly evaluate parameters for feature selection. 90007 90002 As always, I welcome feedback and constructive criticism. I want to emphasis that I’m looking for help on the 90003 FeatureSelector 90004. Anyone can contribute on GitHub and I appreciate advice from those who just uses the tool! I can also be reached on Twitter @koehrsen_will. 90007.90000 A «short» introduction to model selection | by David Schönleber 90001 90002 An overview over hyperparameter selection & algorithm selection with big and small data 90003 90004 90005 In this post I will discuss a topic central to the process of building good (supervised) machine learning models: model selection. This is not to say that model selection is the centerpiece of the data science workflow — without high-quality data, model building is vanity. Nevertheless, model selection plays a crucial role in building good machine learning models.90006 90002 Model selection at different scales 90003 90005 So, what is model selection all about? Model selection in the context of machine learning can have different meanings, corresponding to different levels of abstraction. 90006 90005 For one thing, we might be interested in selecting the best 90012 90013 hyperparameters 90014 90015 for a selected machine learning method. Hyperparameters are the parameters of the learning method itself which we have to specify a priori, i.e., before model fitting.In contrast, 90012 90013 model parameters 90014 90015 are parameters which arise as a result of the fit [1]. In a logistic regression model, for example, the regularization strength (as well as the regularization type, if any) is a hyperparameter which has to be specified prior to the fitting, while the coefficients of the fitted model are model parameters. Finding the right hyperparameters for a model can be crucial for the model performance on given data. 90006 90005 For another thing, we might want to select the best 90012 90013 learning method 90014 90015 (and their corresponding «optimal» hyperparameters) from a set of eligible machine learning methods.In the following, we will refer to this as 90012 90013 algorithm selection 90014 90015. With a classification problem at hand, we might wonder, for instance, whether a logistic regression model or a random forest classifier yields the best classification performance on the given task. 90006 90002 The «One More Thing»: model evaluation 90003 90005 Before diving into the details of different approaches to model selection, and when to use them, there is «one more thing» we need to discuss: 90012 90013 model evaluation 90014 90015.Model evaluation aims at estimating the generalization error of the selected model, i.e., how well the selected model performs on unseen data. Obviously, a good machine learning model is a model that not only performs well on data seen during training (else a machine learning model could simply memorize the training data), but also on unseen data. Hence, before shipping a model to production we should be fairly certain that the model’s performance will not degrade when it is confronted with new data.90006 90005 But 90012 90013 why 90014 90015 do we need 90012 90013 the distinction between model selection and model evaluation? 90014 90015 The reason is overfitting. If we estimate the generalization error of our selected model on the same data which we have used to select our winning model, we will get an overly optimistic estimate. Why? 90048 Let’s do a thought experiment. Assume you are given a set of black-box classifiers with the instruction to select the best-performing classifier.All classifiers are useless — they output a (fixed) sequence of zeros and ones. You evaluate all classifiers on your data and find that they get, on average, 50% of the cases right. By chance, one classifier performs better than the others on the data, say, by a margin of 8%. You take this performance as an estimate of the performance of your classifier on unseen data (i.e., as an estimate of the generalization error). You report that you have found a classifier which does better than random guessing.If you had, however, used a completely independent test set for estimating the generalization error, you would have quickly discovered the fraud! To avoid such issues, we need completely independent data for estimating the generalization error of a model. We will come back to this point in the context of cross validation. 90006 90002 If data is not an issue 90003 90005 The recommended strategy for model selection depends on the amount of data available. If 90012 90013 plenty of data 90014 90015 is available, we may split the data into several parts, each serving a special purpose.For instance, for 90012 90013 hyperparameter tuning 90014 90015 we may split the data into three sets: 90012 90013 train / validation / test 90014 90015. The training set is used to train as many models as there are different combinations of model hyperparameters. These models are then evaluated on the validation set, and the model with the best performance on this validation set is selected as the winning model. Subsequently, the model is retrained on training + validation data using the chosen set of hyperparameters and the generalization performance is estimated using the test set.If this generalization error is similar to the validation error, we have reason to believe that the model will perform well on unseen data. Lastly, we retrain the model on the full data (train, validation & test set) before using it in «production». 90048 Since not all data is created equal, there is no general rule as to how the data should be split. A typical split is e.g. 50% / 25% / 25%. In any case, the validation set should be big enough to measure the difference in performance we want to be able to measure: If we care about a 0.1% difference between models, our validation set must not be smaller than 1000 samples, but 10000 samples would suffice. 90006 90005 For 90012 90013 algorithm selection 90014 90015, following the above reasoning, we can use 90012 90013 several train / validation / test sets 90014 90015, one triple per algorithm. Since this method is quite data-demanding, we will discuss an alternative method below. 90006 90002 Learning curves, and why they are useful 90003 90005 But why should we retrain the model after model selection / model evaluation? The answer is best illustrated using 90012 90013 learning curves 90014 90015.In a learning curve, the performance of a model both on the training and validation set is plotted as a function of the training set size. 90012 90013 Fig. 1 90014 90015 shows a typical learning curve: The training score (performance on the training set) decreases with increasing training set size while the validation score increases at the same time. High training score and low validation score at the same time indicates that the model has overfit the data, i.e., has adapted too well to the specific training set samples.As the training set increases, overfitting decreases, and the validation score increases. 90048 Especially for data-hungry machine learning models, the learning curve might not yet have reached a plateau at the given training set size, which means the generalization error might still decrease when providing more data to the model. Hence, it seems reasonable to increase the training set (by adding the validation set) before estimating the generalization error on the test set, and to further take advantage of the test set data for model fitting before shipping the model.Whether or not this strategy is needed depends strongly on the slope of the learning curve at the initial training set size. 90006 Fig. 1: Example of a learning curve. 90005 Learning curves further allow to easily illustrate the concept of (statistical) 90012 90013 bias 90014 90015 and 90012 90013 variance 90014 90015. Bias in this context refers to erroneous (e.g. simplifying) model assumptions, which can cause the model to underfit the data. A high-bias model does not adequately capture the structure present in the data.Variance on the other hand quantifies how much the model varies as we change the training data. A high-variance model is very sensitive to small fluctuations in the training data, which can cause the model to overfit. The amount of bias and variance can be estimated using learning curves: A model exhibits high variance, but low bias if the training score plateaus at a high level while the validation score at a low level, ie, if there is a large gap between training and validation score. A model with low variance but high bias, in contrast, is a model where both training and validation score are low, but similar.Very simple models are high-bias, low-variance while with increasing model complexity they become low-bias, high-variance. 90006 90005 The concept of model complexity can be used to create measures aiding in model selection. There are a few measures which explicitly deal with this trade-off between goodness of fit and model simplicity, for instance the 90012 90013 Akaike information criterion 90014 90015 (AIC) and the 90012 90013 Bayesian information criterion 90014 90015 (BIC). Both penalize the number of model parameters but reward goodness of fit on the training set, hence the best model is the one with lowest AIC / BIC.BIC penalizes model complexity stronger and hence favors models which are «more wrong» but simpler. While this allows to do model selection without a validation set, it can be strictly applied only for models which are linear in their parameters, even though it typically also works in more general cases, e.g. for general linear models such as logistic regression. For a more detailed discussion, see e.g. Ref. [2]. 90006 90002 Divide and conquer — but do it carefully 90003 90005 What we have implicitly assumed throughout the above discussion is that 90012 training, validation, and test set 90015 are 90012 90013 sampled from the same distribution 90014 90015.If this is not the case, all estimates will be plain wrong. This is why it is essential to ensure before model building that the distribution of the data is not affected by partitioning your data. Imagine, for example, that you are dealing with imbalanced data, e.g. a data set with a binary target which is positive only 10% of the cases. Randomly splitting the data into train / validation / test set according to a 50% / 25% / 25% split could e.g. lead to a distribution of 5% positive cases in training and 15% in validation & test set, which could seriously screw algorithm performance estimates.In such a case, you may want to use 90012 90013 stratified sampling 90014 90015 (potentially combined with over- respectively undersampling techniques if your learning method requires it) to make the partitions. 90006 90005 A final word of caution: when dealing with time series data where the task is to make forecasts, train, validation and test sets have to be selected by 90012 90013 splitting 90014 90015 the data 90012 90013 along the temporal axis 90014 90015. That is, the «oldest» data is used for training, the more recent one for validation, and the most recent one for testing.Random sampling does not make sense in this case. 90006 90002 If all you have is small data 90003 90005 But what if small data is all we have? How do we do model selection and evaluation in this case? Model evaluation does not change. We still need a test set on which we can estimate the generalization error of the final selected model. Hence, we split the data into two sets, a training and a test set. What changes compared to the previous procedure is the way we use the training set. 90006 90002 Have your cake and eat it too: cross-validation for hyperparameter selection 90003 90005 For 90012 90013 hyperparameter selection 90014 90015, we can use 90013 K 90014 -fold 90012 90013 cross-validation 90014 90015 (CV).Cross-validation works as follows: 90006 90152 90153 We split the training set into 90013 K 90014 smaller sets. Note that the caveats regarding imbalanced data also apply here. 90156 90153 We set aside each of the 90013 K 90014 folds one time, as illustrated in 90012 90013 Fig. 2 90014 90015. We train as many models as there are different combinations of model hyperparameters on the remaining 90013 K-1 90014 folds and compute the validation score on the hold-out fold. 90156 90153 For each set of hyperparameters we compute the mean validation score and select the hyperparameter set with best performance on the hold-out validation sets.Alternatively, we can apply the «one-standard-error-rule» [2], which means that we choose the most parsimonious model (the model with lowest complexity) whose performance is not more than a standard error below the best performing model. 90156 90169 90005 Subsequently, we train the model with the chosen hyperparameter set on the full training set and estimate the generalization error using the test set. Lastly, we retrain the model using the combined data of training and test set. 90006 Fig. 2: Illustration of 5-fold cross-validation.90005 How many splits should we make, i.e., how should we choose 90013 K 90014? Unfortunately, there is no free lunch, i.e., no single answer that always works best. If we choose 90013 K = N 90014 where 90013 N 90014 is the number of training examples, we are dealing with a method called 90012 90013 leave-one-out cross-validation 90014 90015 (LOOCV). The advantage here is that since we always use almost the entire data for training, the estimated prediction performance is approximately unbiased, meaning that the difference between expected value of the prediction error and «true» prediction error is very low.The disadvantage, however, is that LOOCV is computationally expensive and the variance can be high, meaning that our prediction performance estimate can fluctuate strongly around its «true» value. In contrast, if we choose 90013 K = 5 90014 or 90013 K = 10 90014, the variance of our prediction performance estimate is low, but our estimate might be overly pessimistic since we use only 80-90% of the available data for training ( cf. discussion of the learning curve above). Nevertheless, 10-fold (or 5-fold) CV is recommended as a rule of thumb [2].90006 90002 Inside the matryoshka: nested cross-validation for algorithm selection 90003 90005 For 90012 90013 algorithm selection 90014 90015 we need a more elaborate method. Here, 90012 90013 nested cross-validation 90014 90015 comes to the rescue, which is illustrated in 90012 90013 Fig. 3 90014 90015 and works as follows: 90006 90152 90153 We split the data into 90013 K 90014 smaller sets (outer fold). 90156 90153 Each of the 90013 K 90014 folds we set aside one time. For each learning method we then perform 90013 K ‘90014 -fold CV (following the procedure above) on the 90013 K-1 90014 remaining folds, in which we do we do hyperparameter selection.For brevity, one denotes nested CV with 90013 K 90014 outer folds and 90013 K ‘90014 inner folds as 90013 KxK’ 90014 nested CV. Typical values ​​for 90013 KxK ‘90014 are 90013 5×2 90014 or 90013 5×3 90014. 90156 90153 We use the best hyperparameter set for each algorithm to estimate its validation score on the hold-out fold. 90156 90153 Then we compute the mean validation score (as well as standard deviation) over the 90013 K 90014 folds and select the best performing algorithm 90048 Subsequently, we choose the best hyperparameter set based on CV using the full training set and estimate the generalization error using the test set.90156 90169 90005 Lastly, we retrain the model using the combined data of training and test set. 90006 Fig. 3: Illustration of 5×2 nested cross-validation. 90002 The intricate business of nested cross-validation 90003 90005 90012 90013 Why do we 90014 90015 do hyperparameter selection at all if we do 90012 90013 not use the «best» hyperparameters found in the inner loop 90014 90015 of the nested CV procedure? The reason is that in algorithm selection we are not really interested in finding the best algorithm & corresponding hyperparameter set for a specific data sample (our training set).We rather want an algorithm which generalizes well, and which does not fundamentally change if we use slightly different data for training [3]. We want a stable algorithm, since if our algorithm is not stable, generalization estimates are futile since we do not know what would happen if the algorithm had encountered a different data point in the training set. Hence, we are fine if the hyperparameters found in the inner loop are different, provided that the corresponding performance estimates on the hold-out sets are similar.If they are, it seems very likely that the different hyperparameters resulted in similar models, and that training the algorithm on the full training data will again produce a similar (though hopefully slightly improved) model. 90006 90005 90012 90013 What about preprocessing 90014 90015 such as feature selection? As a rule of thumb, supervised preprocessing (involving the data labels) should be done inside the (inner) CV loop [2]. In contrast, unsupervised preprocessing such as scaling can be done prior to cross-validation.If we ignore this advice, we might get an overly optimistic performance estimate, since setting aside data for validation purposes after relevant features have been selected on the basis of all training data introduces a dependence between training and validation folds. This contradicts the assumption of independence, which is implied when we use data for validation. By doing preprocessing such as feature selection in the inner CV loop, however, we might get a pessimistic performance estimate because the preprocessing procedure might improve when using more data.90006 90002 When prose is not enough 90003 90005 In case you wondered how this slightly cumbersome procedure of nested cross-validation can be implemented 90012 90013 using code 90014 90015, you can find an 90012 90013 example 90014 90015 (python jupyter notebook with scikit-learn ) here. 90006 90005 While by far not comprehensive, this post covers many important concepts and «best practices» concerning model selection. There is more to say about this topic, particularly about statistical tests for model selection and sampling-based methods (e.g. bootstrapping) for estimating the uncertainty of the performance estimates. If this post just got you started, please refer to Refs. [1,2] for further reading. 90006 90002 References 90003 90005 [1] Model evaluation, model selection, and algorithm selection in machine learning by Sebastian Raschka. 90006 90005 [2] Hastie T., Tibshirani R., and Friedman J., The Elements of Statistical Learning, New York, NY, USA: Springer New York Inc. (2008). 90006 90005 [3] See Stackexchange discussions here, here and here.90006.