Multicollinearity

-
Multicollinearity is defined as the linear relationship between two or more independent variables while performing regression analysis between a dependent variables and set of independent variables.

Multicollinearity presents a severe problem during regression modelling. Inclusion of independent variables having linear relationship with each other leads to parameter estimation with higher standard error. This in turn leads to inaccurate parameter estimation. Furthermore, due to inaccurate parameters regression model becomes unstable.

The unstable models performs badly on the validation and test samples. When model is unstable, its performance deteriorate very fast compared to stable model over the period, though model is scored on the data of sample of same population. In such a situation an analyst must investigate for the multicollinearity, before finalizing the model.

Next question is how to investigate and which variable should be kept if some variables are found to be linearly related with each other. This is done by performing linear regression analysis between each pair of independent variables. This gives us a metric called variance inflating factor (VIF). It is equal to 1/sqrt(1-Rsquare) where R is correlation coefficient.

This can be calculated in MS Excel, R, MATLAB, SAS as well as using many other statistical packages available in the market. I discuss here how to use proc reg to measure the multicollinearity among the independent variables.

proc reg data = Dev_samp_cust;
model spend = salary age education tax_paid travelling_spend wealth family_members / vif tol collin;run;
quit; 

The VIF option used with Collin results in Variance Inflation factor whereas TOL results in the tolerance (requested by the tol option) is the proportion of variance in a given predictor that is NOT explained by all of the other predictors, while the VIF (or Variance Inflation Factor) is simply 1 / tolerance.The VIF represents a factor by which the variance of the estimated coefficient is multiplied due to the multicollinearity in the model.Exclusion of correlated variables depends upon the objective of the model, domain knowledge and VIF.

 In banking domain I used to drop the variable if VIF is more than 1.2 but I have also seen analysts in other domains such as retail etc dropping the variable only if VIF is more than 10. At this stage knowledge of data and understanding of the business is very important as it helps in deciding the VIF cut off as well as selecting one variable among the linearly related variables.


Comments

Post a Comment

Popular posts from this blog

Solution for ERROR: Some character data was lost during transcoding in the dataset

-
When I search for the mentioned error I get the link solution provided administration oriented but developers do not have access.     http://support.sas.com/kb/52/716.html Please see my solution below proc options option=config; run; proc options group=languagecontrol; run; /* Show the encoding value for the problematic data set */ %let dsn=item_information_may16; %let dsid=%sysfunc(open(&dsn,i)); %put &dsn ENCODING is: %sysfunc(attrc(&dsid,encoding)); /*Renaming item desc file  (encoding=any) allowed reading****************************/ data tmp.item_info_curr; set hc.item_information_may16 (encoding=any); run; /*gave error*/ data item_info_curr; set item_information_may16; run;
Read more

How to check whether a SAS dataset exist or not and throw an error in the log ?

-
/* if the data set exists, then conditionally execute the step(s). %sysfunc checks the availability of the data in the library location. It means it checks historical sas data. It generates the system error if the not is data available at the defined library  */ %let dsname=&dataval.MKT_DATA;  /**** declaring data set name with library*****/ %macro warning1(name); %if %sysfunc(exist(&name)) %then %do; Proc printto log="&logfile.\WarningLog..txt" new;     /*** printing log in the text file at specifile location &logfile****/ run; %Put WARNING :- &name does not exists; Proc printto  ; Run; %end; %else %do; Proc printto log="&logfile.\WarningLog.txt" ; run; %put &name exists; Proc printto  ; Run; %end; %mend ; %warning1(&dsname); /**** Pros:  If any sas process is dependent on the existence of the data then we can use this macro***/ /**** Cons:  *Even if the SAS data set is available with zero observatio
Read more

2018 plan for getting expertise in Machine Learning and Deep Learning

-
Machine Learning and deep learning are next frontier in the world of innovation. These skill sets are high in demand and demand is going to increase further as we are moving towards a world with connected systems. Although I am experienced professional with SAS having multiple SAS certifications and  using SAS for programming, predictive model building, optimization and data visualization for more than seven years, I will be foolish I do not recognize how important it is to adopt open source platforms for innovation.  2018 plan to acquire expertise in these are are 1. Choose a programming language and I have chose python as my language of preference  2. Do hands on practice to be efficient in Pandas and Numpy. These two libraries are useful and very important for data exploration,wrangling and data cleaning. I find Kernals in Kaggle and below blogs and books to be helpful tutorial for  Pandas  a. https://www.dataquest.io/blog/pandas-tutorial-python-1/  b.https://www.d
Read more