Big Data

-
IBM coined the word called Big Data characterized by the volume and velocity of the data being created in the Universe.  It has become a buzz word since then. Many reports have been published that Big data is next big thing and will generate business valued billions of dollars.

Basically in simple words on daily basis large volume of data are being generated by different activities of human being. I will also emphasis that not only activities of humans is generating tera bytes of data but activities of animals, non living things like planets, asteroids, stars etc. In fact non living things are producing volume of data since ancient times only recently we, human have been able to store and generate with advent of internet (one of the big thing of current era).

With arrival of Big data skills sets to manage it, convert it into usable formate is required in the world and  which is actually in great demand these days.

Big data has given opportunities learn many new things. It has challenged brightest human minds to manage and use it efficiently.  I find this term very interesting. I can find daily  a new article in web on it. Recently I read ten big data jobs to make millions of dollars. Ohh Great.


Comments

Post a Comment

Popular posts from this blog

Solution for ERROR: Some character data was lost during transcoding in the dataset

-
When I search for the mentioned error I get the link solution provided administration oriented but developers do not have access.     http://support.sas.com/kb/52/716.html Please see my solution below proc options option=config; run; proc options group=languagecontrol; run; /* Show the encoding value for the problematic data set */ %let dsn=item_information_may16; %let dsid=%sysfunc(open(&dsn,i)); %put &dsn ENCODING is: %sysfunc(attrc(&dsid,encoding)); /*Renaming item desc file  (encoding=any) allowed reading****************************/ data tmp.item_info_curr; set hc.item_information_may16 (encoding=any); run; /*gave error*/ data item_info_curr; set item_information_may16; run;
Read more

How to check whether a SAS dataset exist or not and throw an error in the log ?

-
/* if the data set exists, then conditionally execute the step(s). %sysfunc checks the availability of the data in the library location. It means it checks historical sas data. It generates the system error if the not is data available at the defined library  */ %let dsname=&dataval.MKT_DATA;  /**** declaring data set name with library*****/ %macro warning1(name); %if %sysfunc(exist(&name)) %then %do; Proc printto log="&logfile.\WarningLog..txt" new;     /*** printing log in the text file at specifile location &logfile****/ run; %Put WARNING :- &name does not exists; Proc printto  ; Run; %end; %else %do; Proc printto log="&logfile.\WarningLog.txt" ; run; %put &name exists; Proc printto  ; Run; %end; %mend ; %warning1(&dsname); /**** Pros:  If any sas process is dependent on the existence of the data then we can use this macro***/ /**** Cons:  *Even if the SAS data set is available with zero observatio
Read more

2018 plan for getting expertise in Machine Learning and Deep Learning

-
Machine Learning and deep learning are next frontier in the world of innovation. These skill sets are high in demand and demand is going to increase further as we are moving towards a world with connected systems. Although I am experienced professional with SAS having multiple SAS certifications and  using SAS for programming, predictive model building, optimization and data visualization for more than seven years, I will be foolish I do not recognize how important it is to adopt open source platforms for innovation.  2018 plan to acquire expertise in these are are 1. Choose a programming language and I have chose python as my language of preference  2. Do hands on practice to be efficient in Pandas and Numpy. These two libraries are useful and very important for data exploration,wrangling and data cleaning. I find Kernals in Kaggle and below blogs and books to be helpful tutorial for  Pandas  a. https://www.dataquest.io/blog/pandas-tutorial-python-1/  b.https://www.d
Read more