MyStory: I became a Data Scientist after working for 10 years in IT Industry

Karthe 01 Sep, 2016 • 7 min read

15e974d

Distant Memories (Prologue)

Getting into Analytics or Data science stream was never my dream. I got into this out of an accident. Prior to getting into data sciences, I was a mainframe programmer all through. The only aim that I had for a very long time was to get into a good MBA programme.

 

Business analytics as a new career Stream

It was 2013. I had 9+ years into software services industry by then. My career was almost stabilized and I could not see much growth out of it. Hence I was planning to turn up to my dreams of management education. But I was quite apprehensive as my long experience would bar me from getting into a decent MBA. It was when I came to know about the new upcoming stream called Business Analytics.

My first step towards getting into analytics was scouting for an opportunity internally. It didn’t worked out for me for two reasons. First, there were not much analytics initiatives happening within the companies at that time. Secondly, as the advanced analytics stream evolved as an extension to the business intelligence division in most of the companies (especially the Information technology services), they could not afford to take a non BI person like me and to train up for their analytics works. Also, there were not much courses online at that time. Or perhaps, I was not aware.

 

A Rude(Self) awakening

That was the time I decided to reskill myself to suit to the analytics industry. I had applied for couple of long term analytics programmes available in India and I managed to convert one of them into admission. But I never know that the class of business analytics will be math (advanced math) and technology heavy, the areas which I was bit apprehensive, until I went to the first day of my class. Apprehensive was due to the fact that I was working on a very old technology and doesn’t require you update constantly.

The extreme main skill that any data scientist should have – more than the art of storytelling – is the art of questioning and analyzing the information in hand. The very first mistake that I did before getting into business analytics was failing to understand what “Business analytics” professional do in their work – atleast its literal meaning. My cluttered mind strongly correlated “Business analytics” with the term “Business analysis”.  At the end of my first day class, one thing I really understood was – The decision to join the business analytics program was taken based on the gut feeling, rather than an informed decision.

 

Beginning of a thousand-mile

I had two stages of transition from being a mainframe programmer to getting into data sciences – First one was undergoing a one year programme and the other being the challenges that I face during the day to day work.

 

Advantages of structured transition

The one year analytics programme was extremely challenging – I would call it as a toughest challenge (yet most rewarding at a later point of time), I had till now in life. Multifaceted responsibility is the first challenge that I had faced as soon as I enrolled to the programme. Being a dad to a 5 year old, a production support lead at the office, coping up with the course was extremely difficult for me. With the kind of rigorousness that came along with the course, be it in terms of week end online classes (apart from the regular classes that we attend at the campus), tests, home works and assignments I was completely bombarded every day.

The one year spent at the programme was a roller coaster ride for all my student mates without any exception. The quality of problems that we were tested as part of assignments and their deadlines driven us to spend sleepless nights.  The effort to put completing every mini project, the early morning hour classes, the late night discussions  along with my family and office responsibilities – all together  would have made my health worse, if had I not been supported by the awesome peer group.  I still remember the days attending an hour of office call and taking up the exam in tandem. But I didn’t realize God was preparing me for the challenges that would follow up later in my career in analytics.

 

The First Date

Job interviews are always like first date. The outcomes are seldom predictable.

The next challenge that I faced was when I started looking for an opportunity to work in data science.  Hiring managers are typically concerned about taking in someone who comes with considerable experience in non-data science stream.  Luckily, the capstone project that I did as part of the course work with one of the prominent retail brands came in handy for me. The interviewer typically liked the process that was followed in building up the solution during this project. Thanks to awesome professors who mentored continuously.

These days, I also see many people who apply and get selected in interviews just by participating actively in data science competitions. Infact, as far as I have seen, data science competitors outshined the one with real time work experience in the interviews.

 

The Swell is really big

If the challenges pertaining during the academic stage of my transition was pertaining to the mathematical part, at work I was facing the challenges from the process and domain front. Here with I am briefing some of the lessons that I learned as part of my journey:

Be good in domain – One of the foremost challenge that any data scientist would face is the short-come in domain knowledge.  It is imperative for any data scientist to be good in the domain without which an analytics engagement can never be successful. Without the domain knowledge, a data scientist would never be able to answer any of the business question. He/She cannot build a good predictive model without the right set of variables. The insights that he generates will go in vain without the business knowledge.

Setting the expectation right with the customer – Most important and vital to satisfy every stakeholder. Customers may not know about analytics. All they know is that if the data is given, data scientists do some magic and generate insights which can be used to improve the business.

 A data scientist should exhibit patience and teach them what can be possible with data science and what not. I remember the days I spent with the customer explaining why a statistical model is much better than predicting manually, what is mean by a model by itself, how to interpret confusion matrix etc.

Knowledge of SQL and databases – I had an initial assumption that it is the duty of the data engineer to extract the relevant data required for building the use cases. It is false. Companies may not be interested in investing one more person for pulling the data. A data scientist is expected to know how to extract the data that he requires, and transform the data to the required format.

Patience for analyzing the data– Data analysis has to be done iteratively from various perspectives.  We never know what kind of pattern exist inside the data. However, by following structured data analysis methods and little bit of patience, the pattern that the data follows can be identified.

Presenting the solution– In my view, this is more important than building a model. All the hard work put in by a data scientist can be showcased only in this area. Also it is important to know the context of to whom the presentation is made. We may not talk about that adjusted R square or an ROC curve to end user.

Be comfortable when you fail – Most of the work that I did during the initial days failed to larger extent. Every time I presented the solution to the customer, there was a high probability that it gets rejected. I looked stupid every time I made silly and big mistakes. But I eventually learned that all these are part and parcel of the career in data science.  Just try to have a mentor if you are new to the analytics projects or start small if you don’t have a mentor.

 

The Meta Critics:

Here are some the questions that you should ask yourself before getting into a career transition:

Why do you want to make a career shift? – I meet many peoples who come to me and say that they wanted to shift to analytics just because they have problem with existing job/manager/company etc. If this is the case with you, try to see if something can be made interesting in your current work. Changing the work stream may not solve your problem.

Shifting career with high work-ex:  If your overall experience is more than 7 years, please think thrice before changing stream.  Please remember that with more than 7 years of experience, you may not be allowed to experiment in your work rather you will be expected to deliver.

Why do you want to get into data science?  If the answer is that data sciences present lucrative opportunity, then you might want to think again!! You will be getting into a field where you can’t sail smooth atleast for few years down the line.  You will be expected to keep yourself updated very frequently and it is very difficult for many people whom I had seen, including myself.

What do you know about analytics to make a career in it? –  Test this by doing some self-learning. Try some online courses in coursera or edx or udacity to check whether you will be comfortable with data science. Typically, you should not take data science as your career if you are averse to mathematics or if you show dislike to databases and querying tools like SQL. If you are new to all these, check these at khanacademy.com (for math courses) and w3schools.

 

Dare to Dream

Life holds special magic for all those who are dare to dream beyond their abilities. Some tips for the people who would want to hold their dream and looking to get into data sciences.

  1. Do minimum level of self-analysis before getting into data sciences. Test your fitment for data sciences in this link.
  2. Try to speak to some of them who already transitioned to this field. Their experience can give you lot of insights to your new career. If you don’t know anyone in person, post your questions in forums like discuss. Someone will be there to help you out.
  3. Read and subscribe to articles in Analytics Vidhya, data science central before even committing to the pricy courses available. Follow forums like discuss, quora data science Also if possible try to take couple of 101 challenges from datahack (and later kaggle). You will get a first-hand experience of how to solve a data science problem in these portals. I learned more by solving the challenges than working in real time environment.
  4. Never believe in claims which promises to make you a data scientist in less than 3/6/12 months. Every career has its own path of learning curve and data science is no exception to it.

 

Foot notes

  1. This article was aimed mostly for the people who want to enter into data sciences / analytics with considerable work experience. The challenges mentioned here may not necessarily apply for a fresher or with less than 5 years’ of work experience.
  2. Data science / business analytics are used interchangeably.

 

Karthe 01 Sep 2016

I work in data analytics domain with exposure in the capital markets and retail businesses. I use data as a tool to solve the business problems of our customers. I mine massive datasets to develop analytical framework that help clients to address their complex business challenges. I build predictive analytics based solutions using the statistical and machine learning algorithms to the forefront of our Client's decision making process.

Frequently Asked Questions

Lorem ipsum dolor sit amet, consectetur adipiscing elit,

Responses From Readers

Clear

Aditya
Aditya 02 Sep, 2016

Hi Karthe could you please suggest the details of the 1 year program you did.

Shahnawaz
Shahnawaz 02 Sep, 2016

Good post.I agree with most of your pointers.I have started engaging myself with online courses for data science few months. I don't see technical knowledge as a hurdle but the biggest challenge is to understand the domain/industry.Could you please let me know what else do you did to acquire decent domain knowledge.Do capstone projects help you to leverage your domain knowledge? I tried to solve a couple of online challenge but was lost somewhere in the middle.Please suggest.

Gagan
Gagan 02 Sep, 2016

Thanks a Lot Karthe ... Article written by you is Inspiring for me !!

Mallikarjun
Mallikarjun 03 Sep, 2016

Thank you very much.. surely it will help me to take wise decision.:)

Mahesh Jadhav
Mahesh Jadhav 06 Sep, 2016

Hi Karthi First all Thanks for wonderful article which i was looking for from a long time. I have recently joined the CBA course offered by ISB -Aug 2016 batch. My story is similar to you, in the sense I am also currently working as Technical lead/Project lead basically a mainframe developer with 8 yrs experience currently working at BNY Mellon. I have done my Masters from IIT Roorkee in Physics and Bachelors in Mathis , Physics and Computer science. So in a sense Good at maths and research..I have fair knowledge of OOP and Databases I am very much interested in pursuing a career in data analytics and would like to establish myself as a data scientist . I am selected for the higher education program by my company where 50% course fees is sponsored by company. By the time I will complete this course i,e, Jan 2018 I will be having around 9.5 yrs exp in Mainframe mainly in Banking, domains like cards, mortgages and securities. With this profile of mine can you Please let me know what are the challenges which i need to be ready to face as someone from a non analytics background in finding a job in Non banking/ banking domain. What do you recommend about the project , should we do within/outside our company (if given a chocie) and with in the same domain we are currently working or in some other domain which may help us in finding a Good Job. Also do you recommend moving internally within the organisation if given scope or should we try outside understanding that within the organisation we may not get any hike after this transition. also can you please advise on which topics we need to focus more while we pursue this course at ISB I think you would be right person to advise on above queries, as more or less our story is similar. if possible can you share your email id. Thanks Mahesh Jadhav [email protected] Thanks Mahesh

Mahesh Jadhav
Mahesh Jadhav 06 Sep, 2016

Thanks karthe , for taking your valuable time to answer my queries. But there is one point which I wanted to know specifically. As a Mainframe developer what are the challenges one may face when he joins into this field At what level is one recruited if he has about 10 yrs non analytical experience-assuming he cracked the interview- what roles can he expect.,assuming he is above average in all the concepts he learnt. Learning and interest is one thing but I also want to know how much hike can one expect when he makes such a transition and what is expected of such a person.\ How was your life when you made this transition , do you had a team with which you worked and gained some of the knowledge on Job or how it goes .. Pls advise. Thanks Mahesh

Mahesh
Mahesh 07 Sep, 2016

Thanks Karthik..well said..those are really some valid points you said. Completely agree with you. Thanks Mahesh

Sreekanth
Sreekanth 07 Sep, 2016

Hi Karthe, Am working as Middleware Consultant from last 2+ years and with Engineering background. I want to switch into Analytics domain. From last couple of weeks been searching for the same on internet. Am not able to trust online courses for my career transition. Am open for two options 1. ISB 2.Great Lakes, Pune. As am from Mumbai, ideally ISB (Hyderabad) wont be suitable due to distance and job commitments. Option available is Great Lakes, Pune. Could you please advise me whether am on right track to get into GreatLakes for business analytics or should i think about some other institutes. Am into big dilemma re from where i should pursue my analytic plans. Please shed some light on me to get into right trick. Thanks in advance!

Sameer Subhedar
Sameer Subhedar 08 Sep, 2016

Karthe, Your article is truly inspiring, I am inspired by it. I am planning to take up the IIIT Bangalore Program which is happening in partnership with UpGrad. It is an online course, The course which you took up was a full time residential program or an online course ?? Thanks !!

Arrif
Arrif 11 Sep, 2016

Well articulated Karthe, actually I had discussed with you on call before (~3 months back) regarding getting into data science. I'm following what you suggested to get into data science unit within my org. I'm in negotiation, hopefully I will get into it.

Vishwa
Vishwa 13 Sep, 2016

Hello Mahesh, First of all congratulations and wish you many success in this new adventure, here are few questions that I need to ask, I am almost 18 years in IT, have been in multiple roles such as manager, coach, sr manager, quality head, delivery head have worked in both India and abroad and right now in a senior role but don't find it a lot too interesting and learning has come down significantly.., I am bit of pseudo technical but have been largely into delivery management and facilitation roles. I know where the industry is heading today and I see a major challenge for folks who are into delivery and program management and whether its good or bad, the change in focus for IT is going to impact all the management people around, With 16 years I have seen a plenty of customers, have a black belt and lean certification under my hat. I have almost lost touch with programming and looking for complete shift to secure myself for the next 10-15 years, that's when I got this site and read your post. I am not averse to mathematics but you can imagine that I haven't done any mathematics other than using Minitab or statistics few years ago. what is your experience and input to shift the career to data science, do you think folks like me will be looked at if we decide to make a shift

Guddu
Guddu 14 Sep, 2016

I would like to address one thing which many of you have in your mind "Taking a course to transition into Analytics - Would it payoff?" Simple answer is Yes but you really need to think about the time it takes to get your returns. Let me elaborate with an example - Person A, has arnd 5 years of experience with Data (Business Intelligence, Reporting, Data warehousing, ETL, SQL, any sort of descriptive analytics etc.) wants to get into Data science and Person B, has arnd 5 yrs of exp. has done nothing related to Data, wants to get into Data science Year 2016 - For Person A - it is a lot easier for him to take up a course of Data Science and get into this field in an Organization by advancing his resume. He would also most likely to get a position and compensation at par if not more for such transition. Building up further on this would be straight forward. Hence it could start paying him off in may be less than 2 years. Year 2018. Year 2016- For Person B - having no knowledge of legacy analytics and taking up a course of Data Science is like taking an admission into 5th class without having run through the basic classes. He would have to struggle for long time to build the base and then come upto the speed of Data scientist's expectations. One has to start as a fresher and have to compromise on compensation. It might take him more than 5 years to reach to a level when it actually starts paying off. Year 2021. Now imagine where would the Person A be in Year 2021. India is full of people like Person A and most of them are doing something to get into Data science.

Rajesh
Rajesh 20 Sep, 2016

Hi Karthe, Fantastic post !!! Thanks for sharing as it can be inspirational for many like me who belong to Mainframes background and 10+ years of IT experience. I have 1 question : Since we are Mainframe programmer/lead,do we need to learn front end technologies too - Java/.Net/HTML5 or CSS3/ Hadoop. I believe a knowledge of SAS/R/Statistics/Machine Learning/SQL should be enough to move forward. Can you please advise other than these skills (and Attitude) do we need something else to land data science jobs. Also, what role should we targeting as considering 10 years of current experience. Thanks Rajesh

Venkat
Venkat 28 Sep, 2016

Hi Karthe, Well written and unbiased article! Even when people tempted you to speak on salaries, etc. you managed well :) I am searching on AV regarding Analytics courses and stumbled upon your article. Can you please suggest me on consulting prospects after completing analytics course and 2-3 yrs work ex? Just to give you background, unlike most other members, I have a PhD in Mechanical Engineering and currently working as a Mechanical Designer / Analyst. Main reason I am planning to join Analytics is my interest in patterns in data and a bit of love with numbers! Down the lane I intend to offer consultancy / training services in Analytics from a Tier-2 city in India. I would like to know if I am thinking in right direction and making right assumptions regarding consultancy environment for analytics? Which domain can I serve better with knowledge in analytics. What is your advice in terms of which course to pick to have strong fundamentals and applied knowledge. Thank you for your time and advice.

Venkat
Venkat 29 Sep, 2016

Hi Karthe, Thank you for detailed and encouraging reply.. I will post my question on domain in the forum.

Yashi
Yashi 17 Nov, 2016

HI Karthe , I am totally new to the field of data science, I have 1.4 yrs of exp. in front end development. From where I should start and what things should I cover so that I can get into this field of data science.

Rajesh kumar singh
Rajesh kumar singh 08 Jul, 2017

I have +10 years of IT software experience. Planning to move to business Analytics. Can you please suggest me about IIMC EPBA course. Does this course covering most of analytics area. Also does this course include any dummy project . Regards Rajesh

Sandeep
Sandeep 17 Jul, 2017

Hi Karthe, Thank you very much for sharing your experience with us. Let me ask one question, currently working in Accenture in operations domain(1 year experience)which is totally non technical and I want to upgrade myself and always looking to enter into analytics domain. I just wanted to ask, is it good to change my domain form operations to analytics. Kindly give me the your suggestion and what are the things need to learn as a fresher. Thanks, Sandeep.

Swagata Das
Swagata Das 01 Aug, 2017

I have 10 years work experience as snr. research associates for Math. I want to shift to data analytics. Ned a suggestion.

pvs
pvs 24 Sep, 2017

Hi Karthe, It was really an inspiring story. Thanks for sharing. I have around 10 years of experience in the production support field of an investment bank. Currently I am looking for a career change and thinking about joining a good course in analytics. When I apply for a job after completing the course, would like to know if they will consider me as a fresher while deciding compensation package or some weightage will be given to my 10 years experience. Kindly advice. regards Pvs

Kapil Sharma
Kapil Sharma 30 Oct, 2017

Hi Karthe, Thanks for a detailed documentation on the Data Science. I want some help and I am confident you can guide me. I am a Maths and Statistics graduate and passed in 1998. After that worked on Software Development side for 11 years and worked for Application Support for 6 years. Recently my support project was transitioned to big company that did not hired me as they cant afford the 10+ Lacs person on the project. I need to rebuild my carrier and you can say need to start once again. Is I am a right candidate for Data Science? If yes how to start and where to start. Please help!

  • [tta_listen_btn class="listen"]