Dear AVian,
2015 was year of growth for us – we transformed from a blog to a community of data scientists. We launched our discussion portals, hackathon platform and job portal – and each of this initiative is shaping up well. Our mission of building a nation of data scientists is coming live every day. And all of this happened only because of your enthusiasm and support. We couldn’t have thought of this journey without direction from you. Your comments, emails, letters and queries helps us decide our next steps.
So, on this day of new year, I want to hear more from you. I want to hear from you what bothers you the most about data science? What aspects of data science you find hard to learn? What is blocking your successful journey? What is one thing you would want us to do right in 2016?
Simply reply to this email on write in comments below.
I don’t know how things will turn out. But, I promise, I’ll try my best to give you many reasons of happiness & joy in 2016.
Wishing you all a year full of joy, health, curiosity, knowledge and perseverance.
I’m waiting for your reply.
Regards
Kunal Jain
Founder & CEO
Analytics Vidhya
Dear Kunal,
Happy New Year and an even more successful 2016.
Keep up the good work.
I am currently in Bombay and will be heading back to USA on the 5th. Pl. let me k now if you are travelling around this are in the next 2 days.
Best Regards,
Srikar
202-445-0183
Srikar,
Looks like I’ll miss you. I plan to be in Mumbai a couple of weeks down the line.
All the best for your trip.
Regards,
Kunal
The biggest problem I have had is access to realistic data. I would like to see a repository of “scan data”, mfg rep sales data, etc., and other types of data sets for analysis and feedback from other Data Analysts on how they approached the problem. It is extremely difficult to get good realistic data as companies jealously guard their data and trade secrets.
Alvin,
This sounds a lot like our hackathons. Do check them out – http://datahack.analyticsvidhya.com
We release data after working with our clients for these hackathons and the community usually shares their solutions on our discussion platform.
Hope to see you competing in our hackathons this year.
Regards,
Kunal
Dear kunal
HAPPY NEW YEAR
Regards.
kishore
Thanks Kishore. Same to you.
I am totally new in analytics but i got almost 10 years of experince in stock market,i want to build some stock price prediction model can we do that
Sagar,
We are working on creating a problem on these lines.
Regards,
Kunal
Happy new year Kunal
Happy New Year to you too Golam Kabir.
Hi Kunal,
Wish you and your team a very happy new year. All around the web we only find analysis on small sample of available datasets.
It would be a great idea for a section on analyzing extremely large datasets (million records strong). I have learned that a completely different approach is required with such large datasets, as you not only focus on the algorithm, but also on the data, speed and efficiency of the algorithm used. I am looking around for more guidance on the subject, and would appreciate if AV could do a series on this.
Regards,
Jigar
Thanks Jigar for the suggestion. We will definitely put up a series on this.
In the meanwhile, do look out for our hackathon datasets – they are reasonably big and can help you get flavour of big data sets.
Regards,
Kunal
Happy new year Kuna !
Keep going on what you’re doing. You do a very good job. It help me a lot in my work and in my personal development.
I just hope that you can create something like learning paths but devoted to more theory in order to “unblackbox” statistical tools like random forests for example.
Amine,
Thanks for your suggestion.
We will definitely work on it.
Regards,
Kunal
Hello Kunar,
First of all thanks to you and your friends for this usefull site. I wish you a great 2016.
In my experience is one of a kind, and you and Andrew Ng (I’m quite sure you heard of him) are my personal heroes. So a deep thanks to you and Andrew for sharing your deep knowledge.
My personal barrier to learn is to find someone giving me credits enough to work for him on real task. I tried also to offer my services for free, but I was unlucky.
So my target for next year is to read all your articles, and try to participate in a competition, hardening myself until I will find a job in this industry.
Gianfranco from Italy wishes to all friends here a very good 2016 !
Dear Gianfranco,
Thanks for those wonderful wishes. Prof. Andrew Ng has made huge contribution to the field. While it is flattering to be mentioned along side him, I have a long way to go before I can come any where close to the contributions he has made.
Looking forward to interact with you further in 2016.
Regards,
Kunal
Hi Kunal,
Happy New year. You are doing a great job and I would like to thank you for starting AV and actively working on improving the platform.
I would like to suggest you something. I’ve an intermediate level of knowledge in data science and big data technologies. I used to participate in some Kaggle competitions also. But Hackathon and competitions are platforms that focus on model building and optimizing. I believe that in real projects, data collection and feature engineering steps have importance. I wish if there were some opensource projects or platforms were people can collaboratively work on solving real world issues. For example exploring data provided by government, NGOs, Survey platforms, etc. and developing a good insights from it.
I wish you even more successful 2016.
Thanks & Regards,
Niyas
Thanks Niyas for the suggestion. We will definitely work on it.
Regards,
Kunal
Happy new year 2016!
The best wishes for everybody.
I would like to read more about multivariate data analysis, specific for QlikView.
Thank you
Hv
Thanks Hector. Happy new year to you too.
Will make sure we address this need.
Regards,
Kunal
Hi Kunal,
Wish you happy new year!!!
Currently I m working as qlikview developer. I want to move to analytics. Kindly help me further learning.
Regards
Viresh
Viresh,
Happy New Year to you too.
You can start by following our learning path on R or Python.
Regards,
Kunal
You are really doing a good job, now I recommend to start guiding members toward main certificates in big data and build skills matrix for big data to help employers.
I think certificate section may be a good addition to your web site and build a window for those who got these certificates to tell us about their experience before and after getting it.
Hossam,
Good suggestion. It could be a good fit with 2016 plan.
Regards,
Kunal
Hi Kunal,
Two areas where I would like help are –
1) Finding real-time work – working in Kaggle and other competitions are fine but it would be of great help if we can get to work (under any Data Scientist) on real time projects. This will increase our learning scope tremendously.
2) Exploring and Explaining (top) successful solutions in different competitions (Kaggle/AV/CrowdAnalytix etc). Discussing various aspects like – what feature engineering was done which differentiated top rankers from others, what algorithms did top rankers used etc.
Along with the learning paths your team is publishing, if you can write few posts on above topics, it would be of immense help to newbies trying to swim through the ocean of Analytis 🙂
Highspirits,
1. Good thought – it might be a bit difficult to channelize something like this. But we have a few thoughts to test out in 2016.
2. We have done a few articles on this line and would continue to do so. On our hackathons, people usually share their approach on the discussion portals – so look out for them.
Have an awesome 2016.
Regards,
Kunal
Hi Kunal,
Your articles on Analytics are always informative and helpful. Above all your committment to help fellow Data Analysts and Data Scientists is amazing. We always look forward to what comes from you. Keep the good work going and we wish you a wonderful New Year 2016.
Really appreciate if you could clarify the following questions.
Q1) What tools do you suggest for text mining preferbly any tools that require no or little scripting?
Q2) I am working on a project where we have 600,000 plus records in MySql database table. I need to search for a string of two words in the column2 (long text). Then pick 8 character numeric that follows the two word search string and insert both 8 character numeric from column2 and ID from column1 into a separate table..Please suggest what is the best fit tool for this requirement? If you have any code snippet please share.
Thank you as always,
Trinadh
Trinadh,
Best place to start looking for 1 is ‘tm’ library in R or ‘nltk’ library in Python.
For the second question – please post it on our discussion portal: http://discuss.analyticsvidhya.com along with a few lined of data and what you want. I am sure some one in the community will help you out.
Regards.
Kunal
Kunal,
Thank you for your response. That is helpful and I will pursue this further on your discussion portal.
Regards,
Trinadh
Hi Kunal,
Wish u a happy new year.. hope this year will be the fantastic year for you and Analytics vidhya Community.
Regards,
Vidya