Data engineering and data science for leveraging high frequency streaming data  

Internet of Things (IoT) mainly focuses on how to enable general objects to see, hear, and smell the physical world for themselves, and make them connected to share the observations. These often require the use of sensors providing high-frequency streaming data. In this talk, we would discuss how one can leverage these high-frequency sensor data for IoT analytics.

There are primarily 3 building blocks for a quality analysis of high-frequency sensor data. These building blocks comprise of:

  • Data engineering techniques to ingest and store high-velocity data and providing a fail-safe mechanism to avoid data loss. These involve configuring BigData machines for successful real-time storing as well as configuring messaging queues for streaming data.

  • Feature engineering and development of supervised and unsupervised models using high-velocity data. Feature engineering is crucial for extracting maximum value from data. Additionally, one must also understand the intricacies of different algorithms/modelling methodologies for correct selection of methodology.

  • Implementing models in real time. One can build a high precision model but if it’s not implementable it’s of no use. One has to configure the systems to correctly ingest models for perfect execution.

Key takeaways:

The following are the key takeaways from the talk.

  • Understanding BigData machines and messaging queues
  • Techniques for good feature engineering
  • Variable reduction techniques
  • How to choose between models – parameters for selection
  • Implementation challenges and solutions



Hindol, is the CEO of Actify Data Labs (to be incorporated), which is a TrueNorth Company focusing on analytics for health care and mid-market companies. He brings in about 16 years of analytics consulting experience spanning across multiple industry verticals and markets. Hindol had led multiple analytics consulting and customer centricity initiatives in the Indian, Asia Pacific and North American market across financial services, travel and telecommunication verticals. In India, he had been the pioneer in establishing credit bureau analytics for the Indian market; he had developed the first set of credit bureau scores for CIBIL. He had also lead analytical consulting assignments for Indian companies across financial services, manufacturing, travel and hospitality, telecom, health care and retail sectors. He has developed many frameworks that leverages the power of sensor and machine data for machine learning. Hindol holds a bachelor in engineering from IIT Kharagpur and an MBA from IIM Bangalore. He is also the co-author of the book titled “Business Analytics – Applications to Consumer Marketing – Sandhya Kuruganti and Hindol Basu” published by McGraw Hill India.

Bijoy Khandelwal

Bijoy brings in a unique combination of technology and data science expertise which has helped him build scalable machine learning solutions. He combines depth of expertise in managing high-volume, high-velocity data and creating machine learning models based on high frequency data.

Bijoy has extensively worked on building scalable technology stack that are scaled to handle high transaction sizes coupled with extremely fast response times. He has a number of native Hadoop implementations to his credit. He has developed streaming and IOT data management solutions across sensor data, click stream data and social data. He has also developed a number of traditional data warehouse and data mart solutions for consumer and financial services industries

Buy Ticket
Social media & sharing icons powered by UltimatelySocial