Swiftapply – A Python Package for Efficient and Superfast use of Pandas ‘apply’ Function
Overview
- Swiftapply works on the pandas ‘apply’ function to make it efficient and quicker
- The package runs the apply operation is a vectorized style; failing that, it automatically decides if it’s faster to perform task parallel processing or use a simple pandas apply
- You can install it from pip using the command given below
Introduction
Time is of the absolute essence when you’re working on a machine learning project. You don’t want to be waiting around for pandas to finish it’s loop before you can proceed with your model building, do you? But this is exactly what happens in most cases.
When you use the ‘apply’ function in python, it lets you apply any function you specify on the pandas dataframe. But as pointed out so well in this article, the apply function is basically a for loop and it takes it’s own sweet time to run. How can we go about combating this?
Well, this is what led to the creation of the ‘Swiftapply’ package. Developed by Jason Carpenter, this package aims to make the execution of the apply function more efficient, and less time consuming. As the developer has explained in this blog post, “swiftapply tries to run your operation in a vectorized fashion. Failing that, it automatically decides whether it is faster to perform dask parallel processing or use a simple pandas apply”.
How to go about installing it? You can do so using pip. Follow the below command:
$ pip install swifter
You can view the GitHub respository for Swiftapply here. To read more about how this apply function works, we recommend reading this documentation.
Our take on this
Pandas is an amazing flexible library and most of the operations you perform using it are still done fairly quickly. If you have a Linux or Mac machine, you can also try Pandas on Ray to boost your code’s performance. You can also try compiling with cython or numba. All in all, there are quite a lot of options available in the market now for accelerating your operations.
Use this package and let us know your experience in the comments section below!
Subscribe to AVBytes here to get regular data science, machine learning and AI updates in your inbox!
