Learn everything about Analytics

Home » FFmpeg — Your powerful video/audio helper in your application

FFmpeg — Your powerful video/audio helper in your application

This article was published as a part of the Data Science Blogathon

The name is inspired by MPEG, which means Moving Picture Expert Group. The FF means Fast Forward. The MPEG group is the alliance of working groups behind setting many of the essential video standards today, including the MP4 format, which we are all familiar with.

Outcomes

  1. What is FFmpeg?
  2. Why do people use FFmpeg?
  3. How to set up FFmpeg in Windows, Ubuntu, and macOS.
  4. Libraries are available in FFmpeg for media manipulations.
  5. Tools that FFmpeg comes with for command-line applications.
  6. Usage and examples for FFprobe and FFplay.

Why do people use FFmpeg?

It is often referred to as the Swiss Army knife of media transcoding or streaming. We can use FFmpeg to perform a lot of functions. The code is written in C language and optimized for best performance. It commands are simple to run. Once you get familiar with the concepts, it is very flexible to all the filters and options you can use to fit your needs.

FFmpeg supports a wide range of codes, formats, devices, and protocols, making it an ideal choice as a transcoding engine. Unlike many discontinued projects, it is still in active development for more than 20 years. There’s a large community of developers, users, and contributors who constantly develop new features and fixes.
FFmpeg has been used in the core processing for video platforms like YouTube and iTunes. Most of us used a media player like VLC to play video files. VLC uses FFmpeg libraries as its core. Some video editors and mobile applications also use FFmpeg under the hood.

Setting up FFmpeg in Windows, Ubuntu, and macOS

There are a few things you need to know before setting up FFmpeg. There’s no official build for any operating systems. But it is an open-source project, which you can get hold of the FFmpeg source code from the internet. So, you can choose to build it yourself from the source code. You can download a snapshot of the current source code here or do a checkout from their git repository and follow the instructions to compile it to your machine. This can be a little complex and time-consuming and may require you to understand and choose some configurations options. It may also need additional tools and compilers to be installed at first. So unless you have a good reason to build it yourself, you may not want to go this route.

Instead, you can choose to download and install a pre-build FFmpeg package for your operating system. There are two ways to set up these pre-build binaries. If you have a package manager like APT in Ubuntu and Homebrew in macOS, it is easier to use it since it takes care of downloading FFmpeg and its dependencies.

Let’s keep the thing as simple as possible and choose the easiest way to get started. The procedures below show how you can set up using pre-build packages.

macOS

On macOS, the easiest way to get FFmpeg installed is by using Homebrew. Make sure you installed Homebrew in your macOS, or you can follow this tutorial on how to install Homebrew.
We are using macOS 11.4 in this case.

  • Open Terminal and type brew — version . You should see the brew version printed out in Terminal if Homebrew is installed successfully.
mac FFmpeg
Brew version in macOS
  • For macOS in x86 Architecture, use the commandbrew install ffmpeg while forM1 please use the command aarch arm64 brew install ffmpeg .
mac FFmpeg 2
Downloading and installing necessary dependencies
  • After the installation is done, you can type ffmpeg -version in Terminal. If the FFmpeg version is printed, means FFmpeg is installed successfully.
FFmpeg terminal

Windows

Unlike APT on Ubuntu and Homebrew on macOS, there’s no well-known package manager on Windows, which you can use to quickly install FFmpeg. For Windows, you need to down and set it up manually.

There are 4 Windows FFmpeg build variants available can be found here:

  1. git full – built from a master branch with a large set of libraries.
  2. git essentials – built from a master branch with commonly-used libraries.
  3. release full – built from the latest release branch with a large set of libraries.
  4. release essentials – built from the latest release branch with commonly-used libraries.

We will use FFmpeg release essentials in our case.

  • Download FFmpeg release essentials here.
  • Extract the files to a place from where you will use them in the future. In my case, I put them in my E Drive.
FFmpeg folder
  • Inside the bin folder is containing those executables we want.
bin folder FFmpeg
  • Add bin folder to Windows Environment Path Variable so that we can run this tool without having to specify the full path every time. Search for Environment Variables in Search Bar and click on Environment Variables…
envt variable FFmpeg
  • Click on Path in User Variable, click New, and paste the full path of the bin folder.
FFmpeg user variable
  • Open Command Prompt and type ffmpeg -version . If you see the FFmpeg version is printed, means FFmpeg is installed successfully. We also can check ffprob -version and ffplay -version .
FFmpeg command

FFmpeg Libraries

FFmpeg has several valuable libraries that you can use directly from your application code. Each library contains different functions related to a particular area.

Some of the famous libraries are:

  1. libavcodec — Contains all the encoders and decoders that FFmpeg supports.
  2. libavformat — Has all the muxers and demuxers for dealing with various containers format.
  3. libavfilter — Consists of many filters that you can use to modify the audio or video according to the requirements.
  4. libavdevice— Support several different input and output devices.
  5. libavutil — Aid portable multimedia programming.
  6. libswscale — Performs highly optimized image scaling and colorspace and pixel format conversion operations.
  7. libswresample — Performs highly optimized audio resampling, remixing, and sample format conversion operations.

FFmpeg Tools

Besides the libraries which libraries can use from other applications, the FFmpeg package contains a few tools on its own.

Some of the tools are:

  1. FFmpeg — Main transcoding engine. It is often invoked from a command line or another process.
  2. ffplay — A minimal tool for playing audio or video.
  3. ffprobe — Quickly inspecting and media to extract valuable information, like how many streams there are in the media, the frame rate of a video, etc.

FFprobe Usage and Examples

Usage

ffprobe [OPTIONS] [INPUT_FILE]

  • man ffprobe — Refer to the manual.
  • ffprobe -h — Shows included help.
  • -v error — Hide logs unless there’s an error.
  • -show_format — Show formatted video metadata using Tag.
  • show_streams — Show all the streams in the file (both video and audio).
  • -print_format —Print metadata in specific format (CSV/XML/JSON).
  • -select_streams
  • -show_entries stream= — Show only the specific entity and its nested dependencies.
  • -show_entries stream= default-noprint_wrappers=1 — Show only the specific entity.
  • You can find out more FFprob usage here.

    Basic Examples

    • Show all video metadata.
    $ ffprobe -v error -show_format -show_streams input.mp4
    Output:
    [STREAM]
    index=0
    codec_name=h264
    codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
    profile=High
    codec_type=video
    codec_time_base=1/50
    codec_tag_string=avc1
    codec_tag=0x31637661
    width=320
    height=240
    has_b_frames=2
    sample_aspect_ratio=1:1
    display_aspect_ratio=4:3
    pix_fmt=yuv420p
    level=13
    color_range=N/A
    color_space=unknown
    color_transfer=unknown
    color_primaries=unknown
    chroma_location=left
    timecode=N/A
    refs=4
    is_avc=1
    nal_length_size=4
    id=N/A
    r_frame_rate=25/1
    avg_frame_rate=25/1
    time_base=1/12800
    start_pts=0
    start_time=0.000000
    duration_ts=384000
    duration=30.000000
    bit_rate=34761
    max_bit_rate=N/A
    bits_per_raw_sample=8
    nb_frames=750
    nb_read_frames=N/A
    nb_read_packets=N/A
    DISPOSITION:default=1
    DISPOSITION:dub=0
    DISPOSITION:original=0
    DISPOSITION:comment=0
    DISPOSITION:lyrics=0
    DISPOSITION:karaoke=0
    DISPOSITION:forced=0
    DISPOSITION:hearing_impaired=0
    DISPOSITION:visual_impaired=0
    DISPOSITION:clean_effects=0
    DISPOSITION:attached_pic=0
    TAG:language=und
    TAG:handler_name=VideoHandler
    [/STREAM]
    [STREAM]
    index=1
    codec_name=aac
    codec_long_name=AAC (Advanced Audio Coding)
    profile=LC
    codec_type=audio
    codec_time_base=1/44100
    codec_tag_string=mp4a
    codec_tag=0x6134706d
    sample_fmt=fltp
    sample_rate=44100
    channels=1
    channel_layout=mono
    bits_per_sample=0
    id=N/A
    r_frame_rate=0/0
    avg_frame_rate=0/0
    time_base=1/44100
    start_pts=-1024
    start_time=-0.023220
    duration_ts=1324024
    duration=30.023220
    bit_rate=56517
    max_bit_rate=N/A
    bits_per_raw_sample=N/A
    nb_frames=1293
    nb_read_frames=N/A
    nb_read_packets=N/A
    DISPOSITION:default=1
    DISPOSITION:dub=0
    DISPOSITION:original=0
    DISPOSITION:comment=0
    DISPOSITION:lyrics=0
    DISPOSITION:karaoke=0
    DISPOSITION:forced=0
    DISPOSITION:hearing_impaired=0
    DISPOSITION:visual_impaired=0
    DISPOSITION:clean_effects=0
    DISPOSITION:attached_pic=0
    TAG:language=und
    TAG:handler_name=SoundHandler
    [/STREAM]
    [FORMAT]
    filename=input.mp4
    nb_streams=2
    nb_programs=0
    format_name=mov,mp4,m4a,3gp,3g2,mj2
    format_long_name=QuickTime / MOV
    start_time=-0.023220
    duration=30.024000
    size=368644
    bit_rate=98226
    probe_score=100
    TAG:major_brand=isom
    TAG:minor_version=512
    TAG:compatible_brands=isomiso2avc1mp41
    TAG:title=FFprobe Tips
    TAG:encoder=Lavf56.15.101
    [/FORMAT]
    • From the above example, if you would only want duration=30.024000 .
    $ ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1 input.mp4
    Output: size=368644
    • If you would only want the value with no key.
    $ ffprobe -v error -show_entries format=size -of default=noprint_wrappers=1:nokey=1 input.mp4
    Output: 368644
    • Getting width and height (resolution) of video.
    $ ffprobe -v error -select_streams v:0 -show_entries stream=height,width -of csv=s=x:p=0 input.mp4
    Output: 1280x720

    FFplay Usage and Examples

    Usage

    ffplay [OPTIONS] [INPUT_FILE]

    • man ffplay — Refer to the manual.
    • ffplay -h — Shows included help.
    • ffplay  — Play the video on full screen.
    • -x -y  — Play the video with specific height and width popup windows size in center. It will automatically fill the extra space between the video area and the window area if the aspect ratio of the popup window and the video are different.
    • -noborder — Hide title bar while playing video in popup windows.
    • -[top/left/bottom/right]  — Make popup window in specific coordinate. By default, it will pop up in the center of the screen.

    You can find out more FFplay usage here.

    Basic Examples

    • Play a video file named video.mp4 in fullscreen.
    $ ffplay video.mp4
    • Play a video file in a popup window with sizes 600 widths, 600 height, and no title bar.
    $ ffplay video.mp4 -x 600 -y 600 -noborder
    • Play a video file in a popup window in the top left corner.
    $ ffplay video.mp4 -x 600 -y 600 -noborder -top 0 -left 0

     

    Conclusion

    We had covered all the basics in FFmpeg. Feel free to read their official documentation here if you want to have a deeper understanding!

    References

    About Author:

    This article is written by Han Sheng, an AI Full Stack Application Developer from Cameron Highlands, Malaysia. He has a passion for Deep Learning, Computer Vision, and also Edge Devices. He did several AI-based Web/Mobile Applications to help clients solving real-world problems. Feel free to read about him via his portfolio.

 

The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.
You can also read this article on our Mobile APP Get it on Google Play