ffmpeg: an introduction

Speaker: Fabrizio A. Vitale

ffmpeg: an introduction

Speaker: Fabrizio A. Vitale

Who am I?

Fabrizio A. Vitale

Frontend developer engineer & lead developer at Deltatre.

Contacts

Github

What's the talk about?

a gentle introduction to ffmpeg
ffplay
ffprobe
ffmpeg (CLI app)
a few practical examples
how to use ffmpeg with JS

Disclaimer

No birds were harmed in the making of this talk.

What's ffmpeg?

FFmpeg is the leading multimedia framework, able to decode, encode, transcode, mux, demux, stream, filter and play pretty
much anything that humans and machines have created.
ffmpeg website

A powerful, quick and relatively intuitive suite of CLI tools
to manipulate audio and videos.
Me

A bit of history

The ffmpeg project started in 2000 by Fabrice Bellard.

It was led by Michael Niedermayer from 2004 until 2015.

It's opensource and is published under the LGPL-2.1-or-later or GPL-2.0-or-later.

ffmpeg today

ffmpeg is widely used for basic and advanced video and audio editing.

It is used by software such as Videolan (VLC), Blender, Youtube and Google Chrome.

It's available for Mac, Windows and Linux.

ffmpeg is a suite of tools

After downloading ffmpeg, you'll have 3 executables installed on your machine.

ffplay /1

a simple CLI media player.

syntax

ffplay [options] input_file
ffplay [options] input_file

example

ffplay -ss 24   public/media/big_buck_bunny.mp4
ffplay -ss 24   public/media/big_buck_bunny.mp4

ffplay /2

ffprobe /1

Simple multimedia streams analyzer.

ffprobe -hide_banner public/media/big_buck_bunny.mp4
ffprobe -hide_banner public/media/big_buck_bunny.mp4

-hide_banner Hides verbose banner.
public/media/big_buck_bunny.mp3 file to analyze.

ffprobe /2

ffmpeg /1

Hyper fast Audio and Video encoder.

ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...
ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

ffmpeg
[options]
[[infile options] -i infile]
{[outfile options] outfile}
ffmpeg
[options]
[[infile options] -i infile]
{[outfile options] outfile}

ffmpeg /2

What can you do with ffmpeg?

Let's take a look at some examples.

How to change video format? /1

mp4 (3.8MB) -> webm (2.2MB)

ffmpeg -y -i public/media/big_buck_bunny.mp4 \
-vcodec libvpx -acodec libvorbis public/media/big_buck_bunny.webm
ffmpeg -y -i public/media/big_buck_bunny.mp4 \
-vcodec libvpx -acodec libvorbis public/media/big_buck_bunny.webm

-y overwrite output files.
-i public/media/big_buck_bunny.mp4 input file.
-vcodec libvpx set video codec.
-acodec libvorbis set audio codec.
public/media/big_buck_bunny.webm output file.

How to change video format? /2

Input (.mp4)

Output (.webm)

How to extract the audio from a video? / 1

ffmpeg -y -i public/media/big_buck_bunny.mp4 public/media/big_buck_bunny.mp3
ffmpeg -y -i public/media/big_buck_bunny.mp4 public/media/big_buck_bunny.mp3

-y overwrite output files.
-i public/media/big_buck_bunny.mp4 input file.
public/media/big_buck_bunny.mp3 output file.

How to extract the audio from a video? / 2

Input (.mp4)

Output (.mp3)

How to create a gif from a video? /1

ffmpeg -y -ss 24 -t 3 \
-i public/media/big_buck_bunny.mp4 public/media/big_buck_bunny.gif
# or
ffmpeg -y -ss 24 -to 27 \
-i public/media/big_buck_bunny.mp4 public/media/big_buck_bunny.gif
ffmpeg -y -ss 24 -t 3 \
-i public/media/big_buck_bunny.mp4 public/media/big_buck_bunny.gif
# or
ffmpeg -y -ss 24 -to 27 \
-i public/media/big_buck_bunny.mp4 public/media/big_buck_bunny.gif

-y overwrite output files.
-ss 24 | -ss 00:00:24 seek to 24th second.
-t 3 set duration to 3 seconds.
-to 27 | -ss 00:00:27 end video at the 27th second.

How to create a gif from a video? /2

Input

Output

Issues

The output gif is quite big and the framerate is off.

ls -lh public/media/big_buck_bunny.gif  | awk -F " " {'print $5'}
1.4M
ls -lh public/media/big_buck_bunny.gif  | awk -F " " {'print $5'}
1.4M

Can we do better?

Let’s lower the frame rate and scale the gif /1

ffmpeg -y \
-ss 24 -t 3 -i public/media/big_buck_bunny.mp4 \
-vf "fps=12,scale=320:-1" \
public/media/big_buck_bunny_fps_scale.gif
ffmpeg -y \
-ss 24 -t 3 -i public/media/big_buck_bunny.mp4 \
-vf "fps=12,scale=320:-1" \
public/media/big_buck_bunny_fps_scale.gif

-y overwrite output files.
-vf 'fps=12' Sets output framerate to 12fps.
-vf "scale=320:-1" Scales width to 320px while preserving the aspect ratio.

Let’s lower the frame rate and scale the gif /2

ls -lh public/media/big_buck_bunny.gif  | awk -F " " {'print $5'}
1.4M # before

ls -lh public/media/big_buck_bunny_fps_scale.gif  | awk -F " " {'print $5'}
184K # after
ls -lh public/media/big_buck_bunny.gif  | awk -F " " {'print $5'}
1.4M # before

ls -lh public/media/big_buck_bunny_fps_scale.gif  | awk -F " " {'print $5'}
184K # after

Before

After

Take a screenshot of a video /1

ffmpeg -y \
-ss 00:00:24.82 \
-i public/media/big_buck_bunny.mp4 \
-frames:v 1 \
public/media/big_buck_bunny_pic.jpg
ffmpeg -y \
-ss 00:00:24.82 \
-i public/media/big_buck_bunny.mp4 \
-frames:v 1 \
public/media/big_buck_bunny_pic.jpg

-y overwrite output files.
-ss 00:00:24.82 seek input file to 24th second.
-frames:v 1 set the number of frames to output 1.

Take a screenshot of a video /2

Input

Output

Is it possible to run ffmpeg with JS?

Of course!

Node

// run with "node examples/node/extract-audio.mjs --input  public/media/big_buck_bunny.mp4"
import { spawn } from "node:child_process";
import { z } from "zod";
import process from "node:process";
import parse from "yargs-parser";

function extractAudio(params) {
  const parsedArgs = z.object({ input: z.string().min(3) }).parse(params);
  const childProcess = spawn(
    "ffmpeg",
    ["-y", "-i", parsedArgs.input, `${parsedArgs.input}-node.mp3`],
    { stdio: "inherit", shell: true }
  );

  childProcess.on("error", (err) => { console.log(err); process.exitCode = 1; });
  childProcess.on("close", (code) => {
    if (code) {
      console.error(`ffmpeg exit code=${code}`);
      process.exitCode = 1;
    }
  });
}

extractAudio(parse(process.argv.slice(2), { string: ["input"] }));
// run with "node examples/node/extract-audio.mjs --input  public/media/big_buck_bunny.mp4"
import { spawn } from "node:child_process";
import { z } from "zod";
import process from "node:process";
import parse from "yargs-parser";

function extractAudio(params) {
  const parsedArgs = z.object({ input: z.string().min(3) }).parse(params);
  const childProcess = spawn(
    "ffmpeg",
    ["-y", "-i", parsedArgs.input, `${parsedArgs.input}-node.mp3`],
    { stdio: "inherit", shell: true }
  );

  childProcess.on("error", (err) => { console.log(err); process.exitCode = 1; });
  childProcess.on("close", (code) => {
    if (code) {
      console.error(`ffmpeg exit code=${code}`);
      process.exitCode = 1;
    }
  });
}

extractAudio(parse(process.argv.slice(2), { string: ["input"] }));

Node (ffmpeg.wasm)

// run with "node examples/node/extract-audio-wasm.mjs --input public/media/big_buck_bunny.mp4"
import { z } from "zod";
import process from "node:process";
import parse from "yargs-parser";
import path from "node:path";
import fs from 'node:fs';
import { createFFmpeg, fetchFile } from '@ffmpeg/ffmpeg';

const ffmpeg = createFFmpeg({ log: true });

async function extractAudio(params) {
  const parsedArgs = z.object({ input: z.string().min(3) }).parse(params);
  const inputBasename = path.basename(parsedArgs.input)
  const outputFilepath = `${parsedArgs.input}-node-wasm.mp3`
  const outputBasename = path.basename(outputFilepath)

  await ffmpeg.load();
  ffmpeg.FS('writeFile', inputBasename, await fetchFile(parsedArgs.input));
  await ffmpeg.run('-i', inputBasename, outputBasename);
  await fs.promises.writeFile(outputFilepath, ffmpeg.FS('readFile', outputBasename));
}

await extractAudio(parse(process.argv.slice(2), { string: ["input"] }));
process.exit(0); // You have to force the exit with ffmpeg.wasm
// run with "node examples/node/extract-audio-wasm.mjs --input public/media/big_buck_bunny.mp4"
import { z } from "zod";
import process from "node:process";
import parse from "yargs-parser";
import path from "node:path";
import fs from 'node:fs';
import { createFFmpeg, fetchFile } from '@ffmpeg/ffmpeg';

const ffmpeg = createFFmpeg({ log: true });

async function extractAudio(params) {
  const parsedArgs = z.object({ input: z.string().min(3) }).parse(params);
  const inputBasename = path.basename(parsedArgs.input)
  const outputFilepath = `${parsedArgs.input}-node-wasm.mp3`
  const outputBasename = path.basename(outputFilepath)

  await ffmpeg.load();
  ffmpeg.FS('writeFile', inputBasename, await fetchFile(parsedArgs.input));
  await ffmpeg.run('-i', inputBasename, outputBasename);
  await fs.promises.writeFile(outputFilepath, ffmpeg.FS('readFile', outputBasename));
}

await extractAudio(parse(process.argv.slice(2), { string: ["input"] }));
process.exit(0); // You have to force the exit with ffmpeg.wasm

Deno

// run with "deno run -A examples/deno/extract-audio.ts --input  public/media/big_buck_bunny.mp4"
import { parse } from "https://deno.land/std@0.187.0/flags/mod.ts";
import { z } from "https://deno.land/x/zod@v3.21.4/mod.ts";

async function extractAudio(params: unknown) {
  const parsedArgs = z.object({ input: z.string().min(3) }).parse(params);
  const command = new Deno.Command("ffmpeg", {
    args: [
      "-y",
      "-i",
      parsedArgs.input,
      `${parsedArgs.input}-deno.mp3`,
    ],
  });
  const { code } = await command.spawn().output()

  if (code) {
    throw new Error(`ffmpeg exit-code=${code}`);
  }
}

await extractAudio(parse(Deno.args, { string: ["input"] }));
// run with "deno run -A examples/deno/extract-audio.ts --input  public/media/big_buck_bunny.mp4"
import { parse } from "https://deno.land/std@0.187.0/flags/mod.ts";
import { z } from "https://deno.land/x/zod@v3.21.4/mod.ts";

async function extractAudio(params: unknown) {
  const parsedArgs = z.object({ input: z.string().min(3) }).parse(params);
  const command = new Deno.Command("ffmpeg", {
    args: [
      "-y",
      "-i",
      parsedArgs.input,
      `${parsedArgs.input}-deno.mp3`,
    ],
  });
  const { code } = await command.spawn().output()

  if (code) {
    throw new Error(`ffmpeg exit-code=${code}`);
  }
}

await extractAudio(parse(Deno.args, { string: ["input"] }));

Conclusions /1

What we've shown in this talk is just the tip of the iceberg.

ffmpeg is a tool with seemingly endless possibilities.

Conclusions /2

So... are you ready to go down the rabbit hole?

Thank you!

Slides

Speaker

Fabrizio A. Vitale