Video Streaming Server Using FFmpeg

livebooks/ffmpeg-streaming-server.livemd

Akash Hiremath

@akash-akya

exile

More notebooks

Video Streaming Server Using FFmpeg

Mix.install([
  {:req, "~> 0.4.5"},
  {:ex_cmd, "~> 0.10.0"},
  {:plug, "~> 1.15"},
  {:kino, "~> 0.11.3"},
  {:bandit, "~> 1.1"}
])

Introduction

Elixir and Erlang provide Port a Port for running external programs. However, communicating with these programs over stdio can lead to various issues. ExCmd and Exile libraries address these concerns. Along with that they provide beloved streaming capabilities. In otherwords they let you stream input, output all the way down.

Let’s take an example to illustrate the streaming capabilities. Here, we’ll create an HTTP server to watermark a video.

The server should perform the following:

accept request with video URL and watermark text
stream input video
adds text as watermark using FFmpeg
stream output video as response, using chunked encoding

FFmpeg

We need a sample video to test. Let’s fetch and display it using Kino

# download sample video
video_url =
  "https://upload.wikimedia.org/wikipedia/commons/transcoded/8/81/Beachfront_B-Roll-_Fireworks_%28Free_to_Use_HD_Stock_Video_Footage%29.webm/Beachfront_B-Roll-_Fireworks_%28Free_to_Use_HD_Stock_Video_Footage%29.webm.480p.vp9.webm"

tmp = System.tmp_dir!()
input_path = Path.join(tmp, "input.webm")
video = Req.get!(video_url).body
:ok = File.write!(input_path, video)

# lets display it
File.read!(input_path)
|> Kino.Video.new(:mp4, autoplay: true, muted: true)

Creating helper module to construct ffmpeg command to add watermark. The details about the command parameters are not important.

defmodule FFmpeg do
  @moduledoc """
  Returns ffmpeg command as list of string
  """

  @doc """
  Returns ffmpeg command with arguments for adding watermark
  """
  @spec watermark(String.t(), String.t(), String.t(), map()) :: [String.t()]
  def watermark(input, text, output, text_opts \\ []) do
    # add text with white color font and transparency of 0.5
    filter_graph =
      [
        text: "'#{text}'",
        fontsize: text_opts[:fontsize] || 80,
        fontcolor: "white",
        x: text_opts[:x] || 300,
        y: text_opts[:y] || 350,
        alpha: text_opts[:alpha] || 0.5
      ]
      |> Enum.map(fn {k, v} -> "#{k}=#{v}" end)
      |> Enum.join(":")

    [
      "ffmpeg",
      "-y",
      ["-i", input],
      ["-vf", "drawtext=#{filter_graph}"],
      ~w(-codec:a copy),
      # output should be MP4
      ~w(-f mp4),
      # add flag to fix error while reading the stream
      ~w(-movflags empty_moov),
      # output location
      output
    ]
    |> List.flatten()
  end
end

Let’s see it in action by giving it the sample video we have!

output_path = Path.join(tmp, "output.mp4")

["ffmpeg" | args] = cmd = FFmpeg.watermark(input_path, "Fireworks", output_path, x: 220, y: 30)

# print the constructed ffmpeg command
IO.puts(Enum.join(cmd, " "))

# run the command, this might take while
{"", 0} = System.cmd(System.find_executable("ffmpeg"), args)

File.read!(output_path)
|> Kino.Video.new(:mp4, autoplay: true, muted: true)

Streaming Output Video

The method with System.cmd works, but it makes us create and deal with temporary files unnecessarily. If you’re using the output right away and then throwing it away, a better way is to use streams. Ffmpeg can read from and write to UNIX pipes, and you can use this easily with ExCmd.

# `pipe:0` : read input from STDIN
# `-` : write output to STDOUT
cmd = FFmpeg.watermark("pipe:0", "Fireworks", "-", x: 220, y: 30)

# ExCmd returns output as lazy stream. You can also replace `ExCmd` with `Exile`
output_stream = ExCmd.stream!(cmd, input: video)

# To display the video, Kino.Video needs it as binary blob.
# So we have collect stream as binary
Enum.into(output_stream, <<>>)
|> Kino.Video.new(:mp4, autoplay: true, muted: true)

FFmpeg Server

With that, the only thing remaining is to set up the HTTP server. We utilize Plug to create a GET /watermark route. This route accepts a video URL and watermark text as query parameters and returns the output video as a stream using chunked response, maximizing the streaming capability.

Streaming Input

ExCmd facilitates input streaming by supporting both Enumerable and Collectable. Given that HTTP Client (Req) we are using supports pushing responses to Collectable, we can simply leverage that capability.

output_stream =
  ExCmd.stream!(cmd,
    input: fn ex_cmd_sink ->
      # Req pushes video input to ex_cmd as chunks
      Req.get!(video_url, into: ex_cmd_sink)
    end)

Streaming Output

Achieving streaming response can be done using Plug.Conn.chunk/2, allowing us to push chunks as soon as they become available from the ffmpeg command.

defmodule FFmpegServer do
  @moduledoc """
  HTTP server for demonstrating FFmpeg streaming
  """
  use Plug.Router
  require Logger

  plug(Plug.Parsers, parsers: [], pass: ["*/*"])
  plug(:match)
  plug(:dispatch)

  get "/watermark" do
    %{"video_url" => video_url, "text" => text} = conn.params

    cmd = FFmpeg.watermark("pipe:0", text, "-", x: 20, y: 20)
    output_stream = ExCmd.stream!(cmd, input: &amp;Req.get!(video_url, into: &amp;1))

    conn =
      conn
      |> put_resp_content_type("video/mp4")
      |> send_chunked(200)

    Enum.reduce_while(output_stream, conn, fn chunk, conn ->
      case chunk(conn, chunk) do
        {:ok, conn} ->
          Logger.debug("Sent #{IO.iodata_length(chunk)} bytes")
          {:cont, conn}

        {:error, :closed} ->
          Logger.debug("Connection closed")
          {:halt, conn}
      end
    end)
  end
end

Start the server at port 8989

webserver = {Bandit, plug: FFmpegServer, scheme: :http, port: 8989}
{:ok, supervisor} = Supervisor.start_link([webserver], strategy: :one_for_one)

Now you can test it at command line using curl

curl \
  --include \
  --get 'http://127.0.0.1:8989/watermark' \
  --data-urlencode 'text=FFmpeg Rocks!' \
  --data-urlencode 'video_url=https://upload.wikimedia.org/wikipedia/commons/transcoded/8/81/Beachfront_B-Roll-_Fireworks_%28Free_to_Use_HD_Stock_Video_Footage%29.webm/Beachfront_B-Roll-_Fireworks_%28Free_to_Use_HD_Stock_Video_Footage%29.webm.480p.vp9.webm'

You can even directly play the video stream from command line using ffplay!

curl \
  --get 'http://127.0.0.1:8989/watermark' \
  --data-urlencode 'text=FFmpeg Rocks!' \
  --data-urlencode 'video_url=https://upload.wikimedia.org/wikipedia/commons/transcoded/8/81/Beachfront_B-Roll-_Fireworks_%28Free_to_Use_HD_Stock_Video_Footage%29.webm/Beachfront_B-Roll-_Fireworks_%28Free_to_Use_HD_Stock_Video_Footage%29.webm.480p.vp9.webm' \
  | ffplay -

Notice the logs in the Livebook cell above. As you watch the video, observe how the video chunks are fetched, encoded, and pushed as responses in chunks, together, on the fly!

Another noteworthy feature is that if the connection drops midway (close ffplay using Ctrl+C), the streaming pipeline stops gracefully, avoiding the wasteful encoding of the entire video.

# stop the server
Supervisor.stop(supervisor)

Back