Membrane intro

Logger.configure(level: :info)

Mix.install([
  :membrane_core,
  :membrane_hackney_plugin,
  :membrane_mp3_mad_plugin,
  :membrane_wav_plugin,
  :membrane_file_plugin
])

Membrane intro

Membrane

• Targets mostly live streaming
• API similar to GStreamer
• Written in Elixir
• Modular

Membrane - goals

• To make work with multimedia a more pleasant experience than it is now.
• To provide a welcoming ecosystem for learning multimedia development.
• To power resilient, maintainable and scalable systems.

Membrane - links

• website: membrane.stream
• github: github.com/membraneframework
• docs: hexdocs.pm, for example hexdocs.pm/membrane_core/0.12.9

Pipelines

graph LR
HTTP_client --> MP3_decoder --> WAV_serializer --> File

Task 1

Let’s create a pipeline spec in the handle_init callback below, to reflect the graph above. The spec can be created the following way:

• An element can be spawned with the child function:

  • child(SomeElement) to just spawn SomeElement
  • child(%SomeElement{option: value}) to spawn SomeElement and pass an option to it

• The elements can be linked together into a pipeline with the pipe |> operator, for example:

  child(SomeElement) |> child(AnotherElement)

The pipeline should contain the following elements:

Membrane.Hackney.Source with the following options: location: mp3_url, hackney_opts: [follow_redirect: true], linked to Membrane.MP3.MAD.Decoder, linked to Membrane.WAV.Serializer, linked to Membrane.File.Sink with option location: "~/output.wav".

defmodule MyPipeline do
  use Membrane.Pipeline

  @impl true
  def handle_init(_ctx, mp3_url) do
    spec =
      child(%Membrane.Hackney.Source{location: mp3_url, hackney_opts: [follow_redirect: true]})
      |> child(Membrane.MP3.MAD.Decoder)
      |> child(Membrane.WAV.Serializer)
      |> child(%Membrane.File.Sink{location: "~/output.wav"})

    {[spec: spec], %{}}
  end
end

mp3_url =
  "https://raw.githubusercontent.com/membraneframework/membrane_demo/master/simple_pipeline/sample.mp3"

{:ok, _supervisor, pipeline} = Membrane.Pipeline.start_link(MyPipeline, mp3_url)

Membrane.Pipeline.terminate(pipeline)

Element

Task 2

Let’s fill the handle_process callback in the module below, so that it changed the audio volume according to the value of gain. It should:

• Split buffer’s payload into 3-byte chunks, so that each contains a single sample, with Bunch.Binary.chunk_every/2
• Use Enum.map/2 to transform samples. For each sample:

  • Use pattern matching to decode the sample from the binary to an integer:

    <> = sample

  • Multiply the sample value by state.gain.

  • Use analogical syntax to encode the sample back to the binary form.

• Use Enum.join/2 to concatenate samples into a single binary.

defmodule VolumeKnob do
  @moduledoc """
  Membrane filter that changes the audio volume
  by the gain passed via options.
  """
  use Membrane.Filter

  alias Membrane.RawAudio

  def_input_pad(:input, accepted_format: %RawAudio{sample_format: :s24le}, flow_control: :auto)
  def_output_pad(:output, accepted_format: %RawAudio{sample_format: :s24le}, flow_control: :auto)

  def_options(
    gain: [
      spec: float(),
      description: """
      The factor by which the volume will be changed.

      A gain smaller than 1 reduces the volume and gain
      greater than 1 increases it.
      """
    ]
  )

  @impl true
  def handle_init(_ctx, options) do
    {[], %{gain: options.gain}}
  end

  @impl true
  def handle_process(:input, buffer, _ctx, state) do
    sample_size_bytes = 3

    payload =
      buffer.payload

    # |> ...
    # implement volume changing here

    buffer = %Membrane.Buffer{buffer | payload: payload}
    {[buffer: {:output, buffer}], state}
  end
end

Task 3

Let’s put the new element into our pipeline. It operates on the raw audio, so it should be linked right after the MP3 decoder. Set the gain option to some reasonable value, for example 0.2. Run the pipeline again and check if it plays quieter.

Task 4

Let’s try a more complex example: a server that accepts RTMP and outputs HLS. Please follow the instructions here.