Getting Started with Fred

Mix.install([
  {:fred, "~> 0.4.0"},
  {:vega_lite, "~> 0.1"},
  {:kino_vega_lite, "~> 0.1"}
])

Introduction

FRED® (Federal Reserve Economic Data) provides access to over 800,000 economic time series from 100+ sources including the Bureau of Labor Statistics, the Bureau of Economic Analysis, and the Federal Reserve Board. This library was written to allow readers of Elixir For Finance to collect, analyze and visualize economic data from Fred, but it is a complete Fred API client and can be used outside the context of the book.

To learn how you can analyze and visualize the financial markets using Livebook, Explorer, Scholar and Nx, be sure to pick up a copy of our book:

Setup

To being, start by installing the notebook dependencies. This notebook uses the fred library for API access and VegaLite for charting.

You’ll need a FRED API key before making any API calls. You can get your free API key from the FRED API website. After you have a FRED API key, add it to your Livebook secrets undet the key FRED_API_KEY so the the downstream code can access it.

With your API key in place, you can set the application configuration for the Fred library and attach the default logger.

alias VegaLite, as: Vl

# API key pulled from Livebook secrets
Application.put_env(:fred, :api_key, System.fetch_env!("LB_FRED_API_KEY"))

# Attach the default logger to keep an eye on requests
Fred.Telemetry.Logger.attach(level: :info)

With that you are ready to rock and roll! Let’s now take a look at some simple endpoints to see what data we can extract from FRED.

Exploring a Series

We’ll begin by looking up metadata for the Unemployment Rate series (which has the key UNRATE). The meta data will include things like the frequency of the series, the human readable name, and the units of the series.

# We use the `Fred.Series.get/1` function to get metadata about the `UNRATE` series and pluck
# the head of the response list.
{:ok, %{"seriess" => [data | _]}} =
  Fred.Series.get("UNRATE")

# Print out some of the metadata from the series
IO.puts("""
Title:       #{data["title"]}
Frequency:   #{data["frequency"]}
Units:       #{data["units"]}
Seasonal:    #{data["seasonal_adjustment"]}
Last Update: #{data["last_updated"]}
""")

:ok

Fetching Observations

Now let’s pull the actual unemployment rate data from 2000-01-01 onward. We’ll work with the raw response data from the FRED API and massage it into a format that works with VegaLite.

# Get the observation data for the `UNRATE` series after (and including) 2000-01-01
observation_start = ~D[2000-01-01]

{:ok, %{"observations" => observations}} =
  Fred.Series.observations("UNRATE",
    observation_start: observation_start,
    frequency: :m
  )

# Parse into a list of maps, extract the necessary data and cast the float values. FRED returns
# "." for dates without any data so we'll ignore those.
formatted_observations =
  observations
  |> Enum.reduce([], fn
    %{"value" => "."}, acc ->
      acc

    %{"date" => date, "value" => value}, acc ->
      [%{date: Date.from_iso8601!(date), value: String.to_float(value)} | acc]
  end)
  |> Enum.sort_by(
    fn %{date: date} ->
      date
    end,
    Date
  )

# Get the min and max Dates for the series
{%{date: min_date}, %{date: max_date}} = Enum.min_max_by(formatted_observations, &(&1.date), Date)

# Print some information about the series
IO.puts("Fetched #{length(formatted_observations)} observations")
IO.puts("Date range: #{min_date} to #{max_date}")

# Preview the first few rows
Enum.take(formatted_observations, 5)

With the UNRATE data formatted nicely inside of the observations variable, let’s now see how we can plot this data using the VegaLite library inside of Livebook.

Plotting with VegaLite

With our unemployment data now formatted as a list of maps with the date and value, we can now plot the simple time series data using a VegaLite line chart:

Vl.new(width: 750, height: 400, title: "U.S. Unemployment Rate (#{min_date} - #{max_date})")
|> Vl.data_from_values(formatted_observations)
|> Vl.mark(:line, tooltip: true, color: "#2563eb")
|> Vl.encode_field(:x, "date",
  type: :temporal,
  title: "Date",
  axis: [format: "%Y"]
)
|> Vl.encode_field(:y, "value",
  type: :quantitative,
  title: "Unemployment Rate (%)",
  scale: [zero: false]
)

Adding Recession Shading

FRED provides the National Bureau of Economic Research (NBER for short) recession indicator as series "USREC". The data value is a 1 during a recession and 0 otherwise. We’ll convert this time series data set into a list of tuples to mark the beginning and end of recession periods which we’ll then use to overlay on top of the unemployment data in VegaLite.

# Fetch the recession indicator for the same date range
{:ok, %{"observations" => reccession_data}} =
  Fred.Series.observations("USREC",
    observation_start: observation_start,
    frequency: :m
  )

# Convert monthly 0/1 flags into recession periods: [{start, end}, ...]
recession_periods =
  reccession_data
  |> Enum.reject(fn %{"value" => value} ->
    value == "."
  end)
  |> Enum.chunk_while(
    nil,
    fn
      %{"value" => "1", "date" => date}, nil ->
        {:cont, date}

      %{"value" => "1"}, start_date ->
        {:cont, start_date}

      %{"value" => "0"}, nil ->
        {:cont, nil}

      %{"value" => "0", "date" => date}, start_date ->
        {:cont, %{start: Date.from_iso8601!(start_date), end: Date.from_iso8601!(date)}, nil}
    end,
    fn
      nil ->
        {:cont, nil}

      start_date ->
        {:cont, %{start: start_date, end: Date.utc_today()}, nil}
    end
  )

With the recession date ranges, aggregated, we can now create a new VegaLite chart using the VegaLite.layers/2 function in order to overlay the recession ranges along with the unemployment data.

# Layer recession bands behind the unemployment area chart
Vl.new(width: 750, height: 400, title: "U.S. Unemployment Rate with Recession Shading")
|> Vl.layers([
  # Gray recession bands
  Vl.new()
  |> Vl.data_from_values(recession_periods)
  |> Vl.mark(:rect, color: "#3f3f46", opacity: 0.25)
  |> Vl.encode_field(:x, "start", type: :temporal)
  |> Vl.encode_field(:x2, "end", type: :temporal),

  # Unemployment area
  Vl.new()
  |> Vl.data_from_values(formatted_observations)
  |> Vl.mark(:area,
    tooltip: true,
    color: "#2563eb",
    opacity: 0.5,
    line: [color: "#2563eb"]
  )
  |> Vl.encode_field(:x, "date",
    type: :temporal,
    title: "Date",
    axis: [format: "%Y"]
  )
  |> Vl.encode_field(:y, "value",
    type: :quantitative,
    title: "Unemployment Rate (%)",
    scale: [zero: true]
  )
])

Using FRED’s Built-in Transformations

The FRED API can also compute percent changes and format the response payload accordingly. You can do this via the :units option. Let’s look at month-over-month percent change in the unemployment rate.

{:ok, %{"observations" => pch_observations}} =
  Fred.Series.observations("UNRATE",
    observation_start: observation_start,
    frequency: :m,
    units: :pch
  )

formatted_pch_observations =
  pch_observations
  |> Enum.reduce([], fn
    %{"value" => "."}, acc ->
      acc

    %{"date" => date, "value" => value}, acc ->
      value = String.to_float(value)

      data = %{
        date: Date.from_iso8601!(date),
        value: value,
        direction: if(value >= 0, do: "Increase", else: "Decrease")
      }

      [data | acc]
  end)
  |> Enum.sort_by(
    fn %{date: date} ->
      date
    end,
    Date
  )

With our formatted percent change data, we can once again lean on the VegaLite.layers/2 function in order to overlay the recession periods along with the unemployment rate percent change.

Vl.new(width: 750, height: 400, title: "Unemployment Rate - Month-over-Month % Change")
|> Vl.layers([
  # Gray recession bands
  Vl.new()
  |> Vl.data_from_values(recession_periods)
  |> Vl.mark(:rect, color: "#3f3f46", opacity: 0.25)
  |> Vl.encode_field(:x, "start", type: :temporal)
  |> Vl.encode_field(:x2, "end", type: :temporal),

  # Unemployment area
  Vl.new()
|> Vl.data_from_values(formatted_pch_observations)
|> Vl.mark(:bar, tooltip: true)
|> Vl.encode_field(:x, "date",
  type: :temporal,
  title: "Date",
  axis: [format: "%Y"]
)
|> Vl.encode_field(:y, "value",
  type: :quantitative,
  title: "% Change"
)
|> Vl.encode_field(:color, "direction",
  type: :nominal,
  scale: [domain: ["Increase", "Decrease"], range: ["#ef4444", "#22c55e"]],
  legend: nil
)
])

Searching for Observation Series

In addition to fetching data for known observation series, you can search FRED’s full catalog via full text search. Let’s find the most popular series related to “inflation”.

{:ok, %{"seriess" => search_results}} =
  Fred.Series.search("inflation",
    order_by: :popularity,
    sort_order: :desc,
    limit: 5
  )

search_results
|> Enum.map(fn s ->
  %{
    id: s["id"],
    title: s["title"],
    frequency: s["frequency_short"],
    popularity: s["popularity"]
  }
end)