Newsletter

Mix.install([
  {:jason, "~> 1.4"},
  {:kino, "~> 0.9", override: true},
  {:youtube, github: "brooklinjazz/youtube"},
  {:hidden_cell, github: "brooklinjazz/hidden_cell"}
])

Navigation

Home Report An Issue PicChat: Infinite ScrollLists Vs Tuples

Review Questions

Upon completing this lesson, a student should be able to answer the following questions.

• How do we use Swoosh and SendGrid to send emails?
• How do we configure environment variables?
• How do we use Oban to schedule jobs?

Overview

SendGrid

SendGrid is an email provider that allows businesses to send email communications to customers and prospects. It provides a platform for email delivery, which includes a web-based interface for managing contacts and creating and sending email campaigns, as well as APIs for integrating email functionality into applications.

Swoosh

Swoosh is an Elixir library for sending emails.The library allows you to send emails using various email providers such as Sendgrid, SMTP, Mailgun and more.

Oban

Oban is an Elixir library for running background jobs. It is a powerful and flexible job queue built on top of OTP. Oban allows you to enqueue jobs and process them in the background, which can be useful for tasks that are time-consuming or need to be run independently of the application’s main process. The jobs can be executed concurrently and can be scheduled to run at a specific time.

Mailer

By default, Phoenix defines a App.Mailer (where App is the app name) module in app/mailer.ex, which uses the Swoosh.Mailer module.

defmodule App.Mailer do
  use Swoosh.Mailer, otp_app: :app
end

The Mailer module is configured with some adapter for sending emails with Swoosh and some email provider such as SendGrid.

Follow Along: Newsletter

To learn more about sending emails with Swoosh and SendGrid we’re going to create a newsletter subscription page.

You can see the Newsletter Demonstration Project if you need to refer to the complete project.

Create SendGrid Account

To use SendGrid with Swoosh, we’ll need a SendGrid API Token.

To get a SendGrid API Token, complete the following steps.

1. Sign up for a Free SendGrid Account.
2. Create a Single Sender. This will be the configuration used for sending emails and receiving replies.
3. verify your sender identity through the confirmation email.
4. Set up MFA (Multi-Factor Authentication)
5. Create an API key. You can create a Full Access key if you would like, but it’s safer to create a Restricted Access key with the “Mail Send” permission. Make sure to save your API key someplace safe where others will not be able to view it.

Upon completing the above, review your sender to ensure they have been successfully set up.

Create Newsletter Project

Create the newsletter project from your command line.

$ mix phx.new newsletter

Create the database.

$ mix ecto.create

Initialize git in the project folder.

$ git init

Install Hackney, which is a required dependency for swoosh. Add the following to your deps/0 in mix.exs. Ensure you’re using the latest version.

{:hackney, "~> 1.18"}

Send A Development Email

Build The Email

We’re going to use the IEx shell to send an email. First, create a newsletter/emails.ex file with the following content. This will build our email.

Make sure you set the @sender_email and @sender_name to use the name and email for the sender you created with SendGrid.

defmodule Newsletter.Emails do
  import Swoosh.Email

  @sender_email "senders@sender_domain.com"
  @sender_name "sender name"

  def welcome(user) do
    new()
    |> to({user.name, user.email})
    |> from({@sender_name, @sender_email})
    |> subject("Welcome to the DockYard Academy Newsletter")
    |> html_body("
Hello #{user.name}, welcome to the DockYard Academy Newsletter
")
    |> text_body("Hello #{user.name}\n, welcome to the DockYard Academy Newsletter")
  end
end

In the code above, we use functions from Swoosh.Email to build the Email Fields including who to send the email to, who to send the email from, the subject, and the html_body content with a text_body fallback if HTML does not work.

This builds a Swoosh.Email struct we’ll use with our applications Mailer module to deliver the email.

Run the server from the IEx shell.

$ iex -S mix phx.server

Then try calling the welcome/1 function with a user map. You’ll see the resulting Swoosh.Email struct.

iex> email = Newsletter.Emails.welcome(%{name: "example name", email: "example@example.com"})  
%Swoosh.Email{
  subject: "Welcome to the DockYard Academy Newsletter",
  from: {"sender name", "senders@sender_domain.com"},
  to: [{"example name", "example@example.com"}],
  cc: [],
  bcc: [],
  text_body: "Hello example name\n, welcome to the DockYard Academy Newsletter",
  html_body: "
Hello example name, welcome to the DockYard Academy Newsletter
",
  attachments: [],
  reply_to: nil,
  headers: %{},
  private: %{},
  assigns: %{},
  provider_options: %{}
}

To deliver this email, we can use the deliver/2 function defined for us under the hood in our Newsletter.Mailer module.

iex> Newsletter.Mailer.deliver(email)
{:ok, %{id: "9a4f1966eab157432f11b7a1a6a00d41"}}

Visit http://localhost:4000/dev/mailbox/ to see your sent email. This route is defined in router.ex, and makes it easier to debug email in our development environment.

You can find the following code in router.ex.

# Enables The Swoosh Mailbox Preview In Development.
#
# Note That Preview Only Shows Emails That Were Sent By The Same
# Node Running The Phoenix Server.
if Mix.env() == :dev do
  scope "/dev" do
    pipe_through :browser

    forward "/mailbox", Plug.Swoosh.MailboxPreview
  end
end

Configure The Email Adapter

By default, our application uses a local development-only adapter that does not actually send emails, but instead displays them in the development mailbox for debugging purposes.You can find this configuration in config.exs.

# For Production It's Recommended To Configure A Different Adapter
# At The `config/runtime.exs`.
config :newsletter, Newsletter.Mailer, adapter: Swoosh.Adapters.Local

# Swoosh API Client Is Needed For Adapters Other Than SMTP.
config :swoosh, :api_client, false

In production, we want to actually send emails. To send emails, we have to configure Swoosh with an email provider such as SendGrid.

Normally we would configure this in runtime.ex to only run in the :prod environment. You can find an example in runtime.ex.

# ## Configuring The Mailer
#
# In Production You Need To Configure The Mailer To Use A Different Adapter.
# Also, You May Need To Configure The Swoosh API Client Of Your Choice If You
# Are Not Using SMTP. Here Is An Example Of The Configuration:
#
# Config :newsletter, Newsletter.Mailer,
# Adapter: Swoosh.Adapters.Mailgun,
# Api_key: System.get_env("MAILGUN_API_KEY"),
# Domain: System.get_env("MAILGUN_DOMAIN")
#
# For This Example You Need Include A HTTP Client Required By Swoosh API Client.
# Swoosh Supports Hackney And Finch Out Of The Box:
#
# Config :swoosh, :api_client, Swoosh.ApiClient.Hackney
#
# See Https://hexdocs.pm/swoosh/Swoosh.html#module-installation For Details.

For demonstration purposes only, we’re going to add the configuration to config.ex so that we can manually test sending emails. You should not do this in real systems, as you risk sending production emails in your development environment.

Comment out your development configuration, and add the following config that configures our Newsletter.Mailer module with the Swoosh.Adapters.Sendgrid adapter.

We should always protect API keys, so we’ll use the SENDGRID_API_KEY environment variable to avoid putting any API keys in our git history.

# For Production It's Recommended To Configure A Different Adapter
# At The `config/runtime.exs`.
# Config :newsletter, Newsletter.Mailer, Adapter: Swoosh.Adapters.Local

# Swoosh API Client Is Needed For Adapters Other Than SMTP.
# Config :swoosh, :api_client, False

config :newsletter, Newsletter.Mailer,
  adapter: Swoosh.Adapters.Sendgrid,
  api_key: System.get_env("SENDGRID_API_KEY")

config :swoosh, :api_client, Swoosh.ApiClient.Hackney

Now, let’s start the server again (make sure to stop it first) and provide the SENDGRID_API_KEY environment variable with the API key you created earlier in SendGrid. Replace YOUR_API_KEY with the API key.

$ SENDGRID_API_KEY=YOUR_API_KEY iex -S mix phx.server

This defines the SENDGRID_API_KEY environment variable retrieved by System.get_env/2.

Send A Production Email

Now send an email from the IEx shell. Replace example@example.com with your email address so that you can verify it’s sent.

iex> email = Newsletter.Emails.welcome(%{name: "example name", email: "example@example.com"})
iex> Newsletter.Mailer.deliver(email)

You can view sent emails on the SendGrid activity feed.

Signup Users

Now that we’ve proven we can send production emails, let’s create a form for users to to subscribe to our newsletter. We’ll let users sign up using their name and email.

$ mix phx.gen.html Subscribers Subscriber subscribers name:string email:string:unique

Run migrations.

$ mix ecto.migrate

Add our routes in router.ex.

scope "/", NewsletterWeb do
  pipe_through :browser

  get "/", PageController, :index
  resources "/subscribers", SubscriberController
end

Visit http://localhost:4000/subscribers/new and you should see the subscriber form.

Send New Subscriber Welcome Email

Whenever we create a new newsletter subscriber we want to send them a welcome email.

We can do this whenever we successfully create a subscriber from the controller. This prevents sending users an email if they did not successfully join our subscriber list.

Send an email in subscriber_controller.ex.

  def create(conn, %{"subscriber" => subscriber_params}) do
    case Subscribers.create_subscriber(subscriber_params) do
      {:ok, subscriber} ->
        # Welcome the user when they sign up.
        Newsletter.Emails.welcome(subscriber)
        |> Newsletter.Mailer.deliver()

        conn
        |> put_flash(:info, "Subscriber created successfully.")
        |> redirect(to: Routes.subscriber_path(conn, :show, subscriber))

      {:error, %Ecto.Changeset{} = changeset} ->
        render(conn, "new.html", changeset: changeset)
    end
  end

Environment Variables

It’s fairly tedious to include the SENDGRID_API_KEY every time we start the Phoenix server. Instead, we can store or private environment configuration in files ignored by git. This keeps our secret values protected while still being convenient.

There are two main methods for private environment variables that we’ll cover.

1. Create a .env file with the environment variables and load them into your environment.
2. Create a *.secrets.exs file with the configuration.

We’re going to use a .env file.

Create a .env file in the top level newsletter project folder with the following content. Replace "SENDGRID_API_KEY" with your API key in double quotes.

export SENDGRID_API_KEY="SENDGRID_API_KEY"

Use source to load the environment variable.

$ source .env

Make sure to ignore this file in your .gitignore, to ensure you don’t commit your API key to git.

.env

It’s common practice to create an env_template file in the top level project folder with obfuscated (removed) environment variables. This is useful to let other developers know they need certain API keys if they want to use the project.

Create an env_template file with the following content.

export SENDGRID_API_KEY="SENDGRID_API_KEY"

Scheduling Emails

In our demonstration, we send an email upon signing up to the newsletter. However, it’s common in many production systems to schedule emails on some kind of interval, on a specific date, or after a given amount of time.

To learn more about scheduling emails, we’ll use the popular Oban library to send subscribers a daily email.

Configure Oban

Follow the Oban Installation Instructions to add Oban to your project.

We’ll outline the steps here.

First, add oban to your list of dependencies in mix.exs. Make sure your version is up to date.

{:oban, "~> 2.14"}

Configure Oban in config.exs.

config :newsletter, Oban,
  repo: Newsletter.Repo,
  plugins: [Oban.Plugins.Pruner],
  queues: [default: 10]

Add Oban testing configuration in test.exs.

# Oban
config :newsletter, Oban, testing: :manual

Add Oban to your application’s supervision tree.

def start(_type, _args) do
  children = [
    # Start the Ecto repository
    Newsletter.Repo,
    # Start the Telemetry supervisor
    NewsletterWeb.Telemetry,
    # Start the PubSub system
    {Phoenix.PubSub, name: Newsletter.PubSub},
    # Start the Endpoint (http/https)
    NewsletterWeb.Endpoint,
    # Start a worker by calling: Newsletter.Worker.start_link(arg)
    # {Newsletter.Worker, arg},
    {Oban, Application.fetch_env!(:newsletter, Oban)} # Added Oban
  ]

  # See https://hexdocs.pm/elixir/Supervisor.html
  # for other strategies and supported options
  opts = [strategy: :one_for_one, name: Newsletter.Supervisor]
  Supervisor.start_link(children, opts)
end

Worker

Oban let’s use create workers that perform work in a scheduled amount of time.

Create a newsletter/workers/daily_newsletter.ex worker with the following content.

defmodule Newsletter.Workers.DailyNewsletter do
  use Oban.Worker, queue: :default, max_attempts: 10

  @impl true
  def perform(%{args: user}) do
    IO.inspect(user, label: "PERFORMING JOB")
    :ok
  end
end

Right now this worker is basically empty, but it’s going to contain the logic for sending an email every day.

First, let’s see how workers..work. Open the IEx shell and enter the following to create a job we’ll send in 5 seconds.

iex> job_changeset = Newsletter.Workers.DailyNewsletter.new(%{name: "example name", email: "example@domain.com"})

To actually schedule this job, we call Oban.insert!/3.

iex> Oban.insert!(job_changeset)

After five seconds you should see the printed inspect value.

PERFORMING JOB: %{"email" => "example", "name" => "example"}

Now when we schedule the job, we want to send an email. We could write some code to create a custom daily email, but that’s beyond the scope of this demonstration. Instead, we’ll re-use the welcome email.

Replace the perform/1 function with the following.

def perform(%{args: user}) do
  Newsletter.Emails.welcome(user)
  |> Newsletter.Mailer.deliver()
end

Daily Worker

We’re able to schedule an email in a certain amount of time. Now instead of scheduling a single job, we want to schedule a job every day. To send the same job every day, we can use a Recursive Job which schedules itself.

Oban automatically retries failed attempts, so we don’t want to schedule another email in a day, otherwise this could result in duplicates.

flowchart
ScheduleDay1 --> Attempt1 --> S2[ScheduleDay2]
ScheduleDay1 --> Attempt2 --> S1[Duplicate ScheduleDay2]

Modify your daily_newsletter.ex with the following.

defmodule Newsletter.Workers.DailyNewsletter do
  use Oban.Worker, queue: :default, max_attempts: 10

  @day 60 * 60 * 24

  # schedule tomorrow's job if it is the first attempt
  @impl true
  def perform(%{args: user, attempt: 1}) do
    # schedule job in one day
    user
    |> new(schedule_in: @day)
    |> Oban.insert!()

    Newsletter.Emails.welcome(user)
    |> Newsletter.Mailer.deliver()
  end

  # do not schedule tomorrow's job if it is any other attempt
  def perform(%{args: user}) do
    Newsletter.Emails.welcome(user)
    |> Newsletter.Mailer.deliver()
  end
end

If you want to verify this works, change the @day value to a smaller number such as every 5 seconds and add an IO.inspect/2 inside of the perform functions. Then, run the following in the IEx shell. Change the name and email to your own name and email.

iex> Newsletter.Workers.DailyNewsletter.new(%{name: "example name", email: "example@domain.com"}) |> Oban.insert!(job_changeset)

Scheduling Daily Email Upon Subscribing.

Now, we want to schedule this recurring email when a user signs up. Add the following in subscriber_controller.ex.

  def create(conn, %{"subscriber" => subscriber_params}) do
    case Subscribers.create_subscriber(subscriber_params) do
      {:ok, subscriber} ->
        # send the DailyNewsLetter immediately, then schedule recurring job.
        Newsletter.Workers.DailyNewsletter.new(%{name: subscriber.name, email: subscriber.email})        
        |> Oban.insert!()

        conn
        |> put_flash(:info, "Subscriber created successfully.")
        |> redirect(to: Routes.subscriber_path(conn, :show, subscriber))

      {:error, %Ecto.Changeset{} = changeset} ->
        render(conn, "new.html", changeset: changeset)
    end

Now whenever a user subscribes to the newsletter, they will receive a daily email!

Visit http://localhost:4000 to subscribe a user to the daily email. Keep in mind, this will only schedule jobs when our local application is running, so you don’t need to worry about sending yourself a daily email.

Move Configuration

We don’t want to send emails in development, so revert back to using the Local Swoosh adapter in config.exs.

# For Production It's Recommended To Configure A Different Adapter
# At The `config/runtime.exs`.
config :newsletter, Newsletter.Mailer, adapter: Swoosh.Adapters.Local

# Swoosh API Client Is Needed For Adapters Other Than SMTP.
config :swoosh, :api_client, false

Then configure Swoosh with SendGrid in runtime.exs. It should be towards the bottom with the following comment.

# ## Configuring The Mailer
#
# In Production You Need To Configure The Mailer To Use A Different Adapter.
# Also, You May Need To Configure The Swoosh API Client Of Your Choice If You
# Are Not Using SMTP. Here Is An Example Of The Configuration:
#
# Config :newsletter, Newsletter.Mailer,
# Adapter: Swoosh.Adapters.Mailgun,
# Api_key: System.get_env("MAILGUN_API_KEY"),
# Domain: System.get_env("MAILGUN_DOMAIN")
#
# For This Example You Need Include A HTTP Client Required By Swoosh API Client.
# Swoosh Supports Hackney And Finch Out Of The Box:
#
# Config :swoosh, :api_client, Swoosh.ApiClient.Hackney
#
# See Https://hexdocs.pm/swoosh/Swoosh.html#module-installation For Details.
config :newsletter, Newsletter.Mailer,
  adapter: Swoosh.Adapters.Sendgrid,
  api_key: System.get_env("SENDGRID_API_KEY")

config :swoosh, :api_client, Swoosh.ApiClient.Hackney

Removing Unused Files (Optional)

By using the generators, we created resources you don’t need. For example, there are many unused actions in the SubscriberController and we no longer need the Page resource.

If you would like to use this project for your resume, you might consider removing any unused files to clean up the project. You may also wish to have the subscriber routes be under the base url "/" instead of "/subscribers" to improve the UX of the project. You may have to modify tests and template files to resolve changing or removing routes.

Clean up the project code and ensure all tests pass.

mix test

Connect To GitHub

This newsletter can serve as a project for your future portfolio.

Create a Repository on GitHub and follow the instructions to connect your local project to the remote repository.

Further Reading

Consider the following resource(s) to deepen your understanding of the topic.

• HexDocs: Oban
• HexDocs: Swoosh

Commit Your Progress

DockYard Academy now recommends you use the latest Release rather than forking or cloning our repository.

Run git status to ensure there are no undesirable changes. Then run the following in your command line from the curriculum folder to commit your progress.

$ git add .
$ git commit -m "finish Newsletter reading"
$ git push

We’re proud to offer our open-source curriculum free of charge for anyone to learn from at their own pace.

We also offer a paid course where you can learn from an instructor alongside a cohort of your peers. We will accept applications for the June-August 2023 cohort soon.

Navigation

Home Report An Issue PicChat: Infinite ScrollLists Vs Tuples