Would you like to see your link here? Contact us

Notesclub

created by hec & contributors

terms privacy

Book Search: Book Content

book_search_book_content.livemd

Ricardo Trejos

@cardotrejos

learning-elixir

More notebooks

Book Search: Book Content

Mix.install([
  {:kino, github: "livebook-dev/kino", override: true},
  {:kino_lab, "~> 0.1.0-dev", github: "jonatanklosko/kino_lab"},
  {:vega_lite, "~> 0.1.4"},
  {:kino_vega_lite, "~> 0.1.1"},
  {:benchee, "~> 0.1"},
  {:ecto, "~> 3.7"},
  {:math, "~> 0.7.0"},
  {:faker, "~> 0.17.0"},
  {:utils, path: "#{__DIR__}/../utils"},
  {:httpoison, "~> 1.8"},
  {:poison, "~> 5.0"}
])

Navigation

Return Home Report An Issue

Setup

Ensure you type the ea keyboard shortcut to evaluate all Elixir cells before starting. Alternatively, you can evaluate the Elixir cells as you read.

Overview

We’re going to add book contents to books in our BookSearch application. While we could simply add a :contents field in each Book, this would likely result in loading too much data when we list all of our books.

Instead, we’ll put book contents in a separate table. Each BookContents record will contain the full-text contents of the book in a :content field.

classDiagram
  class BookContents {
    book_id: :id
    content: :text
  }

Books and BookContents have a one-to-one relationship. Each BookContent record belongs to a Book. And each Book has one book.

Instead of creating a new table, we could have simply added a new :content field to each Book. However, this could result in each book having a huge amount of data, so we’ve instead chosen to put BookContent into a separate table to avoid potential performance issues.

We could still have performance issues loading an entire book into memory, but that problem goes beyond the scope of this lesson.

Typically one-to-one relationships have a parent and a child entity. In this case, the Book is the parent, and the BookContents is the child. Our Database stores the foreign key of the parent record on the child record to track the relationship.

Books Table

id	name
123	“We are Legend (We are Bob)”

Book_contents Table

id	book_id	content
456	123	…but as for me, I am tormented with an everlasting itch for things remote.

classDiagram
  direction RL
  class Book {
    title: :string
  }

  class BookContents {
    book_id: :id
    content: :text
  }

  BookContents "1" --> "1" Book :belongs_to

Run Book Search Project

Ensure you have completed the BookSearch project from the previous lesson. If not, you can clone the BookSearch project on the tags branch. If you get stuck at any point during this lesson, you can reference the existing BookSearch/book_content branch with the completed version of this project.

Before starting, all tests should pass.

$ mix test

Start the server from the book_search project folder.

$ mix phx.server

Books Contents Migration

BookContents will be closely tied to it’s parent resource. When we create or update a Book we will also create or update BookContents.

We need to create a book contents table, so let’s generate a new migration.

$ mix ecto.gen.migration create_book_contents

In the generated migration, create the book_contents table with the :content field and a reference to the books table. Both fields should be required, so we’ll add null: false.

# priv/repo/migrations/_create_book_contents.exs

defmodule BookSearch.Repo.Migrations.CreateBookContents do
  use Ecto.Migration

  def change do
    create table(:book_contents) do
      add :content, :text, null: false
      add :book_id, references(:books, on_delete: :delete_all), null: false

      timestamps()
    end

    create index(:book_contents, [:book_id])
  end
end

Run the migration.

$ mix ecto.migrate

Book Contents Schema

Now add a schema file lib/book_search/books/book_content.ex to match our previous migration.

# lib/book_search/books/book_content.ex

defmodule BookSearch.Books.BookContents do
  use Ecto.Schema
  import Ecto.Changeset

  schema "book_contents" do
    field :content, :string
    belongs_to :book, BookSearch.Books.Book

    timestamps()
  end

  @doc false
  def changeset(book, attrs) do
    book
    |> cast(attrs, [:content])
    |> validate_required([:content])
  end
end

Book Schema

We’ll use has_one/3 to define the relationship between books and book contents.

# lib/book_search/books/book.ex

defmodule BookSearch.Books.Book do
  use Ecto.Schema
  import Ecto.Changeset

  schema "books" do
    field(:title, :string)
    belongs_to(:author, BookSearch.Authors.Author)
    many_to_many(:tags, BookSearch.Tags.Tag, join_through: "book_tags", on_replace: :delete)
    has_one(:book_contents, BookSearch.Books.BookContents)

    timestamps()
  end

  @doc false
  def changeset(book, attrs) do
    book
    |> cast(attrs, [:title])
    |> validate_required([:title])
  end
end

Create Book Content

We want to create the associated book contents whenever we create a book. So let’s start with a test. This test will ensure the created book has a book_contents association with a content field.

test "create_book/1 with book content creates a book with book content" do
  author = author_fixture()
  valid_attrs = %{title: "some title", author: author, book_contents: %{content: "some content"}}

  assert {:ok, %Book{} = book} = Books.create_book(valid_attrs)
  assert book.title == "some title"
  assert book.book_contents.content == "some content"
end

We can use cast_assoc/3 to create associations from parameters. cast_assoc/3 is often useful when managing associations with external data, such as parameters coming from a form.

# lib/book_search/books.ex

def create_book(attrs \\ %{}) do
  {author, attrs} = Map.pop!(attrs, :author)
  {tags, attrs} = Map.pop(attrs, :tags, [])

  author
  |> Ecto.build_assoc(:books, attrs)
  |> Book.changeset(attrs)
  |> Ecto.Changeset.put_assoc(:tags, tags)
  |> Ecto.Changeset.cast_assoc(:book_contents)
  |> Repo.insert()
end

We may need to run migrations in our test environment.

$ MIX_ENV=test mix ecto.migrate

All tests should pass.

$ mix test
66 tests, 0 failures

Update Book Content

We need want to be able to update the associated book contents, so let’s write our test.

test "update_book/2 with book content" do
  author = author_fixture()
  book = book_fixture(author: author, book_contents: %{content: "some content"})

  update_attrs = %{
    title: "Name of the Wind",
    book_contents: %{content: "some updated content"}
  }

  assert {:ok, %Book{} = book} = Books.update_book(book, update_attrs)
  assert book.title == "Name of the Wind"
  assert book.book_contents.content == "some updated content"
end

This test fails because we do not update book contents.

test books update_book/2 with book content (BookSearch.BooksTest)
     test/book_search/books_test.exs:72
     Assertion with == failed
     code:  assert book.book_contents.content == "some updated content"
     left:  "some content"
     right: "some updated content"

Make it pass by casting the association.

# lib/book_search/books.ex
def update_book(%Book{} = book, attrs) do
  {tags, attrs} = Map.pop(attrs, :tags, [])

  book
  |> Book.changeset(attrs)
  |> Ecto.Changeset.put_assoc(:tags, tags)
  |> Ecto.Changeset.cast_assoc(:book_contents)
  |> Repo.update()
end

For the sake of completeness, let’s make a test for the controller as well that will already pass. Add this test inside the existing "update book" describe block.

# test/book_search_web/controllers/book_controller_test.exs

test "with book content", %{conn: conn, book: book} do
  tag = tag_fixture(name: "Fantasy")
  update_attrs = %{title: "Name of the Wind", }

  conn =
    put(conn, Routes.author_book_path(conn, :update, book.author_id, book), book: update_attrs)

  assert redirected_to(conn) == Routes.author_book_path(conn, :show, book.author_id, book)
  conn = get(conn, Routes.author_book_path(conn, :show, book.author_id, book))
  
  assert html_response(conn, 200) =~ tag.name
end

Managing Associations

So far, we’ve seen three different functions for managing associations.

Let’s take a moment to consider the differences.

put_assoc/4

Puts the given association as a change in the changeset.

flowchart LR
Record --> Association

It allows you to add, remove, or change all associated records at once.

Example Return Value:

#Ecto.Changeset<
  action: nil,
  changes: %{
    tags: [
      #Ecto.Changeset, valid?: true>,
      #Ecto.Changeset, valid?: true>
    ],
    title: "some title"
  },
  errors: [],
  data: #BookSearch.Books.Book<>,
  valid?: true
>

build_assoc/3

Builds a struct from the given association. Often used when we want to work with the child association of an existing parent record such as author and book. It returns a child struct with the foreign key to the parent record, such as author_id.

flowchart LR
Parent --> Child

Example Return Value:

%BookSearch.Books.Book{
  __meta__: #Ecto.Schema.Metadata<:built, "books">,
  author: #Ecto.Association.NotLoaded,
  author_id: 603,
  book_contents: #Ecto.Association.NotLoaded,
  id: nil,
  inserted_at: nil,
  tags: #Ecto.Association.NotLoaded,
  title: nil,
  updated_at: nil
}

We’ve chosen to use build_assoc/3 in the BookSearch project for demonstration purposes. However, we could have used put_assoc/4 instead.

For example, you can replace create_book/1 with the following, and tests continue to pass.

# lib/book_search/books.ex

def create_book(attrs \\ %{}) do
  {author, attrs} = Map.pop!(attrs, :author)
  {tags, attrs} = Map.pop(attrs, :tags, [])

  %Book{}
  |> Book.changeset(attrs)
  |> Ecto.Changeset.put_assoc(:tags, tags)
  |> Ecto.Changeset.put_assoc(:author, author)
  |> Ecto.Changeset.cast_assoc(:book_contents)
  |> Repo.insert()
end

cast_assoc/3

> Casts the given association with the changeset parameters. > This function should be used when working with the entire association at once (and not a single element of a many-style association) and receiving data external to the application. > > * cast_assoc/3

We use cast_assoc/3 when we want to use parameters to create our association. (https://hexdocs.pm/ecto/Ecto.Changeset.html#cast_assoc/3) is often useful for external data such as the parameters we receive from a form.

flowchart LR
Params --> Association

Example Return Value:

#Ecto.Changeset<
  action: nil,
  changes: %{
    book_contents: #Ecto.Changeset<
      action: :insert,
      changes: %{content: "some content"},
      errors: [],
      data: #BookSearch.Books.BookContents<>,
      valid?: true
    >,
    tags: [],
    title: "some title"
  },
  errors: [],
  data: #BookSearch.Books.Book<>,
  valid?: true
>

Book Contents Form Input

We’ve already mentioned that cast_assoc/3 is useful for params from a form, so let’s add book contents to our existing book form.

We can use inputs_for/4 to create inputs for an association.

Modify the book form template file to include an input for book contents.

# lib/book_search_web/templates/book/form.html.heex

<.form let={f} for={@changeset} action={@action}>
  <%= if @changeset.action do %>
    
      <p>Oops, something went wrong! Please check the errors below.</p>
    
  <% end %>

  <%= label f, :title %>
  <%= text_input f, :title %>
  <%= error_tag f, :title %>

  <%= label f, :tags %>
  <%= multiple_select f, :tags, tag_options(), selected: if assigns[:book], do: Enum.map(@book.tags, fn book -> book.id end), else: [] %>
  <%= error_tag f, :tags %>

  <%= inputs_for f, :book_contents, fn i -> %>
    <%= label i, :content %>
    <%= text_input i, :content %>
  <% end %>

  
    <%= submit "Save" %>

We need to preload the book contents association; otherwise, this form will crash. Notice we have some failing tests.

$ mix test
...
)test edit book renders form for editing chosen book (BookSearchWeb.BookControllerTest)
     test/book_search_web/controllers/book_controller_test.exs:101
     ** (ArgumentError) using inputs_for for association `book_contents` from `BookSearch.Books.Book` but it was not loaded. Please preload your associations before using them in inputs_for
...
2) test update book renders errors when data is invalid (BookSearchWeb.BookControllerTest)
     test/book_search_web/controllers/book_controller_test.exs:135
     ** (ArgumentError) using inputs_for for association `book_contents` from `BookSearch.Books.Book` but it was not loaded. Please preload your associations before using them in inputs_for
...

Preload the :book_contents.

# lib/book_search/books.ex

def get_book!(id), do: Book |> preload([:tags, :book_contents]) |> Repo.get!(id)

Now all tests should pass.

$ mix test
...
65 tests, 0 failures

We can view our new book contents field when we visit http://localhost:4000/authors/1/books/new.

Now that we have a form let’s add a test to ensure we can send the book contents in our BookController.create/2 action and that we display the book contents on the book show page. Put the following test inside the existing "create book" describe block.

# test/book_search_web/controllers/book_controller_test.exs

test "with book content", %{conn: conn} do
  author = author_fixture()
  create_attrs = %{title: "some title", tags: [], book_contents: %{content: "some content"}}
  conn = post(conn, Routes.author_book_path(conn, :create, author), book: create_attrs)

  assert %{id: id} = redirected_params(conn)
  assert redirected_to(conn) == Routes.author_book_path(conn, :show, author, id)

  conn = get(conn, Routes.author_book_path(conn, :show, author, id))
  assert html_response(conn, 200) =~ "some content"
end

Add book contents to the book show page. We haven’t enforced that a book must have book contents, so we’ll ensure book contents exist before displaying them.

<h1>Show Book</h1>

<ul>

  <li>
    <strong>Title:</strong>
    <%= @book.title %>
  </li>
  <li>
    <strong>Tags:</strong>
    <%= for tag <- @book.tags do %>
      <%= tag.name %>
    <% end %>
  </li>

  <%= if @book.book_contents do %> 
    <li>
      <strong>Content:</strong>
      <%= @book.book_contents.content %>
    </li>
  <% end %>

</ul>

<span><%= link "Edit", to: Routes.author_book_path(@conn, :edit, @author_id, @book) %></span> |
<span><%= link "Back", to: Routes.author_book_path(@conn, :index, @author_id) %></span>

All tests should pass!

$ mix test
...
67 tests, 0 failures

Update Book

We’ll also want to update book content when we update a book. So let’s write a test to ensure we can update the book content.

test "update_book/2 with book content" do
  author = author_fixture()
  book = book_fixture(author: author, book_contents: %{content: "some content"})

  update_attrs = %{
    title: "Name of the Wind",
    book_contents: %{content: "some updated content"}
  }

  assert {:ok, %Book{} = book} = Books.update_book(book, update_attrs)
  assert book.title == "Name of the Wind"
  assert book.book_contents.content == "some updated content"
end

Now we need to create the :book_contents association using put_assoc/4.

# lib/book_search/books.ex

def update_book(%Book{} = book, attrs) do
  {tags, attrs} = Map.pop(attrs, :tags, [])
  {content, attrs} = Map.pop(attrs, :content, "")

  book
  |> Book.changeset(attrs)
  |> Ecto.Changeset.put_assoc(:tags, tags)
  |> Ecto.Changeset.put_assoc(:book_contents, %{content: content})
  |> Repo.update()
end

We’ll also test the controller for the sake of comprehensive testing.

# test/book_search/books_test.ex

test "with book content", %{conn: conn, book: book} do
  update_attrs = %{
    title: "Name of the Wind",
    book_contents: %{content: "some updated content"}
  }

  conn =
    put(conn, Routes.author_book_path(conn, :update, book.author_id, book), book: update_attrs)

  assert redirected_to(conn) == Routes.author_book_path(conn, :show, book.author_id, book)
  conn = get(conn, Routes.author_book_path(conn, :show, book.author_id, book))
  assert html_response(conn, 200) =~ "some updated content"
end

Now all tests should pass!

$ mix test

Large Data

We want to ensure our book contains a large amount of data. We can use the Faker library to generate large amounts of fake text.

Add Faker to our list of dependencies.

# mix.exs

defp deps do
  [
    {:phoenix, "~> 1.6.10"},
    {:phoenix_ecto, "~> 4.4"},
    {:ecto_sql, "~> 3.6"},
    {:postgrex, ">= 0.0.0"},
    {:phoenix_html, "~> 3.0"},
    {:phoenix_live_reload, "~> 1.2", only: :dev},
    {:phoenix_live_view, "~> 0.17.5"},
    {:floki, ">= 0.30.0", only: :test},
    {:phoenix_live_dashboard, "~> 0.6"},
    {:esbuild, "~> 0.4", runtime: Mix.env() == :dev},
    {:swoosh, "~> 1.3"},
    {:telemetry_metrics, "~> 0.6"},
    {:telemetry_poller, "~> 1.0"},
    {:gettext, "~> 0.18"},
    {:jason, "~> 1.2"},
    {:plug_cowboy, "~> 2.5"},
    # Added Dependencies
    {:faker, "~> 0.17.0"}
  ]
end

Make sure to install dependencies.

$ mix deps.get

The Faker.Lorem module handles generating specified amounts of fake text. The Faker.Lorem module is named after Lorem ipsum, which is commonly used for fake text.

# test/book_search/books_test.exs

test "create_book/1 with large amounts of book content" do
  author = author_fixture()

  valid_attrs = %{
    title: "some title",
    author: author,
    book_contents: %{content: Faker.Lorem.paragraph(1000..2000)}
  }

  assert {:ok, %Book{} = book} = Books.create_book(valid_attrs)

  assert book.book_contents.content == valid_attrs.book_contents.content
end

The test already passes, but it’s good to verify we can have books with lots of content!

Seed Book Content

Let’s add a book with some large content to our seed file to manually test our application.

alias BookSearch.Tags
alias BookSearch.Authors
alias BookSearch.Books

if Mix.env() == :dev do
  {:ok, author1} = Authors.create_author(%{name: "Patrick Rothfuss"})
  {:ok, author2} = Authors.create_author(%{name: "Dennis E Taylor"})

  [tag1, tag2, tag3, tag4] =
    ["fiction", "fantasy", "history", "sci-fi"]
    |> Enum.map(fn tag_name ->
      {:ok, tag} = Tags.create_tag(%{name: tag_name})
      tag
    end)

  Books.create_book(%{
    title: "Name of the Wind",
    author: author1,
    tags: [tag1, tag2],
    # Added book contents
    book_contents: %{content: Faker.Lorem.paragraph(1000..2000)}
  })

  Books.create_book(%{
    title: "We are Legend (We are Bob)",
    author: author2,
    tags: [tag3, tag4]
  })
end

Then reset the Database and run the seed file.

$ mix ecto.reset

Now we can visit http://localhost:4000/author/1/books/1 and see our book content.

Your Turn

In your seed file, change the size of :content in :book_contents to be significantly larger such as 100000..1000000 sentences. Do you experience any performance issues?

Commit Your Progress

Run the following in your command line from the project folder to track and save your progress in a Git commit.

$ git add .
$ git commit -m "finish book search book content section"

Other notebooks:

@andyl

elix_util

Examples

vegalite.livemd

vega_lite jason

2022-8-18
Chris Martin
@trbngr

elixir_cqrs_tools

Using cqrs_tools with Commanded

commanded.livemd

commanded cqrs_tools ecto etso jason

2022-8-18
@TomBers

livebookNotes

Attractors

attractors.livemd

decimal vega_lite kino

2022-8-18
@TomBers

livebookNotes

Input test

inputs.livemd

httpoison

2022-8-18
@DockYard-Academy

curriculum

Phoenix 1.7

phoenix_1.7.livemd

jason kino youtube hidden_cell

2023-3-21
Stephen Ball
@sdball

livebooks

Advent of Code 2021 - Day 6

day06.livemd

kino vega_lite kino_vega_lite

2022-8-18
Ricardo Trejos
@cardotrejos

learning-elixir

Keyword Lists

keyword_lists.livemd

kino_lab vega_lite kino_vega_lite benchee ecto math faker kino utils

2022-8-24

Back