Web Servers

Mix.install([
  {:jason, "~> 1.4"},
  {:kino, "~> 0.9", override: true},
  {:youtube, github: "brooklinjazz/youtube"},
  {:hidden_cell, github: "brooklinjazz/hidden_cell"},
  {:finch, "~> 0.16.0"}
])

Finch.start_link(name: MyApp.Finch)

Navigation

Review Questions

Upon completing this lesson, a student should be able to answer the following questions.

• What is TCP?
• What is the lifecycle for a TCP request?

Overview

Web development is one sub-field within programming. Broadly speaking, web developers create websites.

Websites are programs that run on a web server. Clients communicate with these web servers over a computer network. We’ve already seen how we can connect to a web server as a client using HTTP requests in the APIs section.

In this lesson, we’ll cover how we can create the web server that the client connects to. Together, we’ll create a web server from scratch to demystify how they are built.

However, be aware that in general we’ll rely on the Phoenix Framework, which is a web framework for Elixir, rather than building our own web server from scratch.

Also, this lesson relies on port 4000 being free. If you have any programs using port 4000 this lesson may not work properly.

Transmission Control Protocol (TCP)

Transmission Control Protocol (TCP) is the communication protocol applications use to communicate and exchange data over a network. TCP is the underlying protocol used by HTTP requests.

We use the built in :gen_tcp library from Erlang to start a server that uses TCP to listen for connections on a network.

This server creates a socket connection on a specified port of the machine. A port is a communication endpoint for a particular service. For example, by default this livebook application runs on port 8080 which you can see in the URL of your browser, http://localhost:8080/sessions/. A socket allows for two way communication over this port to send and receive data.

graph LR;
  S[Server]
  P[Port]
  LSO[Listen Socket]
  C[Client]
  S --listen on --> P --establish--> LSO
  C --request --> LSO
  LSO --response--> C

This socket listens for connections from clients, and accepts them when they connect, reads the request, then send a response to the client.

sequenceDiagram
participant C as Client
participant S as Server
S->> S: start socket
C--> S: accept connection
C->> S: send request
S->> S: read request
S->> C: send response
C--> S: close connection

Below, we simulate a web server that accepts only a single request, and returns "Hello, world!" to the client. We then send the web server a request and receive a response.

port = 4000

spawn(fn ->
  # start socket
  {:ok, listen_socket} = :gen_tcp.listen(port, active: false, reuseaddr: true)

  # accept connection
  {:ok, connection} = :gen_tcp.accept(listen_socket)

  # read request
  {:ok, request} = :gen_tcp.recv(connection, 0)
  IO.inspect(request, label: "Client Request")

  # send response
  response = "HTTP/1.1 200\r\nContent-Type: text/html\r\n\r\n Hello world!"
  :gen_tcp.send(connection, response)

  # close connection
  :gen_tcp.close(connection)

  # we kill the listen_socket to avoid address already in use issues.
  # Typically, the listen socket will remain open and recursively accept connections indefinitely.
  Process.exit(listen_socket, :kill)
end)

# Send Request
Finch.build(:get, "http://localhost:#{port}") |> Finch.request!(MyApp.Finch)

The code above is purely for demonstration purposes. We won’t need to write our own webserver, instead we’ll rely on frameworks like Phoenix that do all of this for us.

Web Server

Putting all of this together, we can make a WebServer module. The WebServer module will listen on port 4000 and recursively accept client requests indefinitely.

Start the WebServer socket below by executing the code cell, then navigate to http://localhost:4000 in a new tab in your browser. You should see a message from the web server with the current time.

defmodule WebServer do
  def start(port) do
    {:ok, listen_socket} = :gen_tcp.listen(port, active: false, reuseaddr: true)

    accept(listen_socket)
  end

  def accept(listen_socket) do
    # accept connection
    {:ok, connection} = :gen_tcp.accept(listen_socket)

    # send response
    current_time = Time.to_string(Time.utc_now())
    response = "HTTP/1.1 200\r\nContent-Type: text/html\r\n\r\n It is #{current_time}"
    :gen_tcp.send(connection, response)

    # close connection
    :gen_tcp.close(connection)

    accept(listen_socket)
  end
end

WebServer.start(4000)

This server will remain open indefinitely and recursively call accept/2 every time a new client connects to the socket on port 4000.

All websites use a similar (albeit more fully featured) web server to accept client connections, receive requests, and send responses back.

Hopefully, this example provides you context on how web servers operate under the hood. However, it’s rarely practical to build our own web server, and instead we’ll rely on the Phoenix Framework which hides these implementation details from use.

Your Turn

We’ve seen a simple text response, but web servers are also able to send more complex responses containing an entire web page.

In the WebServer module above, modify the original response:

response = "HTTP/1.1 200\r\nContent-Type: text/html\r\n\r\n It is #{current_time}"

To a response using Hyper Text Markup Language (HTML). HTML is the code used to structure a web page. For example, we can modify the response to use an HTML header tag

.

response = "HTTP/1.1 200\r\nContent-Type: text/html\r\n\r\n 
It is #{current_time}
"

Visit http://localhost:4000 to see your modified content. You may need to stop the WebServer Elixir cell and start it again.

Commit Your Progress

DockYard Academy now recommends you use the latest Release rather than forking or cloning our repository.

Run git status to ensure there are no undesirable changes. Then run the following in your command line from the curriculum folder to commit your progress.

$ git add .
$ git commit -m "finish Web Servers reading"
$ git push

We’re proud to offer our open-source curriculum free of charge for anyone to learn from at their own pace.

We also offer a paid course where you can learn from an instructor alongside a cohort of your peers. We will accept applications for the June-August 2023 cohort soon.

Navigation