Would you like to see your link here? Contact us

Notesclub

created by hec & contributors

terms privacy

Talking to Azure Storage and the ARM API - From Erlang

azure.livemd

Dr. Christian Geuer-Pollmann

@chgeuer

elixir-livebook-azure-dem...

More notebooks

Talking to Azure Storage and the ARM API - From Erlang

Section

Full demo on Github: chgeuer/elixir-livebook-azure-demo

Just enough Elixir to understand the code

12 - 5

The - operator is just an infix syntax for the Kernel.-/2 function, i.e. the function with name - in the Kernel module, which takes 2 arguments (arity):

Kernel.-(12, 5)

The |> (pipe) operator in Elixir pipes the left-hand value as first argument into the subsequent function call, i.e. a |> f(b,c) is equivalend to f(a,b,c).

Those familiar with F# might see the similarity; the difference is that in F#, the piped argument becomes the last in the function call, i.e. a |> f(b,c) in F# is equivalend to f(b,c,a), simply because all functions in F# only have a single argument, so that this is f(b)(c)(a)…

12 |> Kernel.-(5)

12
|> Kernel.-(5)

Processes and actors

self()

Pull in the Azure SDK for Elixir/Erlang

The Mix.install() function locally installs a collection of software libraries. In this case:

The chgeuer/ex_microsoft_azure_storage contains a REST client for the Azure storage APIs
The chgeuer/ex_microsoft_azure_management package contains REST APIs for the ARM API
The chgeuer/ex_microsoft_azure_utils are some helpers handling Azure AD communications

Running this command for the first time times a few minutes, as all the source code gets pulled from Github, and all packages (and their dependencies) get compiled.

armSdk = fn (k,v) -> { k,
  github: "chgeuer/ex_microsoft_azure_management",
  sparse: "Microsoft.Azure.Management.#{v}",
  app: false }
end

[
  {:azure_utils, github: "chgeuer/ex_microsoft_azure_utils", app: false},
  armSdk.(:arm_resources, "Resources"),
  armSdk.(:arm_subscription, "Subscription"),
  armSdk.(:arm_storage, "Storage"),
  {:azure_storage, github: "chgeuer/ex_microsoft_azure_storage", app: false}
]
|> Mix.install()

The alias section is similar to using or import statements, as we can omit parts of the module names.

alias Microsoft.Azure.ActiveDirectory.DeviceAuthenticator
alias Microsoft.Azure.ActiveDirectory.DeviceAuthenticator.Model.State
alias Microsoft.Azure.ActiveDirectory.DeviceAuthenticatorSupervisor
alias Microsoft.Azure.Storage
alias Microsoft.Azure.Storage.{Container, Blob, Queue, BlobStorage}

Store a bunch of ARM API versions in a map:

api_version = %{
  :resource_groups => "2018-02-01",
  :subscription => "2016-06-01",
  :storage => v
}

And let’s start an additional token fetching process for the ARM API management.azure.com…

{:ok, management_pid} =
  %State{
    resource: "https://management.azure.com/",
    tenant_id: "chgeuerfte.onmicrosoft.com",
    azure_environment: :azure_global
  }
  |> DeviceAuthenticatorSupervisor.start_link()

management_pid

Process.alive?(management_pid)

Many processes are already running on our VM

:erlang.registered()

management_pid
|> DeviceAuthenticator.get_stage()

Start the device code flow.

Note the pattern matching here, we get a rather complex structure back from DeviceAuthenticator.get_device_code/1, but we only care about the user_code

{:ok, %{user_code: uc}} =
  management_pid
  |> DeviceAuthenticator.get_device_code()

"Use user code #{uc}"

Initially, Erlang process (actor) is in :polling state:

management_pid
|> DeviceAuthenticator.get_stage()

Now go to the microsoft.com/devicelogin

Now the underlying Erlang process (actor) is in :refreshing state, i.e. it has both an access_token and a refresh_token, and refreshes the access_token as needed.

management_pid
|> DeviceAuthenticator.get_stage()

After authenticated, we can get the token.

{:ok, %{access_token: management_token}} =
  management_pid
  |> DeviceAuthenticator.get_token()

IO.puts("The JWT value is\n#{management_token}")

Equivalent C# would be like

Console.WriteLine(
   String.Join(
     JWT.Parse(managementToken)
         .Fields
         .Select((key, val) => $"{key} {val}")
         .ToArray()
     , "\n"));

management_token
|> JOSE.JWT.peek()
|> Map.get(:fields)
|> Enum.map(fn {k, v} -> "#{k |> String.pad_trailing(12, " ")}: #{inspect(v)}" end)
|> Enum.join("\n")
|> IO.puts()

IO.puts("https://jwt.ms/#access_token=#{management_token}")

Now let’s create an HTTP client which has our access token:

conn =
  management_token
  |> Microsoft.Azure.Management.Resources.Connection.new()

Now call the subscription_list API, to get a detailed list of subscriptions, and filter for ‘our’ subscription.

subscriptionName = Console.ReadLine().Trim();

subscriptionID = client
   .GetSubscriptions()
   .First(sub => sub.DisplayName == subscriptionName)
   .SubscriptionID;

alias Microsoft.Azure.Management.Subscription.Api.Subscriptions, as: SubscriptionMgmt

subscription_name = IO.gets("Subscription Name") |> String.trim()

{:ok, %{value: subscriptions}} =
  conn
  |> SubscriptionMgmt.subscriptions_list("2016-06-01")

subscription_id =
  subscriptions
  |> Enum.filter(&amp;(&amp;1 |> Map.get(:displayName) == subscription_name))
  |> hd
  |> Map.get(:subscriptionId)

alias Microsoft.Azure.Management.Storage.Api.StorageAccounts, as: StorageMgmt

{:ok, %{value: accounts}} =
  conn
  |> StorageMgmt.storage_accounts_list("2018-02-01", subscription_id)

accounts

accounts
|> Enum.map(&amp;(&amp;1 |> Map.get(:name)))

Storage access

Starting a device authentication flow with Azure AD

This starts an Erlang process (an ‘actor’), which handles the communication with Azure AD. The #PID stuff you see is the Erlang process ID.

{:ok, storage_pid} =
  %State{
    resource: "https://storage.azure.com/",
    tenant_id: "chgeuerfte.onmicrosoft.com",
    azure_environment: :azure_global
  }
  |> DeviceAuthenticatorSupervisor.start_link()

And give us the device code with which we need to authenticate to AAD (microsoft.com/devicelogin)

{:ok, %{user_code: storage_usercode}} =
  storage_pid
  |> DeviceAuthenticator.get_device_code()

storage_usercode

{:ok, %{access_token: storage_token}} =
  storage_pid
  |> DeviceAuthenticator.get_token()

storage_token
|> JOSE.JWT.peek()
|> Map.get(:fields)
|> Enum.map(fn {k, v} ->
  "#{k |> String.pad_trailing(12, " ")}: #{inspect(v)}"
end)
|> Enum.join("\n")
|> IO.puts()

"https://jwt.ms/#access_token=#{storage_token}"

Now let’s talk to Azure Blob Storage

We’re now using an input cell in this LiveBook to allow the user to specify the storage account name.

The aad_token_provider is a lambda function which, on each call, asks the process which keeps our storage token fresh, for the current access token.

accounts
|> Enum.map(&amp;(&amp;1 |> Map.get(:name)))
|> IO.inspect(label: "Available storage accounts")

:ok

storage_account_name = IO.gets("Storage Account Name") |> String.trim()

IO.puts("We'll be using storage account '#{storage_account_name}'")

aad_token_provider = fn _resource ->
  storage_pid
  |> DeviceAuthenticator.get_token()
  |> elem(1)
  |> Map.get(:access_token)
end

storage = %Storage{
  account_name: storage_account_name,
  cloud_environment_suffix: "core.windows.net",
  aad_token_provider: aad_token_provider
}

Set HTTP proxy

Given we’re running in a Docker container on WSL2, we could still ask the SDK to funnel all outgoing calls through Fiddler on the Windows side.

# System.put_env("http_proxy", "192.168.1.10:8888")

# System.delete_env("http_proxy")

List storage containers

{:ok, %{containers: containers}} =
  storage
  |> Container.list_containers()

containers
|> Enum.map(&amp; &amp;1.name)

Create a new storage container

container_name = IO.gets("Container Name") |> String.trim()

storage
|> Container.new(container_name)
|> Container.create_container()

storage
|> Container.new(container_name)
|> Container.list_blobs()

storage
|> Container.new(container_name)
|> Blob.upload_file("/data/TheseGoToEleven.mp4")

{:ok, %{blobs: blobs}} =
  storage
  |> Container.new(container_name)
  |> Container.list_blobs()

blobs
|> Enum.map(&amp;(&amp;1 |> Map.get(:name)))
|> Enum.join("\n")
|> IO.puts()

Who’s using it already?

Talk about prod customers of the SDKs
SDKs

Damn Meeting Minutes

> How FastTrack job forced me rewire my brain - A geek-out session on Keyboards

How I felt in meetings

hunt-and-peck

What I wanted to feel

Touch typing

touch typing

Diving into the rabbit hole - Layouts

QWERTY / QWERTZ - My brain accumulated 34 years of bad habits on this
Dvorak - from 1936. Didn’t feel fancy
Workman - “… in honor of all who type on keyboards for a living”
Colemak Mod-DH
All have Windows Keyboard drivers…

usage-viz-grid2

How many kilometers do your fingers travel?

Huckleberry finn

Learning pages

Colemak.Academy - Learn your new layout slowly
keybr.com - Grill your weaknesses
- sign-in just with a magic link like https://www.keybr.com/login/yIkX1rpIDZ
Monkeytype.com - More interesting texts, love the “Quote” mode