Migration steps

Main Section

The key steps for data migration are:

1. we create the non-event structs
2. we creatre the event structs
3. we upload the mp3 files (to minio for local dev)

1 and 2 shall rely on a json dump . 3 shall rely on mp3 files that we have.

Defines the location of the json dump file that we are using

mypath =
  "#{File.cwd!()}/data/Elixir.Vyasa.Written.Source"
  |> File.ls!()
  |> Enum.sort(:desc)
  |> List.first()

# A module to support re-compilation from within a livebook cell. Call it from anywhere.
# defmodule R do
#   def recompile() do
#     Mix.Task.reenable("app.start")
#     Mix.Task.reenable("compile")
#     Mix.Task.reenable("compile.all")
#     compilers = Mix.compilers()
#     Enum.each(compilers, &Mix.Task.reenable("compile.#{&1}"))
#     Mix.Task.run("compile.all")
#   end
# end

# R.recompile()

# Vyasa.Release.rollback(Vyasa.Repo, 0)

Ecto.Migrator.run(Vyasa.Repo, :down, all: true)

# 20241515170225
Vyasa.Release.migrate()

inits the non-event structs

Path.expand(mypath, "#{File.cwd!()}/data/Elixir.Vyasa.Written.Source")
|> Vyasa.Corpus.Migrator.Restore.run()

Migrates the events

Events are dependent on other structs, hence it’s added in later

Path.expand(mypath, "data")
|> Vyasa.Corpus.Migrator.Restore.read()
|> Enum.map(fn x ->
  # &1["translations"]
  x["events"]
  |> Enum.map(&(&1 |> Vyasa.Medium.create_event()))
end)

Add in the voice files

We already have necessary functions in order to upload to minio, so we can just call the changeset and pass it the map of information we want and let that changeset logic handle it.

alias Vyasa.Medium.Store
alias Vyasa.Medium

# in the target media folder, put in the mp3 files. The names for the files should be the id itself and this is in a way "hardcoded". The ids must match.
target_media_folder = File.cwd!() <> "/media/voices"

File.ls!(target_media_folder)
|> Enum.map(fn x ->
  %{"id" => hd(String.split(x, ".mp3")), "file_path" => target_media_folder <> "/" <> x}
end)
|> Enum.map(fn v -> Vyasa.Medium.Voice.gen_changeset(%Medium.Voice{}, v) end)

# can test if events are seeded correctly here:
voice_id = "b1db78be-0bdf-443c-afdf-3221bac38758"

loaded_voice =
  Vyasa.Repo.get(Vyasa.Medium.Voice, voice_id)
  |> Vyasa.Medium.get_voices!()
  |> List.first()
  |> Vyasa.Medium.Store.hydrate()

loaded_voice.events