Would you like to see your link here? Contact us

Notesclub

created by hec & contributors

terms privacy

Concurrency in Elixir

ch_1.1_concurrency_in_elixir.livemd

Arpan Ghoshal

@Arp-G

async-elixir

More notebooks

Concurrency in Elixir

Navigation

Home Immutability and memory management

Concurrency vs Parallelism

Concurrency and parallelism are often used interchangeably but are two distinct concepts. Concurrency refers to the execution of multiple tasks that overlap in time, with each task being interrupted and resumed intermittently by the CPU(context switching). This can create an illusion of tasks running simultaneously, but in reality, they are taking turns executing on a single time-sliced CPU.

Parallelism, on the other hand, involves the simultaneous execution of multiple tasks on a hardware system that has multiple computing resources, such as a multi-core CPU. Parallelism allows tasks to run literally at the same time, without having to share CPU time.

In essence, concurrency deals with handling multiple tasks at once, while parallelism deals with actually performing multiple tasks at the same time. While a system can exhibit both concurrency and parallelism, it is possible to have a concurrent system that is not parallel.

In todays world with machines having power multi-core CPUs writing code that can run cocurrently and parallely can lead to huge performance benifits. We should strive to leverage the capabilities of modern hardware advancements by writing code that can fully utilize them.

Furthermore, as we develop software, we often encounter problems that require background tasks and can benefit greatly from the use of parallel programming. Examples of such tasks include image processing, video transcoding, and make third party api calls, to name a few.

Concurrency and parallelism in Elixir

Due to the functional and immutable nature of Elixir writing parallel and concurrent code becomes much simpler. Unlike many other languages that require locks and mutexes to handle issues related to shared state in parallel programming, Elixir’s design mitigates these problems. As a result, parallel and concurrent code is a first-class citizen in Elixir, requiring less effort and complexity to implement effectively.

In Elixir, the Erlang Virtual Machine (BEAM) serves as the backbone for managing concurrency and parallelism. Let’s take a closer look at how it works under the hood to provide us with these superpowers.

Elixir leverages lightweight processes that are expertly managed by the VM. These processes are not true OS processes, making them highly lightweight and allowing for thousands or even millions of them to run concurrently without impacting performance.

This lightweight process model has given rise to several powerful applications, including the Cowboy web server which creates a process for every incoming web request to keep heavy work or errors within a single request from affecting others. Other examples include Phoenix Channels and Phoenix live view that employ an Erlang process per WebSocket connection.

Processes in Elixir

In Elixir and Erlang, the term “processes” does not refer to operating system processes or threads. Instead, they are akin to green threads or actors. These processes run concurrently on a single-core CPU and in parallel on multiple cores, managed and scheduled by the Erlang Virtual Machine.

Surprisingly, each process in Elixir and Erlang requires only around 300 words of memory and takes microseconds to start, making them incredibly lightweight. In fact, within the Erlang Virtual Machine, every entity executing code operates within a process.

For instance, in Phoenix, when making a regular HTTP request using Phoenix controllers, the corresponding connection is allocated its own process. This process is swiftly terminated once the response is sent and the connection is closed. In LiveView we keep that process alive since we work with websockets.

Each process is capable of executing code and possesses a first-in-first-out mailbox to which other processes can send messages. Likewise, it can send messages to other processes. Processes in Elixir and Erlang are inherently sequential, meaning they handle one message at a time.

Similar to an operating system scheduler, the Erlang VM has the ability to start, pause, or preempt work as needed (In computing, preemption is the act of temporarily interrupting an executing task, with the intention of resuming it at a later time).

While waiting for a message, a process is completely ignored by the scheduler. As a result, idle processes do not consume any system resources.

Reductions

Erlang uses “reductions” as work units to decide when a process might be paused. A reduction in Erlang is a unit of work done by BEAM, including tasks like function application, arithmetic operations, and message passing. The scheduler monitors reductions for each process, pausing a process once it reaches a set reduction count, which lets another process take its turn to run. This ensures fairness in scheduling by preventing processes from hogging the CPU.

Additionally, reductions are applied flexibly based on the operation type. For instance, I/O operations consume reductions differently, allowing the scheduler to handle various operations effectively. Unlike traditional blocking I/O, Erlang’s non-blocking model lets processes continue working during I/O waits, improving overall system performance.

Scheduling in BEAM

BEAM, the underlying virtual machine, employs a single OS thread per core, and each thread runs its own scheduler. Every scheduler is responsible for pulling processes from its own run queue, with the BEAM being responsible for populating these queues with Erlang processes for execution.

(Note: To utilize more than one core the Erlang Runtime System Application(ERTS) has to be built in SMP mode. SMP stands for Symmetric MultiProcessing, that is, the ability to execute a processes on any one of multiple CPUs.)

The scheduler manages two queues: a ready queue containing processes that are prepared to run and a waiting queue containing processes that are waiting to receive a message.

When a process is selected from the ready queue, it is handed over to BEAM for the execution of one CPU time slice. BEAM interrupts the running process and places it at the end of the ready queue when the time slice expires. However, if the process is blocked in a receive operation before the time slice runs out, it is added to the waiting queue.

Loadbalancer

A load balancer is also in place, responsible for executing migration logic to allocate processes across the run queues on separate cores. This logic assists in maintaining load balance by taking jobs away from overloaded queues (known as task stealing) and assigning them to empty or underloaded queues (known as “task migration”).

In simpler terms, if one scheduler’s queue becomes crowded due to processes taking an extended time, other schedulers step in to distribute the workload more evenly. For example, if a process accumulates a high number of function calls (reductions), without completing, the scheduler will preemptively pause it which means freeze it, mid-run, and send it back to the end of the work queue. This preemptive multitasking approach ensures that no single task can monopolize the system for an extended period, ensuring consistently low latency, a key feature of the BEAM.

The load balancer strives to maintain an equal maximum number of run-able processes across schedulers.

Looking beyond Erlang’s internal run queues, the operating system also manages the scheduling of threads onto CPU cores at an OS level. This means that processes can not only swap within Erlang’s run queue but also undergo complete context switches or be relocated to different cores by the OS.

You can find the numer of schedulers in your IEX session using the System.schedulers/0 function.

# Returns the number of schedulers in the VM.
System.schedulers()

# Returns the number of schedulers online in the VM.
# Here online means total number of schedulers which are active and actually being used.
System.schedulers_online()

Process priority

Erlang’s priority system has four levels: low, normal, high, and max (reserved for Erlang’s internal use). Each level has its own run queue and follows a round-robin scheduling method, except for max.

Processes in max or high priority queues are executed exclusively, blocking lower-priority processes until they’re done. This design emphasizes efficiency for critical tasks but can cause bottlenecks if high-priority processes are overused, impacting overall application responsiveness.

Low and normal queues are more flexible, allowing interleaved execution without blocking each other. However, using high priority sparingly is crucial to avoid performance issues.

Additionally, Erlang permits communication across different priority levels, although a high-priority process waiting for a message from a lower-priority one will effectively lower its own priority.

Process priority can be changed in elixir using Process.flag(:priority, :high)

Resources

Navigation

Home Immutability and memory management

Back