Ruby & the Multicore CPU: Part One - Introduction

I’m a big fan of both ruby and elixir. For quite some time, one perceived weakness of ruby has been its inability to make best use of multiple cores in modern CPUs.

When Matz published the first version of ruby in 1995, Moore’s Law was still in effect, and the multi-core architecture seen today was not a consideration. Consequently, ruby is often seen as a poor performer on modern CPU architectures.

I remember an early conversation about elixir in which a colleague was bemoaning the fact that a [mri] ruby web server only ran in a single process, and it you wanted to exploit multiple cores you would have to spin up multiple web server processes and load balance between them. This has a performance and memory overhead, since processes are heavyweight.

However, there is plenty you can do to make the most of modern CPUs with ruby, and there are some exciting new developments on the way in Ruby 3.0, coming this Christmas¹.

In this series of posts, I want to talk about

• How concurrency and parallelism are achieved in ruby.
• How to make best use of the CPU and other resources available to you.
• What ruby can currently be best used to optimise concurrency and parallelism.
• Some interesting changes in ruby 3.0 that provide even more features.
• How to write better code now, so that the upcoming changes are easy to take advantage of.

Glossary

Before we get into it, let’s define some common terms, so we’re all on the same page.

Concurrency vs Parallelism

Imagine you have a computer with an old single core CPU. The operating system will allow you to run multiple programs at once. It deals with scheduling processes, dealing with interrupts and sharing memory and resources so that processes seem to be responsive. You can listen to music while to write a text document, while you download something in the background.

But, there is only one CPU, and only one set of instructions for one task are being ever executed at one time. Instructions for each task are interleaved. The operating system decides when to switch tasks between processes to make best use of IO and resources and to ensure that the computer seems responsive. Each time, the CPU has to do a certain amount of work to switch contexts so that processing can continue where it left off.

Concurrency is the management of multiple, independently executing sets of instructions.

Now imagine you have two computers with old single core CPUs. The CPU on each computer can each process instructions independently from each other. It might be that the processes on each computer can send messages to one another, but that’s not required for the configuration to be parallel.

Parallelism is the simultaneous execution of multiple sets of instructions.

Concurrency is what you need to think about when managing shared resources between processes and when optimising the use of resources. So, although different and independent, the concepts of concurrency and parallelism work very effectively together. Concurrency is what you need to think about if you want to make the most effective use of parallel processing.

Processes

Processes are the operating system task that runs your program. The MRI ruby virtual machine is a process. You can achieve a simple form of parallelism simply by spinning up more than one ruby process.

However, processes are heavyweight. They require a whole new memory space, and take more time to create and manage than some other solutions.

Threads

A thread is a subset of a process that can be managed and scheduled independently from, and execute concurrently with the other parts of the process. Threads can access shared memory from their parent process, and do not require as much overhead to create as processes.

Native Threads are threads that are managed by the operating system.

Green Threads are threads that are managed at the user level rather than the kernel. So, they are created and managed by their parent process (usually a virtual machine), rather than the operating system. They are typically quicker to create and manage than OS native threads, but, since the OS is not managing them, they do not generally run across multiple CPU cores. Therefore, they are not so useful in optimising CPU-bound processes. They are still good for optimising IO-bound processes. Goroutines are an example of green threads.

Fibers

Thread processing can either be scheduled pre-emptively (where the OS decides that it is now time to pass processing time onto a different thread; or cooperatively, where a process can signal that it is ready to cede processing time to a different thread (for example, because the current thread is waiting for user input).

Fibers are threads that are designed to be cooperatively scheduled.

Synchronicity

Somewhat orthogonal to concurrency and parallelism is the concept of synchronous and asynchronous calls.

A synchronous method call is such that the caller must wait for the response from the callee before further processing can take place. Each instruction or method call takes place sequentially and the process must wait for each instruction to complete before the next one takes place.

An asynchronous method call doesn’t have to wait around. The caller can request that processing takes place, then continue to process other instructions. When the requested work is done, the callee sends a message back, informing the caller that the work is complete and the result can now be processed further. This is useful for, say, a web site that needs to do a bunch of things when a new user is registered, (like create a welcome letter and send them an email) but you don’t want to block the web server request while those tasks are being done.

Concurrency and parallelism are related to resource management; synchronicity is related to control flow.

Coming Up

So, there is a lot to go through in the next series of posts. Here is a breakdown of the whole series:

1. Introduction
   1. Concurrency vs. Parallelism
   2. Processes
   3. Threads
   4. Fibers
   5. Synchronicity
2. Ruby up to 2.7
   1. MRI Ruby
   2. JRuby
   3. Rubinius
   4. TruffleRuby

3. Processes and Threads

4. Current Concurrency Paradigms
   1. Queues and Jobs
   2. Communicating Sequential Processes
   3. Actor Model
   4. Reactor Model
5. Ruby 3.0 Concurrency and Parallelism

Pick something of interest above, or go straight to the next post in the series.