Moby - Product Vision and Problem Analysis
What is the Moby project? At the first glance it might not sound like something that revolutionized the world of software engineering in the last decade. However, it is in fact a community-driven upstream for Docker1, the widely known containerization ecosystem.
In this blog we talk about Moby and Docker’s capabilities, common use cases, as well as analyze their stakeholders. We also discuss the key quality attributes, project’s roadmap and potential ethical considerations.
Inspiration and goal
Before the first version of Docker was released in 2013, isolation of the processes from the operating system’s files and resources was done with various utilities like UNIX chroot, Linux LXC or FreeBSD jails. From the perspective of the processes it often seemed as if they were running on a separate virtual machine, since it did not have access to the full process tree, connected devices and files of the host operating system. They could also have a limited amount of RAM and CPU assigned. However, unlike a fully-fledged virtualization done with KVM, VMware ESXi or HyperV, such system-level solutions were much more lightweight. Unfortunately, they also often required extensive administrative knowledge to configure and were used mainly in order to secure deployed server applications.
Docker provides standardized process isolation solutions in the form of templated containers, which led to a massive rise in popularity for both Docker itself and its approach to containerization.
System’s structure and capabilities
The main one is the Docker engine daemon, which runs in the background, communicates with the container runtime of choice (containerd2 by default) and uses it to start and control the containers. Container runtime leverages various capabilities of the operating system in order to isolate processes and provide communication between them and the host. Container images are lightweight, immutable templates that include everything needed to run an application: binaries, libraries and the base system. Images are used by the Docker engine to create containers. Each container is a standalone, isolated environment which runs a process, usually the application for which it was created. Containerization allows that process to always work in the same way, independent of the host operating system, its binaries and libraries.
Images can be created manually or pulled from and pushed to the registry. The registry can be public or private. The Docker Hub is the most widely used public registry, where developers can push their custom images for everybody to use.
Since containers are ephemeral by design, Docker provides the volumes capability, which allow for data persistence even after container’s deletion or recreation. They can be mounted into the container and refer to the local directory on the machine or the remote server like NFS.
Docker is able to perform basic container orchestration by operating in the swarm mode. In that mode, several Docker daemons are deployed on the different machines, forming a cluster. Containers can be deployed in multiple instances and spread among the machines, providing high availability and load balancing for the applications which they run.
Use cases
One of the main uses for precursors of the modern containerization was isolating deployed application processes from their host operating system for security purposes. Docker’s simplicity took that even further and streamlined the workflow in other fields as well3, effectively making it the Swiss Army Knife of the DevOps world. The images, provided in various public container repositories, can be deployed and serve as building blocks of the infrastructure - not only testing, but also production. Orchestration systems like Docker Swarm4 or Kubernetes5 allow engineers to deploy multiple containers on clusters, consisting of hundreds or thousands of machines.
What’s more, containers allowed public cloud providers like Amazon6, Microsoft7 or Google8 to introduce a new category of infrastructure: serverless computing9. Instead of managing the individual machines or clusters, the provider takes responsibility for managing those and engineers only provide their application’s code or the containers that are meant to be deployed. The workloads are scaled up on demand or even down to zero when they’re not needed. This approach abstracts away the concept of machine entirely, vastly simplifying the administration and maintenance.
Containers can be also used to make the development environment as close as possible to the production one. For example by running local instances of the database system instead of mocking the data. This enables developers to easily replicate bugs without the extra overhead that launching separate virtual machines would require.
The ephemeral nature of Docker containers turned out to be a perfect fit for CI/CD jobs, used for building, testing and deploying software. Being able to quickly launch a dedicated, fresh environment for these jobs was a huge step up in comparison to the older methods, which relied on the building agent being configured appropriately in advance. Docker allows many services like Github, Gitlab, Bitbucket, Azure Pipelines and more to host and provide the CI/CD service on shared machines.
The amount of potential uses for the containers is very broad. It will likely grow as people find more applications in which the containers’ characteristics are beneficial.
Stakeholder analysis
Stakeholders in open source software are considered to be an important factor to decide in which direction the software is heading. It is noteworthy that Moby refers to Docker when more end user applications are required. In the table below the target audience of Moby is mapped to explore its stakeholders and their categorization.
| Who? | What do they require from the project? |
|---|---|
| Contributors | Clear and standardised codebase, software development tool (issues, bugs, tasks), Documentation (architecture, dependencies, onboarding, contribution guidelines, building from scratch) |
| Infrastructure providers | Documentation, integration capabilities, extensibility, long term goals and roadmap, licensing information |
| Software Developers | Clear documentation, forum and/or helpdesk, standard compliance, ease of use |
| Researchers | Communication channel, documentation (architecture, components), history, roadmap |
| System administrators | Integration with existing CI/CD tools, reliability, documentation, API stability |
Next, we classify the stakeholders of Moby by interest and influence based on the arc42 documentation. The graph is divided into four categories. In the high interest and influence section we see the Docker Inc company and its owner Mirantis, as well as Moby’s Technical Steering Committee. In the lowest interest and impact we see the Cloud Native Computing Foundation, the projects of which make heavy use of Docker’s standardized container image and runtime.
Key quality attributes
Since Moby is meant as a product for developers who want to build systems based on the project, internal quality attributes like code quality also become interesting for external stakeholders. The Moby project has laid out four key principles. From these we can identify the key quality attributes.
Modularity
The Moby project is created with modularity in mind. It consists of a large number of different components. These components have well defined APIs, allowing for a mix and match of different components depending on the use case. This principle also defines that the components should work well together, which leads to another key concept: composability.
“Batteries included but swappable”
It means that all components combined to give a functional product, however, the components should be designed in such a way that they can be substituted by a similar external component. This means that the ease of integration is a very important quality attribute.
Usable security
The intended audience of the Moby project is developers that will use the project to create a final product. However, design decisions in the Moby project also influence the usability of the product from the perspective of the end-user. If the CLI component is poorly designed, the product will be less usable. Therefore Moby has to keep in mind both the usability for developers and deployers. The second part of the principal demands that additionally, the project is secure. Moby aims to achieve this by designing components with sane default behaviour without compromising usability (example rootless mode10).
Developer focused
This principle further emphasises that Moby is not an end-product, but rather a set of components “functional and useful to build powerful tools”. By separating the Moby project from the Docker application, it has allowed its functionalities to be applied in a more general sense. Reusability is another key quality attribute that would allow the code to be used as an upstream for different projects.
Product roadmap
Moby provides a roadmap to understand where the project is going and help determine if a contribution could be conflicting with some longer-term plans.
The main focus of the project right now are ongoing runtime improvements and decoupling of the internal components. Moby maintainers also aim to remove all dependencies on Docker trademarks, in order to clarify its relation with its downstream product.
Since the Docker daemon works as a root user by design, many security concerns were raised by the community. This led to implementation of the “rootless mode”, which allows the daemon to work as an unprivileged user. Ongoing development of the rootless mode focuses on maximising its functionality, without having to require permission elevation.
Additionally, a lot of work is put into the BuildKit toolkit, which focuses on extending Docker’s capabilities for building the container images in CI/CD pipelines. This includes providing the external cache functionality in addition to Docker image layer caching, as well as the parallel builds.
Ethical considerations
In 2017 Docker Inc created the Moby project in order to facilitate research and development around modern containerization stack, as well as encourage software engineers to use Moby’s components in creating custom container-based systems. They are also committed to using Moby as an upstream for their products: Docker Community and Enterprise editions.
Docker had revolutionised the concept of containerization. In 2015 the Open Container Initiative (OCI) was created as a group under the Linux Foundation. Docker played a large contributing factor in the specification of the standards. This has allowed the industry to further innovate and has led to the widespread use of containerization software in the industry. It also allowed tools like Podman, Buildah and others to build on top of the existing container stack and broaden the use cases while offering granularity, simplicity while focusing on specific parts of the ecosystem. For example, Podman aims to offer a containerization engine without an extra daemon process, while Buildah focuses on image creation.
However, the company behind Docker has also restricted the use of Docker by companies. Docker Desktop, the product which made it easy to use Docker as a development tool on any operating system, has become a paid product in 2021 for use by businesses with over 250 employees.
Final thoughts
In conclusion, Moby is a collaborative project, which aims to provide building blocks that are used to build Docker, in a way that allows developers to make their own container-based systems as well as experiment with the containerization stack. The impact Moby and its precursor Docker had on the modern software engineering world is substantial and altered it on all of its stages - from making development easier, all the way through creating new ways of deploying systems in a highly available and scalable manner. It is safe to say that we will see more applications of containers in the future, and many of these might incubate as part of Moby before becoming mainstream, just as the modern containerization itself did.