Mattermost Server - Architectural decisions

Main architecture

Mattermost consists of several applications that are interconnected so that the project can run as a whole. Although our project is mattermost-server, we will present Mattermost as a whole in this section, while focusing on mattermost-server in the further sections. Mattermost describes its backend as:

“… a single-compiled Go binary that is exposed as a Restful JSON web server with Javascript and Go clients.”1

We observe that the key architectural styles and decisions are:

  • Client-server architecture There are multiple client applications that request the server for information and for executing some processes. This makes Mattermost a multi-platform application.
  • Monolithic architecture The mattermost-server is a single codebase compiled to a single binary that serves all the purposes of the mattermost application. Although there is a separation of concerns within this repository, it is deployed as a whole. For complex scenarios, clustered replicas of this monolith are used.
    • Despite the monolith offering most functionalities, there are two other independent smaller services offering push notifications and email support.
  • Plugins There is a lot of importance placed on third-party integrations and there are a plethora of plugins for the client and server applications2.
  • Flexibility Along with plugins, Mattermost also offers access to all server-side APIs to build extensions, support for custom slash commands, webhooks, etc. This is a key feature that enables organizations to tailor Mattermost to their specific needs.

Containers

According to the C4 model, containers are independently deployable/runnable units that represent a boundary of sorts within which different pieces of code can run and together constitute a process. Consequently, communication between different containers would typically represent inter-process communication. In this section, a “container” will be in reference to this definition.

As there are multiple moving parts in the functioning of Mattermost, there are primarily four broad executional environments for different pieces of the software.

Figure: Fig1, Mattermost containers view

  • The mattermost-server deployment is run behind a proxy. This container is responsible for fetching data from and storing data to the database container and the file storage container. If used in a big enterprise, and/or the scenario of use is more complex with a high volume of communication, the server container is deployed as a cluster with multiple replica instances.1

  • The database is used to store structured information like the messages, profile information, group memberships, etc. In simple scenarios, this is a standalone MySQL or Postgres deployment, whereas bigger enterprises and organizations might have read and search replicas alongside the main database3.

  • The file storage is used to store unstructured data like documents, images and videos that users upload on to mattermost channels.

  • The mattermost-server instance exposes REST APIs that are used by a number of client side applications through HTTPS (Hypertext Transfer Protocol Secure) or WSS (WebSocket Secure protocol). These can be mobile clients on iOS and Android, desktop client or the mattermost web application. As we focus on the server, we do not elaborate further on the client side containers.

In addition to the aforementioned containers, auxiliary services are deployed on the backend infrastructure. They include plugins to integrate with third-party applications, push notifications service, email service, etc. These are clearer and more relevant while considering the different connections in use. This is elaborated upon in the next section.

To get a good grasp with detailed information about different containers and how they are independently deployable, the deployment guide of Mattermost4 is a great resource.

Components and connectors

Figure: Fig2, Mattermost network diagram

In reference to the C4 model again, a component is

a grouping of related functionality encapsulated behind a well-defined interface5.

This refers to units that are not separately deployable. While this strictly is not the implementation style in mattermost-server, the team has defined the main groups of functionalities164:

  • RESTful JSON web service: The entirety of the Mattermost server is accessible through a RESTful Web Service API. The API can be completely accessed by developers creating custom applications for Mattermost either directly or via Javascript and Golang drivers.
  • Authentication client: Authenticates users by email or username, and password.
  • Authentication provider: Enables authentication of Mattermost server to other services with Authentication Client interface using OAuth2.
  • Notification service: Sends notifications via SMTP email and mobile push notifications via Mattermost Push Notification Service (MPNS).
  • Data management service: Connects to and manages supported databases and file stores.

These functionalities are not grouped in the source-code the same way they are depicted in Mattermost diagrams. The separation of these functionalities into directories is covered in the next section.

There exist different types of connectors between the components and between the containers. Mattermost utilizes straight-forward types of connections that one would expect to find in most software projects. The main connections can be seen in Figure 2 above4.

  • The connectors within various components in the mattermost-server container consist of package imports and function calls from different packages in the same codebase.
  • The client applications communicate with the server through HTTPS connections (offering page renders and core functionality) or WSS connections (for real-time updates and notifications). All communication passes through a proxy that is setup at the customer’s infrastructure. These communications include push notifications, emails, and any REST requests/responses from the mattermost-server4.
  • The communication to the relational database and the file store is achieved through the related driver packages. These data store communications and other connections on the backend infrastructure (with MPNS, plugins, active directory, etc.) use TCP connections over specific ports1.

Development view

Code Structure

At first, when looking at the directory structure of the mattermost-server, one may get quickly overwhelmed by the sheer number of substructures within the project. However, as the project’s documentation outlines, there are five main directories within the project6:

  • api4/ holds all API and application-related code.
  • model/ holds all data model definitions and the Go driver.
  • store/ holds all database querying code.
  • utils/ holds all utilities, such as the mail utility.
  • i18n/ holds all localization files for the server.

Dependencies

Based on the go.mod file, the project has 121 dependencies, ranging from essential Go development tools like gorilla/mux for web routing7 to self-maintained database migration tools8.

Development tools

Developers get useful terminal tools, including various make commands for building the project, resetting the database, cleaning docker and checking the code for potential issues. Additionally, developers can make use of mattermost commands for generating test data and interacting with the database9.

Runtime view

Mattermost is a big application that has many activites during the runtime. To change how key functionalities behave during the runtime, one needs to modify a config.json file10. It supports a dozen of settings - enabling plugins and integrations, database configurations, changing authentication providers, etc. This configuration file decides how components shall interact during the runtime. Consequently, some of the key usage scenarios would involve:

  • Authentication. Mattermost provides several configurable authentication options, including typical sign in with email, GitLab, Google, etc. In order to authenticate, the user’s credentials get handled by Authentication Clients, that communicate with external Authentication Providers (such as Google).
  • Push notifications. During the runtime, mattermost-server communicates with configured push notifications servers (e.g., Apple Push Notification Service, Google Cloud Messaging), in order to send out notifications per various functionalities in the application.
  • Changing data in the database. While running, mattermost-server uses configurable Data management services (local, Amazon S3) for updating and accessing data within the Mattermost instance.

Users get access to Mattermost’s functionality by utilizing HTTPS and WSS connections. While running, mattermost-server provides an extensive REST API11 that can be used by any other application. Routes related to all Mattermost functionalities can be found in api4/api.go file - by using them, one can perform HTTPS requests for retrieving or manipulating data. Consequently, WSS connections get used for receiving real-time updates.

Key quality attributes

To effectively realize key quality attributes, Mattermost has been adhering to certain architectural guidelines. Each outlined key quality attribute is supported by different architectural decisions, as can be seen in the following overview.

Scalability

As Mattermost’s clients range from individual users and small startups to large enterprises, Mattermost has different solutions for different needs. The team presents several ways of deploying a Mattermost instance, along with relevant installation guides and hardware requirements.

  • Single Machine Deployment. This configuration can handle up to 2000 users.
  • Multi Machine Deployment. This can handle between 2000 and 10000 registered users on a three-machine Mattermost installation with a proxy, an application server and a database server.
  • Cluster-based Deployment. This solution is suitable for cases with over 10000 users with moderate usage. It is achieved by clustering multiple servers and introducing a load balancer

    to distribute requests from users across multiple Mattermost application servers, allowing the system to scale beyond the limits of any single server12.

Mattermost can be deployed on AWS, Google Cloud, Microsoft Azure, and Oracle Cloud. The scalability guide13 presents multiple sample architectures for different use cases, including suggestions for instances involving 100000 users.

Availability

This aspect is crucial, since many clients rely on Mattermost for important communication that defines their workflow. As Mattermost instances are either self-hosted or cloud-hosted, availability may vary highly among instances. Companies can opt for better availability by using multiple servers and load balancers. In general, cloud solutions can provide amazing results. For example, Amazon offers solutions that have 99.99%14 per-year availability and even 99.999%15 per-year availability.

Modularity

This quality is clearly realized by Mattermost’s plugin system. As stated by Mattermost:

Mattermost plugins are isolated pieces of code written in Go and/or React. They are separate from the main repositories and are used to extend the functionality of the Mattermost server and webapp16.

By following the extensive documentation17, one can create their own plugin and, later on, publish it on Mattermost Marketplace. This greatly boosts modularity of the whole system, as independent developers can create easy-to-install code pieces that are accessible by other Mattermost users.

Compatibility

To ensure that the system’s components remain compatible, Mattermost maintains an extensive changelog18. This way, Mattermost’s stakeholders get to know the breaking related to individual components. Additionally, Mattermost also has clear guidelines19 for upgrading server instances.

API design principles

To serve a standardized API, Mattermost uses OpenAPI Standard. This is a widely adopted specification, which allows for the Mattermost APIs to be more easily understood amongst developers. Furthermore, it allows the newcommers to deliver code that can be easily integrated in the existing specification. Additionally, Mattermost uses ReDoc document generator, that allows for generation of a well-organised and beautiful API documentation.

To deliver new contributions to the APIs, developers are given an exstensive guideline. For new additions to be successfully integrated, one needs to complete the following checklist.

  • Document the endpoint.
  • Implement the API handler on the server.
  • Add a function to the Go driver.
  • Write a unit test.
  • Submit the implementation.

References

NB: All figures in this essay are taken from Mattermost documentation.

  1. Mattermost Docs. Architecture Overview. https://docs.mattermost.com/getting-started/architecture-overview.html ↩︎

  2. Mattermost. Marketplace. https://mattermost.com/marketplace/ ↩︎

  3. Mattermost Docs. Databases. https://docs.mattermost.com/deploy/deployment-overview.html#databases ↩︎

  4. Mattermost Docs. Deployment overview. https://docs.mattermost.com/deploy/deployment-overview.html ↩︎

  5. C4 Model. https://c4model.com/ ↩︎

  6. Mattermost developers. Project structure. https://developers.mattermost.com/contribute/server/developer-workflow/#workflow ↩︎

  7. Gorilla. Go Dev Tools. https://github.com/gorilla/mux ↩︎

  8. Mattermost. Morph. https://github.com/mattermost/morph ↩︎

  9. Mattermost Developers. Useful commands. https://developers.mattermost.com/contribute/server/developer-workflow/#useful-mattermost-commands ↩︎

  10. Mattermost Docs. Configuration settings. https://docs.mattermost.com/configure/configuration-settings.html#configuration-settings ↩︎

  11. Mattermost. API. https://api.mattermost.com/ ↩︎

  12. Mattermost. Cluster-based development. https://docs.mattermost.com/scale/scaling-for-enterprise.html#cluster-based-deployment ↩︎

  13. Mattermost Docs. Scalability for enterprise. https://docs.mattermost.com/scale/scaling-for-enterprise.html#scaling-for-enterprise ↩︎

  14. Amazon. Amazon Aurora. https://aws.amazon.com/rds/aurora/?aurora-whats-new.sort-by=item.additionalFields.postDateTime&aurora-whats-new.sort-order=desc ↩︎

  15. Amazon. High availability hosting. https://aws.amazon.com/marketplace/solutions/infrastructure-software/high-availability ↩︎

  16. Mattermost Developers. Mattermost Plugins: An Introduction. https://developers.mattermost.com/contribute/plugins/ ↩︎

  17. Mattermost Developers. An Overview of Mattermost Plugins. https://developers.mattermost.com/integrate/plugins/overview/ ↩︎

  18. Mattermost Docs. Changelog. https://docs.mattermost.com/install/self-managed-changelog.html ↩︎

  19. Mattermost Docs. Upgrade guidelines. https://docs.mattermost.com/guides/deployment.html#upgrade-mattermost ↩︎