SerenityOS - Quality and Evolution

Degree of achieving Quality attributes

We have explained three key quality attributes in Essay 1: Functionality, Usability, and Consistency. This section will introduce how the extent to which these quality attributes are achieved.

Functionality: As an operating system, SerenityOS undoubtedly does a good job of providing basic operating system functionality. This includes a web browser, email, desktop composer, and various desktop applications and games. All the features that can be implemented are included in the project manual¹.

Usability: SerenityOS was originally intended to be a private operating system, but as it grew into a multi-person effort, additional functional apps were added to make it more useable. It is, however, still a long way from being a full-fledged operating system that can enter the market. It is not suitable for non-technical individuals in some ways. We hope that additional enthusiasts will join the project in the future, allowing Serenity OS to reach a wider audience.

Consistency: The project pursues a uniform code style, and the commit policy and code style are clearly defined in GitHub². Consistency has been basically satisfied in terms of implementation.

Software Quality Process

In order to ensure the Software quality of the project, the SerenityOS team has applied several methods to ensure the quality of the project. Firstly, SerenityOS’ primary developers rely on volunteers to maintain the software’s quality. The SerenityOS allows developers to create issues and pull requests in Github. Second, the maintainers have put together a contribution guide for anyone interested in contributing to the operating system³. The developers were asked by the maintainers to determine the direction of their improvements and additions to the project and to begin with tiny steps. Furthermore, the project has a commit policy that specifies the code’s coding style². Lastly, in addition to assessing requests on Github, the maintainers have created a bug bounty program ⁴ in order to recruit more technical employees to help improve the code and the project.

Continuous integration processes

The project repository doesn’t allow the building of a new branch in the project repository for new implementing new features or bugs fixing. The usual way of contributing code is as follows.

1. Pick a contribution or issue

If you want to contribute to the project, you can claim an issue by posting a comment under the issue you want to work on in the issues tab of the project. These issues usually will be checked by the project maintainers after being posted. They will decide whether this issue needs to be solved or not, if not they will close the issue.

You can also contribute to implementing some features, functions, or applications by rising a draft pull request in a unified naming format. These pull requests will also be checked by the project maintainers.

But usually, you should communicate with the other developers and the key developers through the Discord of the project the bug fixing contribution can be communicated in the Github issues tab. The developers are more prefer to communicate with each other on Discord before deciding to implement some new features or functions. This is also indicated in the issue policy of the project.⁵

2.Implementation

After deciding what you want to contribute, fork the project to your repository which will create a copy of the project in your repository. Then

The implementation can be tested locally on your machine with the toolchain provided to rebuild the operating system. You can then use the rebuilt SerenityOS that implemented your modifications to test them.

3. Pull request

After creating a pull request, the steps shown in the figure below will be automatically done.

It will run the linter check of the commit to ensure the code structure and style satisfy the contribute policy then. Then It will test whether the SerenityOS modified by the contributor can be built on ubuntu 20.04 on x86_64 and i686 chips. And then will run the CI test through an Azure cloud server(Azure pipeline).

It will also notice the project maintainer who responds to the relative aspect of the project, this is indicated in the title of the pull request which is also required by the contribution policy.

After these automatic tests on the critical part of an operating system, one or multiply project maintainers will take a look at the code. Passing these automatic CI pipelines is not mandatory because the change will be checked by the maintainers.

Usually, they will provide some comments on some issues still needed to improve such as the coding style, or the way of realizing a certain function to let you improve it. After several times improvement and communication on Github and they saw nothing that needs to be improved, they will approve the pull request and close it.

There is a chance that the way of implementation of a function is not as what they imagine or there is no response from the contributor, the pull request will also be closed but not merged.

The rigor of the test processes and the role of test coverage

As shown in the figure in the last section for a new pull request the tests are done in the Azure pipeline, the CI pipeline runs on the Azure cloud server.

The pipeline will try to build the SerenityOS on the different operating systems, Linux and macOS. If the building is successful, then there are 122 function tests, but they are basically testing whether the SerenityOS kernel functions still working correctly after merging the modification as shown in the figure below. The TestArray, TestBase 64 are basically testing the data structures provided in the kernel of the SerenityOS.

The pipeline will also try to build on different chip structures x86_65 and i686. If the building is successful, it will also run the function tests mentioned above.

beside the building test and kernel function test, there is a Toolchain test to test whether the toolchain for building the SerenityOS is working correctly.

The tests for the newly implemented features or functions and fixed bugs are not included, whether these contribute work and whether the codes are written in good style are manually checked by the project maintainers.

Hotspot components

For analyzing hotspot components, we use CodeScene, which provides dynamic analysis of code repositories by providing various metrics and information. We will mostly use code analyzers and heatmaps. To categorize hotspots, we check the latest commit for the next version. This will give us an overview of code access points in terms of the distribution of lines of code and commits across components.

The figures below were generated using Codescene.

From the past code changes, the top three hotspot components are: Userlands, Kernel, and Meta. Userland is the one that contains all the Interface and custom UI of Serenity Os, thus it’s reasonable that Userland is one of the hotspots.

Identifying access points allows developers to discover code that is inefficient, expensive, and potentially difficult to maintain. These are potential targets for refactoring to avoid expensive changes in the future.

The largest hotspot component is the preferences dialog implementation, contained in the inkscape-preferences.cpp file, which is the large red dot on the left in the figure above. The file is responsible for the look of the preference dialog that the user can open. The largest hotspot file is Paser.cpp, which provide the CSS Parser for LibWeb library. The other hotspot is AST.cpp, which has 2986 lines of Code and 130 commits per year.

Code quality

Code quality determines the ease of understanding, writing, and maintaining code. At the most basic level, it includes maintaining a specific coding style. To keep the code of SerenityOS readable and maintainable, SerenityOS has developed standard code styles for the different languages used in the project. The most important one is the C++ Code Style, but Javascript code, HTML, shell and c code, and SerenityOS Cmake Files also have a defined code-style. There is a document in the repo that strictly require the contributors to follow, called Serenity C++ coding style, it includes the coding style for Names, Classes, Cmake, Functions, structs, Punctuation and the document illustrates many examples of each coding style requirement. Also SerenityOS has requirement for low-level styling (spaces, parentheses, brace placement, etc), all code should follow the format specified in .clang-format in the project root and Make sure the contributor use clang-format version 11 or later. According to our analysis, most contributor follow the rules when making PRs, and if not, the SerenityOS will comment their mistake patiently.

Then we can see that all the hotspots have either long method or complex method issues identified by the number of code lines and high cyclomatic complexity rating, respectively.⁶

Quality Culture

In this section, we are going to analyze the quality culture of SerenityOS as evidenced in actual discussions and tests taking place in architecturally significant feature and pull requests.

Contributions to SerenityOS have to follow the Contribution Policy, which specifies all the important things that contributors are expected to take care of when trying to make contributions. It is encouraged that pull request beginners start with small improvements to get familiar with the project and its development processes. Although everyone is welcome to work on SerenityOS, people shouldn’t start by adding a new application, library or other large components.

Most of the communication happens on the Discord server: SerenityOS Discord. Both users and develops ask questions in different channels on Discord, ranging from build-problems, development and code-review to announcement, userland and off-topic chats. For example, developers advertise their pull requests on code-review channel and ask if someone could review it. It is preferred to ask for review help on the Discord rather than because many developers turn off the Github notification due to its large volume.

The Contribution Policy, consisting of issue policy, human language policy and code submission policy, is designed to be pretty strict. Not only does it regulate the general aspect, such as:

• When contributing to SerenityOS, make sure that the changes you wish to make are in line with the project direction.
• Use proper spelling, grammar, and punctuation.
• Don’t comment on issues just to add a joke or irrelevant commentary. Hundreds of people get notified about comments so let’s keep them relevant.

It contains also more very specific rules, such as

• Choose expressive variable, function and class names. Make it as obvious as possible what the code is doing.
• One issue per bug. Putting multiple things in the same issue makes both discussion and completion unnecessarily complicated.
• The official project language is American English with ISO 8601 dates and metric units.

The maintainers are constantly reviewing and commenting on pull requests and issues made by the community. Its easy to determine how successful this system is, and how much technical debt it avoids, by looking at numerous issues and pull requests exemplifying the above-mentioned policies. For instance, pull request Shell: Better parser/AST; misc additions implemented a decent parser and AST. The responsible developer had a heated discussion with one of the project mantainers that includes over 100 chats. The code of the argv and a list of redirectons is checked and refined over the process. Finally the main author awesomekling merged the request.

The pull request Userland: Add a ’test’ utility added an implementation of the test utility just because the developer found out they had no shell tests to compare things. He mentioned that it was still not complete and had some minor bugs. Similarly, he discussed the issue with many maintainers and finally got the pull request merged.

Technical Debt

To analyse the technical debt of SerenityOS, the online tools such as Sonarcloud for analyzing the code quality should be used. However, due to it unique attribute as an operating system, SerenityOS can’t be analyzed by tools requiring building. Moreover, since the size of SerenityOS exceeds 1 million lines, it also unable to be analyzed by bettercodehub Instead, we use another online tool, codacy, to analyze the code. Following shows the result.

The grade of SerenityOS is A. Accoding to the figure, problems of this project mainly include the code style (Code formatting and syntax problems, such as variable names style and enforcing the use of brackets and quotation marks) and error prone (Code that may hide bugs and language keywords that should be used with caution).

References

---

1.  SerenityOS Manual https://man.serenityos.org/ ↩︎

2. Github SerenityOS Codingstyle https://github.com/SerenityOS/serenity/blob/master/Documentation/CodingStyle.md ↩︎

3. Github SerenityOS Contribution https://github.com/SerenityOS/serenity/blob/master/CONTRIBUTING.md ↩︎

4. SerenityOS bug bounty program https://serenityos.org/bounty/ ↩︎

5. Contributing to Serenity. Retrieved February 27, 2022, from https://github.com/SerenityOS/serenity/blob/master/CONTRIBUTING.md#issue-policy/ ↩︎

6. Code scene.Cyclomatic Complexity is a poor Predictor of Code Complexity.https://codescene.com/blog/bumpy-road-code-complexity-in-context/ ↩︎