Quality and Evolution

Sonic Pi: Quality and Evolution

In the previous essays, we discussed about the project vision and the architectural view of Sonic Pi. In this essay we will discuss about the quality control of Sonic Pi, and its evolution process.

Key attributes

Figure: The key quality attributes of Sonic Pi

Figure ‘The key quality attributes of Sonic Pi’ shows the key quality attributes of Sonic Pi, where we divide them into external qualities and internal qualities.

External Qualities

Sonic Pi’s strong functionality and usability has been proved by the growing number of its users. It produces sounds upon codes with a high correctness. For learnability, the detailed and inspiring tutorial 1 provides the beginners with a friendly and quick learning experience. As for efficiency, the Supercollider music synthesis tool is always ready for producing output sounds on a thread different from the main GUI thread. This process wouldn’t hamper the performance of visuals and sounds and finishes quickly, thus efficient.

However, completedness is calling for constant improvement. A considerable number of users (and developers) are putting issues on Github suggesting additional synths such as trumpet, violin and less-popular instruments. 2 This is mainly because the synth tool, SuperCollider, that Sonic Pi uses contains only limited instrument synths. Another issue is that, the translations for the tutorials are far from complete. Also, the language of the GUI is currently not configurable. 3

Flexibility in general is also an issue for Sonic Pi. Boot issues are commonly seen in the community, which stresses the configurability problem: Sonic Pi is not configurable enough to cope with different versions of OS environments.

Internal Qualities

The maintainability of Sonic Pi proves to be strong. Since there are a core team is constantly checking on Github for issues and pull request. This is discussed in details in section Quality Culture. From the code level, the C++ and Ruby codes are in constant style. Classes and functions are developed in strict namespace classifications. Functions are clearly divided and referenced with proper naming. These are all important traits that facilitates the maintain process.

The tests of Sonic Pi are also well-developed. details about the rigors of tests and test coverage will be discussed in Section “The Test Process”.

Overall Quality Process

As an open source software, it is very important for contributors to keep in mind the requirements they should follow and the right direction to improve towards. Thus the documentations provided by the core team can be a key to ensuring the code quality.

Sonic Pi provides guidelines for contributors, testers, and maintainers. In each document, the requirements for contributing is mentioned. For instance, in the file contributing.md, they specified that features should be simple enough for children to understand. 4 The core team actively interact with contributors on Github to resolve the issues and deal with pull requests.

Apart from documentations on Github, there is a Sonic Pi community called “in_thread” 5 where developers share there ideas such as posting a very detailed tutorial for writing SuperCollider files for additional synths.

All these efforts have provided an open but also strict environment for Sonic Pi’s contributers and kept them inline with the core aim of the project.

Continuous Integration

The continuous integration process of Sonic Pi is done together by the developer and Github in the simplest way. This helps to ensure the overall quality of the project as mentioned above, and also make it easy for developers to contribute.

First, the developer should pass a suite of API tests on the local machine. This is strongly recommended to be done before sending the pull request. After passing the API tests, there will be a larger suite of test built by the Sonic Pi core team in the Github pipeline that checks whether the pulled code could be execute normally on different platforms. The final step before a pull request is merged is the approval of Sam Aaron. As mentioned in contributing.md, the codes must be aligned with Sam’s vision to the project, and be maintainable for him.

The Test Process

In this section, we discuss about the rigor of the tests as well as Sonic Pi’s test coverage. In our point of view, the rigorousness of the tests of Sonic Pi lies in the fact that it has achieved certain amount of test coverage. From the platform level, it developed tests for different OS platforms. From the dependency level, it ran tests on Ruby, Qt, BEAM, elixir, which are essential dependencies for Sonic Pi to run. From the application level, it tested a myriad of functions in the API. Passing all these tests is a prerequisite for a pull request to be merged.

However, there is still something outside the current testing coverage that plagues the software from being used by all users. In the Issues, we see currently lots of people complaining about unable to run Sonic Pi on their OS, mostly Windows and Linux.

Hotspot components from the past and the future

Hotspot components represent the ones that have the highest development activity in the software but it does not mean there are quality issues inside and they normally represent the modules that play important roles in the business 6. The hotspot components analysis is conducted by leveraging a tool called CodeScene 7, which helps developers dig out the potential risks in the code to reduce the technical debt. Sonarqube is also one of the popular code analysis tools in this field but it mainly focuses on the static analysis instead of the temporal one8.

Figure: File-level hotspot map

Figure: Component-level hotspot map

As shown in the Figure ‘File-level hotspot map’, most hotspot files are involved in the server and gui components. In the hotspot map, if the circle is redder, then the development activity is more frequent inside. If we switch to Figure ‘Component-level hotspot map’, we can see the server is the redest hotspot and the gui is the second redest one since the server provides the core functionality of sonic pi and it is also a crucial part of the business. According to the roadmap of Sonic Pi9, the task with the highest priority is the stability of the server and the second priority is the user interface. In this case, the hotspot components from the future will be still server and gui.

Code quality on hotspot components

To analyse the code quality, we focus on the server component, which provides the core functionalities and is developed most frequently in Sonic Pi. As shown in the Figure ‘Code health score overview’, the code health score of the server component is 3.32 out of 10 according to the analysis outcome since there are three files with low scores inside namely sound.rb, core.rb, and synthinfo.rb. Low cohesion, Bumpy road, and Deeply nested complexity are the main issues that degrade their code health scores. The Low cohesion issue means, if the cohesion is low, the file contains multiple behaviours and the functions inside call each other, which could make code harder to understand and more tests required10. One of the possible reasons that the code with different behaviours is involved in a single file is software had to be launched in a short time so that there was less time for the optimization. To solve this problem, the file can be divided into modules with more cohesion. Bumpy road issue means there is a function that contains multiple nested conditional statements in the file, which would increase the refactoring costs. In the sound.rb, the structure with nested conditional statements is for bus creation. To address this issue, some parts of code inside the function with nested conditional statements can be encapsulated if possible. Bumpy road usually comes with deeply nested complexity and it could lead to the growth of the cognitive load for developers while reading code. Refactoring nested logic can help after the bumpy road issue is resolved11.

Figure: Code health score overview

Figure: File with low health scores

Quality Culture

Quality culture12 refers to the establishment of an environment in which the quality goal is achieved by the members’ trust, participation and communication. Sonic Pi is making a good process in motivating the third party contributors to involve within the development and maintenance. The core team does provide a clear picture13 to the third party contributors what is and will be feature plans of Sonic Pi.

Figure: A screen shot of the project plan

Sonic Pi team also behaves actively in responding to those contributors. The most current issue #3038 was proposed 6 days ago and closed in 3 days. And for issue #2935, the problem ‘can’t run sonic pi on chromebook’ was reported. This issue is still labeled as an open issue, but the member of Sonic Pi replied to this post with a detailed solution. The users can even ask question about their code like issue#2117 #2677 and received timely and warm replies. These indicate the core team is continuously paying attention to those third party contributors. Because of the encouragement given from the core team, the whole community works together positively to reach those qualities attributes mentioned in the previous posts:

Aspect relevant issues/ pull requests(PR)
Code Readability Typos fix: PR #3005, PR#2999, PR#2967. Remove unused string variable: PR#3004.
Adaptation Boot error:Reported by several issues such as Issue#2982, Issue#2986, Issue#3000, and some of them are fixed by PR such as PR#2839.
Functionality Trigger the bell synths with amp: 0: reported by Issue#2730 and fixed by PR#2731. Addition of a new tonewheel organ synth: PR#2991. Fix wrong output of current_debug function: reported by Issue#2932, and fixed in PR#2933.
Usage Simplicity Perfection of translation: PR#2825, PR#2836. Report of missing documentation: Issue#3002.

Technical Debt of Sonic Pi

The term14 was coined by Ward Cunningham. Technical debt can be understood as the sub-optimal solution for delivering the feature on time, and the developer has to spend time to polish and improve the product in the future to pay the debt back. The technical debt is experienced by all systems more or less, and Sonic Pi is not an exception.

As mentioned in the first post, Sonic Pi was designed as a software that helps teaching computer science and music for educational purposes, but it was later developed into a widely used code-based music creation software. The change of the objective causes an insufficient up-front definition. This implies a continuous redefinition of the requirements, and is proven by version updating history15. More than 20 versions have been updated, and the Version 4.0.0 is expected to be released soon with solving problems like adaptive problems in macOS, extra functionalities on GUI and ect. . However, a reduction in version updating frequency also shows the technical debt of Sonic Pi is getting smaller in the recent two years.

Another proof of the existence of technical debt is the number of open issues. There are 480 open issues in its repository currently, the oldest of open issues was reported in October of 2014, and the closest one happened in one month. 22 of them are under the bug label. This indicates the whole system is still repaying its debts.

Besides the perspectives mentioned above, the quality of the current code which was analyzed in the previous section also indicates potential technical debt still exists because of the low score caused by low cohesion, bumpy road and deeply nested complexity.

References

  1. The tutorial of Sonic Pi: https://github.com/sonic-pi-net/sonic-pi/tree/dev/etc/doc/tutorial ↩︎

  2. An issue about lack of synths: https://github.com/sonic-pi-net/sonic-pi/issues/2411 ↩︎

  3. An issue about GUI translations: https://github.com/sonic-pi-net/sonic-pi/pull/2604 ↩︎

  4. The contributing readme file, https://github.com/sonic-pi-net/sonic-pi/blob/dev/CONTRIBUTING.md ↩︎

  5. Sonic Pi community: https://in-thread.sonic-pi.net ↩︎

  6. Hotspots, https://codescene.io/docs/terminology/codescene-terminology.html#hotspots ↩︎

  7. Codescene, https://codescene.com/ ↩︎

  8. Codescene vs Sonarqube, https://systemverification.com/en/articles/difference-static-code-analysis-codescene/ ↩︎

  9. Sonic Pi - Product Vision and Problem Analysis, https://desosa2022.netlify.app/projects/sonic-pi/posts/essay1/ ↩︎

  10. Bio markers, https://docs.enterprise.codescene.io/versions/2.8.0/guides/technical/biomarkers.html ↩︎

  11. Bumpy road, https://codescene.com/blog/bumpy-road-code-complexity-in-context/?__hstc=175703037.bc5a0f1d77c643a5933344edbf446a79.1647274447494.1647460373982.1647466823401.9&__hssc=175703037.27.1647531291473&__hsfp=570208402&_ga=2.7584887.477167447.1647274445-1181300213.1647274445 ↩︎

  12. Quality culture, https://www.testify.io/en/qualityculture-for-successful-qualitymanagement/ ↩︎

  13. Features, https://github.com/orgs/sonic-pi-net/projects/1 ↩︎

  14. debt metaphor, https://www.youtube.com/watch?v=pqeJFYwnkjE) ↩︎

  15. Changes , https://github.com/sonic-pi-net/sonic-pi/blob/dev/CHANGELOG.md ↩︎