Dolphin - Quality and Evolution

Dolphin is an emulator for the GameCube (GC) and Wii and so its main capability is being able to play GC and Wii games. The key quality attributes of the system are therefore mostly focused on functionality and operation attributes. Correctness and completeness, for example, are two main key quality attributes. Correctness, in this case, points to the fact that the game runs exactly as it does on real GC and Wii hardware. Although this would be ideal, unfortunately only 36% of the games run perfectly¹. This is mainly because it is very difficult to emulate the CPU of the GC and the Wii perfectly². The completeness attribute makes sure that users can play the full game and use all its functionalities instead of playing a partial game. Currently, almost 97% of the games can be fully played, so completeness is definitely achieved.

An attribute closely related to this is reliability. Reliability is important when playing games since you do not want Dolphin to crash while playing. Games from the GC and Wii era usually do not feature autosave, so if Dolphin crashes, players would lose all their progress after their last manual save. Luckily, like with completeness, only 3% of all games are flagged to sometimes crash. Therefore the reliability attribute is also satisfied.

Two other key attributes that can be identified for Dolphin are usability and performance. GC and Wii games usually run at a locked framerate. Running these games at a lower frame rate slows down the game speed. It is therefore important for Dolphin to perform well enough to consistently meet the target framerate. This is why, next to completeness, the games also need to run smoothly for them to be usable and enjoyable. As 60% of the games are said to have minor audio or graphical issues, the performance and usability attributes can still be improved. Although they are currently satisfied for the most part.

Last, aesthetics play a big role for Dolphin as the graphics of a game can impact playability. Dolphin strives to make games look as accurate as possible to the games on original hardware, but it also has optional settings to make them look even better, such as increased internal resolution, widescreen hacks, and support for HD textures. As mentioned before, a lot of games cope with minor graphical issues, so this attribute is satisfied but not perfectly achieved.

Code quality process

To make sure that the code quality stays at a high level, Dolphin has guidelines for contributions³. In these guidelines are the general rules for code style and formatting like variable names, indentation style, comment usage etc. To make it easier for contributors to follow these guidelines, a clang-format⁴ code reformatter is included in the Dolphin source code. By simply selecting the “Format Document” option in Visual Studio, the code gets automatically reformatted according to the guidelines of Dolphin. This means that parts like includes get sorted and redundant empty lines get removed. Finally, the code for the Android version of Dolphin has its own code style file that will help developers format their Java code.

For most of the C++ files, there is no documentation or comment about what a certain class is supposed to do. Methods also do not contain any information on their purpose for the most part. This makes the code a lot shorter, but also more difficult to understand. Placing comments for all the classes and functions now would simply take too much time, which can be seen as a flaw in the software quality of Dolphin.

Continuous integration process

There are a lot of developers contributing to Dolphin simultaneously. It is important to have continuous integration (CI) processes in place. These CI processes ensure that every new code change is automatically logically tested and the format is checked. The CI pipeline lowers the workload for code reviewers by rapidly detecting possible bugs.

When a contributor wants to make a code change to Dolphin, he has to create a pull request on GitHub. GitHub will automatically start the CI processes of the new Dolphin version. If all processes succeed, the pull request can be merged by a Dolphin reviewer.

Dolphins CI processes are hosted on different platforms ⁵. Dolphins Central project was developed to be the central notification hub that dispatches and translates events between systems to tie them all together. Central sends instructions to buildbot⁶ master to start certain CI processes. There are 2 main CI process groups that can be defined, build processes and test processes. For every merge to master, buildbot makes sure that there will be a new version for Linux, macOS, Windows, and Android built and deployed on the website⁷. At the same time, buildbot starts the tests processes, details on these will be further explained in the next part.

Dolphin has numerous language options in the application. The developer team chose Transifex as a platform to add and maintain new languages. Everyone can file a request to get edit or review rights in a language of their choice. Once every couple of months, the language files in the repository get updated to the current version on Transifex.

Testing process

Writing good tests for Dolphin is hard. The emulator code is mostly quite old, complicated and GC and Wii specific. To tackle this problem, the Dolphin team released their first version of the FifoCI project in 2015⁸. Dolphin supports recording the data being transmitted between the emulated CPU and emulated GPU. The data is stored in FIFO logs which can be replayed after the fact. This feature enables the possibility to compare these transmitted instructions between different Dolphin versions and even to the real console. The FifoCI project does exactly this. For every new Dolphin version, the FIFO logs are generated for a predefined set of instructions. These logs are then compared to previous versions and notify the developers if there are significant changes.

However, this clever method of emulator integration testing has some flaws. FifoCI is in fact a black-box test. Predefined instructions are used as input and logs are received as output. There is no way to see where a possible bug is situated. Another flaw of this black box is that it is impossible to calculate the test coverage and generate a report. A good thing is that it is fairly easy to create new tests as the desired output for a set of instructions can be generated using the Wii itself.

Components of main development

Since Dolphin’s launch back in 2003, its components and the frequency of their changes have shifted a lot. Throughout its lifespan, a primary focus has always been the accuracy of the emulation relating to the original consoles. In the past this was related to the entire architecture, especially making sure that everything related to the console’s memory is handled. Later on, Dolphin 4.0⁹, released in 2013, added major features including beta support for the Wii official online multiplayer, Alpha support for ARM/Android and an all-new main audio emulation technique. These feature updates were packed together with a new user interface (UI) design, controller support and a lot of general optimizations. Dolphin 5.0¹⁰ was released in 2016 and is currently the latest major version release of the Dolphin emulator. Its release announcement states:

“The three previous releases followed a very distinct pattern: sacrifice performance, hacks, and features in exchange for higher accuracy. As such, Dolphin 3.0, 3.5, and 4.0 progressively grew slower. But thanks to the cleanups put forward throughout those releases, Dolphin 5.0 is the fastest Dolphin has ever been!”

The main focus as stated was optimizing performance, which they did by getting rid of the x86_32 JIT, forcing a 64-bit implementation. A lot of focus was also on the audio and visual performance with new stereoscopic 3D support and a reduction in audio latency by over 80ms.

One thing that stands out from the announcement of Dolphin 5.0 is that they regard the three versions leading up to Dolphin 5.0 as “performance sacrifices and hacks”. This is a great example of code quality being sacrificed for accuracy. Components that were changed in the 5.0 release like the Audio and Visual components had to be completely reworked to gain back performance.

In recent years after the release of Dolphin 5.0 and the coming future, the focus has been more minor feature-driven and game-specific regarding buggy behavior like visual and audio emulation glitches.

Code quality culture

A lot of pull requests (PRs) from JosJuice get merged with little discussion if any. This makes sense as JosJuice has been contributing to Dolphin for years. Not only would he know what he is doing and what the Dolphin quality standards are, but the other contributors would also trust him a lot because of that. For example pull requests #9973, #10401, #10162 and #10055.

PRs from other contributors do seem to get some more discussion. Sometimes they ask for screenshots if the changes have a visual impact. For example pull requests #9956, #10257, #10369 and #10389.

A few PRs from other contributors do get merged without much discussion. For example pull requests #10204, #10244, #10279 and #10215.

It looks like some contributors have a greater influence over certain features than others. For example, it seems MayImilae decides what all UI elements look like. All visual changes have to be approved by her first. For example pull requests #10506 and #10337.

Technical debts

It is natural for an emulator in general to accumulate a lot of technical debt in its early days. It is usually only developed by a few people when very little is known about the system. They have to make a lot of assumptions about how the system works as they are essentially reverse-engineering a black box system. Even if they did implement a feature perfectly, there would be no way to verify it. Most of the time they make flawed assumptions that somehow work out well enough and they continue development from there. This is fine at that time as the goal is to learn about the system and get as much basic functionality to work as soon as possible. As the emulator matures, the goals shift more towards game compatibility, accuracy, and performance. The results of the flawed assumptions become more apparent, but by then it is very difficult to fix as years of progress would have to be rewritten.

Unfortunately, Dolphin is no different. One example of this is the DSP-HLE (Digital Signal Processor - High-Level Emulation) audio backend. A fundamental flaw of this backend used to be that it processed audio asynchronously with the main CPU thread whereas the GC and Wii processed it synchronously. It worked fine for most games most of the time, but it also caused some issues that simply cannot be worked around. Because of this, the DSP-HLE had to be rewritten in 2014 to process the audio synchronously ¹¹.

As for Dolphin’s technical depth today, it is impossible to tell exactly. Developers are still learning about the GC and Wii hardware and software to this day, so a lot of assumptions made back then still don’t have definitive answers now. But it is fair to say that, with its 19-year-old legacy and poor documentation, Dolphin still has a lot to catch up on.