Quality Assurance

It is easy to add features that have to work only in certain situations and not really all the time, most notably not when a new feature is added. Who cares about old features, right? Let’s just add them as fast as the code monkeys can type.

Having a lot of users is a simple matter of advertising budget. There is a reason why only software companies and drug dealers have users, while everyone else has customers or clients. Because it is about getting them hooked, not happy.

With just one big, but bad, name in music publishing software, a brand is also not a helpful criterion in choosing the right software.

So what is? What can one do to make sure that a software has the required features, that they work and will keep working? No idea. Seriously. All I can tell you is how this project ensures that features work and will keep working. With Django Music Publisher being an open source software, all of it is verifiable.

Architecture and tech

First of all, the general architecture and used technologies were carefully selected. The crucial part is a SQL database. A SQL over a no-SQL choice is pretty obvious. The data structure in music publishing is well defined, and with time, data on any element grows. If I have to choose, I go with PostgreSQL, but I prefer to go with a database-agnostic solution for as long as possible.

The code had to be in one of the popular and easily readable programming languages with a good web framework. A web interface is an obvious choice. Basically, this narrows it down to Ruby on Rails and Python/Django, as both come with very powerful built-in interfaces. My personal preference is Python.

Focus

Second, focus. By focusing on original publishers only, the data structure remains relatively simple. This is important for a project that aspires to be community driven. While anyone can fork it and add complexity, Django Music Publisher will keep this focus. The data structure is compatible with CWR, EBR and many spreadsheet formats used by various services, sub-publishers and other entities, so adding new import/export capabilities is a straightforward process.

Code Quality

Third, the code. It has to be self-explanatory to experienced developers. It has to adhere to all the good principles, rules and standard ways of doing things. And it has to contain unit and functional tests, as well as all the features required for continuous integration and deployment. Since this is an article for non-developers, some introduction is required.

Unit tests and functional tests

Unit tests are pieces of code that test other pieces of code. If there is a code that adds a work, then it has to be tested, both with valid data that will actually create a work instance and bad data, that will result in exceptions being raised. Functional (aka integration) tests are testing that small pieces of code work together flawlessly. An example would be creating a new work through the interface. It is not just creating a work instance here, it is about the view with appropriate form sets being shown, that they accept values and that the values can be turned into objects, that are individually and jointly validated and saved, or, in case of issues, that appropriate error messages are being reported back to the user. Ideally, 100% of code should be tested in both ways, but that is a bit unpractical. Tests in Django Music Publisher currently cover about 98% of the code.

This is very important, as adding new features or code refactoring does not require a human to test everything every time new code is deployed. This is done automatically in a process called continuous integration. Each time a code is pushed to the repository (this is the place where the code lives, Django Music Publisher uses GitHub), many things happen.

Continous Integration

There are several good services for continuous integration, and Django Music Publisher uses Travis CI, a service that is free for open source projects. When Travis CI detects changes to the repository, it fetches the new code, installs it in several different environments and runs all the tests. The reason why several environments are required is that Django Music Publisher must work in all of them. So, it is tested with three different versions of Python (3.5, 3.6, 3.7), two different versions of Django (2.0 and 2.1) and with two different SQL databases (SQLite3 and PostgreSQL).

As a comparison, if a developer would need to run all the tests in all the environments manually, it would take at least an hour, while testing everything manually would take a day. This does not ensure that all bugs will be detected, but in reality, it detects almost all of them. Those few that get through and are triggered by users are reported to the developers and dealt with as soon as possible. For our installations, we use Sentry.

The result of the test is then reported to another service, in our case it is Coveralls, that helps with tracking test coverage over time.

Continous Deployment

If all tests pass, the code is automatically deployed to our test server. For this, we use the free tier of Heroku. If there is a deployment issue, Heroku reports this, so it can be dealt with immediately. We occasionally define a “stable release” that can then be automatically deployed to production environments.

Conclusion

While Django Music Publisher has a narrow focus and limited features, and, at this point, only a few publishers that use it in production, the quality assurance procedures are as good as it gets in 2018., due to the fact that all these services are free for open source projects, but also due to the fact that a lot of time was invested in this early on.

Open source is not just about the code being open, it is also about the process being totally transparent and open to reviews. As stated earlier, all of these claims are verifiable. While closed source solutions may make similar claims, as I do for my closed source projects, there is no way for a user to verify them. So, to answer the question what can one do to make sure that a software has the required features, that they work and will keep working has a very different answer in an open environment. Here, one can verify.