Since a while, the Yaal Coop team is working on Canaille, an identity and authorization management software (or IAM). At the occasion of the end of an endeavour that benefited the help of the NLNet foundation, we want to tell the history around Canaille.
At first was the Single Sign On
At Yaal Coop (like anywhere else) we use a bunch of tools to work together as a team. Emails, files, address book, project management, time tracking, accountability, continuous integration, bug reporting… and the list goes on. For the comfort of use (and for security), those tools are generally plugged to a central software that manages user accounts and accesses. Usually, LDAP1 is used for that, and this is called a directory. If a service is connected to a directory, you do not need to register to that service, you can just use your usual password, the one you set in the directory. It is convenient for you since you just have one single password to remember (or to forget). This is what is called Single Sign On (SSO). It is also convenient for people managing the user accounts in your organization, as they can create and revoke accounts for all the plugged services at a time.
LDAP is old and robust, and for software those are qualities. The few decades of experience behind LDAP make it a battle-tested technology compatible with a large selection of services. But unfortunately LDAP is also complicated and abstruse.
Its tree model looks like few other things you might know, the documentation is rare and obscure, conventions are intriguing, with a lot of distressing acronyms, tooling is sparse… Also, this was made for data often read but rarely written. The data models LDAP offers cannot easily evolve. Generally, the provided ones allow to handle basic attributes on users and groups. If you want to manage more advanced aspects like the favourite color of users, then you need to write your own data models2. And you'd better be confident when you do it, because no update mechanism is provided for neither data nor data models3. Then, if you wish to share your data models with the world, it is advised that your register at the IANA4, and provide an online reference of your schemas.
When we took a close look at the LDAP universe, we had the impression that once set up, LDAP installation were left alone for years ; like if people who might had knowledge to share had flew the austerity of that technology without documenting their adventures, and without need to returning to it because it just works. I am barely exaggerating, if we have a look5 at the number of questions in stackoverflow, ldap has approximately 1000 questions, while postgresql or mongodb have around 14.000, and mysql has 45.000.
Then came the Single Log In
During our volunteer time at Supercoop6, among other tasks, we worked to install collaboration software, like we did for ourselves. Obviously, those tools were plugged on a LDAP directory. We noticed that the members management team of the supermarket was spending a certain amount of time dealing with user support, notably with password reset demands. People were failing to use the tools we deployed for them as soon as the first login step. This verdict pushed us to look for Single Login Identification (SLI) software (free and open-source, obviously), in replacement or in addition to our historical directory. What SLI brings, is that users can navigate between services without having to sign in every time. They can switch from the webmail interface to the calendar without having to make a step by the authentication screen, while with a simple directory, they would have to enter their password once for each service. For a non tech-savvy public, this is the opportinity to get rid of several technical screens, limit the error sources and the frustration.
Currently the predilection protocols for SLI are OAuth2 and OpenID Connect (OIDC)7. (There are other existing standards, more historical or more focused towards big organizations, like SAML or CAS.) The OAuth2 and OIDC specifications are recent but they still benefit from a decade of feedback. Tooling and documentation are numerous, the community is alive and active, and standards are still evolving at this day with the work of the IETF oauth workgroup. But here again, everything is not perfect and OIDC comes with its difficulties. First of all, OAuth2 and OIDC make about 30 different standards, some of which contradicting or bringing corrections upon prior standards. All of those are not pertinent for what we want to do, but simply understanding how work the few standards that actually interests us already requires a certain amount of time and work. The OIDC implementations – by the IAMs or the services connected to IAMs – are sometimes partial or bugged. To make nothing better, big industry actors sometimes deliberately take some liberties with the standards, and make some additional developments a requirements to be compatible with their software.8
Let's get back on topic. For Supercoop, we wanted to find a tool that would be utterly simple from several point of views:
- for the final users: the volunteers of the shop. They must be able to use the tool without any prior technical knowledge, and without help;
- for the members management team. We wanted a relatively simple tool where a little bit of learning would be acceptable but not desirable. As the cooperative supermarket operates with benevolent labour, we want to take care of the volunteer time and avoid disagreeable tasks.
- for the IT team. Here again simplicity is at stake. If there is a too steep learning curve on the tools the supermarket uses, there will be an issue for recruiting competent people to help us. The nature of the benevolent participation of the shop make that volunteers are helping punctually, and rarely engage for years, so they do not always have the time to learn the whole technical stack. Ideally the IT team should be able to administrate the tool we are seeking with a superficial knowledge of the underlying protocols. And with the descriptions I made of LDAP and OIDC, you start to make an idea of the problem we have.
Our search showed us that there was a dozen of tools that could answer our needs. However, none was fitting all our expectations. Actually, by their very nature, the IAM software are intended to big organizations, that consequently can afford to hire competent people to administrate them, so complexity is not really an issue. Those are mostly big powerful software, with a large protocol compatibility not pertinent for us, and requiring some technical knowledge for installation, maintenance and administration. Pushed by our curiosity, we finally hacked something on our side.
Canaille
A few times later, our prototype had grown bigger and became Canaille.
The features
Canaille is a user and group management software that allows account creation, edition, administration, registration, forgotten password reset, user invitation etc. In addition, Canaille implements OIDC and provide a SLI layer for other services. Its interface is customizable and… that's all. We try to keep the functional scope tight so Canaille can stay simple to maintain and to operate.
Canaille can be used for:
- bringing an account edition interface on top of a LDAP directory. The tooling around LDAP being quite sparse, Canaille can be a convenient option and allow utilities like password reset.
- bringing a SLI layer on top of a LDAP directory. If your organization has a historical LDAP directory and you want to modernize your stack with SLI, Canaille can discretely integrate your existing stack (almost) without modifying anything, and let you the time to prepare (or not) a migration.
- develop applications relying upon an OIDC server. If, like us, you are doing software development and regularly work on applications that needs to connect to a OIDC server, then you can use Canaille in your development environment. Canaille being very light, this is now our tool of choice to work on client applications.
- test applications relying upon an OIDC server. We also use canaille in unit tests suites, with pytest-iam. This is a pytest plugin that configures and run a Canaille instance so your unit tests suite can achieve actual OIDC connections.
The technical choices
Canaille is written in Python, with Flask for the backend and fomantic-ui and htmx for the frontend. We like the simplicity of Python and Flask, and strongly believe those are arguments that make contributions easier and increase software longevity. We have been shocked by our discovery of HTMX earlier this year, that in short allows us to build dynamical pages without writing Javascript. One language less is as much less complexity in the project.
Under the hood, Canaille can be plugged to a LDAP directory or a SQL database.
The development techniques
We use Canaille in production, at Supercoop where it is used to manage 1800 users, but also in our own stack in our Nubla mutualized cloud services, and on dedicated Nubla Pro instances. That large userbase allows us to collect user feedback, and we improve Canaille gradually in reaction to demands and according to our roadmap.
Cannaille development follows the principles of the Test Driven Development (TDD) when possible. The code coverage reaches 100% (including branches), and this is a prerequisite for new contributions. In addition to giving us confidence in our tests, this allowed us to simply delete unused pieces of code. Bugs are easier to find when there is less code to search them into.
Canaille uses all the modern Python source code linters and formatters, with pre-commit, and the unit tests are run on all the officially supported Python versions.
Finally, the translation of Canaille is done by the community with Weblate.
Contributions
Like most of modern software, Canaille relies on libraries so we don't have to re-invent what is already existing.
At Yaal Coop for all our projects, when we find bugs in libraries, we follow a strategy that consists in implementing fixes both in our code and the libraries at the same time. At first we patch our code with temporary but functional fixes. However all that code we write is a maintenance burden to come, so in parallel we discuss with the maintainers of those libraries, and propose better conceived long term fixes. This is for us a guarantee of the sustainability of those pieces of software we rely upon, and this is a way to contribute to that necessary FLOSS ecosystem. Once the upstream library is patched, we delete our quick fix.
In the same fashion when we wish additional features in libraries, we discuss our motivations with maintainers and sometimes provide implementations. We have for instance developed the OIDC specifications RFC7591 and RFC9068 in authlib, a library Canaille uses to deal with OIDC.
The results of those efforts can be seen in our seasonal blogposts about our FLOSS contributions.
The chance of the contributions brought us to share the maintenance of certain libraries we rely upon, like wtforms that is a form management tool, or nextcloud-oidc-login and the underlying OpenIDConnect-PHP that allow nextcloud (the file manager integrated in Nubla) to connect with Canaille.
We also modestly sponsor the maintainers of fomantic-ui and authlib on a monthly basis.
The help of the NLNet foundation
Canaille has been developed to answer specific needs we met: initially Canaille was only made to work with LDAP directories. Watching the ecosystem, we noticed that more and more software became compatible with OIDC in addition to LDAP. Ultimately, if no service we use directly rely upon LDAP, then we could consider a replacement database with which we feel more comfortable. In addition to this, it is more likely that people interested by the Canaille approach will have experience with things like SQL than with LDAP. To go towards this directory, we sollicitated the NGI Zero Entrust fund of the NLNet foundation in the late 2022, for a amount of 7.000€, to help us realize tasks on Canaille (and around):
- database connection genericity. This was a mandatory preparatory step to plugging additional databases than LDAP;
- SQL databases support. This allows Canaille to connect to a lot of widespread databases like postgresql, mariadb and sqlite, thanks to sqlalchemy.
- pytest-iam. This is a tool that can prepare and run a Canaille instance in the purpose of unit testing Python applications with pytest.
Those developments have been achieved in november 2023, and have implied numerous refactoring, bugfixes, and better conception and reliability on Canaille. We are very grateful towards the NLNet foundation for having allowed us to work on our tool, and help at our scale to contribute to the free and open source ecosystem.
What's next?
Go in beta version
From the beginning we have kept Canaille in alpha, we are currently at version 0.0.35.
We advise in the documentation that the software is still not suited for production environments.
This was for us a way to experiment and break things without having to be worried about compatibility.
We would like to break one last little thing by re-thinking the configuration files before we let Canaille go in bêta, and remove usage warnings.
We track the beta version tasks in this milestone.
Go in stable version
The work we intend to realize to go from a bêta version to a finale version will essentially consists in documentation efforts and ease of installation and usage of Canaille. The goal is to make Canaille really easy to access, in order to create usage and collect more user feedback.
We track the finale version tasks in this milestone.
And beyond…
In the future, we would like to work on provisioning, by implementing the SCIM specification. This will allow to decouple user authentication and account creation. Indeed, at the moment user accounts are created the first time users log in at our services. This can raise issues in some situations: for instance one cannot share files in Nextcloud with users who have not log in yet. SCIM also provides a mechanism to delete user accounts in those services, that is a thing we do more or less manually at the moment.
Among the things we look forward to implement, there are also the Single Log-Out (SLO) OIDC specifications. This will allow users to disconnect from all the services in a single action. SLO is a feature that will bring security and comfort of use. This will need to be developed in authlib in a first time.
At last, we would like to work on advanced group management and permissions, and on multi-factor authentication to bring even more flexibility and security in Canaille.
Canaille is open to contributions, if you want to help us in this endeavour, the door is open!
- or more exactly, a software implementing the LDAP protocol ↩︎
- that are called schemas ↩︎
- at least with OpenLDAP ↩︎
- what we did, our IANA number is 56207 ↩︎
- OK, the comparison has limits since those different databases are not used for the same needs, but the magnitude of the numbers says something about the interest around those technologies ↩︎
- a cooperative supermarket near Bordeaux, France ↩︎
- for sake of simplicity, I will keep to OIDC ↩︎
- They can also be a driving force for new features, like RFC7628 or RFC9068 that were commonly used before they became standards. ↩︎