[Evaluating Free Software] Let’s de-google-ify evaluation: how does Framasoft choose its open source software?

How Framasoft Evaluates Free Software

Walid : Here we are again for a new episode of Projets Libres!. Today, it’s the second episode of a new series called “How to evaluate free software”, the first episode of which was the round table we did with my friend Raphaël, here at the Open Source Experience Show in December 2024. And so, this new series, “How to evaluate free software” we will receive guests and ask them, in their context, how and why they evaluate free software. And so, for the first guests of this new series, we talk about Framasoft again. No, I’m not sponsored by Framasoft, even though it’s the fourth time they’ve been on the podcast. But this episode starts from a discussion I had with Pierre-Yves at the Rencontres Professionnelles du Logiciel Libre. And at the end of the discussion, I said to myself: I’m keeping this in my head, it’s going to be very interesting. And so today, to talk about how Framasoft evaluates free software, I’m lucky enough to have two people with me. I have Thomas Citharel and Pierre-Yves Gosset (Editor’s note: PYG). And with them, we will talk about this subject. And of course, my friend Raphaël Semeteys. If you want to know why I’m doing these episodes with him, I strongly advise you to listen to the round table. He will explain what QSOS is.
Listen, gentlemen, welcome. Hello to all three of you.

Pierre-Yves : Hello.

Thomas : Hello.

Raphaël : Hello.

Presentation of the guests

Walid : So, to start the presentations, there are two of them whose presentations we are going to make very quick because they have already intervened. It’s Pierre-Yves and it’s Raphaël. And the third, who will present himself a little more thoroughly, is Thomas. And so, Thomas, I suggest you start. I give you the floor and welcome to the podcast Projets Libres!.

Thomas : Thank you very much for the invitation. I’ve been at Framasoft for about nine years. I started as an intern who was finishing his studies and looking for an internship at the end of my studies. At the time, I was in the community of a software called Wallabag which, as a result, had relations with Framasoft. In particular because Framasoft hosted an instance of Wallabag called Framabag. And so, I was taken on an internship at that time. I contributed: the purpose of my internship was to create some features for a software called NextCloud that we then released in the Framagenda service. So, these are features that were developed for Framagenda and that were transferred at the same time to NextCloud, which were released around mid-2016. And then, I was lucky enough to be recruited as a developer, with several projects, most of which are no longer there today, but which you may have heard of, such as Framaslide, Framasite, Framastory. After that, I also continued to work on NextCloud, and in particular managed to set up the Framaspace we have today. Afterwards, I was a project manager for a federated project called Mobilizon, which Framasoft developed and supported between about 2018 and the end of 2023. And after this role, I had the opportunity to become co-director. I think it’s a perfect transition for PYG to actually explain who he is today and why. I’m co-director.

Pierre-Yves : But you’re co-director because you’re already worth it! So yes, hello everyone, I’m Pierre-Yves Gosset. Indeed, I have already intervened several times, so I was a volunteer, then general delegate, then director, then co-director. And today, I am Framasoft’s digital services coordinator, since I handed over my role as co-director to Thomas in March 2024.

Walid : Very good. And you, Raphael?

Raphaël : Hello everyone, Raphaël Semeteys. I’ve been in the free software for quite some time. We worked together, Walid, a few years ago, when we were at Atos.

Walid : A long time.

Raphaël : At the Open Source Competence Center that I helped create and where I worked for almost ten years. I was pretty much involved in all the board and business advice on the adoption of free software, the risks, how to adopt well, how to benefit from this adoption, so contribute, that kind of thing. And in this context, yes, indeed, I was doing a lot of Veille Techno studies for clients, ministries, and I designed this QSOS method, which didn’t really invent hot water, but which had the merit, and still has, of formalizing a little bit of an approach on how to evaluate projects and the software they produce from a user’s point of view.

Walid : Great. So, if now, we get to the heart of the matter, for people who are interested, who want to know more about what we’re going to talk about, there are the two episodes on the history of Framasoft (Editor’s note: part 1 and part 2), in which we certainly go into much more detail in the story and in the projects. Here, what we’re going to do is talk more about how this software is chosen, how it’s hosted, etc. So there you go, if you want to know more and you haven’t yet listened to the two episodes on the history of Framasoft, it might be a good time too, to listen to them at the beginning of 2025. So, the first thing I’m going to ask you, Thomas and Pierre-Yves, is can you remind you quite briefly what your function is and what solutions you host within Framasoft

Thomas : So, since 2014 we have had a campaign called ” Degoogle the Internet “, in which we offered up to 40 services and today, we are much smaller, more like twenty. In fact, the principle of this campaign was to show the grassroots that there were solutions to the services offered by the GAFAMs. So, we really published some in the form of alternatives to this or that service. Whereas today, we see things a little more. We had proposed this in the form of this formula, which is: equipping the contribution society. And so, above all, we offer collaborative tools that can be practical, that allow us to exchange, to organize together, to collaborate on many different tasks. And so, that’s all the way around, so the entire list is on the degooglisons-internet.org website. And so, the most well-known, typically, are probably, for most people, are probably Framadate and Framaforms. And we continue to get out of this context, to say to ourselves: we want to equip this society with contribution in an alternative way to what is proposed. And typically, we launched Framaspace a bit in that context. To say that there is a need for collaborative spaces for collectives that work for causes that we find rather interesting. And so, we will make sure that they have the tools to carry out their actions.

The services offered by Framasoft

Walid : So, what we were talking about in the second episode about the history of Framasoft is the fact that, by default, what you’re going to offer people are not software that you’ve written, but software that you’re going to take and integrate into your hosting space.

Pierre-Yves : Most of the time, yes. It turns out that, despite everything, it can happen to us. Or to develop the solutions or at least to participate in their development. Typically, here, I pay tribute to both Thomas and Luc (Editor’s note: Didry), who is our Adminsys. If I take for example Etherpad, which is the software that powers Framapad , which is an online collaborative writing service, we have made improvements to Etherpad to allow for better performance. There have been various improvements. And this is also the case on software like Framalistes, so Sympa which powers Framalistes which is a discussion list tool behind it. NextCloud, where Thomas is one of the world’s first contributors, volunteer… finally he is paid by Framasoft but he is not paid by NextCloud, and where Thomas’ contributions in terms of Dev are very substantial. So, it’s true that it’s not necessarily us who are trying to develop these solutions. And so, that’s something we had talked about quickly in the previous podcasts. But as a result, we are often taken as an association that develops software. However, our specificity is rather to make them available. And so, I would say that the added value that Framasoft has may be in the selection of software that we will talk about next, but it is also in the fact of knowing how to set them up, maintain them and make them available to a very large audience, since today we reach roughly 2 million visitors per month.

What triggers Framasoft’s choice of new software?

Walid : For the people who listened to the Open Source Experience round table, it’s more or less the same profile as what Raphaël explained about his own work as an industrialist. If we start talking about evaluation, the first thing is that before we evaluate, we must already have a need. So, that’s also something that we talked about during the first few episodes, is how you evaluate, how you see that you need a new solution to start with. And then, how do you see if it exists or if, in the case of Peertube or Degoogle, it doesn’t.

Thomas : Again, compared to what I was saying, it was different. At the beginning of Degoogle we really had an idea to make an alternative to every big service that existed. Today, we are no longer sure that we have a set of services that is consistent with the needs that current users of our services and the people with whom we exchange. And so, typically today, we no longer deploy services simply because we have the technical possibility to do so. This is what typically led us to close a lot of services in 2021. Because it was becoming complicated to maintain all these services and it didn’t necessarily make sense to have them all. And so, there is really more of a search for meaning, cohesion in the Framasoft offer. And also obstacles, in particular on tools for example, which allow online publication spaces. So tools, for example, that will allow you to put images online or to create publication spaces such as social networks. That, today, is something we are very careful about because it has consequences. Which are not only about technically managing the service and making sure that users don’t have a problem using it, but also about ensuring that there is moderation that is done, respecting existing laws in terms of moderation, terrorism, penalties, pornography, and so on.

Pierre-Yves : So, if I go back a little bit to what Thomas said, with whom I obviously agree, it’s that we’ve really gone from a phase where: “there’s this software that exists, it may be interesting to install it, let’s install it, then we’ll see if there’s demand” to “we’re looking a little bit at what users tell us as needed”. I don’t know, there, for example, there is one that is emerging quite strongly, which is alternatives to Notion. Notion, which is a software that allows you to do a lot of different and varied things. And so that’s where we’re going to intervene by saying “Ok, so, what are the alternatives to Notion? What are they worth? Where are they? Is it worth making this type of software available or not? ». So it has indeed changed over time, between: “we did it because we could do it” to “we do it, but knowing that if we make new software available – here I am thinking, for example, of a petition software that we could release in the coming weeks potentially – millions of people are using it behind it. And we must not mess up too much on the software we are going to make available.

Raphaël : It’s interesting because it means that with the evolution you’ve had since the Degoogle phase with a maximum of services to show that it exists and then to test somewhere, now there’s a whole that is more coherent and that is more to build this society of contribution. Does this mean that your way of choosing, apart from the need, which is very clear there, but of choosing a software by saying: “we look at what exists, where they are, can we make them run with millions of users?” has it also changed your way, precisely, of selecting or looking at what exists?

Pierre-Yves : I have the impression that it is. I don’t know if Thomas would have anything else to say about it, but in any case, for me, what is certain is that the question, for example of the scalability of the software, was not something that concerned us much before. Today we pay much more attention to it. And the second aspect that Thomas already mentioned earlier is that of moderation. Because it’s all well and good to make collaborative tools available, but the question of how we are going to be able to provide support and moderation on these softwares, it becomes critical for us.

Walid : The choice with scalability can also involve having a step back from the fact that the software is scalable. And so, isn’t it a choice that favors slightly mature solutions?

Thomas : Yes, then there are obviously differences between the type of software. Obviously, there will already be sprocket software a little on the street. As a result, there are spaces that are accessible without an account, for example. So, it can very quickly rise in terms of use without us realizing it. There will also be the fact that if you ever need to make quick changes to a software, you need to know how you can tinker with it in order to be able to identify. Where is there a performance problem, for example? So indeed, it is also about checking our skills internally to be able to start to appropriate these particularities a little. These are things that, indeed, we didn’t ask ourselves too much of the question about until now, with also the increase in audience we had from the Degoogle campaign. The question of the sustainability of the project has always been raised, but indeed we can’t necessarily announce something and start really telling everyone: we’ve found a solution, we’re offering to install it, and indeed say that it’s going to be abandoned in a few months because of technical problems. So what we have typically done this year 2024, about certain services, is to launch them in a space called Framalab and say to ourselves: well, there you go, we have typically identified software and they could be the next services in the Framasoft offer. And so, we suggest you test them to see if, already in terms of user experience and functionality, it can be something that fits. And us, see if it’s something that holds up with this test sample? And can we start to put it a little more widely into production?

The role of languages in evaluation

Walid : You talked earlier about getting into the code, about tinkering a little. Do the languages in which the solutions are written also play a role when you evaluate the tools?

Thomas : Yes, definitely. We have in-house capabilities on a certain number of languages that are there in relation to the stuff we develop ourselves. That doesn’t prevent us from having software bricks on which we don’t have any particular knowledge, but we manage to find our way around from time to time. What will be a little more frustrating, for example, may be the fact that on compiled languages it is more difficult to tinker because you have to compile before replacing. So it takes time. It’s not easy to just make a correction and check if it fixes the problem. Just this afternoon, we had a performance problem on a serve. And the fact that you can quickly identify and test to see if it fixes the problem is really something that is decisive on the fact that you tell yourself that you don’t need to stay with these performance problems for days and days, because it can… We are very agile on this kind of thing. Apart from the question of tinkering, the choice of language will also be in the question of can there be a community that gets around the project. Obviously when you’re using a slightly more exotic language, it’s more difficult for a community to create itself, or even to recreate itself if there is ever a maintenance problem at some point. So, indeed, this can be one of the criteria that can be judged in the question of sustainability.

List of evaluation criteria

Raphaël : So that means that when you bring something into the incubator today, the Framalab, or when it leaves Framalab, for example, to become a service, does today, in these criteria, some of which you mentioned, the technical base, the ability you have to appropriate it, the existence of a community, is it scalableetc. Is this list predefined and you look at it systematically or how does it work?

Walid : Is there a list already?

Pierre-Yves : Very good question. There is no permanent method, there is not something constant. It will also depend on where we are, our temporalities, our financial means, our human resources, etc. So there is a part, I’m not going to say randomness because it’s not randomness, but contextualization that will evolve over time. And so to try to answer your question a little bit, inevitably there are things that are constant. Typically, does the software work, does it have a community, is this community active, is the language accessible to us as Thomas said, will we be able to get our hands dirty or not, etc. So, there are a certain number of criteria that are permanent, but which are not necessarily listed because I really believe… so maybe Raphaël won’t agree with me… But I really believe that there is a feeling thing sometimes, which is quite difficult to explain about how you feel about this software. And sometimes we took software that was not necessarily the most ” bankable ” (Editor’s note: likely to bring in money) but because we had the impression that in terms of accessibility of the community or other there was a capacity for projection that was better with software that perhaps had a community, for example, less solid. However, we are ready to make this bet because we feel it better. Or because quite simply, we sent an email to three Dev teams from three different software programs and one responded to us with enthusiasm and two others, not at all. And it’s a criterion, enthusiasm, which is not very factual and objective, but in any case it is a criterion. But it’s difficult to judge the objectivity of this criterion. And yet, the fact that we have someone who responds quickly and says “Oh but that’s great, I’m thinking for example of the case of LiberaForms , which is a Catalan community that develops software, so it’s called LiberaForms, which we are considering to be the successor to Framaforms. This software is not the most beautiful, it is not the most complete that we have been able to find. On the other hand, they are on the go, so it makes you want to work with them. It may be very simple, but for me it’s a real criterion that means that in the context we are in, where Framaforms is several tens of thousands of forms each month, it is in our interest to find a community that wants to get its hands dirty and not just someone who will say to themselves. “Oh dear, Framasoft is going to use our software and won’t pay us anything in terms of time, in terms of code, in terms of communication behind it.” And so what we’re going to bring is a little, precisely, all that. And so sometimes choosing a software solution over the informal exchange that we will be able to create with another community is something important for us. And it’s not a hyper objective criterion, it’s not the number of commits.

Raphaël : Okay, by the way, the QSOS method we’re not here to talk about that, but we didn’t want to do something that was too precise, because it becomes, we don’t know what we’re talking about anymore. Now, the thing you’re talking about is the human factor. Then in this human factor, in terms of community, it can be responsiveness, the fact that the community is open, or collaborative, We can formulate them like that and it becomes criteria, even if they are more human, but which remain criteria. It’s interesting this aspect, because precisely, and here you were talking about Framaforms, it leads me to two questions. The first was, what makes you want to keep the same service at some point, but replace the solution with another? Is it based on some of these criteria or on anything? That kind of thing. And when you don’t adopt the most bankable but you say: we’re going to be able to work with these people and we feel that it’s going to go well, there’s a feeling indeed, even if objectively, we can say: that’s it, it’s dynamic, they respond, they’re reactive, they’re open, etc. At the same time, since you know that it’s not the most bankable, have you identified points where you say: ah, yes, maybe it’s less of something. In this process, do you also make the choice, sometimes to say it’s less of a thing, but precisely, we will be able to help them with that? And therefore, increase sustainability.

Pierre-Yves : I’m willing to take the answer to the first question and then maybe leave the second to Thomas if he’s OK. So your first question was: why would we change the software and what are the criteria that make that… Not you choose a software, but you choose to abandon it, somewhere. Here again, each answer is very contextual because we have “abandoned” a package of services and each with different reasons. There are many that we abandoned because moderation was no longer possible. That’s what Thomas was talking about earlier, when you start to have requests from the judicial police for terrorism on a software for sharing links, creating shortcuts, links to shortcuts… There is a certain moment when you want to do something else with your days. For Framaforms, the answer is completely different. Framaform is a solution that I developed very quickly, telling myself that there is no form software that corresponds to what I want to do, but there will be one that will be born in the months or years to come. So I’m going to make mine in the meantime, then we’ll see how it works. And then, it worked very well, except that it was based on an aging software stack, which was Drupal 7, which reached its end of life 15 days ago. And so, we’ve been saying to ourselves for several years, in fact we should improve or update the Drupal 7-based software that allows us to do Framforms. And we’ve done a lot of small upgrades over the last few years, but it actually required a lot of dev work. And I come back to what I was saying earlier, which is that Framasoft is not basically an association of developers. There are developers, and typically Thomas is also a developer in addition to being co-director, but it’s not the purpose of the association to develop. And so we found ourselves in a bit of a dead end to have a software that has a very, very great success, really, several hundred thousand monthly users, several tens of thousands of forms per month, but not the ability to maintain it. That’s why at some point we move towards another solution, saying to ourselves: but in fact we’re not going to be able to maintain this software over time. It’s better to change before hitting a technical wall, which is that Drupal 7 is no longer maintained. If tomorrow there is a huge security flaw, we won’t start maintaining Drupal 7, even if there are projects, forks that exist. But as a result, we found ourselves abandoning software because we were entering technical deadlocks. And that’s something that is sometimes quite complex to get across. I don’t know if you’ve experienced that. But to make users understand that the software’s time is a very fast time. You quickly end up with libraries that are no longer up to date, that no longer work, whose security is no longer guaranteed, etc. And so, if I take the case of Framaforms, the software is almost now about ten years old. For me, ten years of existence for a software is quite honorable.

How does Framasoft monitor and contact projects?

Walid : I had two questions about that. The first is how do you do your monitoring? That is to say: there is a need, is the day before something continuous? Is the day before punctual? We have a need that has been raised, we see that there is a real need. We need a service, we’ll look at it. That’s my first question. The second question is earlier you were talking about contacting communities and sending an email, etc. I wanted to know if you have a bit of a process when you are in this phase of evaluating who I contact. Am I… I see a message on a forum, do I try to contact the contact email? Do I contact, I don’t know, the founders? Anyway, how do you organize yourself on this?

Thomas : Indeed, there are two different cases. This is the case when we are actively looking for software in relation to a specific need. And so here we are going to really look at public software forges, other organizations that may have the same needs and therefore may have done the same projects. In fact, it’s rather rare that we discover other things at that time because we still have a collective watch, in fact, passive of several members of the association, several employees on whom we really see free software projects, things that are accomplished, things that we hadn’t heard of. And in fact, we drink a little from permanent staff. Even if we don’t necessarily have a particular need to fill, we note it for later because we know that potentially it can do something. And we come back to see a few months later to find out what the situation is, to see if it has taken off, is there still only one person who contributes or not. And then, the monitoring methods are quite classic for everyone: there are RSS feeds, there are social networks, blogs that will talk about this or that free project. But apart from that, nothing exceptional.

Pierre-Yves : To complete Thomas’ answer, you have to be aware that you work for Framasoft, which has been identified for 20 years in the field as one of the resources, one of the gateways to French-language free software. So when there are new software releases, things that come out a little, sometimes we will pull the information, sometimes it is pushed directly to us with people who come to us and say: “Oh well, do you know this solution?” etc. So we have both a certain experience and a certain notoriety which means that the news also comes to us from time to time. And to answer your second question, Walid, on contact processes, do we have processes to contact them? A bit the same as on the selection aspect, no. And I want to say almost so much the better. So, Framasoft in terms of offering alternative software and services, especially to those of the GAFAMs, is at the French-speaking, or even European, and even global level. However, we do not have very industrialized processes on this side. Most of the time, I know that when we choose a software, here I was talking about LiberaForms for example but I think of other software that we might have wanted to use, I find that it’s just correction and politeness to say: “hello, we’re Framasoft, that’s what we do, we find your software interesting, Maybe we would like to propose it, is it a problem for you or not? And in fact that’s one of the interests of free software, in my opinion, it’s precisely the human dimension that can be behind it. The fact that we are known, including, as I said, almost on a global level, at least in the free software community, as being a relatively large resource center in terms of traffic, etc., means that most of the time people are rather enthusiastic, or at least, they see things in a rather positive way. When you arrive and say: “oh well, we could offer your software” they see it as a showcase, and so I find that it doesn’t cost much to send an email, to introduce yourself, always for me with humility, because it’s not because you’re going to set up a software that its number of downloads or its number of users will explode behind it. We messed up a lot of times on software and services that we have set up. That’s it, it just seems important to me to be polite when contacting the people who make the software. It can be by email, by social network… There is no preferred channel. I prefer email because I’m old. But if we had to contact people by other means, that would be OK too.

The impact of the software business model on evaluation

Walid : There is one criterion that we haven’t talked about yet and that, in my opinion, is one of the most interesting, if not more interesting, and that’s the business model part. When you look at the business model of software, the way it is developed, the potentially commercial part or not, etc., what are the models that you are going to reject directly? What are the models where it can go? And which models do you prefer?

Thomas : Obviously, the models on which we could reject directly are really models on which the open-core side is lagging to the limit, on which we will almost not be able to use as such. And where really the prospects of the open-source part is really to remain a kind of demo of a project that would really just want to free up code for a little more the image than for the common side. But it’s quite rare that we come across this, at least that the software is known and relevant. It’s more common for us to have software on which there is really an economic activity that exists. Within the Degoogle Internet projects, we can think of Framagit or Framateam or Framaspace which are respectively based on GitLab, Mattermost and Nextcloud. And so they have, at least the first two, first-party versions with more features and community versions that are free with most of the features that correspond to us. So clearly it bothers us from the point of view of total openness, well no, which is not total, precisely. But on the other hand, it has a certain advantage to have relatively large companies behind these products, which is that we can expect a technical mastery that is a little higher than most other projects. So as much as we know that with these models, we are always under the risk that the features will not be added in the community version, or even removed as the versions progress. But we can say that, as they make a product for a certain number of customers in a professional way, we are not likely to find any major problems in technical terms, in particular, functional at our level. It has advantages and disadvantages. Typically, when the economic model of one of the projects can be put on hold a little because it’s complicated, digital this year has been a bit complicated, so there are some projects on which the screws are tightening a little. Of course, it weakens our sustainability to say to ourselves: hey, we have such and such a service that is based on such and such software and they can potentially either stop overnight, which can be very complicated because there is no community in addition to the salaried activity on the project, or in any case start to say to themselves: well now we’re going to need to make features that are exclusive to the first and paid version to be able to make some money… But in any case, on the services we have chosen so far, the question has not been too problematic. And so we do well with the fact that there are very varied business models .

Walid : I would link to a conference, which I stumbled upon a bit by chance during the holidays, by the founder of NextCloud, which he gave at FOSDEM 2018, in which he explains his model, why he made ownCloud, why he forked the NextCloud sheet. I stayed really on my ass. It shows that it is possible to do a business that works with a model that is not open-core. So you can quite easily, when you look at a project that is open-core for example, to see if it’s worth investigating or if in fact, basically it’s just a bit of fake open-source, but it’s not worth going any further.

Pierre-Yves : The advantage is that here again we have a little experience on the subject. After 20 years in the free software world, you end up feeling how sincere the sincerity of the company that makes the open-core software is in the fact of saying: in fact, this is a business model, period, or is it just a springboard to finally sell a solution that will no longer be open source in a few months or years? That doesn’t mean you can’t mess up. For me, one of the pretty glaring examples we’ve seen in recent years was rocket.chat and Mattermost. rocket.chat and Mattermost, which are alternatives to Slack, the two software were, let’s say in 2017-2016, substantially equivalent, including on their model. And then little by little it’s rocket.chat that is the first to close a certain number of things with quite strong regressions. They have backtracked on a number of community features that are being switched back to the paid model. People who used rocket.chat found themselves in very complicated situations, almost not being able to use the software without having to pay for it. And so I’d be lying if I said we saw it coming. So sometimes you make choices that are guided by almost chance. We chose Mattermost because it was delivered in omnibus packages with GitLab. And so when we set up Framagit, it was Luc, our Adminsys, who said: “Hey, wouldn’t we deploy Mattermost at the same time?” And we were like, “Ok, let’s go!” And today we have one of the largest Mattermost bodies on the planet. We have, I don’t remember, several hundred thousand accounts and several tens or hundreds of millions of hosted messages. And it works very well. Now again, I will be lying by saying that necessarily it is an analysis grid that allowed us to make this choice which turned out to be the right one, at least until now since Mattermost, which has been there for a year now, is strengthening its open-core model and strengthening the paid part. In fact, the new features now are mainly paid for the paid part. It also raises questions that I find interesting from a software selection point of view: it’s when do you say to yourself, this software is stable and suits me and that you’re not in the pursuit, finally, of going to the latest update to take advantage of the latest features, etc. Because as far as Mattermost and Framateam are concerned, I have the impression that we have reached a level of maturity that is sufficient. The new features essentially revolve around artificial intelligence, which is not necessarily our cup of tea. And so I, from a rather strategic point of view, I would tend to say: it’s good, we’re going to freeze the features, If Matermost version 9 suits us, we’ll stick to version 9. While necessarily from a technical point of view and especially from a security point of view, you tend to think that you have to follow the new features offered by the publisher. And it’s logical, I understand that an Adminsys doesn’t want to end up with old versions and an aging stack . But on the other hand, for me, if it does the service that is expected, finally, for me today there would be zero new features added to Framateam for the next three years. Personally, it doesn’t bother me. And so it comes into the question of selection a little bit. That’s what is the objective you set yourself to achieve with this software, because if you let the publishers do it, they will inevitably tend to want their software to make coffee, while you already have a coffee maker.

Contribute to a community project vs. backed by a company

Raphaël : There is this notion of risk analysis. What risk do you want to take or not? And so it comes back to your need. Precisely, you were talking about adding new features, for example AI because it’s trendy. And earlier you were talking about sometimes substantial contributions to projects, whether it’s Etherpad, Sympa or NextCloud. And in a contribution framework like that, when you contribute to completely community projects, such as Sympa, compared to contributing to projects where there is a company, how does it work? Do you have any interesting feedback or even a vision, a way of doing things?

Thomas : The simple answer would be that indeed community projects are perhaps easier to integrate the changes proposed by Framasoft compared to projects undertaken. Most companies will have a business model that is rather B2B and therefore their interest will be particularly vis-à-vis companies. And so necessarily what we offer for our users is not necessarily the same things. So sometimes there can be indeed, not necessarily conflicts, but divergent interests and equally different priorities within the software. But in practice it is true that we can also, depending on the governance of certain projects, the way things are going, or even the abandonment of the maintenance of certain projects, the fact that our contributions are not easily integrated. Typically, you cited the case of Sympa. It was maintained by one person for quite a long time and it was a bit complicated to manage it, especially with a lot of universities in France that also use the software. We’ve never had any big unpleasant surprises and complete blockages, but that doesn’t prevent the fact that, clearly, it’s also a criterion to say to ourselves: apart from bug fixes, performance, will the project also align with the objectives we want for the service

Walid : I have a question about that. If you take open-core products, you have something that is always in the proprietary version: it’s OAuth authentication for example. You see, that’s like the thing: your project is great, but the Auth part is never in the community version. And so, if you come in and you need it and you develop something on the side, you can’t give it back to the project. Anyway, he’ll tell you: sorry, but it’s in the version… Finally, you always have problems where you want to develop a feature and in fact the project, either it has a paid version, or it tells you: I’m not interested in it because I want to make it payable. And so in fact, indeed, don’t you find yourself at times, precisely, with this kind of contrib where you want to do contribs, but in the end, it doesn’t fit into their roadmap at all. That’s a little bit what you were saying earlier.

Thomas : As Pierre-Yves has already said, in fact, our margin for development is quite limited. So it’s rare that we have this kind of problem where we really have to contribute to entire functionalities. It’s more like small fixes, small improvements at the margin, bug fixes. Depending on the feedback we have in relation to our large panel of users that the projects do not have. And so typically, if a case like this arises, what we’re going to do is rather: we’re going to develop the thing in our corner, make sure that it’s done in a pluggable enough way that we don’t need to keep the fork on our side. And have that until, potentially, it’s somehow integrated or there’s a fork.

Walid : So you have services where you have your own patches of stuff that aren’t integrated?

Thomas : Absolutely. It’s quite rare, but we have patches, especially on Framaspace for example because there are things that NextCloud said about: we’re not particularly interested in that. So we put them in the form of applications. Like that it’s separate, it doesn’t stop us. But there are also hard patches, we have specific versions of certain software. I don’t necessarily have the list in mind but in any case, clearly, everything is on our Framagit forge.

How do you define that your project is mature?

Raphaël : I have one last question on my side. We talked about the subject a little bit when PYG you were talking about Framaforms, the first version based on Drupal, etc. Basically, when you develop your own solutions, I’m thinking in particular of PeerTube for example, randomly, how do you monitor that? You say: hey, the project is mature, we have to pay attention to this aspect, that kind of thing? What are your thoughts on this?

Pierre-Yves : On PeerTube, your question is interesting because we ask ourselves the question of whether we have not already reached a level of maturity that would be sufficient. You can always add things but again, your software in the end ends up making coffee. So, typically for PeerTube it was the question of: OK, wouldn’t we broadcast things in audio-only. But suddenly your software becomes a little bit, I’m not going to say podcast software, but suddenly it’s not the same audience as saying to yourself: we’re broadcasting video. Ah, well yes, now we want to broadcast sound. Oh yes, but then, don’t we also need a studio to edit the videos and to edit the sound inside? And how far do you go like that? The second example we have of this, and Thomas could talk about it of course, is Mobilizon. Mobilizon which was born out of the need, at a given moment, to say that there is no alternative to Facebook pages and events. We could develop this in a decentralized way based on ActivityPub, etc. And so it’s a software that Thomas has been developing for several years. And after 4 or 5 years, we came to say that the software is doing what we had planned it to do. We could continue for 10 years to improve it, add features, fix this or that bug, etc. But at some point, it’s endless. It raises a question, in fact your question raises a much more meta problem, which is: what is software and where does it stop? I don’t think we have an answer to this question, which is a bit complex. But on the other hand, what part of our audience can reproach us for, and this is completely understandable for me, is: oh yes, but we would like there to be permanent improvements in Framadate, in Framaforms, in Mobilizon, in PeerTube, etc. And suddenly you find yourself a full-time publisher of X software. And I think that it is for us a specificity, a singularity that Framasoft can have to say to itself, in fact, the software does what we need it to do. Exactly what I was saying earlier about Mattermost. I need something that allows you to do team chat in a simple way, with good ergonomics, on which you don’t waste time. And there’s not too much energy wasted working together as a team. In fact, when the software does the right thing, I think it’s a singularity of Framasoft to be able to say: ok now it does the job and we’re able to say ok, we’ll stop there. And so, maybe for PeerTube it’s something that’s going to come. PeerTube has been developed now for more than 7 years, and so there is still a… At some point we’re going to say that the software may be worth having forks, maybe, because more towards audio than video, maybe more towards a studio version on which we could integrate, to do video editing very simply within PeerTube. There is the question of the remuneration of videographers in PeerTube, etc. But in fact, we as software publishers today don’t want software to do anything and everything. And so you have to be able, at some point, to say to yourself: well, OK, we’ve reached the level we wanted to reach with this software and we’re finally able to move on. This allows us to free up energy for other software in fact.

Thinking about new services

Walid : I refer you to the episode on PeerTube with Pouhiou and Booteille, which explains the history of PeerTube quite well and that I particularly like since I use PeerTube a lot too. So there you have it, and just a little note: we’ve talked a lot about Drupal, the same, I’ll link in the transcripts to a great conference from the Capitol du Libre on the history of Drupal. And it’s quite fantastic, especially when compared to the governance of WordPress, it’s really crazy, I was glued to this conference. I had one last question before we parted. In these selection issues, do you have any new services that you are thinking about? There we talked about Notion. I’ve been thinking about doing an episode for months on how to start from Notion and move towards free software. Do you have any other services on which you can help? well without…

Pierre-Yves : We can even reveal things, because we’ve always been very transparent about what we would like to do in software. We are considering… once again it will depend on a lot of things… I was talking earlier about petition software, because there aren’t that many free petition services that don’t use the petitioners’ email address databases. There is the question of software around, in particular, the management of online PDFs, which we have identified in Framalab and which seems to be setting up rather well. There are in the software that we have identified, on which there are shortcomings. Basically, you can also look at what people around you are using and say to yourself: what are the free alternatives that exist? And so, for me, there were two software programs that were quite in demand that were alternatives to Notion and Canva. Canva is, let’s say, halfway between PowerPoint, drawing software, etc. And that allows people to communicate. So today, there is no alternative to Canva Libre. There are things that come close to it from different angles, but there is not that yet. For the moment, we are trying to stay very attentive to what alternatives to Notion and Canva we could, as Framasoft, offer tomorrow. On Notion, we are starting to identify 3-4 free software today. We will try to apply our own analysis grids to try to evaluate and select a software. As for the alternatives to Canva, we have chosen for the moment to highlight a software called Aktivisda , which is a software that was developed in particular for environmental activist communities and which is much less powerful than Canva, which is not an alternative to Canva, but which allows us to meet a need which is: I am an environmental activist association or activist What You Want, Human Rights, etc., and to be able to quickly decline visuals. I have a basic visual and very quickly, I’ll be able to edit this visual online with zero graphic skills at the beginning and make my own visuals. Because we’re going to do 10 events that are close together during the year. We’re going to do one event per month, for example, and all we need to change is the date. There is no need to use large gas plants on this. This Aktivisda software, which is developed by a cooperative, it seems interesting to us to say: hey, we could make this software available. So here are two or three examples of software that we could release in 2025.

Walid : You didn’t mention no-code?

Pierre-Yves : Ah, no-code. Indeed we have identified today 3-4 free alternatives to Airtable… I lost the name of the software… Airtable, the proprietary software. So there’s Baserow, there’s Teable, Nocodb I think… yes that’s right, and Grist which are basically the four solutions we have identified, which allow you to do no-code. The difficulty we have behind for us, as Framasoft, is that if we make it available, we have to succeed in doing it. This is called multi-tenant, i.e. multi-organization, or even multi-user. If you look at the memory footprint of a Baserow, which is part of the criteria if we come back to the subject of the episode, we have to do it, if tomorrow we have an instance of Baserow to run, it’s quite simple. If tomorrow, we have to run 3,000 or 4,000 of them, the machine power and therefore the cost for Framasoft’s infrastructure… The cost of the Framasoft infrastructure today is roughly the price of an employee. It’s roughly between 30 and 40,000 euros per year to pay for all the servers. And so if tomorrow we were to make baserow available to everyone, perhaps the cost of the infrastructure would increase from 40,000 euros to 60,000 euros. And so the question is where are we going to find the 20,000 euros that come behind, since Framasoft lives only on donations. So that’s why there’s that in the evaluation phase. Typically, I have the intuition that for example Grist is less resource-consuming than Baserow. But it must be objectified and therefore it must be tested. You have to test it with 1, 10, 100, 1000 users and see what it consumes in CPU, RAM, etc. So, sorry, it probably won’t be in 2025.

Walid : No, but for me, it’s more about having a real free alternative to automation tools, that’s what it’s all about. Where they all are, with models: either open-core, where you see that it’s going to close, or with source-available licenses, that kind of stuff. And that’s really my big thing. OK! We’re coming to the end of the show, I thought we were going to… Well, we’re still on a one-hour format, it’s crazy. Every time, even if we want to do less, every time…

Pierre-Yves : We shouldn’t invite Frama as a result, we’re not the least talkative…

Final Words

Walid : I think it’s because I’m used to this format. So we’re coming to the end, do you have a message to convey on these evaluation subjects? Do you have a little message to convey, each of you, on this?

Thomas : What’s quite interesting is that in open source we say that we’re on the shoulders of giants, but in fact it’s also a bit of a house of cards, because suddenly all the projects on which we base ourselves can sometimes just die overnight for X reasons. And so, in fact, what’s really important is to put the commons side back in the heart a bit, to say to ourselves: if there are several big players interested in a software, whether it’s framasofts, kittens, completely different organizations that have an interest, the state, that’s it. It allows for communities, common interests that allow – whatever the contribution operating model, whether commercial or not – there are potentially, when an accident happens, resumptions and the fact of saying: well, we had chosen the right solution at that time. So it’s true that the easiest way to choose the right software is also to choose the one that is popular, at least in open source and especially to find the one that, potentially, other players may be interested in having. Typically we know that on Framaforms it’s a service that is hyper-used, it’s a software that has almost never been installed. And that’s also what determines the fact that we have to replace this software as well. So there you go, if you are ever a decision-maker somewhere, don’t hesitate to go and install Framasoft services and tell yourself that it’s also good to participate in one way or another.

Walid : Pierre-Yves, do you want to say a word?

Pierre-Yves : yes, on this dimension of the selection and evaluation of software, I really come back to what we said a little bit throughout the episode, which is to say that the intuition part counts. And so, I’m really in favor of having criteria, indeed, on governance, on the daily activity of the software, the number of commits, etc. But sometimes there’s something that comes from the gut, knowing that for us, the stakes are relatively limited. We’re Framasoft, we’re not… the government, the French state, etc. So when you make a choice of software, it won’t necessarily change the lives of billions of people. And so making a choice, sometimes with your gut, is not stupid. The other point was to also say that there is also perhaps a need to deconstruct a little… Finally, there are still many software programs that are with a volunteer dictator for life. And inevitably this model has been criticized a lot, a lot, which I completely understand. But on the other hand, publisher models, I mean companies, are something that is quite understandable, but which, for me, makes me less comfortable with the common part that Thomas was talking about. That is to say, in the end, open source, in this case, is seen as a method, let’s say, of industrial production that works. There is hindsight to say: open source works from an industrial point of view, as an industrial method of software production. But you can quickly lose the common aspect and the slightly … I’m not going to say voluntary, but in any case amateur, that on a personal level – but I don’t think I’m betraying by saying that this is the case at Framasoft – this amateur side is claimed because this amateurism poses problems but also brings solutions, precisely, to a software world that is more and more complex to tackle. For someone who, where a few years ago, was going to tinker with his little software in his corner, was still very, very simple. So, for me, a little bit in this conclusion there is how do we ensure that free software keeps anti-capitalist spaces, where software is not seen only from a financial point of view, but from the angle of the digital commons? And how can we participate in the end, all together? Even if, indeed, it’s not the most beautiful software, even if it’s not the most efficient, etc., the fact of doing together still seems essential to me, at least as a dimension to be preserved, if we don’t want open source to be just absorbed and digested by Microsoft, via Github or other software publishers behind it.

Walid : Raphaël, do you want to make a final word?

Raphaël : Fast. What I appreciate, and what I also align myself with, is the human dimension of the projects. What makes people who get involved, who contribute to projects in the evenings, on weekends and everything, is because there is that, there is this notion of commitment. And that’s what makes communities strong. So yes, to feel things with your guts, to say: such and such a dictator I leave him the title of dictator because I trust him, in fact, somewhere. So he’s a dictator, but hey, we trust him. These kinds of things, these are human aspects that are fundamental and that’s what makes one of the foundations of what we love. Well, at least me personally, in open source and in communities. That’s what makes us happy to discuss and exchange on these aspects. Something that we can’t do when we are in models that are more closed or that are focused on goals that are a little different. There is still this notion and so yes these are things that we can’t necessarily put into equation. It’s the human dimension.

Walid : Finally, I also remember the human part, the part that spoke to me a lot, it was the first contacts with the projects. It reminds me of my own experience, I talk about it in the first episode on GLPI. When you arrive on a project and people answer you and help you right away, it makes you want to continue. And this is not the case for all projects. And so it’s very important, indeed, to know how to seize opportunities. And then also to arrive in a very humble way and say: this is what I would like to do, what do you think? Well, listen, thank you to all three of you for taking the time to discuss these evaluation topics that I must admit that we don’t necessarily talk about much, even though, it’s still a very important subject. Listen, thank you very much. See you again. I think we’ve made our quota of Framasoft for 2025.

Pierre-Yves : What? But it’s only January. Walid, what a cruel disappointment. Well thank you for the invitation.

Thomas : Thank you very much for the organization.

Walid : Thank you, yes, thank you soon. And as usual, share this episode with those around you. If you too have evaluation problems and you think that you have interesting methods or things, do not hesitate to contact me or Raphaël. There you go, we’re very interested because we really like this subject and see you all soon.

Raphaël : Thank you.

This episode was recorded on January 14, 2025

Transcription by Raphaël Semeteys

This article has been automatically translated from the original language into English.

License

This podcast is released under the CC BY-SA 4.0 license or later.