Debian Lessons
Project management lessons from the Debian project.
5 September 2004
This text lists the important things I think the Debian development community has learned about project management in the course of developing the Debian GNU/Linux system. These are my personal opinions, not those of the project or other developers. I hope this list will be useful to Debian, other Linux distributions, and other free software projects.
This is the second version of this text. The first one was written in the year 2000.
Thanks to the people on Debian's IRC channel for feedback since the first private drafts. Thanks to Mark Brown for pointing out IRC versus mailing list for communication between developers.
Make sure things scale up. Problems are always simpler when the project is small. When the number of developers and packages grows a lot, small problems become big problems. Make things distributed so that things can be worked on in parallel.
Make sure the foundation is good. Debian has a pretty good package management system (even though it has lots of minor flaws and warts). This makes it much easier to build the system, when one can concentrate on making packages better, instead of having to fight the base system.
Document important things. When people need to work together, they need to know how to take each other into consideration. Rules are needed for how packages are laid out so that they don't interfere with each other. If these rules aren't written down, they will have to be remembered by the community memory and word of mouth, both of which are unreliable. The Debian Policy documents are a fine example of this. Debian couldn't live without it. (Note that when documenting things it's important to document the reasons behind the decisions as well. Debian documents don't always do this well.)
Automate repetitive tasks when possible. Repetitive tasks should be automated, especially if they are tricky or critical. This reduces the number of human errors, and also frees up developers for more important tasks.
Debian examples include dinstall (moves uploaded packages into the ftp server), lintian (checks for many errors that are easy to make), and the automatic compilation for porting efforts. They're all absolutely necessary for Debian to maintain high levels of quality.
Avoid single points of failure, especially for volunteers. Volunteers will always lack time at the critical point. For critical jobs, have several people share the workload and make sure there are mechanisms for checking that things are going smoothly.
Do not worry about time tables; keep goals realistic. In big volunteer projects, time tables will always slip. That is not a catastrophe, and it must be tolerated, but there should still be some effort in keeping them. Goals must be kept realistic.
Make it easy to work independently. The more interdependencies things have, the more communication is needed (read: less time spent on productive hacking) and the bigger the chance that a single developer or package will cause big problems. For example, if many packages must manipulate the configuration of a server (say, Apache), it is better if they can just drop a file in a directory of configuration files, rather than having to attempt to automatically edit a single configuration file.
Some things must be decided or handled centrally. Security issues are one such example. These must not be de-centralized or trouble will happen.
Do not overload developers. Developers are volunteers with too little free time. Make as few requirements on their time as possible. For example, do not require that they follow high-volume mailing lists. Overloaded developers have a tendency to disappear, or at least their packages will be of lower quality or less up-to-date than they might be. Also do not let developers overload themselves by trying to do too much, and make it easy (culturally and technically) to give up packages, when the workload gets too high.
Be open and keep things public. If you're popular, your users will want to know what is going on. Do not decide things behind closed doors. Allow everyone to read the discussions leading to decisions and, preferably, allow everyone to participate in them, even if they aren't developers. Developers may need a private forum as well, but it should be used as little as possible. By doing things in private, you are hiding your motives and other people will be assuming the worst and react accordingly.
This applies within the project and between developers or groups of developers as well. When things are decided that affect more developers than those involved in a particular discussion, the discussion must be held in a forum that all developers can participate it. For example, IRC is not good enough, since not everyone will be present at any given time. It's OK to start a discussion on IRC, but before any decisions are made, it should be taken to, say, a mailing list.
Make it easy to contribute. There are never too many developers, so make it natural and easy for people to join in and help with what they can, even if it is just reporting bugs. Debian has recruited a thousand developers this way, and manages to have well over ten thousand packages. Many developers only have one package, but it tends to be one they need themselves, so they're motivated to keep it working.
Some barrier to entry may be necessary for quality and security. The Debian New Maintainer process screens and trains prospective new Debian developers. Note that this is a process with two different tasks. On the one hand, it is necessary to make sure the people who become official developers are well-intentioned and have the right attitude: an official developer effectively has root privileges on a large number of machines and so they must be trustworthy. On the other hand, many of the skills required of a Debian developer can be learned during the NM process. The process also helps them become part of the Debian development community.
Have leadership. There must be some way to solve conflicts and arguments, if a natural consensus doesn't occur. Different projects may need different kinds of leadership, but there has to be some way, formal or informal.
Conflicts are natural, but mustn't get out of hand. Developers are often opinionated, strong-minded, and bullheaded. You can't avoid that. They must be allowed to scream at each other every now and then, but fights must not be allowed to escalate. Everyone in the project will have to be responsible for mediating between disagreeing parties, if they can't come to a consensus by themselves. Unfinished fights will easily haunt the project.
Developers do not have to like each other, but they do have to get along, at least to the extent that they don't prevent each other from working on the project. If two people despise each other, they must learn to ignore each other, and especially to learn to ignore insults the other party may throw.
Use a bug tracking system. An open bug tracking system makes it harder for bugs to be forgotten when developers are too busy, or leave the project, or lose their disks. It also lets people see whether they want to use a package before installing it, and to find solutions or workarounds for bugs that are already known but not yet fixed.
Real world politics matter. As the free software movement has grown, and Debian with it, people outside the programmer communities on the Internet have started to take notice. Inevitably, this brings in matters of real world politics. As a result, people within Debian need to mind such political issues as different developers voting for opposite parties or living in countries almost literally at war with each other. Debian developers occasionally need to step very carefully to avoid upsetting each other. One cannot assume Debian developers have anything in common except the desire to work together on a free operating system. The social contract binds us together, pretty much nothing else does.
Controversial issues will result in a flame war and often will never really end. Almost all discussions within Debian take part on mailing lists. This is good in many ways, but has some drawbacks. One such drawback is that the natural limitations on aggressiveness that are present in a face-to-face discussions are missing. This means that controversial issues easily result in flame wars. Another drawback is that there is no natural deadline to a mailing list discussion: in a face-to-face meeting, sooner or later the participants will want to go away and that ends the discussion. On a mailing list, things can easily go on for weeks, and can easily re-surface at the drop of a hat.
These facts also mean that finding a consensus and making a decision as a group is hard.
Compensation helps keep people motivated. Doing much work without any reward does not work in the long run. The best form of compensation depends on the person. For some, the compensation of having a Debian package of something they use themselves is enough. For others, getting thank you notes is what makes them happiest. What rarely works for anyone is getting flamed at for doing their best, even if is in reaction to an error they made.
Never, ever, write a program to mail Debian developers automatically. Always make automatic mails opt-in. Most Debian developers get and process huge amounts of e-mail each day. They develop wildly different e-mail reading habits that are best suited for their different temperaments. They tend to take their e-mail personally, and react strongly to any e-mail they didn't want to get. Thus, sending them automatically generated e-mail (say, to inform them that there is a new upstream version) is a good way of annoying them.
Don't make assumptions about people's background. Don't assume anything about their ethnic, religious and cultural backgrounds, their political opinions, their linguistic ability, their technical training, their geographical location, their lifestyle, or their knowledge about issues not directly related to their packages. For example, don't assume they know what time zone you mean unless you are explicit about it (and giving times of day in UTC is better than assuming that people know what "EST" is). Don't assume they know English well. The fewer assumptions you make, the easier it is to communicate successfully.