This is the final part of the series on one-man open source projects. In the previous entries i talked about the different facets of the issues that face single-developer projects:
There are a lot of big projects out there with dozens of active developers and hundreds of people actively involved behind the scenes. Sometimes, developers of these projects have the luxury to concentrate on what they love to do the most – coding. However, the terms such as steering committee, constitution, incubation and governance models do not apply to the vast majority of the open-source projects. Such projects frequently languish because their developers fail to recognize that all those menial and tedium tasks are crucial for the long term health of their projects.
Sometimes, the solo developer behind a fledgling open-source project has an unreasonably high level of expectations from the target user sector. This frequently leads to frustration from low number of downloads, lack of interest from the blogosphere and may even result in venting your anger against the competing projects in public forums, mailing lists and blog comments. While being the best in your field is an understandable goal, i would advise focusing on other aspects instead.
Is being the best a goal in itself? How about admitting the following three simple things:
- You know you’ll make mistakes
- You’re ready to learn from them
- You hope to not repeat the same mistakes in the future
This way, you will not be afraid to make a mistake, not be afraid to admit a mistake and be very quick to fix one. And in any case, what are your chances of being the best? Next time you attend a technical conference, look around you. What are your chances of being the smartest guy in the room? What are your chances of being the smartest guy at the conference? What are your chances of being the smartest developer using your favorite language? What are your chances of being the smartest guy in the world?
If you have a blog, if you have your own open-source project, even if you comment on somebody else’s blog – you want to be heard. If you’re open to comments, bug reports, enhancement requests and a genuine two-way conversation with your users, you will be surprised at how much you can learn from them. And if you have the ability to cut through the complexity, distill chaotic flows into a clean API and communicate without condescension, they will come.
P.S. This series was originally conceived as a session for this year’s JavaOne and OSCON conferences. Even though the reviewing committees were not overly impressed, i do feel that it is an important topic. You can download the full slides in PDF format. There are no absolutes. This is what works for me.
If you write it they will come. Open source software has thousands of eyeballs monitoring it for flaws. Let the code speak for itself. The code is the best documentation. Haven’t we all heard this before, and haven’t we all had this type of wishful thinking? Alas, the harsh reality of hundreds of thousands of languishing open-source projects is in a stark contrast with the rosy promises and expectations that you might nourish when you start your own one-man open source project. Your pour countless hours into ??????developing and maintaining your project, only to see the weekly download count stuck at single digits, and community at large blissfully ignoring your labor. There is only one thing that you’re missing, and for some it might evoke an uneasy association with “those marketing suites”. In a one-man project, you are the developer and the maintainer, but you are also the tech pub, promoter, evangelist and marketer.
You need to stay passionate to continue developing your project even when nobody comes. You need to stay focused to maintain your project and properly prioritize different tasks. When it comes to promotion, you need to stay committed.
Most of the developers are introvert by nature, preferring to let the code speak for itself. The only problem is that there is too much code out there, and your potential users need to be gently and constantly reminded just how awesome your code is. Obviously, you’re not one of those “marketing suites”, and your target audience is your fellow developers. All you need to do is to think about how to “sell” your project to a guy just like yourself. That would mean highlighting the capabilities of your project, objectively comparing it to the competition (even when you’re lagging behind in some areas), not digressing into marketese and buzzword speak, and, perhaps most importantly, constantly staying on the radar.
The last part can be a turn-off for a coder. If you’re a coder, chances are that you don’t like to write documentation. Also, chances are that you don’t feel the need to discuss the reasons behind a specific design / implementation decision, especially if you’ve built yourself a little ivory tower. Chances are, even, that you don’t feel the need to create and maintain a comprehensive set of test applications that showcase all your awesome APIs. Finally, chances are that all you want to do all day long is to just write the damn code. Indeed, promoting a project takes a serious commitment, especially in the light of all the things that you need to do to grab peoples’ attention and don’t let go.
I have already mentioned this in the previous entry – new features should be the last item in your priority list, just after the documentation. I’ve also mentioned the catch-22 situation – nobody reads the documentation, but you must have it to answer the users’ questions in any case. It is your fault if the users need documentation to understand your library, and it is your fault if the documentation is too complex to immediately find an answer. However, the documentation is the “looks” that sell. Poor documentation or lack of thereof indicate a project that does not place enough value on users’ time. Documentation generated by Maven-like tools may fool the users a little, but the end result is an even more disgruntled user that was tricked into thinking there is documentation, when there is none.
Take a break from your frantic coding and ask yourself a simple question – how easy it is to understand your library for a user that has absolutely no knowledge in this specific area? Do you have a simple and intuitive index to guide the novice users around the basic concepts? Do you have a quick start guide? Is it kept in sync with your latest release / development version? Do you have simple examples that show how to use each single API method? Do you have in-depth tutorials for advanced users? You might be a star coder that can download a new library, load the sources in your text editor and start using it right away, but most of us are not. Do your project a favor and document every single thing that you can think of. And by the way, every time you get a question on “how do i do …” in a project forum or mailing list – it is your fault, even when it is documented.
Developers prefer to develop, especially when we are not confined by the project management to do all the tedium tasks. However, “the source code is the best documentation” is a sorry excuse, especially nearer the end of the release cycle. Instead of postponing the documentation until all the coding is done (which never happens), just spread it across the entire release cycle. You need to consider it as a vital part of the project tasks, and always think from the perspective of existing and potential users. Do not concentrate on how much you hate writing documentation. Instead, just do it.
The same principles apply to the test applications. While you should have a few big test applications that showcase your library as a whole, you should also write a very simple and complete example for each one of your public APIs. Keep them always in sync with the latest release (or the latest development version), interlink heavily and organize in an intuitive manner (grouping by business / related functionality).
Finally, have a blog. Get into the habit of writing 2-3 times a week. You might think that you don’t have anything valuable to say, but you will be surprised. If people want to download your code, they will also want to read your blog. It doesn’t have to be exclusively about your project. On the other hand, it doesn’t have to be about your favorite types of food. Once you get into the habit of writing, it will be much easier to continue doing so, even if English is not your native language. This might even come handy when you’ll need to write documentation at work. When everybody else is unhappy, you just cruise through and get to writing the code before they do.
There is a pleasant side effect when you blog about your project – from time to time you will get feedback from readers that are not users of your project. They might as well be potential users if you pique their interest, and they might provide very valuable feedback even when they have no intentions to be your users. You should never dismiss such feedback just because they are not your users. On the contrary, you might see why they do not intend to become your users, and gain some knowledge about your project, your goals and your vision.
The last entry summarizes the discussion.
The image is available under Creative Commons Attribution license from the following flickr.com stream:
The previous entry on one-man open source projects has talked about the development facet. Today, i’m going to talk about perhaps the most unappreciated and overlooked part of any project – the maintenance. It is my belief that if you want your project to become and stay successful, you must treat the maintenance part with utmost care and make sure that it receives all necessary resources, even much more so than the development does. The typical work environment usually enforces this with the relevant managerial structures. When you’re in charge, it might be very tempting to forego the more “boring” and “tedium”parts in favor of the “fun” and “creation” development tasks. However, it is quite easy to manage both aspects, and doing so is the subject of this entry.
While one needs to stay committed during the development tasks, it takes a concentration and focus for the maintenance.
As with the development, there are three main parts to the maintenance facet of the projects and here is the list with the most important one at the top:
Many developers consider bug reports as a sometimes unwelcome interruption to the development flow. Some will blame the users for their incorrect understanding of the APIs, some will postpone the existing bugs indefinitely and perhaps even half-jokingly call them features, and some will follow the existing industry trend of gathering up all the bug reports until the last two weeks before the release and handle them in a condensed and focused “bug-squashing” effort.
Unfortunately, these approaches ignore the vital part of a bug report – the submitter, or the OP in some parlance (stands for original poster). They all overlook a simple point that the person on the other side of the bug report has gone way beyond what we usually do. Not only has he decided to download your distributables, not only he has found enough time to play with your test applications or even write his own one, he was also courteous enough to find a way to contact you and describe his experience with your product. The least you can do is to fire off an immediate and polite reply.
Not all bug reports are valid. Some will come from users who have no idea what are they doing. Some will not contain enough information to reproduce them. Some will come via channels that you explicitly discourage (for example, personal e-mail instead of the official project tracker). Some will contain vague hints of threatening to switch to alternative products if the bug is not addressed immediately. Some will have nothing to do with your project and originate in other libraries. You just have to remember one thing – it is always your fault.
People just don’t have enough time. They don’t have enough time to read the documentation. They don’t have enough time to search the forums and the mailing lists. They don’t have enough time to look at the source code. Most certainly they don’t have enough time to “download the head and compile”. If your API provides too many ways to achieve the same task, and the bug report is on a non-recommended way – it’s your fault. If your documentation does not have an immediate answer – it’s your fault. In fact, the documentation is a chicken and egg problem – if your library needs documentation, it’s unfortunately your fault as well. The best thing to do is to accept this catch-22 situation and try to assess the best ways to alleviate the user pain.
Make sure that you have as many ways as possible to report a bug or raise a question. Having archived mailing lists and forums is great for the “ivory tower ego” type (with a knee-jerk instinctive reaction to “first search the forums and RTFM”), but people just don’t seem to do this. Even a simple requirement of registering to a mailing list to send a mail is a turn-off for many. Replying to a personal e-mail by redirecting the questions to the archived mailing list / forum might not be a welcome thing for users that do not want to divulge potential IP aspects of their applications in public.
Respond to a bug report immediately, and it will do wonders to establish early adopters and give you extra testers. Mark every bugfix code path with bug number and simple description so that you don’t accidentally refactor it out after a few months. The worst thing that can happen to a new feature is no feedback at all. It doesn’t mean that it’s flawless. It simply means that nobody’s using it.
If i may digress a little, here is what you would frequently see in open-source projects that don’t interact with such entities as steering committees, governance models and incubators:
Add to this a spruce of Maven-generated goodness to create an illusion of documentation and you have a recipe to quite a few open-source projects out there. If you’re fortunate enough, you get a little of code health (more on this in a moment) and real documentation; sometimes you’re not so lucky and too much code health results in rewriting the entire project from the scratch, getting back to what we love the most – throwing out old code and writing new one. In reality, your priorities should look like this:
Having the features at the very bottom of the list might sound alarming for a commercial product (what are we going to sell, exactly), but the first three are overwhelmingly more important for the long term health of a one-man project. Going back to the proposed ten-week release cycle for a minor release – yes, sometimes a minor release will not have any new features at all. In order to understand this priority list, allow me to analyze the internal implementation complexity over the cycle of a few consecutive releases.
If you have a clear project vision as mentioned in the previous entry, every new feature needs to conform to that vision and interact seamlessly with every existing one. Otherwise you end up with featuritis and feature soup. In an ideal world, adding a new feature has the same complexity no matter how many existing ones you already have. In a real world we’re all humans, and we all make mistakes (as we get more experienced, our mistakes tend to get more elaborate and destructive). Wouldn’t it be nice if every once in a while this graph would go down instead of climbing up?
The big question, of course, is how exactly can you do that? The smaller one, but important as well, is can you go back to the internal complexity level of the previous major release? As we’re talking about the internal implementation complexity, lowering means changing. It can be simple refactoring, or can go as far as introducing binary incompatibility and breaking existing applications. How far can you go until your users or even early adopters abandon you? And finally, what is more important to you, your users or your vision?
These are not simple questions, and they do not have simple answers. For me, it boils down to one concept – code health. Code health is the intangible metric of your confindence in the code stability, in your ability to find and fix a bug and in your ability to adapt the code to the new requirements. In my opinion, it is the second most important thing after prompt bug fixes, and it has three main parts:
These are the things that you would do when the code becomes too complex. I would even say that these are the things that you should do before your code becomes too complex. In fact, you shouldn’t take the pride in how complex your code is.
The internal refactoring is, of course, the simplest step to prevent code complexity, but sometimes it is not enough. As i mentioned already, we all make mistakes, and some of the mistakes are in the exposed API layer. It might be a bad method name, a missing parameter, an incorrect modeling of the user flows or even too many ways to achieve the same thing. As long as you either have a better way to achieve the same functionality, or a very compelling and preferably non-technical reason to remove an existing functionality – do not be afraid to break APIs and compatibility. If you’re small, nobody will care because nobody will notice. If you’re big, the users will trust you and come with you. Otherwise, it may be a very good indicator if you can get big or not.
The last thing that i want to talk about is the task schedule. This is how you break those ten weeks and allocate different tasks. Here, you should follow the sources of instability and weigh the risks against the benefits to decide when each task should be done. In general, the instability comes from new features, internal rewrites and bug fixes, and you should minimize each stage and confine it to end as early as possible during the release cycle.
Adding new features is the biggest instability risk and should be done in the first four weeks of the cycle. Here, you can hope that your early adopters and testers will be kind enough to test these features and provide enough timely feedback. If they don’t, and you make a mistake that is exposed only after the release is done – you need to wait for the next major release to remove / replace it. If you follow the proposed a-major-release-a-year schedule, you won’t wait too long.
The internal rewrites should not (in principle) have much effect on the users code. The reality, of course, is not ideal. Your users can depend on an undocumented incidental behavior. They can decide to use internal classes or even resort to hacks without ever asking for a proper way to achieve the required functionality in the project forums and mailing lists. However, the internal rewrites generally have less impact than new features, and as such may be done until two weeks before the release candidate. The recommended release candidate date is two weeks before the final release.
Bug fixes should be done all the time. In fact, the earlier you do it, the better it is for your project. If you engage in this cycle the very same day as the bug report arrives, you have much greater chance of getting an actual confirmation from the user that it works. And if you are lucky enough, every once in a while this user will turn into an early adopter. And once release candidate is out, you should fight your every desire to add a new seemingly miniscule feature or do any sort of refactoring.
Since you have only four weeks to add new features, you’ll need to sharpen your skills in making compromises on what goes in and what stays out and waits until the next release. If your users threaten to switch to an alternative product because of “just this one feature” that is missing – it might be a good thing for your project in the long run. However, just because you “need” something to show to warrant a new release doesn’t mean that you must force a solution to the specific user request or a new feature that you want to add. Let your unconscious work on it for a while.
The next entries will continue talking about the specific project tasks.
- Part 4 talks about the promotion
- Part 5 summarizes the discussion
The image is available under Creative Commons Attribution NonCommercial license from the following flickr.com streams:
Meerkats by 814 carthage
As mentioned in the introduction entry, every one-man hobby open source project has three main parts – development, maintenance and promotion. Today, i’m going to talk about where it all starts, and, if you’re not careful enough, where it all ends.
It all starts, of course, with the development. You take an idea and you put it to a virtual life. This is perhaps, the most enjoyable part of a new project, at least for some of us. This is the time where everything seems possible, where there are no implementation complexities, no bugs to fix and no annoying support requests for esoteric environments. Even if you are doing this on your free time, it doesn’t take a lot of commitment to just sit down in front of your favourite editor and start banging away. It is also, unfortunately, a time where a very sizeable percent of the projects simply die.
It takes a serious commitment to continue developing a new project on your own free time, at the expense of your other hobbies and sometimes even a family. This is not only being enthusiastic about your idea. It is about being passionate. If you’re not passionate about the idea in particular, and about its field in general, you might as well not start at all.
There are three main aspects of the development. These are most certainly not specific to one-man projects. However, due to the very nature of such projects (when that one man leaves, the project is no more), not planning the development cycles in advance can have very adverse impact on the project’s ability to survive in the wild ocean of today’s open-source.
The majority of successful open-source projects (no matter how big the development community is) have a very clear set of tenets. This is sometimes referred to as project goals, vision, direction or guidelines. While the tenets do not need to be set in stone, it is crucially important that a project has a clear, concise and simple definition of what it aims to do. A clear project vision helps immensely in deciding what goes in and what stays out. It helps the users to understand whether the project fits their needs, and it makes sure that the project does not end up being a jack of all trades, and master of none.
It is certainly not my intent to create a bad impression on this specific project, and i’ll let you judge its vision on your own:
[…] is a high performance, vastly scalable, embeddable, internet-centric client/server computing platform with a breakthrough adaptive runtime technology.
Speaking only for myself, this is full of buzzwords and marketing speak. Too much information is crammed in here, with too many generic overused words. I had a hard time remembering it a few days after seeing it on the net. Now, compare it to these very focused and plainly-worded tenets of FormLayout project:
It is a very simple and down-to-earth list. It is also very memorable. The entire library is driven by these four statements which set the clear rules for what goes in and what stays out. It doesn’t claim to want to be the best for everybody, and it doesn’t proclaim to provide a solution for all possible scenarios. Instead, it focuses the APIs around a simple and coherent vision, and even explains why some things (that the author considers bad) are hard to achieve.
I will talk about the features in the next entry, but for now i’d like to skip to the third facet of the development – the release schedule. Here is a real example of one of the projects hosted on SourceForge, illustrating the “anti-pattern” or release schedule planning:
On the face of it, everything looks good. There have been eleven releases over the last six years, and the project looks quite active and maintained. However, there are two big sore points with this table – the leading zero in version numbers and the ever increasing time span between consecutive releases.
In this particular case, even the project’s developer does not consider it to be worthy of “1.0” release label. One might say that it is a minor thing which has nothing to do with the actual quality of the project. However, the outside looks count and not only in real life, unfortunately. As cliche as it might be, dress for success is a very good advice, and in virtual world this means stripping the alpha / beta / 0.* status from your project. You have very short time window to impress the potential users, and a solid release number with no greek letters is a must-have. This is the least you can do after six years of development.
The second sore point of this specific project is the ever-growing time frame for each release. It took 2-3 months for the first six (when the initial level of commitment was quite high), which then dropped to 4-6 months for the next two releases, and 11-14 (and counting) for the last two. The potential users that seriously consider investing in your project (using, sending bug reports or even contributing bug fixes) might be rightfully scared by such a trend.
To each his own, but i found that the following release schedule works best for me:
Here, i have three minor releases in between each pair of major releases. I give ten weeks for .1 and .2 release (more on the inner structure of each release later), twelve weeks for .3 release and twenty weeks for the major .0 release. The extra two weeks in the .3 release cycle are spent on code health which will be the main topic of the next entry. The extra ten weeks in the major .0 release cycle are spent on code health, removing all dead code and deprecated functionality (even if it means breaking binary compatibility and existing clients) and allowing the early adopters some time to test the changes and adapt their applications.
There are quite a few advantages of having a regular and predictable release cycle. As long as you commit to the planned release cycle, it allows your users (existing and potential) to align their plans and schedules and make sure that they are not going to slip their schedule because of you. It also gives a very important outward sense of stability and predictability about your project. In addition, it puts a quite welcome (if not necessarily obvious) pressure on making sure that you continue actively looking for the new ways to improve your project.
I’d like to elaborate a little on the last point. When you only have ten weeks for a minor release, you need to be constantly aware that the release must be “justified”. Allowing yourself to postpone your work on the project just because it’s summer, or just because you met this great new girl, or just because there’s some pressure at work is a luxury that a one-man project simply cannot afford. The more you postpone, the more it becomes a norm, and the end result is seen in the previous slide (going from 2-3 months to 11-14 months in between releases). Waiting for a “perfect moment” to work on all those great new features once you get enough free time does not work. The more time you spend away from the project, the harder it becomes to get reacquainted with it when you get back. When you think in terms of “next weekend” instead of “tonight”, it becomes very easy to postpone it to yet another weekend and the one after that, especially if you’re waiting for a chunk of 10-12 straight hours that you want to dedicate to it.
“Justifying” the release means that you need to show compelling reasons to take a new version. It includes bug fixes, performance improvements, reduced memory consumption and new features. When you only have ten weeks to work on a release, it becomes a cycle that feeds itself (just like getting into the habit of postponing the development is a cycle that feeds itself). Developer’s work is never done. We’re constantly striving to expand our knowledge and grow professionally, and this means that the code we wrote yesterday will ever be questioned tomorrow. The project may be “good enough” to some, but the really successful projects are never good enough in the eyes of their developers.
This release cycle has a nice side effect – it adds up to 52 weeks in between the major releases. This means that your project will have a major release once a year, further strengthening the outward appearance of both stability and progress.
The next entries will continue talking about the specific project tasks.
- Part 3 talks about the maintenance
- Part 4 talks about the promotion
- Part 5 summarizes the discussion
The image is available under Creative Commons Attribution NonCommercial license from the following flickr.com stream:
