?

Log in

Wouldn't it be nice ...

if life were perfect?

10/7/07 11:13 pm - Number 26.

That's my ranking in the "Top Linux blogs," according to Don Marti. I'm a little confused how that happened, given how rarely I've blogged lately. Perhaps he knew I had some more good ideas in the pipeline. =)

8/3/07 11:48 pm - Improving quality in Gentoo

I recently posted about making Gentoo a better tool. A requirement for being a good tool is being a tool that doesn't break—thus, we need to improve our quality to a more reliable level. I'm going to mention a few ideas to start this discussion, which I hope the rest of our community will participate in.

First, we essentially have no code review. About the only time any code in Gentoo is reviewed is before and during a developer's training, with a notable exception being the requirement to post eclasses to the gentoo-dev list. Increasing our code review ought to result in an increase in quality, in ability to justify code in words, and in a stronger community of contributors.

How do we increase code review? One idea is to require reviewer approval prior to committing, but this isn't the best answer for Gentoo. We've always been a pretty open community. Developers aren't prohibited by ACL from committing anywhere in our ebuild repository, so I don't think they would accept additional requirements that increased the burden of contributing.

Instead, let's create a gentoo-commits mailing list or RSS feed(s), with full diffs. We should use this tool in many different ways.
  • Each team should use it internally to review all commits to its packages.

  • Mentors should continue to follow their mentees' commits well after they're granted commit access (6 months minimum, and I recommend forever).

  • Mentees should also review their mentors' commits, first to learn and later to review.

  • Every developer should have at least one reviewer and review at least one other developer. This should be formal and documented to ensure it's happening.

These uses will require that the commit diffs be easily filterable by both committer and files affected. RSS feeds could be made available based on developer or herd, and e-mail lists could contain the information in e-mail subjects or headers.

Second, we should improve our unit testing, where the units are individual packages. This can be both automated and performed by developers and arch testers. Although a number of packages have a useful, working test suite, most lack one. For these packages, we should attempt to provide something automatable in src_test() even when a test suite is absent. Failing that, we should print out a checklist in src_test() of tests to perform before stabilizing a package. There should never be an empty src_test().

Another package-level testing approach is to create solid, automated tinderboxes. This remains unrealistic for our entire database of 10,000+ packages, but we should at least get this going for our "system" set and perhaps for some of the most common sets of packages for servers and desktops. Exactly how to set this up remains a question, since there's a lot of tinderbox code floating around. Bonsaikitten has some almost-working code based on swegener's work; Catalyst has some tinderboxing capability; or we could look into using Mozilla's tinderbox.

Third, we should improve our integration testing, on the entire repository level. Our main source of testing here will be our users, because they have infinitely more combinations of build options and hardware than we can reproduce on Gentoo infrastructure. But how can we take advantage of this testing to improve our quality? By creating an additional, time-lagged set of rsync mirrors with additional QA checks, we could allow users who want to test the latest and greatest software to help those who want stable and solid software.

We already have keywords for ~arch and arch, but they're still too mixed. A problem in ~arch ebuilds can break the entire tree for all users. They really need a stronger separation. Perhaps the separate repositories should be ~arch versus stable. But another way to do it is to add a delay to the second set of repos, anywhere from 24 hours to a week. This delay allows us time to encounter major problems in the fast-sync repos, fix them, and carry the fixes over to the slow-sync repo. But we'll need a way to make this really easy to do. It feels like branching with periodic merges, along with cherry picks of major bugfixes, is the right way to do this. Unfortunately, CVS sucks at this. We may need to migrate to a more capable version-control system before this option becomes realistic. In addition to the user testing, we could add a tinderbox into the slow-sync repos to require that they build with the most common configurations.

To sum up, I want to increase code review, unit testing, and integration testing. These three things will strengthen Gentoo's quality, reputation, and community.

8/3/07 02:51 pm - Democracy as education in OSS projects

Reading the PressThink blog, I came across a couple quotes that apply well to open-source projects:

I kept thinking about a famous passage from Christopher Lasch, the great social critic and historian who died in 1994— before the rise of the Web. In the Revolt of the Elites, he said we learn more from argument than from information, not because opinions are weighter than facts, but because to argue for your ideas (in public) puts those ideas at risk. And that is how we learn. ...

Lasch in his book:

If we insist on argument as the essence of education, we will defend democracy not as the most efficient but as the most educational form of government, one that extends the circle of debate as widely as possible and thus forces all citizens to articulate their views, to put their views at risk, and to cultivate the virtues of eloquence, clarity of thought and expression, and sound judgment… small communities are the classic locus of democracy— not because they are “self-contained,” however, but simply because they allow everyone to take part in public debates. Instead of dismissing direct democracy is irrelevant to modern conditions, we need to re-create it on a large scale.

7/31/07 03:03 pm - Gentoo slides from OSCON

At OSCON, I gave a lightning talk on what's happened in Gentoo in the past year. It was part of a big session covering about 15 or 20 different projects, and it's been a lot of fun both times I've done it. Here's the slides: [ODF] [PDF]

7/23/07 02:34 pm - LWN at OSCON

If any of you are LWN readers and would really like to see coverage of any specific sessions or topics at OSCON, please leave a comment on here and I'll see what I can do.

7/22/07 04:16 am - A productive evening

CIA commit stats: 328 messages so far today

7/20/07 02:36 pm - What has Gentoo accomplished in the past year?

I'm giving a short talk at OSCON next week on Gentoo's progress in the past year. So please comment on my blog to let me know areas you want me to cover. I'm open to any and all ideas.

Thanks!

BTW: Anybody living near Portland got a couch I could crash on Tuesday, Wednesday, or Thursday of next week (July 24-26, 2007)?

7/20/07 02:34 pm - Improving Gentoo as a tool

Gentoo is a metadistribution, meaning you're supposed to built whatever you want with it. And we provide some of the tools to make that a reality, but we stop short. We still make it too hard to do what you want with Gentoo. Here's some of the ideas I want to explore over the next year:

  • Increased binary package support: We already have an experimental tinderbox setup to build packages for a variety of architectures. I want to investigate increasing our collection of binary packages to make a fairly complete installation possible using only binaries.

  • Repositories for user-created distributions: We have tools like Catalyst to create basic installations with stage3 or stage4 tarballs, but it's still hard to get what you want with them. We should make a repository of Catalyst spec files for a variety of purposes, and we should also add an accompanying mirror of the created tarballs so anyone can use them. We could also revive the Seeds project to create not just vanilla stage4 tarballs but fully fledged, preconfigured installations customized for specific purposes like LAMP servers, development machines, various cluster nodes, GNOME/KDE/Xfce desktops, etc. We already have the possibilities; we need to share our tools.

  • Creating the Gentoo metacommunity: In addition to letting you create your own distribution from Gentoo, I want Gentoo's community to be what you want it to be. We're in the middle of adding a few new mailing lists because our primary development list is drowning in noise. We made gentoo-project for non-technical general talk and gentoo-dev-announce for development-related information so developers don't need to slog through gentoo-dev. I want to take that a step further by forming stronger Gentoo microcommunities around specific areas and moving discussion about those areas to their IRC channels and mailing lists. Our development community has grown too big to keep everything on a general development list. Everyone tries to chip into every discussion, even if they have no relation to it or are unaffected by it.

  • Excellence: Gentoo's QA is not the greatest. What can we do to improve this? Some automated tools exist for pre-commit checking; can we add anything server-side? Can we add some build servers for critical system packages, so they don't make it to users before building in a predefined set of common configurations? Can we improve our developer recruitment or add new training for existing developers? How can we renew and strengthen our commitment to excellence? You can't create a tool with Gentoo if it's broken.

  • New tools for new places: This is more of an exploratory idea. I'd like a team of contributors to research all the places people use Gentoo, and put together a collection of tools we're missing. Further, we need to do more extensive research on other distributions to see what sorts of tools they have that we lack. I have no symptoms of NIH syndrome, so I'm happy to pull in working tools written for other distributions; I already did this for many of Fedora's system-config-* tools last year.

  • Resolving the meta vs. specificity conflict: Gentoo's status as a metadistribution sometimes produces conflicts between the goals of various projects and teams within Gentoo. Sometimes a person simply needs to make a decision of how to deal with it: pick one, or find a way to do both? Many people remarked over the past couple of years that we're lacking strong leadership and overarching goals, which would give us some consistent way to decide which projects will "win." As Seemant has said, Gentoo is a framework, a metadistribution, not a place that forces people to go a certain way. We should strive to enable contributors to follow whichever path beckons to them. But where is Gentoo going, and where should it go? You decide. The innovators and leaders can only suggest places for it to go, so the rest of the community has to follow them to our destination.

7/20/07 02:07 pm - Q&A interviews suck

I hate reading Q&A interviews. They're a huge waste of time, and they say to me that the journalist quit halfway through his job. I'm not disparaging the Q&A format as a whole, which can work great outside of interviews, but I despite seeing it in them.

Here's why. When a journalist writes a story, the process goes something like this. First you think of an idea. Then you think about who to talk with about that idea. Next, you make a list of questions to ask them. Then the interview: you ask the questions and write down the answers. This is where the Q&A format gives up. After the interview, you do the most important part of your job—you synthesize the information, making connections between all your interviewees, other sources, and prior knowledge. Then you put time into writing a clear and concise story that doesn't include anything beyond what's needed.

As the reader, I expect you, the journalist, to invest your time wisely so I don't waste mine drawing all the conclusions you should've drawn for me. I expect you to cut the material that's unrelated to the story you're telling. Don't stop at the Q&A; write the story.

7/20/07 01:55 pm - Why I volunteer to work on open-source software

A big ruckus came up on the Gentoo development list in the past couple of weeks, and part of it involved Gentoo contributors and their motivations. I'm sure most of us have heard that open-source software is about scratching your own itches, but I'm not sure that everyone really understands what that means.

For me, what it means is that my time on Gentoo needs to fulfill me in some way. If not, then I've got tons of other places to invest my free time. It does not mean every change I make fixes a problem I encountered. It does mean every change I make has some return in value. What's value?
  • Functionality: A feature or bug fix that directly affects a program I use

  • Pride: My reputation as a developer relies upon clean, bug-free code. It's important that my code is beautiful and functional.

  • Gratitude: Thanks from someone else for the time I spent on their problem

  • Reciprocity: An expectation that others will contribute in areas I care about if I work on things they care about

  • Stimulation: Spending time around other smart people who care about the same things

All of these are selfish, because I'm directly getting some sort of value out of my time and contribution. Many times, Gentoo developers have said they develop for themselves. That's what they mean. Consequently, the people I care about making changes for are people who give me something I value in return. If you're contributing in some way, I want to help you. If not, what's my motivation?
Powered by LiveJournal.com