May 28, 2010

SubmittingPatches is better than "Patches Welcome"

So I received my first patch for Perl code a week ago. Amazingly I didn't have to tell the person patches were welcome, they must have assumed so because my project was licensed under an OSI approved license. Also amazingly I've applied the patch even though it didn't meet my quality standards. I figure that's my fault though because no where did I document what those were.

Back when I forked Regen2 from Funtoo one of the first things I did (and a reason for the fork) was create a policy for patch submission. The policy for Funtoo was that all patches were welcome, and I recall 2 patches by 1 user specifically that should have been 10 patches and 2 (or more) of those 10 patches should have been dropped, because they were doing things they shouldn't have. In order to create this policy as easily as possible, I stole Git's SubmittingPatches policy and modified it to work with Regen2.

Once again I've been faced with patches that just aren't up to par. So once again I'm using Git's policy as a baseline for contributions to my projects. Here's the beginning of the documentation

Checklist (and a short version for the impatient):


- make commits of logical units
- check for unnecessary whitespace with "git diff --check"
before committing
- do not check in commented out code or unneeded files
- the first line of the commit message should be a short
description and should skip the full stop
- the body should provide a meaningful commit message, which:
- uses the imperative, present tense: "change",
not "changed" or "changes".
- includes motivation for the change, and contrasts
its implementation with previous behaviour
- if you want your work included in the main repository, add a
"Signed-off-by: Your Name " line to the
commit message (or just use the option "-s" when
committing) to confirm that you agree to the Developer's
Certificate of Origin
- make sure that you have tests for the bug you are fixing
- make sure that the test suite passes after your commit


- use "git format-patch -M" to create the patch
- do not PGP sign your patch
- be careful doing cut & paste, not to corrupt whitespaces.
- provide additional information (which is unsuitable for
the commit message) between the "---" and the diffstat
- if you change, add, or remove any features or
make some other user interface change, the associated
documentation should be updated as well.
- if your name is not writable in ASCII, make sure that
you send the patch in the correct encoding.

Long version:

The only part I added was to add documentation for features. Obviously some parts under patch need modification as I'm not using a mailing list.

Obviously part of the reason I don't like "Patches welcome" is I've seen that policy lead to the acceptance of really bad patches. Now obviously I make mistakes, everyone makes mistakes, but patches can be improved before they are committed. Even good patches can be flat out rejected for other reasons (I've had this happen. The consensus was not that the patch was bad, but the entire script need to be rewritten).

So what was wrong with this patch if I accepted it? well essentially it wasn't a git patch and it was applied to the wrong directory (because I had the original source in src/ and the generated a / I'm not using CommitBuild to another branch) and, it also broke my test suite (even though it was a patch for a test. The breakage was a bug in another test).

In any event, I believe that having your project be "Open Source" and having a patch submission policy does more to say "Patches Welcome" than "Patches Welcome". You don't have to agree with that, but regardless of whether you do, or not, I think having a policy is a good thing.

P.S. Trolling posts will be deleted. Be constructive or talk elsewhere.

May 21, 2010

Warranties, what do they have to do with anything?

so the whole 'open source' has no warranty thing keeps coming up as a defensible reason that I'm wrong.

Microsoft offers a 90 day limited warranty that the software will work to some degree or another as advertised or some such. If you really want to consider what they're saying... it should work if it doesn't we can send you another disk or you can return for refund. which really is what you could have done at the store anyways. I suppose it also means it won't fry your hardware, does anyone know any stable open source software that does that? I suppose a few things could if tweaked right..

Open Source generally doesn't distribute a physical medium, or fry peoples boxen, so what would we even use a warranty for? oh right... a CYA. We really don't want to be sued because someone downloaded our software and it didn't work or broke something of theirs.

Generally your software warranty doesn't really cover support, and if it does it's for a very limited time, and very limited support. Warranties are more for physical medium. For example my Dell came with a 1 year warranty, but because I ordered it with Ubuntu they would provide hardware support only.

The point is that if you're claiming we shouldn't support our software because it is without warranty... the warranty as far as I can tell doesn't have much to do with the software and everything to do with the medium.

Really, the CYA is a good thing. I don't disagree with it... however, it shouldn't be used as an excuse to do nothing. It just means you can't be legally harmed if you do nothing.

That's not what I'm saying

people seem to think that I'm saying support should be a #1 priority. will someone please cite where I said that? yes I said it should be a priority, I said you should do it. I did not say where on the list it should fall. Even if you're an employee at a company making $100k a year, if you're wife is dying, your priority is that, hopefully your children come before said company in most cases too. Understand, that I'm not saying put your software (paid or unpaid) before your life. That would be stupid. I'm saying get this stuff worked into your schedule for working on software. Support is part of the software lifecycle. If you have 2 hours a day to code set some of that aside for support and documentation (pooma number).

Yes, I think I've proven that volunteering to do open source comes with a responsibility or obligation. However, since there is no governing body, and no one has given us rules of good behavior. What that good behavior is, is what is subject to debate.

So hopefully we can stop arguing that we have no responsibility due to being unpaid volunteers and a license, which is really (I believe) just meant to protect us from lawyers. Once we do that we can actually discuss what it is that we actually should consider ourselves responsible for, and more importantly how we can communicate it.

Software licenses is general do not communicate these things. I think David Golden said is on the right track. Keep in mind some Open Source projects (e.g. PostgreSQL discuss a support policy outside of their licensing ) I think postgres policy is excellent for a project of its size. However, I don't expect anyone to support a single release of a small project for 5 years. 1 year might be appropriate for a Major release version.

Actually, PostgreSQL is an exemplary example of how Open Source should be conducted. I've requested stupid features and been told why they were stupid, and no one cried foul of users requesting features. I've asked for help and always received excellent support for my problems. I personally have never been able to locate more than a minute documentation bug in Postgres, and it seems that all new features are adequately documented as part of their QA.

Why is it that we don't all strive for this level of quality?

p.s. the reason postgres created that policy in the first place was that some people started thinking they were supporting old versions (like 7.4) for too long.

UPDATE: I'm locking comments. I'm tired of people commenting when they obviously haven't complete read what I've written. Either that or there grasp of English is poor. It's irritating to be constantly misquoted.

May 19, 2010

of Volunteers and Work

So, I'm heavily criticized for my unpopular opinion that open source volunteering is a job. I would like to point out the definitions of Volunteer and Work from I have removed a few irrelevant definitions, like those of 'botany'.

A person who performs or offers to perform a service voluntarily: an information booth staffed by volunteers; hospital volunteers.

A person who renders aid, performs a service, or assumes an obligation voluntarily.

v., -teered, -teer·ing, -teers.
To give or offer to give voluntarily: volunteered their services; volunteer to give blood.

To perform or offer to perform a service of one's own free will.

To do charitable or helpful work without pay: Many retirees volunteer in community service and day care centers.
n. 1. a person who freely enrolls for military service rather than being conscripted, especially a member of a force formed by voluntary enrollment and distinct from the regular army.

2. a person who freely offers to take part in an enterprise or undertake a task.

3. a person who works for an organization without being paid.

1. freely offer to do something: he volunteered for the job.

2. offer (help) in such a way: he volunteered his services as a driver for the convoy.

3. work for an organization without being paid.

4. commit (someone) to a particular undertaking, typically without consulting them: he was volunteered for parachute training by friends.
You'll note 'service' and 'work' mentioned. Examples include things like 'fire fighters'and 'hospital' workers and 'military personel'. Imagine if these 'volunteers' didn't feel any responsibility for the work they did. ( actually some open source people do like definition #4 here. I've been volunteered several times in this way )

and since it mentions that volunteering is work

Physical or mental effort or activity directed toward the production or accomplishment of something.

A job; employment: looking for work.

A trade, profession, or other means of livelihood.

Something that one is doing, making, or performing, especially as an occupation or undertaking; a duty or task: begin the day's work.

An amount of such activity either done or required: a week's work.

The part of a day devoted to an occupation or undertaking: met her after work.

One's place of employment: Should I call you at home or at work?

I don't think these definitions say something other than what I have said. I believe they will be similar to what is found in other English dictionaries.

Ultimately, I do not understand why people are so vehement against this definition for open source 'volunteers'. I can't make you believe this is true, or do it. But my believing it and others believing it hurts nobody. In fact those who are helping and believe it probably produce good, perhaps better, work.

Agree, or disagree, for the most part I don't think I'm going to change anyone. It seems I've only angered good people. My thoughts on the matter aren't going to change. If people want to take their contributions more seriously, that can only be a good thing, in my humble opinion.

May 18, 2010

Google trashed my Google Site's Content

When I forked Regen2 from Funtoo over a year ago I created a web page for it on The hosting and uptime were both right for what I needed. The dns has long expired but I went to sites today to find a public link to the content, only to find that at some point they upgraded sites and completely trashed all the content I had put on there. Sure they have a revision control on there now... but it doesn't contain ANY of the content I had left. WTF Google? I'm sure someone is going to say 'backups' but honestly, I killed the project, whatever was there wasn't really important to me anymore. But I didn't expect that it would get trashed in some Google update.

May 10, 2010

GPL 3 and Artistic 2.0 Software::License

As of right now Software::License has no way to combine licenses on the fly. To be honest this deficiency might not be a bug in SL, it could be in Pod::Weaver or Dist::Zilla this is a know deficiency and RJBS plans on fixing it at some point. In the mean time we can implement dual licenses (and multi-licenses) in much the same way the Perl5 license is implemented.

I was probably in legal violation (I am not a lawyer) with Template::ShowStartStop version 0.05 and 0.06 because I used Perl_5 licensing because that's the closest I could get with Dist::Zilla, due to the limitation. So I created Software::License::GPL3andArtistic2 which is sort of a perl5 license, except that it mandates the latest version of GPL and Artistic (at the time of this writing). Version 0.07 of Template::ShowStartStop is back in compliance with the licensing of the code I forked from Template::Timer.

This hasn't, and will probably not be merged into Software::License because it's just a stopgap measure until multi-license support can be implemented.

this is apparently not necessary according to Duncan and the this interview I've updated the module to reflect this. Hopefully, it can serve as an educational source.

Curious... why doesn't Perl5 switch to the Artistic 2.0 then... (I wonder what that would mean for 'under the same terms as perl5' but actually including gpl1 and artistic 1).

May 9, 2010

Documentation! it should be TDDD

UPDATE: I shouldn't have said what I said about RJBS and have apologized publicly

Test Driven Development should be Test and Documentation Driven Development. I don't want to read your code to figure out how to use it. I don't really care if you write the Test, documentation, or the feature first. But you should do all three before moving on to the next one.

I'm gonna pick on Dist::Zilla this seems like a very good tool. Except it's documentation is so sub par it's not funny. The new documentation is quite good, although, I have not decided that I like the 'choose your own adventure' format. However none of the documentation on CPAN is good. It ranges from significantly less than the new tutorial, to virtually non-existant, to completely inaccurate.

You know what? It's not acceptable not to write documentation, especially not when you were apparently paid to do it and have shown that you are capable of doing it. Some people I've talked to think not documenting things is ok. I don't think it is, and it's yet another one of my unpopular opinions. Sure you can not document them... If someone else has volunteered and is actually doing it.

Yet maybe this is one of the reasons perl has fallen by the wayside. Documentation is usually either cryptic, incomplete, or inaccurate. Even the core docs like perlintro has inaccurate and potentially damaging information. Why are we so surprised that so many people don't know how to program perl in a modern way if the documentation is so bad?

Documentation should explain who, what, when, where, why and how. Who should use this module, what they should use it for, when they should (and shouldn't) use it, where they should use it, why they should use it, and how to use it. Unfortunately most documentation fails at all but the how, and the what. Who is an Abstract, and Description thing, you should consider who doesn't need it. Why why can be tightly related to who, but really I want to know why this module and not another, consider who needs a templating engine vs why it should (or shouldn't) be HTML::Zoom. When is also closely related to the previous 2 and will probably be answered with them, although consider when to use a for loop vs when to use a while. Where could be anywhere from top of file, to your view in a catalyst program. How is obviously the actual code and should be covered in the synopsis. Sometimes you'll have to think if you've answered all these, but if you manage to answer them all in a comprehensive and complete manner then you're documentation is probably good enough to be useful.

I don't expect people to be perfect in this, but some effort is better than none. Also if you write documentation, you may end up answering less questions, and save people many hours finagling a problem.

I'm sure even my modules fail to meet my own standards.

The best documentation I've seen is PostgreSQL's and Gentoo's.

perlintro was fixed in 5.12.1

May 5, 2010

Don't Say "Patches Welcome"

Don't say "patches welcome" it does more harm to your community than good. Why? It's a polite way of saying "I don't care. Fuck you. Fix it yourself. End of discussion".

1.) Not a Developer

The person you are saying this to is not developer. They don't know how to do it. PERIOD. At this point you are refusing to help them. This person may now have the opinion that your community isn't helpful. Don't ever for any reason assume someone is a developer, assume they are not. I would wager only 1 person in 10,000 is a developer.

I was once told that I should be able to fix a bug in git (msysgit technically) because git is a developer tool. At that time I couldn't code at all but was using git to store homework or something...

2.) Novice Developer

Some how you know the person can code through some method other than assumption. They aren't as smart as you are, there code is so amateurish that it wouldn't be worthy of inclusion, even if they could figure out how to fix the problem. They can't get that far, they've no idea where to begin. You're probably just frustrating them instead of helping the problem. All people that you know are developers fall into this category (unless they fall into the next), either be willing to help this person patch your code or just give them the real reason (e.g. I've no interest in this problem) for not helping. These people know they could be helping, they may even want to be helping, but you telling them that they could be is not helping. They need guidance, not an invitation.

3.) Doesn't Care

So the person doesn't fall into any of the 2 above categories. Guess what? they don't care about it enough. They don't need you to tell them "patches welcome", they know what open source is. They know they could send you a patch, and that you might accept it. They know all that, but there's some reason they don't want to. You aren't telling them anything they don't know. Again just tell them the truth of why you don't want to work on it. Maybe ask them why they don't. Maybe it's a silly little barrier that you can help with. But only bring that up if you know they are competent enough to do it.

Just Stop

Just stop, after years of hearing this when I couldn't do it. All I hear is "fuck you I don't want to take the time to help you". Why would I want to help if that's what I hear? Don't tell me it's not what you mean. Because I've had that said directly.

Stop acting as though everyone can or should write you a patch. Most of you don't agree that you have a responsibility to the free software you create. No one else has a responsibility to contribute to the software. They aren't leeching if they don't give back, you gave it away. Don't ask them to, don't tell them to. They don't want to hear it anymore than you want to fix their problem. Definitely do NOT "volunteer them" to get something done, that's just plain rude.

Pretty much everyone you've ever told this too already knew that patches were welcome, so there's some other reason they weren't writing one.

Exception: If someone asks you if you mind if they work on your code. You can of course answer them "patches welcome" this is because they asked, as opposed to you volunteering that info.

May 3, 2010

Teaching Perl - Week 3 - Textual Data

So last week you should have covered variables, conditionals, flow control statements, calling functions, stdin/out/err, and using new perl features.

This week we're covering text processing, scalars, external libraries, and reinforcing what we learned last week. Text Processing is one of Perl's specialty's, mostly due to it's powerful regular expression engine.

First install LWP with cpan. cpanp -i Bundle::LWP will do it.

So this is almost completely new... what does it do? and what does it have to do with text?

LWP is short for lib www perl. LWP::Simple is a simple procedural interface to it. getprint is a method which is roughly equivilant to fetch uri and print the content that you got back. So this isn't very useful... (ok it's actually quite useful if you're on arch and you want a PKGBUILD) what we want to do in this case is cut the junk and print the pkgname. As a side note, this code does the exact same thing as the python example in Head First Programming: A Learner's Guide to Programming Using the Python Language but is shorter if you cut the boilerplate (#! ... use strict; use warnings ), and the same length if you don't.

Now let's have a first attempt at getting the package name shall we? We need to read the contents into a variable first. note: we've changed from getprint just to get because it returns the contents to a variable. (note: still shorter than the python.) At this point the book goes into how a string is an array of characters. Perl is contextual so at this point our scalar is essentially a string but just because it's a scalar does not mean it's currently a string. In our previous examples we were handling it in a numeric context.
So here we're looking at the 11 character substring past character 281. The output should be modern-perl. This code is horrible, what should happen if I were to change this PKGBUILD? (if this ever doesn't output modern-perl that's probably exactly what happened). Remember say appends a newline character to it's output print doesn't. We used print previously because the output already had a newline at the end.

There are thousands of PKGBUILD's on AUR let's see if we can get the name from another one by changing the URI.The output at the time of this writing is
and will probably change. A newline is just another character in the scalar so nothing prevents one from being printed within the substring. Our URI got a little long here so we moved split that peace of code up to be on 2 lines for readability. Your code lines shouldn't go over 78 characters long if you can help it. A URI would be an acceptable exception though.

So we use the index function to search the string for some text and return the position we can then use that position to get our substring. The position returned is 1 less than where the string we searched for starts. So we added the 9 to get to the string we actually want. Our output is now
why the '? well "perl-moose" is shorter than "modern-perl" so we've got extra characters in there.

Let's use a regular expression.
regular expressions are really their own mini language, very terse, and several dialects. Have your students go through the Regular Expression Tutorial and the Regular Expression page. In short this regex looks for pkgname='something' in $content and takes something and puts in in $pkgname. the ( ) in the / / tell which part of the regex to assign to $pkgname. [\w-]+ says to match 1 or more alphanumeric or _ or - characters. Regular expressions is a huge topic in and of itself. So don't spend a huge amount of time on it, they are important to know, but you don't have to master them right now.

I'd suggest having a copy of Regular Expression Pocket Reference: Regular Expressions for Perl, Ruby, PHP, Python, C, Java and .NET (Pocket Reference (O'Reilly)) and possibly Mastering Regular Expressions and if you want some common recipes Regular Expressions Cookbook. You should also warn your students that although you can use regular expressions to match any kind of text, that in the case of well defined formats, such as xml, html, csv, ini and more that there are better ways of extracting data. In the cases of things like email and phone numbers they should not write their own regex's but use modules to validate such things as they will undoubtedly make mistakes doing so, and a module has been well tested.

The example in Head First Programming: A Learner's Guide to Programming Using the Python Language goes on to explain how to check the page periodically for updates, and only output when a value is less than a certain amount. Since I have no idea when I'll be updating these PKGBUILDs I don't want to get into that code. It would make a good homework assignment. You can use sleep to make sure you only fetch the code ever so often. I can guarantee barring a buggy release I won't update any PKGBUILD more than once every 86400 seconds (1 day). You'll have to check pkgver and pkgrel to see if either of them are newer than the previously fetched PKGBUILD.

May 2, 2010

Teaching Perl - Week 3 - Prelude Commentary

So I think that Head First Programming: A Learner's Guide to Programming Using the Python Language is a great guideline to teach any programming course, esp a 10 week course as it has 10 chapters. We're one week behind. Arguably you could merge week 2 into week 1 and I may do just that in a rewrite of this series. In fact I'd suggest you do just that. But for now I'm going to continue on and we'll see what people think once I reach the end. No reason I can't rewrite this series, especially since it's not really planned I'm just writing it on the fly.

Although this book is a good guideline (note: I'm also concerned that if I follow it's examples to the letter all the time I might violate fair use), chapter 2 annoys me, it's on textual data and uses html scraping as an example. However, it completely ignores html parsers and any other sane way of doing it. So I had to find a better example and I think I have with parsing PKGBUILD's, which are sort of a microformat bash script.