SubmittingPatches is better than "Patches Welcome"

So I received my first patch for Perl code a week ago. Amazingly I didn't have to tell the person patches were welcome, they must have assumed so because my project was licensed under an OSI approved license. Also amazingly I've applied the patch even though it didn't meet my quality standards. I figure that's my fault though because no where did I document what those were.

Back when I forked Regen2 from Funtoo one of the first things I did (and a reason for the fork) was create a policy for patch submission. The policy for Funtoo was that all patches were welcome, and I recall 2 patches by 1 user specifically that should have been 10 patches and 2 (or more) of those 10 patches should have been dropped, because they were doing things they shouldn't have. In order to create this policy as easily as possible, I stole Git's SubmittingPatches policy and modified it to work with Regen2.

Once again I've been faced with patches that just aren't up to par. So once again I'm using Git's policy as a baseline for contributions to my projects. Here's the beginning of the documentation

Checklist (and a short version for the impatient):

Commits:

- make commits of logical units
- check for unnecessary whitespace with "git diff --check"
before committing
- do not check in commented out code or unneeded files
- the first line of the commit message should be a short
description and should skip the full stop
- the body should provide a meaningful commit message, which:
- uses the imperative, present tense: "change",
not "changed" or "changes".
- includes motivation for the change, and contrasts
its implementation with previous behaviour
- if you want your work included in the main repository, add a
"Signed-off-by: Your Name " line to the
commit message (or just use the option "-s" when
committing) to confirm that you agree to the Developer's
Certificate of Origin
- make sure that you have tests for the bug you are fixing
- make sure that the test suite passes after your commit

Patch:

- use "git format-patch -M" to create the patch
- do not PGP sign your patch
- be careful doing cut & paste, not to corrupt whitespaces.
- provide additional information (which is unsuitable for
the commit message) between the "---" and the diffstat
- if you change, add, or remove any features or
make some other user interface change, the associated
documentation should be updated as well.
- if your name is not writable in ASCII, make sure that
you send the patch in the correct encoding.

Long version:
...

The only part I added was to add documentation for features. Obviously some parts under patch need modification as I'm not using a mailing list.

Obviously part of the reason I don't like "Patches welcome" is I've seen that policy lead to the acceptance of really bad patches. Now obviously I make mistakes, everyone makes mistakes, but patches can be improved before they are committed. Even good patches can be flat out rejected for other reasons (I've had this happen. The consensus was not that the patch was bad, but the entire script need to be rewritten).

So what was wrong with this patch if I accepted it? well essentially it wasn't a git patch and it was applied to the wrong directory (because I had the original source in src/ and the generated a / I'm not using CommitBuild to another branch) and, it also broke my test suite (even though it was a patch for a test. The breakage was a bug in another test).

In any event, I believe that having your project be "Open Source" and having a patch submission policy does more to say "Patches Welcome" than "Patches Welcome". You don't have to agree with that, but regardless of whether you do, or not, I think having a policy is a good thing.

P.S. Trolling posts will be deleted. Be constructive or talk elsewhere.

Iron Man and introductions

So I've decided to "compete" in the Iron Man competition.

My name is Caleb Cushing, I'm a 25 year old, unemployed, student, living in the Greater Lansing, MI, USA, area. I've been programming perl for approximately 6 months. This includes a lot of off time, time to relearn the language from the class I took a couple years ago (and learn modern perl), learning Catalyst and DBIC, all of which is an ongoing process. I would guess I've been writing useful code for about 4 months and even that is a slow going process as I've still got a lot to learn.

Previously, I was the tree maintainer for Funtoo, and tried spinning my own fork of Funtoo/Gentoo, it needed more devs than me to survive though. I now use Arch Linux. After an arch/perl user who went by aCiD2 moved to OS X I adopted most of his perl AUR (arch user repository) packages in addition to ones I had created myself. I now have over 200 CPAN packages on AUR in Arch's PKGBUILD format.

I'm a contributer to Catalyst::Manual and I created Template::ShowStartStop by forking Template::Timer.

I'm sure I'll become more active as time goes on, and have more interesting posts than this one.

Arch, Funtoo, and Regen2's future

as I've been saying here recently I'm burned out, on so many angles it's not funny. I'm apathetic, I'm not really sure I care anymore. I'm actually playing with arch at the moment to see if it's a better fit. I also here debian sid might work. I don't want to maintain the tree 7 days a week anymore, and I'd like an up to date system.

I've been talking w/ drobbins lately about the possibility of a common tree with funtoo, I'm not sure he wants to do it the same way I do, which is unfortunate. doing this might take the load off me that I don't want, but it's entirely possible that it's too late for it to matter.

I'm going to take a few days and see what happens, it's entirely possible that I'm just going to give up at this point. I'll let people know in a few days what I've decided.

USE="experimental-git" sys-apps/portage

drobbins recent tip made me think, that maybe I should point out that his solution to the problem is an annoying work around. I had this problem for a while, it was caused by my having autocrlf = input in my .gitconfig settings. Several people have told me that they have this problem without changing their crlf settings for git. After a while I decided that Funtoo/Gentoo's git handling patches were incomplete at best and unfriendly at worst. So I wrote a patch that brings complete git handling to portage.

Why is it experimental? well it hasn't been officially accepted into portage, and as far as I know I'm the only one using it. One of the reason's it's not officially accepted is I haven't documented it.

WARNING: enabling it will cause auto git detection to fail, so if you don't finish setting it up portage will default to rsync.

So first you need to be using my regen2 tree, or get the ebuild and the patch from my tree, and enable the experimental-git flag for sys-apps/portage.

Now, like all the other protocols portage supports you need to set the SYNC variable in /etc/make.conf.

SYNC="git://github.com/regen2/portage.git"

the first 6 characters need to be git:// for git to work.

if you don't already have a git repository then sync will clone the repository at this url, in fact you may need to delete your portage and do a fresh clone for things to work in certain situations. I never discerned the reason for this.

now you need to set your method the most reliable, yet slowest is checkout.

PORTAGE_GIT_METHOD="checkout"

I believe this method will never experience the problem that drobbins was referring to.

for checkout you also need to set the remote branch you want to run off.

PORTAGE_GIT_REMOTE="origin/regen2.org" changing this variable and resyncing will allow you to change branches. checkout is even immune to upstreams forced updates, which is why I prefer to use it myself as sometimes I force updates on my development branch.

PORTAGE_GIT_LOCAL="regen2.org" is also a valid variable, however checkout doesn't use it. It's used by the methods rebase, which will run the git rebase command, this is recommended for people who don't run their own remote repo yet have there own custom git patches.

The final method available is merge, it works similar to the current pull behavior of portage, except it will actually merge from whatever remote you have specified into whatever local you have specified.

all methods use git remote update to avoid problems figuring out where to get updates from. I had problems with git knowing what to do with pull, when my origin was my personal repo on github and where I really needed to be updating from was funtoo not origin.

WARNING: git-experimental will probably be where I merge patches such as the upcoming manifest patch.

Banned from #funtoo

After significant time, consideration, and consultation with others in #funtoo, there is a consensus that the discussion in #funtoo should focus on Funtoo. Therefore, I have decided to ban you from #funtoo and encourage you to set up your own channel(s) for your personal efforts.
Please utilize Daniel Cordero (TheAppleMan) as your liaison for submitting patches and discussing technical issues.

I like how all of this was done behind the scenes... oh and I was never asked to leave. I just woke up this morning banned. I find it interesting that Gentoo and Sabayon are considered ok topics of discussion, and I probably wouldn't have gotten banned for saying use one of those. But Regen2 seems to be a sore spot. I would have left if asked... I find the ban unnecessary. Or maybe I would have just idled watching for bug reports, I've unfortunately ended up laughing at most of them, as there have been quite a few on qt, and I preemptively masked the new qt, knowing that upstream wouldn't have it all sorted yet. Anyways... I'd like to state that I personally disagree with all the private discussions that go on in Funtoo. In gentoo the problems are public and distracting, when I was part of Funtoo, they were made private, this does not appear to be different now. It's possible this was discussed in #funtoo-dev as I stopped hanging out in that room when I stopped being a funtoo-dev. I think in general it's a bit hypocritical. I suppose you could say, "you were asked to not talk about regen2 what did you expect?", I didn't actually expect not to get banned, I did expect to be asked to leave, or shut up, with the threat of being banned first.

Daniel Cordero (TheAppleMan) has offered to grab patches from the regen2-dev mailing list. no solution for overlays for funtoo was ever decided. I have left it up to them to figure it out, since I refuse to maintain both my tree and apply patches to theirs... (and remove them when they don't want them). I can't actually guarantee either my individual patches or the overlays will be merged.

I'll be continuing to import patches from funtoo, although recently I realized it will be most likely impossible to get them all as drobbins commits quite a few patches during his merges. I recently found out that udev had been updated and had some house cleaning done (there is no stable udev in funtoo now (I think I didn't actually check keywords)) all versions before 135 are gone. So I checked to see if I missed a patch... I didn't which means it was changed in a merge... those are the only commits affecting those directories.

Best of luck to Funtooers.

NOTE: ah... I apologize for the formatting issue in the blockquote, seems to be a problem with blogger. I've fixed it best I can.

open letter to Dan Robbins - merge regen2

I had an interesting conversation with thewtex on #funtoo yesterday. My conclusion is, this divergence of tree's is stupid. We should be working on a common base, at least common enough that users could relatively easily merge our tree's. My tree is currently ahead of yours in content by 3 sunrise merges, 1 mpd, and the 2 merges of the python overlay. Your users can't easily get these updates.

Due to the fact that we have irreconcilable views on what patches we allow, the fork must remain. However, users don't have to suffer for it. The solution is to base funtoo.org on regen2.org. I can't base regen2.org on funtoo.org because I have stricter QA policies, however those policies shouldn't negatively affect funtoo, I can continue to cherry-pick patches that you wish to put into funtoo, so they end up in regen2 as well.

This will require that you, trust me enough to merge the tree. I will be handling the gentoo and overlay merges. You will simply merge my tree daily.

In order to resolve the differences of the tree as they stand I suggest creating a funtoo.org2 branch that's based on regen2.org and discontinuing the maintenance of funtoo.org. cherry-pick any patches I haven't applied to regen2 (I can point out which ones) and apply any other changes you need to. The other option is to merge my tree, this will be painful and arduous for you, but require no work on the part of your users.

This resolves many problems. 1 you won't have to deal with the main tree or overlays, 2 you won't have to run your patches by me and can do what you want, 3 users get the best of both worlds, 4 developers can choose to email you inline patches, or send me pull requests. Everyone should go home happy from this.

what say you?

P.S. I will not have it said that I want a rift in the community, I want whats best for all kinds of users and the status quo is not.

EDIT: in short the answer is no. drobbins doesn't want to merge my tree, as it's supposedly untrusted, he also doesn't want to cherry-pick patches out of my tree. I don't want to spend time merging the overlays just for Funtoo, on top of merging them for Regen2. Apparently my bug fixes, and version bumps aren't good enough for Funtoo.

Forking Funtoo - Regen2 project born

So, I won't be the Tree Maintainer for Funtoo anymore. In light of the new situation I'm forking Funtoo. I'm not going into details why I'm not the tree maintainer anymore on this blog, except to point out how Regen2 will differ.

I'll be continuing to maintain a tree with all the overlays I had added and more to come. you can get a copy of my version of portage at git://github.com/regen2/portage.git my private repo is where I test updates first. I'll be maintaining a gentoo.org and funtoo.org branch as well as my own regen2.org branch. The regen2.org branch doesn't currently differ much from funtoo.org, however I intend to keep it up daily and at some point I may even be able to update it more than once a day.

I've already got mailing lists and I'm working on having permanent channels on IRC. lists are on google groups at regen2-user and regen2-dev I don't mind discussions of gentoo, funtoo, and sabayon in addition to regen2 on these lists.

Regen2 unlike Funtoo is not about 'fun' it's about making Regen2 the easiest, fastest, most secure, most configurable, most stable and bleeding edge source distro, and probably a few other things, too.

funtoo wants as little red tape for developers as possible, to the point of zero red tape and zero accountability on whether things break. If I had continued with funtoo it would have been my job to fix patches that came in with problems on simple things like manifests. It's assumed it takes 1 minute to fix manifest or less when in reality it takes about 10. 2 minutes to catch the error, another 5 to regenerate the manifest make a new commit, and another 2 to test and 1 more to push. This may or may not be a time exaggeration. Now imagine I have to generate every manifest on every ebuild that comes in. This takes time, for me it's easier to ask the person who's submitting the patch to fix there patch. When I sent a patch to git and got a 1 page reaming response about everything I did wrong submitting it, I didn't complain, cry, or quit, I licked my wounds fixed my problems and my patch and resent. I had implemented some red tape to prevent simple broken commits and make better commit logs, these added very little overhead to the actual patch process, and some of them had actually caught what would have been otherwise missed bad patches going into the tree. I will continue checking commits coming into the regen2.org tree.

It has been brought to my attention, that it appears like I'm trying to create a rift between funtoo and regen2. Not at all, I will be sending drobbins bugfix patches, and I will be cherry-picking from funtoo.org. I will not be maintaining overlays, or the tree for him however, this is impractical via email patches, I've noted that he's welcome to pull from me. My primary concern will be the regen2.org tree, however. I'd also like to note that the reason I'm maintaining my own tree is the same reason I agreed to be tree maintainer in the first place. At the time I started tree maintainership it had been 5 days since the tree had been merged. This is too long in my opinion and if I don't maintain my own tree it could happen again. Barring extenuating circumstances the tree will be merged daily.

I'd also like to note that the original vision of funtoo was in part, a distro to build other distro's. So I'm merely fulfilling that vision by starting this.

Hopefully with in a couple of weeks I'll start building stage3's on the funtoo tree, and eventually iso's and stage4's as well. I most likely will not have processor specific or stage1's and stage2's, due to my lack of hardware and the fact that I think stage1/2 is rather pointless.

I will be adding emerge-ng to funtoo, which at first will just be a portage (and some other tools) wrapper, however, as time goes on I will be replacing portage with new code to make it faster. emerge-ng should also be a bit simpler to use and it'll have a better name. More on it to come. For my long term visions of it see the emerge-ng tag.

regen2.org will be the official home, I hope to have something operational there by the end of the weekend.

I hope you all enjoy regen2.

EDIT:

I realize this may not be the most professional, gracious, or diplomatic of partings, and beginnings. I'm not a diplomat, although, perhaps I shouldn't have said this as I did... perhaps it doesn't matter... or perhaps I said what needed to be said. I've nothing to hide, and anyone is welcome to call me out.

Funtoo News

I've decided I'm going to start posting a sort of weekly newsletter (I'm not committing to weekly though) merely to document anything I feel important that happened, or is about to, but isn't quite important enough for it's own post.

RA (ra-- on #funtoo) or r_a@lavabit.com submitted quite a few patches this week including 1 that fixes many of my major borkage on non-latin1 changelog breaks. He's become a regular contributor and I'd like to thank him.

perl-5.10 will be unmasked this evening, if you haven't read it yet, read my post on upgrading, you'll need it, because this is more than just an install and forget, other packages need to be rebuilt, and perl-cleaner revdep-rebuild doesn't find them all. Good news is I've been running 5.10 since the beginning of the month no problems since the beginning.

I pushed samba 3.2.7 into the tree this week, gentoo's been sitting on it for a while, it's hard masked, but seems to build fine, I don't operate on a windows network though so can't say how well it works.

mpd overlay

I'm adding the MPD overlay courtesy of Rullzer who's kind enough to merge and maintain it in tree. Nothing big here, just some more up to date ebuilds, and more clients for MPD.

Funtoo gets the new Perl

Soon, Dan Robbins will be merging my perl-experimental branch which contains the perl-experimental overlay. This overlay contains a handful of packages, the most important of which is perl-5.10.0 which has been out for about a year now and Gentoo still hasn't put in portage. One nuisance of the perl-experimental overlay is they don't use changelogs which means if you want to know what's going on with a package from this you'll have to check out git log as well as the changelog. I feel this is a minor issue, in the long run we should migrate all package logs to git's internal logging facility.


I've been testing perl-5.10.0 for a week or so now with few problems, and none which I've found to be un-resolvable. It's entirely possible that there are packages I don't use that will break on 5.10. When we merge perl-experimental it will be hardmasked at first, and we're looking for people to test for a couple of weeks before we move it to the testing branch, so we can iron out any problems which we weren't able to account for.

upgrading to perl-5.10

So, in theory, all you have to do is run

perl-cleaner all

after unmasking and emerging dev-lang/perl and sys-devel/libperl. In practice however, I found this didn't work. it seemed to only find packages in world which needed upgrading, so between me and one other person, we came up with this little one liner to fix packages were built against perl-5.8

locate 5.8.8 | grep ^/usr/ | grep -v ^/usr/portage | xargs equery belongs | uniq | sed -e s/^/\=/ | xargs emerge --oneshot

once you've run perl-cleaner and the one-liner, you should be completely upgraded.

Issues specific to perl-5.10 can be reported on the Gentoo bugtracker, in the perl 5.10 bug or to us on #funtoo.

Sunrise's over Funtoo

Sunrise a Gentoo overlay is being added to Funtoo. So if you've got layman tracking sunrise and you're using funtoo you'll be able to delete sunrise from your overlays as I'll be merging sunrise's reviewed ebuilds daily.

The hardest part of the whole process was merging package.mask, use.local.desc and categories. Since this is new be on the lookout for bugs or problems.

Want to add an overlay to funtoo yourself? First, it'll have to be a git or svn overlay. If it's git you can skip steps involving git svn.

git svn clone uri://mysvnuri.tld

this imports an entire svn repository into git and will take a while, sunrise had import over 7000 commits. Next check to see if any changes need to be made to the structure of the overlay, in sunrise's case it's not branched properly... they had a subdirectory structure including reviewed and non-reviewed ebuilds, I had to remove that so the top level directory was that of a portdir for merging. if it's already in a nice portdir format move to the next step.

start by cd-ing into your funtoo portdir checkout, then run git pull uri://mygit-svn-cloned-repo, this will attempt to merge the 2 and most likely you will have conflicts. Resolve ebuild, and changelog conflicts first, these are easiest with git mergetool, imho, but don't try to use that to merge manifests. Instead of merging manifests just regenerate them with ebuild ebuildWithScrewedManifest digest. once you've resolved all conflicts run git commit. That's it your done. you've now merged an overlay into your local funtoo tree. If you'd like to see it in funtoo you'll have to talk to Dan Robbins. He's on #funtoo on freenode regularly.

Enjoy!

Regen2 development has officially begun

That's right I'm putting my money where my mouth is. It's small but the first 2 packages when into the regen2 tree this morning. They are MySQL 5.1.30 which was added because the gentoo maintainer refused to add it do to instability. I agree that it is unstable, but we have ~M Hard Masked packages for that reason. It has been added as ~M. The other app added is qsynergy, the ebuild was already on portage and I know it to be good so I added it to the tree, I'll probably stabilize it in a few weeks.

Since Regen2 is a fork of Funtoo you can enjoy all the benefits of Funtoo's changes in my tree, and I will try to get mine accepted into Funtoo.

In the future I will be building stable Regen2 tarballs, although only stage3's.