tag builds

This is just a feature concept. May or may not ever happen. Live builds for portage usually run off the trunk/master branch of a scm tool. This is a great idea, but why couldn't we take this a step further, usually people who use scm's 'tag' there releases, so why couldn't the same 'live' build have an option for using a tag instead of trunk. This would allow releases to roll out much faster too. The downside of this is hard drive space for keeping repository checkouts.

Forking Funtoo - Regen2 project born

So, I won't be the Tree Maintainer for Funtoo anymore. In light of the new situation I'm forking Funtoo. I'm not going into details why I'm not the tree maintainer anymore on this blog, except to point out how Regen2 will differ.

I'll be continuing to maintain a tree with all the overlays I had added and more to come. you can get a copy of my version of portage at git://github.com/regen2/portage.git my private repo is where I test updates first. I'll be maintaining a gentoo.org and funtoo.org branch as well as my own regen2.org branch. The regen2.org branch doesn't currently differ much from funtoo.org, however I intend to keep it up daily and at some point I may even be able to update it more than once a day.

I've already got mailing lists and I'm working on having permanent channels on IRC. lists are on google groups at regen2-user and regen2-dev I don't mind discussions of gentoo, funtoo, and sabayon in addition to regen2 on these lists.

Regen2 unlike Funtoo is not about 'fun' it's about making Regen2 the easiest, fastest, most secure, most configurable, most stable and bleeding edge source distro, and probably a few other things, too.

funtoo wants as little red tape for developers as possible, to the point of zero red tape and zero accountability on whether things break. If I had continued with funtoo it would have been my job to fix patches that came in with problems on simple things like manifests. It's assumed it takes 1 minute to fix manifest or less when in reality it takes about 10. 2 minutes to catch the error, another 5 to regenerate the manifest make a new commit, and another 2 to test and 1 more to push. This may or may not be a time exaggeration. Now imagine I have to generate every manifest on every ebuild that comes in. This takes time, for me it's easier to ask the person who's submitting the patch to fix there patch. When I sent a patch to git and got a 1 page reaming response about everything I did wrong submitting it, I didn't complain, cry, or quit, I licked my wounds fixed my problems and my patch and resent. I had implemented some red tape to prevent simple broken commits and make better commit logs, these added very little overhead to the actual patch process, and some of them had actually caught what would have been otherwise missed bad patches going into the tree. I will continue checking commits coming into the regen2.org tree.

It has been brought to my attention, that it appears like I'm trying to create a rift between funtoo and regen2. Not at all, I will be sending drobbins bugfix patches, and I will be cherry-picking from funtoo.org. I will not be maintaining overlays, or the tree for him however, this is impractical via email patches, I've noted that he's welcome to pull from me. My primary concern will be the regen2.org tree, however. I'd also like to note that the reason I'm maintaining my own tree is the same reason I agreed to be tree maintainer in the first place. At the time I started tree maintainership it had been 5 days since the tree had been merged. This is too long in my opinion and if I don't maintain my own tree it could happen again. Barring extenuating circumstances the tree will be merged daily.

I'd also like to note that the original vision of funtoo was in part, a distro to build other distro's. So I'm merely fulfilling that vision by starting this.

Hopefully with in a couple of weeks I'll start building stage3's on the funtoo tree, and eventually iso's and stage4's as well. I most likely will not have processor specific or stage1's and stage2's, due to my lack of hardware and the fact that I think stage1/2 is rather pointless.

I will be adding emerge-ng to funtoo, which at first will just be a portage (and some other tools) wrapper, however, as time goes on I will be replacing portage with new code to make it faster. emerge-ng should also be a bit simpler to use and it'll have a better name. More on it to come. For my long term visions of it see the emerge-ng tag.

regen2.org will be the official home, I hope to have something operational there by the end of the weekend.

I hope you all enjoy regen2.

EDIT:

I realize this may not be the most professional, gracious, or diplomatic of partings, and beginnings. I'm not a diplomat, although, perhaps I shouldn't have said this as I did... perhaps it doesn't matter... or perhaps I said what needed to be said. I've nothing to hide, and anyone is welcome to call me out.

sync target support

Here I explain why emerge --sync should return to it's original sync form.

I think emerge sync should have the ability to have a target. Currently emerge --sync has a target of the SYNC variable in make.conf or the default hardcoded. However, git support is being added to portage. git is a very powerful tool, and the Funtoo project uses it to manage it's full portage tree. git doesn't work like rsync, it's not so straight forward as rsync uri, which is effectively what emerge sync is doing. git supports branches. The current implementation of git in portage effectively does git pull ... which is great... you know if you pull from the default.

I don't, and I'm not sure I'd always want to, it'd be great to be able to change tree's on the fly with a more distributed model. If I want to test funtoo without pulling it's tree into mine I could emerge sync funtoo funtoo.org, or I could switch to gentoo by doing an emerge sync rsync://rsync.gentoo.org/gentoo-portage and none of this will change the default or require configuring in make.conf.

Biggest reason to do this is branches though. what if I have a development branch and I need to test something... I, in theory should be able to change the branch quickly. with git I will have to cd to PORTDIR then run checkout. with portages current implementation I'll have to reconfigure some of git and then do a sync. However, if I were able to add a target to it I could easily do emerge sync origin development or emerge sync origin master.

The SYNC variable and maybe some GIT_BRANCH variable could still define defaults while adding a target would make development and testing easier for developers and testers respectively.

types of use flags

Ok so I have yet another idea for portage and other portage like package managers. actually this is probably more aimed at EAPI. Different types of USE flags. The 2 I'm thinking of off the top of my head could be described as volatile and non-volatile. Basically a use flag marked volatile means a package needs to be recompiled if it gets changed one marked non-volatile most likely just pulls in another dependency or copies files such as documentation. I'm sure this feature could have lots of uses I'm not thinking of.

accept input from stdin

if possible emerge-ng should be able to accept input from .

e.g. cat packagelist | emergeng

thus far neither portage nore pkgcore have this capability.

minimum permissions and privilegdges

I like security... which means I should be able to run portage as portage (user) and have the umask be 077, or perhaps 027. Unfortunately the last time I checked portage could handle these restrictive permissions (I forget if it was these exactly) except for one set... java. In a perfect ebuild world all ebuilds would be able to be installed under a very restrictive umask.

Should regen2 ever come to fruition this should be fixed.

pkgcore the victor

pkgcore is my pick for the next portage and codebase for emerge-ng. I admit I haven't let paludis have a chance (yet).

Why haven't I let paludis have it's chance?

The reasons are quite simple. Everytime I have tried paludis prior to this it has required extra configuration (meaning it doesn't work out of the box). I believe this was fixed recently. However,

paludis@1208029212: [WARNING] Use of Portage configuration files will lead to sub-optimal performance and loss of functionality. Full support for Portage configuration formats is not guaranteed; issues should be reported via trac. You are strongly encouraged to migrate to a Paludis configuration.
paludis@1208029212: [WARNING] Use of Portage configuration files will lead to sub-optimal performance and loss of functionality. Full support for Portage configuration formats is not guaranteed; issues should be reported via trac. You are strongly encouraged to migrate to a Paludis configuration.
* Done cleaning write cache for ebuild format repositories
paludis@1208029212: [WARNING] Use of Portage configuration files will lead to sub-optimal performance and loss of functionality. Full support for Portage configuration formats is not guaranteed; issues should be reported via trac. You are strongly encouraged to migrate to a Paludis configuration.

This is the end of a paludis --sync. Annoying. Paludis also has it's alpha releases marked as ~arch, this is a no, no. I also found the people in #paludis on irc to be rude and unhelpful. I was annoyed at the time (so being rude myself) as gentoo kde-svn overlay was going to require paludis which means I was going to be forced to use it. Regardless I've never been met with lot's of friendliness or help in #paludis. They are an RTFM crowd. I have been told to use paludis several times without even asking about it (as if it is the solution to all problems). So is pkgcore better? I can't say without a doubt that it is.

Why is pkgcore the victor?

For starter's it's only the victor for me. paludis is a more mature product at this point and has a stronger following of fanatic's/zealot's.

1.) pkgcore it more similar to portage.

pkgcore was originally intended to be portage 3. This leads to a large amount of similarities and backwards compatibility.

2.) pkgcore devs are really friendly. Sometimes patience is a virtue getting an answer on #pkgcore but most of the times I get one. This allowed me to quickly learn what I could and could not (yet) do with pkgcore, including finding a bug within a week.

3.) pkgcore just works. I installed pkgcore and with a quick look at the site was quickly able to start doing work with it.

4.) pkgcore installs faster. I think it took like an hour to build paludis on this machine (maybe more) pkgcore took 2 minutes tops.

5.) pkgcore uses C. For the parts of portage that were really slow they re-wrote them in C using cpython. using both python and c leads to faster development and speed where needed.

How do I use pkgcore
real quick

pmaint sync

this is the biggest adaption from portage, imho and I'm not sure why it's not --sync or pmerge --sync (in some ways it's more similar to portage 2.0 syntax). This will not only sync your main gentoo tree but any repositories you've intalled with layman. as of yet you still have to install them via layman, but this may change in the future.

pmerge packagename

I told you this was easy. Most of the options I use work. The biggest that doesn't is --verbose, this seems to be default now (with --ask) so an option is no longer needed. updating world

pmerge -auDs world

an -s is now needed to work with things like system and world. It's for package set. My one complaint here is the --newuse doesn't currently work with sets. This is a bug and will be fixed in the future.

Is anything wrong with pkgcore in your eyes?

well aside from the plethora of missing features (they'll get done, it's just a time thing). Documentation... I usually learn stuff by reading web documentation (or books) until I'm comfortable enough reading man pages. The web documentation for pkgcore needs a lot of work. I may actually volunteer to help with this. I haven't decided yet, and it depends partially on if they want to answer all my questions as I write the docs.

It'd be nice if at some point pmerge could become emerge etc... (meaning reprise it's roll as portage 3).

for more information on pkgcore go to pkgcore.org

Graphical Package Manager

Anyone that knows how I work in linux, knows that I"m comfortable with the cli, and prefer it to a gui. I know that not everyone is or should have to be. Sometimes a gui can make management easier. Even a die hard cli person like me realizes that. However, you should never have to do things the gui way, they are just front ends. emerge-ng needs to support having a gui even if the development of such a project is not directly part of the emerge-ng project.

buildpkg

emerge-ng should be able to build rpms, and debs. regen2 should be the distro that builds other distro's, not only should it be good at that (gentoo is now) it should be designed for it.

kde-overlay (svn) now requires paludis

details are here

so apparently the folks running the kde overlay have decided to discontinue portage compatibility. I suppose I can't blame them, on the other hand I don't approve. Even though I haven't started testing paludis yet. I am still firmly against it as it is as gentoo's next package manager. I won't be regen2's without heavy modification. Including some re-writes. I do intend to investigate the possibility of using paludis code in emerge-ng.

emerge-ng multiple trees

first I hate repositories, but it is unlikely they will go away. So we must strive to make sure that they are only needed for truly rare things. from my perspective the sunrise repo shouldn't exist. and vcs builds should be in the tree. regen2 should strive to have everything possible in the tree. when it can't an overlay should have all the capabilities of the main tree. meaning that an overlay shouldn't automagically be considered less stable and require the keywording of all the ebuilds in it. also overlay's should be search-able regardless of whether they are installed to expedite the finding of apps.

emerge-ng and vcs builds

gentoo is a source based distribution, and this is a good thing. why then has the ability to have trunk builds (which has been added to many overlay's) seem like such a hack? all non-binary program ebuilds in the tree should have a 'trunk' version option. emerge --sync should then be able to find out whether trunk has been updated so you can correctly rebuild programs as needed with emerge --ask --verbose --update --deep --newuse world

what is regen2 linux?

In short a concept aimed at forking Gentoo Linux. First some history and my own experiences.

Gentoo is a sick adolescent who refuses to acknowledge his illness. Because he refuses to acknowledge the illness he can't be helped. There are many who have ideas about how to help him. Some of those are wrong.

Gentoo's biggest problem? Democracy. Making it actually work is not easy. Most of the really successful, open source, projects have one or two people at the top. A good benevolent leader is needed, imho. Gentoo's developer council did not seem to have things under control for the better part of 2007. Some things seem to have improved but I have lost faith in them.

Dan Robbins (gentoo's creator an prior benevolent dictator) made 2 offers to help, but left quickly after both. It is obvious that he can't/won't (shouldn't?) attempt to save Gentoo.

Political is Gentoo's biggest thorn and as a result it is suffering technically.

Portage (gentoo's ports like package manager) is said to be unmaintainable, buggy, and un-fixable. I believe there are 3 package managers being written separately, with the idea of being alternatives. From what I've seen of Paludis and Pkg-core neither of them have the right idea for an actual replacement. I will go into that in another post.

Gentoo's dev's have no respect for standards or conventions, and as such certain area's which could be easily fixed have suffered. such as Gentoo's Apache binary being named apache and apache2 instead of httpd which is mostly a convention, or the lack of an /srv which is not optional according to FHS.

Gentoo lacks a sense of business, to the point the foundations charter was revoked (or was it almost revoked, plenty of confusion to be sure). If support rallies behind me (or the idea) I fully intend on incorporating for the purpose of commercially supporting the distribution.

In short, I hope ReGen2 will be a commercial funded, rapid developing, fast, versatile, source based linux distribution. Which will continue to be a leader in the open source world.

emerge-ng

at the time of this writing. a concept for software with a name that may not be it's final, and it may never get done.

emerge-ng should ultimately be a drop in replacement for the portage package (the main package manager) in gentoo linux.

for clarity I will be referring to portage as the software and if I mean to refer to portage's tree (basically one giant software repository), I will refer to it that way.

I know of at least 3 other projects to replace the outdated barely maintainable portage. I think all of them are going about it wrong. They are developing different package manager's instead of developing a better version of emerge. They are right that portage needs to be rebuilt from ground up, but that doesn't mean that the interface the user sees is bad. if vim had not tried to be mostly a clone of vi would it have been as successful? so successful that is it more popular than what it cloned? if gnu coreutils weren't posix compliant and didn't mostly match the original unix commands would we be using them? probably not.

I will be writing posts here to detail features that emerge-ng can benefit from.