SYNC_METHOD

I'm going to be pushing out an new feature to portage today, since testing has gone well.

SYNC_METHOD is a new portage variable, I've also gone to the trouble of defining it in make.globals. The default for Regen2. is git, however, rsync and cvs have also been defined although not tested.

the main benefit of SYNC_METHOD is to use one way of detecting the program you should use to detect the tree. previous variations were problematic, as rsync does not have a directory like .git and .git supports the rsync:// protocol.

I've added these defaults to make.globals

# Default sync method
SYNC_METHOD="git"

# Default sync mirror
SYNC="git://github.com/regen2/portage.git"

# default git method specific settings
PORTAGE_GIT_METHOD="checkout"
PORTAGE_GIT_REMOTE="origin/regen2.org"

you may of course override them with make.conf.

changes will officially be in portage-2.2_rc23-r3.

split package.mask

There's been a problem with ${PORTDIR}/profiles/package.mask since I added sunrise, I didn't realize at the time the problem wouldn't be adding atoms, it would be keeping track of when they should be removed. I've decided that the best way to do it is to use portages 'directory concatenation' feature, I turned package.mask into a directory, it now contains a file for each overlay that has a package.mask, and one for gentoo.org and regen2.org. This will make it much easier to keep track of removals. They originals are being merged in from a package.mask 'overlay' on regen2's github account. Let me know if anything is now masked that shouldn't be or unmasked that shouldn't be.

USE="experimental-git" sys-apps/portage

drobbins recent tip made me think, that maybe I should point out that his solution to the problem is an annoying work around. I had this problem for a while, it was caused by my having autocrlf = input in my .gitconfig settings. Several people have told me that they have this problem without changing their crlf settings for git. After a while I decided that Funtoo/Gentoo's git handling patches were incomplete at best and unfriendly at worst. So I wrote a patch that brings complete git handling to portage.

Why is it experimental? well it hasn't been officially accepted into portage, and as far as I know I'm the only one using it. One of the reason's it's not officially accepted is I haven't documented it.

WARNING: enabling it will cause auto git detection to fail, so if you don't finish setting it up portage will default to rsync.

So first you need to be using my regen2 tree, or get the ebuild and the patch from my tree, and enable the experimental-git flag for sys-apps/portage.

Now, like all the other protocols portage supports you need to set the SYNC variable in /etc/make.conf.

SYNC="git://github.com/regen2/portage.git"

the first 6 characters need to be git:// for git to work.

if you don't already have a git repository then sync will clone the repository at this url, in fact you may need to delete your portage and do a fresh clone for things to work in certain situations. I never discerned the reason for this.

now you need to set your method the most reliable, yet slowest is checkout.

PORTAGE_GIT_METHOD="checkout"

I believe this method will never experience the problem that drobbins was referring to.

for checkout you also need to set the remote branch you want to run off.

PORTAGE_GIT_REMOTE="origin/regen2.org" changing this variable and resyncing will allow you to change branches. checkout is even immune to upstreams forced updates, which is why I prefer to use it myself as sometimes I force updates on my development branch.

PORTAGE_GIT_LOCAL="regen2.org" is also a valid variable, however checkout doesn't use it. It's used by the methods rebase, which will run the git rebase command, this is recommended for people who don't run their own remote repo yet have there own custom git patches.

The final method available is merge, it works similar to the current pull behavior of portage, except it will actually merge from whatever remote you have specified into whatever local you have specified.

all methods use git remote update to avoid problems figuring out where to get updates from. I had problems with git knowing what to do with pull, when my origin was my personal repo on github and where I really needed to be updating from was funtoo not origin.

WARNING: git-experimental will probably be where I merge patches such as the upcoming manifest patch.

sync target support

Here I explain why emerge --sync should return to it's original sync form.

I think emerge sync should have the ability to have a target. Currently emerge --sync has a target of the SYNC variable in make.conf or the default hardcoded. However, git support is being added to portage. git is a very powerful tool, and the Funtoo project uses it to manage it's full portage tree. git doesn't work like rsync, it's not so straight forward as rsync uri, which is effectively what emerge sync is doing. git supports branches. The current implementation of git in portage effectively does git pull ... which is great... you know if you pull from the default.

I don't, and I'm not sure I'd always want to, it'd be great to be able to change tree's on the fly with a more distributed model. If I want to test funtoo without pulling it's tree into mine I could emerge sync funtoo funtoo.org, or I could switch to gentoo by doing an emerge sync rsync://rsync.gentoo.org/gentoo-portage and none of this will change the default or require configuring in make.conf.

Biggest reason to do this is branches though. what if I have a development branch and I need to test something... I, in theory should be able to change the branch quickly. with git I will have to cd to PORTDIR then run checkout. with portages current implementation I'll have to reconfigure some of git and then do a sync. However, if I were able to add a target to it I could easily do emerge sync origin development or emerge sync origin master.

The SYNC variable and maybe some GIT_BRANCH variable could still define defaults while adding a target would make development and testing easier for developers and testers respectively.

--sync should be sync

emerge sync never should have become emerge --sync, of course 2 years ago I thought it was the right thing, it isn't though, live and learn. sync is currently an argument with the syntax of an option.



What's the difference?

An argument is what the command, or executable operates on. Some examples

cp file0 file1 copies file0 to file1.

ls list the current working directory

ls dir0 lists directory dir0



Right, so how is --sync an argument and not an option?

Options effect how commands operate on arguments. For example

cp file0 file1 will overwrite file1 if it exists.

cp --interactive file0 file1 will prompt to overwrite if file1 exists.

ls -R will list any files and directories off the current working directory recursively.

ls -R dir0 will list any files and directories off dir0 recursively.



I still don't understand why sync isn't an option, it isn't a file so wouldn't it be an option?

Well that's basically what I used to think, not a file not an argument. It's not true to understand better I suggest picking up a version control system (e.g. svn) and seeing how you can have arguments that aren't files. Example

svn checkout -r 1346 checkout revision 1346

See the -r switch is an option that operates on the checkout argument to the svn command.



Starting to get it...

--sync currently doesn't affect any target, doesn't change a thing but isn't a 'default' either (some options are just there for explicitness and are in effect unless another option is there.

types of use flags

Ok so I have yet another idea for portage and other portage like package managers. actually this is probably more aimed at EAPI. Different types of USE flags. The 2 I'm thinking of off the top of my head could be described as volatile and non-volatile. Basically a use flag marked volatile means a package needs to be recompiled if it gets changed one marked non-volatile most likely just pulls in another dependency or copies files such as documentation. I'm sure this feature could have lots of uses I'm not thinking of.

Regen2 development has officially begun

That's right I'm putting my money where my mouth is. It's small but the first 2 packages when into the regen2 tree this morning. They are MySQL 5.1.30 which was added because the gentoo maintainer refused to add it do to instability. I agree that it is unstable, but we have ~M Hard Masked packages for that reason. It has been added as ~M. The other app added is qsynergy, the ebuild was already on portage and I know it to be good so I added it to the tree, I'll probably stabilize it in a few weeks.

Since Regen2 is a fork of Funtoo you can enjoy all the benefits of Funtoo's changes in my tree, and I will try to get mine accepted into Funtoo.

In the future I will be building stable Regen2 tarballs, although only stage3's.