Jun 5, 2011

My solution to not using PluginBundle:AUTHOR for dzil is git

I am now tired of updating my dist.ini's for my Dist::Zilla projects. For many people this is when they start using a PluginBundle with their authorname. I discussed why you shouldn't do this a year ago. Now that I'm tired of managing my dist.ini's on an individual basis I'm going to show you how I'm going to solve the same problem everyone else is, which is getting tired of updating your dzil configuration for all of your projects. I'm using git to do it. You probably haven't considered this, or am thinking I'm wrong because you believe that git can't merge branches without a common history, which is not true. I did it with Regen2, Funtoo, Portage, and Sunrise, which have way more files than any perl repo, including Perl.


First let's talk about the advantages and disadvantages of doing this with git.

I'll start with the disadvantages. You'll have another repository to manage. Git isn't completely automatic, you'll have to remember to merge your changes. You'll also have to add the repository to your existing repositories. You will have to resolve merge conflicts at least once, and probably occasionally more, though most should be fast-forwards.

The advantages are... now the changes to your dist.ini are getting recorded in your history. You can now have a master dist.ini, but remove items without filters in your individual modules. You can share even more configuration as merged differences allow you to maintain differences in downstream commits. It doesn't automatically update all of your modules. Wait didn't I list that last one in disadvantages? Yep, it's an advantage because what if you update your PluginBundle and that update breaks one of your modules, but you don't know it because you haven't worked on that module in a while. You can use this for more than just dist.ini.

Ok so the first thing you you want to do is create your master dzil repo. This is not a git or dzil tutorial so go do that. I did it by creating a dzil new project and removing the files I couldn't use and making a few tweaks and amending all of those changes to the initial commit. Check out my repo for inspiration. Remember the directory structure has to be the same as a dzil repo structure for any common files.

Now add a remote from your new repo to an existing dzil project. Next you need to merge the branch from the remote into your project. The first time you do this you'll have to resolve conflicts. If they're like mine then they'll be easy and obvious. Once that's done future changes will be fast-forwards unless you change something in your perl module repo, then you might end up with a simple merge conflict. One thing to remember NEVER EVER try to merge from your module repo to your dzil repo, it will cause you extreme pain in the future. cherry-picking that direction is possible but not merging.

After that, all that is left to do is run your dzil tests and fix any breakage in your module. Happy merging.

Regen2 accounts transferred

I killed regen2 about 2 years ago. Today I was contacted by Rafal Kupiec asking about the project. Since it's dead he asked if he could take over the project. I have transfered the github account, and freenode channel #regen2 to him. The google groups are remaining archived. At some point in the future there should be a site at asiotec.org I have no idea what he plans on doing with it.

Jun 1, 2011

Test::Version under maintenance again

About 4 months ago russoz (Alexei Znamensky) sent me a message about quoting the versions generated by Dist::Zilla::Plugin::OurPkgVersion. I was a bit skeptical at first about needing it, and the suggestion seemed to have more to do with style anyways. But after digging a little I found to my horror that OurPkgVersion was generating completely invalid multipart version strings, e.g:

our VERSION = 0.1.0;

Which doesn't work. I was shocked because I was using Test::HasVersion. Well all Test::HasVersion does it checks to see that you have a VERSION defined. My code above will pass a defined test, but you certainly won't get anything if you try to print it.

So I accepted Russoz's patch and fixed up my documentation. Still, I was disturbed that there weren't any Test modules that would have caught this error for me.

So, eventually, I began working on writing a Test module to do this task for me. First it was mostly research and learning which modules and methods I would need. One of the hardest parts was determining how best to extract the VERSION from the module. PPI seemed like an impossible task, probably most due to the fact that I don't understand it. I thought about require-ing modules, but many told me that could lead to a vulnerability in the test. Ultimately decided to use Module::Extract::VERSION which will work for everything except perl 5.12 style versions. Then I had the problem of learning how to write Tests for Test modules, which ultimately doherty (Mike Doherty) contributed some patches to get my first set of tests working.

Once the module was all but finished functionality wise I had to decide on the name, after discussing it on #distzilla it was decided that it might be a good idea to take over an existing module. Test::Version seemed like the most obvious candidate because it has the most obvious namespace for what is trying to be done. So I spent the next few weeks trying to contact particle (Jerry Gay) to get maintainer-ship. Once I did manage to contact him it wasn't an issue to get control of the Module.

So I released a new version of the old Test::Version 0.03 announcing new maintainership, on the off chance anyone was using it in the darkpan, since I'd already searched the CPAN archives for any usage and no current module was. I then released a TRIAL version for testing. The Trial version ended up having several failing tests by cpan tester reports, I thought it was caused by Test::Builder::Tester whitespace sensitivity (and maybe some of it was). rjbs (Ricardo Signes) suggested Test::Tester and I released yet another version with failing tests. This time I analyzed about 5 of the test reports, and come to find out the module seems to run the tests on the perl modules in no particular order. I assume that somewhere there is some speed optimization which results in this behavior. So I had to find a way to write the tests which could handle the fact that I couldn't depend on the order that files were being tested. Ultimately worked that kink out as well in a way that appears to be satisfactory.

So with version 0.07 we now appear to have a working Test::Version. That checks to see if modules have versions, and that they are valid. It's also capable of checking an entire dist. The docs describe use, and if you use Dist::Zilla you can check out Dist::Zilla::Plugin::Test::Version or use Dist::Zilla::Plugin::TestingMania which will be including it.

Going forward I plan to take version 0.07 if it turns out to have no serious bugs, and release version 1.0.0 as it stands. After that I have a few features releases in mind. Including supporting is_strict if you'd like to make sure that your VERSION passes those tests as well. I'd also like to pull in the functionality of Test::GreaterVersion and Test::ConsistentVersion. I'm open to other suggestions as well.

I hope people can make some good use out of this module to avoid silly mistakes like I made.

Happy Testing!

May 31, 2011

New beginnings

Since I truly last posted a lot has changed. I moved to Houston, TX, USA (from MI) for a position as a Linux System Administrator. As of this week I've been moved to the programming department in my company, coding Perl. I'm also an elected moderator on StackExchange's Unix & Linux. Of course there's always updates to my CPAN dists.

Apr 29, 2011

Adding and Deleting subdomains with Plesk on the Command Line

Plesk has the disadvantage that everything is done through the
database. So we can't just modify Bind's Zone files.

To add an A record of a subdomain you have to do the following:

/usr/local/psa/bin/dns -a domain.tld -a subdomain -ip 127.0.0.1

the first -a is for add to zone which is why it must specify the top
level domen, the second is for A record, and must not contain the full
domain only the subdomain. If you use the full domain
subdomain.domain.tld you will end up with a record like
subdomain.domain.tld.domain.tld because plesk does not allow you to
terminate with a .. Thank you plesk for a shitty interface. The record
this create's looks like

subdomain.domain.tld. IN A 127.0.0.1

To delete this record you would use the following

/usr/local/psa/bin/dns -d domain.tld -a subdomain -ip 127.0.0.1

you'll note that it's the exact same thing except you replace the
first -a which is for add, with -d for delete.

Nov 10, 2010

Writing a simple Dist::Zilla::Tester test

Hopefully, someone will use the blog post to write an actual doc patch, seeing as how this is undocumented.

I finally wrote A test for DZP::OurPkgVersion with the help of CJM. So I figure it's best to share the knowledge imparted upon me to all those who are writing plugins without tests.

Before we get started I'm going to advise that this test will only check the output that dzil built, if you need it to test anything more sophisticated, you'll have to learn more.

First you'll want to create a corpus repo like /corpus/MyDZTRepo with a basic minimal repo. This repo is simply a repo that you are using to test your plugin against, to make sure it works right. You put it in corpus so that if you have tests that you have to check in your corpus, those tests themselves aren't run when the test suite is run. The dist.ini doesn't need to contain anymore than the basic stuff needed to build. You .pm files need not have anymore data than what you're going to need to make your dzil plugin do its job. In the case of DZP::OurPkgVersion I only needed to test that the output found the # VERSION string correctly in a couple of scenario's. So that meant having # VERSION in the .pm's and [OurPkgVersion] in the dist.ini.






First DZT (Dist::Zilla::Tester) doesn't provide any tests of its own so you still need to use Test::More or some other testing framework. Next you need to initialize the tester object by telling it where the root of your corpus repo is. After that, unless you need to do other work, you can run $tzil->build so that the build is run.

So now lets slurp a file into memory so we can check to see if it was built right. You'll want to look in 'build/*' as the basic root of the build directory. So 'build/t/test1.t' if you need to slurp a test.



Now that we've pulled our build files into memory lets code up what the result should be. We can just do this with a simple heredoc, obviously you can do it another way.



Now that we've gotten that, all we have to do is compare the file that we expect dzil to output and the file that dzil actually built. This is just standard Test::More



Now let's take a look at it all together.


Pretty simple huh? Hope this means more dzil modules getting tested now. Including more of mine.

Nov 3, 2010

Making Secure Recoverable Passwords ( part 3 )

If you haven't read Part 1 please do so.

Although the criticism of Part 2 should be taken with a grain of XKCD Salt, and even the primary plaintiff admitted that it would take him 2 months to crack the final salted sha512. Usually your attacker shouldn't have your shadow file, and having stored them as anything less is just plain not secure. I will acknowledge it has some merit.

Doing a base64 transformation on hex only digits is a bad, idea, and does not have nearly enough possible combinations. Nothing is going to be more secure than random, but random isn't really recoverable, if you lose it. That's why I do some kind of transformation.

I believe that somewhere someone suggested that it would be better to convert from the binary digest into base64 as it would be more random than from hex. I believe this is accurate, but the method suggested was in Perl, which is kind of messy, and more importantly hard to remember. So I asked, on unix.stackexchange.com, how I could do this on the command line. Here's the answer I decided to accept:

echo -n `date` | openssl dgst -binary -sha512 | base64

Remember you should slightly modify the result in a way that you can remember in your head to make it random, and probably use something in place of the "date" command, since it's not reproduce-able.

Again: this is not meant to be as secure as random passwords, just secure enough compared to non random alternatives.

Sep 2, 2010

Ubuntu isn't Linux, What's a Beginner anyways?

Today Joel Spolsky announced The results of the poll of whether ubuntu SE and unix SE would be merged, they won't. In short the Unix community was in favor of it but the Ubuntu community was not. Joel had this to say:
So, Ubuntu, Linux, I get it, it’s clearly not the same thing. If you love Ubuntu, we have a site for you. If you love Linux and Unix, we have a site for you. A Stack Exchange can’t work without a community that loves a subject, and love is very… specific. Fighting human nature is hard: the factionalism and fork-happiness of the Unix world has been a hallmark of that community ever since BSD vs. System V, and Stack Exchange can no more bridge that gap than we can unite the Judean resistance.

I am horribly disappointed in this decision, this factionalism doesn't help the cause, this set's precedence that it's ok in the StackExchange community to be a separatist. Arch Linux's bash must not be the same as Ubuntu's or Fedora's, and of course they've made so many changes to upstream Gnome as to be unrecognizable as such.

The main argument in favor of this separatist movement is that Ubuntu is for beginners, if that's so then why isn't it beginner.unix SE? What's a beginner anyways?

I'd dare wager that 99.999% (POOMA) of all Linux users are windows Power users, veterans, and refugees. So what's a beginner? someone that doesn't know what Firefox is? No they probably do. Maybe they don't know where it is in the menu? I can see that... but I don't see how that can't be answered with a Screenshot and a Ubuntu tag on Unix SE. Maybe it's someone who doesn't know how to use the CLI? Well where do you define beginner from advanced? I realize someone reading there first shell programming Tut/Book/etc is a beginner, but are they no longer a beginner after that? how do you know you're not a beginner anymore? When you choose to move beyond Ubuntu? If I've been using Linux for 5 years and I choose to move to FreeBSD am I a Beginner?

Why is that the answer to a beginner from an advanced user isn't welcome? Is it good that beginner are exclusively answering other beginner questions?

In other news it looks like I may become a moderator ( temporary at least ) for unix SE.

Aug 6, 2010

I need a new favorite OSI license or Why I don't like the GPL anymore

I don't like the GPL anymore after the WordPress Theme fiasco. I don't like the idea that a theme, plugin, addon, extension, whatever is 'derived' from the thing it's adding on to and thus must also be GPL.

If I build a deck onto the back of my house is that deck derived from my house? No! if I take the blueprints of my house and build another one that's 80% like it is that house derived from my house? yes. If I offer a painting service to paint houses and I offer people preexisting color schemes are those schemes derived from their house? should the manufacturer/contractor (whatever) be able to tell me I can't do that, or that I have to share my colorschemes (and how I mix my paint) with him? How STUPID, IMO. But that's what the GPL does, you can't help other people addon to their house without sharing it with the house builder.

Now I'm all for sharing source code back, and I think if you modify OSS you should have to give your source back. But if you don't modify it and you are just using the the public API, why should you have to give all of your code up for that? or more importantly why shouldn't be you be able to choose your own OSI approved license? I mean what if I have a GPL library, a Artistic Library and a BSD library I want to use to build an application, which has an API, must be GPL and so must everyone's apps that add to it. No you may not make a DWTFYWWI plugin, no you may not make a BSD plugin, no you may not release that code into the public domain, because the GPL says so, and I used a library that made me use it.

Well ya know what? I'm not going to use the GPL from now on. I need to find a new license that fits my needs (and will not be toxic to others).

Desire #1. He who modifies my code, and distributes it, must share his code (with me).
Desire #2. He who wants to use my API may do so (so long as he doesn't modify it)
Desire #3. He who wants to copy my code must share it, and any changes.
Desire #4. Some patent protection would be nice
Desire #5. letting users outside your organization utilize it counts as distribution (a'la AGPL)

I think the LGPL fits this bill, except for #5.

seriously why is it that you could optimize the internals of wordpress and make your own hosted blogging platform and you wouldn't have to share the changes at all... but with a theme you do...

Are there any other licenses I should consider or considerations I should make?

(honestly you don't have to agree with me about not liking the GPL here, this is just me)

Aug 3, 2010

Some Stack Exchange site's I'd like to see come to fruition

  • Unix & Linux: I think it'd be more used than Super User and ServerFault for some questions.
  • Outer Join: A place to ask questions about databases and queries
  • User Interface: because this is something we could all do better with.

Please consider following/committing. esp the unix/linux one (since I don't think Ubuntu should get to have its own, while leaving the rest of us in the cold.)