Jun 27, 2010

Announcing Dist::Zilla::Plugin::Catalyst

So I just recently finished reading Restful Web Services and decided I wanted to go back and play with Catalyst and REST some.

The original way to create a Catalyst skeleton is to run catalyst.pl MyApp. This creates a lot of nice files to get you started. dzil new basically does the same thing for a generic cpan module. Honestly, without any plugins dzil new isn't that useful. However, once you add Git::Init , you remove several steps from the creation of a new module and repository. Git::Init also makes your first commit of everything it added. I got to thinking why on earth would I want to do the following to get a cat module going and convert it to dzil.

catalyst.pl MyApp && cd MyApp
vi dist.ini # and add numerous lines
rm Makefile.PL README t/* # and maybe more since dzil is better at managing these
git init
git add .
git commit


when I could just be doing this

dzil new -p catalyst MyApp


A lot simpler huh? to get you started you need a few things but then creating cat apps will be easy.

first you can run dzil setup or if your version of dzil doesn't support that yet, you need to create the following ~/.dzil/config.ini file by hand obviously fill in your own credentials and license preferences. This file is pretty much needed for any dzil new operations.

Next you need to create a minting profile (not necessary for barebones doesn't do much for you). run mkdir -p ~/.dzil/profiles/catalyst/ (note: catalyst is arbitrary, it can be anything). now create a profile.ini in that directory. The only mandatory line in this file for this to work is the [Catalyst ...] one, I think you'll want the other two however. You also want to set AUTHOR="your name youremail@example.com since that's how Catalyst::Helper inserts it into its files (I'll probably work on fixing helper later).

Now you can just run dzil new -p catalyst MyApp. Hopefully, this simplifies your catalyst app creation a little bit.

Good patches are welcome, so are feature suggestions and bug reports. Also I registered it as Dist::Zilla::Plugin::Catalyst So if there are any other plugins that you think could be useful that are specific to Catalyst I'd be willing to add them.

Special thanks to Tomas Doran who helped me (ok... he wrote most of it) create this module.

Jun 26, 2010

Solving code generation problems in dzil

Firstly I want to clarify a bit on my opinions of PluginBundle::USERNAME modules, as some comments there have inspired this post. I don't think you should use them because it makes it harder to disable plugins, and I think Robin Smidsrød put it best:
Mostly it is because the Dist::Zilla::PluginBundle::USERNAME doesn't actually say anything about its intention. It only says use Dist::Zilla as this person does, but what does that actually mean? If you don't know the person it doesn't really tell you anything.

I'd much more prefer PluginBundles that actually advocate certain types of standards or behaviors.
...

Essentially Bundle's like @Git and @Basic don't cause problems because they're generic well defined and contained within their distributions. A bundle that wasn't contained within its dist might not cause a problem if it's for a generic well defined purpose. I'd be in favor of a BundleQATests or similar so long as the author was considerate that some tests (PodSpellingTests) don't work well on some *nix distributions and isn't included in the bundle. But Something like @Xeno (my cpan username) would be my own special settings... who wants to use something that's only for me? and more importantly why would they ever think they have a right to bug something that the name itself implies it's only for me.

But another problem was brought up... code generation. Here's what Nilson said:
dzil is very nice, but I don't if I like the idea of different line numbers and files in the repository vs. the CPAN release. I'm still trying to fully digest this idea.

I remember one of the main mentioned drawbacks of source filters were the possibilities of error messages in the wrong lines. And now, everyone seems to embrace this without hesitation.

So to start dzil's generation isn't quite as bad as a source filter which will give you the wrong line number period, using this generation will still give you the right line number for the module being run at the time. It just means the cpan line number may not match the repo line number. I'm going to show you how to fix this, but you don't always have to. The most common place I've found problems with in code generation has been tests created by extending InlineFiles. In these cases you just want to go and bug the author of the Plugin. I think using these plugins is much better than writing your own EOL Tests, NoTabs, Critic, and Kwalitee etc. It helps keep your dist quality up without extra work, ultimately leaving you with just the responsibility of writing good tests that are specific to just your distribution.

Now on to making sure that your repository matches your cpan dist. The first thing you want to do is use [Git::CommitBuild] (if you're not using git look for something like it or write something like it) This will take all the generated output and commit it to another branch each time you build. Then make sure this branch is pushed to your public repository. I think the biggest problem this solves is having things like your README and your LICENSE actually be in your public repository (if your repo doesn't have a LICENSE... what's the legal situation?). If you're using github you can also change this to be the default branch to be displayed in the repo admin settings.

Next thing is don't use ::Plugin::Prepender This will evil-y insert lines at the beginning of your code which will definitely throw off the line numbers being output. It even seems to suggest using it to insert use strict; use warnings;. If you need output prepended to all files, I suggest writing a plugin that takes advantage of dzil new and maybe just write a script that you can fire off whenever you need to create a new file. This is the only module I know for sure that does this, but avoid ANY that insert actual code or prepend lines to your files (in a way that isn't added to your actual 'master'/'trunk' that should be patched).

POD can also screw up your line numbers, if you're using PodWeaver (or module that has similar side effects) with dzil, which you likely are. Perl Best Practices page 140 will save you here (actual quote pg 475 a summary chapter).
  • Keep all user documentation in a single place within your source file. [Contiguity]
  • Place POD as close as possible to the end of the file. [Position]
This includes the # ABSTRACT: my abstract here line. If you put the # ABSTRACT and any actual pod after the code then all the pod generated will be after the code in your build, and thus any line number errors will be correct. Of course this doesn't save you if your error is in the output pod, but I suspect that's not the original complaint anyhow, and there are lots of dzil plugins to help you keep your pod sane.

Essentially don't do anything that will change line numbers for the code in the resulting build output. Following this will ease contribution, and debugging; I do not believe it significantly increases maintainer load. Happy Hacking!

UPDATE:
just remembered... dzil inserts a BEGIN block... sigh... can't win for nothing.

UPDATE:
I recommend using Dist::Zilla::Plugin::OurPkgVersion to avoid dzil's BEGIN block / VERSION insertion.

Jun 19, 2010

please don't use Dist::Zilla::PluginBundle::USERNAME

or create them. Here's the problem.... (short version is Don't put PodSpellingTests in them)
normally you'd have
[pluginA]
[pluginB]
[pluginC]
[pluginXTests]
[pluginYTests]
[pluginZTests]


and one of them doesn't work on your system (for whatever reason), well you can just do this.
[pluginA]
[pluginB]
[pluginC]
;[pluginXTests]
[pluginYTests]
[pluginZTests]


the ; is a comment in ini, now dzil won't use that plugin. But people will say well you don't want to do that of course I want that plugin enabled. Here's why you may not temporarily. Casual user X has a bug in /your/ module that's using dzil, they code up a patch, and they want to run your test suite. The can't, because [pluginXTests] won't even install properly on their system due to a non perl dependency. They could have just commented it out, but now because you've used this PluginBundle it become difficult. They can't just comment it out.

They might be able to
[@Filter]
-bundle = @USERNAME
-remove = pluginXTests

IF they can get your bundle installed in the first place. This requires them find a way to install your Bundle without installing said broken module.

as I stated at the top my problem is with [PodSpellingTests] it itself isn't broken, but it's dependency Test::Spelling is on some systems, due to the lack of a 'spell' command.

Yes I know there's a workaround... I could just copy a shell script into my path that makes aspell or something work, but really that's not the right solution. No I'm not packaging GNU Spell for my system either, the other spell programs work much better (from what I've read).

Also I find modules with your own username in them to be fairly obnoxious.

If you'd like less code, can't we start making a few more generic Bundle's? I wouldn't minde seeing a Bundle for Tests, and maybe another one for other stuff. I think 1/3 of the lines in my dist.ini's are tests.

Jun 17, 2010

RT git workflow

git was designed to be very flexible in its workflow. One of the things it was designed to do was handle email patches, since there are a lot of patches sent to and from the mailing list. This is a good thing, even if you don't have a mailing list, or you have your own bug tracker (in addition to RT), you can use RT to receive email patches from git. First you have to configure your MUA (mail client)(and go write a patch), some suggestions are given in git.git's SubmittingPatches document. If you like the CLI I suggest you use the git send-email command. I'm going to walk you through using it, but you'll have to learn how to set it up for your smtp server on your own (one hint: if you don't provide an smtp pass as an argument or as a config option you will be prompted for it).

now you have a patch that you haven't pushed yet. you can run one of the following sets of steps to get going.
git format-patch -M HEAD~1 # commitish. commit before the number of patches you want to generate
vi 0001-patch-name # and add some description between the --- and the diff stat
git send-email 0001-patch-name # can replace this step with MUA of choice

git format-patch -M HEAD~5 # generate a patchset of the last 5 patches
git send-email --annotate *patch # will open the core.editor you set in gitconfig to edit the patch

git send-email --annotate --format-patch -M --to=destination-email origin/master
# generate all the patches that aren't committed against origin/master branch in your local branch


for the time being you should probably just send it to one of your own projects on cpan so you can see. If you look at the web interface for the bug you'll notice it's clobbered your formatting, same goes with the email RT sends you... so this is useless right? well... certainly doesn't help... now look at the 'with headers' version. Their are lots of headers here, but all your content is intact. Now download this file (right click save as should do it).

you may now ( you may have to run git reset --hard HEAD^ first to undo you last commit).
git apply downloaded-file.txt

which will only apply the diff no commit log or anything (reset without HEAD^ now).

or you can git am -i downloaded-file.txt -i is interactive and will allow you to choose whole folders full of mail patches and which to apply.

I think this is a good workflow for 1 off patches. obviously you don't send them to yourself, I'm just trying to show you what it's like to send and receive.

Jun 16, 2010

Test Perl 5.12 on Arch Linux

Perl 5.12 has been languishing in the testing repo for a while, there are now several packages that have been rebuilt in testing and community testing. All of the rebuilds I reported have been fixed. If you use arch please make sure you're using their rebuilt packages (and not one you rebuilt because it took like 2 months to do this) if you have any problems file a bug. If you aren't sure which package is really the problem feel free to drop me a line and I'll see if I can help you tack it down. I have over 200 cpan packages on my box right now all of which successfully pass their test suites.

Jun 14, 2010

github renders POD

Not much to say here, but if you didn't know it if you put a file.pod in your github repo it will be parsed and formatted, here's an example: Unfortunately you have to actually visit the GIST to see it rendered.

Jun 13, 2010

Semver.org probably not compat with all downstream

For those that don't know I currently create packages for AUR, and I used to work on Funtoo, as well as a fork of it Regen2 (which died by my hand). So I've got some experience working with BSD-ish style distro's. I can't comment on the Debians or Red Hats of the world, other than REPO HELL DRIVES ME NUTS. Now to the topic. Semver seems sane except for 1 thing that it doesn't mention and an example that encourages it.

A normal version number MUST take the form X.Y.Z where X, Y, and Z are integers. X is the major version, Y is the minor version, and Z is the patch version. Each element MUST increase numerically. For instance: 1.9.0 < 1.10.0 < 1.11.0.



Not all packaging systems are capable of dealing with multipart as actual multiple parts. Lets give a more complete example to break it down. For instance: 1.9.0 < 1.10.0 < 1.11.0 < 2.0.0. That's what's supposed to happen... but that's not how it works. Arch for example see's versions as strings... so let me show you what happens.

190
1100
1110
200


This is essentially what you could interpret the "."'s are essentially ignored ( Ok it's not actually how it works... juster's comment is more accurate than me but I think it's a good rule of thumb). They don't matter they're like spaces between words, they simply make it easier for a human to read. Never ever, ever, EVER! change the length of your versions to anything BUT larger. You will cause downstream annoyances, sure we've tools to work around it, but I don't want to and it shouldn't take any extra effort from upstream to make sure this doesn't happen.

In order to solve this if you ever need to move anything to double digits just don't go backwards. In other words: 1.9.0 < 1.10.0 < 1.11.0 < 2.00.0 would actually evaluate properly. I find it more likely that in an X.Y.Z schema that you will need X.Y.ZZ though, and maybe X.YY.ZZ. For a project More than 10 years old XX.YY.ZZ could happen. Anything more is probably ridiculous. If you find yourself hitting 99 slow your roll before you reach it by moving to a time based release, no more than once a month for example.

Some people will say that Arch doesn't have a sane internal representation. Keep in mind that maintain over 300 package now for arch... of those 300 maybe 10 have ever caused me to create a workaround because of this issue. It's not difficult, so please just be considerate of downstream, it's a lot easier to acknowledge and be mindful of this than it is to fix downstream in a way that would work for all upstream packages (e.g. we've got more to consider that just your package).

UPDATE: Read juster's comment if you want more information on actual implementation and how I'm slightly wrong.

Selling Open Source Software (specifically games)

This is a pseudo reply to Jason Calacanis for what he said about using an Open Source game engine to build a game in This Week in Startups #54, Ask Jason segment. For more shows ThisWeekIn.com.

I believe Jason is implying that you can't make money with open source software(that's not a support contract variant) directly, but it's a good resume builder. True there aren't any big billion dollar open source companies, and companies like Red Hat make money off support, but I don't believe he's correct. But there are 100 million dollar open source companies (Red Hat), and companies that make billions, and have a successful open source strategy, not all of which is support.

For starters I am not a lawyer, successful entrepreneur or venture capitalist, I am a student about to start ~ his last year of his CS bachelors. I will mostly be referring to the GPL family of open source licenses, as a generalization on open source, so if what I say doesn't match your specific OSI approved license that's why.

Piracy

Probably the biggest concern you have with open sourcing your software is that people will be able to use it for free. Guess what? almost all software worth using is available for free if you know where to look. Those that aren't are usually web only or have there distribution channel well locked (e.g. like apple store not available on cd). All this DRM-ed software out there has people distributing it and making cracks and keys available for it. So even if you built everything and locked it all up with DRM people would still find a way to not pay for it.

Licensing

Selling

Absolutely nothing in Open Source says that you can't sell your product or your source code. In fact the GPLs explicitly allow it. Not only can you sell the resulting binary and medium that it's distributed on, but you can sell the source too, you just have to make it available.

Using

All versions of the GPL allow you to modify the code for the use of the entity (business or personal) so long as you do not redistribute that code (meaning you can modify gpl to your hearts content inside of a business and not have to give anything back).

LGPL

The LGPL specifically allows you to call their API in a proprietary piece of software, and only if you modify the LGPL software itself do you have to share that code.

GPL

Mandates that you share any code that is a derived work of the GPL work under a GPL Compatible License. In the specific case of wordpress themes, what derived means is contested. I particularly like this:

The alleged derivative must “physically incorporate a portion of a copyrighted work… [or] supplant demand for a component of that work.” -- Lewis Galoob Toys, Inc. v. Nintendo of America, Inc

Seem's the GPL FAQ clarifies all various circumstances in the next few sections. I certainly wouldn't want to be a python programmer... I do believes this means you can't make any python code that uses a core python library proprietary.

One way around it may be how NVidia releases their drivers when the Kernel is GPL2. They create a small wrapper module in kernel and release a binary blob outside.

Another way is if your program is served by a server, technically under the GPL this is NOT considered derived as the program runs on YOUR server, thus you don't have to share the changes. A webapp based on GPL software can be as proprietary as you want it to so long as you aren't distributing it (Not sure about the legal case on software hosted by a hosting company). It also means if it's your companies hardware, you don't have to open source it (this has been pulled on 'rented routers').

AGPL

This is the strictest of licenses, and is meant to insure open code in the event of the final paragraph of the last section. You must share code if anyone outside of your entitity is using the software.

Binaries

Nothing says you have to release the source of your games with your binaries. If your game hits the shelves of best buy and works.. most people will never bother to look see that the source is available, and that is completely legal.

There are 2 types of game(er)s... casual like Farmville, and Bejeweled... and hardcore games like Half-Life, StarCraft, Neverwinter Nights, to name a few. Casual gamers are usually not sophisticated computer users. "I should update my antivirus? what's a browser?", are not things I would be surprised to here them say. Hardcore gamers are however, and most can probably rattle of their system specs and know not only to update their AV but how to disable while gaming, many even know how to overclock their system.

Casual gamers won't be able to figure out how to assemble your source to even begin to consider pirating it. Many Hardcore gamers could, and might just to try to get more performance out of it. Most would still pay for it though, because your code doesn't include any DRM, which is becoming more annoying to them (and of course you made a great game worth buying right?).

Building software binaries isn't easy, I don't think that any license in open source requires that you include documentation on how to do so. Although you might be seen in a negative light for doing this and someone else will figure it out and publish it. This will require your users to download a toolchain of software and use it. This is not corporate IT, most of them haven't done this and it'd be too intimidating for all but the most determined (for novice software builders).

Contributers

Almost all of the Really long lived games such as Half-Life, and StarCraft had the ability to be modded, sure the source code for the engine wasn't available, but you could make your own 'mini games'. That couldn't be played with out the original. If people had the source for your game engine too they could do even more than standard modding to make your game better. Heck, they might even port your game to a new operating system where you could make money on it from the game. As your company grows you could recruit these contributors to your company.

Marketplace

Or should I say Mod Store, why not create a marketplace for gamers to sell or give away their modules. They can pay you a percentage of royalties.

MMO

Did I mention that whatever is on your servers doesn't have to be Open Source? what good is World of Warcraft without the servers? and how much is it really hindered by a free competitors on their software like The Spirit Realm. Which just to clarify is a free version of World of Warcraft built on proprietary WoW run on their own servers. I don't see Blizzard bleeding money over it, or suing.

Giving your competitor an advantage

This is the main reason that games don't open source their engines, they are afraid the competition will steal it. I don't really get that, you're a gamer, you play good games, and you play and buy more than one. Most of the time you only buy one copy so it's not like they'll make more money convincing you their product is the best. But you can charge to license your engine. What'd be wrong with that?

My conclusion is that I don't know really understand why a company that's big enough doesn't try it. EA and Activision both release enough crap to try it with one game. I honestly think you can make money on any quality Open Source software that you could make money on if it were proprietary. If you do it though, much like Red Hat, you may not be able to expect to be a leader financially, but that doesn't mean your revenue's will be anything to sniff at.

Jun 11, 2010

I am a $explicitive, or Formal Apology to RJBS

A lot of people think I'm a $explicative (insert your favorite one there). I've said some things that I shouldn't have because I was pissed at the Person(s) at the moment. I would like to formally apologize to RJBS for the negative things that I said, about him. He's been nothing but kind to me and is a great contributor to the perl community (and maybe others?). I greatly disrespected RJBS and I actually do regret it. I should have posted this weeks ago when I realized I shouldn't have said it.

I hope that RJBS will accept my apology.

I would however hope that RJBS or any other member of the perl community would not take exception to my contributions even though I can be an $explicit.

Jun 4, 2010

Using ref to fix 5 year old bug

So I haven't been hacking perl for 5 (or more) years but I forked Template::ShowStartStop from Template::Timer which is that old. since I forked it this test has bugged me since I didn't really understand the test, the section of code it referred to or the actual problem.

This is an approximation of the error you'd get.
Couldn't render template "undef error - Can't call method "name" on
unblessed reference at /usr/lib/perl5/site_perl/5.8.0/Template/Timer.pm line
66. "
This error is actually a copy from bug #15457 but I happen to know it's basically the same error you'd get with an eval.

So what's the problem? can you spot it? ( original timer code )

Ugh! That's ugly, let's take a cue from Perl Best Practices and format it as a tabular ternary instead. Now we can read it...

Bug #13225 suggests a problem with eval and the following fix (which I translated into something more similar to the current code).

well that should fix our eval problem... since an eval will be a reference of type SCALAR. but what happens if it's a reference not of type ARRAY or SCALAR? then we'll still get that error.

According to the ref function documentation
If the referenced object has been blessed into a package, then that package name is returned instead. You can think of ref as a typeof operator.
what this means is I could check what type of object it was that I was normally getting. In order to do this I put $what into the output I get. I found out that most of the time $what is a Template::Document. So now I optimized my code for that situation.

See the thing is that we really don't want $what->name method to be called unless it's a Template::Document which actually has that method. I'm not sure that I'm not missing any tests at this point, but I'm pretty confident that, at least, this kind of bug won't crash my module anymore.

Jun 3, 2010

Testing TT Template's

So the poorly made patch the other day converted my Test::More test to use Template::Test which removed quite a bit of code from the test itself. I hadn't seen Template::Test beforehand.

Here's a very simple example of a test you could write to make sure a template is being output ok.

If you want more examples I now have quite a few for Template::ShowStartStop which you can see in the Test directory of the master branch.