Premature optimization is not evil

Or rather people should stop saying this because most of the people that say it don't actually seem to actually know what is meant by "Premature Optimization" or how to determine when it is evil. I've heard people say premature optimization is evil to asking. "Is there a 3rd party library that does this more efficiently?" (knowing if there are better options is premature optimization?), "Thinking about architecting your app for horizontal scalability is premature optimization" (it is if the design is significantly more complex, but if it's just between using REST and ensuring stateless (which is about the same complexity up front, but it'd be harder to convert later)), "wanting to do Dependency Injection is..", "making that code easier to read and simpler and thus faster", and on and on. On the other hand, no one seems to think that requiring Redis, Mongodb, and NodeJS because it's webscale is premature optimization, even if the clustering is horribly convoluted and you end up in callback hell (not saying you are, just saying). Basically, you're not asking to do the thing that everyone else is doing, is premature optimization.

So let's talk about what the hell premature optimization is. Premature spending a week making sure you can spin up infinite instances on AWS because someday you might get slashdotted. Premature optimization is writing a method in a less than clear manner because you think it's faster. Premature optimization is rewriting String.format to StringBuilder because StringBuilder is faster. Premature optimization is any time that you write code that is less readable for the sake of performance, or spend an inordinate amount of time ensuring optimizing it without benchmarking to see if it's slow.

I've spent a significant amount of time in the past few months working on optimizing code, why? because no one ever thought about optimization, it never occurred to the author, in one case, that querying the same data from a database that had been previously queried, in a loop, outside of a transaction, was inefficient. It never occurred to the author that not refetching from the database to do an on screen sort every time you sort would be inefficient. Why think about what you're writing? because premature optimization is evil, or at least that's what I'd be told.

Here's are examples of premature optimization that are not evil. I choose to use EnumMap when I'm storing Enum's in a Map, I presume it's more efficient, so I do it, increase in code? a class name to the constructor. StringBuilder is faster than StringBuffer, so when I come across StringBuffer I convert it, increase in code, none. I use dependency injection to wire stateless (or unchanging state) singletons so I'm not constantly creating instances, code increase is use of a DI framework. I use onClick handlers to ensure that things happen lazily, only when needed.

Basically what I'm saying is that "Premature optimization is evil" is sadly used anytime when anyone is even thinking about anything that could remotely be considered optimization. I personally optimize my code for paradigmn/pattern matching the problem first (which leads to 2 and 3), readability second, performance last. Making smarter decisions about how to write your code is not premature optimization.

I think the real "evil" is encouraging people not to think about performance, or to further understand their craft.

Using Spring to create a full REST API in less than 60 lines of code

Spring with Spring Data is awesome. Seriously, I've never been able to throw up a full HATEOAS REST web service this fast. To start, I'll admit my headliner lie, I'm not counting the pom.xml.

cloc .                                                                 slave-vi
       5 text files.
       5 unique files.
       2 files ignored.

http://cloc.sourceforge.net v 1.62  T=0.04 s (104.8 files/s, 3930.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Maven                            1              6              7             65
Java                             3             15              0             57
-------------------------------------------------------------------------------
SUM:                             4             21              7            122
-------------------------------------------------------------------------------

The basics of the web service is we want to be able to create tasks, like those on a todo list, for now we want the simplest tasks possible, in as little code possible. We should use UUID's so that our service can scale horizontally, so that we can easily generate known test ID's and we know that no two entities will share an id if we ever wanted to flatten things. We need to be able to perform basic CRUD on all of our entities as well as list them.

First let's create our Task. As you can see it's incredibly simple, we have our UUID identity, the uuid and uuid2 basically are telling Hibernate and H2/PostgreSQL to use UUID's. You might ask why limit description to 100 characters, well, since these are quick tasks, I might want to share them in a tweet, and this allows enough room for a url shortner plus the description. I think the rest is pretty self explanatory.

Now let's create our Repository. Well that doesn't do anything... oh but it does, and although it doesn't show it, because this application doesn't need it, there's a nifty method signature parser dsl that allows you to build queries just by writing a method signature.

Here's our Application. ... and pom for dependencies and stuff.

Here's the output of some curl commands I ran.

For a slightly more in depth tutorial you can see the official spring date rest getting started page. In the future I'll try to write about how to actually connect to PostgreSQL and set up API Authentication and Authorization

People are always telling me how verbose Java is, how much less typing their language (especially Perl is). I'd love to see a Perl app that can do all this in fewer lines of Perl (restriction, no line may be longer than 120 characters, and must be humanly readable), I personally don't think it can be done at this time (not with full HATEOAS and as many response codes), but I'm waiting for the day it can, and can be structured this simply.

Java Privacy, broken by design

It is worth prefixing that none of the following arguments apply to anything using the keyword static which makes things more procedural (or in some cases functional, than Object Oriented.

The suggestion in Java is to give the least required permission, but this, in my humble opinion, violates the Open-Closed Principle. Java has four privacy levels. Giving something the least permission required to function is fine in a Security context, privacy in programming however is simply there to discourage developers from doing stupid things. In most cases, unlike security, it only makes them difficult, not impossible. I believe that any SOLID principle should make your code more easily extensible, so while in fact Java's privacy is not in literal violation of Open-Closed, it does make extension more difficult than it otherwise should be, thus violating the spirit of the principle.

Before I continue on to how I think Java's design, and common usage, violates the Open Closed Principle, I should explain how I interpret the Principle, as my interpretation appears to be slightly different from what's on Wikipedia. The Principle as described on Wikipedia appears to be combining it with two other SOLID Principles, namely Liskov Substitution and Interface Segregation. So first let's assume that The principle stands alone, and that although it'd be bad design to not be completely SOLID, Open-Closed by itself does not require a subclass to support the same interface. Let's also assume that Not modifying the source to add features is also an unrealistic expectation. The purpose of Open-Closed is to ensure that your subclasses are not modifying the the structure or data of their child classes and that a child may easily add to, or change the behavior it got from its parent (Liskov says that it must be substitutable for its parent).

First let's talk about final, marking a class as final, means you can't extend it. This by the very definition is in violation of Open-Closed, because the class is not Open for extension. Classes such as UUID are marked final, you might ask, why would I want to extend a UUID? maybe I want to give it a toURISafeBase64 method. That wouldn't break any of the orignal behavior, and is almost as legitimately belonging as representing the UUID as hex. What if I wanted to extend a nested final class like an Iterator on a Map? I can't do that, which means I have to completely reimplement the Iterator to add simple functionality. In fact the way those are implemented I have to implement much more than just the Iterator.

It is recommended by the official Java Docs, and the community, to make member variables private unless otherwise necessary. Private variables are only accessible to the current class and nested classes, they are not visible to subclasses, in or out of the package. In my opinion this violates Open-Closed because now, if I subclass I need to reimplement all the fields, or use getters/setters. Getters and Setters for every single attribute are actually almost no better than the attribute itself, and an object that is nothing more than those is an Anemic. Now it could be argued that making subclasses call methods makes them more... impervious to change, because if you change the data structure you can preserve the methods. The problem is that most classes wouldn't use their own getters internally, and thus break this, because then extending that getter won't actually modify the class as completely as desired. Also remember that subclasses are by definition, tightly coupled, usually changes to the superclass require taking a look at the subclasses. So if you are using getters and setters to ensure extensibility and preserve internal/external interface changes, use them exclusively, meaning only they can have raw access, all constructors, and business logic methods must go through them. At that point they are the replacment for direct member access and private won't matter as much (I will probably advocate a variant of this in the next article). However if you still want to access some member data hidden by the class directly, you should ensure that your subclasses can easily do so as well. You should only make a member private if it would actually cause a bug in any subclass.

So if we go on to assume that all subclasses, even ones in a different package (because you know people using your code are going to extend things) then we should be making all members protected. This would mean that all subclasses could reuse the member variables. Of course the problem is now your data is not encapsulated in your package, once a member variable is not private, is is available to your entire package. To me this also seems like a bad idea, other classes in my package don't need to see my objects internals unless they're a subclass. So now you have to choose, make all classes easily extended? or protect people who are programming in your package from themselves. You can probably control who's modifying your package and how, and have static code analysis to check that you're not calling obj.foo only this.foo. But nothing can give you back extensibility you've taken away (outside of adding it back).

So let's look at interfaces, interfaces generally have two options, public, or protected. This is fine, but has a problem, protected interfaces are only applicable to the package that has the interface defined. Methods implementing the interface must have the same privacy level. Most of the time what I actually want is an interface which I've defined globally as a contract, but I want the implementations to only be called by their package. For example, a DAO (Data Access Object) might be able to share the same interface (with judicious generic usage), between entities. However if you do this, you may find that your interface must be public, so it can be between packages, now the DAO itself must have these methods as public, even if it's being called only by something in the same package, because the interface was public so that the interface could be shared. I don't see that you can get away with this whether you use package by feature or package by layer. If you follow this through with previous design thoughts such as everything is an Interface, and those end up being public, and you want nice subclassibility, whether through protected members or through interfaced getters/setters, now everything is public, and we've completely lost any real encapsulation.

So how could it be done better? have a privacy type subclass which makes the method or member available to only subclasses and not throughout the package. Allow interfaces that have global definitions, but implementations of the methods can be at a package or subclass level. I feel like this could still be accomplished, perhaps by creating an interface type that is a "contract", and a new privacy keyword for "subclass". Contracts could define that methods be subclass, or protected, in their implementation. At that point you could have all kinds of methods that are still hidden to the general world. You could then build package by feature, have all methods that are required within the package have contracts, but share contracts between features, so all CRUD controllers would have the same method signatures, all repositories would share signatures, etc, etc.

What if I actually want more privacy? well you could not share interfaces between packages, and then have interfaces not be public. You could also not use an interface at all unless it's for a method on your bounded context that must be public. You can also say that ease of extensibility is not a goal and continue to not use your getters/setters internally, and yet make your members private.

You could also say, privacy is irrelevant, if the language is then preventing good, SOLID, design. Specifically here, Open-Closed, Liskov Substitution, and Interface Segregation. If you go this route you'll need conventions, and to trust other developers, because a lot of things will be public or protected. I recommend Perl's convention of prefixing subclass private methods with _ and assuming that all member fields are subclass/trait private and should never be called outside of their inheritance hierarchy.

Documentation! it should be TDDD

UPDATE: I shouldn't have said what I said about RJBS and have apologized publicly

Test Driven Development should be Test and Documentation Driven Development. I don't want to read your code to figure out how to use it. I don't really care if you write the Test, documentation, or the feature first. But you should do all three before moving on to the next one.

I'm gonna pick on Dist::Zilla this seems like a very good tool. Except it's documentation is so sub par it's not funny. The new documentation is quite good, although, I have not decided that I like the 'choose your own adventure' format. However none of the documentation on CPAN is good. It ranges from significantly less than the new tutorial, to virtually non-existant, to completely inaccurate.

You know what? It's not acceptable not to write documentation, especially not when you were apparently paid to do it and have shown that you are capable of doing it. Some people I've talked to think not documenting things is ok. I don't think it is, and it's yet another one of my unpopular opinions. Sure you can not document them... If someone else has volunteered and is actually doing it.

Yet maybe this is one of the reasons perl has fallen by the wayside. Documentation is usually either cryptic, incomplete, or inaccurate. Even the core docs like perlintro has inaccurate and potentially damaging information. Why are we so surprised that so many people don't know how to program perl in a modern way if the documentation is so bad?

Documentation should explain who, what, when, where, why and how. Who should use this module, what they should use it for, when they should (and shouldn't) use it, where they should use it, why they should use it, and how to use it. Unfortunately most documentation fails at all but the how, and the what. Who is an Abstract, and Description thing, you should consider who doesn't need it. Why why can be tightly related to who, but really I want to know why this module and not another, consider who needs a templating engine vs why it should (or shouldn't) be HTML::Zoom. When is also closely related to the previous 2 and will probably be answered with them, although consider when to use a for loop vs when to use a while. Where could be anywhere from top of file, to your view in a catalyst program. How is obviously the actual code and should be covered in the synopsis. Sometimes you'll have to think if you've answered all these, but if you manage to answer them all in a comprehensive and complete manner then you're documentation is probably good enough to be useful.

I don't expect people to be perfect in this, but some effort is better than none. Also if you write documentation, you may end up answering less questions, and save people many hours finagling a problem.

I'm sure even my modules fail to meet my own standards.

The best documentation I've seen is PostgreSQL's and Gentoo's.

UPDATE:
perlintro was fixed in 5.12.1

Quick sha1sum with Crypto++

I've been working on Korama. I was originally planning on hashing each music file and using the sha1 for the track primary key. However, after benchmarking an actual implementation. I've decided that it's too slow, 20 minutes for 3k+ tracks, also my program was about 5 minutes slower than find ... -exec sha1sum not sure on the reason why. However, since I had problems figuring out how to do it I figure I'll post a basic sha1sum program that I made with the help of people on the Crypto++ Mailing List.

Include's are mangled due to blogger filtering

#include < cryptopp/sha.h >
#include < cryptopp/hex.h >
#include < cryptopp/files.h >
#include < string >
#include < iostream >

using namespace std;

int main(int argc, char *argv[])
{
    char *file = argv[1];
    string result;
    CryptoPP::SHA1 hash;

    CryptoPP::FileSource( ( file ),true,
        new CryptoPP::HashFilter(
            hash, new CryptoPP::HexEncoder(
                new CryptoPP::StringSink(result), false)
        )
    );
    cout << result << endl;
    return 0;
}

it can be built with.

g++ sha1.cpp -lcryptopp -o sha1sum

and tested with

./sha1sum filename
It's not meant to be a exactly compatible implementation of sha1sum. it doesn't output the filename, it isn't capable of handling more than one file argument, and if no argument is provided it crashes. I know I haven't actually explained how the crypo++ code works, but I hope just posting this here will help someone in the future.

To pointer, or not to pointer: That is the question

For a very long time now I've been looking for good explanations of pointers, and how the varying kinds differ.I've finally found a book that does a good job of explaining what they are and how they differ. The book is by Bjarne Stroustrup (the creator of C++) it's called Programming: Principles and Practice Using C++. The book is supposed to teach you how to program (become a good programmer) using C++ not teach the C++ language. The question that Stroustrup poses is How do we choose between using a reference argument and using a pointer argument? his answer to the question is quite in depth, however I'll reiterate his summary here in the hopes that it's useful for someone.

So, the real answer is: "The choice depends on the nature of the the function":

• For tiny objects prefer pass-by-value.
• For functions where "no object" (represented by a 0) is a valid argument use a pointer parameter (and remember to test for 0).
• Otherwise, use a reference parameter.

Ok, that alone seems a little confusing. I'm just going to use function declarations to denote the differences in code.

According to him, pass by value is the least error prone, but it copies memory and is therefore more expensive. To do this you'd declare a function like int func_name(int x);.

"No object" in my quote is another term for "Null pointer" which he defined previous to this but wasn't in my quote. If no argument to the function is valid you'd use that. It's declared like int func_name(int* x);

The last option is to use a reference pointer. these are declared as int func_name(int& x).

I still have some trouble with the what/why/when to use pointers, but I'll probably just go back through this a few times. Stroustrup's explanation is still the best I've seen. I'd suggest his book to any novice programmer. I'm not sure if it's the best for someone who's never seen any code as it goes quite fast. It doesn't spend hardly any time at all on language constructs, which most books do. I don't think so fast that you couldn't use it as a 'first programming book' but it might be good to have a book that covers the constructs (like for loop) in depth.