Making re.pl usable

For starters I'm not a super fan of interactive shell's for non-system shells. I don't see the point as much. When I first tried Devel::REPL I didn't like it at all, compared to every other interactive shell I've used (with the notable exception of Oracle's SQLPLUS, which is the worst shell ever) it was an unfriendly toy not worthy of mention. I couldn't go up in history, there was no tab completion, and all in all the default output was ugly no colors or formatting. I noticed in the docs

By default the re.pl program looks for (all of the following code goes in this file)$HOME/.re.pl/repl.rc, and runs whatever code is in there as if you had entered it at the REPL shell yourself.

and that re.pl has plugins, I also noticed some strange syntax and what seemed to be overly complicated code for just getting a few things working. Sure the file supports perl, but do I really want to write full perl programs in an RC file just to use plugins?

Well fortunately it's not as complex as all that. First thing you want to do is set some good perl defaults. Since this is our private shell might as well get all the current features of perl. Of course if you want to have other modules always accessible load them here to.

Now lets get some sanity into the shell by using plugins for REPL. It's really much simpler than the docs say. Wow that's way easier than what any of the documentation describes.

Ok let's look at some plugins. First if your typing isn't perfect you've probably noticed that you can't use your up arrow to go back in history, to fix this you can enable Readline support by loading Devel::REPL::Plugin::ReadlineHistory like so, This will allow you to up arrow to go back through lines you've previously typed, which you can then edit just like your normal *nix shell prompt. This one thing makes re.pl about 100x less annoying.

But I wish I had some kind of tab completion so I didn't have to type so much. Hey there's a whole set of completion plugins, lets enable those too.

So the Keywords Driver allows you to type pri< tab > and it'll complete print plus showing you printf just like your *nix shell. LexEnv will allow you to auto-complete your variable names and so on. There are a few more besides what I mentioned here.

Next I'll mention the Colors plugin. You may have noticed when you enter code such as my $var = 'blah'; say $var; you get the output

blah
1

or the output text and return code. well if you enable colors the return code is now in green making it easy to distinguish between output and extra info.

Last but not least I'll mention DumpHistory, enabling DumpHistory allows you to Dump your input history to the screen or to a file.

Now we have a fully functioning, 'user' friendly, perl shell.

For your convenience here's my current file

Dist::Zilla vs xenoterracide

I solved my problem with Dist::Zilla. If you agree with me that you should be able to install from your git repository, without requiring your users to have Dist::Zilla installed here's one way of doing it.

First install Dist::Zilla::Plugin::CopyTo. Now Edit your dist.ini. Most people probably use one of the Dist::Zilla bundles. Assuming you use Basic here's what you do. Now as you can see we use GatherDir to get where the source should come from and CopyTo to tell additional places where the output of the script should be sent to. It will still send to the default directory, that's ok, you can just ignore those files. add . to your repo (making sure to add the default generated location to your ignore list) and commit. now if dzil changes any of these you can see it.

If you wanted to with git you could use a git-new-workdir and another branch and copy to it and commit this stuff to a different branch. I haven't found another way to do it in a seperate branch yet, although someone said it's possible.

P.S. 1
If dzil ever gives you some vague error about some util file... run dzil clean I don't know why it was doing that to me occasionally through all this but I spent like 8 hours thinking my config was screwed up when the directory just needed to be cleaned.

P.S. 2 Thanks to rjbs for putting up with my asininity while I asked a lot of questions and ranted.

P.S 3
My stuff is 99% working now... I just have a problem where my licensing is wrong :( I've set it to Perl_5 but it really should be GPL3 and Artistic2 (or whatever those are in Software::License).

cat that displays tabs at 4 spaces

So in the tabs vs spaces war I'm squarely on the side of use tabs. tabs have meaning. tabs allow people to set there editor's to whatever space display width their eyes are comfortable with as opposed to what yours are (e.g. you like 2 spaces I like 4). However, I've heard the argument what about when you cat the file, etc. Well it is actually a bit annoying... so let's fix it.

my requirements when I type cat foo if foo has tabs they will be displayed at 4 spaces, and I can still pass all cli options as I normally would to cat. To do this we're going to create a shell function called etcat()
I'd like to note, that in order for this to work you must use either an absolute path or a call to env cat because otherwise the shell will attempt to call cat as a recursive function. I put this in my .alias file and sourced it.

you can now put alias cat="etcat" in your aliases file, or run etcat. This is to protect you from any problems that could otherwise be caused by overriding cat in scripts you don't control (not that I'm actually aware of any for this function).

P.S. Thanks to @greybot on #bash for the incantation as I was having trouble with the "$@"

UPDATE:
changed name of function.

bash and konsole

So I've had lots of problems using a multi-tab xterminal, for the casual user I'm sure they'd not even notice them, or think I'm silly, but when you spend all day in a shell they become less so.

History

one of the big problems I've had is with history. see when you close a shell then it writes the history. well if you have 5 tabs open each with with a super long history they'll write those histories out in the order they close in. basically what happens is all the history saves at once so one of your open shells histories will be before another's and if each has like 50 entries it can be quite a way back. In fact I'm sure I've had it happen where parts of my history was lost. So what do you do? you have to separate them per tab. One way to do this is with gnu screen. I don't like screen's history because it records the output of the terminal as well as the command, so this wasn't a good solution for me. The solution is that each tab must have its own history file. Well how do we do that? before I get to it I have another problem.

PS1, umask, and default group

I also have a need to have different settings depending on what I'm working on. If I'm working on files that need to be read and written to by my web server they need to have a different group and umask. first I solved this by su - to a different user, then I solved it by having different bashrc's but neither fully solved the history problem.

howto do it

first you have to have one konsole profile for each of the different settings you need. so I have one for web development called webdev. Then you have to add an environment variable to it I'm going to call it KONSOLE_PROFILE and set that to whatever your profile is called. Neither name really matters so long as they don't conflict with others. Then you need to code your bashrc (or other shell to work with it). I have a shortened sample bashrc that you can use in a gist

Adding a new group to an open shell without logging in and out

Most Unix users believe it's impossible to add a group that was recently added to your user account without logging in and logging out. If you're running X and need to give your gui new permissions (like dolphin/konqueoror) this may be true, for the most part. I don't know how to change it for those processes. However, if you need to change it for a shell prompt it's easy. You merely execute the command newgrp in an open shell prompt and that prompt will now be loaded with the new group. I believe it also changes the default group that shell is running as. So if you run newgrp http; touch test.txt your file will be created with the group http instead of whatever your default is. You can change your group back to the default by using the sg command.

PostgreSQL \edit and Syntax Highlighting

If you've ever used \edit with PostgreSQL you've probably noticed that it's not using syntax highlighting that your favorite editor most likely supports. I've wondered why this is for a while, Postgres uses vim on my system which supports robust highlighting among other things.

I had a few things suggested to me, such as setting putting settings in my .vimrc, most of which would have forced filetype=sql on files that weren't sql, and would have been active outside of psql.

When you have a question what's the first thing you should do? RTFM. The answer was in the man page all along.

Tip: psql searches the environment variables PSQL_EDITOR,
EDITOR, and VISUAL (in that order) for an editor to use.
If all of them are unset, vi is used on Unix systems,
notepad.exe on Windows systems.

so how does this solve the problem? well the only program that uses PSQL_EDITOR is Postgres, and most editors allow you to specify options at startup. In the case of vim, if you want to have use filetype=sql for the default filetype for \edit all you have to do is


export PSQL_EDITOR='vim -c "set ft=sql"'


of course if you don't use vim or you want to set some different options you'll have to read the documentation for your editor, or operating system.

Making secure recoverable passwords

For the basic you need a calculator with a decimal to hex function. Your OS should have one built in (scientific mode), many calculator's do to. For advanced you'll need special hashing software, suggestions for it can be found below.

Windows	HashCalc
Macintosh	MacHash
Linux/(*nix)	Gnu Coreutils

Now that we have the software we need. You'll need an 8 digit (minimum) number that you won't forget. Guess what numbers fit the bill perfectly. Dates. All date's can be expressed in 8 digits, but they require a year. The date we'll use (for example) is Linux's Birthday. August 25, 1991 (I recommend something more personal like your anniversary or birthday).

So we are going to write down the date we picked for the password (note we aren't done yet).

something like

Linux BDay

Date's can be formatted in several ways, Just make sure to remember the numeric format you use. For our example we'll use the ISO 8601 standard. Which leaves our example date in format, YYYYMMDD, looking like 19910825.

So let's make a note of the format (to throw people off use a diff format say DDMMYYYY).

Linux Bday ISO

whatever you do make sure YOU understand your note, but that it isn't easily understandable by OTHERS.

alright... now we're going to take our number and make it hex. So open your calculator enter 19910825, then use the convert to hex function and you should have this 12FD0A9 (if you aren't using our example you may have a different 'number', or it may not be displayed exactly like this, display's vary between calculators). This is your first basic password. you can use it as is, or you can make it a bit more secure by using one of the hexadecimal notations. example: 0x12FD0A9. You can add a word or funny characters to either side of it to make it stronger too, just make sure to note anything you add to it.

Due to a Rainbow Table database you should make sure your password is at least 8 characters but I would recommend no less than 9.

But your hex number is less than 9 and what do the programs you listed have to do with this?

good point... let's make something a bit more crypto and less predictable.

Take the same date and plug it in to your hash creator, I'm going to use an md5sum for my first example.

if you put 19910825 into md5sum you get 6f9822851dfc6c1045c6fef827e5d729 (for you nix people enter the number like this 'echo -n "19910825" | md5sum'
on the cli otherwise you might end up with newline issues because different operating systems use different newlines).

so lets say you need an 12 character password you could just use the first 12 characters of your hash, example: 6f9822851dfc if you need an 8, example: 6f982285 ,etc etc.

if a stupid 'cracker' or black hat hacker get his hands on your actual password he might think he hasn't decrypted it fully because it'll look like a hash or memory error. want to throw people off even more just remember you aren't using the first 8 but the first 8 after the first 2 example 9822851d.

Or use a different has like sha1 or sha512.

Ultimately you could make your note to yourself like this

Sha1LinuxBDayISO to a non IT person this will look like a random password as is.

Also the cryptographic hashes take words too. So you could do your name. Caleb into an md5sum is

9ced73b8525de1db8e232ad575baa9dd

but it's different if I use caleb.

2f0154d7db348840676529dd72f1c034

if you always enter them the same you will always get the same result back. I guarantee after a couple days typing your new password you won't need to have it written down. If you work at a place that requires things like monthly changes.. just put the date you changed it on in to create the hash. Just make sure no one knows but the number/word(s) you used and the hash you used or they will be able to duplicate your password. Of course you're smart and you added somthing to it right? like a symbol (!@#$%^*{}|:"<>?[]\;',./) will all work find if the auth methods allow it.

have fun never having a hard time creating passwords again.

busybox vi

although there isn't much to say about it, it's even more limited than the original vi. If you are like me and have been using a vi clone for so long that other editor's are unimaginable. I suggest trying busybox vi on your *nix cli. At least until you can install vi(m).

iptables for the average desktop user

The Best guide for learning the basics of iptables is here
Linux 2.4 Stateful firewall design for the most part it continues to apply to the 2.6 kernel. The only things that won't apply to your linux system will be: emerge if you aren't on gentoo, and the kernel options which have changed since 2.4 and even a couple of times during 2.6.

I'm not going to cover those here. If you need help building your kernel or installing iptables I suggest that you consult with either the iptables home page or even better your distribution. Chances are it is already installed, and may even be configured.

First Let's see if we have any rules.

The following commands require root access, and can be run in a root shell, with sudo, or in a shell script by root

iptables -L -v

your output should look something like this if you have no rules

Chain INPUT (policy ACCEPT 211 packets, 27413 bytes)
pkts bytes target prot opt in out source destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 312 packets, 211K bytes)
pkts bytes target prot opt in out source destination


If it looks different no worries it just means that your distro has already installed rules.

If so make sure BEFORE CONTINUING to flush them. FIRST check to make sure your policies are set to accept (you can see that in caps above) if they are anything else run


iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT


now that your policies are clean you should flush the rules

iptables -F #flush all rules
iptables -X #delete all chains


all right now hopefully you are at a clean state. If you have ever used iptables for nat you may not be, but that is beyond the scope of this article.

Now we need to create rules, since this is for normal deskop users you shouldn't have any services listening, also you shoudn't be routing anything.

to disable routing with iptables (I'm ignoring the kernel setting for this)


iptables -P FORWARD DROP

# set all forwarded packets to go bye bye if they reach the end of the chain

the output chain is good on accept for the normal user, only a masochist would want to write rules for it. You generally should trust your outbound traffic.

Now to secure input

iptables -A INPUT -m state --state INVALID -j DROP
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -P INPUT DROP

#this rule drops all packets with a bad state
#accept any packets that have something to do with ones we've sent on outbound
#accept any packets coming or going on localhost (this can be very important
# set all packets not matching these rules to drop

Various linux distro's may require you to do something special to save these rules so that they survive a reboot. consult your distributions community.

If you have rules that you want to keep but also want to use mine I suggest putting mine first, you should do the -A rules in reverse but use a -I for insert.

When dealing with iptables always take care when applying iptables -P DROP. You can be locked out of the machine or the internet if you apply this without the appropriate rules in place.

That's it. your desktop should be secure from an attacker that you aren't allowing in. There are of course other things that you can do to make it even more secure but those are beyond the scope of this tutorial.

really nice

if you are using the cfq io scheduler in linux you can use ionice to nice your io. so to really nice something you would do


nice -n 19 ionice -c 3 programname


this may not be the best idea on all programs but it works. If a program is beating your hd and cpu and you don't want it to run this when starting it.