Override DNS on a Linux system without root

I had this problem for a long time, and no one ever proposed a good solution. Recently I got a new answer on my, almost 2 year old, Unix and Linux StackExchange question. This information seems very obscure and so I thought I'd share it, if you too have had this problem and were unable to find this, or at least found finding it hard, consider upvoting the answer.

Problem

You're using a Linux system that you don't have root on, you need to override the DNS of the system. You usually want to do this because you're testing a service (web site) that does not have a proper hostname, but needs one in order to function properly. In the hosting world this comes up often enough.

Solution

You can set the HOSTALIASES environment variable before running your client program. HOSTALIASES is an environment variable that points to a file that is essentially alias value pairs.

$ echo "foo www.google.com" > ~/my_hosts
$ HOSTALIASES=~/my_hosts wget foo

See hostname(7).

Ubuntu isn't Linux, What's a Beginner anyways?

Today Joel Spolsky announced The results of the poll of whether ubuntu SE and unix SE would be merged, they won't. In short the Unix community was in favor of it but the Ubuntu community was not. Joel had this to say:

So, Ubuntu, Linux, I get it, it’s clearly not the same thing. If you love Ubuntu, we have a site for you. If you love Linux and Unix, we have a site for you. A Stack Exchange can’t work without a community that loves a subject, and love is very… specific. Fighting human nature is hard: the factionalism and fork-happiness of the Unix world has been a hallmark of that community ever since BSD vs. System V, and Stack Exchange can no more bridge that gap than we can unite the Judean resistance.

I am horribly disappointed in this decision, this factionalism doesn't help the cause, this set's precedence that it's ok in the StackExchange community to be a separatist. Arch Linux's bash must not be the same as Ubuntu's or Fedora's, and of course they've made so many changes to upstream Gnome as to be unrecognizable as such.

The main argument in favor of this separatist movement is that Ubuntu is for beginners, if that's so then why isn't it beginner.unix SE? What's a beginner anyways?

I'd dare wager that 99.999% (POOMA) of all Linux users are windows Power users, veterans, and refugees. So what's a beginner? someone that doesn't know what Firefox is? No they probably do. Maybe they don't know where it is in the menu? I can see that... but I don't see how that can't be answered with a Screenshot and a Ubuntu tag on Unix SE. Maybe it's someone who doesn't know how to use the CLI? Well where do you define beginner from advanced? I realize someone reading there first shell programming Tut/Book/etc is a beginner, but are they no longer a beginner after that? how do you know you're not a beginner anymore? When you choose to move beyond Ubuntu? If I've been using Linux for 5 years and I choose to move to FreeBSD am I a Beginner?

Why is that the answer to a beginner from an advanced user isn't welcome? Is it good that beginner are exclusively answering other beginner questions?

In other news it looks like I may become a moderator ( temporary at least ) for unix SE.

Some Stack Exchange site's I'd like to see come to fruition

• Unix & Linux: I think it'd be more used than Super User and ServerFault for some questions.
• Outer Join: A place to ask questions about databases and queries
• User Interface: because this is something we could all do better with.

Please consider following/committing. esp the unix/linux one (since I don't think Ubuntu should get to have its own, while leaving the rest of us in the cold.)

Managing CPAN on Arch Linux

So if you're running Arch Linux and want to install a bunch of packages from CPAN the best way is NOT to use the official cpan client, or even the new cpanminus. No your best bet is to use AUR or lacking AUR packages, create your own, and I'm gonna walk you through how I do it. The really nice thing about installing packages with Pacman is that unlike cpan there is a utility to uninstall them.

The first thing you'll want is a utility that allows you to easily manage aur. Right now the Recommended tool appears to be a CLI utility called packer you'll want to download the PKGBUILD, on the packer page, to a directory of it's own and run makepkg -s in that directory and then (as root) pacman -U pkgname-pkgver.pkg.tar.gz. You've now installed Packer, which means you'll never have to install an AUR package like that again (barring a new install of arch). If you really want you can stop now and just use the packages provided by arch, myself, and other aur contributors. The general conversion in name is is the CPAN module DBIx::Class becomes perl-dbix-class. However running packer dbix-class will find it. packer is capable of searching all of aur and the official repositories and giving you options if more than one is available. If you want to learn how to make your own packages, or just plain ignore aur and use cpanp to install read on.

So now you'll want to install CPANPLUS with packer. This is easy just run packer perl-cpanplus. This should install CPANPLUS and any of its deps. Now install CPANPLUS::Dist::Arch
with packer perl-cpanplus-dist-arch. There are other 'cpan' options but this is the only one that's on both CPAN and AUR, and it has some really nice functionality.

Now you have your entire installer stack but you still need to configure C::D::A. Run setupdistarch. This will configure cpanplus to automatically create any package it creates as an Arch pkg, it can even install them. For example packer perl-moose and cpanp -i Moose will both install Moose with Pacman. The first will use the package I've uploaded to AUR and the second will use C::D::A to create one (which is exactly what I did).

So now you know the basics of Installing packages, but I'm guessing you still want to know, how I'm managing over 200 perl packages. C::D::A is actually making this easier all the time (esp since I talk to the author). first you'll want to create a directory to store all your AUR packages, the name isn't important. So you'll want to create your first package like cpan2aur -d Catalyst::Runtime you'll note that this creates the directory perl-catalyst-runtime in that directory there is a PKGBUILD.tt it contains syntax similar to Template::Toolkit, it is however home grown by the author of C::D::A. you can use it to add stuff the the resulting PKGBUILD. To create the PKGBUILD you'll run cpan2aur perl-catalyst-runtime it will create the PKGBUILD and the tarball for AUR. However, if you're really planning on making this for AUR and no package is on AUR or you own the one on AUR you can just run cpan2aur -u perl-catalyst-runtime and it will create all the aforementioned things plus upload them with your AUR account. It's worth noting that if you attempt to run cpan2aur and a PKGBUILD already exists it will ask you if you wish to overwrite, unless you've made changes to it say yes, if you aren't updating to a new version on cpan it will ask if you want to update the pkgrel. I specifically requested that because it was annoying to increment by hand and I needed to update the PKGBUILD because of things like dep or provides changes.

Unfortunately cpan2aur cannnot generate an entire dep tree worth of directories. It's worth saying that you don't really have to run cpan2aur -d at all, you could just go right to cpan2aur -u, but then you'd miss out on the wonder that is cpan2aur --check. You can run cpan2aur --check perl-* and it will automagically check for updates of any directory that starts with perl- and upload the updates to AUR.

Unfortunately C::D::A has some limitations. The first is that it really Doesn't use Template Toolkit or even Template::Tiny (which I suggested). This doesn't seem to matter to much... but I wonder if it could help me with the other problem. C::D::A can't correctly determine optdepends because it gets the deps from the generated Makefiles. I'm not sure if this could be fixed or not, I don't yet understand this part of making CPAN packages enough.

Happy packaging.

UPDATE:

you don't need cpan2aur -u --check, cpan2aur --check implies -u

Port forwarding in OpenWRT

So you need the outside world to have access to some box's on your internal network. first you should use Static DHCP to tell static the IP's of the computer you're routing to. Then you of course need to know the inbound from the outside world and the port on the computer you are forwarding too. For this example we'll forward WAN (Wide Area Network (or (probably) the Internet)) port 8080 to a local dev box running Apache on httpd (we assume you know how to set apache up and make sure it's working on the LAN(Local Area Network)).

edit /etc/config/firewall. note: /etc/firewall.user is for manual iptables commands and is basically a shell script. I only recommend this if /etc/config/firewall doesn't do what you want (like multiport).

config redirect
        option src              wan
        option src_dport        8080
        option dest             lan
        option dest_ip          192.168.1.3
        option dest_port        80
        option proto            tcp

So options src and dest merely define the too and from interfaces (to be honest I'm not sure they actually do anything. as I'm not seeing any changes to what's in iptables). src_dport is what external port you want to listen on. dest_port is what you want it to go to. dest_ip is what ip you want it to go to and proto is the protocol it listens on (yes you have to specify separately for udp if what you're forwarding uses both tcp and udp).

After you've edited and saved the file to fit your case run /etc/init.d/firewall restart and it should work

IMPORTANT: there's a bug in 7.x and pre 8.09.2 with 2.4-bcrm kernels and netfilter code... it 'causes port drift. if the bug is affecting you. (it did me) reboot the system. you will be able to see it with a -j LOG iptables rule. the output was thus for me
IN=eth0.1 OUT=br-lan SRC=66.98.131.131 DST=192.168.1.3 LEN=44 TOS=0x00 PREC=0x20 TTL=47 ID=63450 DF PROTO=TCP SPT=54402 DPT=82 WINDOW=5840 RES=0x00 SYN URGP=0 as you can see the destination port(DPT) is 82 instead of 80 like it's supposed to be. 8.09.2 should be out in a few weeks it's currently at rc2.

Adding a new group to an open shell without logging in and out

Most Unix users believe it's impossible to add a group that was recently added to your user account without logging in and logging out. If you're running X and need to give your gui new permissions (like dolphin/konqueoror) this may be true, for the most part. I don't know how to change it for those processes. However, if you need to change it for a shell prompt it's easy. You merely execute the command newgrp in an open shell prompt and that prompt will now be loaded with the new group. I believe it also changes the default group that shell is running as. So if you run newgrp http; touch test.txt your file will be created with the group http instead of whatever your default is. You can change your group back to the default by using the sg command.

iptables browsing samba shares

NOTE: this assumes that you've read my previous post on basic iptables setup on the desktop

So I just spent the longest time trying to determine what ports I needed to browse and use samba shares. The sad answer is it's just one.


iptables -A INPUT -p udp --sport 137 -j ACCEPT


the catch with all the information I found with google was that most of it was for samba servers. I didn't want that. I just want to be able to browse my roommates network shares, and download from them. None of the information google provided suggested I need to allow the source port on the other end.

Regen2 development has officially begun

That's right I'm putting my money where my mouth is. It's small but the first 2 packages when into the regen2 tree this morning. They are MySQL 5.1.30 which was added because the gentoo maintainer refused to add it do to instability. I agree that it is unstable, but we have ~M Hard Masked packages for that reason. It has been added as ~M. The other app added is qsynergy, the ebuild was already on portage and I know it to be good so I added it to the tree, I'll probably stabilize it in a few weeks.

Since Regen2 is a fork of Funtoo you can enjoy all the benefits of Funtoo's changes in my tree, and I will try to get mine accepted into Funtoo.

In the future I will be building stable Regen2 tarballs, although only stage3's.

Finally Linux gets something first

... and it's something we didn't build. We get lots of things first but we usually build them. We never get proprietary software first. At best we get proprietary software the same day. Generally we get it months later. However, today linux takes another giant step towards being the dominant next OS, we are the first to receive 64-bit flash from adobe. Windows isn't getting it, Mac OS X isn't getting it, we are.

Why not Sabayon?

I'm going to be short and sweet on this. The last time I checked, Sabayon made you run ~arch, upgrading would break your system, and it was all about bleeding edge gui's. Sabayon is good for the last. But stay away from it if you want/need stability, or gentoo upgrades.

TCO Windows vs Linux (Part 1: a Linux Professonal)

Today fsdaily posted an forum thread asking why they should use linux but not using certain reasons. However, some of the reasons they said can't be used are part of the TCO (Total Cost of Ownership) of a computer.

What is TCO? Here's some of what Wikipedia says about it.

A TCO assessment ideally offers a final statement reflecting not only the cost of purchase but all aspects in the further use and maintenance of the equipment, device, or system considered. This includes the costs of training support personnel and the users of the system, costs associated with failure or outage (planned and unplanned), diminished performance incidents (i.e. if users are kept waiting), costs of security breaches (in loss of reputation and recovery costs), costs of disaster preparedness and recovery, floor space, electricity, development expenses, testing infrastructure and expenses, quality assurance, boot image control, marginal incremental growth, decommissioning, e-waste handling, and more.

By this definition not all computer owners are going to have the same TCO as others, as not everyone knows the same things or needs the same things.

First let's assume that the hardware costs the same. This can happen if you build the computer yourself or you get someone to do it for you.

I'm going to give examples of several types of people and what there approximate TCO is. All of these people are going to go to the locally owned and operated computer store, John's Comps. They all want the $1,200 computer that doesn't have an OS on it, The labor to install the OS is $50 for either Linux or Windows. So the preliminary cost just for parts and labor is $1,250.

Our first person, Caleb (that's me), is a system administrator and has been using Linux for 4 years, and has been using it more than windows, but doesn't want to take the time building the system himself, so I'm going to get it from John's.

I figure I have 4 choices of what OS to put on this machine since I'm a power users and will probably want all the power toys I can get. Here's what they are going to cost me in Licensing. (these are full licenses as it's a new computer and licensing isn't transferable, tax not included. list prices obtained from amazon and microsoft. )

License	XP Professional	Vista Ultimate	Server 2008 Standard	Linux
Price	$300	$340	$1000	$0
Subtotal	$1550	$1590	$2250	$1250

So right now linux is ahead for me in TCO... but what do I have to add once I get home? We'll say this is replacing my current system so it has to be able to do the same things I do on this system. This include writing office documents, security, running server daemons, and doing development.

Even on windows all the software I need to secure my system and make it useful is free. So I won't add the cost of anything other than my time to install them. I don't have any scripts that install these things for me so I have to add all the repositories (and check the box's) or download all the software, and run the executables by hand. Time is money, let's say a persons time is worth $20 an hour and I already know where to get everything I need.

I'm going to estimate 1 hour for Linux, to get everything downloaded, installed and configured since I've already got the config files. everything but the daemons and some multimedia stuff was probably already installed.

on windows I think it will take me 3 hours, It'll take 1 just to download all the software I need, including an anti-virus, anti-spyware, firefox, vim, cygwin, or msys, a codec pack for watching movies, flash, adobe acrobat, open office. All of these things are on various sites and I'll have to navigate them to get to the dowloads. I'll have to wait on some of them and I've left out a few. Some will require reboots, meaning more than one. Then we run update on the ones that have definitions files, the anti-malware stuff, that usually takes 20 minutes. Then I have to configure them, since everything is gui I can't copy most of my config files into place since many of them have the config in the registry not a text file. So assuming all goes well it takes about 3 hours to get my windows system up to optimal usability for me.

Unfortunately I find out that all the apps I used on linux aren't available for windows or are a bit different. So it takes me 4 hours to find decent replacement apps $80, and another 4 hours to learn how to use them $80, but I lose 10 hours of productivity over a few weeks changing my habits. $200 for a subtotal of $360.

Now in 6 months major updates to both systems come along. We'll say a service pack and a distro release. It takes 2 hours for me to install linux and configure it (since I always do a fresh install). It take 30 minutes for the windows service pack (neither included download time which I backgrounded, only actual time I spent working on it). So right now we are at ~3 hours $60 for linux and 3.5 hours or $70 for windows.

Within 2 days on windows I notice I'm having system crashes that seem to be related to the video card, I update my drivers 30 minutes, $80, Linux has much improved and lost no functionality and no new instability.

I realize after some more test that the updated driver didn't solve my problems with windows so I uninstall the service pack. Fortunately I know what I'm doing and uninstalling is easy for me $100. I wait a month and go through this cycle with the driver again... it takes 6 months for my video card manufacturer to make there drivers work fine with this SP, each month I tried again, I waste $220 of my time doing so, for $320 total (I actually had a problem like this when SP2 came out). Linux is up for an install again so another 2 hours, $80.

In 2 years I install Linux 4 more times getting linux up to $120, we'll estimate that I spend a total of 1 hour a month keeping linux and windows up to date. it takes me about the same amount of time to run updates in both, and make sure they went smoothly. so we are looking at another $480-$500 dollars to maintain them. So $1180 for windows over 2 years... and $620 for linux.

License	XP Professional	Vista Ultimate	Server 2008 Standard	Linux
Total	$2730	$2770	$3430	$1870

For me switching to windows.... would be almost $1000 more expensive over the course of 2 years than the cheapest version of windows I could get by with.

Over the next several parts of this series I intend to use several different types of people to see how they would get by with linux or windows on a new computer. None of them will be Linux Professionals.

P.S. If anyone knows why that there is a huge amount of whitespace above the tables I'd love to know.

The real problem with windows is not windows

(note: this was originally intended as a reply to Why Is So Hard for Windows Users to Understand That Linux Is Not Windows, but it got so long I felt it should be it's own post. I'm not sure the title is good, but it's close enough)

(note: I use the term users in this blog to mean people who use computers that do not, develop, administrate, repair, or are other wise considered power users, and professional computer users.)

I don't hate windows anymore, like Dan Craciun says they just aren't the same thing. I switched to linux years ago, I see that it isn't perfect anymore, but I still like it better than windows.

If you told me that I had to ignore the constant rebooting on windows and the lack of gaming on linux. I would suggest that I can make either system as stable, and equal in terms of usefulness on the desktop. Sure it might take me a few more hours to secure windows, get drivers installed, and install security tools, and the software I like. But in the end they'd be close to the same, in usability and long term stability for me and normal users.

The truth? the problem with windows isn't entirely windows... it's users. I could teach a user 99% of what they need to know to maintain their computer and keep it running smooth in an hour, but they wouldn't listen, they'd continue to use IE to browse promiscuously, not keep their antivirus, antispyware that I installed up to date.

If you were to change everyone running windows xp to ubuntu (I don't like ubuntu btw) today, I predict in 6 months to a year ubuntu users would have as many problems as windows users today? why? they wouldn't update there systems... they'd disable their firewalls, they would browse to all sorts of bad sites that contain the new linux malware.

How do I know this? I've put 'users' on linux, specifically opensuse, which has a nice little update checker... I show them how to use that after resolving the problems of codecs, and drivers and whatever else install problems they might have. In a week that icon is red, and will stay that way indefinitely because they never ever run updates.

So the real problem is users not treating there computers like cars, and ignoring the fact that they need proper maintenance and there are just some things you shouldn't do with them. True linux is easier for me to fix, mostly due to the lack of a registry, but the problem is not make and model, but lack of maintenance (and yes some makes and models are easier to maintain). If you don't ever get an oil change or add to your oil your engine will burn up a lot faster than it would if you had take care of it.

The one thing that M$ does that causes me to still hate them, because it affects me, is IE. If I had one thing I wanted from microsoft it would be a browser with the standards compliance on par with the likes of Mozilla Firefox (they don't have to achieve opera, or webkit) and without their own proprietary syntax, they have to do it the same way as everyone else (e.g. no active X calls instead of XMLHttpRequest..) But even if IE8 managed to be this browser, it wouldn't help anytime soon... why? because users don't update their software.

iptables for the average desktop user

The Best guide for learning the basics of iptables is here
Linux 2.4 Stateful firewall design for the most part it continues to apply to the 2.6 kernel. The only things that won't apply to your linux system will be: emerge if you aren't on gentoo, and the kernel options which have changed since 2.4 and even a couple of times during 2.6.

I'm not going to cover those here. If you need help building your kernel or installing iptables I suggest that you consult with either the iptables home page or even better your distribution. Chances are it is already installed, and may even be configured.

First Let's see if we have any rules.

The following commands require root access, and can be run in a root shell, with sudo, or in a shell script by root

iptables -L -v

your output should look something like this if you have no rules

Chain INPUT (policy ACCEPT 211 packets, 27413 bytes)
pkts bytes target prot opt in out source destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 312 packets, 211K bytes)
pkts bytes target prot opt in out source destination


If it looks different no worries it just means that your distro has already installed rules.

If so make sure BEFORE CONTINUING to flush them. FIRST check to make sure your policies are set to accept (you can see that in caps above) if they are anything else run


iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT


now that your policies are clean you should flush the rules

iptables -F #flush all rules
iptables -X #delete all chains


all right now hopefully you are at a clean state. If you have ever used iptables for nat you may not be, but that is beyond the scope of this article.

Now we need to create rules, since this is for normal deskop users you shouldn't have any services listening, also you shoudn't be routing anything.

to disable routing with iptables (I'm ignoring the kernel setting for this)


iptables -P FORWARD DROP

# set all forwarded packets to go bye bye if they reach the end of the chain

the output chain is good on accept for the normal user, only a masochist would want to write rules for it. You generally should trust your outbound traffic.

Now to secure input

iptables -A INPUT -m state --state INVALID -j DROP
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -P INPUT DROP

#this rule drops all packets with a bad state
#accept any packets that have something to do with ones we've sent on outbound
#accept any packets coming or going on localhost (this can be very important
# set all packets not matching these rules to drop

Various linux distro's may require you to do something special to save these rules so that they survive a reboot. consult your distributions community.

If you have rules that you want to keep but also want to use mine I suggest putting mine first, you should do the -A rules in reverse but use a -I for insert.

When dealing with iptables always take care when applying iptables -P DROP. You can be locked out of the machine or the internet if you apply this without the appropriate rules in place.

That's it. your desktop should be secure from an attacker that you aren't allowing in. There are of course other things that you can do to make it even more secure but those are beyond the scope of this tutorial.

really nice

if you are using the cfq io scheduler in linux you can use ionice to nice your io. so to really nice something you would do


nice -n 19 ionice -c 3 programname


this may not be the best idea on all programs but it works. If a program is beating your hd and cpu and you don't want it to run this when starting it.

ReGen2 Linux

I'm starting a blog for my thoughts on forking Gentoo Linux here

New version of SELinux released

also see the selinux home/news page http://www.nsa.gov/selinux/news.cfm

Unix Toolbox

Found this Unix Toolbox on Digg today. Most complete list of commands for Linux/BSD/UNIX that I've ever seen. I could only think of one missing off the top of my head. ionice.

What Open Source Operating System's still need

I'm not going to mention things that are beyond the control of open source such as drivers and multimedia. These things are the fault of the corporations that control them.

The first thing I think these systems need is a unified password manager. Instead both gnome and kde have there own password storage system and so do a million other apps. I think OpenBSD should lead the pack on this one given there reputation for security. This would be for the storage of passwords use to access remote systems, on either the command line or gui. Of course the application designers would still probably want to build there own front end but hopefully we could be assured that the information was secure.

The second thing I've been thinking of is a FHS standard for home directory dot files. God I've come to hate dot files in recent months. I recently wanted to put my $HOME directory on version control. Including the dot files. Then I realized that many of them were really only temporary files that were session specific, or they contained binary data etc, etc, basically things I can't shouldn't or don't want to version. I can't possibly keep track of all the dot files I'd like them to be organized in sub dot files now. with config files in .etc, var like data in .var, and temporary data in .tmp etc.

EDIT: a bug has already been filed on the FHS standard and can be found here dot etc, the users private etc directory proposal