My $PS1 is a creature evolved over the years, it amazes me to consider how much accumulated time I must have spent on this simple little critter. For the last couple of years it’s been pretty stable as the simple “u@h:W$” with some occasional colour. I’ve been through phases of executing commands in my $PS1 to add further data (don’t, it fucks with $?) and I have a custom $PROMPT_COMMAND to issue a terminal control sequence to put similar info in titlebars (where I execute a $(date …) to add a timestamp, this doesn’t bother $? though).

Yesterday I noticed a recurring problem… I keep loosing a $? that I’m interested in. Often because I go to another terminal and do something else while something executes, then I return and do something silly like “automatic ls“. Does anyone else do “automatic ls” I wonder, it’s where you run ls for no particular reason — I tend to do it every time I switch to another terminal, also if I’m just looking at an existing terminal and thinking my fingers seem to have a “background ls” function. It’s like looking around the room I guess.

Anyway, back onto the point. This little problem prompted me to redo my prompt and along the way find a better way to do the terminal title:

    ps1xt='[33[1;35m]$?[33[0m]' # Last exit code, magenta
    ps1us='[33[1;32m]u[33[0m]' # Current username, green
    ps1mc='[33[1;36m]h[33[0m]' # Current hostname, cyan
    ps1wd='[33[0;32m]w[33[0m]' # Current working dir (w is full, W is basename), dark green
    export PS1="[33]0;<$?>u@h:w(t)07]<$ps1xt>$ps1us@$ps1mc:$ps1wd$"

So, no more $PROMPT_COMMAND as the control sequence is embedded in $PS1 now (everything within the first […] group). This gives an added bonus of being able to simply use $PS1 substitutions for all the info. The only real difference between this prompt and what I had before is the addition of the <$?> at the start, so now my exit codes will always been in the terminal backlog! (Until I do something else automatic like a ^L.) Also added the $? to the title bar, which works well since I can see the code when I’m in another window frame this way.

One possible annoyance is the extra length of the prompt, the last thing you want is a prompt taking up overt character width. Two possible reductions would be to” 1) remove the u part, how often are you not sure who you are? 2) Make w a W so the CWD part can’t get too long so easily. Given the colour differences you could also remove the punctuation I guess, save yourself 4 chars. You could also stick a n in before the final $ and make a two-line prompt I guess, I used to have one like that (a long time ago when I first discovered PS1 and put everything but the kitchen sink in there).

A lot of people will hate the colourfulness, I hate it myself sometimes. The main function is that I use different colours for the h on different machines so I can very quickly recognise which machine I’m looking at. The rest of the colour is just for the sake of being garish.

<0>yseth@odysseus:~$ls -e
ls: invalid option -- e
Try `ls --help' for more information.
<2>yseth@odysseus:~$ls -e

“:;”? OR Re: Re: Prompt Insanity

Really, one of these days I might do something about comments.

Sometimes I do get upset at my long and garish prompt and blast it with PS1=’>’.

I hit the good old ‘arg list too long’ fairly often, dealing with very large HTTP corpora. I’m rarely in a corpus directory though so this has yet to break my auto-ls habit.

As for other shells, I’m not diverse enough in my shell usage to consider the world outside of bash. I would have thought that .bashrc’s PS1 setting wouldn’t matter there.

Finally, I pretty soon realised that embedding my usual PROMPT_COMMAND into PS1 has some occasional issues – it isn’t “magically invisible” when you have a console login … so now I have made it conditional in a $TERM case…esac
block.

For the “good” of the web:

    ps1xt='[33[1;35m]$?[33[0m]' # Last exit code, magenta
    ps1us='[33[1;32m]u[33[0m]' # Current username, green
    ps1mc='[33[1;36m]h[33[0m]' # Current hostname, cyan
    ps1wd='[33[0;32m]w[33[0m]' # Current working dir (w is full, W is basename), dark green
    case x$TERM in
    xxterm|xrxvt)
        ps1pc='[33]0;$?:u@h:w(t)07]'
        ;;
    esac
    export PS1="$ps1pc$ps1xt:$ps1us@$ps1mc:$ps1wd$"

For random primates, such as myself, friendly spiders and assorted maladroit suckers of all the Internet’s most rank drivel must represent near 100% of our readership. Since, in truth, most people are little better than the few lines of code behind my most frequent website visitors I bid you all welcome. You’re welcome to be my friends Googlebot, Baiduspider, Gigabot, TurnitinBot, Zeusbot, msnbot and the other 65 or so eaters of my robots.txt I have seen in the last year. But those of you who shun my robots.txt, especially those of you lacking decent user-agent strings, can crawl back into your dingy holes with the slugs and worms (I’m looking at you: bots from EZRSSFeeds, WebSense (Konqueror my arse) and other houses of deception). Alas for you, even these clammy denizens of dank and musty places will probably shun your presence.

One of your number seems to have more in common with the leech than any other form of life. To me this nefarious creature appears to propose: “I’ll make it easy for you to steal content to put on your website to fool Google into thinking you actually have content of your own.”.

Highlights:

• No mention of copyright or content ownership on the site, none that I can find.
• The “spider” page doesn’t tell you about the spider employed, it tries to sell you some kind of “spider”.
• The bot grabs RSS with high regularity. (>30 hits in the last 8 days.)
• The bot doesn’t advertise its self via user-agent, it doesn’t send a user-agent string at all. (But it’s IP reverses to the domain name: 147.202.50.50)
• I’m guessing here, but I bet the bot pays no attention to robots.txt! (The IP above started hitting RSS on my site in September 2006 and has never requested the robots.txt file).

I’m blocking the little bugger’s IP now, for general bad behaviour and likely evilness… but that’s only effective up until it starts crawling with a different IP. In truth, if you put stuff on “the Web” there isn’t any way to protect it, consider it “fair game”. With just a little work this bot could be made much harder to identify, since you’re already behaving in a questionable way why not start employing bot-nets to do the surfing, and use some legitimate UA strings! You’re a dumb bot! As a friend of mine might say: no bot-biscuit for you! I think there is a viewpoint floating around that sees providing an RSS feed as permission to play free and easy with the content. People who write weblogs are essentially attention whores so any distribution of their content must be a good thing in their eyes, right?

Now, to some squishy human life-forms: If you’re considering using the service associated with this bot, or anything similar, you might want to consider potential copyright implications. It might be fine, maybe it just provides excerpts and properly references the source, or maybe not. Like I said, their website makes no mention of copyright and their bot doesn’t identify its self, this is incriminating behaviour in my opinion. If it is legitimate why doesn’t it do the right thing?

Alternatively, just write some bloody content you poop fairy.

To the leeches: My apologies if I offended you.

Back to the good bots: Goodnight my friends.

One difference between Sydney and England is that you’ll find a lot of turkey in the supermarket here; whole turkeys, turkey breasts, turkey escallops , turkey drumsticks, and turkey mince… the list goes on. By comparison turkey is rather rare in Australia, I wonder why. As a result I’ve never really thought of cooking it much – in my mind turkey is one those things that traditionalists had for Christmas dinner (I don’t recall having it myself, but we probably did one Xmas or another).

It turns out that turkey actually makes for a decent meal, it has a distinctive and pleasant taste. What’s more it can be very low in fat (lean breast) and high in protein, it also contains a smattering of beneficial vitamins and minerals. Turkey mince tends to have a similar fat content to beef mince because it usually has skin/fat minced in with it, but good turkey breast mince will have less fat than typical lean beef mince. It is mince I’m working with today (2006-11-19), I decided to try a bolognaise style pasta sauce using turkey mince (those who know my various renditions of bolognaise will recognise that this is a pretty broad “style” in my world of bog).

So, without further adieu, here’s the ingredients:

All fine chopped:

• 2 – Medium Brown Onions
• 1 – Red Pepper
• 5 – Medium Chestnut Mushrooms
• 3 – Small Carrots
• 2 – Sticks of Celery
• 4 – Cloves of Garlic
• 2 – Small Hot Chilies

The rest:

• 500g – Turkey Mince
• 1 tbsp – Light Olive Oil
• 1 tbsp – Dried Oregano
• 2 stars – Star Anise
• 1 tbsp – Plain Flour
• 1 tsp – Ground Cinnamon
• 1 tsp – Ground Coriander Seed
• ½ cup – Dry Wine
• 2 tins – Chopped Tomato (Organic if you can get it)
• 1 litre – Beef Stock

Seasoning:

• 4 tbsp – Fine Chopped Parsley
• Pecorino Romano (or similar)
• Extra Virgin Olive Oil
• Salt/Pepper

Heat the oil in a decent sized saucepan (about 28cm wide by 28cm high in my case), when hot put in the turkey mince a bit at a time to break it up. Cook this on high heat until dry and browning, it’ll take a while to evaporate out the water content. When slightly browned add the wine and dry herbs and spices, again cook this until the liquid is evaporated.

When all excess liquid is evaporated add in the chopped onion, garlic and chillies. Cook until onion becomes translucent then put in the rest of the vegetables. Cook this, tossing often, for about 5 minutes.

Sprinkle on the flour (this works as a thickening agent) and toss through the mixture then add the tinned tomato and stock. Amount of stock may need adjusting, the liquid should cover about an inch over the settled solids. Mix well and reduce heat so that the liquid is just simmering, let simmer in this way for about an hour – stirring every now and again.

When the hour is up take off the heat and boil an appropriate amount of spaghetti (we used a very nice wholemeal spaghetti). Stir most of the fine chopped parsley into the bolognaise and serve on top of spaghetti, sprinkle with remaining chopped parsley, freshly grate some percoino romano over it, grind on some black pepper and drizzle with some good extra virgin olive oil.

Gobble gobble gobble!

This recipe created us two large sized serves with two more in the fridge for another day.

My lordy, it’s just like Uni message boards 🙂 And as usual Sam knows more.

At this time of the night after getting back from the local I’m not going to try to better any of that, and since it’s Sam it is probably foolish to try. Some points didn’t happen to apply to my “just now”, it’s is lucky my filenames had the decency to not contain spaces. Learnt something new about awk there, I fear my main weakness is sticking to the little I know until told off and given a better alternative! As for fmt, I didn’t know of an autoconf macro for getting to the thing (maybe it is so ubiquitous I shouldn’t need one), awk seems to be the defacto standard but these days bombing countries to death and sending in troops seems to be the defacto standard for making the world a better place, and nobody in their right mind actually thinks autoconf doesn’t kill puppies in its spare time.

Peter Weinberger? I never said he was the one I didn’t recognise! Sounds like a sausage in a bun to me… and means about as much 😉 sam I am… not, actually, but Sam is so that’s okay I guess.

Shell scriptsi/aliases that do the job are so good, they save my poor old fingers, mine’s strangely called count and will soon probably be called add for both it’s accuracy and finger saving properties (I also have a count.pl that strangely does something completely different).

So much to learn…

I use awk a lot in my day to day work.

Just now: Do a subversion move of all files in the cwd to uppercase file names (I’m not going to explain why):

ls | awk '{u=toupper($1);if(u!=$1){system("svn move "$1" "u);}}'

And regularly I do things like: Get average size of files in a corpus:

find ./corpus/ -type f -exec stat {} ; 
        | grep 'Size:' 
        | awk '{c+=1;d+=$2}END{print d/c/1024 " kB"}'

Does it ever annoy you that grep doesn’t offer a total sum of matching lines?

grep -rihc 'foo' ./corpus/ | awk '{c+=$1}END{print c}'

Or that cut is only really useful with single char delimiters? The default awk delimiter is s+, so the second example above works as expected (and you can override the field separator with -F, even with simple regular expressions).

I’ve written some largish bits of code in awk before, it isn’t that far off using the likes of perl, but I’d generally recommend using more modern options for larger scripts[1] A sample of something I did in a fit of taking cross-platform too far[2] was to create an autoconf macro around this little monkey:

echo "The rabbit-hole went straight on like a 
tunnel for some way, and then dipped suddenly 
down, so suddenly that Alice had not a moment 
to think about stopping herself before she found 
herself falling down a very deep well." 
| awk -v width=50 -v pre='* ' '
BEGIN{
    line=pre;
}
{
    for (i = 1; i <= NF; i++)
    {
        if (length(line)+length($i) > width)
        {
            print line;
            line=pre;
        }
        line=line$i" "
    }
}
END{
    print line;
}'

I’ve modified it to make it into a hideous one-liner. In the original form it is an autoconf macro where the echoed string, the “pre” and the “width” are arguments. All to make failure messages that little bit neater.

What exactly does it do? Wraps a single line of text to a given width with a given prefix (in the process turning lumps of whitespace into single spaces), in this case the output is:

* The rabbit-hole went straight on like a tunnel 
* for some way, and then dipped suddenly down, so 
* suddenly that Alice had not a moment to think 
* about stopping herself before she found herself 
* falling down a very deep well.

Thank you awk (in fact in my particular case I should thank mawk).

There are plently of awk examples and docs out there.

[1] I don’t mean to slight awk too much, it is a complete programming language and is quite easy to work with. It’s been around since 1977 and was designed and written by Alfred Aho, Peter Weinberger and Brian Kernighan (you should recognise at least two of those names!).

[2] You never know, there might be a system out there somewhere without perl installed. The project only compiles on Linux? Hrm, but we have to be prepared for the future!

My recent experiences with Firefox 2.0 are less than ideal, I’m not using the latest release I’m using the version in Ubuntu’s Edgy. I’ve been checking apt for updates since upgrading and have been disappointed so far (I’d really prefer not to have to go outside the distro, going down that path one may as well roll-their-own).

My Firefox 2.0 keeps freezing, maybe it is one of my plugins doing the evil? I have no way of knowing… this is most upsetting. I got used to a Firefox that didn’t crash several times a day (unless you used a Java plugin in which case you’re on your own against some bodgey Java developer) sure it used huge wads of memory and strange amounts of CPU but I can live with that if it doesn’t bloody crash.

Finally we have upgraded from an underpowered little VPS to a real piece of HW with the additional bonus of lower ping times. We have a “km1000” from KeyWeb.de (which I notice is already out of date, it’s a pity that the slightly cheaper km800 wasn’t available a month ago!), essentially the specs are 2.66 GHz P4 with 0.5GB of RAM and the difference in usability between this and the old VPS is huge.

Generally I suspect our problems with the JVDs VPS may have been related to them packing more VPSes onto one system; this probably happened as soon as they were bought out by a larger company a while back.

The changeover has also given me a chance to do a few nice things I’ve wanted to get around to for a while, like setting up services such as Apache and bind in chroots. Itemising the improvements:

• VPS -> BigIron
• 128MB RAM -> 512 MB
• 3 GB Disc (200MB free) -> 73 GB (65 GB free!)
• who knows shared CPU -> 2.66 GHz
• 90ms ping -> 35 ms ping (joy!)

The RAM is one of the more important bits and that’s just because of bloody spam, our spam filtering is easily the most memory hungry thing we run and along with all the other services we were right at the 128MB limit on the VPS and wanted to push more features into anti-spam but couldn’t (and doubling the RAM on the VPS would have more than doubled the cost!). The dedicated machine is a lot more expensive than the VPS. The JVDs VPS was 20 USD per month, this server is something like 50 EUR a month – around 65 USD but that’s not too bad considering most US offerings are pegged at 99 USD and UK offerings are around 50 GBP.

Good stuff.