Useful Shell Trick
Sometimes, when doing a bunch of data munging in the shell, I want to pass the contents of a file through a shell pipeline, and then directly out into the file again. Unfortunately, doing this the naive way will obliterate the file's contents.
So a while back, I saw a neat little utility called "sponge" which simply buffered data until it ended, and then output it. While useful, this wasn't a standard Unix tool, so I couldn't very well rely on having it available.
Well today, after literally minutes of searching, I have found a command that seems to work just as well:
It outputs all lines in the input, beginning with line zero. Due, I suppose, to the way tail is implemented, it buffers all input before outputting anything.
Not exactly groundbreaking or anything, but a useful little trick to know when playing around in the shell.
What the Hell, XDG?
Take a look at the XDG Base Directory Specification. It looks good, doesn't it? It really makes sense to have some "official" directories to store this kind of stuff in. Now look at the default locations listed:
$XDG_CONFIG_HOME - Looks like they nailed that pretty well. I might rather see some user-editable config files placed directly in '~/.app/', but '~/.config/app' is alright as a second choice.
$XDG_CACHE_HOME - Dead on. It is incredibly nice to have a single home for all those nasty little cache files that are a little too persistent to place in '/tmp'.
$XDG_DATA_HOME - Okay, it's a little bit ugly to store extra data in '~/.local/share', why couldn't they just put it in '~/.local'? But on the other hand, most applications don't even need extra data of that sort anyway, and it will rarely, if ever, be edited by a human, so it's at least a little bit acceptable.
$XDG_DATA_DIRS - Yeah, they're back on form for this one. Those two default directories cover basically every app in existence, so not a single fault can be found there.
$XDG_CONFIG_DIRS - WHAT THE FUCK! Why the hell do they decide every application needs to store their system-wide configuration in a special '/etc/xdg' directory? On my system, I count a grand total of three applications in there, of which I use one and have configured zero. For comparison, I count 157 files and directories in '/etc', other than '/etc/xdg'. Obviously, people will kind of expect to see their system-wide config files placed directly in '/etc'
What reason can there be for that kind of annoyance? When I have to go in and manually edit a system-wide config file, I will look for it in '/etc'. It would never even have occurred to me to look in '/etc/xdg'. At the very least, couldn't the default search path have been '/etc/xdg', followed by '/etc'?
This kind of trivial little issue weakens the entire specification, because such a seemingly inconsequential little rough edge can completely sour people's opinions of the whole document. And it would be a shame if that happened, because I would rather see the majority of these things become common convention.
Escaping Strings in a Shell Pipeline
A useful little sed script I wrote, which turned out to be much more useful than I expected. It simply takes a list of files, one per line, and outputs that same list, but with all the spaces and special characters escaped. It's mainly intended for use right between find and xargs, as in "find . -name 'asd' -print | shell-escape | xargs -I % echo %'
The list of characters is taken from IEEE Std 1003.1
#
# Shell-Escape
# A very basic script to escape characters in the shell.
# Mainly intended for use in shell pipelines, right between
# 'find' and 'xargs'
#
s@\\@\\\\@g
s@ @\\ @g
s@'@\'@g
s@"@\\"@g
s@(@\\(@g
s@)@\\)@g
s@|@\\|@g
s@&@\\&@g
s@;@\\;@g
s@<@\\<@g
s@>@\\>@g
s@`@\\`@g
Haskell: A Pretty Nice Language
I was going through one of those "every language sucks" periods a few days back, so I decided to do something productive and learn me a Haskell. So far, I'm liking it: the pure functional style isn't as hard as I expected, monads are okay after a bit (though I'm still a little suspicious of them), the strict typing isn't too much of a burden, the polymorphic types are bliss to use, and the pattern matching is simply wonderful. It also comes with an extensive set of libraries and a central repository, which is nice to find in a non-mainstream language. Overall, I think Haskell is a good language to know.
A Few Handy File Transfer Tools (All Written in Python)
File transfer: the eternal problem.
The file is on your computer, you want to get it onto your friend's computer, and you're in a hurry. Luckily, there are tools to help. All of these tools are single files, written in python, and (probably) work on Windows (although I haven't checked)
Single-File Tools
Droopy
Droopy is a wonderful little script. It's wonderfully useful when your friend has a file they need to send to you, and you don't want to make them install anything. Just run it (possibly with some arguments) and it will start a little miniature web-server to which anyone can upload a file, and then it exits. Command line options make it possible to optionally display a message and/or an icon on the upload page.
Woof
Woof (Web Offer One File) is the opposite to Droopy. Instead of recieving a single file, Woof sends one. It can also optionally send an entire folder as a single tar archive. Another nice feature of woof is its "-s" option, which will cause the client to be served an identical copy of the woof.py script.
Heavyweight Contenders
These tools are meant for jobs a little larger than the ones above. The tools already mentioned concern themselves with a single all-important file transfer, which is all well and good. But like the difference between claw hammer and a sledgehammer, sometimes you just need a tool with a little more heft. That's where these scripts come in.
Python HTTP Server
This tool is only barely fancy enough to need mentioning, but the operative word here is "barely". Unknown to almost everyone, a full-featured static HTTP server lurks within the standard python executable under the guise of an "example". All that you have to do is call "python -m SimpleHTTPServer" in a console window, and it will happily begin serving up the current directory on port 8000.
pyftpdlib
All of these scripts are wonderful tools, just perfect for their intended niches. But sometimes, your needs are more complex. What if you need to both send and recieve multiple files? In this case, our old standby FTP comes to hand. In this case, the pyftpdlib library should be our tool of choice. While at first glance it looks rather daunting, a tarball filled with numerous files and subdirectories, the library itself consists of just one file, ftpserver.py. With a few modifications, this file can be the perfect portable, install-less FTP server. I spent about an hour today putting together these changes, and now have about the nicest little FTP server I've ever had the joy of playing with. Just download the FTP server library, replace the test() and main functions at the end with my 50-line changes (basically just giving it command-line options), and you'll have a user-friendly tool for sharing any kind of files you may need.
I love collecting little tools like this. One never knows when they might come in handy.
It's Alive!
Wow. From idle complaints to a working implementation in 12 days. And most of those days I did exactly no work on this project either. I'm surprised.
Long story short, it turns out it's actually not that hard to write a parser, and a lexer is even easier to do if you just use a preexisting regular expression library.
So now I can officially announce the existence of the Helicon Parser Toolkit. It's small, it works for my purposes, and it isn't Lex/Yacc. I don't know what the official classification of the parser is, I just did whatever seemed like a good idea at the time. It has no lookahead currently, but since I just wanted to parse Scheme, that's fine by me.
The lexer and parser together weigh in at 438 lines of code (not including the two header files), and all of it except for about five functions is just bookkeeping.
The lexer uses PCRE for its regular expression matching, and the parser is entirely coded by hand.
As far as Valgrind can tell me, there are no memory leaks other that what GNU Readline brings to the table (maybe this is my fault and I'm using it wrong. Will check later.)
It seems to work so far.
Long Time, No Post
Until last Sunday, I hadn't posted anything recently. Mostly, this is because I got distracted by Super Mario 64 two weeks ago, and went directly on to Paper Mario 64 the week after that. Partly, it's because I experienced a sudden upsurge in my homework load two weeks ago, which took until yesterday to finish all of. A little bit can be blamed on college applications, and many other little things that ate into my time. But I think a large part of the problem was that the first few posts I wrote on this blog were rather long, and I felt something of an obligation to keep up the quantity.
So. Change. Shorter posts, but posts nonetheless. Hopefully now I'll be able to get myself to write more often.