#!/usr/bin/env ruby
#
# torrentdump - prints the contents of a torrent.
#
# Copyright (C) 2011 Paul Legato. All rights reserved.
# Licensed for personal, non-commercial use only.
# This code comes with NO WARRANTY, express or implied.
#
# Prerequisite gem setup:
#   sudo gem install bencode awesome_print
#

require 'rubygems'
require 'bencode'
require 'awesome_print'

unless ARGV[0]
  STDERR.puts <<END

Usage: torrentdump <filename> <-v>

If -v is given, prints the entire contents of the torrent except the binary "pieces" hash data.

If -vv is given, prints everything, including the binary hash data.

Otherwise, prints only the file list.

END
  exit 1
end

# Method from http://stackoverflow.com/questions/3201890/is-there-an-elegant-way-to-remove-a-specific-key-from-a-hash-and-its-sub-hashes
def recursive_delete!(hash, to_remove)
  hash.delete(to_remove)
  hash.each_value do |value|
    recursive_delete!(value, to_remove) if value.is_a? Hash
  end
end

data = BEncode.load(File.read(ARGV[0]).force_encoding("binary"))

if ARGV[1] == "-vv"
  ap data
elsif ARGV[1] == "-v"
  recursive_delete!(data, "pieces")
  ap data
else
  if data["info"]["files"]
    dir = data["info"]["name"] || ""
    ap data["info"]["files"].map {|x| dir + "/" + x["path"].join("/") }
  else
    ap data["info"]["name"]
  end
end

A quick example:

% torrentdump ubuntu-10.04.2-server-i386.iso.torrent
"ubuntu-10.04.2-server-i386.iso"

Torrent dissection: of interest here..

• Ruby 1.9 imposes a character encoding on every string by default. This is fine for dealing with text, but caused errors in the BEncode library. (For some reason, the authors of BitTorrent chose to re-invent the wheel and developed their own binary encoding format.) We use the #force_encoding method on the torrent data to make Ruby treat the data as raw binary data and not try to interpret it.
• More than you ever wanted to know about what’s inside a .torrent file

The Torrentdump source code is available on GitHub.

They’re not “stories”. They’re tests. They don’t tuck you into bed with a glass of warm milk and a nice fairytale to dream about; they assert that certain things should happen and complain when they don’t, in a purely mechanical and precise way. We’re programming computers, machines of pure logic here, not reading to children. Let’s dispense with the pointless cutesy façade and focus precisely on the task at hand.

They don’t deliver what they promise, anyway. Frameworks like RSpec and Cucumber do not let you write tests in “plain English”. They let you write tests in a horrible chimeric programming language that adopts a superficial English-like veneer over top of what’s essentially a regular old unit testing framework.

It winds up giving you the benefits of neither English nor a well written testing API; your code simply becomes excessively verbose for the sake of a (highly fake and brittle) natural language veneer. You’re still highly restricted in what you can say, just like a regular old programming language. The set of Englishisms that you can use is so grossly restricted and precisely defined that you can’t think in anything remotely like English. You have to memorize a precise set of rigid, fixed English labels and use that and only that. You do get the verbosity of natural language, though, along with the restricted semantic domain of the API. You get the benefits of neither and the disadvantages of both.

Since you’re memorizing a highly restricted API anyway, skip the twee cruft and just use a short and precise API from the start, without the pseudo-natural language clutter.

Cocos2d makes it easy to convert your drab standard UIAlertViews into glorious customized color dialog boxes. Above is an actual UIAlertView / custom dialog box transition from my upcoming iPhone game. Here’s how I did it.

Implementing a Custom Dialog Box in Cocos2d

Cocos2d Scene Structure

Cocos2d uses a scene graph to hold everything being displayed at any given point in a game. Informally, this data structure is a hierarchy of objects, each of which might display something onscreen, contain other objects to be displayed onscreen, or both. In Cocos2d, these objects are descendants of the CCNode class.

Related objects can be placed into a single container object, which allows them all to be manipulated as a unit when necessary. The root, or lowest-level, container at any given time is an instance of the CCScene class. A CCScene contains CCLayer objects, which themselves contain other objects such as menu labels or sprites to be displayed. Layers, as the name implies, are displayed superimposed upon each other onscreen, which makes a CCLayer perfect for our custom dialog box.

Custom CCLayer

Here’s the code for DialogLayer.h and DialogLayer.m. These implement a basic but perfectly servicable custom iPhone dialog box. It can be used as-is or expanded in any number of ways for particular situations. Take a look through the finished product, and then we’ll go over it below.

DialogLayer.h

//
//  DialogLayer.h
//  concentrate
//
//  Created by Paul Legato on 12/4/10.
//  Copyright 2010 Paul Legato. All rights reserved.
//

#import <Foundation/Foundation.h>
#import "cocos2d.h"

@interface DialogLayer : CCLayer {
  NSInvocation *callback;
}

-(id) initWithHeader:(NSString *)header andLine1:(NSString *)line1 andLine2:(NSString *)line2 andLine3:(NSString *)line3 target:(id)callbackObj selector:(SEL)selector;
-(void) okButtonPressed:(id) sender;

@end

DialogLayer.m

//
//  DialogLayer.m
//  concentrate
//
//  Created by Paul Legato on 12/4/10.
//  Copyright 2010 Paul Legato. All rights reserved.
//

#import "DialogLayer.h"
#import "GameSoundManager.h"

#define DIALOG_FONT @"bicho.fnt"

@implementation DialogLayer
-(id) initWithHeader:(NSString *)header andLine1:(NSString *)line1 andLine2:(NSString *)line2 andLine3:(NSString *)line3 target:(id)callbackObj selector:(SEL)selector
 {
   if((self=[super init])) {

     NSMethodSignature *sig = [[callbackObj class] instanceMethodSignatureForSelector:selector];
     callback = [NSInvocation invocationWithMethodSignature:sig];
     [callback setTarget:callbackObj];
     [callback setSelector:selector];
     [callback retain];

     CGSize screenSize = [CCDirector sharedDirector].winSize;

     CCSprite *background = [CCSprite node];
     [background initWithFile:@"Dialog.png"];
     [background setPosition:ccp(screenSize.width / 2, screenSize.height / 2)];
     [self addChild:background z:-1];

     CCLabelBMFont *headerShadow = [CCLabelBMFont labelWithString:header fntFile:DIALOG_FONT];
     headerShadow.color = ccGRAY;
     headerShadow.opacity = 127;
     [headerShadow setPosition:ccp(243, 262)];
     [self addChild:headerShadow];

     CCLabelBMFont *headerLabel = [CCLabelBMFont labelWithString:header fntFile:DIALOG_FONT];
     headerLabel.color = ccBLACK;
     [headerLabel setPosition:ccp(240, 265)];
     [self addChild:headerLabel];

     //////////////////

     CCLabelBMFont *line1Label = [CCLabelBMFont labelWithString:line1 fntFile:DIALOG_FONT];
     line1Label.color = ccBLACK;
     line1Label.scale = 0.84f;
     [line1Label setPosition:ccp(240, 200)];
     [self addChild:line1Label];

     CCLabelBMFont *line2Label = [CCLabelBMFont labelWithString:line2 fntFile:DIALOG_FONT];
     line2Label.color = ccBLACK;
     line2Label.scale = 0.84f;
     [line2Label setPosition:ccp(240, 160)];
     [self addChild:line2Label];

     CCLabelBMFont *line3Label = [CCLabelBMFont labelWithString:line3 fntFile:DIALOG_FONT];
     line3Label.color = ccBLACK;
     line3Label.scale = 0.84f;
     [line3Label setPosition:ccp(240, 120)];
     [self addChild:line3Label];

     CCMenuItemImage *okButton = [CCMenuItemImage itemFromNormalImage:@"OKButton.png" selectedImage:@"OKButtonSelected.png" target:self selector:@selector(okButtonPressed:)];
     [okButton setPosition:ccp(0, 60)];

     CCMenu *menu = [CCMenu menuWithItems: okButton, nil];
     menu.position = ccp(240,0);
     [self addChild:menu];
   }
   return self;
 }

-(void) okButtonPressed:(id) sender
 {
   [[GameSoundManager sharedManager].soundEngine playEffect:@"OK.caf"
                                    pitch:1.0f
                                    pan:0.0f
                                    gain:1.0f];

   [self removeFromParentAndCleanup:YES];
   [callback invoke];
 }

-(void) dealloc
 {
   [callback release];
   [super dealloc];
 }
@end

Custom dialog box code analysis

Overview

The dialog box is implemented as a subclass of CCLayer. An initWithHeader method is provided to initialize the layer with a given message and a specified callback method that will be invoked when the dialog’s “OK” button is pressed. It also plays a confirmation sound and switches the “OK” button’s image to a “pressed button” when the user taps it.

DialogLayer.m requires a few external supports, which must be available as project resources. Dialog.png is the dialog box’s background image. bicho.fnt is an AngelCode format bitmapped font, used for the labels, and can be substituted with any font you like. OKButton.png and OKButtonSelected.png are the images to be used for the button in unpressed and pressed forms. Finally, OK.caf is the audio file to be played when the user taps the button.

Analysis

Storing the callback

     NSMethodSignature *sig = [[callbackObj class] instanceMethodSignatureForSelector:selector];
     callback = [NSInvocation invocationWithMethodSignature:sig];
     [callback setTarget:callbackObj];
     [callback setSelector:selector];
     [callback retain];

This code is Objective C’s (massively redundant) way of storing the callback to be activated when the dialog’s button is pressed. It’s based on the callback code in Cocos2d itself. (We could also have used Objective C’s new closure data type, called “blocks”, but that syntax grates my Lispy sensibilities even more than the overly verbose Javaish mess here, besides not working out of the box on older versions of iOS.)

The initWithHeader method is given a target object and a method selector. It looks up the class of the target object and then looks up the selector on that class at line 1, resulting in a NSMethodSignature object which encapsulates the class-method combination. Line 2 transforms that class-method signature into an “invocation,” which represents the act of actually calling that method on some as yet unspecified object. Line 3 associates the invocation with the particular target object we were given, and line 4 associates the selector (which is utterly odd, since the desired method is encoded in the signature object already.) Finally, we retain the callback so that we can save it for later (to be activated when the user actually presses the “OK” button) and it won’t be deallocated at the end of the present scope.

Setting up the background and labels

The remainder of the initWithHeader method, lines 25 to 68 in the main listing above, is pretty straightforward. A CCSprite object is created to store the background image and added to self, that is, to the new custom layer being created. Its position is set to be the center of the screen, and its level on the Z axis is set to -1 so that it will appear below anything else added in the layer.

CCLabelBMFonts are then created for the given text and added to the new layer. A shadow effect is created for the header by writing the same thing in grey at a slight offset underneath the main text. All lines except the header are then scaled down to 0.84 of their natural size, so as to enhance the prominence of the header.

A CCMenuItemImage is finally created to be the “OK” button. It’s added to a CCMenu, which handles the logic of button presses for us, and the CCMenu is then added to the new layer. It will take a little experimentation to get the exact placement of the OK button correct (lines 64 and 67) relative to your background image.

Note that the OK button itself takes callback arguments of exactly the same type that our initWithHeader method does (line 63). We might be tempted to forego the construction of our own callback object and just pass the given target and selector on to the OK button, simplifying the design of our DialogLayer a bit, but then we wouldn’t have the chance to play a sound when the OK buton is pressed. More seriously, we would have no way to destroy the dialog layer and remove it from the screen. So, we give the OK button a target of self and tell it to call the okButtonPressed method when OK is pressed.

The okButtonPressed callback

The CCMenuItemImage will invoke this method when the user presses “OK”. This means that it’s time to remove the DialogLayer from the scene (easy enough, line 80). Here, we also play a little click-type sound to make the game seem more interactive; also easy with CocosDenshion, lines 75-78. If you don’t want to play a sound, you can remove those lines and the GameSoundManager.h include file.

dealloc

Nothing fancy here; we just release our retained callback object and then defer to super.

Using the custom DialogLayer

Adding a custom dialog layer to your game scene is very easy:

   CCLayer *dialogLayer = [[[DialogLayer alloc]
                            initWithHeader:@"Level 1"
                            andLine1:@"Tilt The Phone"
                            andLine2:@"To Balance"
                            andLine3:@"The Ball!"
                            target:self
                            selector:@selector(beginLevelOne)] autorelease];
   [self addChild:dialogLayer z:200];

Future Expansions

DialogLayer is a simple modal dialog box that can only accept one action from the user. It can easily be enhanced in many ways, depending on your needs. The most obvious enhancement is to add multiple buttons to allow the user to make a choice rather than simply dismissing the box. It could be modified to accept an arbitrary amount of text and divide it up automatically into an aesthetically pleasing configuration.

One thing that DialogLayer does not do is pause the game. I decided that that is not necessarily desirable in all cases; many game scenarios need to present information to the user without stopping the action. It’s easy enough to pause manually when activating the dialog.

Little enhancements like this are critical as ‘polish’ for your game.

This HowTo tells you what software you need and how to do it on your Mac.

Recording projectM Videos to Disk

1. Install Jack OS X. Jack is an audio routing program — it allows you to connect the inputs and outputs of different programs together as you like. We’ll be using it to connect iTunes’ output (that is, our song) to projectM’s input (as well as playing it over the speakers.)
2. Install projectM. There are many different precompiled versions available on the projectM download page. We want the latest build of projectm-jackosx.
3. Install ScreenFlow or other screen capture software. I personally like the rudimentary video editor included with ScreenFlow for preparing my movies as it saves the trouble of having to run a separate video editor, and is perfectly adequate for the relatively simple editing task at hand. Many people like Screenium, too. Use whatever you prefer.
4. Start the JackPilot app. This is the audio routing utility included with Jack OS X, where you actually specify which applications’ sound output should be routed to others’ input. Click “Start” to activate Jack. Then click the “Routing” button to bring up the “Connections Manager,” Jack’s routing table.

5. Change the default sound output to “JackRouter”. Since iTunes doesn’t allow you to specify which audio device to use as its output, you will need to go into System Preferences -> Sound and change the systemwide default sound output to “JackRouter”. When you do this, a new item called iTunes should appear in the “Send Ports” list in JackPilot’s Connections Manager, as illustrated to the right.

   Connections Manager represents its connection information in a rather counterintuitive way, but it’s fairly straightforward once you understand what’s going on. At any time, there is one “focused” device whose connection information is shown in the GUI. It might be in either the Send or Receive column. The focused device has white text on a blue background. It’s “system” in the Receive column in the screenshot to the right. “System” devices are the built-in microphone and speaker/headphone connections.

   Devices of the opposite polarity that are connected to that device are printed in red text. All connections of the focused device (the one highlighted in blue) are also redundantly shown in the “Connections” column on the far right. In this case, we can see that iTunes, in the Send column, is connected to system. The effect here is that any sound sent (played by) iTunes is routed to the system device, which plays it over the speakers or headphones. If you focus on “iTunes” by clicking it in the Send column, the display reverses; iTunes is highlighted in blue and system is printed in red and displayed in the “Connections” column.

6. Start projectM. The Mac build is a bit flaky here. Launching it from Finder has never worked well for me. I start it directly from a terminal window with /Applications/projectM-jack.app/Contents/MacOs/projectM-jack. This allows me to see any error messages rather than sending them to the black hole.

   At this point, you should get a visualization window that is visualizing whatever music you’re playing in iTunes. If it doesn’t respond to the music, check your JackPilot Connections Manager to make sure it looks like the screenshot.

7. Set projectM’s settings as you like. Different visualizations blend better with different types of music. Experiment with the playlist and see what works for your tune. I like to use Ctrl-L, lock to preset, to prevent random switching and switch manually during the performance.

   The projectM user interface is not exactly Mac-friendly. Viewing the hotkey list from the help menu blocks all user input, meaning that you have to close the hotkey list before you can actually use any of the hotkeys. Several hotkeys, such as Ctrl-E (open preset editor) and Ctrl-B (Show/hide menu and status bar) don’t seem to do anything on my computer. For our purposes, the most interesting are:

   • Ctrl-B: disable status bar. Be sure to do this before recording, or you’ll lose screen real estate to a white strip with the names of the presets you play.
   • Ctrl-L: lock/unlock active preset
   • Ctrl-N, Ctrl-P: next/previous preset
   • Ctrl-R: random preset
   • Ctrl-F: fullscreen mode
   • Ctrl-M: Show playlist (called “menu display” in projectM, for some reason)

   Be careful with the playlist. Pressing the third button along the top clears the entire playlist, and it’s really annoying to rebuild it by hand.

8. When you’re happy with your visualization choices, start a screen capture. I don’t record audio with the screen grab; I add it back in afterwards from the original lossless file. This avoids audio degradation that would occur by re-capturing the analog signal and digitizing it again via Jack.
9. Edit your video as necessary. You might want to trim the video to only show the active projectM window, if not running in fullscreen mode, and if you didn’t capture audio with the video you’ll want to add audio, of course.
10. Export and upload to YouTube! They prefer H.264 1280×720 or 640×480 video with MP3 or AAC audio.

Check out my YouTube channel for more of my music and visualizations.

iPhone Pros

• iPhones are all (almost) exactly the same. 4th generation iPhones do support an optional higher screen resolution than previous models, but they’re 100% binary backwards compatible. This is a trivial difference when put up against the dozens of different Android devices available with radically different hardware, and the hoops that you as a developer have to jump through to make sure your app runs correctly on all of them.
• Large, high-end user base that demonstrably spends money on apps. iPhone users have already shown their willingness to buy large numbers of apps; Android users, not so much. This seems to be slowly changing over time as the Android app store matures, but for now, iPhone is still the cash cow.
• Beautiful and well-designed UI and system architecture. iPhones have been designed as an integrated and coherent experience. Apple’s renowned mastery of intuitive human-computer interaction is evident at every step while using the device. There is an overall (and well documented) unifying strategic vision of how different subsystems should fit together seamlessly and present a unified interface to the user. Their development libraries have been designed to support this theme at every level. Android, by contrast, reads much more like KDE or Gnome; it was bolted together out of disparate pieces written by different groups working in isolation with minimal coordination. Perceived technical coolness from the perspective of a software engineer who is already intimately familiar with the system’s internals took precedent over usability. Android’s user interface is generally much uglier and much less intuitive.
• iPhones support low latency audio. Android uses ALSA internally, which does support low-latency audio, but the low-latency API is not exposed to developers, and Google does not seem to be interested in fixing this. Android latency is unacceptably high for realtime sound synthesis unless you hack it and use the ALSA API, which is officially forbidden and in any case could change significantly and without warning between Android operating system updates and between phones. iPhone, on the other hand, has an officially supported low latency audio API, and there are many successful realtime sound synthesizer apps in the app store.
• XCode. XCode is a really well done IDE. I normally don’t like IDEs. I normally use Emacs for everything. But, in all fairness, I decided to give XCode and Eclipse with the Android plugins a shot. Eclipse is nice, but a little rough around the edges. I ran into a few bizarre IDE error messages while developing my Android test projects that required extensive Googling and hours of experimentation to unravel. XCode, as one would expect from a flagship Apple product, is immaculately polished and completely integrated from top to bottom.

iPhone cons

• Objective C. While Objective C was really cool and innovative in 1988, today it seems archaic. Nobody uses it except Apple. It’s laden with tons of cruft. It still feels like it’s a bolt-on afterthought preprocessor for a C compiler. For example:
  • Selectors: you can’t refer to method names directly; you have to use the special @selector() compiler directive to look up their index number. This probably made great sense when CPUs were very slow and doing a lookup in an ASCII table was a significant performance hit over doing an integer offset jump, but on modern computers, this is just annoying. CPU time is cheaper than engineer time.

  • Manual reference-counted memory management is an obsolete and extremely fragile pain in the ass. Lisp had GC ~40 years ago. The vast set of problems introduced by manual memory management, such as buffer overflows and memory leaks, is just not worth the small gain in efficiency that you get from not periodically running a garbage collector. Again, CPU time is cheaper than engineer time. Android does GC on a phone just fine, so I don’t buy the argument that a phone can’t support that kind of overhead. Garbage collection was recently introduced for the Mac OS X desktop system, but is as of yet not available on the iPhone.

    (I’m aware that C and C++ are widely used and both still make you do manual memory management, too. Far from regarding this as a vehicle for a macho display of programming prowess, I find manual memory management to be more of an annoyance, something that the computer should be handling for me automatically, much like how figuring out the exact memory address of a function was automated long ago and is no longer something we as application-level programmers ever need to worry about much.)

  • The use of + and - to mark methods as class or instance methods. Principle of least surprise; why not just use, say, the English word class or static or something to that effect to designate class methods, and leave instance methods unmarked? Perhaps using 1 character symbols rather than entire English words as the token improved compile times on large, complex projects in the ’80s, but the time savings is infinitesimal on a modern computer, so I’ll vote for obviousness and clarity rather than crypticity.
  • The use of [] (square brackets) for message passing, and the requirement for named (but non-reorderable) arguments.

    Perhaps this was done out of a well-intentioned effort to make the programmer notice that these are messages and not function calls or C++ methods, and to make the code self-documenting. I find the introduction of new syntax rather unwarranted and distracting. I’d much prefer the familiar object.message(argument1, argument2) over the verbose and unwieldly [object message:argument1 parameterName:argument2]. You can’t even re-order the named arguments, so including the argument name is just noise in the code. I guess I’ve been spoiled by Clojure and Lisp, where there’s almost no syntax.

    If they really felt strongly about the need to pedantically differentiate messages from function calls or methods at the syntax level, they should at least have retained the general function/method syntactic template and chosen a different separator character, so as to enable programmers to re-use most of their neural syntax parsing circuitry rather than force us to introduce an entirely new syntactic template that performs only 1 task. For example, a more intuitive syntax that still emphasizes the fact that this is not a function or method call could be something like object#message(argument1, argument2) or object<-message(argument1, argument2).

• Closed platform. The source code to the vast Cocoa library is proprietary. You can develop with things Apple tells you you may develop with, and nothing else. They only recently relaxed the restriction on using any third party tools at all. The iPhone's CPU is widely reported to have built-in hardware level Java bytecode support, but that's off-limits.
• Vendor lock-in. This is a huge and obvious minus for the iPhone. Your ability to generate revenue is 100% dependent upon Apple's whims, upon remaining in Apple's good graces. Uncountable horror stories circulate online about developers who invested months and many thousands of dollars into a project only to have it rejected by Apple, often after a considerable delay and with little or no explanation.

Android pros

• Java (almost). Android apps are written in Java. (Well, technically not in Java due to licensing issues, but in a language that is 99.9% the same as Java.) If you already know Java, as I do, this is a big plus. You can easily re-use your code elsewhere, on any platform, with minor or no modifications. Objective C code is rather limited in what you can do with it: you can re-use it in your Apple desktop apps. (Yes, there is a free GNU Objective C compiler, but almost all real-world Objective C code is locked into Apple's proprietary Cocoa runtime system.)
• A JVM, almost. Android doesn't run a true JVM, unfortunately, as part of a licensing dispute between Google and Sun, but it's close. Android's Dalvik is a cleanroom reimplementation of part of Java, designed to avoid triggering what Google regards as onerous licensing fees for the use of a true JVM on a mobile handset. In any case, it's close enough that the door is open to potentially running Clojure and other JVM languages on it someday.
• No upfront developer fees. Develop for free. You can download the SDK and go nuts, upload your code to a real handset to test, send your apps to the app store for sale, whatever. There is no $99 developer tax.
• Larger ultimate user base. Android phones are now outselling iPhones, and this trend looks set to continue. The price you pay is coding for 200 slightly different handsets.
• Open source platform. Most of the Android system is freely available, precluding vendor lock-in, at least in theory.

Android cons

• Android emulator lacks hardware accelerated graphics. This is a big minus. I made a very simple bouncing-ball demo with Rokon that got about 6 or 7 fps in the emulator. Full games often take multiple seconds per frame, completely unusable. If you want to write games (or anything else where graphical performance is key), you absolutely need to have a hardware handset from the start. If you don't already own an Android phone anyway, this means you have to invest hundreds of dollars upfront, just to see if the platform is for you.
• Lack of interest from Google. Google has many, many pots cooking at once. While they do devote significant resources to Android's maintenance and expansion, it's nowhere near as important a product for them as the iPhone is for Apple, and you can tell. The iPhone is a core strategic product for Apple, whereas Android is a relatively minor satellite project for Google. Correspondingly, Apple's general support of the iPhone and its developers seems far better.
• Relatively few libraries. For the moment, there seem to be more third party libraries available for the iPhone, and they seem to be more mature and robust than what's out there for Android. This will change in time, as more and more app developers begin to use Android.

It was a close call. I did some exploratory coding on both platforms — there's no substitute for direct experience. Though I much prefer Java over Objective C as a language, the universe of extant Android libraries is much smaller and less mature. I may have been able to live with that in order to be able to code in Java and avoid Apple's vendor lock-in. In the end, the tipping point was practical: the Android emulator's lack of hardware accelerated graphics left me with an unusable framerate, so I'd have to spend hundreds of dollars on a handset to do even trivial demo development, whereas I already own an iPhone.

I'll be checking back in on Android periodically, as I expect its long-term position will improve substantially in time, but for now, I'm going with the iPhone.

Dell provides the biosdisk utility to make a bootable FreeDOS image with the BIOS upgrade executable on it, but it doesn’t work with the Windows-only PowerEdge 4400 BIOS upgrade, since that won’t run on FreeDOS. They claim that biosdisk isn’t intended for PowerEdge systems, since they all have Linux BIOS flashes available; this does not seem to actually be the case for the 4400.

The Windows self-extractor does run under WINE, but it demands an actual floppy drive device to write to, which my computer doesn’t have. WINE also doesn’t seem to support mounting an image file as a floppy device, for some reason. You’ll need to use something like VMWare or a computer with a physical floppy drive to get this to extract itself.

Once you have a floppy image, here’s how to flash without biosdisk or Windows available:

1. Use WINE to extract the archive. For maximum fun, the upgrade I have refuses to just extract itself; instead, it requires that it has access to a floppy drive to write to directly. Sigh. Ok. Working on that. Updates to come. winecfg
2. Install syslinux. (apt-get install syslinux). We are going to use memdisk, a boot loader that creates a RAM drive and then loads an operating system, to load FreeDOS.
3. Get a FreeDOS boot disk image and add your flashing .exe to it.Mount with something like mount -t vfat /path/to/freedos.img /mnt -o loop, then just cp your flashing .exe to /mnt. (If you got a gzipped image, gunzip it first.
4. Edit /etc/grub.d/40_custom to add a boot menu entry for memdisk/FreeDOS.

   menuentry "FreeDOS" {
       set root=(hd0,2) #### <--- Change this to your /boot partition!
       linux16 /memdisk floppy
       initrd16 /freedos.img
   }
   

   As noted, change the root entry to point to your actual /boot partition, so GRUB can find the images. Also note that the partition normally mounted at /boot will be mounted as the root directory (/) at boot time, so it’s not necessary to prefix /boot before the image file names.

   Then, reboot and select FreeDos from the GRUB menu. Run your DOS-based flash program, and all is well.

   ]]> http://www.paullegato.com/blog/dell-poweredge-bios-linux/feed/ 0 http://www.paullegato.com/blog/swing-clojure-gui-black-scholes/ http://www.paullegato.com/blog/swing-clojure-gui-black-scholes/#comments Thu, 15 Jul 2010 07:55:56 +0000 Paul Legato http://www.paullegato.com/?p=352 Now that we have implemented Black-Scholes in Clojure, let’s make a Swing GUI for it. The Swing GUI will have text boxes for all the necessary inputs, and calculate prices and Greeks when the button is pressed. It’s a simple and straightforward way to get started in Swing GUI programming in Clojure. Here’s what it looks like.

   The GUI is written entirely in Clojure with the Swing toolkit. Calculation state is stored in a series of atoms. Watches are used to update the output table automatically when an atom changes, an idea from Kotka. I used the excellent MiG Layout for general layout functionality, and generic Swing widgets (JTextFields and a JTable) for the input and output.
   

   Clojure code

   The Black-Scholes GUI is launched with themain function. The Swing GUI is actually created in the initialize-gui function, but since Swing widgets are not threadsafe, we must use the do-swing* helper function from Clojure Contrib to spin off the real GUI creation code into the Swing event thread.

   (defn initialize-gui
     []
     (let [frame (JFrame. "Black-Scholes Option Modeler")]
   
       (doto frame
         (.setDefaultCloseOperation JFrame/DISPOSE_ON_CLOSE)
         (-> .getContentPane
             (.add (miglayout (JPanel.)
   
                              (input-panel
                               (fn [_]
                                 (do-swing
                                  (doto frame
                                    (.setVisible false)
                                    (.dispose)))))
   
                              (result-table)
   
                              )))
   
         (.setDefaultCloseOperation JFrame/DISPOSE_ON_CLOSE)
         (.pack)
         (.setVisible true))
        ))
   
   (defn main
     []
     (do-swing* :now initialize-gui))
   

   This is pretty straightforward. We make a (locally scoped) JFrame to be the main container at line 3. Within the context of the let, we add a JPane using MiG Layout to its content area (starting at line 8). The JPanel itself contains two subwidgets, which are those returned from the input-panel and result-table functions.

   Note that we are passing an anonymous function to input-panel at line 11. The input panel contains the “Quit” button, and that button needs to know what to do when it’s pressed. We want it to close the GUI; that is, it should destroy the outer JFrame. To do that, it needs a reference to it. However, as a local variable in initialize-gui, the JFrame is not in scope elsewhere; we have to pass it in. The given function closes over the JFrame reference (line 13) and transports it to input-panel, where that function will be called as an event listener when the “Quit” button is pressed.

   We could also have simply passed the bare JFrame rather than a function, and moved the frame-closing code into input-panel, but that would imply that the input panel must always be in a JFrame, which will not necessarily the case. Passing in an external anonymous function to be activated by the “Quit” button allows the input panel to be used generically in any sort of container, with the container deciding what to do when “Quit” is pressed.

   The Input Panel

   
   (defn- input-panel [quit-action]
       (let [
             ;; Fields
             spot (doto (JFormattedTextField. (NumberFormat/getNumberInstance))
                    (.setColumns 6)
                    (.setValue @*spot*))
   
             strike (doto (JFormattedTextField. (NumberFormat/getNumberInstance))
                      (.setColumns 6)
                      (.setValue @*strike*))
   
             days-till-expiry (doto (JFormattedTextField. (NumberFormat/getIntegerInstance))
                                (.setColumns 6)
                                (.setValue @*days-till-expiry*))
   
             riskfree (doto (JFormattedTextField. (NumberFormat/getNumberInstance))
                        (.setColumns 6)
                        (.setValue @*riskfree*))
   
             volatility (doto (JFormattedTextField. (NumberFormat/getNumberInstance))
                          (.setColumns 6)
                          (.setValue @*volatility*))
   
             ;; Buttons
             calculate (JButton. "Calculate")
             quit (JButton. "Quit")
             ]
   
         (add-action-listener
          calculate
          (fn [_]
            (update-all-calculations
             (Double/parseDouble (.getText spot))
             (/ (Integer/parseInt (.getText days-till-expiry)) *trading-days-per-year*)
             (Double/parseDouble (.getText strike))
             (Double/parseDouble (.getText riskfree))
             (Double/parseDouble (.getText volatility)))))
   
         (add-action-listener
          quit
          quit-action)
   
         (miglayout (JPanel.)
                    :layout  [:wrap 2 ]
   
                    (JLabel. "Spot") [:align "right"]
                    spot
   
                    (JLabel. "Strike") [:align "right"]
                    strike
   
                    (JLabel. "Risk-free rate") [:align "right"]
                    riskfree
   
                    (JLabel. "Volatility") [:align "right"]
                    volatility
   
                    (JLabel. "Days till expiry") [:align "right"]
                    days-till-expiry
   
                    calculate
                    quit)))
   

   The input widgets are local variables in the input panel function, along with the two buttons. The widgets are initialized from the state atoms (which are then unused again; see discussion below.) The “Quit” button is given the event callback passed in as an argument (so it can manipulate the enclosing JFrame as per above.) The “Calculate” button is given an event callback that calls the update-all-calculations function with the values in the input widgets.

   The lot are then stuffed into a JPanel with a MiG Layout and returned.

   The Output Table

   (defn- update-cell
     "Updates the given AbstractTableModel cell and fires a change event for that cell."
     [model data row col]
     (.setValueAt model data row col)
     (.fireTableCellUpdated model row col))
   
   (defn- result-table []
     (let [
           column-names ["" "Call" "Put"]
           row-names ["Price" "Delta" "Theta" "Rho" "Gamma" "Vega"]
           table-model (proxy [AbstractTableModel] []
                         (getColumnCount [] (count column-names))
                         (getRowCount [] (count row-names))
                         (isCellEditable [] false)
                         (getColumnName [col] (nth column-names col))
                         (getValueAt [row col]
                                     (condp = col
                                         0 (nth row-names row) ;; Return the row name for column zero
                                         (condp = [row col]
                                             [0 1] @*call-price*
                                             [0 2] @*put-price*
   
                                             [1 1] @*call-delta*
                                             [1 2] @*put-delta*
   
                                             [2 1] @*call-theta*
                                             [2 2] @*put-theta*
   
                                             [3 1] @*call-rho*
                                             [3 2] @*put-rho*
   
                                             [4 1] @*gamma*
                                             [4 2] "=="
   
                                             [5 1] @*vega*
                                             [5 2] "=="
   
                                             nil))))
   
           table (doto (JTable. table-model)
                   (.setGridColor java.awt.Color/DARK_GRAY))
           ]
   
       (add-watch *call-price* ::update-call-price (fn [_ _ _ newprice] (update-cell table-model newprice 0 1)))
       (add-watch *put-price* ::update-put-price (fn [_ _ _ newprice] (update-cell table-model newprice 0 2)))
   
       (add-watch *call-delta* ::update-call-delta (fn [_ _ _ newval] (update-cell table-model newval 1 1)))
       (add-watch *put-delta* ::update-put-delta (fn [_ _ _ newval] (update-cell table-model newval 1 2)))
   
       (add-watch *call-theta* ::update-call-theta (fn [_ _ _ newval] (update-cell table-model newval 2 1)))
       (add-watch *put-theta* ::update-put-theta (fn [_ _ _ newval] (update-cell table-model newval 2 2)))
   
       (add-watch *call-rho* ::update-call-rho (fn [_ _ _ newval] (update-cell table-model newval 3 1)))
       (add-watch *put-rho* ::update-put-rho (fn [_ _ _ newval] (update-cell table-model newval 3 2)))
   
       (add-watch *gamma* ::update-gamma (fn [_ _ _ newval] (update-cell table-model newval 4 1)))
       (add-watch *vega* ::update-vega (fn [_ _ _ newval] (update-cell table-model newval 5 1)))
   
       ;; This shrinks the table's preferred viewport down to its actual size.
       ;; (The default is to make a huge viewport, even though the table is small.)
       (.setPreferredScrollableViewportSize table (.getPreferredSize table))
   
       (JScrollPane. table)
       ))
   

   The output table itself is a standard JTable. Its model is initialized from the output state atoms. The watches at lines 44-57 are the most interesting part; they allow arbitrary code to be called when the output state atoms change. This allows the Black-Scholes calculation to happen independently of the output widget. When one of those atoms is changed, it self-updates automatically via the watches.

   The helper function at lines 1-5 updates the appropriate model cell and fires an event to tell the JTable to repaint it.

   The scrollable viewport business at line 61 is to work around a questionable Java design decision where a JTable, by default, tells its container to make its viewport enormous, even though it have very little data. (That’s not an error, that’s subjunctive.)

   Design considerations

   GUI and business logic decoupling

   Above all, the GUI has to be decoupled from the model itself. Internal changes to one should never affect the other, and the two should be independently testable. In such a simple application, this was easy enough. A more complex app that requires several information round trips would be trickier. We now have a good foundation for such an app. No business calculations are performed in the GUI code namespace. The sole point of connection is in the update-all-calculations function:

   
   (defn- update-all-calculations [spot timeleft strike riskfree sigma]
     (swap! *call-price* (fn [_] (bs/call spot timeleft strike riskfree sigma)))
     (swap! *put-price* (fn [_] (bs/put spot timeleft strike riskfree sigma)))
   
     (swap! *call-delta* (fn [_] (bs/call-delta spot timeleft strike riskfree sigma)))
     (swap! *put-delta* (fn [_] (bs/put-delta spot timeleft strike riskfree sigma)))
   
     (swap! *call-theta* (fn [_] (bs/call-theta spot timeleft strike riskfree sigma)))
     (swap! *put-theta* (fn [_] (bs/put-theta spot timeleft strike riskfree sigma)))
   
     (swap! *call-rho* (fn [_] (bs/call-rho spot timeleft strike riskfree sigma)))
     (swap! *put-rho* (fn [_] (bs/put-rho spot timeleft strike riskfree sigma)))
   
     (swap! *vega* (fn [_] (bs/vega spot timeleft strike riskfree sigma)))
     (swap! *gamma* (fn [_] (bs/gamma spot timeleft strike riskfree sigma))))
   

   Atoms to hold state

   As we can see above, set of atoms holds state for the GUI, in keeping with the general Clojure principle of isolating state into transactional memory. (Actually, I cheated a bit. Our full state consists of the input values and the results of our Black-Scholes calculations. Although I created atoms for the input state values as well, I calculate the results directly from the widgets. It is a trivial exercise to alter the “Calculate” button callback to use the atoms. This would be more useful in the case where changes to a widget fire an event that triggers recalculation, which is not done in this demo as per below. I would use the atoms in a real application.)

   I/O is inherently stateful, as are GUIs. There was therefore the temptation to use the Swing widget objects themselves as state-holders, and in fact it requires extra code to shuttle data back and forth between the widgets and the atoms. In such a simple app as this, using the GUI widgets as state-holders does not make much practical difference, but in a more complex app, a well designed set of state atoms and watches can automate much of the tedious GUI logic.

   Using the widgets directly as state holders implies a tight coupling between the input widgets, the business logic, and the output widgets. Changing any one of them would invariably have required changes to the linking logic.

   For example, suppose we were using widgets-as-state, and we changed the “days left till expiry” input to a slider rather than a text box, or suppose we wanted to read it from a file or from an API. We would have to update the code that reads it, calculates the model, and puts the results into the output table. Touching that code implies a non-zero likelihood of breaking it. Why even introduce the potential for breaking something that is unrelated to the task at hand, in this case the business logic and output code?

   Again, this seems trivial with such a very small application that is only a few lines long, but the design principle is what’s important. Imagine applying it to a much larger application with hundreds of inputs spread across 4 windows, 3 databases, and 6 web feeds to see why this is a good idea.

   Auto-updating results when input changes

   I wanted the results to auto-update anytime an input is changed, without resorting to a “Calculate” button. Having the button trigger the calculation and output update allows the state of the input and the output to become decoupled, which can be confusing for the user. Worse, it could be unnoticed by a user who is doing other things at the same time.

   Due to typical Java overengineering, there is no straightforward way to simply (add-action-listener) on the JTextField and get notifications when it changes. Instead, each Swing widget implements full-blown model-view separation, and so you have to get the document underlying the JTextField. Yes, the view components of your MVC application themselves contain both models and views. Each and every field contains an entire implementation of the document object and associated cruft, all for itself. If you want to monitor changes in the widget, you need to build out a DocumentListener to watch for such events.

   In a real application, I’d bite the bullet and implement it, but it’s not worth the extra time for the purposes of this demo, so we are left with a “Calculate” button and manual updating.

   Clojure-Options on Github

   The entire package so far is now available on my GitHub page. Patches are welcome. Atomizing the input widgets and adding the appropriate event callbacks and watches to eliminate the “Calculate” button is particularly welcome.

   ]]> http://www.paullegato.com/blog/swing-clojure-gui-black-scholes/feed/ 2 http://www.paullegato.com/blog/black-scholes-clojure/ http://www.paullegato.com/blog/black-scholes-clojure/#comments Sun, 11 Jul 2010 00:51:27 +0000 Paul Legato http://www.paullegato.com/?p=345 The Black-Scholes option pricing model, implemented in Clojure based on the description at Wikipedia and these code samples.
   

   ;;
   ;; Black-Scholes option pricing model
   ;;
   ;; Copyright (C) 2010 Paul Legato. All rights reserved.
   ;; Licensed under the New BSD License. See the README file for details.
   ;;
   ;; Disclaimer: This code comes with NO WARRANTY, express or implied.
   ;; There may be any number of bugs. Use at your own risk.
   ;;
   ;; References:
   ;; - https://secure.wikimedia.org/wikipedia/en/wiki/Black%E2%80%93Scholes
   ;; - http://www.espenhaug.com/black_scholes.html
   
   (ns clojure-options.black-scholes
     (:require incanter.distributions))
   
   (defn d1
     [spot timeleft strike riskfree sigma]
     (/
      (+ (Math/log (/ spot strike))
         (* (+ riskfree (/ (* sigma sigma) 2))
            timeleft))
      (* sigma (Math/sqrt timeleft))))
   
   (defn d2
     [spot timeleft strike riskfree sigma]
     (- (d1 spot timeleft strike riskfree sigma) (* sigma (Math/sqrt timeleft))))
   
   (defn- n [val] (incanter.distributions/cdf (incanter.distributions/normal-distribution) val))
   
   (defn call
     "Returns the theoretic value of a European call option on the given underlying based on the Black-Scholes model.
   
    * spot - spot price of the underlying
    * timeleft - time to expiration, in years
    * strike - option strike price
    * riskfree - annual continuous risk-free interest rate
    * sigma - volatility of the underlying
   
   "
     [spot timeleft strike riskfree sigma]
     (- (* spot (n (d1 spot timeleft strike riskfree sigma)))
        (* strike
           (Math/exp (* (- riskfree) timeleft))
           (n (d2 spot timeleft strike riskfree sigma)))))
   
   (defn put
     "Returns the theoretic value of a European put option on the given underlying based on the Black-Scholes model.
   
    * spot - spot price of the underlying
    * timeleft - time to expiration, in years
    * strike - option strike price
    * riskfree - annual continuous risk-free interest rate
    * sigma - volatility of the underlying
   
   "
     [spot timeleft strike riskfree sigma]
     (- (* strike
           (Math/exp (* (- riskfree) timeleft))
           (n (- (d2 spot timeleft strike riskfree sigma))))
        (* spot (n (- (d1 spot timeleft strike riskfree sigma))))))
   
   (defn call-delta
     [spot timeleft strike riskfree sigma]
     (n (d1 spot timeleft strike riskfree sigma)))
   
   (defn put-delta
     [spot timeleft strike riskfree sigma]
     (- (n (d1 spot timeleft strike riskfree sigma)) 1))
   
   (defn- nprime [val] (incanter.distributions/pdf (incanter.distributions/normal-distribution) val))
   
   (defn gamma
     [spot timeleft strike riskfree sigma]
     (/ (nprime (d1 spot timeleft strike riskfree sigma))
        (* spot sigma (Math/sqrt timeleft))))
   
   (defn vega
     [spot timeleft strike riskfree sigma]
     (* spot (nprime (d1 spot timeleft strike riskfree sigma)) (Math/sqrt timeleft)))
   
   (defn call-theta
     [spot timeleft strike riskfree sigma]
     (-
      (- (/ (* spot (nprime (d1 spot timeleft strike riskfree sigma)) sigma)
            (* 2 (Math/sqrt timeleft))))
      (* riskfree strike (Math/exp (* (- riskfree) timeleft)) (n (d2 spot timeleft strike riskfree sigma)))))
   
   (defn put-theta
     [spot timeleft strike riskfree sigma]
     (-
      (- (/ (* spot (nprime (d1 spot timeleft strike riskfree sigma)) sigma)
            (* 2 (Math/sqrt timeleft))))
      (* riskfree strike (Math/exp (* (- riskfree) timeleft)) (n (- (d2 spot timeleft strike riskfree sigma))))))
   
   (defn call-rho
     [spot timeleft strike riskfree sigma]
     (* strike timeleft (Math/exp (* (- riskfree) timeleft)) (n (d2 spot timeleft strike riskfree sigma))))
   
   (defn put-rho
     [spot timeleft strike riskfree sigma]
     (* (- strike) timeleft (Math/exp (* (- riskfree) timeleft)) (n (- (d2 spot timeleft strike riskfree sigma)))))
   

   This code uses the new Incanter Distributions module, which hasn’t been released yet, so it requires the latest master version from Github.

   Most of the values agree with those produced at BloBek’s calculator, with the exception of theta. I imagine this has to do with the units used for the “days left to expiration” field; my calculator uses fractions of a year, while BloBek doesn’t specify whether his days are to be trading days or calendar days. It could also be a bug in one of our implementations.

   In Part 2, I’ll build a simple Swing GUI for the calculator.
   Update: Part 2, with the Swing GUI, is now available. You can also get the entire project’s source code from GitHub.

   ]]> http://www.paullegato.com/blog/black-scholes-clojure/feed/ 2 http://www.paullegato.com/blog/lazy-spy-clojure-logging-lazy-sequences/ http://www.paullegato.com/blog/lazy-spy-clojure-logging-lazy-sequences/#comments Mon, 24 May 2010 02:27:16 +0000 Paul Legato http://www.paullegato.com/?p=315 Clojure.contrib’s logging/spy macro is really useful for quick investigation of bugs with unclear stack traces, and to check that intermediate values in a calculation are being returned and composed correctly. Lazy sequences don’t get logged correctly, however:

   > (log/spy [1 2 3 4 5])
   [1 2 3 4 5] ;;; prints " [1 2 3 4 5] => [1 2 3 4 5]" to the log
   > (log/spy (take 3 [1 2 3 4 5]))
   (1 2 3) ;;; prints "(take 3 [1 2 3 4 5]) => clojure.lang.LazySeq@8592" to the log
   

   The spy macro logs a string realization of the sequence per se, rather than its constituents. In the case of a concrete, fully realized sequence such as the literal vector in the first example, all values are known in advance, and the vector’s string realization in the log can helpfully be its actual contents. In the case of the lazy sequence returned by take, however, we get a less than helpful class name and some sort of instance identifier code rather than the actual contents of the sequence. A lazy sequence saves resources by merely promising to provide the values when asked. If they’re never asked for, they’re never computed. Since they haven’t been computed, they can’t be printed. A lazy sequence can also potentially be infinitely long, another good reason why they are not realized by default when converting the sequence to a string.

   In our case, we don’t care about the resource usage, and we’re sure the sequence is finite. We want to force all values to be computed for manual inspection. (There are several ways of going about this.) The following modification of the spy macro converts the lazy sequence into a regular seq:

   (defmacro lazy-spy
     "Evaluates expr and outputs the form and its result to the debug log.
      If the result is a lazy sequence, it is realized concretely in the log.
      Returns the result of expr."
     [expr]
     `(let [a# ~expr] (log/log :debug (str '~expr " => "
                                       (if (= clojure.lang.LazySeq (class a#))
                                         (seq a#)
                                          a#)))
           a#))
   

   This gives us:

   > (lazy-spy  (take 3 [1 2 3 4 5]))
   (1 2 3) ;;; prints "(take 3 [1 2 3 4 5]) => (1 2 3)" to the log
   

   Clojure posts you might also like:

   • Log4J and Clojure Configuration
   • Setting Clojure’s Log Level
   • SQL WHERE clauses in Clojure from S-Expressions
   • XML DTD Validation in Clojure: Turning It Off, Parsing Malformed XML
   ]]> http://www.paullegato.com/blog/lazy-spy-clojure-logging-lazy-sequences/feed/ 0 http://www.paullegato.com/blog/log4j-clojure/ http://www.paullegato.com/blog/log4j-clojure/#comments Sun, 16 May 2010 22:12:48 +0000 Paul Legato http://www.paullegato.com/?p=292 Clojure, Log4J, and clojure.contrib.logging can play nicely together. Here’s a reasonable default configuration.

   1. If using Leiningen, add

      :dependencies [
                       [log4j "1.2.15" :exclusions [javax.mail/mail
                                                    javax.jms/jms
                                                    com.sun.jdmk/jmxtools
                                                    com.sun.jmx/jmxri]]
      ]
      

      to your project.clj file and rerun lein deps to auto-install the Log4J JAR.

      If not using Leiningen, acquire a Log4J JAR and put it on your classpath.

   2. Create a file called log4j.properties on your classpath and put the following into it:

      # Based on the example properties given at http://logging.apache.org/log4j/1.2/manual.html
      # Set root logger level to DEBUG and its only appender to A1.
      log4j.rootLogger=DEBUG, A1
      
      # A1 is set to be a ConsoleAppender.
      log4j.appender.A1=org.apache.log4j.ConsoleAppender
      
      # A1 uses PatternLayout.
      log4j.appender.A1.layout=org.apache.log4j.PatternLayout
      log4j.appender.A1.layout.ConversionPattern= %-5p %c - %m%n
      

   That’s the short version. Read on for the gory details.

   Logging philosophy

   My logging requirements are simple at the moment. I want to write messages to the console, each tagged with a log level (“INFO”, “WARN”, “ERROR”, etc.) I want to be able to set a log level, and only see log messages of that severity or higher. That’s it. I take this to be the canonical base case of logging that covers 80% of needs. All other logging features (e.g. logging to a file, differentiating by class or namespace) are nice, but much less common, and ought to be handled via configurable deviations from this default case.

   clojure.contrib.logging does not presently make it easy to do any of this common base case out of the box. The Lispy façade quickly falls apart; you’re dumped into the deep end of the Java overengineering mire without a life vest when you simply want to change the log level temporarily.

   What’s wrong with clojure.contrib.logging’s defaults?

   clojure.contrib.logging with no special configuration done prefaces each and every logged line on my system with:

   [null] May 16, 2010 1:08:48 PM clojure.contrib.logging$eval__2077$impl_write_BANG___2088 invoke
   

   I have no idea why. This makes it very hard to read the log when logging any nontrivial amount of information.

   clojure.contrib.logging’s documentation contains a rather cryptic offhanded remark, with no further explanation: Note: your log configuration should display the name that was passed to the logging implementation, and not perform stack-inspection, otherwise you’ll see something like “fn__72$impl_write_BANG__39__auto____81″ in your logs.

   My log configuration? Stack inspection? Why do I need to have a log configuration to just dump strings to the console? Why is the default case to print a verbose and useless header for each line? Isn’t this Javaesque mess — obscure 50 line XML configuration files just to get basic logging functionality working — exactly what we’re trying to avoid?

   Log4J with clojure.contrib.logging configuration

   Ah, well, there’s nothing for it, I suppose. Better just get a serious logging library and figure out how to configure it. The docs say that clojure.contrib.logging delegates to the first supported logging implementation that it finds. I am apparently using the builtin JDK logger by default, and Apache Commons’ logger is just another wrapper, so I added Log4J to Leiningen’s project.clj and it installed the jar for me.

   Now, I get:

        [null] log4j:WARN No appenders could be found for logger (my-project-name).
        [null] log4j:WARN Please initialize the log4j system properly.
   

   Followed by dead silence.

   Hm.

   I can only imagine that Log4j was never intended to be used without a custom configuration. Odd. Why people design software that comes without reasonable defaults pre-set to cover the common base case is beyond me. I fully understand that strange setups will require customized configuration, but “dumping strings to the console” does not seem like a bizarre edge case for a logging package. A default case of “send all log messages to the black hole unless configured to do otherwise” makes little sense.

   So, how hard can it be to configure Log4J for this base case functionality? The “short introduction to log4j” is 19 pages long. Sigh. (The complete manual is proudly described as “over 200 pages.” I sometimes wonder if they overengineer Java libraries on purpose just to drive book sales.)

   I do not want to know what a Nested Diagnostic Context is, or the precise semantic distinction between “Loggers,” “Appenders,” and “Layouts.” I don’t care about “Appender Additivity” and “Named Hierarchies,” and I certainly don’t want to write an XML configuration file. I Just want to log tagged strings to the console, and have them print without a bizarre autogenerated header line. That’s it. I fully appreciate that those things will be useful for some people — maybe even for me, when my project grows to become more complex. I have nothing against their existence; but for the base case, one should not need to know of it.

   On page 9 of the “short” manual, after a boatload of irrelevant information, we have an answer: there is a class called BasicConfigurator with a static method that does reasonable initialization for logging to the console. We can just do (org.apache.log4j.BasicConfigurator/configure) and get that.

   However, we are still getting log messages like:

        [null] 3702 [8633270@qtp-3149669-2] INFO my.namespace - log message
   

   We have to learn something about Layouts now. The thread names generated by Clojure are apparently not very useful, so we’re going to turn them off. And while interesting when profiling, the number of milliseconds since program start is just going to be visual clutter 98% of the time, so I’m going to turn that off, too, until I actually need it.

   The result is the log4j.properties file listed above. Just copy and paste it into your classpath. If you want more or different information in your logs, take a look at the PatternLayout API documentation and alter the ConversionPattern line according to your preferences.

   The result:

        [null] INFO  my.namespace - log message
   

   I’m still not sure where the [null] comes from, but overall, it’s very readable.

   Clojure posts you might also like:

   • Lazy-spy: Clojure logging/spy for lazy sequences
   • Setting Clojure’s Log Level
   • SQL WHERE clauses in Clojure from S-Expressions
   • XML DTD Validation in Clojure: Turning It Off, Parsing Malformed XML
   ]]> http://www.paullegato.com/blog/log4j-clojure/feed/ 1