|Spare Time Labs 2.0|
PIC CDC ACM
Fun with HC08
Java Goes Native
Gas Fired Furnace
Down Memory Lane
This is an article draft I wrote for Doctor Dobb's Journal in autumn of 2006. After more than six months and numerous unanswered emails later I still do not know if they rejected it or not. Tired of waiting I'm putting it up here , before it goes totally stale, in the hope that someone may benefit from it
Java on Desktop Goes Native
As more developers are catching the Jave desktop train this article should fill some gaps. In this article I describe a 100% Pure Java cross platform a solution for both development and deployment of Java desktop applications that look and behave like native applications. For added bonus the solution is free in every sence of the word . The solution is part of my free (GPL) desktop application framework jApp which is freely available from my web site at http://www.sparetimelabs.com .
BackgroundAt work I write rich client desktop Java applications  which we deploy using various technologies including Java Web Start. On my sparetime I hone my coding skills by working on my two software projects jApp and jDraft . Both are relevant to this story.
jDraft is a 100% Pure Java 2D CAD freeware application that builds on my Free application framework, jApp.
At work we do most of the development with Borland JBuilder which can generate native executables for the tree main platforms: MacOS, Linux and Windows. On my spare time I prefer Eclipse . Not only because I think it is an excellent tool but also because it is free, which is a thing to consider when one is developing an application with zero income model.
For the past five years Java Web Start  has been my only deployment mode for jDraft. Recently it has shown up on my radar that not everybody shares my enthusiasm for Web Start. And indeed the user experience is less than optimal in some key areas, especially in the integration with the Desktop regardless of weather your desktop manager is Finder, Explorer, Gnome or KDE.
My philosophy is that for a good user experience the user should not be aware that the application is written in Java. Rather the application should blend in with the crowd and look like any first class citezent. The Java Swing toolkit gives a passable, if not perfect, imitation of the native look and feel, and it should be even better with the up coming Java 6 release . Reference  gives a good introduction to the subject. But on the desktop integration side the Java API is conspicious by absence.
The most fundamental aspect of a point and click user interface is just that, the user needs to be able to launch applications and open documents by just clicking at the icons on the desktop.
The icons should represent the functionality and content of the application and documents respectively.In addition to file associations I also wanted a solution that does not require installers. Although installers are common, it is a fact that an installer adds nothing positive to the user experience and mostly serves to patch shortcoming in the desktop's application deployment system.
Building the executablesBefore you can create file associations for your application you need to build native executables. An article  on 'developer.apple.com' explains the process of Mac OS X integration pointing out the 'JarBundler' tool , which unfortunately is an interactive GUI based tool not easily incorporated into a build process. While googling for the Apple jarbundler I came accross an Ant task named jarbundler  , which integrates beatifully with my Ant based build process. Although an other article article  on java.net explains the necessary steps for Windows and Mac deployment the article offered no help with Linux integration.
I was also a litle disappointed that the Windows solution involved some commercial JNI code and a native DLL to do the necessary registry magic for file associations. This ment either sacrifying my Pure Java concept or no-installer goal.
A few companies make commercial turnkey solution , a Javalobby article  provides a good list of these.
Luckily I did not have to resort to them because I stumbled on 'jstub' , a free Ant task for packaging Java applications into native executables for Windows and Linux.
An interesting technical detail worth mentioning is how 'jstub' creates the native executables. This is based on the observation that the Java Virtual Machine apparently uses 'zlib'  compression library to read the compiled Java classes. Because 'zlib' does not mind that a file has a preamble before the actual compressed file starts it is possible to add most anything to the beginning of any executable '.jar' file. Just create a stub or bootstrap code that passes the file itself to Java and stick the '.jar' file at the end. For Windows the stub is written in C and for Linux it is actually a shell script.
A delightfull detail is that the even the code inside 'jstub' that inserts the icon resource into the .EXE is pure Java, making it possible to use it on any Java / Ant supported platform . Well done, OrangeVolt . The comple Ant task definition for building the executables is too long to include here but an outline the task defs is in Example 1.
Implementing the file associationsThe above takes care of building the executables for all three platforms and the Mac executable even has the icons and file associations working, as you might say, out of the box.
To do the necessary Windows registry hacking for the file associations JNI code still seemed necessary.
Fortunatelly all modern (NT based) Windows come with a command line utility REG.EXE which can be used to manipulate the registy. And with the Java API 'Runtime.execute()' method it is possible to execute command line tools of the underlying operating system. A pure Java solution, albeit a borderline case!
By making my jDraft application to do the association registration by itself on the first launch a separate installer becames unnecessary.Listing 2 is a Java class that implements a clean interface to the shell command line including the possibility to 'sudo' commands. With this 'Shell' class setting a Windows registry key is as eysy as obtaining a Shell instance and giving it a command line to execute as in example 2.
Handling file associations on Windows is pretty simple and on Mac OS totally trivial but Linux with its multitude of distros, desktop managers and vague and shifting standards is a different story. I guess thas what freedom is fundametally about.
Be that as it may, it boils down to the fact that icon and file associations are handled in the popular Gnome and KDE desktop manager by writing some description files into specific system directories and then calling a system script or two to inform the desktop manager of the newcomers.
Although there is an ongoing effort to standardize the desktop manager interfaces  not every distro is up to speed and the standard leaves some important points open, like the location of some key scripts and config files. To overcome this I attempted to collect a list of all likely place an and work from there. This turned out to be more than I had bargained for and I was about to give up, when someone pointed out that all this has been worked out and the knowledge embedded into a set of scripts called XDG-Utils  .
Unfortunately, in the best tradition of Linux there is no guarantee that these scripts exist in the system and if they do, where they might be. To solve this chicken and egg problem I embedded the scripts as resources into my Java class XDGUtils, listing 2, which writes them out to the disk for execution. With this strategy I can execute these scripts like any other shell command using the the Shell class.
A further complication with Linux desktop integration model and a strange concept for anyone coming from Windows is that in Linux one cannot just write to the system directories and execute commands as one sees fit. In Linux you need to run with root rights to do certain things. That is good from security point of view but it means that your Java application is likely to run with limited user rights making for example some of the installation related directories inaccessible.
Fortunately most distros include the command 'sudo'  which allows you to execute commands with root rights, if you know the password.
So by asking the user for the password and executing commands with 'sudo' most anything can be executed. Granted, you still cannot write to the system directories but you can write the necessary files temporarily to the user's home directory or some such place where the user has write access and then copy the files into the right place with root rights using 'sudo' and 'cp', the Linux copy command.
Using the AssociationServiceI've hidden all the dreary details of registering file association for Linux and Windows into two classes,
To use the
Icons need to conform to a naming convention and stored eiher in your project sources or as resource of some class as described in the table. An application ID is used as a part of icon file names and keys to Windows and Linux registries, so you should take care to pick up a unique name that is unlikely to conflict with other keys.
Mac OS signatures and file types can be registered to ensure uniquenes.
Mime types can be registerd and are usually (for application spefic file types) of the form "application/vnd.company.type".
In Linux the icon for the actual application executable file cannot be changed easily. The convention is that the executable is hidden in somewhere in the system and an 'Application.desktop' file takes it place on the users desktop. To get an idea on how to use AssociationService see example 3.
Packaging for distributionFor distributing applications on a shoe string budget nothing beats Internet.
To get the user's browser to download your nicely packaged application it needs to have a specific type.
For Windows the established format is '.zip' file and for Linux its '.gz'. The premier browsers for each platform, Explorer and Firefox, are configured, out of the box, to download files of these types. To automatically create both as part of your build process using Ant script is trivial with the 'zip' and 'gzip' tasks. (For those working on a budget there is the added advatage that even the most basic internet service providers usually provide download statistics by file type, if not by file, so you can track the number of downloads easily).
The above formats also work for Macintosh Finder but most applications seem to be delivered in the form of a mountable disk image '.dmg'. The advantage is that Safari, the Mac browser, automatically downloads, mounts and opens these images which then appear as ordinary folders. The user can then just drag the application to where he/she wants.
Creating a .dmg disk image is not trivial.
Fortunately Apple has had the foresight to create command line tools in addition to the showy graphical tools and some bright fellow has worked out the details , even if the instructions are slightly out dated by now. With command line tools and Ant's ability to execute them the build process of creating disk images can be totally automated.
Formatting the disk image to put a file system on it requires root priviledges which means either embedding the password into the build script or doing some interactive input during build process, neither of which is a good practice.
Instead I chose to pre-create a large and empty but formated disk image manually, which process needs admin right, and then use the build process to clone that, copy my Mac OS exececutable onto it and then compress the clone creating an optimally sized image, all of which can be done with normal user rights.
Example 4 outlines the Ant script that packages all three executables.
It presuppose the existence of an empty Mac OS disk image file. Correct naming is essential as the script refers to the volume both by filename and volume name. Example 5 gives the shell commands necessary to create that image.
hdiutil create -megabytes 10 EmptyMyApplication-MacOSX.dmg -layout NONE
hdid -nomount EmptyMyApplication-MacOSX.dmg
hdiutil create -megabytes 10 EmptyMyApplication-MacOSX.dmg -layout NONE hdid -nomount EmptyMyApplication-MacOSX.dmg
In this article I've tried to explain the process using cleaned up and simplified examples. They should be usable "as is" out of the box, but of course there is the chance that I've messed things up in creating the example. Therefore, as always, your best bet is to go and look at the real source code. The complete and debugged build script and source code for the for the classes mentioned can be found from my website.
Nobody is perfectThe method described in this article is Pure Java and is based on Ant. I've tested it with Eclipse on Mac OS X, Ubuntu, SuSE and Fedore Linuxes and Windows XP with some success. So a casual reader might think that this allows development on any platform for deployment on any platform (where Java is supported).
Unfortunately that is not quite so. The 'chmod' Ant task only works on MacOS and Linux. chmod is necessary to make the Linux executable, well, executable. Of course this is not absolutely necessary, you can leave this task to the end user, but for best user experience the executable bit should really be factory set. One way around this problem would be to use the 'tarfileset' Ant task that allows setting the mode bits for files packed into '.tar' files which could then be packaged as'.gz' files, which is common practice.
The 'cp' command to copy files is not available in Windows and while Ant support the 'copy' Ant task it does not appear to produce a package that MacOS would regard as a proper functional '.app' package. I suspect that this is because 'copy' does not preserve file permissions which I believe is an inherent limitation of the Java file IO API which cannot not support this concept because Windows, the lowest common denominator, doesn't support it.
One way to get around the afore mentioned problems on Windows would be to install and configure Cygwin  which brings the power of Unix to Windows. It certainly support the 'cp' command and I believe it can simulate file permissions with the 'ntea' option  .
Also there is no way, as far as I know, to create .dmg images on any other platform than Mac OS X. Of course this is not absolutely required if you are happy to deliver your application in '.gz','.zip' or '.tar' archive, which should be almost as good. In summary, if you wan to develop native looking pure Java applications with the described methodology for all platforms then the best platform for development is Mac OS X. In Linux and Windows you need to sacrify packaging the Mac executables in '.dmg' files. Developing in Windows also means finding a way to set the file permission as outlined above.
A word of caution, I've not tested the alternative methods outlined above, after all, I've moved all my development to Mac OS X so I have no need to ...
Also worth mentioning is that at present 'jstub' does not support transparency on appication icons, which is a pitty. As it is an Open Source project you should be able fix this though . While you are at it, please add support for the up coming Windows Vista icons too!