|Spare Time Labs 2.0|
PIC CDC ACM
Fun with HC08
Java Goes Native
Gas Fired Furnace
Down Memory Lane
IntroductionThis project aims to implement a thread safe, Open Source, pure Java, drop-in replacement for Sun's and RXTX project's JavaComm SerialPort on Mac OS X, Linux and Windows platforms. If you are reading this you probably know already what JavaComm is but in short it is a platform indepent way to access serial ports (RS232) from Java applications. Just what the doctor ordered.
So why an another SerialPort when we already have two perfectly functional, battle tested implmementations already?
This project started of as a proof of concept that spiraled out of hand. There was some discussion on the email@example.com mailing list about rewriting the implementation, and while I disagreed with the idea of rewriting I suggested that if such a rewrite would take place it should aim to implement as much of the code as possible in Java not C. And since JNA allows you to call any (well almost) shared/dynamic C-library from Java without writing a single line of C-code I suggested that it [rewrite of SerialPort] could all be done in Java with no C in sight.
This was met with the usual sketsism, so to prove my point and to research the issue (this has some interesting techincal challenges) I sketched non-functional prototype for SerialPort. I'm sceptical of rewrites because they throw away years of experience and debugging, so originally my plan was to leave it at that [non functional prototype], but the idea kept haunting me and I finally put everything else aside for a few days and made working prototype.
Pure Java / JNAWhy this insistance on Pure Java and JNA, especially as this particular task [accessing serial port] presents some challenges that are in some respect easier to handled in C (see Design Notes below) than Java?
My reasoning was that giving the design and implementation challenges a priority is putting your self first instead of your users and customers.
If we take the users point of view, Sun's JavaComm is abandoned and not supported on many platforms and RXTX has its own issues, someone discribed them as paper cuts, which have not been addressed for years. So users are left with the choice of fixing any issues themselfs. And therein lies the problem with C which also bogs down the RXTX development.
The issue is C and C-tool chain.
Most people wanting to use JavaComm SerialPort have a strong Java background and have the Java tools and skills at hand. They can desing, code, debug and test Java code.
But C is a different beast.
It is a nice language, don't get me wrong, but it leaves maddening number of things 'implementation defined', which makes it hard to write portable code. Did you know that the type 'char' in C is not a byte by definition. Did you know that the size of types is defined in terms of the non-byte char? Did you know that the the standard only quarantees that sizeof(char) <= sizeof(short) <= sizeof(int) <= sizeof(long). And so on and so on.
But that is not the worst part. There are heaps of people with the C-skill set who know those things by heart, I'm one of them, so that is not the big issue. By far the worst part of C is the tool chain, ie compiler linker and system headers.
If you are developing in Mac OS X and want to do Windows stuff you need a computer for it and you are in for a day or two installing Windows and you can spend a few more days installing the tool chain, be it Visual Studio, MinGW or Cygwin and all the necessary SDKs and what not.
Multiply that by Linux.
Building binary distributions of any cross platform C-based library is a nightmare.
And you want do that because you want to deliver your library 'batteries included' so that it will work out of the box, instead of asking your users to install this and compile that.
Ok, so you are a developer, you are supposed to know this stuff and take the punishment.
Fair enough, but consider the average JavaComm user (putting the customer first!) who needs to and is willing to fix the paper cuts. How can you expect him/her to set up all this tool chain for one platform, let alone all the platforms (and RXTX supports an amazing number of platforms, hats off!).
With everything in Java, maintenance and debugging is so much easier, you just step into the code with your debugger, see problem, fix it in your platform and can be reasonably confident that you can, if you want to, share the fix and get it incorporated to the code base. Something that has not happened for years in RXTX.
Not to mention deployment issues.
Imaging a world with no DLLs or shared libraries, with no architecture issues,with no dependcies. That is what JNA is all about.
Just a single 'jna.jar' to rule them all.
And this is exactly the same what PureJavComm aspires someday to be: just include purejavacomm.jar in your Java class path and you are done.
Like Timothy Wall, the father and creator of JNA, wrote: "those of you who've never built multi-lingual projects involving C don't know what you're missing".
The project is a work in progress and some of the interfaces 'below' JavaComm level are subject to change without notice.
At this point in time this is probably not for the casual user but the enterpricing coders who are not afraid to get their hands dirty.
The code is functionally close to complete but has as seen very limited testing.
The code passes a simple/naive loopback test
This project will never, realistically thinking, support such a variety of platforms as RXTX, which seems to support: Mac OS X, Linux, Windows, Solaris and them some, if I'm not mistaken.
Like I mentioned before I have no illusions about re-writes -- it will be years, if ever, that this project will reach the quality and breath of something like RXTX.
On the otherhand it can be useful already.
Missing FeaturesPort locking is not implemented and I have no plans to include that. Lock files seem to be more trouble than they are worth and in any case do not solve the exclusive accesse issue between native applications and Java appilications.
So if the operating system allows, you can open to SerialPorts to the same actual serial port. What happens if you do is your headache.
Getting the CodeThe code is hosted at: https://github.com/nyholku/purejavacomm
The original project 'push' is also here as zip-file: purejavacomm.zip for reference only, for latest and greates go to the github.
There is now also a maven repository at:
To use the maven repository just add this to you '.pom' file:
Design NotesAs soon as I started to investigate the issue at hand it was obvious that there are a few major hurdels in operating system APIs that present little difficulty to C-programmers but are a bit of a pain for JNA users.
Namely '#define' constants, macros and global variables, and to a lesser extent the varying native structures and type sizes.
Constants are not a big issues. You just have to look up the value of the constant and declare it in Java. For example:
This C-define :
becomes in Java:
The downside is that it is a bit fragile in that if the constant value changes (what a concept!) the Java code will not know about this whereas the C code will get the new value when it gets recompiled. If it gets recompiled. But for the kind of well established constants we are talking about here, this is a non-issue for me.
The other thing is that the values maybe architecture and/or platform dependent. The C-code again gets the correct values automatically, assuming you have get your tool chain properly configured! But the Java code needs to resort to simulating the constants with 'static ints' without 'final' and they need to be set at runtime based on the platform and architecture. Not a big deal, though it means you cannot use them as case-labels in switch statements.
Conseptually a bigger hurdle are macros, especially the FD_SET -family of macros. The worst of those is FD_SET macro itself, which actually masquarades as a 'type'. Posix nor any other standard does not define what the structure actually is, but looking at the headers it is clear that in most (in every?) case it is just a integer array, in other words a chunck of memory.
As this needs to be platform and architecture dependent with possibly endian issues, and we might as well be prepare for any implementation too, the actual allocation of the FD_SET and the associated SET/CLR operations are abstracted away in the interface.
In practice this means that all those #defines and macros that C-programmers get from the headers Java programmers get from the static class members and methods of
For global variables I see no solution but there is only one crucial that we need here: 'errno'. I've circumvented that problem by using the 'perror()' function to output the error code to the console. At least you can now see it even if you cannot access it from Java.
ArchitectureThere are basically two choise when trying to create a cross platform serial API.
One is to take the SerialPort as the main interface and then create an implementation for each platform. Obviously this has the disadvantage that there is potentially a lot of code and functionality duplication. The other route, which I've taken, is to implement the SerialPort in terms of some idealized serial port API.
It makes sense to model this idealized serial port API according to some existing operating system interface and write a compatibily layer for the others. As Windows is the odd man out and all the unixes are more or less the same it is natural to model the idealized serial port API along the POSIX standard and write an 'impedance matcher' for Windows.
So, without furter ado,
Static methods and fields in that class serve as the POSIX serial API functions and defined constants. This makes it possible to use them using the Java static import like this
after which their usage is about as close to C-usage as it can, leveraging on existing C-knowledge, examples and documentation.
In that same package (
Each of the
This could be the end of the story, here you have a well known (look-a-like) cross platform serial port API, admittedly missing a few features, like port enumeration, but still.
Well, not quite, PureJavaComm wants to build on the existing skills and knowledge of its users and POSIX style termios I/O may not be the most familiar API for Java programmes. But JavaComm SerialPort is a Java programmer friendly API and that is what
To use it you just need to import definitions from
And use it like Sun's
Of course you need to include the
Using the JTermios Library
If you want, you can also use the
Just do a static import for the class
Here is an apetizer:
If you are familiar with termios you can see that you have to look carefully to distinquish this Java code from C!
Using the WinAPI LibraryThe class
Or use it as an example code on how to access those Windows API functions from Java, as such code is hard to find, especially concerning the assynchronous or overlapped I/O.
Here is an apetizer:
LoggingThe library implements a rudimentary logging which you can turn on and set the level of using:
At level 0 (almost) nothing is logged. At level 4 you'll see all the calls to Windows functions and their parameters.
The logging uses a cute little idiom in the code:
What the above does it evaluates the printf like logging text only if loggin is turned on. It uses a global
Now how cute is that!
I know, not everyone likes it but think it is easily memorable, sort of mnemonic, and does not clutter the source code much and at level 0 has very little effect on performance.
Adding More Platforms
To implement more platforms, say for example FreeBSD, do the following.
Implement a new class
Your new class will be pretty much just a very thin wrapper around the JNA calls to the FreeBSD API.
In the constructor of that class initialize the correct values for all the static constants in the
You need to look up those values from the C include headers for the correct architecture and your platform. This maybe a bit tedious and error prone, so you may want to utilize the "c-linux.c" program in the "c" directory which prints out most of the values when compiled with the correct architecture with something like:
gcc -arch i386 -c c-linux.c && ./a.out
gcc -arch i386 -c c-linux.c && ./a.out
Lastly you need to add instantiation code to the static initializer block in
and add your instantiation code there. Note that you may have to implement and instantiate code not only depending
on the platform (Java system property
The code is Copyrighted by me and is licensed under "Simplified BSD License".
I spent a fair amout of time thinking about the license and in the end I chose BSD license as it hopefully creates the least amount of trouble for the users. Of course it can generate issues down the line with forks and contributions that want to add their own licenses.
But there it is, the cat is out of the bag and I little control over what happens next.
I have two concerns.
I would hope that the project will not be immediately forked but that contributions, if any, would be concentrated on this project.
If forking or modifications happen they need to be clearly indentifiable as forks or modifications.
I'm not especially looking for contributions and it maybe that this project will never amount to much more than what is available today, like said, this is a project that spiralled out of hand and at the moment I just want to get this off my chest.
If you insist on contributing please note that I may insist on getting the copyright of the contributions transferred to me, to keep future licensing options free.
I can be contacted at feedback2(@)sparetimelabs.com
with best regards,
Kustaa "Kusti" Nyholm