Introducing Project

PureJavaComm

with

JTermios

Introduction

Some History

So why an another SerialPort when we already have two perfectly functional, battle tested implmementations already?

This project started of as a proof of concept that spiraled out of hand. There was some discussion on the rxtx@qbang.org mailing list about rewriting the implementation, and while I disagreed with the idea of rewriting I suggested that if such a rewrite would take place it should aim to implement as much of the code as possible in Java not C. And since JNA allows you to call any (well almost) shared/dynamic C-library from Java without writing a single line of C-code I suggested that it [rewrite of SerialPort] could all be done in Java with no C in sight.

This was met with the usual sketsism, so to prove my point and to research the issue (this has some interesting techincal challenges) I sketched non-functional prototype for SerialPort. I'm sceptical of rewrites because they throw away years of experience and debugging, so originally my plan was to leave it at that [non functional prototype], but the idea kept haunting me and I finally put everything else aside for a few days and made working prototype.

Pure Java / JNA

My reasoning was that giving the design and implementation challenges a priority is putting your self first instead of your users and customers.

If we take the users point of view, Sun's JavaComm is abandoned and not supported on many platforms and RXTX has its own issues, someone discribed them as paper cuts, which have not been addressed for years. So users are left with the choice of fixing any issues themselfs. And therein lies the problem with C which also bogs down the RXTX development.

The issue is C and C-tool chain.

Most people wanting to use JavaComm SerialPort have a strong Java background and have the Java tools and skills at hand. They can desing, code, debug and test Java code.

But C is a different beast.

It is a nice language, don't get me wrong, but it leaves maddening number of things 'implementation defined', which makes it hard to write portable code. Did you know that the type 'char' in C is not a byte by definition. Did you know that the size of types is defined in terms of the non-byte char? Did you know that the the standard only quarantees that sizeof(char) <= sizeof(short) <= sizeof(int) <= sizeof(long). And so on and so on.

But that is not the worst part. There are heaps of people with the C-skill set who know those things by heart, I'm one of them, so that is not the big issue. By far the worst part of C is the tool chain, ie compiler linker and system headers.

If you are developing in Mac OS X and want to do Windows stuff you need a computer for it and you are in for a day or two installing Windows and you can spend a few more days installing the tool chain, be it Visual Studio, MinGW or Cygwin and all the necessary SDKs and what not.

Multiply that by Linux.

Building binary distributions of any cross platform C-based library is a nightmare.

And you want do that because you want to deliver your library 'batteries included' so that it will work out of the box, instead of asking your users to install this and compile that.

Ok, so you are a developer, you are supposed to know this stuff and take the punishment.

Fair enough, but consider the average JavaComm user (putting the customer first!) who needs to and is willing to fix the paper cuts. How can you expect him/her to set up all this tool chain for one platform, let alone all the platforms (and RXTX supports an amazing number of platforms, hats off!).

With everything in Java, maintenance and debugging is so much easier, you just step into the code with your debugger, see problem, fix it in your platform and can be reasonably confident that you can, if you want to, share the fix and get it incorporated to the code base. Something that has not happened for years in RXTX.

Not to mention deployment issues.

Imaging a world with no DLLs or shared libraries, with no architecture issues,with no dependcies. That is what JNA is all about.

Just a single 'jna.jar' to rule them all.

And this is exactly the same what PureJavComm aspires someday to be: just include purejavacomm.jar in your Java class path and you are done.

Like Timothy Wall, the father and creator of JNA, wrote: "those of you who've never built multi-lingual projects involving C don't know what you're missing".

Project Status

The project is a work in progress and some of the interfaces 'below' JavaComm level are subject to change without notice.

At this point in time this is probably not for the casual user but the enterpricing coders who are not afraid to get their hands dirty.

The code is functionally close to complete but has as seen very limited testing.

The code passes a simple/naive loopback test purejavacomm.testsuite.TestSuite that uses an event listener on Linux, Windows and Mac OS X.

This project will never, realistically thinking, support such a variety of platforms as RXTX, which seems to support: Mac OS X, Linux, Windows, Solaris and them some, if I'm not mistaken.

Like I mentioned before I have no illusions about re-writes -- it will be years, if ever, that this project will reach the quality and breath of something like RXTX.

On the otherhand it can be useful already.

Missing Features

So if the operating system allows, you can open to SerialPorts to the same actual serial port. What happens if you do is your headache.

Getting the Code

The original project 'push' is also here as zip-file: purejavacomm.zip for reference only, for latest and greates go to the github.

There is now also a maven repository at:

http://www.sparetimelabs.com/maven2

To use the maven repository just add this to you '.pom' file:

   <dependency>
      <groupId>com.sparetimelabs</groupId>
      <artifactId>purejavacomm</artifactId>
      <version>xxx</version>
   </dependency>

where 'xxx' is the version number, currently (23.1.2015) it is 0.0.0.22 but it will change at some point, you need to look it up from the github.

Design Notes

Namely '#define' constants, macros and global variables, and to a lesser extent the varying native structures and type sizes.

Constants are not a big issues. You just have to look up the value of the constant and declare it in Java. For example:

This C-define :

#define O_NOCTTY 0x00020000

becomes in Java:

final static int O_NOCTTY 0x00020000;

The downside is that it is a bit fragile in that if the constant value changes (what a concept!) the Java code will not know about this whereas the C code will get the new value when it gets recompiled. If it gets recompiled. But for the kind of well established constants we are talking about here, this is a non-issue for me.

The other thing is that the values maybe architecture and/or platform dependent. The C-code again gets the correct values automatically, assuming you have get your tool chain properly configured! But the Java code needs to resort to simulating the constants with 'static ints' without 'final' and they need to be set at runtime based on the platform and architecture. Not a big deal, though it means you cannot use them as case-labels in switch statements.

Conseptually a bigger hurdle are macros, especially the FD_SET -family of macros. The worst of those is FD_SET macro itself, which actually masquarades as a 'type'. Posix nor any other standard does not define what the structure actually is, but looking at the headers it is clear that in most (in every?) case it is just a integer array, in other words a chunck of memory.

As this needs to be platform and architecture dependent with possibly endian issues, and we might as well be prepare for any implementation too, the actual allocation of the FD_SET and the associated SET/CLR operations are abstracted away in the interface.

In practice this means that all those #defines and macros that C-programmers get from the headers Java programmers get from the static class members and methods of jtermios.JTermios class. See below, Using the JTermios Library.

For global variables I see no solution but there is only one crucial that we need here: 'errno'. I've circumvented that problem by using the 'perror()' function to output the error code to the console. At least you can now see it even if you cannot access it from Java.

Architecture

One is to take the SerialPort as the main interface and then create an implementation for each platform. Obviously this has the disadvantage that there is potentially a lot of code and functionality duplication. The other route, which I've taken, is to implement the SerialPort in terms of some idealized serial port API.

It makes sense to model this idealized serial port API according to some existing operating system interface and write a compatibily layer for the others. As Windows is the odd man out and all the unixes are more or less the same it is natural to model the idealized serial port API along the POSIX standard and write an 'impedance matcher' for Windows.

So, without furter ado, jtermios.JTermios is that idealized interface.

Static methods and fields in that class serve as the POSIX serial API functions and defined constants. This makes it possible to use them using the Java static import like this

import static jtermios.JTermios.*;

after which their usage is about as close to C-usage as it can, leveraging on existing C-knowledge, examples and documentation.

In that same package (jtermios) there are four more idealized classes that serve the same purpose as their POSIX namesakes:

Termios
TimeVal
FDSet
Pollfd

JTermios delegates the actual implementation of 'termios' functionality to the platform and architecture specific JTermiosImpl classes.

When JTermios class loads it instantiates one of those implementing classes which you can find in the jtermios.macosx,jtermios.windows and ,linux packages.

Each of the JTermiosImpl classes need to implement the jtermios.JTermios.JTermiosInterface. So a pretty architecture picture would look something like:

This could be the end of the story, here you have a well known (look-a-like) cross platform serial port API, admittedly missing a few features, like port enumeration, but still.

Mission accomplished.

Well, not quite, PureJavaComm wants to build on the existing skills and knowledge of its users and POSIX style termios I/O may not be the most familiar API for Java programmes. But JavaComm SerialPort is a Java programmer friendly API and that is what purejavacomm.PureJavaSerialPort delivers.

To use it you just need to import definitions from purejavacomm names space, like this:

import purejavacomm.*;

And use it like Sun's javax.comm.JavaComm, in fact, there is no 'javadoc' for PureJavaComm, so you need to refer to the Sun Javacomm API documentation!

Of course you need to include the jna.jar and purejavacomm.jar jars in your classpath.

Using the JTermios Library

If you want, you can also use the jtermios.JTermios library to access the serial port very much in the same way you can from C.

Just do a static import for the class JTermios and you get access to many of the functions and constants that you normallye get from "fcntl.h", "termios.h" and related C-headers.

Here is an apetizer:

import termios.*;
import static jtermios.JTermios.*;
public class JTermiosDemo {
	public static void main(String[] args) {
		port = "/dev/tty.usbserial-FTOXM3NX";
		int fd = open(port, O_RDWR | O_NOCTTY | O_NONBLOCK);
		if (fd == -1)
			fail("Could not open " + port);

		fcntl(fd, F_SETFL, 0);

		Termios opts = new Termios();
		tcgetattr(fd, opts);
		opts.c_lflag &= ~(ICANON | ECHO | ECHOE | ISIG);
		opts.c_cflag |= (CLOCAL | CREAD);
		opts.c_cflag &= ~PARENB;
		opts.c_cflag |= CSTOPB;
		opts.c_cflag &= ~CSIZE;
		opts.c_cflag |= CS8;
		opts.c_oflag &= ~OPOST;
		opts.c_iflag &= ~INPCK;
		opts.c_iflag &= ~(IXON | IXOFF | IXANY);
		opts.c_cc[VMIN] = 0;
		opts.c_cc[VTIME] = 10;

		cfsetispeed(opts, B9600);
		cfsetospeed(opts, B9600);

		tcsetattr(fd, TCSANOW, opts);

		tcflush(fd, TCIOFLUSH);

		byte[] tx = "Not so very long text string".getBytes();
		byte[] rx = new byte[tx.length];
		int l = tx.length;
		int n = write(fd, tx, l);
		if (n < 0) {
			System.out.println("write() failed ");
			System.exit(0);
			}
		System.out.println("Transmitted '" + new String(tx) + "' len=" + n);

		FDSet rdset = newFDSet();
		FD_ZERO(rdset);
		FD_SET(fd, rdset);

		TimeVal tout = new TimeVal();
		tout.tv_sec = 10;

		byte buffer[] = new byte[1024];

		while (l > 0) {
			int s = select(fd + 1, rdset, null, null, tout);
			if (s < 0) {
				System.out.println("select() failed ");
				System.exit(0);
				}
			int m = read(fd, buffer, l);
			if (m < 0) {
				System.out.println("read() failed ");
				System.exit(0);
				}
			System.arraycopy(buffer, 0, rx, rx.length - l, m);
			l -= m;
			}

		System.out.println("Received    '" + new String(rx) + "'");
		int ec = close(fd);
		}
	}

If you are familiar with termios you can see that you have to look carefully to distinquish this Java code from C!

Using the WinAPI Library

Or use it as an example code on how to access those Windows API functions from Java, as such code is hard to find, especially concerning the assynchronous or overlapped I/O.

Here is an apetizer:

import com.sun.jna.Memory;
import static jtermios.windows.WinAPI.*;
import jtermios.windows.WinAPI.*;

public class TestSuite {
	public static void main(String[] args) {
		String COM = "COM5:";
		HANDLE hComm = CreateFileA(COM, GENERIC_READ | GENERIC_WRITE, 0, null, OPEN_EXISTING, FILE_FLAG_OVERLAPPED, null);

		check(SetupComm(hComm, 2048, 2048), "SetupComm ");

		DCB dcb = new DCB();
		dcb.DCBlength = dcb.size();
		dcb.BaudRate = CBR_1200;
		dcb.ByteSize = 8;
		dcb.fFlags = 0;
		dcb.Parity = NOPARITY;
		dcb.XonChar = 0x11;
		dcb.StopBits = ONESTOPBIT;
		dcb.XonChar = 0x13;

		check(SetCommState(hComm, dcb), "SetCommState ");

		COMMTIMEOUTS touts = new COMMTIMEOUTS();
		check(SetCommTimeouts(hComm, touts), "SetCommTimeouts ");

		check(!INVALID_HANDLE_VALUE.equals(hComm), "CreateFile " + COM);
		String send = "Hello World";
		int tlen = send.getBytes().length;

		int[] txn = { 0 };
		Memory txm = new Memory(tlen + 1);
		txm.clear();
		txm.write(0, send.getBytes(), 0, tlen);

		int[] rxn = { 0 };
		Memory rxm = new Memory(tlen);

		OVERLAPPED osReader = new OVERLAPPED();
		osReader.writeField("hEvent", CreateEventA(null, true, false, null));
		check(osReader.hEvent != null, "CreateEvent/osReader");

		OVERLAPPED osWriter = new OVERLAPPED();
		osWriter.writeField("hEvent", CreateEventA(null, true, false, null));
		check(osWriter.hEvent != null, "CreateEvent/osWriter");

		first = false;
		check(ResetEvent(osWriter.hEvent), "ResetEvent/osWriter.hEvent");
		boolean write = WriteFile(hComm, txm, tlen, txn, osWriter);
		if (!write) {
			check(GetLastError() == ERROR_IO_PENDING, "WriteFile");
			System.out.println("Write pending");
			}
		while (!write) {
			System.out.println("WaitForSingleObject/write");
			int dwRes = WaitForSingleObject(osWriter.hEvent, 1000);
			switch (dwRes) {
				case WAIT_OBJECT_0:
					if (!GetOverlappedResult(hComm, osWriter, txn, true))
						check(GetLastError() == ERROR_IO_INCOMPLETE, "GetOverlappedResult/osWriter");
					else
						write = true;
					break;
				case WAIT_TIMEOUT:
					System.out.println("write TIMEOT");
					break;
				default:
					check(false, "WaitForSingleObject/write");
					break;
				}
			}
		System.out.println("Transmit: '" + txm.getString(0) + "' , len=" + txn[0]);

		check(ResetEvent(osReader.hEvent), "ResetEvent/osReader.hEvent ");
		boolean read = ReadFile(hComm, rxm, tlen, rxn, osReader);
		if (!read) {
			check(GetLastError() == ERROR_IO_PENDING, "ReadFile");
			System.out.println("Read pending");
			}

		while (!read) {
			System.out.println("WaitForSingleObject/read");
			check(ResetEvent(osReader.hEvent), "ResetEvent/osReader.hEvent");
			int dwRes = WaitForSingleObject(osReader.hEvent, 1000);
			switch (dwRes) {
				case WAIT_OBJECT_0:
					if (!GetOverlappedResult(hComm, osReader, rxn, false))
						check(GetLastError() == ERROR_IO_INCOMPLETE, "GetOverlappedResult/osReader");
					else
						read = true;
					break;
				case WAIT_TIMEOUT:
					System.out.println("WAIT_TIMEOUT");
					break;
				default:
					check(false, "WaitForSingleObject/osReader.hEvent");
					break;
				}
			}

		System.out.println("Received: '" + rxm.getString(0) + "' , len=" + rxn[0]);
		check(CloseHandle(osWriter.hEvent), "CloseHandle/osWriter.hEvent");
		check(CloseHandle(osReader.hEvent), "CloseHandle/osReader.hEvent");
		check(CloseHandle(hComm), "CloseHandle/hComm");
		}

	private void check(boolean ok, String what) {
		if (!ok) {
			System.err.println(what + " failed, error " + GetLastError());
			System.exit(0);
			}
		}
	}

Logging

	jtermios.JTermios.JTermiosLogging.setLogLevel(1);

At level 0 (almost) nothing is logged. At level 4 you'll see all the calls to Windows functions and their parameters.

The logging uses a cute little idiom in the code:

	log = log && log(3,"lazyly evalueted printf like logging text %s\n","here");

What the above does it evaluates the printf like logging text only if loggin is turned on. It uses a global variable log, short circuit evaluation using '&&' and a static function log() that takes the log level as a parameter and variable number of 'printf' like arguments.

Now how cute is that!

I know, not everyone likes it but think it is easily memorable, sort of mnemonic, and does not clutter the source code much and at level 0 has very little effect on performance.

Adding More Platforms

To implement more platforms, say for example FreeBSD, do the following.

Implement a new class jtermios.freebsd.JTermiosImpl and implement the jtermios.JTermios.JTermiosInterface in that class. It is probably easiest if you copy/paste the code from the jtermios.linux.JTermiosImpl class and take it from there.

Your new class will be pretty much just a very thin wrapper around the JNA calls to the FreeBSD API.

In the constructor of that class initialize the correct values for all the static constants in the jtermios.JTermios such as O_RDWR,O_NONBLOCK etc etc.

You need to look up those values from the C include headers for the correct architecture and your platform. This maybe a bit tedious and error prone, so you may want to utilize the "c-linux.c" program in the "c" directory which prints out most of the values when compiled with the correct architecture with something like:

gcc -arch i386 -c c-linux.c && ./a.out

file a.out

Lastly you need to add instantiation code to the static initializer block in jtermios.JTermios, look for

static { // INSTANTIATION
		if (Platform.isMac()) {
			m_Termios = new jtermios.macosx.JTermiosImpl();
		} else if (Platform.isWindows()) {

and add your instantiation code there. Note that you may have to implement and instantiate code not only depending on the platform (Java system property "os.name") but also on the architecture (Java system property "os.arch").

Licensing/Copyright

The code is Copyrighted by me and is licensed under "Simplified BSD License".

I spent a fair amout of time thinking about the license and in the end I chose BSD license as it hopefully creates the least amount of trouble for the users. Of course it can generate issues down the line with forks and contributions that want to add their own licenses.

But there it is, the cat is out of the bag and I little control over what happens next.

I have two concerns.

I would hope that the project will not be immediately forked but that contributions, if any, would be concentrated on this project.

If forking or modifications happen they need to be clearly indentifiable as forks or modifications.

I'm not especially looking for contributions and it maybe that this project will never amount to much more than what is available today, like said, this is a project that spiralled out of hand and at the moment I just want to get this off my chest.

If you insist on contributing please note that I may insist on getting the copyright of the contributions transferred to me, to keep future licensing options free.

I can be contacted at feedback2(@)sparetimelabs.com

with best regards,

      Kustaa "Kusti" Nyholm