Java NIO - Channels

Channels are the other most important part of the NIO API.

A channel is a communication channel, that is a pipe of data.

The Channel interface has two methods: isOpen() and close() ( the latter may throws an IOException). Most Channels are specified using Interfaces. For example, there is the InterruptibleChannel interface which is a marker interface specifying how the channel behaves when a thread accessing it is interrupted.

There are different types of Channels.

• ReadableByteChannel and WritableByteChannel (Interfaces which define how to read from or write in a channel using a ByteBuffer)
• GatheringByteChannel and ScatteringByteChannel (interfaces defining methods to read from or write to a channel using an array of ByteBuffers)
• InterruptibleChannel ( interface defining the behaviour of the channel when the thread using it is interrupted)
• FileChannel (class for the interaction with a Channel based on a File)
• SocketChannel and ServerSocketChannel, and DatagramChannel ( classes for the communication based on sockets)

Channels cannot be opened directly. The method used to create a Channel object depends on the type of Channels. For example FileChannel are opened using Streams or RandomAccessFiles.

ChannelsUtilityClass There is a Utility Class called java.nio.channels.Channels, which has a number of utility methods to convert between the old and the new API. There are two Channel creating methods: newChannel(InputStream) and newChannel(OutputStream) which return a ReadableByteChannel and a WritableByteChannel respectively, and there are the newInputStream(ReadableByteChannel ch) and newOutputStream(WriteableByteChannel ch) which return InputStream and OutputStream respectively. The same kind of methods exist also for Readers and Writers: newReader and newWriter methods return the appropriate classes.

FileChannel FileChannel have different facets. Using the utility class Channels we can open FileChannel instances. Note that Channels opened from InputStreams are only readable while Channels from OutputStreams are only writable.

A number of methods are important for FileChannels. The force() methods writes to the disk the data which is actually cached ( for performance reasons). The truncate() methods gives a way to cut off all the data which is further than a given data. The methods position() and positions(long positions) are methods to decide at which part of the file the Channel should go.

Pipe Another important class is the Pipe class in general it is a conduit through which data can only go in one direction. Each pipe has a SourceChannel and a SinkChannel. The information written in the SinkChannel (a WritableChannel) is then transmitted to the SourceChannel (a ReadableChannel).

MemoryMappedFiles This type of Buffer and Channels are a far reaching topic so we treat them separately. SocketChannels SocketChannels are also a separate topic so we treat them also separately. Locks The topics of locks will also be covered in its own entry.

Java NIO - Buffers

We will start the explanation of the Java NIO main classes by taking a look at buffers.

Buffers

The Buffers contain a fixed amount of data (obtained using the capacity() method). They have a number of useful methods, for example: mark, reset, clear, flip and rewind. mark remenbers a position to which on can come back by using reset.

Access methods for the buffers are the get() and put() methods (these can be absolute or relative, i.e either one specifies the position in the buffer to set or retrieve or one gets or retrieve the current element).

There are one type of Buffer for each of the main non Boolean primitive types ( plus one called the MappedByteBuffer used when using memory-mapped files). All these Buffer types are abstract but possess a factory for that type of Buffer in order to create one. Different methods exist for example in the case of a CharBuffer: allocate( int capacity), or wrap(char[] array ) which return CharBuffers either of the given capacity or containing the elements found in the array. Note: the changes to the underlying array will be seen in the buffer and vice versa. This type of buffer have a non direct access. In other words they have an array used to contain the data. using the method hasArray() allows you to check whether this Buffer is non direct or not.

A special case has to be explained that direct buffer are actually of type ByteBuffer and are created using the allocateDirect(in capacity) method. The ByteBuffer class also provides the possibility to create View Buffers. This allows the possibility of obtaining instances of other types of Buffers, e.g CharBuffer or IntBuffer without taking care of the problem of conversion. This is performed for example using the asCharBuffer() method.

Many methods of the Buffer classes return also buffer references. Thus you can perform sequence of actions on the buffer in a simple way. This is called invocation chaining.

Other methods can also create views on other buffers. These methods allow for example the duplication of a buffer (using the duplicate() method), its view as a read only buffer (using the asReadOnlyBuffer() method), or a view of only part of this buffer (using the slice()) method).

Java NIO - Intro

Here is an entry on the topic of Java NIO in order to keep in mind the main aspects of these API. First of all, Java NIO stands for Java New I/O. There are a number of useful classes in this new API:

• buffers (they represent fixed size arrays of primitive data elements wrapped with supplementary state information)
• Channels (a model for a communiation connection)
• File locking and memory mapped files (provides classes for monitoring locking of files as well as the possibility of mapping files to memory thus improving the process)
• sockets (these classes provies new methods of interacting with network sockets)
• selectors (this allows the possibility to watch the status of different channels to monitor them more easily
• regular expressions (Perl like regular expressions for processing of Java)
• character sets (the mappings between characters and byte streams)

Models for Enterprise Architectures

According to Wikipedia articles there exist a number of models for Enterprise Architectures.

• The Open Group Architecture Framework (TOGAF)
• Zachman Framework (IBM Framework from the 1980s, now claimed as a de facto world standard)
• Department of Defence Architectural Framework
• Federal Enterprise Architecture Framework (United States Office of Management and Budget Federal Enterprise Architecture)
• United Kingdom Ministry of Defence Architectural Framework
• AGATE (French Délégation Générale pour l'Armement Atelier de Gestion de l'ArchiTEcture des systèmes d'information et de communication)

I think taking a look at these different modelling approaches may prove helpful in order to discuss more easily company structures as well as technical aspects.

GIT

The original developer of Linux has built another versioning system in order to help his linux kernel maintenance job. This tool is git. The Karlsruher Linux User Group has a presentation summing up the features of this distributed versioning system which I am going to use. Git repositories seem also to be smaller in size than their equivalent repositories under CVS or SVN. The presentation gives an example for gcc, which is 450 MB with git while 11GB using SVN. One of the important feature is that this system is distributed. Only few commands are needed, and help is available: > git command -h gives a short help while > git command --help gives a manual page Initialisation is performed using the following commands: > git config --global user.name "Your Name" > git config --global user.email your@e.mail Initalisation of a new repository is performed using: > git init in the location of the repository (i.e. > cd intherepositorydirectory beforehand) To clone an existing repository: > git clone giturl [dir] where giturl is the url of the repository (with http, git, ssh, user@host:repository, dir are acceptable as protocols). See git clone --help for the possibilities. Optimisations are even possibles using hardlinks and softlinks or reuse existing informations. To prepare commits, useful commands: status and diff. I don't need to explain why they are here for. To add elements to the repository, the add command is necessary which also adds directories recursively. changes are added with git-add. Commits are performed using the command: > git commit -m ’message’ Of course, it is possible to tell which files should be commited. This is done by using -- filenames at the end of the commandline. Also take a look at git commit -a -m ’message’. There is also a menu controlled used. for this use: > git commit --interactive -m ’message’ To check the code out, use: > git checkout [-m] -b newbranch [head] you can also list the branches, using git branch, or change branch: > git checkout branch or delete branches: git branch -d name. There is also the possibility to add remote repositories using: > git remote add name giturl where name is the name of the remote repository and giturl the url of the remote repository. you can fetch the changes using: > git fetch [name] [refspec] where name is the name of the remote repository and refspec is the mapping: +srcbranch:dstbranch you can merge changes using > git pull [name] [refspec] There is also > git merge but its use is infrequent. Other features are the publication of changes using mails or using the git server. There are also commands for the maintenance: > git fsck makes concistency checks on the repository and > git gc compresses the repository. Some other features exist like: signed tags, find bugs semi automatically, ...

Google Web Toolkit

The Google Web Toolkit (GWT) is a framework used to develop Webapplications. It works by transforming the Java in Javascript. Moreover, GWT has the following additional features:

1. Interfaces for RPC Calls
2. Integration with JUnit
3. Lots of Widgets for GUI Design (similar to Swing in the way of its use

It seems that GWT uses mainly Java to develop the application and theat therefore the typical Java development tools can be reused for the purpose

AJAX

Thsi is a beginning of a long serie on the technologies which make AJAX. The basic principle of AJAX is making Web pages more responsive by allowing more complex interaction with the servers without requiring the reloading of a page. The name AJAX means asynchronous Javascript with XML. Two main important applications:

1. load pages more quickly through an asynchron loading of the page(i.e the whole page is not loaded immediately).
2. depending on the actions of the user, appropriate data can be loaded without requiring the reloading of the page

Most applications require for this task the use of XMLHttpRequest Objects.

StAX- Streaming API for XML in Java

A number a way exist to access XML Data from Java. The DOM and SAX API are the two most traditional ones. A new alternatives under the form of the Streaming API for XML. This explanation uses the informations found in this tutorial.

The principle behing this API it to give the programmer the control over the next element of the file to be treated. This simplifies things greatly when there are many elements.

There are mainly two sets of APIs with the StaX API: cursor and iterator APIs.

Cursor API The cursor walks the documents one infoset element at a time and always forward. For this there are two interfaces: XMLStreamReader and XMLStreamWriter. These interfaces are very similar to the SAX handlers.

Iterator API

The Iterator API works on the idea, that the XML documents can be seen as a sequence of Events. Each of these Events are of a given type: StartDocument, StartElement, Attribute, ....

Differences. First of all, there are some things that are possible with the Iterator API which are not possible with the Cursor API. However, with the cursor API the code tends to be smaller and more efficient.

For processing events pipelines or modifiying the event stream use the iterator API.

In general, it is preferable to use the Iterator API, since this API is more easily extensible, thus it probably requires less refactoring for future versions of the applications.

Use of StaX APIs

The use of the StaX APIs is performed using the XMLInputFactory, XMLOutputFactory and XMLEventFactory classes. These factory classes are configured using the setProperty() method.

JAXB

Another interesting Java technology pushed by Sun is JAXB, it stands for "Java Architecture XML Binding". Most of the information found here comes from Sun's JAXB tutorial. It is a means of coding Java Object into XML using the descriptive power of the XML Schemas. It is possible to generate Java Object from XML descriptions using a Schema. To simplify the process, it is also possible to use JAXB annotations of Java classe in order to generate the schemas needed for such schemas. So the important steps to consider when using this is:

1. write the classes of the objects which should be stored as XML files, or DOM nodes, etc...
2. annotate them so that the corresonding schema can be generated
3. generate content according to schema
4. unmarshal the content (i.e load into content tree in Java)
5. validate if required
6. marshall, i.e export again as XML

This page gives a list of (XML)Schema-Java correspondance tags. As specified, the binding can be generated either using annotations inside the Java files, or it is also possible to use external configuration files outside these files.

Data Mining

This is the beginning of a series of post on the topics of Data Mining and Knowledge Discovery or KDD. There exists a large amount of resources on these topics. I plan to draw a picture of the possible open source solutions for Data Mining. I also want to recall which are the main applications of Data Mining. Open Source Systems The main open source systems for Data Mining are:

• Pentaho (used to be called WEKA)
• Rapid Miner (used to be called YALE, some parts of the systems were using Weka)
• Orange

for Text Mining, it is certainly important to add: the GATE system. I will review each of these systems. Supervised and unsupervised tasks One of the main distinction for algorithms and problems in Data Mining is whether the task to be performed is supervised or unsupervised. An algorithm is supervised if the goal of the algorithm is to create a model which is appropriate to categorise data according to an existing model. On the contrary, an unsupervised task consists in building a model which is appropriate to describe the data. For example, suppose we have a list of topics and news items which should be categorised in this list of topic. This is a typical supervised task. If we had no topics and were supposed to group the news items together according to some similarity criteria. This would be a unsupervised task. N.B: This entry does not really please me yet and is subject to change.