A couple of weeks ago, GridGain came to may attention, totally out of the blue. Now imagine my surprise when I decided to test it, following some of their examples. The freaking thing is easy to use and works out of the box. Run a script, a node is running. Do it again, a second one is running. Of course, for more complex scenarios you will have to configure a few things, but the point is that I could starting testing it right away.

The way to build and “deploy” services is ridiculously easy as well. I coded the Hello World example. When running, you start a new node in your main class, and in it you say what tasks you want to execute. GrigGain will then find all other available nodes and, depending on the task implementation, split the jobs between those nodes. Something I liked here is that the task implementation is sent to the other peers in a completely transparent way – you don’t even notice what’s happening.

I won’t dive too much into GridGain here, nor show any code, simply because of another great quality it has: documentation. There is a lot of stuff available, including a lot of easy-to-follow examples. Take a look for yourself. I won’t be using this framework for now, but if I end up needing to implement grid-related stuff, I certainly know where to start looking.

Direct link to the GridGain documentation.

The first thing to understand is that Mercurial is a decentralized version control system. So, each developer will have its own local repository. When you commit, you’re not committing to the server, but instead to your local repository. This is cool because this way you can commit very often, and without breaking anyone else’s code. Also, you can easily work offline (airplane, anyone?), and synchronize when you get internet access.

So, when you want to get the most recent version of the project, you pull it from the main repository. And then of course solve eventual conflicts. This allows you to be sure that your code is fine with the server’s, i.e. all the tests passes and so on. After that, you are ready to push your code to the main repository.

So, don’t forget that now, after commiting you code, it is NOT sent to the main repository just yet. It will only get there after you issue a Push from your local repository to the main one.

The exact way to do this will be different depending on the IDE you are using, but the concept is the same. Please consult the official site mentioned earlier and the documentation of your IDE for more detailed information. For NetBeans 6.1, for example, you can find something here.

Jxta is a platform for developing peer-to-peer applications. It supports the definition stated above, plus a lot of infrastructure features that most peer-to-peer applications might need. One example of such a feature is firewall traversal, which is done transparently for us. Also, it has some other very important qualities:

• It is open source and thus publicly available;

• It is programming language agnostic – there is at least a Java, a C++, a Ruby and a mobile implementation;
• It is communication protocol agnostic: it can run on top of TCP, HTTP, Bluetooth etc.

It is important to note that Jxta is not an API – that’s why it can be independent of the programming language. Jxta defines a set of protocols, that can be implemented in a variety of different platforms. Currently, these are the protocols specified by Jxta:

• Peer Resolver Protocol (PRP): provides the foundation for sending and responding generic queries between peers
  
• Rendezvous Protocol (RVP): used to help propagating queries between peers
• Peer Discovery Protocol (PDP): used to allow peers to publish and find resources (like other peers, services etc)
• Endpoint Routing Protocol (ERP): provides the mechanism that a peer can use to find routes to other peers, hiding low level details
• Pipe Binding Protocol (PBP): provides the means to make a connection between two peers
• Peer Information Protocol (PIP): used to get information from a peer, like its status or provided services for example

Those are of course very basic definitions of the protocols, but should be enough for now.

Jxta is not difficult to learn but it is extensive. If you are to fully use it, you have a lot to learn. You can find a lot of resources on the internet. There is the official site, which is the best starting point. And the community is active in the mailing lists. Also, I’m currently reading this book, which you can download for free.

So, first question: What?

After some discussions, we decided to use guigoh‘s conference as a first scenario to test the ideas. But before that, we should be able to have some independent peers communicating. This would be the very foundation for any work that comes later. Something like this:

peers communicating across a network

This means that Peer1 must be able to reach Peer2, and any other Peer, across a network. It shouldn’t have to worry about the details on how to find and send a message to them. Also, the peers cannot depend on any central server or any kind of node that could represent a single point of failure. And the same must be true for all peers.

Based on some ideas from guigoh‘s conference application, and from the Dynamic Virtual Super Peer (DVSP) model (more on this one in a future post, if Amir and Sotiris don’t mind ), plus the small concept explained above, we planned what should be done in terms of functionality. So this is it, in two simple steps:

The conference initialization: One peer creates a new conference, store its information in what we are calling ConferenceContext, and send invitations to the other peers it wants to have part in the conference. The ConferenceContext is also sent with the invitation. Each of these peers can accept or decline the invitation, which will then define who the conference participants are.

An ongoing conference: When the time for the conference arrives, a conference window is displayed at the peer’s screen. In this window, the user can type messages that the peers will exchange between themselves. In the end, it is a simple chat, but in a pure peer-to-peer architecture.

Now for the difficult question: How?

The next decision was technological: how is this going to be implemented? Doing everything from scratch is probably not a very good idea. Just Imagine having to write firewall traversal code by yourself, for a simple example of the trouble. There is just too much basic infrastructure work that would be necessary. And thus a lot of resources, which we don’t have.

There are a few options around the web, but not that much actually. I won’t list them here because I don’t tend to remember things I put aside (sorry :-p). The important thing to mention is the chosen technology: Jxta.

I won’t dive into Jxta right now. Instead, the next post will probably be about it. So if you have any questions feel free to send it to me, and I’ll try to address them.