Extreme Feedback from the Nebula: The Oscilloscope

Extreme Feedback

When I was reading the Hudson manual a few days ago, I came across a section called "Extreme Feedback" where people are experimenting with feedback from the build system. Devices vary from cute bears to traffic lights and actual smell emitters.

Now I am looking for an genuine hospital heartbeat monitor to top all that. If you find one, let me know.

Until that time ...

We are currently building a new software machine that is based on OSGi services. Some of these services are really important and without them the machine will not run. For example, there is the Store service to persist and retrieve data. I don't know why but a few months ago I got this idea that a dashboard with some heartbeat monitors would be a nice way to a) impress our customers and b) to get immediate feedback in case one of the services would die.

After playing around for a few hours on a sunny Sunday when I should really not sit behind my computer, the first version of the OSGilloscope was born. I especially liked the name, but changed it later. The first version was slow as a snail and consumed all of my CPU and I had to tinker quite a bit to be able to run many scopes without problems.

After adding a few nice options, including sound (Heartbeat, Flatline, ..) I asked the Nebula guys if they wanted to have it, and they did!

I have included the scope into the Nebula Examples view. If you want to take a look please follow these five easy steps:

1. Start Eclipse with a new workspace

2. Create a java project

3. Create a file called widget.psf and load it with this content

4. Right click on the file and select "Import Project Set ..."

5. Open the nebula.product file from the example project and press "Launch an Eclipse Application"

After this is done, open the Oscilloscope tab and play around with the options. It is a lot of fun.

How to use the scope

If you want to use the widget please look at the included snippets project. You can just launch each snippet as a Java application. The widget comes with a special abstract OscilloscopeDispatcher class that keeps the animation running if you call the dispatch() method.

It does that by putting a request on the Display queue with a delay of 10-n milliseconds. The higher the delay, the slower the scope will run. You can specify the delay by implementing the getDelayLoop() method.

Off course, the scope wants to display a value. You get an opportunity to put a value in the scope every "Pulse". The value of the pulse can be specified by implementing the getPulse() method. If the pulse is 60 and the delay loop is 20 then every 1200 milliseconds something can be displayed in the scope. This can be a single value which will spike the scope by calling the getOscilloscope().setValue(int) method or

it can be a value series by calling the getOscilloscope().setValues(int[]) method. e.g. the Oscilloscope.HEARTBEAT field will draw a heartbeat like shape. The aforementioned method can lose values if you overflow the stack. The stack is circular meaning that if you push too many values in the method then the tail will overwrite the top.

If you want to draw something in "real time" then the pulse must be set to one. This will give you the opportunity to input a value every cycle.

Alternatively you can register a listener that gets notified if the stack is empty. An example of this can be found in Snippet4.

Please give it a spin. All feedback is appreciated, even extreme feedback.

Helios Bloghaton: Fun with Eclipse Remote Services - 1

In many hotels there is no 13th floor, let alone a room with number 13. Everyone knows that there are 13 steps leading up to the gallows and if you suffer from friggatriskaidekaphobia then you don't like Friday the 13th.

Nevertheless, the OSGi Alliance decided to assign number 13 to the Remote Services chapter. This is particularly creepy since the book starts with Chapter 1 and then jumps straight to chapter 13. After chapter 13 the next one is numbered 101. It is as if the Alliance almost wanted to proof that superstition is just that.

Eclipse Remote Services Implementation
And the proof sticks because Chapter 13 is a great chapter. ECF has implemented the remote services specification in February with the 3.2 release of ECF. Remote services come in two parts Discovery [1] and Distribution [2]. This article is about remote services in combination with the new Zookeeper based discovery provider.

To the fun

We have created a few Example projects to show you how remote services work and the user interface of this one is depicted above. It uses a cool new SWT Widget which I have named the OSGilloscope (watch out for a separate blog on this subject.) The other part of the UI consists of a tabbed folder. On one page a new quote is displayed every 15 seconds by a discovered remote service, and on the other page there is information about this remote service.

Get the sources

Please fire up a Helios with a new workspace and get the two bundles needed for this example from this project set** (copy this file in a project, give it the extension .psf, right click and choose Import Project Set...).

** Our recent move of non-epl code to github has rendered the psf file unusable for the org.eclipse.ecf.services.quotes project. Please get these projects from github and import it using "File/Import../Existing projects into workspace" and then point to the archive.

If this is a fresh Helios then you also have to download the ECF components. Point your p2 to http://download.eclipse.org/releases/helios and get the Communication Framework extensions from the EclipseRT project.

Run it

Inside the consumer bundle there is a file ending with .product. Open it and click on the link "launch an Eclipse application".

Yazafatutu.com

My server is named after somebody I once met in a dream. A long and strange story which I will tell you for a beer at Eclipse Summit Europe. This server is hosting the other side of this example and it uses ECF Zoodiscovery which in turn uses Apache Zookeeper from the Hadoop project. Zookeeper servers can be grouped to form a network of continuously synchronizing servers. Zookeeper is designed to replicate configuration data and it does that very fast and very reliable.

At Yazafatutu.com I have started an OSGi container and in there the ECF remote services and Zoodiscovery are running. Zoodiscovery is configured to run in StandAlone mode which means that there is no replication but only one central server. A so called hub-and-spoke configuration.

In this OSGi container there are three implementations of the QuoteService interface which is located in the other project you have just downloaded: org.eclipse.ecf.services.quotes. Each implementation resides in a separate bundle and the bundles are started and stopped one by one. Starting a bundle causes the Zoodiscovery software to broadcast a publication message to its clients.

Eclipse Twitter

Press the connect button on your user interface and your client will connect the the Zoodiscovery instance running on yazafatutu.com. After a few seconds, your OSGilloscope will come to live and quotes are shown every time a new bundle is started on the server. One of the three implementations of the QuoteService runs an Eclipse Twitter quote service. If you are on twitter then your quote could be there as well!

The Info tab

The Info tab shows some information about the discovered server. Take a look at the "component id" property because that conveniently counts the number of times a service was published. You can see from its count that the ECF Discovery service can handle a few publications. It needs about 25 weeks to hit a cool million. At the time of this writing it has run about 4 days.

How does it work

It is very easy once you know how. The application itself is a cross between a standalone Java application and an RCP, which is an interesting concept in itself. When the product is started, the Application class is given control. In the start method, the UI is setup and then a listener is installed to listen for service publications. This code is very easy to follow. For every service that comes in, it is checked to be a QuoteService. If this is the case then the UI is updated. The ECF remote services takes care of the remoting and your service is published as if it was a local service.

Please remember to place your code in a try block: You are working with remote services. If the remote server shuts down or decides to stop the service, the communication framework will throw an Exception. You always have to be aware of the fact that OSGi services are dynamic and remote services even more. Code in a defensive way suitable for network programming.

How is the framework started

Please look at the Configuration tab of the product configuration. A few of bundles are started to make sure that the underlying framework is initialized. The connect button of the UI does the manual work by creating a Container which is the ECF concept for an endpoint.

try {
zooContainer.connect(
zooContainer.getConnectNamespace().createInstance(
new String[] { "zoodiscovery.flavor.centralized="
+ getServers().getText() }), null);

Clients most of the time run in centralized mode and point to a server with configuration information and so does our client. We can remove the manual connect by setting the following Java options:

-Dzoodiscovery.autoStart=true
-Dzoodiscovery.flavor=zoodiscovery.flavor.centralized=yazafatutu.com

Conclusion

This is a small example on how to use the ECF Zoodiscovery from a consumer perspective. Various parts of Zookeeper can be configured to achieve an optimum for your specific needs and network. More information can be found on the Eclipse wiki about the configuration of Zoodiscovery.

Many thanks to the developers of OSGi, Equinox, Zookeeper and ECF for enabling us to glue this together to this very useful piece of software.

Need Help?
We can help you with professional services and training to implement this and other parts of Eclipse. Please drop a line to sales@industrial-tsi.com for more information.

Links

• ECF and OSGi 4.2 Remote Services
• ECF ZooDiscovery
• ECF Helios New and Noteworthy
• ECF Downloads

Opinion: The Class of Projects

This blog is about projects that are not backed up by companies. We, Eclipse users, are very lucky that there are still companies and individuals that invest in time and money for projects like Equinox, ECF, Platform, JDT, CDT, etc..

I learned from Arnoud Engelfriet that there are three types of projects: Base, Commodity and Differentiator.

Base:

If we look at end users, nobody gives a damn about the base because it is just not visible. What does a user care if Eclipse runs on Equinox or on a homegrown executable or if it runs on Swing or on SWT? However, it is needless to say how important these projects are.

Commodity:

These are projects that are an expected part of the product but that are visible to the user. In case of a IDE, think of completion, refactoring, project management, team support, etcetera. The user cares about these things but they are a part of every Java IDE and if you don't have it, you are not in the game. However, it is needless to say how important these projects are.

Differentiator:

Now we come to the interesting business projects because this is stuff that no other IDE has. Think projects as Mylyn, Swordfish, Rienna, etcetera. Needless to say how important these projects are.

Differentiators form an interesting business area and you see that these projects are mostly backed up by companies who can and will help their committers and contributors with infrastructure.

If you are a project that is a Base or Commodity project and not backed up by a company then it can be hard to get funding. A recent example is the Eclipse telecommunication infrastructure.

I end up paying $40 every week for a teleconference call with our group because the foundation does not have a local dial-in for my country. Use Skype you say. Sure we will if it is possible but you know that Skype does not always operate as expected when teleconferencing with people around the world and we don't want to waste time with technical issues (thats the first time I ever said that).

The foundation has no money to pay for this infrastructure. I respect that but who will pay for it then?

In other words, it can be very hard for a Base or Commodity project to get proper funding.

And this is also something that the Foundation should be aware of. It is logical that the parents are very proud of the children who are popular but the other children need attention too.

Base and Commodity projects do not get enough attention. Not only in terms of funding, but also in terms of marketing power, love and mentions. When was the last time SWT was placed in the spotlight?

Therefore, I think bug 313479 starts a good discussion.

Re: Patently Ridiculous

After reading Ed's post about software patents I remembered a story that occurred around the time when I fell in love for the first time with the game of Go.

It must have been a good ten years ago, or more, when I frequently played on the Internet Go Server. When you are a beginner you have to work your way up the ladder by winning games. One way of winning is to challenge players of around your ranking.

When I challenged one of the lower rated players, I stumbled upon a guy who was hiding from his fellow Masters in Go by logging in with an alias. However, he forgot to unset the "open" flag so I found myself playing against a much stronger opponent. This is something you just cannot win (there is no such thing as luck in Go.)

While we were playing and chatting, he introduced himself as Jean-loup Gailly. I Yahooed him (no Google back then) and found out that he is one of the authors of gzip and masters several other crafts very attractive to the inner-geek (like his work on pulsars).

While I don't know his stance on software patents, I assume that he is against them. Jean-loup has analyzed several patents to make sure that the gzip software avoids all of them.

Jean-loup on his website [3]:

"I have probably spent more time studying data compression patents than actually implementing data compression algorithms. I maintain a list of several hundred patents on lossless data compression algorithms, and I made sure that gzip isn't covered by any of them. In particular, the --fast option of gzip is not as fast it could, precisely to avoid a patented technique."

He continues by making an interesting remark:

"The first version of the compression algorithm used by gzip appeared in zip 0.9, publicly released on July 11th 1991. So any patent granted after July 11th 1992 cannot threaten gzip because of the prior art, and I have checked all patents granted before this date."

I understand from this that patents are void if somebody can prove that the invention was done before it was filed. So filing patents for simple things might not be so damaging as it appears (somebody try to file a patent for the singleton pattern and see what happens). This is called prior art.

The big crunch

One interesting story of Jean-loup is about a compression technique claiming to be so effective that it could compress any file with at least one bit. Of course, a child can see that this would recursively compress the output file to zero bits. The US patent office worked on this patent for three years before finally granting the patent.

--

[1] While I was researching for this blog post I found out that there is a free iPhone client for Go.

[2] Carl Barks, the famous Donald Duck cartoonist, once drew a story where Donald Duck and his nephews salvaged a ship by pumping thousand of ping-pong balls into it. The word goes that this story prevented granting of a patent (unfortunately this appears to be a hoax).

[3] Jean-loup Gailly's site

Invitation to a sunny party

Hi There!

We would like to invite you to attend the Eclipse DemoCamp in Nieuwegein which will take place at June 23.

We will celebrate the Helios release at the end of June and the e4 release in July.

The Eclipse DemoCamps are an opportunity to showcase all of the cool technology being built by the Eclipse community. This is also an opportunity for you to meet Eclipse enthusiasts in your area.

Thanks

to the Eclipse Foundation for the text and Vahid "Pixels" Vafai for the cool graphics.

ECF Discovery: How many Zookeepers does it take to screw in a lightbulb?

If you use OSGi remote services (See OSGi Compendium Specs chapter 13) you have to know the other end. In large installations this configuration can be quite cumbersome.

At Remain Software we develop software to manage nodes in a network. We want our nodes to register themselves to us when they are in the network. This is fine if you manage a small office but not if you manage smart lightbulbs in a sky scraper.

As soon as the lightbulb is screwed into its socket, it can tap some power to activate its OSGi runtime. The runtime will activate the ILightBulb interface with methods dim(int), on() and off() as a remote service. Now, how do we get this service to interested parties...

ECF For the Win
The ECF discovery framework enabled us to create a Zookeeper based Discovery implementation. An addition to the already existing JmDNS (Zeroconf/Bonjour) and jSLP implementations.

An Apache Zookeeper server will replicate configuration data between other Zookeeper servers. The Zookeeper servers know each other and clients know one Zookeeper Server. So the smart bulb (which runs OSGi or did I mention that already?) is preconfigured with the address of its nearest Zookeeper server or gets this information dynamically by some kind of IP broadcast.



When the lightbulb publishes its Remote Service, ECF wakes up and publishes this service through the provided Discovery implementations. The Zookeeper discovery provider will immediately notify its nearest peers and the new lightbulb service is registered in all Zookeeper instances. When the Zookeeper instance that is connected to an interested party receives the data, the Discovery implementation will publish this service in that OSGi container.

The Lightbulb Control Center is waiting for the ILightBulb service and magically creates a UI in its console (the sparkles DO NOT appear in reality. I made that up).

The Building Maintainer can now control the lightbulb.

More..

For more information or if you want to Tinker with the code, please visit the wiki.

Acknowledgements

Ahmed Aadel, one of the great developers working at Remain Software created the Zookeeper Discovery Implementation. Thanks to Scott Lewis and Markus Kuppe from our ECF team for support and Chris Aniszczyk for easy access to Tinkerbell (tm)

Animations done with the Gimp.