Tuesday, February 9, 2010

OSGi Clipboard Monitor for Java (on Windows using JNA)

Before you read on you should be familiar with JNA and OSGi as I will make heavy use of it in this posting.

Preface

For my CopyTo Eclipse plugin's preference page I wanted to enable the "Paste" button for label/URL combinations only if there is some text in the clipboard (and that text can be converted to a "CopyTo" target). So I would need to monitor the content of the systems clipboard.

SWT to the rescue! ... not!

So I checked if SWT can already help me here. There should be a Clipboard.addListener method that would allow me to register a listener with the clipboard and let SWT notify me about changed clipboard content. Unfortunately there is no such functionality in SWT. And I would soon know why that is ;)

The long way... using a Windows Clipboard "Viewer"

Since I am developing on a Windows machine, I fired up the Windows SDK help to see what it offers in regard to the clipboard. There is a SetClipboardViewer function that in a somewhat awkward way let your window become part of the clipboard viewer chain. Windows will then send 2 specific messages to this windows message proc to inform it about clipboard changes (WM_DRAWCLIPBOARD) or changes in the clipboard viewer chain (WM_CHANGECBCHAIN , if someone else calls SetClipboardViewer).

The first thing that bugged me was, that I do not have a window that I could give to the function and that could receive messages from Windows. Then, I remembered my old Win32 coding days in C++ and that one can create a hidden window just for messages. Instead of creating my own window class for that I decided to use an existing one. The "STATIC" window class does not need a lot of resources and does not receive a lot of messages so its the perfect candidate. We will just create an invisible instance of such window using JNA:

viewer = User32.INSTANCE.CreateWindowEx(0, "STATIC", "", 0, 0, 0, 0, 0, null, 0, 0, null);

Then we register the window with the clipboard viewer chain according to the API specs:

nextViewer = User32.INSTANCE.SetClipboardViewer(viewer);

The next thing to do is to redirect all messages to a custom window proc instead of the default one of the "STATIC" window class. This is called "subclassing" in Windows and in a sense is something like class subclassing in the Java world. Only on a "message only" basis.

Our new window proc will handle the two clipboard related messages and redirect the other messages to the original window proc of the "STATIC" window class.

User32.INSTANCE.SetWindowLong(viewer, User32.GWL_WNDPROC, this.ourProc);

Please note that ourProc is a member field, so that the Java GC will not remove its reference and JNA would no longer be able to send messages to our callback!

Thats how the callback looks like:

class OurProc implements WNDPROC {
public int callback(HWND hWnd, int uMsg, WPARAM wParam, LPARAM lParam) {
switch (uMsg) {
case User32.WM_CHANGECBCHAIN:
  // If the next window is closing, repair the chain.
  if (nextViewer.toNative().equals(wParam.toNative())) {
    nextViewer = new HWND(Pointer.createConstant(lParam.longValue()));
  } // Otherwise, pass the message to the next link.
  else if (nextViewer != null) {
    User32.INSTANCE.SendMessage(nextViewer, uMsg, wParam, lParam);
  }
  return 0;
case User32.WM_DRAWCLIPBOARD:
  try {
    onChange(new ClipboardEvent(this));
  } finally {
    User32.INSTANCE.SendMessage(nextViewer, uMsg, wParam, lParam);
  }
  return 0;
case User32.WM_DESTROY:
  User32.INSTANCE.ChangeClipboardChain(viewer, nextViewer);
  break;
}
return User32.INSTANCE.DefWindowProc(hWnd, uMsg, wParam, lParam);
}

That's pretty much the code from the Windows SDK help for programming a Clipboard Viewer. There is some strange chain repairing code included. Speaking of bad API design. How easily could that break, if one programmer gets it wrong?

Polling the message queue without wasting CPU cycles

Now, for the sub-classed static window to receive any messages we need to poll the message queue. In Windows you can only poll your own threads message queue. And, important to know, Windows only creates a message queue if a thread creates a window or calls one of the message queue related functions like GetMessage or PeekMessage. So a simple message poller in our threads run() method would look like this:

while (User32.INSTANCE.GetMessage(msg, null, 0, 0)>0) {
  User32.INSTANCE.TranslateMessage(msg);
  User32.INSTANCE.DispatchMessage(msg);
}

However, GetMessage is a blocking call and so our Thread would consume all the available CPU cycles it could get until a new message arrives in the queue. That's certainly not what we want. Instead we have to create more sophisticated message queue polling logic. We also want to be able to end our clipboard monitor at any given time. We do that by signaling a special Win32 kernel event object. The MsgWaitForMultipleObjects allows us to wait for message to arrive in the queue as well as events that get signaled. So our new message polling loop that cost (almost) no CPU cycles looks like this:

final HANDLE handles[] = { event };
while (true) {
  int result = User32.INSTANCE.MsgWaitForMultipleObjects(handles.length, handles, false, Kernel32.INFINITE, User32.QS_ALLINPUT);

  if (result == Kernel32.WAIT_OBJECT_0) {
    User32.INSTANCE.DestroyWindow(viewer);
    return;
  }
  if (result != Kernel32.WAIT_OBJECT_0 + handles.length) {
    // Serious problem, end the thread's run() method!
    break;
  }

  while (User32.INSTANCE.PeekMessage(msg, null, 0, 0, User32.PM_REMOVE)) {
    User32.INSTANCE.TranslateMessage(msg);
    User32.INSTANCE.DispatchMessage(msg);
  }
}

Please note that we poll all messages out of the queue until there are no more left and only then return to another round of MsgWaitForMultipleObjects until our halt event is signaled.

That's it about the gory details of implementing such a "viewer" in Windows (without displaying anything). The hard thing to figure out was, that all user level related calls had to be made from the thread that will read out the message queue.

Why Microsoft decided to put the responsibility of a chain into the hands of the programmer is beyond me. Clearly bad API design.

Adding some spin - OSGi services

Of course when programming in Java I will always use OSGi whenever possible. The Clipboard Monitor is no exception. Granted, the whole code can be used without OSGi too. But then you would have to implement a kind of listener management yourself. That's up to the reader of this entry and I welcome everyone to contribute a plain old Java implementation of AbstractWindowsClipboardMonitor that integrates hand crafted listener management instead of using the OSGi service registry.

So there is a monitor component that will consume ClipboardListener services registered in the system and inform them about changes in the clipboard.

Listener that publishes an event using the EventAdmin

Now we can have a fairly simple listener that publishes an event on every clipboard change:

public class EventAdminClipboardListener implements ClipboardListener {
  private static final String TOPIC = "clipboard/monitor/event"; //$NON-NLS-1$

  private final AtomicReference ref = new AtomicReference();

  protected void bind(EventAdmin eventAdmin) {
    ref.set(eventAdmin);
  }

  protected void unbind(EventAdmin eventAdmin) {
    ref.compareAndSet(eventAdmin, null);
  }

  public void onEvent(ClipboardEvent event) {
    EventAdmin eventAdmin = ref.get();
    if (eventAdmin != null) {
      eventAdmin.postEvent(new Event(TOPIC, (Map) null));
    }
  }
}

Pretty simple.

The SWT Clipboard Listener service

Then we have this listener service that will examine the content of the clipboard and use the OSGi EventAdmin to publish specialized events with topics describing the content of the clipboard. EventHandler services can so easily react on specific changes in the clipboard. The following event topics are currently implemented:

  • clipboard/monitor/swt/TEXT
  • clipboard/monitor/swt/URL
  • clipboard/monitor/swt/IMAGE
  • clipboard/monitor/swt/RTF
  • clipboard/monitor/swt/HTML
  • clipboard/monitor/swt/FILE

So you could register an EventHandler that reacts on changes in the clipboard, and only if the new clipboard content contains (also) text. Your EventHandler service would register with the topic set to "clipboard/monitor/swt/TEXT". The beauty of this componentized approach is that you do not even have to know that there is a clipboard monitor installed and running in the system. You have no dependencies on it. You simply register for events of a specific topic and get informed about changes. You can then use SWT to read out the clipboard.

Conclusion

I hope this rather lengthy posting gave you another clue what can be done using clean OSGi component design. And maybe some of you can even put the components to some use in your own projects. It's all licensed under EPL 1.0 so feel free to use the code.

Here you can watch a little video demonstration (if you cannot see the video below).

Code at GitHub

You are invited to check out the full source code over at GitHub and you are invited to fork/clone and contribute, if you want. Maybe someone can add a GTK/Linux clipboard viewer?

13 comments:

  1. I think the event dispatching is a reasonable idea, but it probably needs more work than what is covered at present. For example, data can be pasted onto the clipboard in multiple formats simultaneously (and/or converters can be used) so having a 'publish' event which is tied to the format is a bad idea. You ideally want to be able to publish the 'clipboard has changed' event and then let the filtering be done by properties of the event, not the message. There is support for the OSGi event admin to filter the events prior to dispatch in any case.

    Furthermore, using capitalised variants (like TXT) are pretty limiting in the face of MIME types (text/plain, text/html). Indeed, Apple has recently changed their type hierarchy to handle nested MIME types, called UTIs:

    http://www.roughlydrafted.com/2009/09/22/inside-snow-leopards-uti-apple-fixes-the-creator-code/

    Lastly, I don't think 'swt' is needed at all in this mention. One can write (headless) applications which still may yet use the clipboard.

    ReplyDelete
  2. Oh my God! You don't need to reinvent the wheel and can use ISelectionService instead http://help.eclipse.org/ganymede/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/api/org/eclipse/ui/ISelectionService.html

    ReplyDelete
  3. Thanks Alex, for your comment. The sample swt bundle was not well thought out. It was merely a demonstration of one possible use case. I thought filtering on the topic is a good idea. The bundle posts multiple events for each detected clipboard format. You are right, that it should better just send one event and set a String[] property for the formats and the EventListener could then filter with a target filter.
    Unfortunately "swt" is needed to inspect the Clipboard and its also platform dependent. In the sample swt bundle I wanted to create event that do not necessarily imitate MIME types, but a more general approach of just describing roughly whats in the clipboard after the change. After all, the Windows clipboard does not understand the concept of MIME types, neither does the SWT Clipboard widget. And since most application save uncompressed DIB data for images in the Windows clipboard (instead of compressed JPEG or PNG data) I wanted to just send and "IMAGE" event.
    What MIME-Types do you suggest for the various (SWT supported) clipboard formats?


    @Eugene: Thank you for your comment. However, the ISelectionService has nothing to do what is presented here. It is in no way related to the (SWT) clipboard. If at all, it gives you only a possible *source* for a clipboard operation. It does not help you to enable UI elements based on whats in the system clipboard.

    ReplyDelete
  4. First of all you don't really have to use system clipboard, but selection of the workbench part that has the focus. Then you can just attach the listener that will update the UI according to selection changes.

    ReplyDelete
  5. Eugene, thanks for your comment.

    I understand the use of the selection service.
    This post is about a system clipboard monitor. There are no dependencies to the Eclipse Workbench here.

    Let's imagine a simple image editing software based on Eclipse RCP. You have a toolbar button/menu entry that should only be enabled if there is image data in the system clipboard. Otherwise it would be enabled always and only when you try to paste the clipboard content in your app you will realize that there is actually only text data in the clipboard. A clipboard aware UI is just a usability helper.
    And totally independent on Eclipse. The only dependencies introduced here are the (optionally) ones to SWT. They are used to show a more fine grained notification mechanism that informs listeners of clipboard changes about the new content of the clipboard.
    But then, one could easily provide a non-SWT implementation using JNA to read out the clipboard data and replace the SWT implementation. Or use AWT to read out the clipboard.
    Once again, it's not about the selection service. The use case covered with this monitor are totally non-related to it.

    ReplyDelete
  6. Philipp, I wasn't arguing about the clipboard tracking. But we have different take on a particular application. You are tracking the clipboard and I am saying that issue can be tackled without clipboard tracking. I believe with my solution user would have to do less, i.e. don't need to copy into clipboard before he'll be able to click that button on the toolbar, so button reacts on selection instead of content of the clipboard. That is all

    ReplyDelete
  7. I understand that you can use the selection service or whatever Eclipse provides to track the current selection. The clipboard has nothing to do with the selection.

    If I have an Eclipse based application that allows me to create new editor parts for images, and I copy an image into the clipboard (i.e from Firefox) then in my Eclipse application the "paste" button will be enabled and allows me to create a new editor part with the content of the clipboard. Such use case would not be possible with the the selection service ;)

    ReplyDelete
  8. Philipp, thanks for you blog. I have question, how to stop clipboard listener. I have simple app that listnen to clipboard and paste something on clipboard. After paste it captures two times what has been pasted?

    ReplyDelete
  9. Veljko, thanks for trying it out. What kind of information is copied to the clipboard? I would like to reproduce this here.

    ReplyDelete
  10. Phil, I created thread based on your blog to listnen clipboard change. The sam application is putting data on clipboard clipboard.setContents(data, dataTypes) and listener thread received two clipboard changes events. If you want to check my prototipe please see http://sourceforge.net/projects/clipcomrade/

    ReplyDelete
  11. I can't get it to work. The loop of "while (User32.INSTANCE.PeekMessage(msg, null, 0, 0, User32.PM_REMOVE))" only enters at first run of the while(true) loop. The callback() is never called. I have no idea where I'm failing, I copied the whole code, got all the libraries. Even the WindowsClipboardTests does nothing. I want only to get the clipboard items, without any complications...

    ReplyDelete
  12. Got it. SetWindowLong was returning 0. Added at User32, "long SetWindowLongPtr(HWND hWnd, int nIndex, WNDPROC proc);", and now it works. I'm at Win7 x64. Now I'm trying to figure out how to use this :p

    ReplyDelete
  13. Great implementation solution! I was looking for exactly this, as the Java access to System Clipboard on my Windows box is broken. Thanks, Aaron

    ReplyDelete