Getting developers working in this way requires a culture change. While the shift is ongoing, many have embraced the idea. We chose not to remove the core bundles or refactor them directly in one go. Rather than having n-sided developer battles, we decided to move to the right design in new work. Refactoring can in this case be done later, once ideas spread organically, by training and sticking to good practice in new bundles. Our problem with core bundles does not have to be solved right away, but we are chipping away at it using the no-dependency bundles.
Real world problem #4: The static, non-modular algorithm
Today in our server there around a hundred OSGI declarative services for things like loading files, getting interfaces to hardware, writing data to a fast distributed file system (we use GPFS and Luster), talking to FPGA-based devices by description language, sending text messages to a port on a custom Linux device that controls a detector, and much more besides! In fact, depending on the experiment, the various bridging bundles and device libraries can easily outnumber the scanning algorithm itself.
The scan algorithm is the heart of the data acquisition system. It is one of the parts that brings together separate concepts like devices and file writing and runs them together in order to collect useful data for the user. On the face of it, there wasn’t much wrong with our existing scanning system. Having been honed by several generations of developers (using the standard developer lifetime of seven years), it was pretty fast, robust, and had a useful Jython layer with which to extend it.
The scan did have a problem, though, in that it could not deal with a new file-writing design, which was introduced in a separate project and had to be integrated to our software. It was intended to store data statically and written in a non-modular way, which made it expensive to adapt. We decided to solve the file-writing requirement and at the same time migrate scanning to OSGi. So scanning was one of the few parts of the system, and the most important part, that we did choose to rewrite. The final algorithm spanned a few thousand lines of code and its bundle is shown expanded below.
Figure 2. The OSGi bundle for the scan algorithm. Image credit: Matthew Gerring.
The main algorithm of the scan is an iteration over n-dimensional motor positions running objects that manage fork/join pools. I’ve printed part of it here, and it’s also available under an open source license on GitHub.
Listing 1. The main algorithm for Diamond Light's scanning service
Sign up for CIO Asia eNewsletters.