OFEP3: Configuration of the OpenFlexure Microscope server
| OFEP | 3 |
|---|---|
| Author | Richard Bowman |
| Created | 16-Nov-2021 |
| Status | Approved, not implemented |
| Approved date | 08-May-2024 |
| Requires implementation | Yes |
| Implemented date | N/A |
| Updated dates (post-approval) | N/A |
1. Introduction
This OFEP clarifies the thinking behind the currently-proposed changes to how the microscope server is configured. It's important to explain and discuss this, because it has implications for how it will be used in other projects, and for how we manage the medical device version of the project. This OFEP considers three aspects of configuration in particular:
- Setting up the stage
- Setting up the camera
- Enabling/disabling extensions
2. Current state
Currently, there are very few options for the microscope hardware - the camera can either be managed by a PiCameraStreamer class, or a MissingCamera class if the former fails to load. Similarly, only a SangaStage or, if that fails, a MissingStage will be used for the microscope stage (though two versions of the SangaStage are provided to cope with delta geometry). In both cases, the server will attempt to load the hardware, then fall back to the emulated device if it is not present. This can be helpful when working on the microscope, because the server will start even if hardware is missing. However, in some circumstances it would be better to fail with a helpful error than to load the server with missing hardware components.
There is currently no way to add a class to handle a new camera or stage without modifying the server code; this means that, for example, the UC2-Hi2 system needs to use a fork of the microscope server. In principle, any subclass of BaseStage or BaseCamera could be substituted in the initialisation code, but at the moment that requires adding another hard-coded reference and import to openflexure_microscope.microscope.
Extensions are currently loaded from a folder - any extension in the folder will be loaded. If extensions fail to load, the server still starts and it is not immediately obvious to a client that anything is wrong. There is no notion of "expected" extensions: if an extension is not found or fails to load, the server does not currently consider anything to have gone wrong. The only indication of this is that the extension in question is missing from the list of extensions. Installing extensions can also be complicated, especially if they have dependencies, though this is a subject for another OFEP currently under consideration.
The microscope's configuration is currently split into several files:
* microscope_settings.json contains settings that may change during the server's execution.
* microscope_configuration.json contains configuration information that usually does not change, although it is managed in the same way as the above (i.e. is dynamically written to when settings change).
3. Proposed changes
We propose to move away from a predefined set of hardware classes, and instead to specify entry points in the microscope_configuration.json file. This has the significant benefit that the hardware control class need not be part of the microscope server package, thus it becomes possible to implement and test wrapper code for different cameras and stages, without needing to modify the OpenFlexure server. The entry points mechanism allows us to easily enumerate all the compatible camera or stage classes that are available in the current Python environment, which will enable helpful error messages and automatic or semi-automatic configuration in the future. Making the server more flexible in this way allows it to become a useful component in custom microscopes beyond just the OpenFlexure microscope.
We propose not to use a fallback or autoconfiguration method when loading the microscope configuration, in order to ensure it is always running with all hardware connected. Builds on the CI server should use a configuration file that explicitly selects the MissingCamera in order to avoid errors in the console. There should be the option for a fallback server that gives the option to display a graceful error message via HTTP, instead of simply failing to start. That would be appropriate when the microscope is run as a service. Autoconfiguration on first run is desirable, but is probably best implemented in a separate web service providing a management interface. That way, we separate out the task of determining what hardware/software is available from the task of starting up the server in a given state. This would allow for a medical device to run only in the correct configuration or not at all, while a development build gains a friendly interface to pick the appropriate components in the event that a configuration file doesn't specify a working configuration.
Similarly, assuming the extensions OFEP is accepted we will use entry points to identify extensions. We propose to store, in microscope_configuration.json, a list of extensions to load. On startup, the server should attempt to load all of these extensions. If any extension fails to load, the server will not start (or will only serve an error page) and will raise a prominent error. Errors can be avoided by disabling extensions that are unable to load. Specifying a list of extensions makes it explicit which are required, allows us to flag missing extensions, and provides a mechanism whereby extensions that are installed but not required may be disabled. Allowing some extensions to fail complicates the code and introduces potentially confusing failure modes, so we propose not to make provision for this. v3 of the software refers to Things rather than Extensions: exactly the same logic would apply.
The microscope_configuration.json file could be made read-only in a medical device, thus preventing the server from being changed by an untrained user such that it loads in unverified external code.
In the future, we anticipate that extensions could read some configuration information from the configuration file, rather than just the settings file. This allows static configuration of the extensions to be distinct from the user-controlled settings, again a useful feature for medical devices or other products where aspects of the configuration should be fixed.
4. Alternatives and rejected options
Explicitly defining fallback options, e.g. to allow one server configuration to work on several different versions of hardware, was considered at various points. Similarly the idea of "nice-to-have but optional" Extensions was mooted. However, both of these make it harder to see when code has failed, and create potentially confusing failure states. As stated above, a management interface, capable of modifying the configuration file and reloading the microscope server, would be a better solution - though obviously it is additional effort.