Microscope image stabilization

When taking automated captures, especially at high magnification, any vibration can ruin your result.

In this article I describe the method used with my motorized microscope.

For almost a year I have been using this method for all my panoramas, and it has become essential.

Physical vibration dampening

First of all, there is no point in implementing the method described here if you have not added any physical vibration dampening on your microscope.

The idea is very simple.

  1. You need enough weight on your microscope.
    For this you can use some granite surface plate, optical breadboard, or anything heavy enough and stable enough.

  2. You need some damping material under the plate / microscope.
    Something soft enough to absorb vibrations.
    Sorbothane seems to be ideal but is expensive so I just use foam (make sure it’s soft enough). Dampening feet are also a possible solution.

Software vibration detection

The idea is very simple. Compare consecutive frames to detect vibrations.

Here is my implementation. It should works with any magnification and for any sample. The method is very robust in theory and practice.

void CameraStabilization::update()
{
    static_assert (BufCount >= 2, "");
    
    if(!_ctx->refresh_request) {
        return;
    }
    
    // Ring buffer
    // A B C D => {} A B C D => D A B C {} => D A B C
    // discard last buffer but keep the other ones in the right order
    {
        ImageBuf empty_buffer;
        // be careful of insert initializer_list ...
        _buffers.insert(_buffers.begin(), std::move(empty_buffer));
        std::swap(_buffers.front(), _buffers.back());
        _buffers.resize(BufCount);
    }
    
    // read new frame
    ImageBuf& buf = _buffers[0];
    {
        std::lock_guard<std::mutex> lock(_ctx->mutex_callback);
        if(buf.size() < _ctx->im_data.size()) {
            buf = _ctx->im_data;
        } else {
            std::copy(begin(_ctx->im_data), end(_ctx->im_data), begin(buf));
        }
        _image_size = _ctx->image_size;
    }
    
    // resolution changed
    if(_buffers[0].size() != _buffers[1].size() ||
       _buffers[0].empty())
    {
        _stabilized_frames = 0;
        _distance = MAX_DIST;
        _distance_prev = MAX_DIST;
        return;
    }
    
    // compute distance
    unsigned d2 = 0;
    
    // fast: only process the green channel
    {
        const auto& buf0 = _buffers[0];
        const auto& buf1 = _buffers[1];
        
        for(unsigned i=1; i<buf0.size(); i+=4) {
            int d = int(buf0[i]) - int(buf1[i]);
            d2 += d*d;
        }
    }
    
    
    _distance_prev = _distance;
    _distance = std::sqrt(d2);
    _distance /= 255;
    
    const float delta = std::abs(_distance - _distance_prev);
    
    if(delta < 0.2f) {
        ++_stabilized_frames;
    } else {
        _stabilized_frames = 0;
    }
}

The interesting part is after ‘compute distance’. This is an Euclidean distance between two images, using only their green channel.

Using the distance directly is not enough, as sharpness of the image will affect the value obtained.

This is why you have to compute the delta value.

Two consecutive stabilized images are enough, and that’s what I use.

Go further: noise reduction

One interesting side effect of this method is that you can use the last stabilized frames to reduce noise.

The idea is very simple. Just average all stabilized frames into one image.

If you only have 2 stabilized frames, you will reduce your noise by 50%.

It’s not huge, but quite visible on the red/blue channels. And it’s free, so why not take advantage of it?

Note: this is why I’m using a ring buffer in my implementation, so I can keep the previous images to average them later.

Tradeoffs

You can expect a slightly longer capture time if your motors are a bit noisy.

Motors vibrations can be reduced a lot with TMC2208 (or equivalent) drivers and microstepping.

You can also use 0.9deg/step stepper motors instead of 1.8deg/step. There is a significant difference between the two, and this is very important to use a 0.9deg/step motor for the Z axis.

When using my 100x oil objective, movements are slow and therefore capture times are the same with and without software stabilization.

Hardware vibration detection

I’ve done a few tests but overall it’s not conclusive.

Vibration detection is generally not sensitive enough, and the amount of work required to obtain something usable is too high to make it worthwhile.

If you are still interested, here is an interesting project:


Designing a simple vibration sensor.