FlimFast - Software for Fluorescence Lifetime Imaging at Video Rate

by Christoph Gohlke and Oliver Holub, Laboratory for Fluorescence Dynamics, December 2002.

Fluorescence lifetimes provide valuable information that is not available from steady-state fluorescence. However, acquisition, processing and display of fluorescence lifetime images require much longer times than simple intensity images due to the complexity of the analysis and the visualization of multi-parameter information. Therefore, fluorescence lifetime imaging (FLI or FLIM) has not yet met the fast continuous interactive operation required by many biological and medical applications. We have developed software (FlimFast) to automatically control and acquire data from fast frequency-domain, full-field image acquisition hardware.

Screenshot of FlimFast

Screenshot of FlimFast showing multiple, consistent views of
fluorescence lifetime image data of a 3T3 cell stained with BODIPY.

FlimFast has been designed and programmed from scratch in order to complement our customary lifetime imaging software, to overcome its limitations, and to unlock the full potential of our FLI instrumentation and modern personal computer hardware. The software enables continuous mode fluorescence lifetime imaging at video-rate with concurrent image analysis, user interaction, and advanced visualization of multi-parameter image information. Rates up to 26 fluorescence lifetime images per second are achieved for 320x240 pixel images. Image acquisition and lifetime analysis are optimized for speed and flexible operation modes such as continuous integration and non-sequential phase sweeps. Common image processing tasks such as binning, convolution filtering, contrast enhancement, and threshold masking are applied in real time. If available, dual CPUs and a 3D graphics processor are made use of for parallelizing and distributing processing tasks and to increase responsiveness of user interactions. Real-time visualization of multi-parameter image information in form of multi-textured shaded surface renderings, color-coded images using anti-aliasing and multi-dimensional palettes, and combined 3D scatter plot / 2D histograms have been implemented to allow the operator to focus on the generally most relevant aspects of the vast amount of image information that is generated at video rate. Operating the instrument has been automated to the possible extent and the user interface was streamlined to further hide much of the complexity of fluorescence lifetime imaging. A new file format, FLIF, has been defined for archival, streaming, and post-measurement analysis, containing all relevant parameters and acquired image information in a single file. While FlimFast has been developed with experimental applications in mind (e.g. fast sample screening), many of the concepts that are implemented are necessary in order to successfully introduce lifetime imaging to a medical environment (e.g. medical diagnostics using an endoscope). The software has been developed in C++ using the Microsoft Foundation Classes and OpenGL (SGI). Interface libraries for PCI cards are loaded dynamically at runtime, such that FlimFast is capable of running on computers without installed device interface drivers.

Multi-textured shaded surface
Multi-textured shaded
surface rendering

Anti-aliasing using
Anti-aliasing using
palette interpolation

Multi-dimensional
Multi-dimensional
color coding

RGB channel coding
RGB channel coding 

3D scater plot
3D scater plot
2D histograms


Features

Data Acquisition

  • Frequency domain (single frequency), full field, homodyne, triggered image acquisition
  • Region of interest
  • Interactively adjustable number of images per phase and zero phase
  • Single, up-down, optimized, or continuous phase sweeps
  • Automated hardware control of frequency synthesizers, light shutter, digital phase shifter, and CCD camera

Real-Time Data Processing

  • Integration and binning of images
  • Per pixel calculation of phase shift and demodulation, or apparent single lifetimes
  • Convolution filters
  • Region of interest
  • Multi-channel threshold masking
  • User defined or smart auto scale
  • Histogram equalization
  • Image statistics

Input/Output

  • Native FLIF input/output
  • FLImage, SimFCS, FLIE, and FLIez input (INT, PHS, MOD, OUT, I16)
  • F32 output of images and 3D histogram
  • BMP and TXT output of images
  • AVI output of frame buffer and main window
  • Clipboard output of  frame buffer and images

Data Visualization

  • Multi-parameter image visualization
  • Consistent multiple views/windows
  • Global image mask
  • Double buffered display
  • Color coded image bitmaps
  • Color coded 1D image histograms
  • Color coded 2D image histograms
  • Histogram of polar plot (phase, modulation, "Phasor")
  • Plot of pixel raw signal and fit
  • Image bar chart
  • 3D scatter plot
  • The Matrix view
  • Teapot rendering

Multi-textured shaded surface rendering

  • Real time (up to 200 fps)
  • Interactive rotation and scaling
  • Image surface plot (height field)
  • Bi-linear image scaling
  • Antialiased color coding using
    palette interpolation and mip-mapping
  • Multi-parameter color coding
    (2D and 3D palettes)
  • Iso-contours or iso-lines
  • Display of image metrics
  • Texture mapping
  • Per pixel or vertex lighting
  • Bump mapping
  • Environment mapping
  • N patch surface rendering

Speed

  • Video-rate acquisition, processing and visualization
  • Intelligent calibration of instrumentation
  • Optimized and parallelized processing pipeline
  • Running Digital Fourier Transform
  • Caching of intermediate data
  • Hardware accelerated 3D rendering via OpenGL
  • Multi-threading for efficient use of dual and hyper-threading CPUs

Ease of Use

  • Standard, unobtrusive user interface: MDI, toolbar, console output, full screen mode
  • No installation required
  • Driver DLLs loaded on demand at runtime
  • Automatic instrument initialization
  • One click calibration
  • Non modal status and error messages
  • Double-click or drag&drop file open
  • Meaningful initial view of data
  • Consistent view of data
  • Virtual trackball

System Requirements

  • Microsoft Windows XP
  • 550 MHz Pentium III CPU
  • 384 MB RAM
  • OpenGL 1.3 capable GPU
  • 1024x768 true color display
  • Recommended dual or Hyperthreaded CPU and GeForce 3 GPU

Download

FlimFast.zip (6.5 MB). Requires Microsoft Windows XP with OpenGL 1.4 ICD. The WebCam plugin requires OpenCV.

Note to ATI/AMD Radeon graphics card owners: disable "Compiled Vertex Arrays" in the "Program Settings" dialog.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


Further Information

PhD. Thesis of Oliver Holub, Technische Universität Berlin, 2003. Fluorescence lifetime imaging at video-rate - a new technique in photosynthesis research.

Poster (PDF, 2.1 MB) presented at the 45th Annual Meeting of the Biophysical Society, Boston, Massachusetts, February 17-21, 2001. FlimFast - fluorescence lifetime image acquisition and visualization at video-rate. Christoph Gohlke, Oliver Holub, and Robert M. Clegg.

7 Minutes Presentation (PDF, 3.25 MB) given at the Advisory Committee of the Laboratory for Fluorescence Dynamics, 2002. FlimFast - software for fluorescence lifetime imaging at video-rate.

Movie (QuickTime MP4 format, 8.2 MB) or movies 1, 2, 3, and 4 (WMV8 format, 7-18 MB) demonstrating continuous fluorescence lifetime image measurements with concurrent analysis and visualization at 3-26 Hz.

Renderings by FlimFast appear on the book cover of Molecular Fluorescence and the flyer of the Understanding Complex Systems 2002 symposium.


Copyright © 2004-2017, Christoph Gohlke, Laboratory for Fluorescence Dynamics