FlimFast - Software for Fluorescence Lifetime Imaging at Video Rate

by Christoph Gohlke and Oliver Holub, Laboratory for Fluorescence Dynamics, December 2002.

Fluorescence lifetimes provide valuable information that is not available from steady-state fluorescence. However, acquisition, processing and display of fluorescence lifetime images require much longer times than simple intensity images due to the complexity of the analysis and the visualization of multi-parameter information. Therefore, fluorescence lifetime imaging (FLI or FLIM) has not yet met the fast continuous interactive operation required by many biological and medical applications. We have developed software (FlimFast) to automatically control and acquire data from fast frequency-domain, full-field image acquisition hardware.

Screenshot of FlimFast showing multiple, consistent views of
fluorescence lifetime image data of a 3T3 cell stained with BODIPY.

FlimFast has been designed and programmed from scratch in order to complement our customary lifetime imaging software, to overcome its limitations, and to unlock the full potential of our FLI instrumentation and modern personal computer hardware. The software enables continuous mode fluorescence lifetime imaging at video-rate with concurrent image analysis, user interaction, and advanced visualization of multi-parameter image information. Rates up to 26 fluorescence lifetime images per second are achieved for 320x240 pixel images. Image acquisition and lifetime analysis are optimized for speed and flexible operation modes such as continuous integration and non-sequential phase sweeps. Common image processing tasks such as binning, convolution filtering, contrast enhancement, and threshold masking are applied in real time. If available, dual CPUs and a 3D graphics processor are made use of for parallelizing and distributing processing tasks and to increase responsiveness of user interactions. Real-time visualization of multi-parameter image information in form of multi-textured shaded surface renderings, color-coded images using anti-aliasing and multi-dimensional palettes, and combined 3D scatter plot / 2D histograms have been implemented to allow the operator to focus on the generally most relevant aspects of the vast amount of image information that is generated at video rate. Operating the instrument has been automated to the possible extent and the user interface was streamlined to further hide much of the complexity of fluorescence lifetime imaging. A new file format, FLIF, has been defined for archival, streaming, and post-measurement analysis, containing all relevant parameters and acquired image information in a single file. While FlimFast has been developed with experimental applications in mind (e.g. fast sample screening), many of the concepts that are implemented are necessary in order to successfully introduce lifetime imaging to a medical environment (e.g. medical diagnostics using an endoscope). The software has been developed in C++ using the Microsoft Foundation Classes and OpenGL (SGI). Interface libraries for PCI cards are loaded dynamically at runtime, such that FlimFast is capable of running on computers without installed device interface drivers.

Multi-textured shaded
surface rendering

Anti-aliasing using
palette interpolation

Multi-dimensional
color coding

RGB channel coding

3D scater plot
2D histograms

Features

Data Acquisition

Frequency domain (single frequency), full field, homodyne, triggered image acquisition
Region of interest
Interactively adjustable number of images per phase and zero phase
Single, up-down, optimized, or continuous phase sweeps
Automated hardware control of frequency synthesizers, light shutter, digital phase shifter, and CCD camera

Real-Time Data Processing

Integration and binning of images
Per pixel calculation of phase shift and demodulation, or apparent single lifetimes
Convolution filters
Region of interest
Multi-channel threshold masking
User defined or smart auto scale
Histogram equalization
Image statistics

Input/Output

Native FLIF input/output
FLImage, SimFCS, FLIE, and FLIez input (INT, PHS, MOD, OUT, I16)
F32 output of images and 3D histogram
BMP and TXT output of images
AVI output of frame buffer and main window
Clipboard output of frame buffer and images

Data Visualization

Multi-parameter image visualization
Consistent multiple views/windows
Global image mask
Double buffered display
Color coded image bitmaps
Color coded 1D image histograms
Color coded 2D image histograms
Histogram of polar plot (phase, modulation, "Phasor")
Plot of pixel raw signal and fit
Image bar chart
3D scatter plot
The Matrix view
Teapot rendering

Multi-textured shaded surface rendering

Real time (up to 200 fps)
Interactive rotation and scaling
Image surface plot (height field)
Bi-linear image scaling
Antialiased color coding using
palette interpolation and mip-mapping
Multi-parameter color coding
(2D and 3D palettes)
Iso-contours or iso-lines
Display of image metrics
Texture mapping
Per pixel or vertex lighting
Bump mapping
Environment mapping
N patch surface rendering

Speed

Video-rate acquisition, processing and visualization
Intelligent calibration of instrumentation
Optimized and parallelized processing pipeline
Running Digital Fourier Transform
Caching of intermediate data
Hardware accelerated 3D rendering via OpenGL
Multi-threading for efficient use of dual and hyper-threading CPUs

Ease of Use

Standard, unobtrusive user interface: MDI, toolbar, console output, full screen mode
No installation required
Driver DLLs loaded on demand at runtime
Automatic instrument initialization
One click calibration
Non modal status and error messages
Double-click or drag&drop file open
Meaningful initial view of data
Consistent view of data
Virtual trackball

System Requirements

Microsoft Windows XP
550 MHz Pentium III CPU
384 MB RAM
OpenGL 1.3 capable GPU
1024x768 true color display
Recommended dual or Hyperthreaded CPU and GeForce 3 GPU

Download

FlimFast.zip (6.5 MB). Requires Microsoft Windows XP with OpenGL 1.4 ICD. The WebCam plugin requires OpenCV.

Note to ATI/AMD Radeon graphics card owners: disable "Compiled Vertex Arrays" in the "Program Settings" dialog.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Further Information

PhD. Thesis of Oliver Holub, Technische Universität Berlin, 2003. Fluorescence lifetime imaging at video-rate - a new technique in photosynthesis research.

Poster (PDF, 2.1 MB) presented at the 45th Annual Meeting of the Biophysical Society, Boston, Massachusetts, February 17-21, 2001. FlimFast - fluorescence lifetime image acquisition and visualization at video-rate. Christoph Gohlke, Oliver Holub, and Robert M. Clegg.

7 Minutes Presentation (PDF, 3.25 MB) given at the Advisory Committee of the Laboratory for Fluorescence Dynamics, 2002. FlimFast - software for fluorescence lifetime imaging at video-rate.

Movie (QuickTime MP4 format, 8.2 MB) or movies 1, 2, 3, and 4 (WMV8 format, 7-18 MB) demonstrating continuous fluorescence lifetime image measurements with concurrent analysis and visualization at 3-26 Hz.

Renderings by FlimFast appear on the book cover of Molecular Fluorescence and the flyer of the Understanding Complex Systems 2002 symposium.