9 results
Search Results
Now showing 1 - 9 of 9
- ItemcellSTORM - Cost-effective Super-Resolution on a Cellphone using dSTORM(San Francisco : Public Library of Science, 2019) Diederich, Benedict; Then, Patrick; Jügler, Alexander; Förster, Ronny; Heintzmann, RainerHigh optical resolution in microscopy usually goes along with costly hardware components, such as lenses, mechanical setups and cameras. Several studies proved that Single Molecular Localization Microscopy can be made affordable, relying on off-the-shelf optical components and industry grade CMOS cameras. Recent technological advantages have yielded consumer-grade camera devices with surprisingly good performance. The camera sensors of smartphones have benefited of this development. Combined with computing power smartphones provide a fantastic opportunity for “imaging on a budget”. Here we show that a consumer cellphone is capable of optical super-resolution imaging by (direct) Stochastic Optical Reconstruction Microscopy (dSTORM), achieving optical resolution better than 80 nm. In addition to the use of standard reconstruction algorithms, we used a trained image-to-image generative adversarial network (GAN) to reconstruct video sequences under conditions where traditional algorithms provide sub-optimal localization performance directly on the smartphone. We believe that “cellSTORM” paves the way to make super-resolution microscopy not only affordable but available due to the ubiquity of cellphone cameras.High optical resolution in microscopy usually goes along with costly hardware components, such as lenses, mechanical setups and cameras. Several studies proved that Single Molecular Localization Microscopy can be made affordable, relying on off-the-shelf optical components and industry grade CMOS cameras. Recent technological advantages have yielded consumer-grade camera devices with surprisingly good performance. The camera sensors of smartphones have benefited of this development. Combined with computing power smartphones provide a fantastic opportunity for “imaging on a budget”. Here we show that a consumer cellphone is capable of optical super-resolution imaging by (direct) Stochastic Optical Reconstruction Microscopy (dSTORM), achieving optical resolution better than 80 nm. In addition to the use of standard reconstruction algorithms, we used a trained image-to-image generative adversarial network (GAN) to reconstruct video sequences under conditions where traditional algorithms provide sub-optimal localization performance directly on the smartphone. We believe that “cellSTORM” paves the way to make super-resolution microscopy not only affordable but available due to the ubiquity of cellphone cameras.
- ItemUsing machine-learning to optimize phase contrast in a low-cost cellphone microscope(San Francisco, CA : Public Library of Science, 2018) Diederich, Benedict; Wartmann, Rolf; Schadwinkel, Harald; Heintzmann, RainerCellphones equipped with high-quality cameras and powerful CPUs as well as GPUs are widespread. This opens new prospects to use such existing computational and imaging resources to perform medical diagnosis in developing countries at a very low cost. Many relevant samples, like biological cells or waterborn parasites, are almost fully transparent. As they do not exhibit absorption, but alter the light’s phase only, they are almost invisible in brightfield microscopy. Expensive equipment and procedures for microscopic contrasting or sample staining often are not available. Dedicated illumination approaches, tailored to the sample under investigation help to boost the contrast. This is achieved by a programmable illumination source, which also allows to measure the phase gradient using the differential phase contrast (DPC) [1, 2] or even the quantitative phase using the derived qDPC approach [3]. By applying machine-learning techniques, such as a convolutional neural network (CNN), it is possible to learn a relationship between samples to be examined and its optimal light source shapes, in order to increase e.g. phase contrast, from a given dataset to enable real-time applications. For the experimental setup, we developed a 3D-printed smartphone microscope for less than 100 $ using off-the-shelf components only such as a low-cost video projector. The fully automated system assures true Koehler illumination with an LCD as the condenser aperture and a reversed smartphone lens as the microscope objective. We show that the effect of a varied light source shape, using the pre-trained CNN, does not only improve the phase contrast, but also the impression of an improvement in optical resolution without adding any special optics, as demonstrated by measurements.
- ItemPatterned illumination single molecule localization microscopy (piSMLM): user defined blinking regions of interest cellSTORM - Cost-effective Super-Resolution on a Cellphone using dSTORM(Washington D.C. : Optical Society of America, 2018) Chen, S.-Y.; Bestvater, F.; Heintzmann, Rainer; Cremer, ChristophSingle molecule localization microscopy (SMLM) has been established as an important super-resolution technique for studying subcellular structures with a resolution down to a lateral scale of 10 nm. Usually samples are illuminated with a Gaussian shaped beam and consequently insufficient irradiance on the periphery of the illuminated region leads to artifacts in the reconstructed image which degrades image quality. We present a newly developed patterned illumination SMLM (piSMLM) to overcome the problem of uneven illumination by computer-generated holography. By utilizing a phase-only spatial light modulator (SLM) in combination with a modified Gerchberg-Saxton algorithm, a user-defined pattern with homogeneous and nearly speckle-free illumination is obtained. Our experimental results show that irradiance 1 to 5 kW/cm2 was achieved by using a laser with an output power of 200 mW in a region of 2000 µm2 to 500 µm2, respectively. Higher irradiance of up to 20 kW/cm2 can be reached by simply reducing the size of the region of interest (ROI). To demonstrate the application of the piSMLM, nuclear structures were imaged based on fluctuation binding-activated localization microscopy (fBALM). The super-resolution fBALM images revealed nuclear structures at a nanometer scale.Single molecule localization microscopy (SMLM) has been established as an important super-resolution technique for studying subcellular structures with a resolution down to a lateral scale of 10 nm. Usually samples are illuminated with a Gaussian shaped beam and consequently insufficient irradiance on the periphery of the illuminated region leads to artifacts in the reconstructed image which degrades image quality. We present a newly developed patterned illumination SMLM (piSMLM) to overcome the problem of uneven illumination by computer-generated holography. By utilizing a phase-only spatial light modulator (SLM) in combination with a modified Gerchberg-Saxton algorithm, a user-defined pattern with homogeneous and nearly speckle-free illumination is obtained. Our experimental results show that irradiance 1 to 5 kW/cm2 was achieved by using a laser with an output power of 200 mW in a region of 2000 µm2 to 500 µm2, respectively. Higher irradiance of up to 20 kW/cm2 can be reached by simply reducing the size of the region of interest (ROI). To demonstrate the application of the piSMLM, nuclear structures were imaged based on fluctuation binding-activated localization microscopy (fBALM). The super-resolution fBALM images revealed nuclear structures at a nanometer scale.
- ItemVideo-rate multi-color structured illumination microscopy with simultaneous real-time reconstruction(Berlin : Nature Publishing, 2019) Markwirth, A; Lachetta, Mario; Mönkemöller, V.; Heintzmann, Rainer; Hübner, Wolfgang; Huser, Thomas; Müller, MarcelSuper-resolved structured illumination microscopy (SR-SIM) is among the fastest fluorescence microscopy techniques capable of surpassing the optical diffraction limit. Current custom-build instruments are able to deliver two-fold resolution enhancement with high acquisition speed. SR-SIM is usually a two-step process, with raw-data acquisition and subsequent, time-consuming post-processing for image reconstruction. In contrast, wide-field and (multi-spot) confocal techniques produce high-resolution images instantly. Such immediacy is also possible with SR-SIM, by tight integration of a video-rate capable SIM with fast reconstruction software. Here we present instant SR-SIM by VIGOR (Video-rate Immediate GPU-accelerated Open-Source Reconstruction). We demonstrate multi-color SR-SIM at video frame-rates, with less than 250 ms delay between measurement and reconstructed image display. This is achieved by modifying and extending high-speed SR-SIM image acquisition with a new, GPU-enhanced, network-enabled image-reconstruction software. We demonstrate high-speed surveying of biological samples in multiple colors and live imaging of moving mitochondria as an example of intracellular dynamics.
- ItemThermal illumination limits in 3D Raman microscopy: A comparison of different sample illumination strategies to obtain maximum imaging speed(San Francisco : Public Library of Science, 2019) Hauswald, Walter; Förster, Ronny; Popp, Jürgen; Heintzmann, RainerConfocal Raman microscopy is a powerful tool for material science and biomedical research. However, the low Raman scattering cross-section limits the working speed, which reduces the applicability for large and sensitive samples. Here, we discuss the fundamental physical limits of Raman spectroscopy with respect to signal-to-noise, sample load and how to achieve maximal imaging speed. For this, we develop a simple model to describe arbitrary far field light microscopes and their thermal influence on the sample. This model is used to compare the practical applicability of point- and line-confocal microscopes as well as wide-field-, light sheet- and light line illumination, for the measurement of 3D biological samples. The parallelization degree of the illumination can positively affect the imaging speed as long as it is not limited by thermal sample heating. In case of heat build-up inside the sample, the advantages of parallelization can be lost due to the required attenuation of excitation and the working speed can drop below that of a sequential method. We show that for point like illumination, the exposure time is thermally not as critical for the sample as the irradiance, while for volume like illumination, the exposure time and irradiance result in the same thermal effect. The results of our theoretical study are experimentally confirmed and suggest new concepts of Raman microscopy, thus extending its applicability. The developed model can be applied to Raman imaging as well as to other modes (e.g. two- or three- photon imaging, STED, PALM/STORM, MINFLUX) where thermal effects impose a practical limit due to the high irradiance required.
- ItemAutomated distinction of shearing and distortion artefacts in structured illumination microscopy(Washington D.C. : Optical Society of America, 2018) Förster, Ronny; Müller, Walter; Richter, Renè; Heintzmann, RainerAny motion during an image acquisition leads to an artefact in the final image. Structured illumination microscopy (SIM) combines several raw images into one high-resolution image and is thus particularly prone to these motion artefacts. Their unpredictable shape cannot easily be distinguished from real high-resolution content. We previously implemented a motion detection specifically for SIM, which had two shortcomings which are solved here. First, the brightness dependency of the motion signal is removed. Second, the empirical threshold of the calculated motion signal was not a threshold at a maximum allowed artefact. Here we investigate which artefacts are still acceptable and which linear movement creates them. Thus, the motion signal is linked with the maximal strength of the expected artefact. A signal-to-noise analysis including classification successfully distinguishes between artefact-free imaging, shearing and distortion artefacts in biological specimens. A shearing, as in wide-field microscopy, is the dominant reconstruction artefact, while distortions arise not until surprisingly fast movements.
- ItemMotion artefact detection in structured illumination microscopy for live cell imaging(Washington, DC : Optical Society of America, 2016) Förster, Ronny; Wicker, Kai; Müller, Walter; Jost, Aurélie; Heintzmann, RainerThe reconstruction process of structured illumination microscopy (SIM) creates substantial artefacts if the specimen has moved during the acquisition. This reduces the applicability of SIM for live cell imaging, because these artefacts cannot always be recognized as such in the final image. A movement is not necessarily visible in the raw data, due to the varying excitation patterns and the photon noise. We present a method to detect motion by extracting and comparing two independent 3D wide-field images out of the standard SIM raw data without needing additional images. Their difference reveals moving objects overlaid with noise, which are distinguished by a probability theory-based analysis. Our algorithm tags motion-artefacts in the final high-resolution image for the first time, preventing the end-user from misinterpreting the data. We show and explain different types of artefacts and demonstrate our algorithm on a living cell.
- ItemImpact of deuteration on the ultrafast nonlinear optical response of toluene and nitrobenzene(Washington D.C. : Optical Society of America, 2019) Karras, Christian; Chemnitz, Mario; Heintzmann, Rainer; Schmidt, Markus A.Nonlinear pulse propagation inside highly nonlinear media requires accurate knowledge on the temporal response function of the materials used particular in the case of liquids. Here we study the impact of deuteration on the ultrafast dynamics of toluene and nitrobenzene via all optical Kerr gating, showing substantially different electronic and molecular contributions, which was quantified by fitting a multichannel decay model to the data points. Specifically we found that deuteration imposes the time-integrated nonlinearities to reduce particular for toluene which could be caused by both reduced electronic hyperpolarizabilities as well as weaker intermolecular interactions. The results achieved reveal that deuterated organic solvents represent promising materials for infrared photonics since they offer extended infrared transmission compared to their non-deuterated counterparts while maintained strong nonlinear responses.
- ItemOne-shot phase-recovery using a cellphone RGB camera on a Jamin-Lebedeff microscope(San Francisco, California, US : PLOS, 2019) Diederich, Benedict; Marsikova, Barbora; Amos, Brad; Heintzmann, RainerJamin-Lebedeff (JL) polarization interference microscopy is a classical method for determining the change in the optical path of transparent tissues. Whilst a differential interference contrast (DIC) microscopy interferes an image with itself shifted by half a point spread function, the shear between the object and reference image in a JL-microscope is about half the field of view. The optical path difference (OPD) between the sample and reference region (assumed to be empty) is encoded into a color by white-light interference. From a color-table, the Michel-Levy chart, the OPD can be deduced. In cytology JL-imaging can be used as a way to determine the OPD which closely corresponds to the dry mass per area of cells in a single image. Like in other interference microscopy methods (e.g. holography), we present a phase retrieval method relying on single-shot measurements only, thus allowing real-time quantitative phase measurements. This is achieved by adding several customized 3D-printed parts (e.g. rotational polarization-filter holders) and a modern cellphone with an RGB-camera to the Jamin-Lebedeff setup, thus bringing an old microscope back to life. The algorithm is calibrated using a reference image of a known phase object (e.g. optical fiber). A gradient-descent based inverse problem generates an inverse look-up-table (LUT) which is used to convert the measured RGB signal of a phase-sample into an OPD. To account for possible ambiguities in the phase-map or phase-unwrapping artifacts we introduce a total-variation based regularization. We present results from fixed and living biological samples as well as reference samples for comparison.