Microscope with multiple image sensors for fluorescence imaging of multiple locations and/or wavelengths

This U.S. patent application is a Continuation-in-Part of U.S. patent application Ser. No. 14/991,208, filed on Jan. 8, 2016, and claims priority thereto under 35 U.S.C. §120. The disclosure of the above-referenced U.S. patent application is incorporated herein by reference.

1. Field of the Invention

The present invention relates to biological microscopy, and more particularly concerns a microscope probe for in vivo or in vitro fluorescence imaging of multiple locations and/or wavelengths using multiple image sensors integrated within the microscope.

2. Background of the Invention

When making microscopic observations of in vivo biological specimens, minimally invasive techniques are required. However, conventional microscopes feature large size microscope objectives. Large objectives cannot be implanted within a sample without causing significant damage to the sample, such as removing structures in order to facilitate access to deep layers within the sample. Less invasive fiber-optic cannulae that are typically used in optogenetics experiments are typically configured only for optical stimulation or silencing of a certain class of neurons without imaging capabilities. More recent solutions include miniaturized microscopes that can be detachably coupled to a base plate mounted on a subject, such as those disclosed in U.S. Pat. No. 9,195,043 and U.S. Patent Application Publication U.S. 20150309295.

While miniaturized microscopes are available, as noted above they are limited in their flexibility of performing fluorescence measurements.

Therefore, it would be desirable to provide an optogenetic compatible fluorescence microscope with additional facilities.

The invention encompasses microscopes capable of performing fluorescence imaging on a sample having an attached optical cannula, which is a fixture on the sample having one or more optical probes focusing and/or extending into the sample for delivering light into and receiving light from regions or points within the sample. The invention also encompasses a method of performing fluorescence imaging on a sample.

The microscope includes a first connector for receiving an optical illumination connection coupling light from one or more illumination sources and a second connector adapted to mechanically connect to the cannula. The second connector includes an optical interface for coupling light returning from the sample to the microscope and coupling illumination from the illumination sources to the sample. The microscope further includes a pair of optical splitters: a first optical splitter that separates the light returning from the sample from the illumination light provided to the second connector, and a second optical splitter for splitting the light returning from the sample into two images, which are provided to corresponding image sensors within the microscope.

The second optical splitter may be an optical filter that splits light of two different wavelengths returning from the sample, or may be an intensity-division or polarization beam-splitter in the case in which the two different images are of the same wavelengths, but from two different object planes, e.g., depths within the sample.

The foregoing and other objectives, features, and advantages of the invention will be apparent from the following, more particular, description of the preferred embodiment of the invention, as illustrated in the accompanying drawings.

The novel features believed characteristic of the invention are set forth in the appended claims. The invention itself, however, as well as a preferred mode of use, further objectives, and advantages thereof, will best be understood by reference to the following detailed description of an illustrative embodiment when read in conjunction with the accompanying drawings, wherein like reference numerals indicate like components, and:

FIG. 1 is a side cross-section view of an exemplary miniaturized microscope system.

FIG. 2 is a front cross-section view of the exemplary miniaturized microscope system of FIG. 1 .

FIG. 3 is a side cross-section view of an exemplary cannula that can be used with the exemplary miniaturized multimodal microscope systems disclosed herein.

FIG. 4 is a side cross-section view of another exemplary miniaturized microscope system.

FIG. 5 is a pictorial diagram illustrating an optical path length shift in an exemplary miniaturized microscope system.

FIGS. 6-7 are graphs depicting optical transmission band characteristics of filters that may be used within the miniaturized multimodal microscope systems depicted herein.

FIG. 8 is a pictorial diagram illustrating an optical path length and object plane shift in an exemplary miniaturized microscope system.

FIG. 9 is a side cross-section view of an exemplary miniaturized microscope system illustrating an object plane shift for single-wavelength measurement.

The systems described herein are miniaturized multi-imaging systems of generally less than a cubic inch in size that combine capability for imaging of two different images, which may be two different colors (wavelengths) or imaging two different object planes using a pair of internal image sensors. Typical multi-color microscopes are bulky and traditionally limited to tabletop applications. An application for the microscopes disclosed herein is fluorescence imaging of fluorescent proteins for applications requiring monitoring of cell activity. The system includes an objective lens to create a magnified image of one or two regions of the sample on two (or more) image sensors integrated within the microscope. The objective lens may be integrated in the microscope or be integrated in a cannula to which the microscope is attached. The microscope includes two optical splitters: one to separate illumination light from fluorescence signals and another to separate the fluorescence signals into two images for measurement by the image sensors.

In some applications, a sample may be labeled with two different fluorescent markers having distinct fluorescence emission spectra and distinct excitation spectra, which avoids cross-talk between measurement channels. While the first marker is used to measure activity, the second marker may be present as a reference, or also for cell activity monitoring. Providing two different fluorescence measurement channels provides for simultaneous capture of images due to the presence of the two fluorescent markers. In other applications, the two channels can be configured to image two different object planes at different depths within the sample. The microscope also includes a connection for receiving illumination light from one or more external light sources with an optical arrangement to bring illumination light to the subject for fluorescence imaging. For multi-color measurement, light from two illumination sources can be combined and provided to the illumination connection, or for sequential multi-color measurement, two different illumination sources can be sequentially connected. For measurements of the same wavelength at different object planes, only one illumination source may be required.

The miniaturized microscope disclosed herein is configured to easily connect to and disconnect from the cannula and includes a specific connector adapted to provide precise optical alignment with the cannula, which may be a cannula such as those described in U.S. Patent Application Publication US20150366437A1, the disclosure of which is incorporated herein by reference and also in the above-incorporated parent U.S. patent application. The imaging system is designed to observe one or many fields of view smaller than 0.5 mm 2 (to reduce invasiveness) with a spatial resolution at the micron scale. An optical splitter set is included to separate one or more bands of illumination light from the fluorescence signal returning from the sample, to provide the illumination light to the sample, and to separate the fluorescence signals and direct them to corresponding image sensors. The objective lens may be incorporated in the microscope body, or in the cannula and may image light from an implant inserted in the sample, which may be a relay lens, e.g. a GRIN lens, and/or a glass rod. Alternatively, the objective lens may image one or more regions near the surface of the sample or at some depth within a transparent or translucent sample. Other configurations and combinations, such as those disclosed herein or otherwise provided by combining elements that are shown by the present disclosure or their equivalents, are contemplated and the examples chosen as illustrations should not be considered limiting. In the examples given herein, the field of view of the microscope described herein is less than 0.5 mm 2 and having a spatial resolution at the micron scale, which also differs from existing miniature microscope systems. By limiting the field of view to less than 0.5 mm 2 , the volume of the microscope can be reduced and the image quality improved over the field of view. To improve the portability and facilitate the access to a confined region of interest, the total size of the miniaturized microscope can be maintained below 1 in 3 with a weight of under 4 g.

Referring now to FIG. 1 , an example of a miniaturized microscope system for fluorescence measurement is shown in a side cross-section view, respectively. A microscope body 1 A has a connector adapted for connection to a cannula 2 A that is attached to a sample, which is generally a biological test specimen for in vivo measurements as described in the above-incorporated U.S. patent applications. The microscope, which includes microscope body 1 A, also includes an optical interface at two different object planes 5 A and 5 B of the optical system below an objective lens 7 A. In the illustrated microscope, two object planes 10 A and 10 B are co-located, i.e., they are at the same depth within a sample to which cannula 2 A is attached. Due to optical path length differences due to a difference in wavelength for the two images generated by the microscope system of FIG. 1 , co-located object planes 10 A and 10 B of the system are imaged on object planes 5 A and 5 B, respectively, of microscope body 1 A. Object planes 5 A and 5 B are not co-located, and the corresponding detection pathways 4 A and 4 B to a pair of corresponding image sensors 6 A and 6 B differ. Image sensors 6 A and 6 B may be CMOS or CCD sensors mounted to corresponding circuit boards 22 A and 22 B, and are interfaced to an external image processing system via an electrical cable 15 terminated at a high speed connector 16 B and connected to microscope body 1 A via another high speed connector 16 A that provides an interface to signals on printed circuit boards 22 A and 22 B.

A first optical splitter 12 A, which in the illustrated embodiment is a dichroic filter, directs light returning from the sample that contains both images corresponding to two different wavelengths of fluorescent emissions from the location of object planes 10 A and 10 B toward a second optical splitter 13 A. First optical splitter 12 A is generally a dual-bandpass filter designed to pass illumination of two different excitation wavelengths, and thus has pass-bands centered around the excitation wavelengths and reflects both of the fluorescence wavelengths to direct the light returning from the sample to second optical splitter 13 A. In some applications, the excitation wavelength is a single excitation wavelength that excites two spectrally distinct fluorescence markers, e.g., Förster resonance energy transfer (FRET) applications. In such applications, first optical splitter 12 A will generally have a short-pass filter dichroic characteristic. Second optical splitter 13 A, in the depicted embodiment, is also a dichroic filter that divides the light returning from the sample into two images, one provided at image sensor 6 A via transmission through optical splitter 13 A and the other provided at image sensor 6 B by reflection from optical splitter 13 A, which is inclined at 45 degrees. A pair of corresponding emission filters 23 A and 23 B, which are generally single-passband filters, are provided to remove stray light, illumination leakage and leakage from the other channel from the fluorescence signals corresponding to the images of object planes 10 A and 10 B provided to image sensors 6 A and 6 B, respectively. Therefore, emission filters 23 A and 23 B have pass-bands 44 A and 44 B, as shown in FIG. 7 , centered on the respective wavelengths of the corresponding fluorescent markers that are measured by respective image sensors 6 A and 6 B.

A pair of lenses 8 A and 8 B provide adjustment of the magnification between object planes 5 A- 5 B and image sensors 6 A- 6 B. To optimize aberration correction and increase the numerical aperture of the objectives, a high numerical aperture lens is used as objective lens 7 A. In the illustrated embodiment, the numerical aperture of objective lens 7 A is between 0.3 and 0.5. To reduce the total size of the system, the focal length of the objective lens, the length of the detection pathways 4 A- 4 B, and the focal length of lenses 8 A- 8 B are chosen to produce a magnification ratio between 2× and 10× between object planes 5 A- 5 B and image sensors 6 A- 6 B. The lengths of the detection pathways 4 A- 4 B and the focal length of the lenses 8 A- 8 B are adjusted to obtain the same magnification in both detection pathways 4 A- 4 B. The insertion of lenses 8 A- 8 B in detection pathways 4 A- 4 B also reduces the angular divergence at the surface of emission filters 23 A- 23 B and therefore improves the efficiency of emission filters 23 A- 23 B. In the depicted embodiment, a lens 9 may also be used to relay the images of object planes 10 A- 10 B within the subject to the respective object planes 5 A- 5 B of the microscope body.

In the depicted example, lens 9 , which in the depicted embodiment is a gradient index (GRIN) relay lens, extends into the sample to image a region of interest within the sample at two different wavelengths providing images of object planes 10 A and 10 B at different wavelengths and the same depth. GRIN relay lens 9 is integrated in cannula 2 A and is optically aligned with the optical axis 11 of the objective lens 7 A. GRIN relay lens 9 is not required for imaging surfaces just below cannula 2 A. In other embodiments illustrated below, the objective lens is integrated in a cannula and no relay lens is used. An illumination pathway 3 extends from a connector 19 at the top of microscope body 1 A that receives an optical coupling, such as an optical fiber, from one or more illumination sources that provide light of at least one nominal wavelength for exciting fluorescence in the sample. The depicted microscope system further includes an optical patch cord 14 , e.g., multi-mode optical fiber or fiber bundle that conducts the illumination light from the external illumination source. A threaded female connector 18 mates with male threaded connector 19 of microscope body 1 A to secure optical patch cord 14 to microscope body 1 A. Connector 18 may be of a standard type for connection to various illumination sources, which may include laser diodes, solid state lasers, gas lasers, fiber lasers, etc. In the particular embodiments described herein, the illumination source is a laser, LED or Ce:YAG incoherent illumination source providing illumination for imaging of cells in biomedical applications. A bored recess 20 in microscope body 1 A receives a ferrule 17 that holds the terminated end of optical patch cord 14 , which contacts a hub 21 providing an optical interface receiving the illumination light, i.e., the light for fluorescence excitation, which is directed to an optical interface at object planes 5 A- 5 B of objective lens 7 A for introduction to GRIN relay lens 9 in cannula 2 A. An illumination lens system 50 forms a collimated beam along illumination pathway 3 that is filtered by an excitation filter 24 A, and which is then collimated at the object plane(s) of the objective lens 7 A, for introduction to GRIN relay lens 9 in cannula 2 A along optical axis 11 after passing through optical splitter 12 A. Excitation filter 24 A includes at least one optical transmission band corresponding to the nominal wavelength of the illumination source coupled through connector 19 and cleans the illumination light spectrum before it is introduced to GRIN relay lens 9 , removing light artifacts such as auto-fluorescence generation in the optical fibers coupling the illumination light to connector 19 .

Referring additionally now to FIG. 2 , further details of the microscope shown in FIG. 1 are illustrated. Illumination lens system 50 includes a high numerical aperture aspheric lens 25 A and a plano-convex spherical lens 26 A, and forms a collimated beam that is filtered by excitation filter 24 A. Illumination lens system 50 is configured to provide a collimated and uniform illumination over the entire field of view in object planes 5 A- 5 B, in order to avoid artifacts due to illumination discontinuities. Illumination lens system 50 is configured so that the output of the optical fiber at the tip of ferrule 17 is imaged at the back focal plane of objective lens 7 A, which ensures that defects or dust present at the interface of ferrule 17 after multiple connections and disconnections have been made are not imaged in object planes 5 A- 5 B. A separate high numerical aperture aspheric lens 25 A and plano-convex spherical lens 26 A are not required. In alternative embodiments, a single high numerical aperture aspheric lens 25 A may be used to focus illumination light at the back aperture of objective lens 7 A. However, using two lenses reduces the angle of incidence of light at the surface of excitation filter 24 A and thereby improves filtering efficiency.

FIG. 2 and additionally FIG. 3 show further details of an attachment mechanism for cannula 2 A, a cannula similar to which is described in detail in the above-incorporated Parent U.S. patent application U.S. Ser. No. 14/991,208. In the depicted embodiment, use of such a cannula provides convenient attachment and removal of microscope body 1 A from cannula 2 A, but is not a requirement to practice the invention, except as recited in particular Claims. Cannula 2 A includes an orientation key 54 that provides rotational alignment of cannula 2 A with microscope body 1 A. An interior top surface 51 of cannula 2 A is shaped to adapt cannula 2 A to accept a guiding taper 29 around the bottom portion of microscope body 1 A and a cylindrical shoulder 27 contacts a corresponding shoulder 28 on microscope body 1 A to prevent movement of microscope body 1 A with respect to cannula 2 A once microscope body 1 A is secured to cannula 2 A. Shoulder 27 defines a slot 37 around the exterior surface of cannula 2 A. A precise optical alignment between objective lens 7 A and GRIN relay lens 9 is achieved once microscope body 1 A is secured to cannula 2 A. As seen in FIG. 2 , additional components that secure microscope body 1 A to cannula 2 A are shown. Latch hooks 35 at the end of a pair of latches 33 capture shoulder 27 when latches 33 are expanded as inner surfaces of extensions 52 of latches 33 slide along side walls 53 of microscope body 1 A. Side walls 53 of microscope body 1 A are shaped to form an acute angle with respect to the cylindrical axis of the cannula 2 A, so that, when latches 33 are moved downwards toward cannula 2 A, side walls 53 act as guides for expanding latch hooks 35 to release shoulder 27 of cannula 2 A. The upper end of latches 33 forms a retention shoulder 34 which compresses a compression spring 36 that provides for secure latching of latch hooks 35 by pressing against the lower surface of retention shoulder 34 and a shoulder 38 of microscope body 1 A to pull latches 33 upwards. Cannula 2 A and microscope body 1 A are connected by pressing springs 36 and pressing on latches 33 toward cannula 2 A. A pin may be inserted between latch hooks 35 and cannula 2 A to release latch hooks 35 from slot 37 . Cannula 2 A is secured to the specimen being observed, in general, by fastening a flange 32 of cannula 2 A to the specimen with an adhesive and/or with fasteners such as screws. The flange 32 is attached to a body 31 of cannula 2 A. To adjust the focus and the working distance of the microscope system (or to adjust the penetration depth of GRIN relay lens 9 if present), cannula 2 A includes an adjustment ring 30 that is glued and/or screwed on the specimen via flange 32 . In the example, body 31 of the cannula 2 A includes a male threaded portion on an outer surface. A threaded female inner surface of adjustment ring 30 mates with a threaded male portion of body 31 and is rotated to adjust the distance between the microscope system and the specimen (or the penetration depth of GRIN relay lens 9 ) when adjustment ring 30 is attached to the specimen.

Referring now to FIG. 4 , another example of a microscope system for fluorescence measurement is shown in a side cross-section view. The microscope system of FIG. 4 differs from the microscope system of FIGS. 1-2 in that an objective lens 7 B is integrated in a cannula 2 B detachably coupled to a microscope body 1 B and is provided by a high numerical aperture type of objective lens 7 B. Since the optical path differs from the microscope system of FIGS. 1-2 , a high numerical aperture aspheric lens 25 B, a plano-convex spherical lens 26 B, an excitation filter 24 B, optical splitters 12 B and 13 B, lenses 8 C- 8 D, emission filters 23 C- 23 D may be of different dimensions. Locating objective lens 7 B in cannula 2 B increases the working distance and image field of view up to 0.5 mm 2 without increasing invasiveness in the subject. There is no GRIN relay lens in the depicted embodiment, and an output window 60 is used to seal the microscope.

Each of the above-described microscope configurations can be arranged to image a single object plane (single depth within the sample) at two different emission wavelengths, or two distinct object planes (different depths within the sample) at one or two emission wavelengths. As described above, two illumination sources can be coupled to the microscopes to provide stimulus for emission at two different wavelengths, or in some instances, two measured emission wavelengths can be stimulated by a single illumination wavelength. The characteristics of optical splitters 12 A, 13 A and 12 B, 13 B are selected based on the type of measurement being performed. For example, if the microscope of FIG. 1 or FIG. 4 is used to image object planes at different depths but the same wavelength, optical splitter 13 A or 13 B is an intensity division beam-splitter or polarization beam-splitter rather than a dichroic filter.

Referring now to FIG. 5 , a two-color (two emission wavelength) single object plane configuration for the above-described microscopes is shown in a pictorial diagram. GRIN relay lens 9 is shown twice for clarity. First object plane 10 A is imaged at a first image plane 39 A and second object plane 10 B, co-located with first object plane 10 A is imaged at a different second image plane 39 B. The optical path length between first object plane 10 A and first image plane 39 A differs from the optical path length between second object plane 10 B and second image plane 39 B, which is at least due to chromatic aberration within GRIN relay lens 9 as the emission wavelength received at first image plane 39 A differs from the emission wavelength received at second image plane 39 B. A resulting chromatic focal shift 40 corresponds to the difference in optical path length of the detection pathways (e.g., detection pathways 4 A and 4 B of FIG. 1 ) at their corresponding emission wavelengths, which are adjusted to co-locate object planes 10 A and 10 B at their corresponding emission wavelengths. The detection pathways of the microscope (e.g., detection pathways 4 A and 4 B of FIG. 1 ) are adjusted to co-locate object planes 5 A and 5 B of the microscope outside of objective lens 7 A with a corresponding one of first image plane 39 A and second image plane 39 B, so that the two detection channels of the microscope ultimately image first object plane 10 A and second object plane 10 B at their corresponding emission wavelengths when, for example, cannula 2 A is attached to microscope body 1 A. In a configuration that does not employ GRIN relay lens 9 , such as the microscope of FIG. 4 , the detection pathways are adjusted to co-locate object planes 5 A, 5 B of the microscope at their corresponding emission wavelengths, which are set to some depth within the sample.

Referring now to FIG. 6 , characteristics of a first optical splitter (e.g., optical splitter 12 A of FIG. 1 ) and an excitation filter (e.g., excitation filter 24 A of FIG. 1 ) are shown for a two illumination color configuration. In the instant configuration, optical splitter 12 A is a two pass-band dichroic filter for passing two illumination wavelengths and reflecting the emission wavelengths. Passbands 42 illustrate the passbands of optical splitter 12 A. Excitation filter 24 A may generally have a narrower passband characteristic 41 for purifying the spectra of the excitation illumination sources. In instances in which the two emission spectra are stimulated by a single illumination wavelength, a single passband will be present in each of optical splitter 12 A and excitation filter 24 A.

Referring now to FIG. 7 , characteristics of a second optical splitter (e.g., optical splitter 13 A of FIG. 1 ) and emission filters (e.g., emission filters 23 A- 23 B of FIG. 1 ) are shown for a two emission color configuration. In the instant configuration, optical splitter 13 A is a short-pass filter having a short-pass characteristic 43 for passing a first emission wavelength and reflecting a longer second emission wavelength, thereby separating the light returning from two imaged object planes 5 A and 5 B at two different corresponding wavelengths. Passbands 44 A and 44 B illustrate the passbands of emission filters 23 A and 23 B, respectively. For single emission wavelength applications, optical splitter 13 A will generally be a wavelength-agnostic beam-splitter (intensity-division beam-splitter) or polarization beam-splitter that divides the light returning from the two object planes 5 A and 5 B equally and emission filters 23 A and 23 B will generally have an identical passband characteristic.

Referring now to FIG. 8 , a single-color (single emission wavelength) dual object plane configuration for the above-described microscopes is shown in a pictorial diagram. GRIN relay lens 9 is again shown twice for clarity. A first object plane 10 C is imaged at a first image plane 39 C and a second object plane 10 D, at a different depth within the sample from first object plane 10 C is imaged at a different second image plane 39 D. The optical path length between first object plane 10 C and first image plane 39 C does not differ from the optical path length between second object plane 10 D and second image plane 39 D, since the emission spectra is the same for both light received at first image plane 39 C and light received at second image plane 39 D. The optical path length of the detection pathways differ at the emission wavelength, in order to separate object planes 10 C and 10 D. Referring additionally to FIG. 9 , a microscope having detection pathways 4 C and 4 D illustrates an arrangement that separates object planes 10 C and 10 D at a single emission wavelength by adjusting detection pathways 4 C and 4 D to locate object planes 5 C and 5 D at corresponding image planes 39 C and 39 D of FIG. 8 . As described above for the instant configuration, an excitation filter 24 C has a single-passband configuration, an optical splitter 13 C is an intensity-division beam-splitter and emission filters 23 C and 23 D have identical single band-pass characteristics. An optical splitter 12 C may have a short-pass characteristic that passes the excitation illumination wavelength and reflects the longer wavelength emitted light returning from the sample. In other configurations of the two-object plane system described above, an additional illumination wavelength may be provided for opto-genetic stimulation for stimulating light-driven ion channels in the sample. In such an application, excitation filter 24 C has a dual band-pass characteristic to clean the illumination at the fluorescent excitation and opto-genetic stimulation wavelengths and optical splitter 12 C also has a dual pass-band to pass both illumination wavelengths and to reflect the fluorescent emissions returning from the sample.

While the invention has been particularly shown and described with reference to the preferred embodiment thereof, it will be understood by those skilled in the art that the foregoing and other changes in form, and details may be made therein without departing from the spirit and scope of the invention.