CHEMIJA. 2022. Vol. 33. No. 1. P. 7–11
© lietuvos mokslų akademija, 2022

Three different techniques to reconstruct 3D view of SEM images by using only free available software

Edvardas Brimas1,

Ramūnas Skaudžius1,

Gintautas Brimas2,

Algirdas Selskis3,

Rimantas Ramanauskas3,

Aivaras Kareiva1 *

1 Department of Inorganic Chemistry, Vilnius University, 24 Naugarduko Street, 03225 Vilnius, Lithuania

2 Clinic of Gastroenterology, Nephrourology and Surgery, Department of General Surgery, Vilnius University, 29 Šiltnamių Street, 04130 Vilnius, Lithuania

3 Center for Physical Sciences and Technology, 3 Saulė tekio Avenue, 10257 Vilnius, Lithuania

In the present work, three different techniques were used to obtain a 3D view of the SEM images of adipose tissue samples taken from volunteer obese patients by only freely available software. The extrapolation of spatial data demonstrates the most promising results compared to the results of a stereoscopic view or interactive surface plots. The 3D reconstructed view of SEM images by extrapolation of spatial data allows one to obtain the depth of the sample from the SEM images. The reconstructed model also might be printed with a 3D printer, which extends the usage possibilities of this technique.

Keywords: 3D reconstruction, SEM, biological tissue, free available software

INTRODUCTION

Microscopy is a powerful tool used in many fields of science and technology. However, it has some limitations, like resolution and depth of focus [1]. In this sense, scanning electron microscopy (SEM) is superior to light microscopy, although a SEM micrograph is still a 2D projection of a three-dimensional object [2]. As a consequence, interpretation of such data may remain qualitative or even misleading [3]. So there is a strong interest in improving the reliability of quantitative microstructural analysis and 3D reconstruction is an essential tool [4].

Focused ion beam (FIB) sectioning in combination with electron microscopy for 3D analysis has been known for a while and successfully applied in studies of inorganic systems [3, 5]. 3D FIB-SEM imaging has also been useful in studies of biological samples like mammalian cells [6] or bone tissue [7]. Unfortunately, this is a destructive and time-consuming technique so alternatives are being developed.

3D surface reconstruction of SEM images using stereo-photogrammetry has been introduced a long time ago [8], but since it demands high computational capacities only in recent years it started receiving more interest [2, 9, 10]. With the advancement of technology, nowadays it is a simple and inexpensive task to make a digital 3D model of an object using a smartphone camera, a home computer and freely available software. However, same workflows could not be easily applied on SEM micrographs since images have a different nature. Nevertheless, specialised techniques and algorithms are being developed [11, 12] and applied in many fields, including structural biology [13–15]. 3D imaging could be a comprehensive tool in histology [16–18]. Moreover, such data could be applied in 3D printing of biocompatible biomaterials [19, 20]. In the present study, three different ways to obtain a 3D view of the SEM images of biological tissue by using only free available software were demonstrated.

EXPERIMENTAL

Preparation of adipose tissues

Adipose samples were taken from volunteer patients at the Department of General Surgery of Vilnius University Hospital. Men and women aged 18–65 years with a BMI more than 30 kg/m2 were enrolled in the study. The study protocol was approved by the Lithuanian Ethics Committee, with the aim and design of the study explained to each subject, who in turn gave their informed consent. The adipose tissue samples were stored in –70°C temperature before the chemical analysis was performed. The adipose tissue samples were homogenised and extracted using a modified Folch extraction procedure. The lipids were extracted by 20 ml of chloroform/methanol (2:1, v/v), as reported previously [21].

Characterisation

Morphology of the adipose tissue samples was evaluated by field emission scanning electron microscopy (FE-SEM, SU70, Hitachi) equipped with an energy dispersive X-ray spectrometer, and the spectrometer was controlled by the INCA software (Oxford Instruments) or by Hitachi TM3000 and Helios NanoLab 650 instruments. The database of SEM images of the adipose tissue taken from 3 layers of adipose tissue (subcutaneous, preperitoneal and visceral) was collected. The most characteristic examples were selected for 3D reconstruction. The approaches of three different techniques were compared to obtain a 3D view of adipose tissue engaging only the freely available software:

Stereoscopic view (special glasses needed): the approach by combining two SEM images of a specimen from two slightly different angles was used for the reconstruction of 3D SEM images using stereoscopic technique. The combination of two SEM images into an anaglyph was done with the freely available software AnaMaker. The more detailed information about the stereoscopic technique used to characterise particles of Au was discussed in more details by S. Kareiva et al. [12]. The same technique was used in the present work for the adipose tissue samples.

Interactive 3D Surface Plot plugin for Fiji: the plugin creates interactive surface plots from all image types. The luminance of an image is interpreted as the height for the plot. Internally the image is scaled to a square image using the nearest neighbour sampling. For selections, the bounding box of the selection is used for the surface plot [22].

Extrapolation of spatial data: this approach was done from at least three or more SEM images made by changing the tilt by 5 or 10° step. The 3D model of SEM images reconstructed by the VisualSFM and MeshLab open source software is based on the following steps:

1. Convert the SEM images to a format suitable for processing (i.e. JPG).

2. Find local features in the given set of images.

3. Find matching feature-sets between the image pairs.

4. Construct the spatial positions of feature points based on their trajectories.

5. Map the image data (colour) to the spatial points.

6. Connect the spatial points to build a vertex or other surface-like structure.

In addition, the Fiji (ImageJ) software was used as an additional software to analyse the surface of the samples obtained from the SEM images. The FigureJ plugin [23] was used to prepare the figures for this publication.

RESULTS AND DISCUSSION

SEM images of adipose tissue

First of all, the data base of SEM images of different adipose tissue was collected. The SEM images were divided into two big groups: the group of patients who has metabolic syndrome (a collection of risk factors that increase the chance of developing heart disease, stroke and diabetes) and the one who has not. The random selected images are presented in Fig. 1. Note that the SEM images of the same type of adipose tissue were very similar in the patients having the metabolic syndrome and the patients without it. Nevertheless, the adipose tissue from the deeper level exhibits a more rough surface. The hypothesis was proposed that the SEM images of the patients having the metabolic syndrome contains more white edges on the sample surface caused by the inflammation. To confirm/decline the hypothesis the ImageJ (Fiji) software was empowered to find the edges (function on the process menu) of each image. Then the images were turned to binary images (function on the process menu) in order to calculate the total length of the edges lines. Unfortunately, there was no statistical difference between the samples of the patients who have the metabolic syndrome and the ones who have not independent of the type of adipose tissue.

img

Fig. 1. SEM images of a different type of the adipose tissue of the volunteer patient who has the metabolic syndrome recruited at the Department of General Surgery of Vilnius University Hospital: a, subcutaneous adipose tissue; b, the deeper level of subcutaneous adipose tissue; c, preperitoneal adipose tissue; d, visceral adipose tissue

Stereoscopic view and 3D reconstruction of adipose tissue

By using any online tool (a stereo images generator from the photo) or freely available software such as AnaMaker the stereo image might be obtained in several seconds. Figure 2 demonstrates a stereo image of the selected subcutaneous adipose tissue of the volunteer patient at the Department of General Surgery of Vilnius University Hospital. The proper view is seen only with 3D glasses, which makes this method less convenient. Moreover, the stereo image still has very poor information about the real depth of surface irregularities and the 3D surface is not fully described. Therefore, this technique was rejected for the further development. The second approach to obtain a more detailed surface reconstruction was realised by the Interactive 3D Surface Plot plugin as a tool. This technique gives the detailed view of the surface (Fig. 3). However, the main disadvantage of this visualisation is that the pixel size values of x and y are known, though the z values are dimensionless. The z value has to be interpreted and set manually, therefore, the different technique should be used additionally to obtain a reasonable z value.

img

Fig. 2. An example of a SEM image of the deeper level of subcutaneous adipose tissue (the same sample as depicted in Fig. 1b) turned into a stereoscopic image (3D glasses needed for a proper view)

img

Fig. 3. Interactive 3D surface of the deeper level of subcutaneous adipose tissue. The same sample as depicted in Fig. 1b

The very first results of the 3D model of SEM images reconstructed by the VisualSFM and MeshLab open source software have demonstrated that the reconstruction is very dependent on the equipment. FE-SEM, SU70, Hitachi instrument demonstrated the best results of 3D reconstruction, therefore it was used for the further investigation. The 3D reconstruction method is also very sensitive to the noise levels of the images and to the changes of the intensity of the grey scale of images. Another difficulty was to keep exactly the same area for measurements. Every time when the tilt and/or the rotation were changed, it was really difficult to get back to the same position. To sum up, it was expected that the 1–5° tilt would be optimal for the reconstruction [24], while the experimental work has demonstrated that the 10° step of tilt was optimal for the reconstruction. The developed model for the 3D reconstruction designed for regular photos was successfully applied for the investigation of adipose tissues of volunteer patients at the Department of General Surgery of Vilnius University Hospital. The 3D reconstructed view of SEM images is presented in Fig. 4. It is clearly seen that the surface was fully reconstructed. The very tentative results demonstrate that the depth size of the sample might be extrapolated (Fig. 4e). Moreover, the depth size obtained by this 3D model might be used for the development of the surface visualisation obtained by the Interactive 3D Surface Plot plugin in Fiji. The reconstructed 3D view helps to visualise the adipose tissue in more details. Moreover, the reconstructed model might be printed with a 3D printer, which extends the usage possibilities. The short demonstration of 3D reconstruction results of subcutaneous adipose tissue of the patient having the metabolic syndrome as well as the 3D reconstruction results of the different types of adipose tissue of the patient who has not the metabolic symptom are presented as an additional information.

img

Fig. 4. a–d: SEM images of subcutaneous adipose tissue made by changing the tilt; e: an example of a 3D reconstructed view of the SEM images of subcutaneous adipose tissue of the patient having the metabolic syndrome

CONCLUSIONS

Three different techniques were tested to construct the 3D view from the SEM images of adipose tissue. The stereoscopic view gave the poor information, while the reconstructed surface of adipose tissue formed by the Interactive 3D Surface Plot plugin in Fiji demonstrated more rich data. Nevertheless, since the pixel does not have the depth dimension z, the interpretation of the data is very difficult. The 3D reconstructed model by the VisualSFM and MeshLab free available software gave the best and the most promising results. The surface is fully reconstructed and the pixel z dimension might be easily extrapolated. The reconstructed view also might be easily printed by a 3D printer which gives a new area of applications. To conclude, it was demonstrated that free available software might be successfully applied for the 3D reconstruction of the SEM images of adipose tissue.

Received 2 December 2021

Accepted 23 December 2021

References

1. N. T. Goldsmith, Image Anal. Stereol., 19, 163 (2000).

2. F. Marinello, P. Bariani, E. Savio, et al., Meas. Sci. Technol., 19, 065705 (2008).

3. L. Holzer, F. Indutnyi, P. H. Gasser, et al., J. Microsc.-Oxford, 216, 84 (2004).

4. J. L. Pouchou, D. Boivin, P. Beauchene, et al., Microchim. Acta, 139, 135 (2002).

5. M. Ritter, P. A. Midgley, J. Phys. Conf. Ser., 241, 012081 (2010).

6. J. A. Heymann, D. Shi, S. Kim, et al., J. Struct. Biol., 166, 1 (2009).

7. P. Schneider, M. Meier, R. Wepf, R. Muller, Bone, 49, 304 (2011).

8. G. Piazzesi, J. Phys. E, 6, 392 (1973).

9. L. C. Gontard, J. D. Lopez-Castro, L. Gonzalez-Rovira, et al., Ultramicroscopy, 177, 106 (2017).

10. V. N. Tondare, J. S. Villarrubia, R. A. Vlada, Microsc. Microanal., 23, 967 (2017).

11. A. P. Tafti, A. B. Kirkpatrick, Z. Alavi, et al., Micron, 78, 54 (2015).

12. S. Kareiva, A. Selskis, F. Ivanauskas, S. Sakirzanovas, Pure Appl. Chem., 87, 283 (2015).

13. M. Eulitz, G. Reiss, J. Struct. Biol., 191, 190 (2015).

14. A. Kremer, S. Lippens, S. Bartunkova, et al., J. Microsc., 259, 80 (2015).

15. K. Narayan, S. Subramaniam, Nat. Methods, 12, 1021 (2015).

16. D. Cretoiu, E. Hummel, H. Zimmermann, et al., J. Cell. Mol. Med., 18, 2157 (2014).

17. S. Kaul-Strehlow, T. Stach, Front. Zool., 10, 12 (2013).

18. G. Brimas, R. Skaudzius, V. Brimiene, et al., Med. Hypotheses, 136, 109526 (2020).

19. H. N. Chia, B. M. Wu, J. Biol. Eng., 9, 4 (2015).

20. W. Zhu, X. Ma, M. Gou, et al., Curr. Opin. Biotechnol., 40, 103 (2016).

21. B. Buckus, G. Brimas, A. Stašinskas, et al., Chemija, 26, 98 (2015).

22. K. U. Barthel, in: Proceedings of the ImageJ User and Developer, Luxembourg (2006).

23. J. Mutterer, E. Zinc, J. Microsc., 252, 89 (2013).

24. S. Kareiva, V. Klimavicius, A. Momot, et al., J. Mol. Struct., 1119, 1 (2016).

* Corresponding author. Email: aivaras.kareiva@chgf.vu.lt

Edvardas Brimas, Ramūnas Skaudžius, Gintautas Brimas, Algirdas Selskis, Rimantas Ramanauskas, Aivaras Kareiva

TRYS SKIRTINGI SEM NUOTRAUKŲ 3D VAIZDO ATKŪRIMO BŪDAI, NAUDOJANT TIK LAISVAI PRIEINAMĄ PROGRAMINĘ ĮRANGĄ

Santrauka

Šiame darbe buvo naudojami trys skirtingi metodai, norint gauti riebalinio audinio mėginių, paimtų iš savanorių nutukusių pacientų, SEM nuotraukų 3D vaizdą. Naudota tik laisvai prieinama programinė įranga. Parodyta, kad erdvinių duomenų ekstrapoliacija, palyginti su stereoskopinio vaizdo ar interaktyvių paviršiaus diagramų rezultatais, yra perspektyviausias metodas. 3D rekonstruotas SEM nuotraukų vaizdas, ekstrapoliuojant erdvinius duomenis, leidžia atlikti mėginio giluminį (tūrinį) vaizdinimą. Siūlomas 3D rekonstrukcijos modelis taip pat gali būti spausdinamas 3D spausdintuvu, o tai praplečia šios technikos panaudojimo galimybes.