{"id":29946,"date":"2021-03-08T00:00:06","date_gmt":"2021-03-08T08:00:06","guid":{"rendered":"https:\/\/vermont.salk.edu\/?post_type=disclosure&#038;p=29946"},"modified":"2024-01-30T14:36:46","modified_gmt":"2024-01-30T22:36:46","slug":"new-method-could-democratize-deep-learning-enhanced-microscopy","status":"publish","type":"disclosure","link":"https:\/\/www.salk.edu\/de\/news-release\/new-method-could-democratize-deep-learning-enhanced-microscopy\/","title":{"rendered":"New method could democratize deep learning-enhanced microscopy"},"content":{"rendered":"<p>LA JOLLA\u2014Deep learning is a potential tool for scientists to glean more detail from low-resolution images in microscopy, but it\u2019s often difficult to gather enough baseline data to train computers in the process. Now, a new method developed by scientists at the Salk Institute could make the technology more accessible\u2014by taking high-resolution images, and artificially degrading them.<\/p>\n<p>The new tool, which the researchers call a &#8220;crappifier,&#8221; could make it significantly easier for scientists to get detailed images of cells or cellular structures that have previously been difficult to observe because they require low-light conditions, such as mitochondria, which can divide when stressed by the lasers used to illuminate them. It could also help democratize microscopy, allowing scientists to capture high-resolution images even if they don\u2019t have access to powerful microscopes. The findings were published March 8, 2021, in the journal <a href=\"https:\/\/www.nature.com\/articles\/s41592-021-01080-z\" rel=\"noopener\" target=\"_blank\"><em>Natur Methoden<\/em><\/a>.<\/p>\n<div class=\"row\" style=\"\"><div class=\"col-md-12 col-md-push-0\"><div class=\"video-anchor\" id=\"video-4L9yrZvFyx8\"><\/div><div class=\"embed-responsive embed-responsive-16by9\"> <iframe class=\"embed-responsive-item\" src=\"\/\/www.youtube.com\/embed\/4L9yrZvFyx8?rel=0\" webkitallowfullscreen mozallowfullscreen allowfullscreen><\/iframe><\/div><!-- .embed-responsive --><\/div><!-- .col-md-*size --><\/div><!-- .\/row --><div class=\"row\" style=\"\"><div class=\"col-md-12 col-md-push-0\" style=\"font-size: .9em; margin-bottom: 20px;\">A low-resolution time-lapse of a mitochondrial network (cell's powerhouse) inside a cancer cell (left) is enhanced using artificial intelligence (right). Inset at bottom left highlights a fission (splitting) event that is detectable in the high-resolution version (right), but not the low-resolution version (left).<br \/><a href='https:\/\/www.salk.edu\/wp-content\/uploads\/2021\/03\/Fluorescence_example.jpg'>Klicken Sie hier<\/a> for a high-resolution still image.<br \/>Salk Institute\/Waitt Advance Biophotonics Core<\/div><\/div>\n<p>\u201cWe invest millions of dollars in these microscopes, and we\u2019re still struggling to push the limits of what they can do,\u201d says <a href=\"https:\/\/www.salk.edu\/de\/scientist\/uri-manor\/\">Uri Manor, <\/a>director of the Waitt Advanced Biophotonics Core Facility at Salk. \u201cThat\u2019s the problem we were trying to solve with deep learning.\u201d<\/p>\n<p>Deep learning is a type of artificial intelligence (AI) in which computer algorithms learn and improve by studying examples. To use deep learning to improve microscope images\u2014either by improving the resolution (sharpness) or reducing background \u201cnoise\u201d\u2014the system would need to be shown many examples of both high- and low-resolution images. That\u2019s a problem, because capturing perfectly identical microscopy images in two separate exposures can be difficult and expensive. It\u2019s especially challenging when imaging living cells that might be moving around during the process.<\/p>\n<p>That\u2019s where the crappifier comes in. According to Manor, the method takes high-quality images and computationally degrades them, so that they look something like the lowest low-resolution images the team would acquire.<\/p>\n<p>Manor\u2019s team showed high-resolution images and their degraded counterparts to the deep learning software, called Point-Scanning Super-Resolution, or PSSR. After studying the degraded images, the system was able to learn how to improve images that were naturally poor quality.<\/p>\n<p>That\u2019s significant because, in the past, computer systems that learned on artificially-degraded data still struggled when presented with raw data from the real world.<\/p>\n<p>\u201cWe tried a bunch of different degradation methods, and we found one that actually works,\u201d Manor says. \u201cYou can train a model on your artificially-generated data, and it actually works on real-world data.\u201d<\/p>\n<p>&#8220;Using our method, people can benefit from this powerful, deep learning technology without investing a lot of time or resources,&#8221; says Linjing Fang, image analysis specialist at the Waitt Advanced Biophotonics Core Facility, and lead author on the paper. &#8220;You can use pre-existing high-quality data, degrade it, and train a model to improve the quality of a lower-resolution image.&#8221;<\/p>\n<p>The team showed that PSSR works in both electron microscopy and with fluorescence live cell images\u2014two situations where it can be extraordinarily difficult or impossible to obtain the duplicate high- and low-resolution images needed to train AI systems. While the study demonstrated the method on images of brain tissue, Manor hopes it could be applied to other systems of the body in the future.<\/p>\n<p>He also hopes it could someday be used to make high-resolution microscopic imaging more widely accessible. Currently, the most powerful microscopes in the world can cost upwards of a million dollars, because of the precision engineering required to create high-resolution images. \u201cOne of our visions for the future is to be able to start replacing some of those expensive components with deep learning,\u201d Manor says, \u201cSo we could start making microscopes cheaper and more accessible.\u201d<\/p>\n<p>Other authors on the study are Sammy Weiser Novak, Cara R. Schiavon, Tong Zhang and Melissa Wu of the Salk Institute; Fred Monroe of the Wicklow AI Medical Research Initiative; Lindsey Kirk and Kristen Harris of the University of Texas at Austin; Seungyoon B. Yu and Gulcin Pekkurnaz of the University of California San Diego; Kyle Kastner of the Universit\u00e9 de Montr\u00e9al, Yoshiyuki Kubota of the National Institute for Physiological Sciences, Okazaki, Japan; Zhao Zhang of the University of Texas at Austin, and Alaa Abdel Latif, Zijun Lin,, Andrew Shaw and Jeremy Howard of the University of San Francisco.<\/p>\n<p>The research was supported by the National Science Foundation, the Chan-Zuckerberg Initiative, the Waitt Foundation, the National Cancer Institute, the National Institute on Deafness and Other Communication Disorders, the National Institute of Mental Health, the Wicklow AI Medical Research Initiative, the Parkinson\u2019s Foundation, the National Institute of Health, and the Japan Society for the Promotion of Science.<\/p>","protected":false},"featured_media":30051,"template":"","faculty":[405],"disease-research":[332],"class_list":["post-29946","disclosure","type-disclosure","status-publish","has-post-thumbnail","hentry","faculty-uri-manor","disease-research-computational-biology"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.3 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>New method could democratize deep learning-enhanced microscopy - Salk Institute for Biological Studies<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.salk.edu\/de\/news-release\/new-method-could-democratize-deep-learning-enhanced-microscopy\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"New method could democratize deep learning-enhanced microscopy - Salk Institute for Biological Studies\" \/>\n<meta property=\"og:description\" content=\"LA JOLLA\u2014Deep learning is a potential tool for scientists to glean more detail from low-resolution images in microscopy, but it\u2019s often difficult to gather enough baseline data to train computers in the process. Now, a new method developed by scientists at the Salk Institute could make the technology more accessible\u2014by taking high-resolution images, and artificially degrading them.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.salk.edu\/de\/news-release\/new-method-could-democratize-deep-learning-enhanced-microscopy\/\" \/>\n<meta property=\"og:site_name\" content=\"Salk Institute for Biological Studies\" \/>\n<meta property=\"article:modified_time\" content=\"2024-01-30T22:36:46+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.salk.edu\/wp-content\/uploads\/2021\/02\/Fluorescence_example-767.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"767\" \/>\n\t<meta property=\"og:image:height\" content=\"768\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.salk.edu\\\/news-release\\\/new-method-could-democratize-deep-learning-enhanced-microscopy\\\/\",\"url\":\"https:\\\/\\\/www.salk.edu\\\/news-release\\\/new-method-could-democratize-deep-learning-enhanced-microscopy\\\/\",\"name\":\"New method could democratize deep learning-enhanced microscopy - Salk Institute for Biological Studies\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.salk.edu\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.salk.edu\\\/news-release\\\/new-method-could-democratize-deep-learning-enhanced-microscopy\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.salk.edu\\\/news-release\\\/new-method-could-democratize-deep-learning-enhanced-microscopy\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.salk.edu\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/Fluorescence_example-767.jpg\",\"datePublished\":\"2021-03-08T08:00:06+00:00\",\"dateModified\":\"2024-01-30T22:36:46+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.salk.edu\\\/news-release\\\/new-method-could-democratize-deep-learning-enhanced-microscopy\\\/#breadcrumb\"},\"inLanguage\":\"de-DE\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.salk.edu\\\/news-release\\\/new-method-could-democratize-deep-learning-enhanced-microscopy\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"de-DE\",\"@id\":\"https:\\\/\\\/www.salk.edu\\\/news-release\\\/new-method-could-democratize-deep-learning-enhanced-microscopy\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.salk.edu\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/Fluorescence_example-767.jpg\",\"contentUrl\":\"https:\\\/\\\/www.salk.edu\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/Fluorescence_example-767.jpg\",\"width\":767,\"height\":768},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.salk.edu\\\/news-release\\\/new-method-could-democratize-deep-learning-enhanced-microscopy\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.salk.edu\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"New method could democratize deep learning-enhanced microscopy\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.salk.edu\\\/#website\",\"url\":\"https:\\\/\\\/www.salk.edu\\\/\",\"name\":\"Salk Institute for Biological Studies\",\"description\":\"The Power of Science\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.salk.edu\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.salk.edu\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de-DE\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.salk.edu\\\/#organization\",\"name\":\"Salk Institute for Biological Studies\",\"url\":\"https:\\\/\\\/www.salk.edu\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de-DE\",\"@id\":\"https:\\\/\\\/www.salk.edu\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"http:\\\/\\\/www.salk.edu\\\/wp-content\\\/uploads\\\/2023\\\/03\\\/salk_logo_696.jpg\",\"contentUrl\":\"http:\\\/\\\/www.salk.edu\\\/wp-content\\\/uploads\\\/2023\\\/03\\\/salk_logo_696.jpg\",\"width\":696,\"height\":696,\"caption\":\"Salk Institute for Biological Studies\"},\"image\":{\"@id\":\"https:\\\/\\\/www.salk.edu\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"New method could democratize deep learning-enhanced microscopy - Salk Institute for Biological Studies","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.salk.edu\/de\/news-release\/new-method-could-democratize-deep-learning-enhanced-microscopy\/","og_locale":"de_DE","og_type":"article","og_title":"New method could democratize deep learning-enhanced microscopy - Salk Institute for Biological Studies","og_description":"LA JOLLA\u2014Deep learning is a potential tool for scientists to glean more detail from low-resolution images in microscopy, but it\u2019s often difficult to gather enough baseline data to train computers in the process. Now, a new method developed by scientists at the Salk Institute could make the technology more accessible\u2014by taking high-resolution images, and artificially degrading them.","og_url":"https:\/\/www.salk.edu\/de\/news-release\/new-method-could-democratize-deep-learning-enhanced-microscopy\/","og_site_name":"Salk Institute for Biological Studies","article_modified_time":"2024-01-30T22:36:46+00:00","og_image":[{"width":767,"height":768,"url":"https:\/\/www.salk.edu\/wp-content\/uploads\/2021\/02\/Fluorescence_example-767.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.salk.edu\/news-release\/new-method-could-democratize-deep-learning-enhanced-microscopy\/","url":"https:\/\/www.salk.edu\/news-release\/new-method-could-democratize-deep-learning-enhanced-microscopy\/","name":"New method could democratize deep learning-enhanced microscopy - Salk Institute for Biological Studies","isPartOf":{"@id":"https:\/\/www.salk.edu\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.salk.edu\/news-release\/new-method-could-democratize-deep-learning-enhanced-microscopy\/#primaryimage"},"image":{"@id":"https:\/\/www.salk.edu\/news-release\/new-method-could-democratize-deep-learning-enhanced-microscopy\/#primaryimage"},"thumbnailUrl":"https:\/\/www.salk.edu\/wp-content\/uploads\/2021\/02\/Fluorescence_example-767.jpg","datePublished":"2021-03-08T08:00:06+00:00","dateModified":"2024-01-30T22:36:46+00:00","breadcrumb":{"@id":"https:\/\/www.salk.edu\/news-release\/new-method-could-democratize-deep-learning-enhanced-microscopy\/#breadcrumb"},"inLanguage":"de-DE","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.salk.edu\/news-release\/new-method-could-democratize-deep-learning-enhanced-microscopy\/"]}]},{"@type":"ImageObject","inLanguage":"de-DE","@id":"https:\/\/www.salk.edu\/news-release\/new-method-could-democratize-deep-learning-enhanced-microscopy\/#primaryimage","url":"https:\/\/www.salk.edu\/wp-content\/uploads\/2021\/02\/Fluorescence_example-767.jpg","contentUrl":"https:\/\/www.salk.edu\/wp-content\/uploads\/2021\/02\/Fluorescence_example-767.jpg","width":767,"height":768},{"@type":"BreadcrumbList","@id":"https:\/\/www.salk.edu\/news-release\/new-method-could-democratize-deep-learning-enhanced-microscopy\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.salk.edu\/"},{"@type":"ListItem","position":2,"name":"New method could democratize deep learning-enhanced microscopy"}]},{"@type":"WebSite","@id":"https:\/\/www.salk.edu\/#website","url":"https:\/\/www.salk.edu\/","name":"Salk-Institut f\u00fcr biologische Studien","description":"Die Macht der Wissenschaft","publisher":{"@id":"https:\/\/www.salk.edu\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.salk.edu\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de-DE"},{"@type":"Organization","@id":"https:\/\/www.salk.edu\/#organization","name":"Salk-Institut f\u00fcr biologische Studien","url":"https:\/\/www.salk.edu\/","logo":{"@type":"ImageObject","inLanguage":"de-DE","@id":"https:\/\/www.salk.edu\/#\/schema\/logo\/image\/","url":"http:\/\/www.salk.edu\/wp-content\/uploads\/2023\/03\/salk_logo_696.jpg","contentUrl":"http:\/\/www.salk.edu\/wp-content\/uploads\/2023\/03\/salk_logo_696.jpg","width":696,"height":696,"caption":"Salk Institute for Biological Studies"},"image":{"@id":"https:\/\/www.salk.edu\/#\/schema\/logo\/image\/"}}]}},"ACF":{"hero":"https:\/\/www.salk.edu\/wp-content\/uploads\/2021\/03\/Fluorescence_example-header-2.jpg","line_1":"New method could democratize deep learning-enhanced microscopy","line_2":"Salk researchers teach AI to improve microscope image quality\u2014by taking high-resolution images and artificially degrading them","poster_quote":"","paper_url":"https:\/\/www.nature.com\/articles\/s41592-021-01080-z","journal_title":"Nature Methods","paper_author_list":"Linjing Fang, Fred Monroe, Sammy Weiser Novak, Lyndsey Kirk, Cara R. Schiavon, Seungyoon B. Yu, Tong Zhang, Melissa Wu, Kyle Kastner, Alaa Abdel Latif, Zijun Lin, Andrew Shaw, Yoshiyuki Kubota, John Mendenhall, Zhao Zhang, Gulcin Pekkurnaz, Kristen Harris, Jeremy Howard, Uri Manor","doi":"0.1038\/s41592-021-01080-z","paper_title":"Deep Learning-Based Point-Scanning Super-Resolution Imaging","subhead":"Salk researchers teach AI to improve microscope image quality\u2014by taking high-resolution images and artificially degrading them","home_photo":"","listing_photo":"","legacy_boilerplate":[],"hide_boilerplate":[],"disable_date":false,"listing_excerpt":"","descriptive_blurb":"","has_journal_cover":false,"og_image_override":false,"gallery":false},"_links":{"self":[{"href":"https:\/\/www.salk.edu\/de\/wp-json\/wp\/v2\/disclosure\/29946","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.salk.edu\/de\/wp-json\/wp\/v2\/disclosure"}],"about":[{"href":"https:\/\/www.salk.edu\/de\/wp-json\/wp\/v2\/types\/disclosure"}],"version-history":[{"count":20,"href":"https:\/\/www.salk.edu\/de\/wp-json\/wp\/v2\/disclosure\/29946\/revisions"}],"predecessor-version":[{"id":30081,"href":"https:\/\/www.salk.edu\/de\/wp-json\/wp\/v2\/disclosure\/29946\/revisions\/30081"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.salk.edu\/de\/wp-json\/wp\/v2\/media\/30051"}],"wp:attachment":[{"href":"https:\/\/www.salk.edu\/de\/wp-json\/wp\/v2\/media?parent=29946"}],"wp:term":[{"taxonomy":"faculty","embeddable":true,"href":"https:\/\/www.salk.edu\/de\/wp-json\/wp\/v2\/faculty?post=29946"},{"taxonomy":"disease-research","embeddable":true,"href":"https:\/\/www.salk.edu\/de\/wp-json\/wp\/v2\/disease-research?post=29946"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}