Research Initiation Award: Integrating Image and Text Information for Biomedical Literature-Based Cross and Multimodal Retrieval

研究启动奖：基于图像和文本信息的生物医学文献交叉和多模态检索整合

• 批准号: 1601044
• 负责人: Md Rahman
• 金额: $ 29.78万
• 依托单位: Morgan State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-06-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1601044&HistoricalAwards=false
• 关键词: Research; Initiation; Award; Integrating; Image

The Historically Black Colleges and Universities-Undergraduate Program (HBCU-UP) Research Initiation Awards (RIAs) provide support to STEM junior faculty at HBCUs who are starting to build a research program, as well as for mid-career faculty who may have returned to the faculty ranks after holding an administrative post or who needs to redirect and rebuild a research program. Faculty members may pursue research at their home institution, at an NSF-funded Center, at a research intensive institution or at a national laboratory. The RIA projects are expected to help further the faculty member's research capability and effectiveness, to improve research and teaching at his or her home institution, and to involve undergraduate students in research experiences. With support from the National Science Foundation, Morgan State University will conduct research in information retrieval using search strategies based on techniques from image processing as well as natural language processing. This would enable public access to both visual information and take away messages from journal articles. This project will provide valuable research experience and mentorship for several minority undergraduate students at Morgan State University. In addition, the project will help Morgan State University build its research capacity and enhance the educational and research experiences of their undergraduate students. Within the larger goal of expanding queries for information retrieval, the project will 1) use a crowdsourcing based approach to perform large scale manual annotation of visual regions of interest (ROIs) by pairing automatically detected ROIs to concepts occurring in a brief caption, 2) use a feature learning approach to extract discriminative features from ROIs and automatically map the ROIs to concepts in an existing textual ontology, such as RadLex, 3) aided by a visual ontology, consider the semantic relations between the visual words when assessing the distance between images described with the bag-of-visual-words feature representation scheme, 4) in addition to cross modal search by mapping image regions to concepts in ontology, perform multimodal search by fusing weighted text and image features generated by a multi-response linear regression (MLR)-based meta-learner in a classification-driven task-specific manner, and 5) evaluate the retrieval techniques using benchmark and realistic datasets by participating in the yearly ImageCLEF retrieval evaluation campaign. The labeled set of biomedical images with annotated regions of interest will be made available to the research community.

历史上的黑人学院和大学本科生计划（HBCU-UP）研究启动奖（RIAs）为HBCU的STEM初级教师提供支持，他们开始建立研究计划，以及职业生涯中期的教师，他们可能在担任行政职务后返回教师队伍，或者需要重新定向和重建研究计划。教师可以在他们的家乡机构，在NSF资助的中心，在研究密集型机构或在国家实验室进行研究。RIA项目预计将有助于进一步提高教师的研究能力和效率，改善他或她的家乡机构的研究和教学，并让本科生参与研究经验。在国家科学基金会的支持下，摩根州立大学将利用基于图像处理和自然语言处理技术的搜索策略进行信息检索研究。 这将使公众能够获得视觉信息，并从期刊文章中获取信息。 这个项目将为摩根州立大学的几名少数民族本科生提供宝贵的研究经验和指导。 此外，该项目将帮助摩根州立大学建立其研究能力，并提高其本科生的教育和研究经验。在扩展信息检索查询的更大目标内，该项目将1）使用基于众包的方法，通过将自动检测到的感兴趣的视觉区域（ROI）与简短标题中出现的概念配对来执行感兴趣的ROI的大规模手动注释，2）使用特征学习方法从ROI中提取区分特征，并将ROI自动映射到现有文本本体中的概念，例如RadLex，3）在视觉本体的帮助下，当评估用视觉词袋特征表示方案描述的图像之间的距离时，考虑视觉词之间的语义关系，4）除了通过将图像区域映射到本体中的概念的跨模态搜索之外，通过以分类驱动的任务特定方式融合由基于多响应线性回归（MLR）的元学习器生成的加权文本和图像特征来执行多模态搜索，5）通过参加每年的ImageCLEF检索评估活动，使用基准数据集和真实数据集评估检索技术。 带有注释的感兴趣区域的标记生物医学图像集将提供给研究界。