Research Initiation Award: Integrating Image and Text Information for Biomedical Literature-Based Cross and Multimodal Retrieval

历史上的黑人学院和大学本科项目（HBCU-UP）研究启动奖（RIAs）为hbcu中开始建立研究项目的STEM初级教师提供支持，也为那些在担任行政职务后可能已经回到教职员工队伍或需要重新定向和重建研究项目的职业中期教师提供支持。教师可以在自己的学校、美国国家科学基金会资助的中心、研究密集型机构或国家实验室进行研究。RIA项目将有助于进一步提高教师的研究能力和效率，改善其所在机构的研究和教学，并使本科生参与研究经验。在美国国家科学基金会的支持下，摩根州立大学将利用基于图像处理和自然语言处理技术的搜索策略开展信息检索研究。这将使公众既能获得视觉信息，又能从期刊文章中获取信息。该项目将为摩根州立大学的几名少数民族本科生提供宝贵的研究经验和指导。此外，该项目将帮助摩根州立大学建立其研究能力，并提高其本科生的教育和研究经验。在扩展信息检索查询的更大目标中，该项目将1)使用基于众包的方法，通过将自动检测到的roi与简短标题中出现的概念配对，对视觉感兴趣区域（roi）进行大规模手动注释；2)使用特征学习方法，从roi中提取区分特征，并将roi自动映射到现有文本本体（如RadLex）中的概念；3)借助于视觉本体；4)除了通过将图像区域映射到本体中的概念进行跨模态搜索外，还通过基于多响应线性回归（MLR）的元学习器以分类驱动的任务特定方式融合加权文本和图像特征来进行多模态搜索。5)通过参加年度ImageCLEF检索评估活动，使用基准和现实数据集评估检索技术。标记的生物医学图像集，加上感兴趣的注释区域，将提供给研究界。

The Historically Black Colleges and Universities-Undergraduate Program (HBCU-UP) Research Initiation Awards (RIAs) provide support to STEM junior faculty at HBCUs who are starting to build a research program, as well as for mid-career faculty who may have returned to the faculty ranks after holding an administrative post or who needs to redirect and rebuild a research program. Faculty members may pursue research at their home institution, at an NSF-funded Center, at a research intensive institution or at a national laboratory. The RIA projects are expected to help further the faculty member's research capability and effectiveness, to improve research and teaching at his or her home institution, and to involve undergraduate students in research experiences. With support from the National Science Foundation, Morgan State University will conduct research in information retrieval using search strategies based on techniques from image processing as well as natural language processing. This would enable public access to both visual information and take away messages from journal articles. This project will provide valuable research experience and mentorship for several minority undergraduate students at Morgan State University. In addition, the project will help Morgan State University build its research capacity and enhance the educational and research experiences of their undergraduate students. Within the larger goal of expanding queries for information retrieval, the project will 1) use a crowdsourcing based approach to perform large scale manual annotation of visual regions of interest (ROIs) by pairing automatically detected ROIs to concepts occurring in a brief caption, 2) use a feature learning approach to extract discriminative features from ROIs and automatically map the ROIs to concepts in an existing textual ontology, such as RadLex, 3) aided by a visual ontology, consider the semantic relations between the visual words when assessing the distance between images described with the bag-of-visual-words feature representation scheme, 4) in addition to cross modal search by mapping image regions to concepts in ontology, perform multimodal search by fusing weighted text and image features generated by a multi-response linear regression (MLR)-based meta-learner in a classification-driven task-specific manner, and 5) evaluate the retrieval techniques using benchmark and realistic datasets by participating in the yearly ImageCLEF retrieval evaluation campaign. The labeled set of biomedical images with annotated regions of interest will be made available to the research community.