基于策略爬行与混合索引的医药行业垂直搜索引擎的设计与实现

随着互联网应用的深入,越来越多的用户希望通过搜索引擎获得特定行业的相关信息,通用搜索引擎无法有效地满足相应需求。文中主要介绍医药行业垂直搜索引擎的设计与实现。设计基于智能搜索引擎的架构,采用了任务驱动的聚焦搜索、隐藏搜索技术;字词混合倒排索引及优化的字倒排索引、检索技术。提供了资源收集阶段的可控策略爬行,和高效的索引、检索功能。实现了针对医药行业的高专业度、高准确率、高效率的信息垂直搜索。     

基于Web的DCI垂直搜索引擎的研究与设计

为了解决用户能够快速、准确的搜索互联网上数字作品信息的问题,分析设计了一个对数字作品版权唯一标识符(Digital Copyright Identifier简称DCI)数字作品的垂直搜索引擎.首先基于Heritrix网络爬虫技术,对互联网上的数字作品进行数据采集和正文信息抽取,并将抽取的数据保存到本地;然后基于Lucene的全文检索工具包,对本地数据进行分词、倒排索引、索引检索和改进的相关度排序等处理,最终设计实现了一个通用可扩展的DCI垂直搜索引擎.实验结果表明,该搜索引擎在很大程度上提高了网页信息抽取的准确度和数据的检索效率.     

搜索引擎分块索引技术研究

搜索引擎通常使用签名文件、倒排索引、后缀树/后缀数组来构建索引,其中倒排文件由于性能可靠、实现简单,得到广泛应用.但是在现今海量数据情况下,有时倒排索引仍显力不从心,导致检索性能不佳,提出了分块索引术,可以有效提高基于倒排索引搜索系统检索效率.     

基于XML文档的藏文网页倒排索引的研究与实现

如今互联网上藏文信息也不断的扩充,藏文搜索引擎作为常用的信息检索的工具和渠道,倒排索引又是搜索引擎的核心技术之一,倒排索引直接影响搜索引擎检索的结果和响应的速度。之所以文章详细介绍了一个自主开发的藏文网页倒排索引系统,它以XML文档的标签内容作为索引对象,定义了文档和文档属性等概念,采用C#语言对文藏文网页正文构建倒排索引的关键技术和实现方法进一步的阐述,实现了基于XML文档的藏文网页倒排索引数据库的底层实现,提供了技术参考。利用这种方法藏文搜索引擎中信息检索的速度和准确率有所提高。     

杏林搜索:中医药垂直搜索引擎的设计与实现

随着中医药行业的快速发展,中医药相关数字化资源急剧增加,传统的全文搜索引擎因专业性不够强、内容繁多等问题导致用户无法快速方便地找到所需信息.为此,设计并实现了一款面向中医药领域的垂直搜索引擎具有重要的意义和价值.采用Heritrix与Lucene等技术构建中医药垂直搜索引擎——杏林搜索,重点介绍了杏林搜索的设计与实现,包括网络爬虫模块、网页解析模块、索引库和Web端等.实际应用效果表明,杏林搜索可以为用户提供更加快速、专业的中医药资源检索服务.     

分布式FTP并行搜索引擎的设计与实现

构建了搜索引擎系统结构,实现了在FTP文件服务器上所需资料的快速查找。设计和实现了搜索引擎的核心部分即数据搜索器、索引器、检索器、引擎控制台以及Web搜索界面五个模块。数据搜索器的分布并行多线程机制,实现了多台PC分布搜索服务策略;同时每台PC设置多线程搜索,多次尝试和断点续搜功能,保证了在网络不稳定的情况下也能够完成搜索、建库任务。索引器和检索器的双字母倒排索引机制对提高建库、索引和查找速度,效果非常明显。     

FileFinder:桌面搜索引擎的设计与实现

为了满足用户对桌面搜索引擎准确、快速、易用的迫切要求,将通用搜索技术、垂直搜索技术中“先建立索引再进行搜索”的搜索机制和若干功能借鉴到桌面搜索技术,设计并实现了桌面搜索引擎FileFinder.功能测试验证了该软件不仅实现了关键字搜索、文件类型搜索等常用功能,而且具有结果匹配度排序、在搜索结果中再次搜索、以文件附加属性进行搜索等其他桌面搜索引擎不具备而用户需要的功能.对比测试结果既体现了FileFinder在搜索速度上与Windows桌面搜索引擎相比的优势,也表明在不实现全文检索功能的前提下,FileFinder建立索引的时间仅约为Google桌面搜索引擎的一半.     

基于倒排索引的可搜索加密数据共享方案

针对当前医疗数据共享难、数据被攻击、密文搜索效率低的问题，提出了一种基于倒排索引的可搜索加密数据共享方案。采用基于私有链和联盟链的双链结构存储数据并实现数据共享；设计了一种新型倒排索引结构，防止敏感数据被攻击；提出了一种基于新型倒排索引结构的密文搜索算法，利用可搜索加密技术将陷门信息提交至倒排索引结构中实现密文搜索。实验表明，所提出的方案可以有效保证数据安全，提高搜索效率。     

探析监管信息搜索系统的设计与实现

信息时代将信息技术应用到监管领域具有广阔前景。监管敏感信息搜索系统设计可以实现信息的检索、数据挖掘、信息采集以及定位。本文通过对敏感信息搜索系统的设计,应用搜索引擎技术、检索技术、数据挖掘技术等,实现基本功能。     

基于倒排列表的网流索引检索与压缩方法

随着计算机的广泛应用以及互联网的飞速发展,互联网流量呈现爆炸式增长的态势。为了应对日益严重的网络滥用以及网络安全事件,出于安全取证的需要,必须对互联网流量进行收集、存储和分析。互联网流量的监控需要及时统计网络流量的源地址、目的地址、源端口、目的端口、协议、时间戳等信息,以便进行流量统计和综合分析。但是网络流量信息是海量的,如何快速检索相关流量是一个挑战性问题。在搜索引擎中,为了处理海量数据检索,倒排索引是快速搜索技术的关键方法。文章把搜索引擎中的倒排索引方法和索引压缩算法应用到互联网网流信息检索中。通过实验测试和验证,在网流信息检索中,倒排索引以及索引压缩算法能够有效提高检索速度。     

Web网页并行爬行研究*

首先分析Web环境对并行爬行的需求,然后对并行爬行带来的优势和面临的问题进行探讨,进一步对并行爬行模式和策略展开深入研究。     

Development of a biomimetic miniature robotic crawler

The paper presents the development of segmented artificial crawlers endowed with passive hook-shaped frictional microstructures. The goal is to find design rules for fabricating biomimetic, adaptable and mobile machines mimicking segmented animals with hydrostatic skeleton, and intended to move effectively along unstructured substrates. The paper describes the mechanical model, the design and the fabrication of a SMA-actuated segmented microrobot, whose locomotion is inspired by the peristaltic motion of Annelids, and in particular of earthworms (Lumbricus Terrestris). Experimental locomotion performance are compared with theoretical performance predicted by a purposely developed friction model -taking into account design parameters such as number of segments, body mass, special friction enhancement appendixes—and with locomotion performance of real earthworms as presented in literature. Experiments indicate that the maximum speed of the crawler prototype is 2.5 mm/s, and that 3-segment crawlers have almost the same velocity as earthworms having the same weight (and about 330% their length), whereas 4-segment crawlers have the same velocity, expressed as body lengths/s, as earthworms with the same mass (and about 270% their length). Arianna Menciassi (MS, 1995; PhD, 1999) joined the CRIM Lab of the Scuola Superiore Sant’Anna (Pisa, Italy) as a Ph.D. student in Bioengineering with a research program on the micromanipulation of mechanical and biological micro-objects. The main results of the activity on micromanipulation were presented at the IEEE International Conference on Robotics & Automation (May 2001, Seoul) in a paper titled “Force Feedback-based Microinstrument for Measuring Tissue Properties and Pulse in Microsurgery”, which won the “ICRA2001 Best Manipulation Paper Award”. In the year 2000, she was offered a position of Assistant Professor in Biomedical Robotics at the Scuola Superiore Sant’Anna and in June 2006 she obtained a promotion to Associate Professor. Her main research interests are in the field of biomedical microrobotics, biomimetics, microfabrication technologies, micromechatronics and microsystem technologies. She is working on several European projects and international projects for the development of minimally invasive instrumentation for medical applications and for the exploitation of micro- and nano-technologies in the medical field. Samuele Gorini received his Laurea Degree in Mechanical Engineering (with honors) from the University of Pisa, Italy, in 2001. In 2005 he obtained the Ph.D. in Microsystem Engineering with a thesis on locomotion methods and systems for miniaturised endoscopic devices. Since 2000, he has been working at the CRIM Lab of the Scuola Superiore Sant’Anna in Pisa, Italy. His research interests are in the field of biomedical robotics with a special focus on actuation technologies. Starting from the year 2004 he has been president of Era Endoscopy S.r.l., a start-up company of Scuola Superiore Sant’Anna developing novel devices for endoscopy. Dino Accoto (MS 1998, PhD 2002) is Assistant Professor of Biomedical Engineering at Scuola Sant’Anna (Pisa, Italy). He received the Laurea degree in Mechanical Engineering from the University of Pisa (cum laude) in 1998, the diploma in Engineering from the Scuola Sant’Anna (cum laude) in 1999 and the PhD degree from the Scuola Sant’Anna in 2002. From October 2001 to September 2002 he has been visiting scholar at the RPL-Lab, Stanford University (Ca, USA). Since 2004 he cooperates with the Biomedical Robotics & EMC Lab at Campus Bio-Medico University in Rome. His main research field is the modelling and development of small electromechanical systems, with a special attention to multi-physics and multi-domain approaches. The research, often inspired by the analysis of natural mechanisms, has been mainly applied to hybridizing microtechnologies, including microfluidics, and robotics. He has co-authored more than 30 papers, appeared in international journals and conference proceedings. Paolo Dario received his Dr. Eng. Degree in Mechanical Engineering from the University of Pisa, Italy, in 1977. He is currently a Professor of Biomedical Robotics at the Scuola Superiore Sant’Anna in Pisa.. He also teaches courses at the School of Engineering of the University of Pisa and at the Campus Biomedico University in Rome. He has been Visiting Professor at Brown University, Providence, RI, USA, at the Ecole Polytechnique Federale de Lausanne (EPFL), Lausanne, Switzerland, at Waseda University, Tokyo, Japan, at the College de France, Paris, and at the Ecole Normale Superieure de Cachan, France. He was the founder of the ARTS (Advanced Robotics Technologies and Systems) Laboratory and is currently the Co-ordinator of the CRIM (Center for the Research in Microengineering) Laboratory of the Scuola Superiore Sant’Anna, where he supervises a team of about 70 researchers and Ph.D. students. His main research interests are in the fields of medical robotics, bio-robotics, mechatronics and micro/nanoengineering, and specifically in sensors and actuators for the above applications, and in robotics for rehabilitation. He is the coordinator of many national and European projects, the editor of two books on the subject of robotics, and the author of more than 200 scientific papers (75 on ISI journals). He is Editor-in-Chief, Associate Editor and member of the Editorial Board of many international journals. Prof. Dario has served as President of the IEEE Robotics and Automation Society in the years 2002–2003. He has been the General Chair of the IEEE RAS-EMBS BioRob’06 Conference and he is the General Co-Chair of ICRA 2007 Conference. Prof. Dario is an IEEE Fellow, a Fellow of the European Society on Medical and Biological Engineering, and a recipient of many honors and awards, such as the Joseph Engelberger Award. He is also a member of the Board of the International Foundation of Robotics Research (IFRR).     

蠕动式管内移动机构的一种模型

本文研制开发了一种蠕动式管内移动机器人机构，该机构克服了轮式、覆带式管内移动机构牵引力与附着力之间的矛盾，可以提高机构的输出牵引力，并且可以顺利通过变直径管道。     

基于主题关联联想的Web知识发现

提出了一种新的面向主题的关联挖掘和联想算法，并提出了主题关联联想概念以及挖掘主题的间接关联关系。实验表明，该算法能有效地实现Web主题知识的自动获取和自学习。     

针对主题挖掘的通用设计与性能评估框架的研究

本文不是设计一种新的主题挖掘机,而是借助分层思想,为每层赋予特定的任务,采用容器管理机制,提出了一种针对主题挖掘的通用设计框架,该设计框架旨在规范和指导主题挖掘机的研发设计步骤;同时提供了相应的评估框架,其中包括几个重要的评估参数,根据这些参数的实验值可以估量挖掘机的性能,以此推动高效挖掘机的研发设计。     

基于SVM的主题爬虫技术研究

随着互联网的快速发展,网络信息呈现海量和多元化的趋势。如何为互联网用户快速、准确地提取其所需信息,已成为搜索引擎面临的首要问题。传统的通用搜索引擎虽然能够在较大的信息范围内获取目标,但在某些特定领域无法给用户提供专业而深入的信息。提出基于SVM分类的主题爬虫技术,其将基于文字内容和部分链接信息的主题相关度预测算法、SVM分类算法和HITS算法相结合,解决了特定信息检索的难题。实验结果表明,使用基于SVM分类算法的爬取策略,能够较好地区分主题相关网页和不相关网页,提高了主题相关网页的收获率和召回率,进而提高了搜索引擎的检索效率。     

定题搜索引擎Robot的设计与算法

定题搜索引擎将信息检索限定在特定主题领域，提供特定主题的信息检索服务，是新一代搜索引擎的发展方向之一。该文介绍了一个定题搜索robot系统NetBat 2．02版，它可以实现在web上爬行下载主题相关网页。定题搜索的关键技术是主题相关信息的搜索及网页相关度分析。该文分析了传统定题搜索算法的优缺点，提出了基于反向链接结合超链文本分析的定题搜索算法。文章还对基于内容的网页相关度分析算法进行了详细的论述。对比搜索实验表明系统有着较好的性能，能准确地爬行到主题相关网页。     

Web本体搜索中链接评价方法的设计与实现

基于对本体搜索的分析,本文从多个方面对聚焦搜索的链接评价方法进行改进,同时介绍了新方法实现中的多个关键步骤.     

一种新的主题爬行算法

爬虫是搜索引擎的关键组成部分，本文提出了一种可利用之前爬行数据自动改进其分析算法与种子URL集合的主题爬行算法，并讨论了其首次爬行和再次爬行算法。实验结果表明该算法的准确率在大多数情况下优于基于宽度策略、基于PageRank和基于内容相似度分析的爬虫。