SUMLOW: early design‐stage sketching of UML diagrams on an E‐whiteboard

Most visual diagramming tools provide point‐and‐click construction of computer‐drawn diagram elements using a conventional desktop computer and mouse. SUMLOW is a unified modelling language (UML) diagramming tool that uses an electronic whiteboard (E‐whiteboard) and sketching‐based user interface to support collaborative software design. SUMLOW allows designers to sketch UML constructs, mixing different UML diagram elements, diagram annotations, and hand‐drawn text. A key novelty of the tool is the preservation of hand‐drawn diagrams and support for manipulation of these sketches using pen‐based actions. Sketched diagrams can be automatically ‘formalized’ into computer‐recognized and ‐drawn UML diagrams and then exported to a third party CASE tool for further extension and use. We describe the motivation for SUMLOW, illustrate the use of the tool to sketch various UML diagram types, describe its key architecture abstractions and implementation approaches, and report on two evaluations of the toolset. We hope that our experiences will be useful for others developing sketching‐based design tools or those looking to leverage pen‐based interfaces in software applications. Copyright © 2007 John Wiley & Sons, Ltd.     

Online Personalised Non‐photorealistic Rendering Technique for 3D Geometry from Incremental Sketching

This paper presents an online personalised non‐photorealistic rendering (NPR) technique for 3D models generated from interactively sketched input. This technique has been integrated into a sketch‐based modelling system. It lets users interact with computers by drawing naturally, without specifying the number, order, or direction of strokes. After sketches are interpreted as 3D objects, they can be rendered with personalised drawing styles so that the reconstructed 3D model can be presented in a sketchy style similar in appearance to what have been drawn for the 3D model. This technique captures the user's drawing style without using template or prior knowledge of the sketching style. The personalised rendering style can be applied to both visible and initially invisible geometry. The rendering strokes are intelligently selected from the input sketches and mapped to edges of the 3D object. In addition, non‐geometric information such as surface textures can be added to the recognised object in different sketching modes. This will integrate sketch‐based incremental 3D modelling and NPR into conceptual design.     

A language for representing and extracting 3D geometry semantics from paper-based sketches

Every product that exists, ranging from a toothbrush to a car, has first been conceived as a mental concept. Due to its efficacy in rapidly externalizing concepts, paper-based sketching is still extensively used by practising designers to gradually develop the three-dimensional (3D) geometric form of a concept. It is a common practice that form concepts are sketched on paper prior to generating 3D virtual models in commercial Computer-Aided Design (CAD) systems. However, the user-interface of such systems does not support automatic generation of 3D models from sketches. Furthermore, the inherent characteristics of form sketching (e.g. idiosyncrasy) pose a challenge to computer-based understanding of the form concept semantics expressed on paper. To address these issues, this paper is therefore concerned with the development of a visual language that is prescribed and to be used by product designers to annotate paper-based sketches such that the form geometry semantics can be formally represented; parsing the annotated sketch allows for the automatic generation of 3D virtual models in CAD. Inspired by re-usable 3D CAD modelling functions and the related environmental constraints and requirements, a prescribed sketching language, PSL, has been developed to annotate paper-based form sketches. The framework architecture which parses the annotated sketch and subsequently extracts the form concept semantics is described. Based on this framework, a prototype computer tool has been implemented and evaluated. Evaluation results provide a degree of evidence, first on the suitability of PSL in representing the semantics of a range of forms, and secondly on the designers׳ acceptance of taking up this annotated sketching approach in practice.     

SmartCanvas: Context‐inferred Interpretation of Sketches for Preparatory Design Studies

In early or preparatory design stages, an architect or designer sketches out rough ideas, not only about the object or structure being considered, but its relation to its spatial context. This is an iterative process, where the sketches are not only the primary means for testing and refining ideas, but also for communicating among a design team and to clients. Hence, sketching is the preferred media for artists and designers during the early stages of design, albeit with a major drawback: sketches are 2D and effects such as view perturbations or object movement are not supported, thereby inhibiting the design process. We present an interactive system that allows for the creation of a 3D abstraction of a designed space, built primarily by sketching in 2D within the context of an anchoring design or photograph. The system is progressive in the sense that the interpretations are refined as the user continues sketching. As a key technical enabler, we reformulate the sketch interpretation process as a selection optimization from a set of context‐generated canvas planes in order to retrieve a regular arrangement of planes. We demonstrate our system (available at http:/geometry.cs.ucl.ac.uk/projects/2016/smartcanvas/ ) with a wide range of sketches and design studies.     

Interactive Sketch‐Driven Image Synthesis

We present an interactive system for composing realistic images of an object under arbitrary pose and appearance specified by sketching. Our system draws inspiration from a traditional illustration workflow: The user first sketches rough ‘masses’ of the object, as ellipses, to define an initial abstract pose that can then be refined with more detailed contours as desired. The system is made robust to partial or inaccurate sketches using a reduced‐dimensionality model of pose space learnt from a labelled collection of photos. Throughout the composition process, interactive visual feedback is provided to guide the user. Finally, the user's partial or complete sketch, complemented with appearance requirements, is used to constrain the automatic synthesis of a novel, high‐quality, realistic image.     

A Survey on Sketch Based Content Creation: from the Desktop to Virtual and Augmented Reality

Sketching is one of the most natural ways for representing any object pictorially. It is however, challenging to convert sketches to 3D content that is suitable for various applications like movies, games and computer aided design. With the advent of more accessible Virtual Reality (VR) and Augmented Reality (AR) technologies, sketching can potentially become a more powerful yet easy-to-use modality for content creation. In this state-of-the-art report, we aim to present a comprehensive overview of techniques related to sketch based content creation, both on the desktop and in VR/AR. We discuss various basic concepts related to static and dynamic content creation using sketches. We provide a structured review of various aspects of content creation including model generation, coloring and texturing, and finally animation. We try to motivate the advantages that VR/AR based sketching techniques and systems can offer into making sketch based content creation a more accessible and powerful mode of expression. We also discuss and highlight various unsolved challenges that current sketch based techniques face with the goal of encouraging future research in the domain.     

线积分卷积与双色调映射相结合的彩色素描模拟方法

目的 目前学者已经设计了很多模拟油画、水彩、水墨等风格的非真实感绘制方法,而能够生成彩色素描的算法还不是很多。针对这一课题,在前人工作的基础上,结合线积分卷积与双色调映射技术,改进了一种彩色素描模拟方法。方法 首先基于K-means聚类对彩色图像进行分割,通过计算色彩差异性为每个区域指定两种基本色,并利用双色调映射技术计算每种颜色的密度。而后利用线积分卷积分别生成两个基本色层的素描纹理,并将两层纹理相融合来生成彩色纹理。与此同时,利用霓虹变换生成素描轮廓线。最后,将轮廓与彩色纹理相融合来得到彩色素描效果。结果 实验结果表明,本文方法能够实现由彩色图像到彩铅画的自动、实时转化。结论 本文方法从轮廓和纹理两个角度模拟了真实的彩铅绘画过程。基于K-means聚类的分割方法得到的结果能够更好地反映彩色图像的颜色分布特性。通过色彩差异性计算指定基本色的策略提高了该环节的效率,满足了实时性要求。由于粉笔、蜡笔等绘画风格的调色与彩铅画类似,本文不同颜色层上下叠加的方式可以扩展到对其他介质绘画的模拟当中。     

Categorical aspects of data type constructors

Sketches are introduced as presentations of many-sorted algebraic theories and data types are described as initial algebras for such sketches. A construction A⁺ofB is described where A⁺ is a suitably structured sketch and B is a sketch. In this construction each operation from A⁺ operates on all the sorts of B. Many examples are given. The two main theoretical results are the theorem about the tensor product ⊗ for structured sketches such that A⁺of(B⁺of C)≃(A⁺⊗B⁺)of C (Theorem 3.17), and the “homomorphism” theorem 4.2 which describes the operation of structured sketches on the fibred category of all models of all sketches.     

Drawing for Illustration and Annotation in 3D

We present a system for sketching in 3D, which strives to preserve the degree of expression, imagination, and simplicity of use achieved by 2D drawing. Our system directly uses user-drawn strokes to infer the sketches representing the same scene from different viewpoints, rather than attempting to reconstruct a 3D model. This is achieved by interpreting strokes as indications of a local surface silhouette or contour. Strokes thus deform and disappear progressively as we move away from the original viewpoint. They may be occluded by objects indicated by other strokes, or, in contrast, be drawn above such objects. The user draws on a plane which can be positioned explicitly or relative to other objects or strokes in the sketch. Our system is interactive, since we use fast algorithms and graphics hardware for rendering. We present applications to education, design, architecture and fashion, where 3D sketches can be used alone or as an annotation of an existing 3D model.     

计算机支持的智能草图技术研究及实现

首先论述了作为CAD技术当前研究前沿的计算机支持的智能草图技术研究现状。构建出智能草图的技术体系,包括基于单笔划的自然手势界面,徒手草图输入,草图识别与基于草图的几何建模。最后给出基于著名几何造型核心ACIS开发的智能草图工具原型系统ISID,它具有一个支持笔输入的用户界面,并能和典型的CAD系统实现集成。初步应用证明该系统具有很强的实用性和可扩展性。     

结合连续卷积算子的自适应加权目标跟踪算法

目的 在视觉跟踪领域中,特征的高效表达是鲁棒跟踪的关键,观察到在相关滤波跟踪中,不同卷积层表达了目标的不同方面特征,提出了一种结合连续卷积算子的自适应加权目标跟踪算法。方法 针对目标定位不准确的问题,提出连续卷积算子方法,将离散的位置估计转换成连续位置估计,使得位置定位更加准确;利用不同卷积层的特征表达,提高跟踪效果。首先利用深度卷积网络结构提取多层卷积特征,通过计算相关卷积响应大小,决定在下一帧特征融合时各层特征所占的权重,凸显优势特征,然后使用从不同层训练得到的相关滤波器与提取得到的特征进行相关运算,得到最终的响应图,响应图中最大值所在的位置便是目标所在的位置和尺度。结果 与目前较流行的3种目标跟踪算法在目标跟踪基准数据库（OTB-2013）中的50组视频序列进行测试,本文算法平均跟踪成功率达到85.4%。结论 本文算法在光照变化、尺度变化、背景杂波、目标旋转、遮挡和复杂环境下的跟踪具有较高的鲁棒性。     

笔式三维草图绘制中的轮廓线技术研究*

详细介绍了三维模型轮廓线技术及其检测算法,并深入分析了轮廓线技术在三维草图绘制中的作用;然后设计并实现了一个三维草图绘制系统。该系统提供了用笔绘制三维模型的交互环境,通过轮廓线将三维模型与二维交互手势联系起来,使用户可以自由灵活地操纵和控制三维模型,获得自然、高效的交互体验。     

Sketching asynchronous data streams over sliding windows

We study the problem of maintaining a sketch of recent elements of a data stream. Motivated by applications involving network data, we consider streams that are asynchronous, in which the observed order of data is not the same as the time order in which the data was generated. The notion of recent elements of a stream is modeled by the sliding timestamp window, which is the set of elements with timestamps that are close to the current time. We design algorithms for maintaining sketches of all elements within the sliding timestamp window that can give provably accurate estimates of two basic aggregates, the sum and the median, of a stream of numbers. The space taken by the sketches, the time needed for querying the sketch, and the time for inserting new elements into the sketch are all polylogarithmic with respect to the maximum window size. Our sketches can be easily combined in a lossless and compact way, making them useful for distributed computations over data streams. Previous works on sketching recent elements of a data stream have all considered the more restrictive scenario of synchronous streams, where the observed order of data is the same as the time order in which the data was generated. Our notion of recency of elements is more general than that studied in previous work, and thus our sketches are more robust to network delays and asynchrony. The work of the authors was supported in part through NSF grants CNS 0520102 and CNS 0520009. A preliminary version of this paper appeared in Proceedings of the ACM Symposium on Principles of Distributed Computing (PODC) 2006, pages 82–91. Work done while the third author was at Rensselaer Polytechnic Institute. Authors are listed in reverse alphabetical order.     

The eternal sunshine of the sketch data structure

In the past years there has been significant research on developing compact data structures for summarizing large data streams. A family of such data structures is the so-called sketches. Sketches bear similarities to the well-known Bloom filters [B.H. Bloom, Space/time trade-offs in hash coding with allowable errors, Communications of ACM, 13 (7) (1970), 422–426] and employ hashing techniques to approximate the count associated with an arbitrary key in a data stream using fixed memory resources. One limitation of sketches is that when used for summarizing long data streams, they gradually saturate, resulting in a potentially large error on estimated key counts. In this work, we introduce two techniques to address this problem based on the observation that real-world data streams often have many transient keys that appear for short time periods and do not re-appear later on. After entering the data structure, these keys contribute to hashing collisions and thus reduce the estimation accuracy of sketches. Our techniques use a limited amount of additional memory to detect transient keys and to periodically remove their hashed values from the sketch. In this manner the number of keys hashed into a sketch decreases, and as a result the frequency of hashing collisions and the estimation error are reduced. Our first technique in effect slows down the saturation process of a sketch, whereas our second technique completely prevents a sketch from saturating.¹ We demonstrate the performance improvements of our techniques analytically as well as experimentally. Our evaluation results using real network traffic traces show a reduction in the collision rate ranging between 26.1% and 98.2% and even higher savings in terms of estimation accuracy compared to a state-of-the-art sketch data structure. To our knowledge this is the first work to look into the problem of improving the accuracy of sketches by mitigating their saturation process.     

草绘3维人体建模的模板形变方法

目的 采用草绘交互方式直接构造3维人体模型是当前人体建模研究的重要课题之一.提出一种草绘3维人体建模的模板形变方法.方法 针对输入的草图,首先,采用关节点定位方法获取草图中的人体关节点,根据人体结构学约束识别人体骨架结构,通过解析人体轮廓草图获取人体草图特征;其次,通过骨架模板和外观轮廓模板形变,将草图特征映射到3维人体模型,实现3维人体建模.结果 草图解析方法能有效提取草图特征,通过模板形变方法生成3维人体模型,并在模型上保持草图特征;能适应不同用户的绘制习惯,且生成的3维人体模型可用于人体动画设计.结论 提出一种草绘3维人体建模的模板形变方法,支持用户采用草绘方式进行3维人体模型设计,方法具有良好的用户适应性,对3维动画创作具有重要意义.     

Volume-based large dynamic graph analysis supported by evolution provenance

We present an approach for the visualization and interactive analysis of dynamic graphs that contain a large number of time steps. A specific focus is put on the support of analyzing temporal aspects in the data. Central to our approach is a static, volumetric representation of the dynamic graph based on the concept of space-time cubes that we create by stacking the adjacency matrices of all time steps. The use of GPU-accelerated volume rendering techniques allows us to render this representation interactively. We identified four classes of analytics methods as being important for the analysis of large and complex graph data, which we discuss in detail: data views, aggregation and filtering, comparison, and evolution provenance. Implementations of the respective methods are presented in an integrated application, enabling interactive exploration and analysis of large graphs. We demonstrate the applicability, usefulness, and scalability of our approach by presenting two examples for analyzing dynamic graphs. Furthermore, we let visualization experts evaluate our analytics approach.

     

SERIAL SKETCHING: VISUAL PROBLEM SOLVING IN DESIGNING

Architectural designing is seen as a process of small-step transformations of partial images of a still nonexisting entity. A design problem is solved when a satisfactory visual representation of a design concept is produced. To deal with pictorial properties of the design concept, the designer utilizes visual thinking, which is represented through sketching. In serial sketching the designer systematically transforms images of the entity that is being designed: each sketch provides feedback that informs the generation of subsequent representations. The process is seen as an exemplar of information processing in problem solving and is investigated through cases from on-line experiments as well as examples from the architectural literature.