Recently, transforming graphical user interface (GUI) mockups into code becomes a common challenging practice for current software developers. However, this transformation usually takes time ...especially when GUI changes keep pace with evolutionary features. There are many studies admitted this challenge and presented solutions in terms of computer-based GUI mockups. However, there is a research gap in this kind of research as very few of them adopted hand-drawn mockups as an input. In this study, the authors employed YOLOv5 is a fast and accurate deep learning framework to automate the process of converting hand-drawn GUI mockups into Android-based GUI prototype. The process starts with detecting all GUI mockups in an input image and determining their bounding boxes, classifying these mockups into their corresponding GUI objects, then finally aligning these objects together to form the output prototype based on the layout presented in the input image. Experimental results show the effectiveness of the proposed approach in generating a visually appealing Android GUI from hand-drawn mockups with a recognition accuracy of 98.54% when tested on various hand-drawn GUI structures designed by five developers.
We present Lyra, an interactive environment for designing customized visualizations without writing code. Using drag‐and‐drop interactions, designers can bind data to the properties of graphical ...marks to author expressive visualization designs. Marks can be moved, rotated and resized using handles; relatively positioned using connectors; and parameterized by data fields using property drop zones. Lyra also provides a data pipeline interface for iterative, visual specification of data transformations and layout algorithms. Visualizations created with Lyra are represented as specifications in Vega, a declarative visualization grammar that enables sharing and reuse. We evaluate Lyra's expressivity and accessibility through diverse examples and studies with journalists and visualization designers. We find that Lyra enables users to rapidly develop customized visualizations, covering a design space comparable to existing programming‐based tools.
In the past decades, H2 has attracted significant attention as a potentially low, zero, or negative-emissions fuel depending on how it is produced. However, how H2 will evolve in terms of its ...production, demand, and transport is not very clear. To help fill this gap, we developed a Python-based tool called the Hydrogen Economy Assessment & Resource Tool (HEART), specifically focused on the Association of Southeast Asian Nations (ASEAN) region. The tool consists of three parts: a user-friendly GUI, a default database consisting of costs and emissions for various supply/demand pathways, and an analysis and optimization engine. Users can use the default data and/or user inputs to run case studies for insightful results. The analysis and optimization engine can carry out techno-enviro-economic analysis for H2 production and H2 transport, project H2 demand, and carry out the optimization and planning of H2 supply chain over a long-term period. In this paper, we introduce the framework of HEART, the workflow of various projects, and show the capabilities and strengths of HEART through a case study. The case study indicates the tool can provides many useful results, like the landed cost and carbon footprint of H2. Most importantly, by using HEART, H2 supply chain pathways to meet projected demand can be obtained and optimized for total cost or total emissions.
Display omitted
•A Python-based tool is developed for hydrogen economy planning and optimization.•The framework, workflow, and capabilities of HEART is introduced.•Hydrogen supply chains for multiple sectors are planned for an ASEAN case study.
Data visualization is now a popular medium for journalistic storytelling. However, current visualization tools either lack support for storytelling or require significant technical expertise. ...Informed by interviews with journalists, we introduce a model of storytelling ions that includes state‐based scene structure, dynamic annotations and decoupled coordination of multiple visualization components. We instantiate our model in Ellipsis: a system that combines a domain‐specific language (DSL) for storytelling with a graphical interface for story authoring. User interactions are automatically translated into statements in the Ellipsis DSL. By enabling storytelling without programming, the Ellipsis interface lowers the threshold for authoring narrative visualizations. We evaluate Ellipsis through example applications and user studies with award‐winning journalists. Study participants find Ellipsis to be a valuable prototyping tool that can empower journalists in the creation of interactive narratives.
VRML (Virtual Reality Modeling Language) is a nonproprietary language used for scientific simulation and visualization. VRML alone do not have the capability of holding numerical values but with the ...help of slider design, the numeric values can be dynamically varied. The VRML Graphic User Interface (GUI) presented in this paper demonstrates dynamic simulation of Seed Grader Machine Parameters and its effect on final output. This paper describes how VRML capabilities can be utilized for simulating Seed Grader Machine Parameters in virtual Environment. The concept can be applied for variety of applications involving similar requirements.
An animation method for creating 3D motion effects from a single realistic, fluid-like 2D image/photo is developed. Several 3D basic velocity patterns with 3D Navier Stokes equations are designed to ...provide a timedependent motion for a 2D image. Their velocity patterns are transformed onto a 2D image plane using perspective transformation and a 2D advection equation is used to generate a new image sequence. Two GUIs are also developed to allow quick motion generation by mouse, integrated with image-matting and image in-painting methods. For continuing textural changes, source and sink regions can be defined and used as well. Experiments with 3D visual effects using single images, such as a cloud and a waterfall, are described. Results show that our proposed image generation method and GUI have quick operability with less distorted textures.
Rapid GUI prototyping has evolved into a widely applied technique in early stages of software development to facilitate the clarification and refinement of requirements. Especially high-fidelity GUI ...prototyping has shown to enable productive discussions with customers and mitigate potential misunderstandings, however, the benefits of applying high-fidelity GUI prototypes are accompanied by the disadvantage of being expensive and time-consuming in development and requiring experience to create. In this work, we show
RaWi
, a data-driven GUI prototyping approach that effectively retrieves GUIs for reuse from a large-scale semi-automatically created GUI repository for mobile apps on the basis of Natural Language (NL) searches to facilitate GUI prototyping and improve its productivity by leveraging the vast GUI prototyping knowledge embodied in the repository. Retrieved GUIs can directly be reused and adapted in the graphical editor of
RaWi
. Moreover, we present a comprehensive evaluation methodology to enable (i) the systematic evaluation of NL-based GUI ranking methods through a novel high-quality gold standard and conduct an in-depth evaluation of traditional IR and state-of-the-art BERT-based models for GUI ranking, and (ii) the assessment of GUI prototyping productivity accompanied by an extensive user study in a practical GUI prototyping environment.
Currently, the rapid growth of mobile apps requires automated testing technology to ensure their quality. Automated testing of mobile apps is usually closely related to the recognition and judgment ...of their graphical user interface (GUI), but there usually are numerous isomorphic GUIs with different styles and contents, and similar structure and function in mobile apps. In automatic testing, isomorphic GUI is easy to cause the issue of state space explosion, which leads to low efficiency or failure of testing. In view of the limitations of traditional automatic recognition of isomorphic GUI, this paper presents a GUI similarity judgment approach based on visual feature information. Firstly, the GUI component elements are identified by object detection, and then the GUI skeleton is abstracted. Secondly, the visual features of the GUI skeleton are extracted by a convolutional autoencoder. Finally, the isomorphic GUI judgment is completed by comparing the similarity of GUI visual features. The experimental results show that the proposed approach can effectively shield the influence of GUI style and content, complete the isomorphic GUI recognition more accurately and optimize the efficiency of automated mobile app testing.
当前快速增长的移动应用程序迫切需求自动化测试技术以保证其质量。移动应用的自动化测试与其图形用户界面(GUI)的识别与判断紧密相关, 但移动应用却通常存在大量的样式与内容有差异而结构与功能相类似的同构GUI。在自动化测试中, 同构GUI容易引发应用状态空间的爆炸问题, 进而导致测试的低效或失败。针对传统自动化识别同构GUI的局限性, 提出一种基于视觉特征信息的GUI相似度判断方法。通过目标检测技术识别GUI组件元素进而抽象出GUI结构框架; 采用卷积自编码器提取出GUI结构视觉特征; 对比GUI视觉特征的相似度完成同构GUI判断。经过实验验证, 所提方法能够屏蔽GUI的样式、内容等影响, 从而更精确地完成同构GUI识别, 优化自动化测试效率。