Abstract
IoT platforms are in charge of extracting and processing the data that come from IoT networks, generating additional value, and providing access to the user through usable interfaces. ...However, the ever growing number of devices, networks, services and applications within the IoT ecosystem, and the recently adopted edge/cloud architecture, increase the complexity. Therefore, IoT platforms should integrate monitoring and visualization tools to facilitate deployment, management and maintenance tasks. In this work, we present the implementation and performance evaluation of an IoT modular platform for distributed architectures that combines the use of Elastic Stack tools (Elasticsearch, Kibana and Beats) and Apache Kafka. We have developed a monitoring framework based on Beats agents that supervise the platform performance attending to different metrics; and adapted the Kibana visualization tools to provide friendly and accessible information to platform administrators and users. Finally, we have deployed and evaluated the IoT platform in four real use cases, identifying the factors that affect the performance of the different modules: Edge Node, Data Streaming, Cloud Server and Search Engine.
Big data collection involves enormous amounts of raw data. To boost the sustainability of corporate value and support business intelligence and decision-making systems, in-depth data analysis is ...necessary. The data storage, analysis, and visualization methods, as well as the discovery of patterns and linkages, all depend on extensive data analysis. This study aims to process datasets to learn things like how ratings impact market sales transactions and how much of an impact factor connected to consumers and items have on ratings. Elasticsearch and Kibana were used for the dataset processing. This study evaluated traits related to the test parameters using a variety of test procedures. The product is scored as a representation of the product types involved in the sales transaction, and the name is assessed as a reflection of the customer. Kibana and Elasticsearch, a full-text search engine, were used in this work to do extensive data analysis on data sets. It is a visualization tool that is employed in a controlled environment to evaluate how ratings impact market exchanges for electronic goods, and it offers suggestions. The study found a substantial relationship between electronic product sales on the Amazon marketplace from 2012 to 2018. It suggested the importance of buyer constituents as users and how different product categories relate to ratings in business transactions. Doi: 10.28991/HIJ-2023-04-03-09 Full Text: PDF
Дана стаття присвячена розгляду подальших актуальних шляхів забезпечення процедури відслідковування помилок у високонавантажених веб-додатках реалізованих мовою програмування Javascript. У статті ...досліджено та визначено, що помилки, які виникають при розробці та використанні сучасних високонавантажених веб-додатків є дуже небезпечними, оскільки впливають на повноцінну життєдіяльність інформаційної системи в цілому та можуть призводити до порушення конфіденційності та цілісності персональної інформації користувачів. В статті авторами розглянуті питання обробки помилок у мові програмування Javascript, проблема та необхідність відслідковування помилок у високонавантажених веб-додатках, поняття високонавантажених веб-додатків, існуючі методи та підходи відслідковування помилок, принципи побудови сучасних високонавантажених веб-додатків, порівняння існуючих рішень для відслідковування помилок у високонавантежних веб-додатках реалізованих мовою програмування Javascript. Результатом даних досліджень стало створення авторського програмного модулю відслідковування помилок у висконавантажених веб-додатках для вирішення проблеми логування помилок, аналіз логів на повноту, обробку помилок та вирішення їх в майбутньому. Також впровадження такого рішення дозволяє зменшити розмір програмного додатку для завантаження до 5 кілобайт та зберігати історію помилок. Розроблений програмний модуль відслідковування помилок у високонавантажених веб-додатках складається з двох частин клієнтської та серверної. Кожна частина є незалежним програмним модулем та може бути переконфігурована з мінімальними змінами конфігурації на будь-якому іншому ресурсі. Така реалізація дає змогу повністю збирати метрики про кожен XMLHTTP запит, збирати інформацію про оточення користувача в якому сталася помилка, збирати інформація про те, чим саме була викликана помилка, визначати конкретне місце, де сталася помилка при виконанні програмного коду, за допомогою власноруч розробленого алгоритму, зберігати історії помилок у журналі Kibana. Можливі напрямки розвитку цієї роботи пов’язані із розширенням алгоритму відслідковування помилок, для збору більшої кількості даних та удосконалення їх агрегації, на основі розширення метрик. Авторами в подальшому планується ряд науково технічних рішень розробки та впровадження ефективних методів, засобів забезпечення вимог, принципів та підходів забезпечення кібернетичної безпеки та організації захисту на основі використання авторських підходів відслідковування помилок у високонавантажених веб-додатках в дослідних комп’ютерних системах та мережах.
Twitter can be considered as a large scale network. People's opinions matter a lot to analyze how knowledge spreads impact lives. In this project, we took advantage of the Apache Spark Streaming fast ...and memory computing platform to retrieve live tweets and perform sentiment analysis. The primary purpose is to provide a tool to evaluate the score of sentiments in streams. This paper reports on the nature of an analysis of emotions, collecting vast numbers of tweets. Results identify the view of users through tweets into positive, neutral and negative about coronavirus. This project on Spark Streaming to analyze tweets, hashtags or specific keyword/keywords such as (corona) from live twitter data streams. Data is collected from input sources like Twitter and processed downstream using Spark Streaming. Then, how sentiment scores can be generated for tweets and build visualization dashboards on the data using Elasticsearch and Kibana.
Using Users Profiling to Identifying an Attacks Aarthi, M; Nivetha, N; Sharvesha, J ...
Turkish journal of computer and mathematics education,
04/2021, Letnik:
12, Številka:
7
Journal Article
Odprti dostop
Nowadays crime activities have increased in almost all areas, but this paper only focuses on who performs illegal activities within the organization. A user may perform insider and phishing attacks ...in the organization. A legitimate user of an organization may try to login in the administrator id, and then perform some illegal activities. Due to these activities, sensitive data can be modified or corrupted. Identification of illegal user's behavior is very difficult within the organization. The scope of this work is to analyze the log files, to filter out the user profiles of those who are involved in suspicious activity and to detect the suspicious activity of the user. In any organization, large number of log files is being generated, log manger system helps to take an optimal solutions. Although, a variety of log supervisor gadget exists, however, they are not providing that much efficiency. This paper analyses the ELK stack working principles and compare it with Splunk. ELK stack include many additional features such as indexing, preprocessing a large amount of logs and producing graphical representation output using kibana.
A log management system is essential for the networks administrator. With a log management tool, we can collect, store, analyze, archive, and finally dispose of the log information. In this paper, we ...propose the architecture model of a log management system using ELK Stack with Ceph to provide a safe network, good Wi-Fi signal strength, and adequate backup data mechanism. In this case, we use our campus data of Wi-Fi log and NetFlow log. First, we collect and store data of our Wi-Fi log using Filebeats tool, and then, we use Elasticsearch, Logstash, and Kibana Stack to visualize the Wi-Fi log data. Second, we collect and store our NetFlow log using NFDUMP, and then, we also use ELK Stack to visualize the NetFlow log data. Third, we integrate the Wi-Fi log and NetFlow log data in one architecture using a distributed storage Ceph file system (CephFS). Moreover, we also compare the performance of RADOS Gateway and CephFS for better storage mechanism.
This paper examines the use of Elasticsearch for data warehousing and analyses of geo-referenced sensor data. Elasticsearch has several advantages compared to its direct competitors. For example, it ...is capable of handling time series, spatial data, and objects. Moreover, it is natively connected with the data shippers Beats, Logstash, and the visualisation tool Kibana. This paper proposes a method to implement and query multidimensional models in Elasticsearch. No prior work has evaluated Elasticsearch for data warehouses and analytical queries, especially for sensor environmental data. This paper therefore also presents extensive experiments to evaluate its querying performance. The proposed approach is applied to the analysis of sensor data used in the context of CEBA, an environmental cloud solution developed to collect, store, and analyse environmental data. An experimental performance analysis is also provided.
Industry 4.0 is the latest trend in the manufacturing sector that focuses on intelligent manufacturing and smart factories. This leads to opportunities in automation, optimization, asset management ...and predictive maintenance, which helps reduce downtime and increase revenue. In this paper, we propose the solution that was created for a particular SMT PCB manufacturing facility in Mysuru (Vinyas IT), the features it has to offer and the methodologies that were implemented in order to achieve our goals. We will also highlight the important aspects of the solution that will be showcased during the demonstration and the impact of our solution.
In the era of technology, where for almost everything we are dependent on the internet, has been successful enough to dominate our social lives too. So, with excess usage of internet, creating huge ...log files containing hidden valuable information in it. Hence efforts have been made to overcome the shortcomings of log management and perform effective search to extract the real crux from pool of data which is needed for analysis. Further we require an efficient graphical visualizer which can completely express any shape of data. This paper demonstrates the working of open source tools i.e. elastic search, logstash, and kibana which has been clubbed together to have complete insight and visualization of data. Elastic search is used for searching and indexing, Logstash for slicing and dicing the raw data, maintaining and managing events whereas kibana is a graphical front end for data held in elastic search. By implementing these tools we are performing sentiment analysis of data taken from social networking blogging service like twitter.
PDF
