Research on intelligent analysis and visualization of big data based on ELK

Tao Xie; Mingjiang Zhang; Zhe Wang; Linna Yang

doi:10.1117/12.2661192

16 December 2022 Research on intelligent analysis and visualization of big data based on ELK

Tao Xie, Mingjiang Zhang, Zhe Wang, Linna Yang

Proceedings Volume 12500, Fifth International Conference on Mechatronics and Computer Technology Engineering (MCTE 2022); 125004K (2022) https://doi.org/10.1117/12.2661192
Event: 5th International Conference on Mechatronics and Computer Technology Engineering (MCTE 2022), 2022, Chongqing, China

Abstract

In the era of big data, the analysis and processing of data has many difficult problems, such as difficult integration of heterogeneous data, inability to guarantee timeliness, weak analysis ability and weak visualization ability. In response to these problems, this paper conducts secondary development of the open-source ELK (Elasticsearch, Logstash, Kibana) log analysis system to better analyze and visualize the data. In order to show the effectiveness of the system more clearly, this paper takes the collected operation and maintenance data of a certain server as an example, and presents the important information obtained by searching, summarizing and analyzing the logs in the form of various dynamic charts or visualizations. It can be seen from the results that the system can better and more vividly display data distribution and operation and maintenance and provide a method and tool for data processing and visual presentation, which can help people better grasp information or make auxiliary decision-making.

Citation Download Citation

Tao Xie, Mingjiang Zhang, Zhe Wang, and Linna Yang "Research on intelligent analysis and visualization of big data based on ELK", Proc. SPIE 12500, Fifth International Conference on Mechatronics and Computer Technology Engineering (MCTE 2022), 125004K (16 December 2022); https://doi.org/10.1117/12.2661192

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available