Import CSV and Log Data into Elasticsearch from Kibana with File Data Visualizer
Blog post from Elastic
File Data Visualizer, introduced in Elastic Stack 6.5, is a feature designed to simplify the import of delimited files, NDJSON, and semi-structured text into Elasticsearch, facilitating data exploration in Kibana and Machine Learning without deep technical knowledge. It utilizes the find_file_structure endpoint to analyze file content, suggest ingest pipelines, and create mappings for seamless data integration into Elasticsearch. Users can configure settings such as data format, delimiter, and field names through an intuitive interface, offering both simple and advanced import modes. Although currently experimental and not recommended for production processes, the tool allows users to import data into new indices and provides insights through Kibana for further analysis. The feature is particularly beneficial for users with limited technical backgrounds, as demonstrated by a marketing team member who successfully imported and visualized earthquake data. The tool also supports importing semi-structured log files, using grok patterns to identify fields, and offers the ability to edit field names and types for accuracy.