How To: Exporting dashboards, visualizations and saved searches from Kibana v4.0.1 … and some software philosophizing

Categories Engineering, How To, Tools
The philosophizing …

There are dimensions of software products that differentiate them from other systems. To a large extent compatibility between iterations is one such dimension. This probably is because of the tempo at which software pieces are being delivered compared to other systems – if you want to release often and build the piece incrementally you better make sure that people will not have to waste time recreating everything they have done with the previous versions… or that the added value of the new version is greater than the pain of the migration. Compatibility of the systems is the main criteria by which iterations of the same product are counted in the semantic versioning system: e.g.: in v2.5.96 the 2 denotes the fact that it is incompatible with everything that is v1.X.X or v0.X.X. Migration goes hand in hand with compatibility so this abstract philosophical smear of an introduction is not entirely w/o merit b/c I now turn to the topic of migration.

The exporting …

The UI export functionality Kibana was re-introduced in Kibana v4.1. It apparently was available in the 3.X versions. If you are running versions below v4.1 you will have to resort to the _search API of ElasticSearch.

Kibana_5.2.2

First we need to know that Kibana objects (searches, visualizations, dashboards) are saved in the .kibana index. In that case the query template would look like:

curl -s -u u_name:p_word -XPOST 'https://{{es_domain|es_ip}}:{{es_port}}/.kibana/{{dashboard|visualization|search}}/_search?q=title:{{*|some_specific_title}}&size={{0...999999}}'

An example of a query would look like:
curl -s -u u_name:p_word -XPOST 'https://es.example.com/.kibana/search/_search?q=title:*&size=100' | python -m json.tool

To talk about the compatibility-breaking changes, I will add a truncated output example:
{
    "_shards": {
        "failed": 0,
        "successful": 1,
        "total": 1
    },
    "hits": {
        "hits": [
           {
                "_id": "all-hosts",
                "_index": ".kibana",
                "_score": 1.0,
                "_source": {
                    "description": "",
                    "kibanaSavedObjectMeta": {
   "searchSourceJSON":"{\"index\":\"YYYY.MM.DD\",\"query\":{\"query_string\":{\"query\":\"host: *-host\",\"analyze_wildcard\":true}},\"filter\":[]}"
                    },
                    "title": "All hosts",
                    "version": 1,
"visState":"{\"type\":\"area\",\"params\":{\"addLegend\":true,\"addTooltip\":true,\"defaultYExtents\":false,\"mode\":\"stacked\",\"shareYAxis\":true},\"aggs\":[{\"id\":\"1\",\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"type\":\"date_histogram\",\"schema\":\"segment\",\"params\":{\"field\":\"@timestamp\",\"interval\":\"auto\",\"min_doc_count\":1,\"extended_bounds\":{}}},{\"id\":\"3\",\"type\":\"terms\",\"schema\":\"group\",\"params\":{\"field\":\"host\",\"size\":0,\"order\":\"desc\",\"orderBy\":\"1\"}}],\"listeners\":{}}"
                },
                "_type": "visualization"
            }
        ],
        "max_score": 1.0,
        "total": 1
    },
    "timed_out": false,
    "took": 1
}

How to return only the array of Kibana objects in JSON without the metadata returned by the search _search API using python:

What’s breaking?

When you import the JSON array extracted, you will see this if you have visualizations with terms aggregations that use "size":0 as the size parameter for the query (with the intention of ):

ES does not support "size":0 shorthand for “all available results anymore” – read more here and in the ES version change log. If you want to grab all of the data you will need to use the "size":9999999 parameter in your aggregations, which is allowed:

size_999999

Leave a Reply