asahi/rikako-note

Fork 0

Files

wuxiangkai 500af61cc4 阅读es pipeline aggregation文档

2024-12-30 19:31:46 +08:00

66 KiB

Raw Blame History

ElasticSearch

简介

ElasticSearch是一个分布式的搜索和分析引擎、可缩放的数据存储、矢量数据库（vector database）。

用例场景

如下是ElasticSearch的用例场景

日志：es可以用于收集、存储和分析日志
full-text search：通过倒排索引，es可以用于构建全文本搜索方案

安装

下列安装示例基于Ubuntu 22.04

add elasticsearch GPG keys

wget -q https://artifacts.elastic.co/GPG-KEY-elasticsearch -O- | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

Add Elasticsearch 8.x APT Repository

echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list

install elastic search

sudo apt update && sudo apt install elasticsearch

Indices, documents, and fields

在ES中，index是存储的基本单元，是存储数据的逻辑namespace，位于同一index下的存储数据共享相似的特征。

在ES服务部署之后，需要创建index，并在index中存储数据。

index是一系列document的集合，通过name或alias唯一标识，在查询或其他操作中，通过unique name来定位index。

Documents and fields

ElasticSearch以json文档的格式来序列化和存储数据。一个document是fields的集合，field则是对应的key-value pair。每个document都有一个唯一的id，文档id可以手动指定，也可以让ES自动生成。

一个ES文档的格式如下所示：

{
  "_index": "my-first-elasticsearch-index",
  "_id": "DyFpo5EBxE8fzbb95DOa",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "found": true,
  "_source": {
    "email": "john@smith.com",
    "first_name": "John",
    "last_name": "Smith",
    "info": {
      "bio": "Eco-warrior and defender of the weak",
      "age": 25,
      "interests": [
        "dolphins",
        "whales"
      ]
    },
    "join_date": "2024/05/01"
  }
}

metadata field

一个indexed document包含data和metadata。

metadata fields为系统fields，用于存储document的相关信息，在elastic search中，metadata field以下划线_开头，例如，如下field为metadata field：

_id：文档id，文档id在每个index中都是唯一的
_index：标识该文档存储在哪个index中

Mappings and data types

每个index都有mapping或schema，用于指定document中的fields如何被索引。

一个mapping定义了每个field的data type，以及该field如何被索引、该field如何被存储。

在将document添加到index时，对于mapping有如下两种选择：

Dynamic Mapping：让ES自动检测data type并创建mapping。在引入dynamic mapping后，可能会针对某些用例场景产生次优结果
Explicit Mapping：针对每个field手动指定data type

向ElasticSearch中添加数据

General content

General content是不包含时间戳的数据，对于general content，可以通过如下方式添加到ES中：

API：可以通过HTTP API向ES中添加数据

Timestamped data

Timestamped data代表包含timestamp field的数据，如果使用了Elastic Common Schema(ECS)，那么timestamp field的名称为@timestamp，这些数据可能是logs, metrics, traces。

查询和分析数据

可以通过如下方式来查询和分析数据

Rest Api

可以通过rest api来管理elastic search集群，并索引和查询数据。

query language

ES提供了多种查询语言来和数据进行交互

Query DSL: ES主要的查询语言
ES|QL: 8.11中新增的piped query language和计算引擎

Query DSL

query DSL是一种json格式的查询语言，支持复杂的查询、过滤、聚合操作，是ES最原始也是功能最强的查询语言

_search endpoint接收Query DSL格式的查询

query DSL支持如下查询：

全文本搜索：搜索已经被分析和索引过的文本，支持短语或临近查询、模糊匹配等
关键词查询：支持精确的关键词匹配
语义查询
向量查询
地理位置查询

Query DSL分析

如果要通过Query DSL对elastic search数据进行分析，那么Aggregations是主要的工具。

Aggregations允许根据数据构建复杂的数据摘要，并获取指标、模式和趋势。

aggregations利用了和查询相同的数据结构，故而聚合的速度十分快，可以实时的对数据进行分析和可视化。

在使用ES时，可以在同一时刻对相同的数据同时进行文档查询、结果过滤、数据分析操作，聚合是在查询请求的上下文中进行计算的。

ES支持如下类型的Aggregations：

Metric：计算metrics，例如field的总和或平均
Bucket：基于field value、范围或其他指标对文档进行分组
Pipeline：在其他聚合操作结果集的基础上执行聚合操作

ES | QL

Elasticsearch Query Language是一个piped query language，用于对数据进行过滤、transforming、分析。ES|QL基于新的计算引擎，查询、聚合、transformation方法是直接在Elasticsearch中执行的。在Kibana工具中可以使用ES|QL语法。

ES|QL支持Query DSL中的部分特性，例如聚合、过滤、transformation

使用ElasticSearch Api索引和查询数据

创建索引

可以通过如下方式来创建一个名为books的索引：

PUT /books

返回相应结构如下，代表索引创建成功：

{
  "acknowledged": true,
  "shards_acknowledged": true,
  "index": "books"
}

向索引中添加数据

可以向ElasticSearch中添加json形式的数据，json格式数据被称为document。ElasticSearch将添加的数据保存到可搜索的索引中。

向索引中添加单个document

POST books/_doc
{
  "name": "Snow Crash",
  "author": "Neal Stephenson",
  "release_date": "1992-06-01",
  "page_count": 470
}

该请求的返回体中包含ElasticSearch为该document生成的元数据，包含索引范围内唯一的_id，在索引范围内唯一标识该document。

{
  "_index": "books", 
  "_id": "O0lG2IsBaSa7VYx_rEia", 
  "_version": 1, 
  "result": "created", 
  "_shards": { 
    "total": 2, 
    "successful": 2, 
    "failed": 0 
  },
  "_seq_no": 0, 
  "_primary_term": 1 
}

向索引中添加多个document

可以使用/_bulk接口来在单个请求中添加多个document。_bulk请求的请求体由多个json串组成，json串之间通过换行符分隔。

bulk请求示例如下所示：

POST /_bulk
{ "index" : { "_index" : "books" } }
{"name": "Revelation Space", "author": "Alastair Reynolds", "release_date": "2000-03-15", "page_count": 585}
{ "index" : { "_index" : "books" } }
{"name": "1984", "author": "George Orwell", "release_date": "1985-06-01", "page_count": 328}
{ "index" : { "_index" : "books" } }
{"name": "Fahrenheit 451", "author": "Ray Bradbury", "release_date": "1953-10-15", "page_count": 227}
{ "index" : { "_index" : "books" } }
{"name": "Brave New World", "author": "Aldous Huxley", "release_date": "1932-06-01", "page_count": 268}
{ "index" : { "_index" : "books" } }
{"name": "The Handmaids Tale", "author": "Margaret Atwood", "release_date": "1985-06-01", "page_count": 311}

如果上述请求被正确处理，将会得到如下返回体：

{
  "errors": false,
  "took": 29,
  "items": [
    {
      "index": {
        "_index": "books",
        "_id": "QklI2IsBaSa7VYx_Qkh-",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 2,
          "failed": 0
        },
        "_seq_no": 1,
        "_primary_term": 1,
        "status": 201
      }
    },
    {
      "index": {
        "_index": "books",
        "_id": "Q0lI2IsBaSa7VYx_Qkh-",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 2,
          "failed": 0
        },
        "_seq_no": 2,
        "_primary_term": 1,
        "status": 201
      }
    },
    {
      "index": {
        "_index": "books",
        "_id": "RElI2IsBaSa7VYx_Qkh-",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 2,
          "failed": 0
        },
        "_seq_no": 3,
        "_primary_term": 1,
        "status": 201
      }
    },
    {
      "index": {
        "_index": "books",
        "_id": "RUlI2IsBaSa7VYx_Qkh-",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 2,
          "failed": 0
        },
        "_seq_no": 4,
        "_primary_term": 1,
        "status": 201
      }
    },
    {
      "index": {
        "_index": "books",
        "_id": "RklI2IsBaSa7VYx_Qkh-",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 2,
          "failed": 0
        },
        "_seq_no": 5,
        "_primary_term": 1,
        "status": 201
      }
    }
  ]
}

定义mapping和data type

使用dynamic mapping

当使用dynamic mapping时，elastic search默认情况下将会自动为新field创建mapping。上述示例中向索引中添加的document都使用了dynamic mapping，因为在创建索引时，并没有手动指定mapping。

可以向books索引中新增一个document，新增document中包含当前索引documents中不存在的字段：

POST /books/_doc
{
  "name": "The Great Gatsby",
  "author": "F. Scott Fitzgerald",
  "release_date": "1925-04-10",
  "page_count": 180,
  "language": "EN" 
}

此时，针对books索引，新字段language之前并不存在，会以text的data type被新增到mapping中。

可以通过/{index_uid}/_mapping请求来查看索引的mapping信息：

GET /books/_mapping

其返回的响应为：

{
  "books": {
    "mappings": {
      "properties": {
        "author": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "name": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "new_field": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "page_count": {
          "type": "long"
        },
        "release_date": {
          "type": "date"
        }
      }
    }
  }
}

手动指定索引的mapping

如下示例会展示如何在创建索引时手动指定索引的mapping：

PUT /my-explicit-mappings-books
{
  "mappings": {
    "dynamic": false,  
    "properties": {  
      "name": { "type": "text" },
      "author": { "type": "text" },
      "release_date": { "type": "date", "format": "yyyy-MM-dd" },
      "page_count": { "type": "integer" }
    }
  }
}

上述示例中请求体含义如下：

"dynamic": false: 在索引中禁用dynamic mapping，如果提交的document中包含了mapping中不存在的field，那么该提交的document将会被拒绝
"properties"：properties属性定义了document中的fields及其数据类型

将dynamic mapping和手动指定mapping相结合

如果在创建索引时手动指定了索引的mapping，那么在向索引中添加document时，document必须符合索引的定义。

如果要结合dynamic mapping和手动指定mapping，有如下两种方式：

使用update mapping Api
手动指定mapping时，将dynamic设置为true，此时向document中添加new field时无需对mapping执行update

搜索索引

搜索所有文档

GET books/_search

上述请求将会搜索books索引中所有的文档

响应如下：

{
  "took": 2, 
  "timed_out": false, 
  "_shards": { 
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": { 
    "total": { 
      "value": 7,
      "relation": "eq"
    },
    "max_score": 1, 
    "hits": [
      {
        "_index": "books", 
        "_id": "CwICQpIBO6vvGGiC_3Ls", 
        "_score": 1, 
        "_source": { 
          "name": "Brave New World",
          "author": "Aldous Huxley",
          "release_date": "1932-06-01",
          "page_count": 268
        }
      },
      ... (truncated)
    ]
  }
}

其中，响应体的字段含义如下：

took：es执行该搜索请求花费的时间，单位为ms
time_out：代表该请求是否超时
_shards：代表该请求的分片数和成功数
hits：hits对象中包含了执行结果
total：total对象中包含了匹配结果的总数信息
max_score：max_score包含了在所有匹配documents中最高的relavance score
_index：该字段代表了document所属的索引
_id：该字段代表document的唯一标识id
_score：_score字段代表当前document的relavance score
_source：该字段包含了indexing过程中提交的原始json对象

match请求

可以通过match请求来查询特定field中包含指定值的documents。这是全文本查询的标准查询。

如下示例中会查询索引中name field中包含brave的文档：

GET books/_search
{
  "query": {
    "match": {
      "name": "brave"
    }
  }
}

响应体结构如下：

{
  "took": 9,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 0.6931471, 
    "hits": [
      {
        "_index": "books",
        "_id": "CwICQpIBO6vvGGiC_3Ls",
        "_score": 0.6931471,
        "_source": {
          "name": "Brave New World",
          "author": "Aldous Huxley",
          "release_date": "1932-06-01",
          "page_count": 268
        }
      }
    ]
  }
}

删除索引

如果要删除创建的索引从头开始，可以使用如下方式：

DELETE /books
DELETE /my-explicit-mappings-books

删除索引将会永久删除其document、shards、元数据。

全文本搜索和过滤

如下示例展示了如何实现cook blog的搜索功能。

创建索引

创建cooking_blog索引

PUT /cooking_blog

为索引定义mapping：

PUT /cooking_blog/_mapping
{
  "properties": {
    "title": {
      "type": "text",
      "analyzer": "standard", 
      "fields": { 
        "keyword": {
          "type": "keyword",
          "ignore_above": 256 
        }
      }
    },
    "description": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword"
        }
      }
    },
    "author": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword"
        }
      }
    },
    "date": {
      "type": "date",
      "format": "yyyy-MM-dd"
    },
    "category": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword"
        }
      }
    },
    "tags": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword"
        }
      }
    },
    "rating": {
      "type": "float"
    }
  }
}

上述mapping定义含义如下：

对于text类型的field，如果analyzer没有指定，那么会默认使用standard analyzer
在上述示例中，使用了multi fields，将字段既作为text来进行全文搜索，又作为keyword来进行聚合和排序。在该字段上，既支持全文搜索，又支持青雀匹配和过滤。如果使用dynamic mapping，那么multi-fields将会自动被创建。
ignore_above不会索引keywordfield中超过256个字符长度的值。默认情况下，keyword field其ignore_above的值为256

multi-field

对同一个字段按不同的方式进行索引有时候很必要，对于multi-fields，一个字符串类型字段可以被映射到text类型用于全文索引，也可以被映射到keyword类型用作排序和聚合。

示例如下：
PUT my-index-000001
{
  "mappings": {
   "properties": {
     "city": {
       "type": "text",
       "fields": {
         "raw": { 
           "type":  "keyword"
         }
       }
     }
   }
 }
}

PUT my-index-000001/_doc/1
{
  "city": "New York"
}

PUT my-index-000001/_doc/2
{
  "city": "York"
}

GET my-index-000001/_search
{
  "query": {
    "match": {
      "city": "york" 
    }
  },
  "sort": {
    "city.raw": "asc" 
  },
  "aggs": {
    "Cities": {
      "terms": {
        "field": "city.raw" 
      }
    }
  }
}

ignore_above

在keyword中指定ignore_above为256，将避免索引长度大于256的字段值。当字段长度大于256时，该字段将不会被索引，但是被忽略字段将会包含在_source中

当ignore_above没有显式指定时，其值默认为256.

批量插入数据

再创建索引后，可以向索引中批量插入文档数据：

POST /cooking_blog/_bulk?refresh=wait_for
{"index":{"_id":"1"}}
{"title":"Perfect Pancakes: A Fluffy Breakfast Delight","description":"Learn the secrets to making the fluffiest pancakes, so amazing you won't believe your tastebuds. This recipe uses buttermilk and a special folding technique to create light, airy pancakes that are perfect for lazy Sunday mornings.","author":"Maria Rodriguez","date":"2023-05-01","category":"Breakfast","tags":["pancakes","breakfast","easy recipes"],"rating":4.8}
{"index":{"_id":"2"}}
{"title":"Spicy Thai Green Curry: A Vegetarian Adventure","description":"Dive into the flavors of Thailand with this vibrant green curry. Packed with vegetables and aromatic herbs, this dish is both healthy and satisfying. Don't worry about the heat - you can easily adjust the spice level to your liking.","author":"Liam Chen","date":"2023-05-05","category":"Main Course","tags":["thai","vegetarian","curry","spicy"],"rating":4.6}
{"index":{"_id":"3"}}
{"title":"Classic Beef Stroganoff: A Creamy Comfort Food","description":"Indulge in this rich and creamy beef stroganoff. Tender strips of beef in a savory mushroom sauce, served over a bed of egg noodles. It's the ultimate comfort food for chilly evenings.","author":"Emma Watson","date":"2023-05-10","category":"Main Course","tags":["beef","pasta","comfort food"],"rating":4.7}
{"index":{"_id":"4"}}
{"title":"Vegan Chocolate Avocado Mousse","description":"Discover the magic of avocado in this rich, vegan chocolate mousse. Creamy, indulgent, and secretly healthy, it's the perfect guilt-free dessert for chocolate lovers.","author":"Alex Green","date":"2023-05-15","category":"Dessert","tags":["vegan","chocolate","avocado","healthy dessert"],"rating":4.5}
{"index":{"_id":"5"}}
{"title":"Crispy Oven-Fried Chicken","description":"Get that perfect crunch without the deep fryer! This oven-fried chicken recipe delivers crispy, juicy results every time. A healthier take on the classic comfort food.","author":"Maria Rodriguez","date":"2023-05-20","category":"Main Course","tags":["chicken","oven-fried","healthy"],"rating":4.9}

执行full-text search

full-text search会在一个或多个document fields之间执行基于文本的查询。这些查询会为每个匹配的文档计算relevance score，relevance score的计算基于文档内容和search terms的关联程度。

ES支持多种查询类型，每种查询类型都有其matching text和relevance scoring的方法。

`match`

match是针对full-text的标准查询，基于每个字段上配置的analyzer，query text将会被分析。

GET /cooking_blog/_search
{
  "query": {
    "match": {
      "description": {
        "query": "fluffy pancakes" 
      }
    }
  }
}

默认情况下，match query在resulting tokens间使用or，故而在上述的查询中，会查找description中包含fluffy或pancakes任一的document。

其会返回结果如下：

{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": { 
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 1.8378843, 
    "hits": [
      {
        "_index": "cooking_blog",
        "_id": "1",
        "_score": 1.8378843, 
        "_source": {
          "title": "Perfect Pancakes: A Fluffy Breakfast Delight", 
          "description": "Learn the secrets to making the fluffiest pancakes, so amazing you won't believe your tastebuds. This recipe uses buttermilk and a special folding technique to create light, airy pancakes that are perfect for lazy Sunday mornings.", 
          "author": "Maria Rodriguez",
          "date": "2023-05-01",
          "category": "Breakfast",
          "tags": [
            "pancakes",
            "breakfast",
            "easy recipes"
          ],
          "rating": 4.8
        }
      }
    ]
  }
}

track total hits

如果想要精确计算hit count，通常需要遍历所有的匹配文档，这将会带来很大开销。

track_total_hists参数允许对如何计算hit count进行控制。

如果设置为true，那么会精确的计算匹配数量，total.relation会一直为eq，代表total.value和实际hit count相同

如果该值为其他值，例如其默认值10000，则该查询数量的下限为10000

如果total.relation为eq，则total.value代表实际hit count

如果total.relation为gte，则total.value为hit count的下界，实际hit count大于或等于total.value

max score

max_score代表所有matching documents中最高的relevance score

_score

_score为指定文档的relevance score，分数越高，匹配越好。

contains all terms

如上所示，如果在query.match.description.query中指定了多个由空格分隔的词，那么可以为query.match.description.operator指定and，那么其会查询description中包含所有分词的文档。

GET /cooking_blog/_search
{
  "query": {
    "match": {
      "description": {
        "query": "fluffy pancakes",
        "operator": "and"
      }
    }
  }
}

应匹配terms的最小个数

通过minimum_should_match参数，可以指定在search result中最少应该匹配的terms个数。

如下实例中，查询文本中包含三个分词，"fluffy", "pancakes", or "breakfast"，但是查询结果中至少应该包含3个中的2个：

GET /cooking_blog/_search
{
  "query": {
    "match": {
      "title": {
        "query": "fluffy pancakes breakfast",
        "minimum_should_match": 2
      }
    }
  }
}

search across mutli fields

当用户输入查询文本时，其通常不知道是否查询文本出现在文档中的哪个字段。multi_match允许同时跨多个字段来进行查询。

mutli_match查询示例如下：

GET /cooking_blog/_search
{
  "query": {
    "multi_match": {
      "query": "vegetarian curry",
      "fields": ["title", "description", "tags"]
    }
  }
}

在上述示例中，将会在title, description, tags三个字段中查询vegetarian curry，其中每个字段的重要性都相同。

但是，在某种场景下，在部分字段中的匹配比其他字段中的匹配更加重要（例如，相关论文检索时，关键词出现在摘要中要比出现在正文中更加重要）。

因此，可以在fields中针对多个字段进行重要性加权的调整：

GET /cooking_blog/_search
{
  "query": {
    "multi_match": {
      "query": "vegetarian curry",
      "fields": ["title^3", "description^2", "tags"] 
    }
  }
}

如上述示例所示，^语法应用于指定字段：

title^3：title的重要性为非增强字段的3倍
description^2：description的重要性为非增强字段的2倍
tags：非增强字段

filter & exact matches

filter允许根据指定条件来缩小查询结果的范围。和full-text search不同的是，filter是二进制的，并且不会影响relevance score。

filter相较于query执行的更快，因为被排除的result不会计算score。

bool query

在如下示例中，bool query只会返回category为Breakfast中的文档：

GET /cooking_blog/_search
{
  "query": {
    "bool": {
      "filter": [
        { "term": { "category.keyword": "Breakfast" } }  
      ]
    }
  }
}

在针对category进行filter时，指定的是category.keyword，其类型为keyword，为精确、区分大小写的匹配。

category.keyword字段其multi field类型为keyword，代表了字段没有经过分析的版本（代表字段没有指定analyzer），故而其进行的是精确、大小写敏感的匹配。

如下两种场景将会使用.keyword：

使用了针对text类型的字段使用了dynamic mappings，此时es会自动创建一个名为.keyword的multi-field

当手动指定text类型字段的.keyword为keyword类型时

date range

当想要查询一定时间范围内的文档时，可以按如下方式来编写查询：

GET /cooking_blog/_search
{
  "query": {
    "range": {
      "date": {
        "gte": "2023-05-01", 
        "lte": "2023-05-31" 
      }
    }
  }
}

exact match

可以通过term来搜索精确术语，term将会在字段中精确的搜索，并不会对传入的文本进行分析。

针对指定term的精确、大小写敏感搜索通常基于keyword进行搜索。

term搜索示例如下所示，搜索作者为Maria Rodriguez的文档：

GET /cooking_blog/_search
{
  "query": {
    "term": {
      "author.keyword": "Maria Rodriguez" 
    }
  }
}

应该避免在text类型的字段上使用term查询，因为text类型的field会被analyzer进行分析和转换

组合搜索

bool query允许将多个子查询组合为复杂的查询。

bool query示例如下：

GET /cooking_blog/_search
{
  "query": {
    "bool": {
      "must": [
        { "term": { "tags": "vegetarian" } },
        {
          "range": {
            "rating": {
              "gte": 4.5
            }
          }
        }
      ],
      "should": [
        {
          "term": {
            "category": "Main Course"
          }
        },
        {
          "multi_match": {
            "query": "curry spicy",
            "fields": [
              "title^2",
              "description"
            ]
          }
        },
        {
          "range": {
            "date": {
              "gte": "now-1M/d"
            }
          }
        }
      ],
      "must_not": [ 
        {
          "term": {
            "category.keyword": "Dessert"
          }
        }
      ]
    }
  }
}

上述查询含义如下：

tag中必须精确匹配vegetarian
rating必须大于或等于4.5
在title或者description中应该包含curry或spicy
category应该精确匹配Main Course
date字段应该大于或等于上个月
category不能为Dessert

must_not

must_not会淘汰不满足指定条件的文档

使用Query DSL分析数据

在使用kibana导入sample ecommerce orders的数据集后，其会创建一个名为kibana_sample_data_ecommerce的索引，其索引结构如下：

{
  "kibana_sample_data_ecommerce": {
    "mappings": {
      "properties": {
        "category": {
          "type": "text",
          "fields": { 
            "keyword": {
              "type": "keyword"
            }
          }
        },
        "currency": {
          "type": "keyword"
        },
        "customer_birth_date": {
          "type": "date"
        },
        "customer_first_name": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "customer_full_name": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "customer_gender": {
          "type": "keyword"
        },
        "customer_id": {
          "type": "keyword"
        },
        "customer_last_name": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "customer_phone": {
          "type": "keyword"
        },
        "day_of_week": {
          "type": "keyword"
        },
        "day_of_week_i": {
          "type": "integer"
        },
        "email": {
          "type": "keyword"
        },
        "event": {
          "properties": {
            "dataset": {
              "type": "keyword"
            }
          }
        },
        "geoip": {
          "properties": { 
            "city_name": {
              "type": "keyword"
            },
            "continent_name": {
              "type": "keyword"
            },
            "country_iso_code": {
              "type": "keyword"
            },
            "location": {
              "type": "geo_point" 
            },
            "region_name": {
              "type": "keyword"
            }
          }
        },
        "manufacturer": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword"
            }
          }
        },
        "order_date": {
          "type": "date"
        },
        "order_id": {
          "type": "keyword"
        },
        "products": {
          "properties": { 
            "_id": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "base_price": {
              "type": "half_float"
            },
            "base_unit_price": {
              "type": "half_float"
            },
            "category": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword"
                }
              }
            },
            "created_on": {
              "type": "date"
            },
            "discount_amount": {
              "type": "half_float"
            },
            "discount_percentage": {
              "type": "half_float"
            },
            "manufacturer": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword"
                }
              }
            },
            "min_price": {
              "type": "half_float"
            },
            "price": {
              "type": "half_float"
            },
            "product_id": {
              "type": "long"
            },
            "product_name": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword"
                }
              },
              "analyzer": "english"
            },
            "quantity": {
              "type": "integer"
            },
            "sku": {
              "type": "keyword"
            },
            "tax_amount": {
              "type": "half_float"
            },
            "taxful_price": {
              "type": "half_float"
            },
            "taxless_price": {
              "type": "half_float"
            },
            "unit_discount_amount": {
              "type": "half_float"
            }
          }
        },
        "sku": {
          "type": "keyword"
        },
        "taxful_total_price": {
          "type": "half_float"
        },
        "taxless_total_price": {
          "type": "half_float"
        },
        "total_quantity": {
          "type": "integer"
        },
        "total_unique_products": {
          "type": "integer"
        },
        "type": {
          "type": "keyword"
        },
        "user": {
          "type": "keyword"
        }
      }
    }
  }
}

其中，geoip.properties, products.properties都为嵌套的object类型，而geo_point类型则是用于地理坐标。

get metrics

计算订单的平均值

通过如下请求，可以计算数据集中所有订单的平均值：

GET kibana_sample_data_ecommerce/_search
{
 "size": 0, 
 "aggs": {
   "avg_order_value": { 
     "avg": { 
       "field": "taxful_total_price"
     }
   }
 }
}

在上述请求体中，各属性分别代表如下含义：

size: 将size设置为0可以避免在返回的结果中包含匹配的文档，size设置为0后，返回结果中只会包含聚合的结果
avg_order_value为该项metric的name
avg为聚合类型，会计算算数平均

该请求的返回结果如下所示：

{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 4675, 
      "relation": "eq"
    },
    "max_score": null,
    "hits": [] 
  },
  "aggregations": {
    "avg_order_value": { 
      "value": 75.05542864304813 
    }
  }
}

返回结果中各属性含义如下所示：

hits.total.value：代表数据集中的订单数量
hits.hits为空，因为请求体中设置了size为0
aggregations中包含聚合结果，请求体中为metric指定的结果为avg_order_value，故而该项metric位于aggregations.avg_order_value

在单个请求中计算订单的多个metrics

如果想要在单个请求中结算多个metrics，可以通过stats聚合类型：

{
  "size":0,
  "aggs":{
    "order_status":{
      "stats":{
        "field":"taxful_total_price"
      }
    }
  }
}

其中，stats聚合类型会返回count, min, max, avg, sum五个metrics。

其返回结果为

{
 "aggregations": {
   "order_stats": {
     "count": 4675, 
     "min": 6.98828125, 
     "max": 2250, 
     "avg": 75.05542864304813, 
     "sum": 350884.12890625 
   }
 }
}

根据category对订单进行分组

可以根据terms聚合类型来对订单进行分组，

GET kibana_sample_data_ecommerce/_search
{
 "size": 0,
 "aggs": {
   "sales_by_category": { 
     "terms": { 
       "field": "category.keyword", 
       "size": 5, 
       "order": { "_count": "desc" } 
     }
   }
 }
}

terms聚合类型会根据该字段的类型对文档进行分组

"size":5和 "order": { "_count": "desc" }设置了只会返回最多的5个category

其返回结果如下所示：

{
  "took": 4,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 4675,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "sales_by_category": {
      "doc_count_error_upper_bound": 0, 
      "sum_other_doc_count": 572, 
      "buckets": [ 
        {
          "key": "Men's Clothing", 
          "doc_count": 2024 
        },
        {
          "key": "Women's Clothing",
          "doc_count": 1903
        },
        {
          "key": "Women's Shoes",
          "doc_count": 1136
        },
        {
          "key": "Men's Shoes",
          "doc_count": 944
        },
        {
          "key": "Women's Accessories",
          "doc_count": 830
        }
      ]
    }
  }
}

doc_count_error_upper_bound

基于es的分布式结构，terms aggregations在多个shards上运行时，document计数可能会有小的误差，doc_count_error_upper_bound的代表计数的最大可能误差

sum_other_doc_count: 由于当前请求体中设置了aggs.sales_by_category.terms.size为5，故而sum_other_doc_count代表未包含在返回结果中的文档数量
aggregations.buckets：该字段代表category buckets数组，根据count来排序

date_histogram

date_histogram类型的聚合会将文档聚合为time-based buckets，其类似于terms，但是其并非基于精确匹配的聚合，而是基于日期的集合。

date_histogram聚合示例如下：

GET kibana_sample_data_ecommerce/_search
{
 "size": 0,
 "aggs": {
   "daily_orders": { 
     "date_histogram": { 
       "field": "order_date",
       "calendar_interval": "day", 
       "format": "yyyy-MM-dd", 
       "min_doc_count": 0 
     }
   }
 }
}

上述示例中，各参数含义如下：

min_doc_count：当min_doc_count被设置为0时，会返回没有订单的天数

请求返回结果如下：

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 4675,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "daily_orders": { 
      "buckets": [ 
        {
          "key_as_string": "2024-11-28", 
          "key": 1732752000000, 
          "doc_count": 146 
        },
        {
          "key_as_string": "2024-11-29",
          "key": 1732838400000,
          "doc_count": 153
        },
        {
          "key_as_string": "2024-11-30",
          "key": 1732924800000,
          "doc_count": 143
        },
        {
          "key_as_string": "2024-12-01",
          "key": 1733011200000,
          "doc_count": 140
        },
        {
          "key_as_string": "2024-12-02",
          "key": 1733097600000,
          "doc_count": 139
        },
        {
          "key_as_string": "2024-12-03",
          "key": 1733184000000,
          "doc_count": 157
        },
        {
          "key_as_string": "2024-12-04",
          "key": 1733270400000,
          "doc_count": 145
        },
        {
          "key_as_string": "2024-12-05",
          "key": 1733356800000,
          "doc_count": 152
        },
        {
          "key_as_string": "2024-12-06",
          "key": 1733443200000,
          "doc_count": 163
        },
        {
          "key_as_string": "2024-12-07",
          "key": 1733529600000,
          "doc_count": 141
        },
        {
          "key_as_string": "2024-12-08",
          "key": 1733616000000,
          "doc_count": 151
        },
        {
          "key_as_string": "2024-12-09",
          "key": 1733702400000,
          "doc_count": 143
        },
        {
          "key_as_string": "2024-12-10",
          "key": 1733788800000,
          "doc_count": 143
        },
        {
          "key_as_string": "2024-12-11",
          "key": 1733875200000,
          "doc_count": 142
        },
        {
          "key_as_string": "2024-12-12",
          "key": 1733961600000,
          "doc_count": 161
        },
        {
          "key_as_string": "2024-12-13",
          "key": 1734048000000,
          "doc_count": 144
        },
        {
          "key_as_string": "2024-12-14",
          "key": 1734134400000,
          "doc_count": 157
        },
        {
          "key_as_string": "2024-12-15",
          "key": 1734220800000,
          "doc_count": 158
        },
        {
          "key_as_string": "2024-12-16",
          "key": 1734307200000,
          "doc_count": 144
        },
        {
          "key_as_string": "2024-12-17",
          "key": 1734393600000,
          "doc_count": 151
        },
        {
          "key_as_string": "2024-12-18",
          "key": 1734480000000,
          "doc_count": 145
        },
        {
          "key_as_string": "2024-12-19",
          "key": 1734566400000,
          "doc_count": 157
        },
        {
          "key_as_string": "2024-12-20",
          "key": 1734652800000,
          "doc_count": 158
        },
        {
          "key_as_string": "2024-12-21",
          "key": 1734739200000,
          "doc_count": 153
        },
        {
          "key_as_string": "2024-12-22",
          "key": 1734825600000,
          "doc_count": 165
        },
        {
          "key_as_string": "2024-12-23",
          "key": 1734912000000,
          "doc_count": 153
        },
        {
          "key_as_string": "2024-12-24",
          "key": 1734998400000,
          "doc_count": 158
        },
        {
          "key_as_string": "2024-12-25",
          "key": 1735084800000,
          "doc_count": 160
        },
        {
          "key_as_string": "2024-12-26",
          "key": 1735171200000,
          "doc_count": 159
        },
        {
          "key_as_string": "2024-12-27",
          "key": 1735257600000,
          "doc_count": 152
        },
        {
          "key_as_string": "2024-12-28",
          "key": 1735344000000,
          "doc_count": 142
        }
      ]
    }
  }
}

上述示例中，key_as_string为human readable日期字符串，而key则是和key_as_string代表相同日期的unix时间戳。

doc_count代表bucket中的文档个数。

Calendar intervals

calendar-aware interval可以通过calendar_interval来进行配置，可以通过时间单位名称month或单个时间单位1M来进行指定。类似的day和1d也是等同的，但是，并不支持2d这种多个时间单位的配置。

可以接受的时间单位为：

minute, 1m
hour, 1h
day, 1d
week, 1w
month, 1M
quarter, 1q
year, 1y

compare category performance

GET kibana_sample_data_ecommerce/_search
{
 "size": 0,
 "aggs": {
   "categories": {
     "terms": {
       "field": "category.keyword",
       "size": 5,
       "order": { "total_revenue": "desc" } 
     },
     "aggs": { 
       "total_revenue": { 
         "sum": {
           "field": "taxful_total_price"
         }
       },
       "avg_order_value": { 
         "avg": {
           "field": "taxful_total_price"
         }
       },
       "total_items": { 
         "sum": {
           "field": "total_quantity"
         }
       }
     }
   }
 }
}

如上所示，在按category.keyword分组聚合后，其又在aggs.categories.aggs中指定了针对每个category计算的metrics。

上述请求的返回结果如下所示：

{
 "aggregations": {
   "categories": {
     "buckets": [
       {
         "key": "Men's Clothing", 
         "doc_count": 2179, 
         "total_revenue": { 
           "value": 156729.453125
         },
         "avg_order_value": { 
           "value": 71.92726898715927
         },
         "total_items": { 
           "value": 8716
         }
       },
       {
         "key": "Women's Clothing",
         "doc_count": 2262,
         ...
       }
     ]
   }
 }
}

analyze daily sale performance

GET kibana_sample_data_ecommerce/_search
{
 "size": 0,
 "aggs": {
   "daily_sales": {
     "date_histogram": {
       "field": "order_date",
       "calendar_interval": "day",
       "format": "yyyy-MM-dd"
     },
     "aggs": {
       "revenue": { 
         "sum": {
           "field": "taxful_total_price"
         }
       },
       "unique_customers": { 
         "cardinality": {
           "field": "customer_id"
         }
       },
       "avg_basket_size": { 
         "avg": {
           "field": "total_quantity"
         }
       }
     }
   }
 }
}

上述请求首先按照order_date字段按天进行了分组，然后针对每天的订单，继续计算sum, cardinality, avg聚合。

其中，cardinality会计算每天不重复的客户个数（distinct）。

上述请求返回结构如下所示：

{
  "took": 119,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 4675,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "daily_sales": {
      "buckets": [
        {
          "key_as_string": "2024-11-14",
          "key": 1731542400000,
          "doc_count": 146,
          "unique_customers": { 
            "value": 42
          },
          "revenue": { 
            "value": 10578.53125
          },
          "avg_basket_size": { 
            "value": 2.1780821917808217
          }
        },
        {
          "key_as_string": "2024-11-15",
          "key": 1731628800000,
          "doc_count": 153,
          "unique_customers": {
            "value": 44
          },
          "revenue": {
            "value": 10448
          },
          "avg_basket_size": {
            "value": 2.183006535947712
          }
        },
        {
          "key_as_string": "2024-11-16",
          "key": 1731715200000,
          "doc_count": 143,
          "unique_customers": {
            "value": 45
          },
          "revenue": {
            "value": 10283.484375
          },
          "avg_basket_size": {
            "value": 2.111888111888112
          }
        },
        {
          "key_as_string": "2024-11-17",
          "key": 1731801600000,
          "doc_count": 140,
          "unique_customers": {
            "value": 42
          },
          "revenue": {
            "value": 10145.5234375
          },
          "avg_basket_size": {
            "value": 2.142857142857143
          }
        },
        {
          "key_as_string": "2024-11-18",
          "key": 1731888000000,
          "doc_count": 139,
          "unique_customers": {
            "value": 42
          },
          "revenue": {
            "value": 12012.609375
          },
          "avg_basket_size": {
            "value": 2.158273381294964
          }
        },
        {
          "key_as_string": "2024-11-19",
          "key": 1731974400000,
          "doc_count": 157,
          "unique_customers": {
            "value": 43
          },
          "revenue": {
            "value": 11009.45703125
          },
          "avg_basket_size": {
            "value": 2.0955414012738856
          }
        },
        {
          "key_as_string": "2024-11-20",
          "key": 1732060800000,
          "doc_count": 145,
          "unique_customers": {
            "value": 44
          },
          "revenue": {
            "value": 10720.59375
          },
          "avg_basket_size": {
            "value": 2.179310344827586
          }
        },
        {
          "key_as_string": "2024-11-21",
          "key": 1732147200000,
          "doc_count": 152,
          "unique_customers": {
            "value": 43
          },
          "revenue": {
            "value": 11185.3671875
          },
          "avg_basket_size": {
            "value": 2.1710526315789473
          }
        },
        {
          "key_as_string": "2024-11-22",
          "key": 1732233600000,
          "doc_count": 163,
          "unique_customers": {
            "value": 44
          },
          "revenue": {
            "value": 13560.140625
          },
          "avg_basket_size": {
            "value": 2.2576687116564416
          }
        },
        {
          "key_as_string": "2024-11-23",
          "key": 1732320000000,
          "doc_count": 141,
          "unique_customers": {
            "value": 45
          },
          "revenue": {
            "value": 9884.78125
          },
          "avg_basket_size": {
            "value": 2.099290780141844
          }
        },
        {
          "key_as_string": "2024-11-24",
          "key": 1732406400000,
          "doc_count": 151,
          "unique_customers": {
            "value": 44
          },
          "revenue": {
            "value": 11075.65625
          },
          "avg_basket_size": {
            "value": 2.0927152317880795
          }
        },
        {
          "key_as_string": "2024-11-25",
          "key": 1732492800000,
          "doc_count": 143,
          "unique_customers": {
            "value": 41
          },
          "revenue": {
            "value": 10323.8515625
          },
          "avg_basket_size": {
            "value": 2.167832167832168
          }
        },
        {
          "key_as_string": "2024-11-26",
          "key": 1732579200000,
          "doc_count": 143,
          "unique_customers": {
            "value": 44
          },
          "revenue": {
            "value": 10369.546875
          },
          "avg_basket_size": {
            "value": 2.167832167832168
          }
        },
        {
          "key_as_string": "2024-11-27",
          "key": 1732665600000,
          "doc_count": 142,
          "unique_customers": {
            "value": 46
          },
          "revenue": {
            "value": 11711.890625
          },
          "avg_basket_size": {
            "value": 2.1971830985915495
          }
        },
        {
          "key_as_string": "2024-11-28",
          "key": 1732752000000,
          "doc_count": 161,
          "unique_customers": {
            "value": 43
          },
          "revenue": {
            "value": 12612.6640625
          },
          "avg_basket_size": {
            "value": 2.1180124223602483
          }
        },
        {
          "key_as_string": "2024-11-29",
          "key": 1732838400000,
          "doc_count": 144,
          "unique_customers": {
            "value": 42
          },
          "revenue": {
            "value": 10176.87890625
          },
          "avg_basket_size": {
            "value": 2.0347222222222223
          }
        },
        {
          "key_as_string": "2024-11-30",
          "key": 1732924800000,
          "doc_count": 157,
          "unique_customers": {
            "value": 43
          },
          "revenue": {
            "value": 11480.33203125
          },
          "avg_basket_size": {
            "value": 2.159235668789809
          }
        },
        {
          "key_as_string": "2024-12-01",
          "key": 1733011200000,
          "doc_count": 158,
          "unique_customers": {
            "value": 42
          },
          "revenue": {
            "value": 11533.265625
          },
          "avg_basket_size": {
            "value": 2.0822784810126582
          }
        },
        {
          "key_as_string": "2024-12-02",
          "key": 1733097600000,
          "doc_count": 144,
          "unique_customers": {
            "value": 43
          },
          "revenue": {
            "value": 10499.8125
          },
          "avg_basket_size": {
            "value": 2.201388888888889
          }
        },
        {
          "key_as_string": "2024-12-03",
          "key": 1733184000000,
          "doc_count": 151,
          "unique_customers": {
            "value": 40
          },
          "revenue": {
            "value": 12111.6875
          },
          "avg_basket_size": {
            "value": 2.172185430463576
          }
        },
        {
          "key_as_string": "2024-12-04",
          "key": 1733270400000,
          "doc_count": 145,
          "unique_customers": {
            "value": 40
          },
          "revenue": {
            "value": 10530.765625
          },
          "avg_basket_size": {
            "value": 2.0965517241379312
          }
        },
        {
          "key_as_string": "2024-12-05",
          "key": 1733356800000,
          "doc_count": 157,
          "unique_customers": {
            "value": 43
          },
          "revenue": {
            "value": 11872.5625
          },
          "avg_basket_size": {
            "value": 2.1464968152866244
          }
        },
        {
          "key_as_string": "2024-12-06",
          "key": 1733443200000,
          "doc_count": 158,
          "unique_customers": {
            "value": 42
          },
          "revenue": {
            "value": 12109.453125
          },
          "avg_basket_size": {
            "value": 2.151898734177215
          }
        },
        {
          "key_as_string": "2024-12-07",
          "key": 1733529600000,
          "doc_count": 153,
          "unique_customers": {
            "value": 42
          },
          "revenue": {
            "value": 11057.40625
          },
          "avg_basket_size": {
            "value": 2.111111111111111
          }
        },
        {
          "key_as_string": "2024-12-08",
          "key": 1733616000000,
          "doc_count": 165,
          "unique_customers": {
            "value": 42
          },
          "revenue": {
            "value": 13095.609375
          },
          "avg_basket_size": {
            "value": 2.1818181818181817
          }
        },
        {
          "key_as_string": "2024-12-09",
          "key": 1733702400000,
          "doc_count": 153,
          "unique_customers": {
            "value": 41
          },
          "revenue": {
            "value": 12574.015625
          },
          "avg_basket_size": {
            "value": 2.2287581699346406
          }
        },
        {
          "key_as_string": "2024-12-10",
          "key": 1733788800000,
          "doc_count": 158,
          "unique_customers": {
            "value": 42
          },
          "revenue": {
            "value": 11188.1875
          },
          "avg_basket_size": {
            "value": 2.151898734177215
          }
        },
        {
          "key_as_string": "2024-12-11",
          "key": 1733875200000,
          "doc_count": 160,
          "unique_customers": {
            "value": 42
          },
          "revenue": {
            "value": 12117.65625
          },
          "avg_basket_size": {
            "value": 2.20625
          }
        },
        {
          "key_as_string": "2024-12-12",
          "key": 1733961600000,
          "doc_count": 159,
          "unique_customers": {
            "value": 45
          },
          "revenue": {
            "value": 11558.25
          },
          "avg_basket_size": {
            "value": 2.1823899371069184
          }
        },
        {
          "key_as_string": "2024-12-13",
          "key": 1734048000000,
          "doc_count": 152,
          "unique_customers": {
            "value": 45
          },
          "revenue": {
            "value": 11921.1171875
          },
          "avg_basket_size": {
            "value": 2.289473684210526
          }
        },
        {
          "key_as_string": "2024-12-14",
          "key": 1734134400000,
          "doc_count": 142,
          "unique_customers": {
            "value": 45
          },
          "revenue": {
            "value": 11135.03125
          },
          "avg_basket_size": {
            "value": 2.183098591549296
          }
        }
      ]
    }
  }
}

pipeline aggregation demo

可以对其他聚合的返回结果应用pipeline aggregation，示例如下：

GET kibana_sample_data_ecommerce/_search
{
  "size": 0,
  "aggs": {
    "daily_sales": {
      "date_histogram": {
        "field": "order_date",
        "calendar_interval": "day"
      },
      "aggs": {
        "daily_revenue": {  
          "sum": {
            "field": "taxful_total_price"
          }
        },
        "smoothed_revenue": { 
          "moving_fn": { 
            "buckets_path": "daily_revenue", 
            "window": 3, 
            "script": "MovingFunctions.unweightedAvg(values)" 
          }
        }
      }
    }
  }
}

上述示例中，aggs.daily_sales.aggs.smoothed_revenue针对aggs.daily_sales.aggs.daily_revenue的聚合结果进行了聚合操作，

上述返回结果如下所示：

{
  "took": 13,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 4675,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "daily_sales": {
      "buckets": [
        {
          "key_as_string": "2024-11-14T00:00:00.000Z",  
          "key": 1731542400000,
          "doc_count": 146, 
          "daily_revenue": { 
            "value": 10578.53125
          },
          "smoothed_revenue": { 
            "value": null
          }
        },
        {
          "key_as_string": "2024-11-15T00:00:00.000Z",
          "key": 1731628800000,
          "doc_count": 153,
          "daily_revenue": {
            "value": 10448
          },
          "smoothed_revenue": { 
            "value": 10578.53125
          }
        },
        {
          "key_as_string": "2024-11-16T00:00:00.000Z",
          "key": 1731715200000,
          "doc_count": 143,
          "daily_revenue": {
            "value": 10283.484375
          },
          "smoothed_revenue": {
            "value": 10513.265625
          }
        },
        {
          "key_as_string": "2024-11-17T00:00:00.000Z",
          "key": 1731801600000,
          "doc_count": 140,
          "daily_revenue": {
            "value": 10145.5234375
          },
          "smoothed_revenue": {
            "value": 10436.671875
          }
        },
        {
          "key_as_string": "2024-11-18T00:00:00.000Z",
          "key": 1731888000000,
          "doc_count": 139,
          "daily_revenue": {
            "value": 12012.609375
          },
          "smoothed_revenue": {
            "value": 10292.3359375
          }
        },
        {
          "key_as_string": "2024-11-19T00:00:00.000Z",
          "key": 1731974400000,
          "doc_count": 157,
          "daily_revenue": {
            "value": 11009.45703125
          },
          "smoothed_revenue": {
            "value": 10813.872395833334
          }
        },
        {
          "key_as_string": "2024-11-20T00:00:00.000Z",
          "key": 1732060800000,
          "doc_count": 145,
          "daily_revenue": {
            "value": 10720.59375
          },
          "smoothed_revenue": {
            "value": 11055.86328125
          }
        },
        {
          "key_as_string": "2024-11-21T00:00:00.000Z",
          "key": 1732147200000,
          "doc_count": 152,
          "daily_revenue": {
            "value": 11185.3671875
          },
          "smoothed_revenue": {
            "value": 11247.553385416666
          }
        },
        {
          "key_as_string": "2024-11-22T00:00:00.000Z",
          "key": 1732233600000,
          "doc_count": 163,
          "daily_revenue": {
            "value": 13560.140625
          },
          "smoothed_revenue": {
            "value": 10971.805989583334
          }
        },
        {
          "key_as_string": "2024-11-23T00:00:00.000Z",
          "key": 1732320000000,
          "doc_count": 141,
          "daily_revenue": {
            "value": 9884.78125
          },
          "smoothed_revenue": {
            "value": 11822.033854166666
          }
        },
        {
          "key_as_string": "2024-11-24T00:00:00.000Z",
          "key": 1732406400000,
          "doc_count": 151,
          "daily_revenue": {
            "value": 11075.65625
          },
          "smoothed_revenue": {
            "value": 11543.4296875
          }
        },
        {
          "key_as_string": "2024-11-25T00:00:00.000Z",
          "key": 1732492800000,
          "doc_count": 143,
          "daily_revenue": {
            "value": 10323.8515625
          },
          "smoothed_revenue": {
            "value": 11506.859375
          }
        },
        {
          "key_as_string": "2024-11-26T00:00:00.000Z",
          "key": 1732579200000,
          "doc_count": 143,
          "daily_revenue": {
            "value": 10369.546875
          },
          "smoothed_revenue": {
            "value": 10428.096354166666
          }
        },
        {
          "key_as_string": "2024-11-27T00:00:00.000Z",
          "key": 1732665600000,
          "doc_count": 142,
          "daily_revenue": {
            "value": 11711.890625
          },
          "smoothed_revenue": {
            "value": 10589.684895833334
          }
        },
        {
          "key_as_string": "2024-11-28T00:00:00.000Z",
          "key": 1732752000000,
          "doc_count": 161,
          "daily_revenue": {
            "value": 12612.6640625
          },
          "smoothed_revenue": {
            "value": 10801.763020833334
          }
        },
        {
          "key_as_string": "2024-11-29T00:00:00.000Z",
          "key": 1732838400000,
          "doc_count": 144,
          "daily_revenue": {
            "value": 10176.87890625
          },
          "smoothed_revenue": {
            "value": 11564.700520833334
          }
        },
        {
          "key_as_string": "2024-11-30T00:00:00.000Z",
          "key": 1732924800000,
          "doc_count": 157,
          "daily_revenue": {
            "value": 11480.33203125
          },
          "smoothed_revenue": {
            "value": 11500.477864583334
          }
        },
        {
          "key_as_string": "2024-12-01T00:00:00.000Z",
          "key": 1733011200000,
          "doc_count": 158,
          "daily_revenue": {
            "value": 11533.265625
          },
          "smoothed_revenue": {
            "value": 11423.291666666666
          }
        },
        {
          "key_as_string": "2024-12-02T00:00:00.000Z",
          "key": 1733097600000,
          "doc_count": 144,
          "daily_revenue": {
            "value": 10499.8125
          },
          "smoothed_revenue": {
            "value": 11063.4921875
          }
        },
        {
          "key_as_string": "2024-12-03T00:00:00.000Z",
          "key": 1733184000000,
          "doc_count": 151,
          "daily_revenue": {
            "value": 12111.6875
          },
          "smoothed_revenue": {
            "value": 11171.13671875
          }
        },
        {
          "key_as_string": "2024-12-04T00:00:00.000Z",
          "key": 1733270400000,
          "doc_count": 145,
          "daily_revenue": {
            "value": 10530.765625
          },
          "smoothed_revenue": {
            "value": 11381.588541666666
          }
        },
        {
          "key_as_string": "2024-12-05T00:00:00.000Z",
          "key": 1733356800000,
          "doc_count": 157,
          "daily_revenue": {
            "value": 11872.5625
          },
          "smoothed_revenue": {
            "value": 11047.421875
          }
        },
        {
          "key_as_string": "2024-12-06T00:00:00.000Z",
          "key": 1733443200000,
          "doc_count": 158,
          "daily_revenue": {
            "value": 12109.453125
          },
          "smoothed_revenue": {
            "value": 11505.005208333334
          }
        },
        {
          "key_as_string": "2024-12-07T00:00:00.000Z",
          "key": 1733529600000,
          "doc_count": 153,
          "daily_revenue": {
            "value": 11057.40625
          },
          "smoothed_revenue": {
            "value": 11504.260416666666
          }
        },
        {
          "key_as_string": "2024-12-08T00:00:00.000Z",
          "key": 1733616000000,
          "doc_count": 165,
          "daily_revenue": {
            "value": 13095.609375
          },
          "smoothed_revenue": {
            "value": 11679.807291666666
          }
        },
        {
          "key_as_string": "2024-12-09T00:00:00.000Z",
          "key": 1733702400000,
          "doc_count": 153,
          "daily_revenue": {
            "value": 12574.015625
          },
          "smoothed_revenue": {
            "value": 12087.489583333334
          }
        },
        {
          "key_as_string": "2024-12-10T00:00:00.000Z",
          "key": 1733788800000,
          "doc_count": 158,
          "daily_revenue": {
            "value": 11188.1875
          },
          "smoothed_revenue": {
            "value": 12242.34375
          }
        },
        {
          "key_as_string": "2024-12-11T00:00:00.000Z",
          "key": 1733875200000,
          "doc_count": 160,
          "daily_revenue": {
            "value": 12117.65625
          },
          "smoothed_revenue": {
            "value": 12285.9375
          }
        },
        {
          "key_as_string": "2024-12-12T00:00:00.000Z",
          "key": 1733961600000,
          "doc_count": 159,
          "daily_revenue": {
            "value": 11558.25
          },
          "smoothed_revenue": {
            "value": 11959.953125
          }
        },
        {
          "key_as_string": "2024-12-13T00:00:00.000Z",
          "key": 1734048000000,
          "doc_count": 152,
          "daily_revenue": {
            "value": 11921.1171875
          },
          "smoothed_revenue": {
            "value": 11621.364583333334
          }
        },
        {
          "key_as_string": "2024-12-14T00:00:00.000Z",
          "key": 1734134400000,
          "doc_count": 142,
          "daily_revenue": {
            "value": 11135.03125
          },
          "smoothed_revenue": {
            "value": 11865.674479166666
          }
        }
      ]
    }
  }
}

66 KiB Raw Blame History Unescape Escape

ElasticSearch

简介

用例场景

安装

add elasticsearch GPG keys

Add Elasticsearch 8.x APT Repository

install elastic search

Indices, documents, and fields

Documents and fields

metadata field

Mappings and data types

向ElasticSearch中添加数据

General content

Timestamped data

查询和分析数据

Rest Api

query language

Query DSL

Query DSL分析

ES | QL

使用ElasticSearch Api索引和查询数据

创建索引

向索引中添加数据

向索引中添加单个document

向索引中添加多个document

定义mapping和data type

使用dynamic mapping

手动指定索引的mapping

将dynamic mapping和手动指定mapping相结合

搜索索引

搜索所有文档

match请求

删除索引

全文本搜索和过滤

创建索引

multi-field

ignore_above

批量插入数据

执行full-text search

match

track total hits

max score

_score

contains all terms

应匹配terms的最小个数

search across mutli fields

filter & exact matches

bool query

date range

exact match

组合搜索

must_not

使用Query DSL分析数据

get metrics

计算订单的平均值

在单个请求中计算订单的多个metrics

根据category对订单进行分组

doc_count_error_upper_bound

date_histogram

Calendar intervals

compare category performance

analyze daily sale performance

pipeline aggregation demo

66 KiB

Raw Blame History

`match`

`must_not`