List of changes impacting docker-elk:
- [logstash]: The output to Elasticsearch is handled as a data stream.
Starting with v8.0.0, the `elasticsearch` output for Logstash sends
log data to a data stream instead of `logstash-*` indices by default.
The name of the default data stream is `logs-generic-default`.
docker-elk remains unopinionated and simply uses Elastic's defaults
like it always has, so users who prefer to retain the old behaviour
need to explicitly opt-out of data streams in their Logstash
pipelines.
Refs:
- https://www.elastic.co/guide/en/elasticsearch/reference/current/data-streams.html
- https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-data-streams
- [logstash]: The (legacy) monitoring data collection is now disabled.
This feature was deprecated since v7.9.0, and removed in v8.0.0.
Ref: https://www.elastic.co/guide/en/logstash/current/monitoring-internal-collection-legacy.html
- [kibana]: An index pattern for `logs-*` indices is automatically
created.
It used to be required to manually create an index pattern for indices
managed by Logstash, even when using the default Logstash indices.
This is no longer the case since the output data is now being handled
as a data stream, and Kibana automatically creates index patterns for
these.
- [elasticsearch]: The command line tool `elasticsearch-setup-passwords`
was deprecated in favour of a new `elasticsearch-reset-password` tool.
Passwords for built-in users must now be generated one by one.
Ref: https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-passwords.html
- [enterprise-search]: Kibana is now the new management interface, and
the only one available moving forward.
The old standalone Enterprise Search interface was removed in v8.0.0.
Ref: https://www.elastic.co/guide/en/enterprise-search/current/user-interfaces.html
The first attempt occasionally fails, probably due to a race with the
listener's startup.
Additionally, perform minor adjustments to the output written to stderr
to ensure it is properly flushed in the CI environment.
Since we restart Logstash to enable the UDP input prior to running this
test, we should also ensure it has successfully come back, otherwise we
start searching for the expected documents in Elasticsearch way too
early and often run into timeouts.
ci: Update Enterprise Search test for v3 index
App Search API keys are now encrypted in Elasticsearch, so we simply
ensure they exist and don't try to read their value.
Prevents the occasional connection timeout when the tool randomly picks
the IP of the ingress interface (usually 10.0.0.x/24) instead of the elk
interface (usually 10.0.1.x/24) in Swarm mode.
