Release Notes

Release cadence
Stable Releases
Loki releases (this includes Promtail, Loki Canary, etc.) use the following
naming scheme: 
MAJOR.
MINOR.
PATCH.
MAJOR (roughly once a year): these releases include large new features and possible backwards-compatibility breaks.
MINOR (roughly once a quarter): these releases include new features which generally do not break backwards-compatibility, but from time to time we might introduce minor breaking changes, and we will specify these in our upgrade docs.
PATCH (roughly once or twice a month): these releases include bug and security fixes which do not break backwards-compatibility.
{{< admonition type="note" >}}
While our naming scheme resembles Semantic Versioning, at this time we do not strictly follow its
guidelines to the letter. Our goal is to provide regular releases that are as stable as possible, and we take backwards-compatibility
seriously. As with any software, always read the release notes and the upgrade guide whenever
choosing a new version of Loki to install.
{{< /admonition >}}
New releases are based of a weekly release which we have vetted for stability over a number of weeks.
We strongly recommend keeping up-to-date with patch releases as they are released. We post updates of new releases in the 
#loki channel
of our Slack community.
You can find all of our releases on GitHub and on Docker Hub.
Weekly Releases
Every Monday morning, we create a new "weekly" release from the tip of the 
main branch.
These releases use the following naming scheme:
weekly-k<week-number> where 
<week-number> is the number of weeks since we began this process (2020-07-06).
These weekly releases are deployed across our Grafana Cloud Logs fleet of instances. We test these releases for stability
by deploying them through development, pre-production, and production instances.
Generally these weekly releases are considered stable enough to run, but we provide zero stability guarantees and these
releases should not be run in production unless you are willing to tolerate some risk.
You can find these releases on Docker Hub.
Which release will my merged PR be part of?
Once your PR is merged to 
main, you can expect it to become available in the next week's
weekly release. To find out which stable or weekly releases a commit is included in, use the following tool:
tools/which-release.sh
For example, this PR was merged into the commit named 
d434e80. Using the tool above, we can see that is part of release 2.8 and several weekly releases:
$ ./tools/which-release.sh d434e80                                 
Commit was found in the following releases:
  release-2.8.x
Commit was found in the following weekly builds:
  k136
  k137
  k138
  k139
  k140
  k141
  k142

V?.?
Grafana Labs is excited to announce the release of Loki ?.?.? Here's a summary of new enhancements and important fixes:
:warning: This a placeholder for the next release. Clean up all features listed below
Features and enhancements
Upgrade Considerations
Bug fixes

v3.0
Grafana Labs and the Loki team are excited to announce the release of Loki 3.0. Here's a summary of new enhancements and important fixes.
For a full list of all changes and fixes, refer to the CHANGELOG.
Features and enhancements
{{< admonition type="note" >}}
Note that Loki 3.0 defaults to using the v13 schema. All of the latest features are built against TSDB and the v13 Schema. This version of the schema is compatible with both Loki 2.9.x and Loki 3.0. The main change is to add support for Structured Metadata which is used by the new OTLP native endpoint and is enabled by default.
{{< /admonition >}}
Key features in Loki 3.0.0 include the following:
Query acceleration with Bloom filters (experimental): This is designed to speed up filter queries, with best results for queries that are looking for a specific text string like an error message or UUID. For more information, refer to Query acceleration with Blooms.
Native OpenTelemetry Support: A simplified ingestion pipeline (Loki Exporter no longer needed) and a more intuitive query experience for OTel logs. For more information, refer to the OTel documentation.
Helm charts: A major upgrade to the Loki helm chart introduces support for 
Distributed mode (also known as microservices mode), includes memcached by default, and includes several updates to configurations to improve Loki operations.
Pattern match filter: LogQL now supports two new pattern match filter operators. You can match any word with just one control character and it is simpler and 10x faster than using regex.
Caching updates: This release includes multiple updates to caching to improve performance, add new configuration options and support for new features, deprecate features no longer needed, and add automatic background checks.
Lambda/Promtail: Support dropping labels (#10755) (ec54c72).
Profiling integration: Added profiling integrations to tracing instrumentation to allow getting a profile for a single request.
Docs improvements: All the Getting Started topics have been revised, including a new Quickstart to help new users get up and running with Loki faster. The Storage, Configuration Reference, and API documentation have been updated to reflect deprecated and removed code, configuration options, and API endpoints.
Deprecations
One of the focuses of Loki 3.0 was cleaning up unused code and old features that had been previously deprecated but not removed. Loki 3.0 removes a number of previous deprecations and introduces some new deprecations. Some of the main areas with changes include:
Deprecated storage options including the deprecation of the BoltDB store.
Deprecated configuration options.
API endpoint deprecations.
To learn more about breaking changes in this release, refer to the Upgrade guide.
{{< docs/shared source="alloy" lookup="agent-deprecation.md" version="next" >}}
Upgrade Considerations
The path from 2.9 to 3.0 includes several breaking changes. For important upgrade guidance, refer to the Upgrade Guide and the separate Helm Upgrade Guide.
Bug fixes
3.0.1 (2024-08-09)
deps: Bumped dependencies versions to resolve CVEs (#13833) (e13011d).
Fixed nil pointer dereference in bloomstore initialization (#12869) (167b468), closes #12270.
3.0.0 (2024-04-08)
All lifecycler configurations reference a valid IPv6 address and port combination (#11121) (6385b19).
deps: Update github.com/c2h5oh/datasize digest to 859f65c (main) (#10820) (c66ffd1).
deps: Update github.com/docker/go-plugins-helpers digest to 6eecb7b (main) (#10826) (fb9c496).
deps: Update github.com/grafana/gomemcache digest to 6947259 (main) (#10836) (2327789).
deps: Update github.com/grafana/loki/pkg/push digest to 583aa28 (main) (#10842) (02d9418).
deps: Update github.com/grafana/loki/pkg/push digest to cfc4f0e (main) (#10946) (d27c4d2).
deps: Update github.com/grafana/loki/pkg/push digest to e523809 (main) (#11107) (09cb9ae).
deps: Update github.com/joncrlsn/dque digest to c2ef48c (main) (#10947) (1fe4885).
deps: Update module google.golang.org/grpc [security] (main) (#11031) (0695424).
helm: Bump nginx-unprivilege to fix CVE (#10754) (dbf7dd4).
helm: Sidecar configuration for Backend (#10603) (c29ba97).
lambda-promtail Fix panic in lambda-promtail due to mishandling of empty DROP_LABELS env var. (#11074).
loki: Respect query matcher in ingester when getting label values (#10375) (85e2e52).
loki Generate tsdb_shipper storage_config even if using_boltdb_shipper is false (#11195).
loki Do not reflect label names in request metrics' "route" label. (11551).
loki Fix duplicate logs from docker containers. (#11563).
loki Ruler: Fixed a panic that can be caused by concurrent read-write access of tenant configs when there are a large amount of rules. (#11601).
loki Fixed regression adding newlines to HTTP error response bodies which may break client integrations. (#11606).
loki Log results cache: compose empty response based on the request being served to avoid returning incorrect limit or direction. (#11657).
loki Fix semantics of label parsing logic of metrics and logs queries. Both only parse the first label if multiple extractions into the same label are requested. (#11587).
loki Background Cache: Fixes a bug that is causing the background queue size to be incremented twice for each enqueued item. (#11776).
loki: Parsing: String array elements were not being parsed correctly in JSON processing (#11921).
promtail: Correctly parse list of drop stage sources from YAML (#10848) (f51ee84).
promtail: Prevent panic due to duplicate metric registration after reloaded (#10798) (47e2c58).
promtail: Fix Promtail excludepath not evaluated on newly added files. (#9831).
tools/lambda-promtail: Do not evaluate empty string for drop_labels (#11074) (94169a0).

v3.1
Grafana Labs and the Loki team are excited to announce the release of Loki 3.1. Here's a summary of new enhancements and important fixes.
For a full list of all changes and fixes, refer to the CHANGELOG.
Features and enhancements
Key features in Loki 3.1.0 include the following:
Query acceleration with Bloom filters (experimental): Continued work on improving the new Bloom filters feature introduced in Loki 3.0. This is designed to speed up filter queries, with best results for queries that are looking for a specific text string like an error message or UUID. For more information, refer to Query acceleration with Blooms.
Helm: Update Helm chart to support distributed mode and 3.0 (#12067) (79b876b). Add pattern ingester support in SSD mode (#12685) (19bfef4). Enable log volume endpoint by default in Helm (#12690) (e39677f). Allow extraObject items as multiline strings (#12397) (af5be90).Support for PVC Annotations for Non-Distributed Modes (#12023) (efdae3d).
LogQL: Support negative numbers in LogQL (#13091) (6df81db). Improve performance of 
first_over_time and 
last_over_time queries by sharding them (#11605) (f66172e). Improve syntax parser for pattern (#12489) (48dae44).
Loki: Add ability to disable AWS S3 dual stack endpoints usage (#13795) (464ac73).
lokitool: Add 
lokitool to replace 
cortextool. (#12166) (7b7d3d4). Introduce 
index audit to 
lokitool (#13008) (47f0236).
Explore Logs: Explore Logs, which lets you explore your Loki data without writing LogQL queries, is now available in public preview. If you are a Grafana Cloud user, you can access Explore Logs in the Grafana Cloud main navigation menu. If you are not a Grafana Cloud user, you can install the Explore Logs plugin.  For more information, refer to the Explore Logs documentation.
Docs: Added a video to the Getting Started demo and updated for Grafana Alloy. Added an interactive sandbox to the Loki Quickstart tutorial. Updated the documentation for the SSD and microservices deployment modes using the Helm charts.  Documented the new meta-monitoring Helm chart.
Other improvements include the following:
API: Add step param to Patterns Query API (#12703) (7b8533e).
API: Expose optional label matcher for label names API (#11982) (8084259).
cache: Add 
Cache-Control: no-cache support for Loki instant queries. (#12896) (88e545f).
canary: Add test to check query results with and without cache. (#13104) (71507a2).
distributor: Optimize log parsing performance by using unsafe package (#13223) (9f31b25).
Enable log volume endpoint by default (#12628) (397aa56).
ingester: Add backoff to flush op (#13140) (9767807).
ingester: Add ingester_chunks_flush_failures_total (#12925) (cc3694e).
ingester: add profile tagging to ingester (#13068) (00d3c7a).
ingester: add recalculateOwnedStreams to check stream ownership if the ring is changed (#13103) (e7689b2).
ingester: Add profile tagging to ingester (#13068) (00d3c7a).
ingester: Include the stream we failed to create in the stream limit error message (#12437) (ec81991).
ingester: New stream count limiter (#13006) (1111595).
Lambda-Promtail: Parameterise the MaximumEventAgeInSeconds, LogGroupName, and IAMRoleName for Lambda-Promtail CloudFormation template (#12728) (8892dc8).
loki/main.go: Log which config file path is used on startup (#12985) (7a3338e).
metamonitoring: Add a version of the mixin dashboards for meta monitoring (#12700) (ec1a057).
Promtail: Added support to install wget on Promtail docker image to support docker healthcheck (#11711) (ffe684c).
Promtail: Support of RFC3164 aka BSD Syslog (#12810) (be41525).
querier: Split gRPC client into two. (#12726) (7b6f057).
reporting: Report CPU usage (#12970) (87288d3).
Split detected fields queries (#12491) (6c33809).
TSDB: Add fast-path to 
inversePostingsForMatcher (#12679) (402d1d7).
Deprecations
One of the focuses of Loki 3.0 was cleaning up unused code and old features that had been previously deprecated but not removed. Loki 3.0 removed a number of previous deprecations and introduces some new deprecations. Some of the main areas with changes include:
Deprecated storage options including the deprecation of the BoltDB store.
Deprecated configuration options.
API endpoint deprecations.
To learn more about breaking changes in this release, refer to the Upgrade guide.
{{< docs/shared source="alloy" lookup="agent-deprecation.md" version="next" >}}
Upgrade Considerations
The path from 2.9 to 3.0 included several breaking changes. For important upgrade guidance, refer to the Upgrade Guide and the separate Helm Upgrade Guide.
BREAKING CHANGE Update Helm chart to support distributed mode and 3.0 (#12067).
{{< admonition type="important" >}}
Out of an abundance of caution, we advise that users with Loki or Grafana Enterprise Logs (GEL) deployments on AWS upgrade their Helm charts or change the names of their buckets, as outlined in the solutions and mitigations section of this blog post.
{{< /admonition >}}
Bug fixes
3.1.2 (2024-10-17)
config: Copy Alibaba and IBM object storage configuration from common (#14316) (7184d45).
docker-compose: upgrade loki and grafana production image tags to 3.1.1 (#14025) (36fe29e).
logql: updated JSONExpressionParser not to unescape extracted values if it is JSON object. (backport release-3.1.x) (#14503) (759f9c8).
promtail: Revert build image to Debian Bullseye to fix libc version issue in Promtail (#14387) (05b6a65).
storage: have GetObject check for canceled context (backport release-3.1.x) (#14421) (f3d69ff).
3.1.1 (2024-08-08)
deps: Bumped dependencies versions to resolve CVEs (#13789) (34206cd).
3.1.0 (2024-07-02)
blooms: Fix duplicate enqueue item problem in bloom download queue when do sync download (#13114) (f98ff7f).
blooms: Add a missing 
continue in fuse which may cause incorrect bloom test result (#12650) (0d1ebeb).
blooms: Allow cluster label override in bloom dashboards (#13012) (987e551).
blooms: bloomshipper no longer returns empty metas on fetch (#13130) (ad279e5).
blooms: Clean block directories recursively on startup (#12895) (7b77e31).
blooms: Correctly return unfiltered chunks for series that are not mapped to any block (#12774) (c36b114).
blooms: Deduplicate filtered series and chunks (#12791) (3bf2d1f).
blooms: Disable metas cache on bloom gateway (#12959) (00bdd2f).
blooms: Do not fail requests when fetching metas from cache fails (#12838) (667076d).
blooms: Don't break iterator conventions (#12808) (1665e85).
blooms: Fix 
partitionSeriesByDay function (#12900) (738c274).
blooms: Fix a regression introduced with #12774 (#12776) (ecefb49).
blooms: Fix findGaps when ownership goes to MaxUInt64 and that is covered by existing meta (#12558) (0ee2a61).
blooms: Fully deduplicate chunks from FilterChunkRef responses (#12807) (a0f358f).
blooms: Handle not found metas gracefully (#12853) (37c8822).
blooms: Reset error on LazyBloomIter.Seek (#12806) (76ba24e).
blooms: Defer closing blocks iter after checking error from loadWorkForGap (#12934) (cb1f5d9).
blooms: Fix bloom deleter PR after merge (#13167) (c996349).
blooms: Fix incorrect sorting of chunks in bloom-filtered response since 
ChunkRef.Cmp method is used in reverse (#12999) (670cd89).
blooms: Remove unused parameter causing lint error (#12801) (33e82ec).
codec not initialized in downstream roundtripper (#12873) (b6049f6).
Change log level since this is a known case (#13029) (ca030a5).
Close res body (#12444) (616977a).
Correct initialization of a few slices (#12674) (0eba448).
distributor: add detected_level info when the info word appears on log message (#13218) (c9bfa3e).
docs: Broken link in getting started readme (#12736) (425a2d6).
docs: Move Promtail configuration to the correct doc (#12737) (1161846).
docs: Updated Loki OTLP Ingest Configuration (#12648) (ff88f3c).
Explore logs: Make detected fields work for both json and proto (#12682) (f68d1f7).
Fix errors reported by the race detector (#13174) (2b19dac), closes #8586.
Fix for how the loop sync is done (#12941) (5cd850e).
Fix log level detection (#12651) (6904a65).
Fix setting of info log level when trying to detect level from log lines (#12635) (0831802).
lokitool: Fix the lokitool imports (#12673) (6dce988).
Helm: Fix indentation of query range values in Helm (#12577) (9823f20).
Helm: Add missing Helm helper loki.hpa.apiVersion (#12755) (3070ea7).
Helm: Add missing OTLP endpoint to nginx config (#12709) (8096748).
Helm: Fixes read & backend replicas settings (#12828) (d751134).
Helm: Set compactor address for distributed mode. (#12748) (521d40a).
Helm: Fix GEL image tag, bucket name and proxy URLs (#12878) (67ed2f7).
Helm: Fix query-frontend and ruler targetPort 'http-metrics' in Service template (#13024) (1ab9d27).
Helm: Fix queryScheduler servicemonitor (#12753) (8101e21).
Helm: Fixed ingress paths mapping (#12932) (5ada92b).
Helm: Only default bucket names when using MinIO (#12548) (2e32ec5).
Helm: Removed duplicate bucketNames from documentation and fixed key name 
deploymentMode (#12641) (0d8ff9e).
Helm: Ingester zoneAwareReplication (#12659) (9edb0ce).
Helm: Missing password for Loki-Canary when loki.auth_enabled is true (#12411) (68b23dc).
indexstats: Do not collect stats from "IndexStats" lookups for other query types (#12978) (1f5291a).
ingester: Replace channel check with atomic bool in tailer.send() (#12976) (4a5edf1).
ingester: Not owned stream count (#13030) (4901a5c).
ingester: Properly return http status codes from ingester to querier for RPC function calls (#13134) (691b174).
ingester: Track bytes discarded by ingester. (#12981) (88c6711).
Introduce feature flag for [last|first]_over_time sharding. (#13067) (6e45550).
Invalidate caches when pipeline wrappers are disabled (#12903) (a772ed7).
ksonnet: Do not generate RBAC for consul if you are using memberlist (#12688) (2d62fca).
Lambda-Promtail: Update s3 filename regex to allow finding of log files from AWS GovCloud regions (#12482) (7a81d26).
Lambda-Promtail: Remove Hardcoded Bucket Name from EventBridge Example CloudFormation Template (#12609) (8c18463).
LogQL: Fix multiple or filters would get wrong filtertype (#13169) (9981e9e).
Loki version prefix in Makefile (#12514) (dff72d2).
Make the TSDB filenames correctly reproducible from the identifier (#12536) (ec888ec).
metamonitoring: Add missing parentheses in meta monitoring dashboards (#12802) (151d0a5).
mixins: loki-operational.libsonnet (#12789) (51a841f).
mixins: Incorrect compactor matcher in loki-deletion dashboard mixin (#12567) (006f88c).
mixins: Fix compactor matcher in the loki-deletion dashboard (#12790) (a03846b).
mixin: Mixin generation when cluster label is changed (#12613) (1ba7a30).
mixin: dashboards $__auto fix (#12707) (91ef72f).
mixins:  Add missing log data source on loki-deletion (#13011) (1948899).
mixins: Align loki-writes mixins with loki-reads (#13022) (757b776).
mixins: Remove unnecessary disk panels for SSD read path (#13014) (8d9fb68).
mixins: Upgrade old plugin for the loki-operational dashboard. (#13016) (d3c9cec).
nix: Lambda-Promtail vendor hash (#12763) (ae180d6).
OTLP: Fix parsing of default per tenant OTLP config (#12836) (7cc9a93).
operator: Add alertmanager client config to ruler template (#13182) (6148c37).
operator: Bump golang builder to 1.21.9 (#12503) (f680ee0).
operator: Configure Loki to use virtual-host-style URLs for S3 AWS endpoints (#12469) (0084262).
operator: Improve API documentation for schema version (#13122) (3a9f50f).
operator: Use a minimum value for replay memory ceiling (#13066) (4f3ed77).
Optimize regular initialization (#12926) (a46d14f).
OTel: Map 500 errors to 503 (#13173) (b31e04e).
packaging: Require online network in systemd unit file for Loki and Promtail (#12741) (57f78b5).
Fix panics when ingester response is nil (#12946) (3cc28aa).
Promtail: Promtail race fixes (#12656) (4e04d07).
Promtail: Clean up metrics generated from logs after a config reload. (#11882) (39a7181).
Promtail: Fix bug with Promtail config reloading getting stuck indefinitely (#12795) (4d761ac).
Promtail: Fix UDP receiver on syslog transport (#10708) (a00f1f1).
Promtail: Handle docker logs when a log is split in multiple frames (#12374) (c0113db).  This changes the current Promtail behavior.  Loki has a default 
max_line_size of 256kb, but
With this fix, if the 
max_line_size config in Promtail has a value, the docker service discovery will now respect that configuration, otherwise it will use the Loki default of 256kb.
querier: Add retry middleware to the "limited" query roundtripper (#13161) (bb864b3).
query sharding: Generalize avg -> sum/count sharding using existing binop mapper (#12599) (11e7687).
regression: reverts grafana/loki #13039 to prevent use-after-free corruptions (#13162) (41c5ee2).
spans: Corrects early-close for a few spans (#12887) (93aaf29).
stores: Do not filter out chunks for store when 
From==Through and 
From==start (#13117) (d9cc513).
structured metadata: Add copyString function to symbolizer to avoid retaining memory (#13146) (86b119a).
Temporarily moving from alloy to alloy dev (#13062) (7ffe0fb).
Update to build image 0.33.2, fixes bug with Promtail windows DNS resolution (#12732) (759f42d).
Updated all dockerfiles go1.22 (#12708) (71a8f2c).
Use to the proper config names in warning messages (#12114) (4a05964).
workflows: Use an intermediate env variable in GitHub workflow (#12905) (772616c).
workflows: Don't run metric collector on forks (#12687) (7253444).

v3.2
Grafana Labs and the Loki team are excited to announce the release of Loki 3.2. Explore Logs is also now Generally Available. Upgrade to Loki/GEL 3.2 to get the best possible experience with Explore Logs. Here's a summary of new enhancements and important fixes.
For a full list of all changes and fixes, refer to the CHANGELOG.
Features and enhancements
Key features in Loki 3.2.0 include the following:
*API: The Loki APIs have been updated to fail log queries when executed on instant query endpoint (#13421) (ce71f1c), add structured metadata to 
/detected_fields API (#13604) (ce02cc2), and avoid looking up unnecessary TSDB symbols during Volume API (#13960) (7c1a849).
Query acceleration with Bloom filters (experimental): The Bloom filters feature introduced in Loki 3.0 remains experimental. Note that the Bloom Compactor component has been removed (#13969) (b75eacc) and replaced by the Bloom Planner and Bloom Builder components (#13997) (be9eb50).
distributor: Ignore empty streams in distributor if all entries fail validation (#13674) (6c4b062), and limit to block ingestion until configured date (#13958) (b5ac6a0).
Explore Logs Is now Generally Available (GA). For the best experience, you should be on Grafana 11.2 or later and Loki 3.2. This release includes enhancements to add _extracted suffix to detected fields conflicts (#13993) (ab1caea), collect and serve pre-aggregated bytes and counts (#13020) (467eb1b), and remove cardinality filter (#13652) (4f534d7).
Helm: This release includes updates to the Helm charts to make gateway container port configurable. (#13294) (05176e4) and to support alibabacloud oss in the Helm chart (#13441) (3ebab6f). It also includes a breaking change to the Helm chart to support distributed mode and 3.0 (#12067).
ingester: Ingester Stream Limit Improvements: Ingester stream limits now take into account "owned streams" and periodically update when the Ingester ring is changed. Non-owned streams are now also flushed when this update takes place. The stream limit calculation has also been updated for improved accuracy in multi-zone ingester deployments. (#13532) (ec34aaa).
lambda-promtail: Add S3 log parser support for AWS GuardDuty (#13148) (2d92fff), build lambda with zip file (#13787) (9bf08f7), and ensure messages to Kinesis are usable by refactoring parsing of KinesisEvent to match parsing of CWEvents + code cleanup (#13098) (dbfb19b).
loki: Add ability to disable AWS S3 dualstack endpoints usage(#13785) (bb257f5), not enforce max-query-bytes-read and max-querier-bytes-read in limited tripperware (#13406) (47f6ea5), and upgrade Prometheus (#13671) (b88583d).
operator: Add alert for discarded samples (#13512) (5f2a02f), add support for the volume API (#13369) (d451e23), enable leader-election (#13760) (1ba4bff), and update Loki operand to v3.1.0 (#13422) (cf5f52d).
storage: Convert WalSegmentWriter to io.ReadSeeker (#13340) (19c0509) and instrument failed chunk encoding/decoding (#13684) (5a87ccb).
docs: Add getting started video for ingesting OTel logs (#13226) (5e560f9).
Other improvements include the following:
chunks-inspect: Support structured metadata (#11506) (1834065).
exporter: Include boolean values in limit exporter (#13466) (4220737).
mempool: Replace 
sync.Mutex with 
sync.Once (#13293) (61a9854).
metrics: Collect duplicate log line metrics (#13084) (40ee766).
Deprecations
One of the focuses of Loki 3.0 was cleaning up unused code and old features that had been previously deprecated but not removed. Loki 3.0 removed a number of previous deprecations and introduces some new deprecations. Some of the main areas with changes include:
Deprecated storage options including the deprecation of the BoltDB store.
Deprecated configuration options.
API endpoint deprecations.
To learn more about breaking changes in this release, refer to the Upgrade guide.
{{< docs/shared source="alloy" lookup="agent-deprecation.md" version="next" >}}
Upgrade Considerations
For important upgrade guidance, refer to the Upgrade Guide and the separate Helm Upgrade Guide.
BREAKING CHANGE - API: Fail log queries when executed on instant query endpoint (#13421).
BREAKING CHANGE - blooms: Remove bloom compactor component (#13969).
BREAKING CHANGE - docker: Remove wget from Promtail docker image(#15101).
BREAKING CHANGE - Helm: Update Helm chart to support distributed mode and 3.0 (#12067).
BREAKING CHANGE - Helm: Fix how we set imagePullSecrets for enterprise-gateway and admin-api. (#13761) (3be5a45).
BREAKING CHANGE - jsonnet: Convert read statefulset into deployment for loki-simple-scalable (#13977).
{{< admonition type="important" >}}
Out of an abundance of caution, we advise that users with Loki or Grafana Enterprise Logs (GEL) deployments on AWS upgrade their Helm charts or change the names of their buckets, as outlined in the solutions and mitigations section of this blog post.
{{< /admonition >}}
Bug fixes
3.2.2 (2024-12-04)
BREAKING CHANGE - docker: Remove wget from Promtail docker image(#15101).
docker: Move from base-nossl to static. This PR removes the inclusion of glibc into most of the Docker images created by the Loki build system. (#15203).
logql: Updated JSONExpressionParser not to unescape extracted values if it is JSON object.  (#14499).
promtail: Switch Promtail base image from Debian to Ubuntu to fix critical security issues (#15195).
storage: Have GetObject check for canceled context. S3ObjectClient.GetObject incorrectly returned nil, 0, nil when the provided context is already canceled (#14420).
3.2.1 (2024-10-17)
config: Copy Alibaba and IBM object storage configuration from common (#14315) (32a9bc0).
logql: updated JSONExpressionParser not to unescape extracted values if it is JSON object. (backport release-3.2.x) (#14502) (e9bbaf3).
promtail: Revert build image to Debian Bullseye to fix libc version issue in Promtail (#14386) (1e913df).
storage: have GetObject check for canceled context (backport release-3.2.x) (#14422) (1b7dd95).
3.2.0 (2024-09-19)
blooms: Cleanup temp blockdir in bloom compactor (#13622) (64215e1).
blooms: Delete outdated metas during planning (#13363) (11e1976).
blooms: Ensure tokenizer cache is reset between series (#13370) (04bc3a4).
blooms: Fix eviction of multiple blockcache items (#13573) (c9950e3).
blooms: Fix panic in bloom gateway (#13303) (66f97b2).
blooms: Ignores bloom filtering errors in bounded shard query planning (#13285) (ede6941).
blooms: Improve error wrap to make ignoreNotFound work when fetching blocks (#13656) (bd20171).
blooms: Improves mempool metrics (#13283) (d36e1d5).
blooms: Minor fixes and improvements for testing in dev (#13341) (d0f56ee).
blooms: Remove backoff from notify planner (#13506) (e506995).
blooms: Remove unused arg (#13343) (fcb9b28).
blooms: Ship chunkrefs in task payload (#13677) (450bbce).
blooms: Add logging to empty bloom (#13502) (c263a68).
blooms: Skip empty blooms on reads (#13500) (bfa6955).
blooms: Suppress error from resolving server addresses for blocks (#13385) (3ac2317).
blooms: Use correct key to populate blockscache at startup (#13624) (2624a4b).
blooms: Fix log line for fingerprint not found (#13555) (aeb23bb).
blooms: Fix panic in BloomStore initialization (#13457) (5f4b8fc).
blooms: Flaky test blockPlansForGaps (#13743) (37e33d4).
blooms: Keep blocks referenced by newer metas (#13614) (784e7d5).
blooms: Lint issues after merge to main (#13326) (7e19cc7).
blooms: Use elements match in gapsBetweenTSDBsAndMetas test (#13722) (4cbe2a6).
ci: Add cleanup step into job 
dist (#13801) (217f928).
ci: Fixed release-please manifest (#13810) (f253db5).
deps: Update github.com/axiomhq/hyperloglog digest to af9851f (#13806) (67295e0).
deps: Update github.com/c2h5oh/datasize digest to aa82cc1 (#13807) (a93f38c).
deps: Update github.com/docker/go-plugins-helpers digest to 45e2431 (#13808) (e5a3994).
deps: Update github.com/grafana/jsonparser digest to ea80629 (#13814) (d5718eb).
deps: Update module github.com/aliyun/aliyun-oss-go-sdk to v2.2.10+incompatible (#13861) (6f79194).
deps: Update module github.com/azure/go-autorest/autorest/adal to v0.9.24 (#13862) (8041bd2).
deps: Update module github.com/azure/go-autorest/autorest/azure/auth to v0.5.13 (#13863) (71c4421).
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.186 (#13864) (3c0e3e2).
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.187 (#13933) (56af84d).
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.188 (#14000) (79039a2).
deps: Update module github.com/cespare/xxhash/v2 to v2.3.0 (main) (#13615) (cfc7b34).
deps: Update module github.com/datadog/sketches-go to v1.4.6 (#13865) (1f3c467).
deps: Update module github.com/docker/docker to v25.0.5+incompatible [security] (main) (#12279) (960c034).
deps: Update module github.com/docker/docker to v27.1.1+incompatible [security] (main) (#13762) (f8bf3bb).
deps: Update module github.com/docker/docker to v27.1.2+incompatible (#13872) (8ab4c20).
deps: Update module github.com/efficientgo/core to v1.0.0-rc.3 (#14001) (90f7e5f).
deps: Update module github.com/felixge/fgprof to v0.9.4 (#13870) (c68848f).
deps: Update module github.com/fsouza/fake-gcs-server to v1.47.7 (#13935) (d43b2de).
deps: Update module github.com/gogo/googleapis to v1.4.1 (#13871) (6da7eb5).
deps: Update module github.com/gorilla/mux to v1.8.1 (main) (#13618) (19b288e).
deps: Update module github.com/gorilla/websocket to v1.5.3 (#13873) (1eb8342).
deps: Update module github.com/hashicorp/consul/api to v1.29.4 (#14002) (e11b244).
deps: Update module github.com/ibm/go-sdk-core/v5 to v5.17.4 (#13892) (b6991f2).
deps: Update module github.com/ibm/ibm-cos-sdk-go to v1.11.0 (#13893) (9b7e7e9).
deps: Update module github.com/klauspost/pgzip to v1.2.6 (#13874) (fdea7a1).
deps: Update module github.com/mattn/go-ieproxy to v0.0.12 (#13876) (775bf8e).
deps: Update module github.com/ncw/swift to v2 (#13951) (246a1df).
deps: Update module github.com/oschwald/geoip2-golang to v1.11.0 (#13934) (3bebba5).
deps: Update module github.com/schollz/progressbar/v3 to v3.14.6 (#13884) (fb9cae4).
deps: Update module github.com/tonistiigi/fifo to v1 (#13952) (96b5c79).
deps: Update module github.com/workiva/go-datastructures to v1.1.5 (#13885) (d817aee).
deps: Update module golang.org/x/text to v0.17.0 (main) (#13794) (df61482).
deps: Update module golang.org/x/time to v0.6.0 (#13910) (dff00bd).
detected fields: Detected fields incorrect type bug (#13515) (f6a94d3).
detected fields: Remove query size limit for detected fields (#13423) (1fa5127).
detected labels: Response when store label values are empty (#13970) (6f99af6).
detected_labels: Add matchers to get labels from store" (#14012) (25234e8).
detected_labels: Remove limit middleware for 
detected_labels (#13643) (2642718).
docs: Fixed typo in ruler URL (#13692) (1476498).
docs: Remove trailing backtick in verify-config for Loki 3.0 (#13640) (498f29a).
Helm: Fix HPA ingester typo (#13158) (4ca9785).
Helm: Fix extraObjects (#13107) (b7fcf2b).
Helm: Fix imagePullSecrets for statefulset-results-cache (#13051) (8434b2f).
Helm: Fixed memcached and provisioner templates (#13788) (1bf9791).
Helm: Removed helm test (#13651) (ef03476).
Helm: Update yaml file 
./production/helm/loki/Chart.yaml (+1 other) (#13392) (b5b861c).
Helm: Update yaml file 
./production/helm/loki/values.yaml (+1 other) (#13426) (fc3904e).
Helm: Querier address in SingleBinary mode (#13297) (29f1ea9).
Helm: Update Loki v3 Helm statefulset-ingester.yaml template (#13118) (5b4e576).
index gateways: Do not retain span logger created with index set initialized at query time (#14027) (bd25ac2).
index gateway: Fix nil pointer dereference panic when using ruler in ring mode (#13436) (304db10).
ingester: Fix panic in ingester.go (#13557) (dbff69a).
ingester: Redo ingester profile tagging (#13239) (32097c8).
ingester: Remove tenant label tagging from profiles to reduce cardinality (#13270) (f897758).
ingester: Stream ownership check (#13314) (5ae5b31).
ingester: Support multi-zone ingesters when converting global to local limits for streams in limiter.go (#13321) (e28c15f).
ingester: Update fixed limit once streams ownership re-checked (#13231) (7ac19f0).
LOgQL: AST left circular reference result in out of memory (#13501) (6dd6b65).
LogQL: Improve execution speed for queries with label filters (#13922) (40f4f14).
LogQL: Panic when parsing and extracting JSON key values (#13790) (5ef83a7).
LogQL: Propagate headers/warnings/stats from quantile downstreams (#13881) (a0c7598).
LogQL: Record datasample queries are limited query type (#13930) (ae938d0).
LogQL: Return empty vector instead of nil for empty evaluator. (#13485) (08615bf).
LogQL: Special case the return values from a sharded first/last_over_time query (#13578) (29a37d5).
log results cache: Include pipeline wrapper disabled in cache key (#13328) (221491c).
loki: Init internal server log along with loki's server instance (#13221) (66b8c9b).
loki-mixin: Fix latency panel for Index Gateway (#13629) (f586c00).
loki-mixin: Various latency panels in operational dashboard should have ms unit type instead of seconds (#13260) (f5a9905).
loki-mixin: Attribute OTLP route correctly to write path (#13943) (b91b782).
loki-mixins: Incorrect pod matcher for compactor in mixin when using ssd mode (#12846) (515e13c).
operator: Allow structured metadata only if V13 schema provided (#13463) (3ac130b).
operator: Don't overwrite annotations for LokiStack ingress resources (#13708) (f523530).
operator: Remove duplicate conditions from status (#13497) (527510d).
operator: Set object storage for delete requests when using retention (#13562) (46de4c1).
operator: Skip updating annotations for serviceaccounts (#13450) (1b9b111).
operator: Support v3.1.0 in OpenShift dashboards (#13430) (8279d59).
operator: Watch for CredentialsRequests on CCOAuthEnv only (#13299) (7fc926e).
querier: Add a retry middleware to all the stats handlers (#13584) (7232795).
querier: Adjust tailer loop criteria so it is actually re-tested (#13906) (dabbfd8).
querier: Fix retry code to handle grpc status codes. updated newer stats retries to be wrapped with spans (#13592) (d3e1edb).
querier: Fixes span name of serializeRounTripper (#13541) (4451d56).
querier: Remove retries on the stats handlers because they already retry (#13608) (1008315).
query engine: Include lines with ts equal to end timestamp of the query range when executing range aggregations (#13448) (e0ca67d).
retry: Fix retries when using protobuf encoding (#13316) (a457c5d).
ruler: Protect ruler remote-write overrides map with a mutex when creating new appenders (#13676) (e9a9c60).
sharding: Use without() grouping when merging 
avg_over_time shard results (#12176) (eb8a363).
storage: Handle block offset exceeding chunk length in memchunk.go (#13661) (d42476a).
storage: Read "404" as object not exist (#13901) (3c9c647).
storage: Separates directory creation from permission checks (#13248) (1086783).
storage: Try reading chunks which have incorrect offset for blocks (#13720) (7e224d5).
structured metadata: Sanitize structured metadata at query time (#13983) (3bf7fa9).
WAL: Fix a bug where AppendRequest with no entries triggers flush (#13672) (8a3ae22).
WAL: Properly reset wal segment writer (#13468) (6ea83b4).

v3.3
Grafana Labs and the Loki team are excited to announce the release of Loki 3.3. Here's a summary of new enhancements and important fixes.
For a full list of all changes and fixes, refer to the CHANGELOG.
Features and enhancements
Key features in Loki 3.3.0 include the following:
Query acceleration with Bloom filters (experimental): The Bloom filters feature introduced in Loki 3.0 remains experimental. There has been a major shift in how Loki leverages bloom filters — pivoting from free-text search to harnessing the power of structured metadata (#14061) (a2fbaa8). This change leverages the growing adoption of OpenTelemetry in observability stacks. The Bloom Compactor component has been removed and replaced by the Bloom Planner and Bloom Builder components (#13997) (bf60455) and blooms has a new block schema (V3) (#14038) (5395daf) which is a breaking change.
Explore Logs: Starting with Grafana v11.3, the plugin for the Explore Logs app is installed in Grafana by default. This release includes enhancements to improve pattern ingester tracing (#14707) (80aec25), to apply patterns line length limit to json message keys (#14296) (41fafd8), add configuration to set 
max-allowed-line-length in the pattern ingester (#14070) (0780456) and move metric aggregation to a per-tenant configuration (#14709) (c1fde26).
logql: We've added an 
approx_topk function to LogQL, which uses probabilistic techniques to give you an approximation of the most frequent values in the input, along with their approximate frequencies. This is great for cases where the equivalent 
topk query hits the maximum series limit or times out before completion. Its also great in cases where you want a faster response to your query and are fine with an approximation of the true answer. (#14243). For more information, visit LogQL docs.
helm: Multiple improvements to the Helm charts, including: Updating the chart with Loki version 3.2.0 (#14281) (11b92ee), replacing bloom compactor with bloom planner and builder (#14003) (08e61ca), adding the ability to configure 
client_max_body_size (#12924) (809a024), adding additional service annotations for components in distributed mode (#14131) (5978f13), adding configurable 
extraEnvFrom to admin-api and enterprise-gateway (#14533) (5d78a3a), adding 
kubeVersionOverride for Helm chart (#14434) (0935d77), adding persistence option to memcached on Helm chart (#13619) (ef1df0e), adding tolerations to pattern-ingester statefulset (#13605) (09530c0), allowing setting node attributes to 
tokengen and 
provisioner (#14311) (c708ae6), updating Loki Helm chart for restricted environments (#14440) (adc7538), and supporting ruler sidecar in singleBinary mode (#13572) (684baf7).
mixins: Multiple improvements to the Loki mixins, including: adding a missing cluster label to mixins (#12870) (547ca70), adding support for partition ingester in dashboards (#14498) (70deebf), adding a Loki compaction not successful alert (#14239) (da04f50), allowing overriding of some labels by parameterizing mixin recording/alert rules (#11495) (f1425b6), allowing disabling of bloom dashboards (#14177) (ce2e6d5), allowing hiding of unused rows in loki-operational (#13646) (3aa4f22), and merging read-resources and write-resources dashboards for SSD into one dashboard that also includes the backend-resources(#13471) (45b8719).
operator: Multiple updates to the Operator, including: Adding support for Loki OTLP limits configuration (#13446) (d02f435), providing default OTLP attribute configuration (#14410) (1b52387), adding a User-guide for OTLP configuration (#14620) (27b4071), updating the Loki operand to v3.1.1 (#14042) (7ae1588), updating the Loki operand to v3.2.1 (#14526) (5e970e5), renaming the Loki API go module (#14568) (976d8ab), introducing 1x.pico size (#14407) (57de81d), migrating the project layout to kubebuilder go/v4 (#14447) (dbb3b6e), declaring feature FIPS support for OpenShift only (#14308) (720c303).
docs: New videos and tutorials, including a New Cloud deployment guide for Deploying Loki on AWS, a new Fluent Bit tutorial and a Getting Started with OpenTelemetry Collector tutorial. Also there are improvements to the Query documentation, including a new Query Best Practices topic, a reorganized Query Template, and a revised LogQL Simulator topic.
Other improvements include the following:
api: Added endpoint to return detected field values (#14350) (7983f94).
blooms: Add disk-backed queue for the bloom-planner (backport k227) (#14927) (1f6828b).
blooms: Only write key and key=value to blooms (#14686) (3af0004).
blooms: Do not add empty blooms to offsets (#14577) (51c42e8).
blooms: Extract task computing into a strategy interface (#13690) (ab5e6ea).
blooms: New bloom planning using chunk size TSDB stats (#14547) (673ede1).
blooms: Remove compression of 
.tar archived bloom blocks (#14159) (cdf084f).
compactor: Add backoff mechanism to the retention process (#14182) (3136880).
distributor: Track discarded data by usageTracker (#14081) (c65721e).
distributor: Add ability to log stream selectors before service name detection (#14154) (d7ff426).
distributors: Use a pool of worker to push to ingesters. (#14245) (f80d68a).
fluentd: Support custom http headers (#14299) (e59035e).
ingester: Implement owned streams calculation using Partition Ring (#14282) (3c36ba9).
ingester: Implement partition shuffle sharding for ingester (#14304) (1a4436c).
jsonnet: Allow to name prefix zoned ingesters (#14260) (fac3177).
labels: Add app_name as a service label (#13660) (f2a16f4).
logcli: Add gzip compression option (#14598) (4d3f9f5).
logql: Micro-optimizations for IP filter (#14072) (c5083c7).
logql: Add query user and query source to "executing query" log lines (#14320) (4d69929).
promtail: Add structured metadata to the promtail push API (#14153) (66cffcb).
querier: Move detected field logic to query frontend (#14212) (36ace66).
ruler: Enable ruler store that uses clients from thanos-io/objstore pkg (#11713) (8bca2e7).
storage: Add retries for s3 ObjectExists calls (#14062) (73cbbb0).
storage: Implement IsRetryableErr for S3ObjectClient (#14174) (fc90a63).
storage: Introduce new 
ObjectExistsWithSize API to (#14268) (ac422b3).
storage: AWS backend using thanos.io/objstore (#11221) (b872246).
storage: Azure backend using thanos.io/objstore (#11315) (5824e3d).
storage: GCS backend using thanos.io/objstore (#11132) (c059ace).
structured metadata: Include structured_metadata size while asserting rate limit (#14571) (a962edb).
Deprecations
One of the focuses of Loki 3.0 was cleaning up unused code and old features that had been previously deprecated but not removed. Loki 3.0 removed a number of previous deprecations and introduces some new deprecations. Some of the main areas with changes include:
Deprecated storage options including the deprecation of the BoltDB store.
Deprecated configuration options.
API endpoint deprecations.
To learn more about breaking changes in this release, refer to the Upgrade guide.
{{< docs/shared source="alloy" lookup="agent-deprecation.md" version="next" >}}
Upgrade Considerations
For important upgrade guidance, refer to the Upgrade Guide.
BREAKING CHANGE - blooms: Introduce a new block schema (V3) (#14038).
BREAKING CHANGE - blooms: Index structured metadata into blooms (#14061).
BREAKING CHANGE - docker: Remove wget from Promtail docker image(#15101).
BREAKING CHANGE - operator: Migrate project layout to kubebuilder go/v4 (#14447).
BREAKING CHANGE - operator: Rename Loki API go module (#14568).
BREAKING CHANGE - operator: Provide default OTLP attribute configuration (#14410).
Bug fixes
3.3.4 (2025-02-14)
deps: Move to Go 1.23.7 (#16681) (f6c6474).
deps: Update jwt and oauth2 dependencies for 3.3.x (#17021) (241f5aa).
deps: Update module golang.org/x/crypto to v0.35.0 [security] (release-3.3.x) (#16590) (454cad2).
deps: Update module golang.org/x/oauth2 to v0.27.0 [security] (release-3.3.x) (#16591) (7b35a41).
docs: Add a note on docker configuration.md doc to explain accepted values for boolean type (#16746) (0102107).
3.3.3 (2025-02-14)
build: Update build info 3 3 x (#16169) (4672625).
ci: Provide correct build info for release builds (#15939) (#16214) (e5f80d3)
ci: Revert "fixed 
Publish Rendered Helm Chart Diff workflow" (#15511) (09d1085).
correct typo in architecture name (amr64 -> arm64) (backport release-3.3.x) (#15715) (9acdc33).
deps: CVE updates for Loki 3.3.2 (#15645) (01a7a0d).
3.3.2 (2024-12-18)
deps: backport update module golang.org/x/crypto to v0.31.0 (#15394) (57a7a2c).
docker-driver: Fix the docker driver build (backport release-3.3.x) (#15478) (23b5fc2).
enable metric aggregation in 
loki-local-config.yaml (backport release-3.3.x) (#15319) (635f04d).
3.3.1 (2024-12-04)
BREAKING CHANGE - docker: Remove wget from Promtail docker image(#15101).
docker: Move from base-nossl to static. This PR removes the inclusion of glibc into most of the Docker images created by the Loki build system. (#15203).
promtail: Switch Promtail base image from Debian to Ubuntu to fix critical security issues (#15195).
3.3.0 (2024-11-19)
blooms: Add tenant limits as dependency to pattern ingester (#14665) (31eea90).
blooms: Check length of tasks before accessing first element in slice (#14634) (601f549).
blooms: Copy chunks from ForSeries (backport k227) (#14864) (d10f79c).
blooms: Do not restart builders when planner disconnects (backport k227) (#14922) (213e8ee).
blooms: Exclude label filters where label name is part of the series labels. (#14661) (d1668f6).
blooms: Fix panic in initialization of the bloom planner and builder (#14110) (8307c42).
blooms: Fix strategy logger and add task test (backport k227) (#14921) (dc36a1e).
blooms: Fix tenants slice on loadTenantTables (backport k227) (#14901) (540dd5a).
blooms: Skip multi-tenant TSDBs during bloom planning (backport k227) (#14888) (631cff3).
blooms: Improve docs for min and max table offsets (backport k227) (#14929) (3161fdc).
blooms: Fix lint errors (#14574) (99ef900).
build: Use Debian Bullseye base image for build image (#14368) (3beb8ff).
canary: Reconnect immediately upon tail max duration (#14287) (9267ee3).
ci: Fix 
Publish Rendered Helm Chart Diff workflow (#14365) (6de6420).
ci: Update helm diff rendering workflow (#14424) (916e511).
config: Copy Alibaba and IBM object storage configuration from common (#14297) (59ff1ec).
config: Migrate renovate configuration (#14646) (a67d8ef).
deps: Update aws-sdk-go-v2 monorepo (#13986) (6f49123).
deps: update aws-sdk-go-v2 monorepo (#14742) (53a1ab7).
deps: update github.com/grafana/dskit digest to 687ec48 (#14395) (c2f38e1).
deps: update github.com/grafana/dskit digest to 7c41a40 (#14277) (f39cdbd).
deps: update github.com/grafana/dskit digest to 931a021 (#14032) (7c18642).
deps: update github.com/grafana/dskit digest to b69ac1b (#14355) (9d7a6ea).
deps: update github.com/grafana/dskit digest to f52de24 (#14319) (a4f3edf).
deps: update github.com/twmb/franz-go/pkg/kfake digest to cea7aa5 (#14590) (688c42a).
deps: update k8s.io/utils digest to 702e33f (#14033) (b7eecc7).
deps: Update module cloud.google.com/go/bigtable to v1.33.0 (#14580) (a0920ed).
deps: Update module cloud.google.com/go/pubsub to v1.45.0 (#14361) (4351238).
deps: Update module cloud.google.com/go/pubsub to v1.45.1 (#14650) (f173708).
deps: Update module cloud.google.com/go/storage to v1.46.0 (#14744) (8e45116).
deps: Update module github.com/alicebob/miniredis/v2 to v2.33.0 (#14721) (7bfda25).
deps: Update module github.com/aws/aws-sdk-go to v1.55.5 (#14715) (03f0f5a).
deps: Update module github.com/axiomhq/hyperloglog to v0.2.0 (#14722) (0167b22).
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.189 (#14044) (7fb34b4).
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.192 (#14337) (6f7cae2).
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.196 (#14651) (478085a).
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.197 (#14682) (b898294).
deps: Update module github.com/coder/quartz to v0.1.2 (#14652) (7459e07).
deps: Update module github.com/felixge/fgprof to v0.9.5 (#14338) (a2ad3aa).
deps: Update module github.com/fsouza/fake-gcs-server to v1.50.2 (#14313) (275c97c).
deps: Update module github.com/hashicorp/raft to v1.7.1 (#14005) (e9cec1d).
deps: Update module github.com/ibm/go-sdk-core/v5 to v5.17.5 (#14045) (677d217).
deps: Update module github.com/ibm/go-sdk-core/v5 to v5.18.1 (#14716) (8395acd).
deps: Update module github.com/ibm/ibm-cos-sdk-go to v1.11.1 (#14342) (aa82a7c).
deps: Update module github.com/klauspost/compress to v1.17.10 (#14352) (e23c5ed).
deps: Update module github.com/minio/minio-go/v7 to v7.0.76 (#14006) (51f9376).
deps: Update module github.com/minio/minio-go/v7 to v7.0.77 (#14353) (d0e3ef7).
deps: Update module github.com/minio/minio-go/v7 to v7.0.80 (#14654) (eec2513).
deps: Update module github.com/ncw/swift/v2 to v2.0.3 (#14356) (c843288).
deps: Update module github.com/prometheus/client_golang to v1.20.5 (#14655) (e12f843).
deps: Update module github.com/schollz/progressbar/v3 to v3.17.0 (#14720) (4419d0f).
deps: Update module github.com/shirou/gopsutil/v4 to v4.24.10 (#14719) (3280376).
deps: Update module github.com/shirou/gopsutil/v4 to v4.24.9 (#14357) (c8e6a9d).
deps: Update module github.com/shopify/sarama to v1.43.3 (#14059) (1cf4813).
deps: Update module github.com/spf13/afero to v1.11.0 (#14060) (bbbd82b).
deps: Update module go.etcd.io/bbolt to v1.3.11 (#14358) (b7bccfc).
deps: Update module golang.org/x/net to v0.29.0 (#14341) (1b6b9da).
deps: Downgrade grpc to fix regression (#14065) (8c38d46).
deps: Fix missing dep PartitionRing for Ingester (#14292) (6354ded).
deps: nix build, downgrade toolchain to go1.23.1 (#14442) (26dfd62).
deps: Revert "fix(deps): Update module github.com/shirou/gopsutil/v4 to v4.24.9 (#14357)" (#14437) (d53955b).
detected_fields: Return parsed labels when parsers are passed (#14047) (aa1ac99).
detected_fields: Always return empty array as 
null (#14112) (93009d4).
docs: Correct OTLP documentation typo (#14602) (063c590).
distributor: Always write detected_level when enabled, even if unknown (#14464) (41c6b6c).
distributor: Validate partition ring is kafka is enabled (#14303) (8438d41).
distributor: Level detection for warning level (#14444) (242a852).
explore logs: Correctly create logfmt string (#14124) (63e84b4).
explore logs: Allow any level for aggregated metrics (#14255) (c001a1d).
helm: Add missing 
loki.storage.azure.chunkDelimiter parameter to Helm chart (#14011) (08c70cc).
helm: Check for 
rbac.namespaced condition before creating roles (#14201) (3f47f09).
helm: Fix persistence configuration for Memcached (#14049) (ee6e1cf).
helm: Fix wrong port name referenced for ingress NetworkPolicy (#12907) (963a25b).
helm: Various fixes and enhancements for bloom components (#14128) (dc0cbd4).
ingester: Report PSRL error message correctly (#14187) (a475153).
ingester: Wait for OwnedStreams service in Ingester startup (#14208) (a4aee4f).
lambda-promtail: Add s3 principal to iam policy attached to sqs in lambda-promtail terraform code (#14619) (db0889e).
logcli: Check for errors before checking for 
exists when fetching data (backport k227) (#14906) (31b2a63).
logcli: Create new tail response for every line (#14525) (bcfd0d1).
logql: Fix panic in json parsing when using empty array index (#14393) (833bf0d).
logql: Update JSONExpressionParser to not unescape extracted values if it is JSON object. (#14499) (08b1a90).
logql: Allow rename of structured metadata labels (#13955) (2d4792a).
logql: Renamed label should use ParsedLabel category (#14515) (82fb2f0).
logql: Skipping label if it contains special symbol (#14068) (55e374e).
mixin: Remove pod label from disk usage aggregation (#14180) (5d45c96).
mixins: Fix loki-resources-overview panel layout (#14178) (8f54ec6).
mixins: Add backend path section in loki-operational for single scalable deployment (#13023) (16881ab).
mixins: Disk space utilization panels with latest KSM versions (#13486) (0ea7431).
mixins: Retention dashboards fix metric name (#14617) (c762b9b).
otlp: Enable service detection for otlp endpoint (#14036) (4f962ef).
operator: Add 1x.pico OpenShift UI dropdown menu (#14660) (4687f37).
operator: Add missing groupBy label for all rules on OpenShift (#14279) (ce7b2e8).
operator: Correctly ignore again BlotDB dashboards (#14587) (4879d10).
operator: Disable automatic discovery of service name (#14506) (3834c74).
operator: Disable log level discovery for OpenShift tenancy modes (#14613) (5034d34).
operator: Fix building the size-calculator image (#14573) (a79b8fe).
operator: Fix make build target for size-calculator (#14551) (e727187).
operator: Move OTLP attribute for statefulset name to stream labels (#14630) (5df3594).
operator: Use empty initialized pod status map when no pods (#14314) (6f533ed).
operator: Update renovate ignore for operator API with new module path (#14581) (c9b2907).
pattern: Fixes latency metric namespace for tee to pattern (#14241) (ae955ed).
promtail: Fix configuration unmarshalling (#14408) (a05431f).
promtail: Fix parser for azureeventhubs message without time field (#14218) (2e62abb).
promtail: Validate scrape_config job name, do not allow duplicate job names (#13719) (f2d3499).
querier: Propagate query stats from quantile & topk queries (#13831) (78b275b).
querier: Correct _extracted logic in detected fields (#14064) (1b3ba53).
querier: Timestamp. Time should be called with milliseconds (#14196) (f8d9143).
querier: Report correct status code for metric and log queries in metrics.go (#12102) (900751c).
query: Fix bug in query result marshaling for invalid utf8 characters (#14585) (f411a07).
Remove usage of unsafe string in label adapter unmarshall (#14216) (758364c).
Rename misspelled filename (#14237) (cf1d4a3).
sharding: Apply offset to both 
from and 
through in shard request (#14256) (17c472d).
storage: Add additional validation for timeout while retrieving headers (#14217) (8322e51).
storage: Do not retain span logger created with index set initialized at query time (#14027) (4e41744).
storage: Expand matching for additional variations (#14221) (71d7291).
storage: Disable client retries when congestion control is enabled in S3(#14588) (cff9f43).
storage: Have GetObject check for canceled context (#14420) (5f325aa).
storage: Transform 
ObjectExistsWithSize into 
GetAttributes (#14329) (2f56f50).
storage: Update AWS storage timeout error for Go 1.23 behavior (#14226) (a4642b5).

v3.4
Grafana Labs and the Loki team are excited to announce the release of Loki 3.4. Here's a summary of new enhancements and important fixes.
For a full list of all changes and fixes, refer to the CHANGELOG.
Features and enhancements
Key features in Loki 3.4.0 include the following:
Time sharding ingestion: Add per-tenant time sharding for long out-of-order ingestion. The setting 
shard_streams.time_sharding_enabled is a per-tenant config that splits incoming log streams by injecting a new 
__time_shard__ synthetic label in such a way that no stream will ever be longer than 
1h (or, rather, 
max_chunk_age/2). This allows the ingestion of very old logs, because the logs in these new streams will never be "too far behind" by design. (#14711) (0d6d68d).
Thanos object client: Previously Loki and Grafana Enterprise Logs used different Go packages to talk to the underlying object store. Migrating to Thanos means that there is now a single object client across Loki and GEL, and across the other Grafana backend databases: Mimir and Pyroscope.(#14891) (fb6789d), (#14638) (8958eb9), (#14867) (947a66f), (#14793) (cfc3819), (#14937) (a629212). The documentation now includes configuration examples using Thanos-based storage clients.
API: Add support for Parquet file format as a query response format. (#15408) (d0c11a6).
Structured metadata: Introduces a new feature,based on per-tenant configuration, that allows for extraction of "fields" into structured metadata at ingest time. Fields can either be regular labels, structured metadata keys, or keys from 
logfmt or 
json formatted log lines. (#15188) (7033091).
lambda-promtail: Adds support for relabeling log entries in lambda-promtail using Prometheus-style relabel configs. This allows for more flexible label manipulation and filtering of log entries before they are sent to Loki. (#15600) (c41a8b4).
Kafka: Loki 3.4 adds an Apache Kafka development environment to support ongoing development work(#15603) (226e9f1).
Helm charts: Loki 3.4 includes multiple improvements to the Helm charts, including: Added support for overrides exporter (#15630) (db72164), Expose topologySpreadConstraints for admin-api pods (#14995) (1b93086), Expose topologySpreadConstraints for distributed deployment (#14853) (d075107), Compute ConfigMap/Secret checksum only over .data content (#15177) (990f71c), Do not explicitly template 
managed-by, as it should be set at install time by the installation manager. (#14763) (10b7478).
Query acceleration with Bloom filters (experimental): The Bloom filters feature introduced in Loki 3.0 remains experimental. In Loki 3.3, there was a major shift in how Loki leverages bloom filters — pivoting from free-text search to harnessing the power of structured metadata. In Loki 3.4, work continues on the new Bloom Planner (#14874) (b646861).
Explore logs: The Explore apps suite for metrics, logs, traces, and profiles has a new name: Grafana Drilldown. To learn more about why we made this switch, you can read about it in this blog post. The new name is reflected in the Grafana Cloud user interface and navigation as of 2025-02-20, and it will be part of the Grafana 11.6 release in March.
Docs: This release includes a heavily revised labels topic and a new Cardinality topic. This release also marks the return of the sizing guidance page, newly updated based on Grafana Labs experience running Loki. Continuing updates to the Query section include a revised logcli topic and new logcli tutorial. There is also two additional new tutorials: Deploying Loki on Azure tutorial and Kubernetes Monitoring Helm tutorial.
{{< admonition type="note" >}}
Note that when appropriate, documentation updates have been backported to previous releases.
{{< /admonition >}}
Other improvements include the following:
analytics: Add TLS config to the analytics client (#15227) (2c5eabd).
approx_topk: Reduce memory usage of HyperLogLog in approx_topk. (#15559) (bef2043).
approx_topk: Improve 
approx_topk performance by reducing allocations. (#15450) (04994ca).
blooms: Add task timing and sizing metrics (#15032) (79cccfd).
blooms: Apply task timeout in bloom builder (#14988) (f92dde0).
blooms: Prefetch bloom blocks as soon as they are built (#15050) (b406015).
blooms: Compute chunkrefs for series right before sending task to builder (#14808) (66e6b1c).
blooms: Replace JSON lib for encoding/decoding metas (#14767) (57f16a0).
distributor: Add a metric ingestion time SM sanitization (#15222) (e9d0c3e).
docker compose: enable feature toggle (#14858) (8f243ee).
docker driver: Add ARM64 build and release of the Docker driver in Drone pipeline(#9247) (e950251).
lambda-promtail: Add terraform variable for s3 bucket notification filter prefix (#15381) (3dd8d7d).
logcli: Add opt-in support for ProxyFromEnvironment in logcli (#11742) (#14950) (c56b95d).
logcli: Include common labels (#15611) (639ac74).
logql: Allow line_format bypass template execution if possible (#15411) (2150fbc).
metrics: Add lines skipped metric to pattern ingesters (#14997) (dea5d78).
operator: Add support for managed GCP WorkloadIdentity (#14752) (7635a5c).
otel/labels: Add otlp attribute deployment.environment.name to list of default labels (#14747) (2d407bd).
querier: Present DF bytes values in queryable format (#15272) (f5d62bd).
querier: Use context propagation to call the same ingester in GetChunksID as Query (#15186) (70d9587).
ruler: Add Rule Name and Type to Query Tags (#15055) (b272573).
storage: Add objstore support for Swift using thanos.io/objstore (#11672) (44523e0).
storage: Skip writeback for chunks fetched by queriers older than a duration (#15605) (3b8d993).
storage: Support config timeout for AlibabaCloud OSS requests (#14856) (ac2e21f).
storage: Allow TSDB index creation in memory only (#14732) (831c0d5).
structured metadata: Sanitize structured metadata during ingestion in the distributor (#15141) (be4f17e).
3.4.5 (2025-07-11)
storage: Add a configuration option for custom GCS endpoints (#16419) (#18419) (b2c87ba).
Deprecations
One of the focuses of Loki 3.0 was cleaning up unused code and old features that had been previously deprecated but not removed. Loki 3.0 removed a number of previous deprecations and introduces some new deprecations. Some of the main areas with changes include:
Promtail is deprecated, as the code has been merged into Grafana Alloy. You can find migration documentation and a utility to convert your Promtail configuration to Alloy configuration in the Alloy documentation. Note that this deprecation does NOT include the lambda-promtail client.
Deprecated storage options including the deprecation of the BoltDB store.
Deprecated configuration options.
API endpoint deprecations.
To learn more about breaking changes in this release, refer to the Upgrade guide.
{{< docs/shared source="alloy" lookup="agent-deprecation.md" version="next" >}}
Upgrade Considerations
For important upgrade guidance, refer to the Upgrade Guide.
BREAKING CHANGE - Docker image: Remove 
wget from Promtail docker image (#15101).
Bug fixes
3.4.6 (2025-09-11)
deps: Update Go minor version (#19137) (2b7b94f).
3.4.5 (2025-07-11)
frontend: Allow resolution of v6 addresses. (backport release-3.4.x) (#18260) (93e0880).
memberlist: Allow resolution of advertise address from IPv6  interfaces defined in common 
instance_interface_names.(backport release-3.4.x) (#18256) (640cd33).
WAL: Handle WAL corruption properly on startup (backport release-3.4.x) (#18410) (1ecc9e4).
3.4.4 (2025-06-09)
ci: Update release code 3.4 (#18015) (4108c5d).
deps: Move to Go 1.23.8 (#17806) (2e18686).
deps: Update module golang.org/x/net to v0.38.0 [security] (release-3.4.x) (#17273) (57cd6ce).
docs: Update configuration.md (backport release-3.4.x) (#17298) (cc8db57).
3.4.3 (2025-04-03)
deps: Move to Go 1.23.7 (#16682) (5de86f1).
deps: Update jwt, oauth and go-redis dependencies (#17023) (e038d0f).
deps: Update module golang.org/x/crypto to v0.35.0 [security] (release-3.4.x) (#16588) (184fe1a).
deps: Update module golang.org/x/oauth2 to v0.27.0 [security] (release-3.4.x) (#16589) (bead91e).
docs: Add a note on docker configuration.md doc to explain accepted values for boolean type (backport release-3.4.x) (#16743) (c39780d).
3.4.2 (2025-02-14)
blooms: Initialize bloom gateway client only once (backport release-3.4.x) (#16268) (1b9829b)
ci: do not disable CGO in Makefile (backport release-3.4.x) (#16272) (4fa045d)
3.4.1 (2025-02-12)
docker: Update the build info in docker images (#16225) (4484080).
3.4.0 (2025-02-12)
aggregated_metrics: Fix the IsError method causing retries (#15296) (7a7b3b1).
azureblob: Return error if Azure returns no service principal token (#13195) (e98a86b).
blooms: Copy chunks from ForSeries (#14863) (bfc2890).
blooms: Do not restart builders when planner disconnects (#14783) (7669385).
blooms: Enable metas cache on backend when running in ssd mode (#15275) (4cce207).
blooms: Fix check for skipping most recent data when filtering blooms (#15300) (78d3c44).
blooms: Fix strategy logger and add task test (#14857) (ab2721d).
blooms: Fix tenants slice on loadTenantTables (#14900) (a8f7851).
blooms: Match series to newest block only (#15481) (5846ea2).
blooms: Skip multi-tenant TSDBs during bloom planning (#14770) (76d35cc).
blooms: Fix data race in bloombuild tests (#15360) (aa7b849).
ci: Fix test compile error (#15404) (db2b6db).
ci: Remove submodule once and for all (#15495) (80e8e60).
ci: Revert "fixed 
Publish Rendered Helm Chart Diff workflow" (#15506) (6c49cc0).
ci: Transform TestChunkSize into a benchmark (#15361) (fed43db).
ci: Update Python version for Helm CI (#15420) (bdbb009).
deps: Downgrade hyperloglog to 0.2.0 (#15424) (#15426) (3b0d815).
deps: Update aws-sdk-go-v2 monorepo (#14820) (a355840).
deps: Update module cloud.google.com/go/bigtable to v1.34.0 (#15581) (9f28f69).
deps: Update module cloud.google.com/go/pubsub to v1.45.3 (#15294) (672fb08).
deps: Update module cloud.google.com/go/storage to v1.50.0 (#15690) (efd3ec3).
deps: Update module github.com/alecthomas/chroma/v2 to v2.15.0 (#15596) (a2d3057).
deps: Update module github.com/alicebob/miniredis/v2 to v2.34.0 (#15501) (61b4ddf).
deps: Update module github.com/aliyun/aliyun-oss-go-sdk to v3 (#14976) (64081c0).
deps: Update module github.com/aws/aws-sdk-go-v2 to v1.32.8 (#15680) (42c43ec).
deps: Update module github.com/aws/aws-sdk-go-v2/config to v1.28.9 (#15663) (c5c9a2e).
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.72.2 (#15685) (0e329e1).
deps: Update module github.com/axiomhq/hyperloglog to v0.2.3 (#15641) (b6df82e).
deps: Update module github.com/azure/azure-storage-blob-go to v0.15.0 (#14746) (ad9bdcd).
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.213 (#15686) (95dac7c).
deps: Update module github.com/bmatcuk/doublestar to v4 (#14977) (a2b66d3).
deps: Update module github.com/docker/docker to v27.3.1+incompatible (#14753) (b4a926c).
deps: Update module github.com/fatih/color to v1.18.0 (#14754) (8fc9b28).
deps: Update module github.com/fsnotify/fsnotify to v1.8.0 (#14755) (0704f5d).
deps: Update module github.com/go-redis/redis/v8 to v9 (#14978) (33a2148).
deps: Update module github.com/gocql/gocql to v1 (#15016) (2a51081).
deps: Update module github.com/golang/snappy to v0.0.4 (#15415) (035f871).
deps: Update module github.com/grafana/loki/v3 to v3.2.1 (#14826) (b8509fc).
deps: Update module github.com/grpc-ecosystem/go-grpc-middleware/v2 to v2.2.0 (#15392) (6b7c3fc).
deps: Update module github.com/hashicorp/consul/api to v1.31.0 (#15540) (8207a9b).
deps: Update module github.com/hashicorp/golang-lru to v2 (#14979) (3abb3b1).
deps: Update module github.com/heroku/x to v0.4.1 (#15370) (20f5414).
deps: Update module github.com/ibm/go-sdk-core/v5 to v5.18.5 (#15673) (b7ec8e3).
deps: Update module github.com/ibm/ibm-cos-sdk-go to v1.12.0 (#15228) (5689e2e).
deps: Update module github.com/ibm/sarama to v1.45.0 (#15636) (9e13abd).
deps: Update module github.com/imdario/mergo to v1 (#15035) (be24862).
deps: Update module github.com/influxdata/telegraf to v1.33.1 (#15688) (d57492c).
deps: Update module github.com/klauspost/compress to v1.17.11 (#15451) (6e8bd9d).
deps: Update module github.com/masterminds/sprig/v3 to v3.3.0 (#14899) (8874878).
deps: Update module github.com/minio/minio-go/v7 to v7.0.82 (#15417) (7b8d82e).
deps: Update module github.com/opentracing-contrib/go-stdlib to v1.1.0 (#15221) (9f7892a).
deps: Update module github.com/pierrec/lz4 to v2.6.1+incompatible (#15502) (60f031a).
deps: Update module github.com/pierrec/lz4/v4 to v4.1.22 (#15400) (36bda72).
deps: Update module github.com/prometheus/common to v0.61.0 (#15352) (860c25d).
deps: Update module github.com/schollz/progressbar/v3 to v3.18.0 (#15691) (5f902b4).
deps: Update module github.com/shirou/gopsutil/v4 to v4.24.12 (#15569) (bc43878).
deps: Update module github.com/sony/gobreaker/v2 to v2.1.0 (#15556) (d188d06).
deps: Update module github.com/spf13/afero to v1.12.0 (#15696) (94cfb90).
deps: Update module github.com/stretchr/testify to v1.10.0 (#15091) (f65ab13).
deps: Update module github.com/twmb/franz-go/pkg/kadm to v1.14.0 (#14911) (1b59525).
deps: Update module go.opentelemetry.io/collector/pdata to v1.22.0 (#15452) (661ae79).
deps: Update module go.opentelemetry.io/otel/trace to v1.33.0 (#15454) (82c2c56).
deps: Update module golang.org/x/crypto to v0.32.0 (#15610) (db1188f).
deps: Update module golang.org/x/net to v0.34.0 (#15612) (0b25012).
deps: Update module golang.org/x/oauth2 to v0.25.0 (#15590) (ccee7f9).
deps: Update module golang.org/x/sync to v0.10.0 (#15253) (3150f07).
deps: Update module golang.org/x/sys to v0.29.0 (#15591) (efb9a81).
deps: Update module golang.org/x/text to v0.21.0 (#15266) (818446c).
deps: Update module golang.org/x/time to v0.9.0 (#15592) (5401e78).
deps: Update module google.golang.org/api to v0.214.0 (#15519) (e34944d).
deps: Update module google.golang.org/grpc to v1.68.1 (#15269) (b607fe0).
deps: Update module google.golang.org/protobuf to v1.36.2 (#15635) (74885a2).
deps: Update module gopkg.in/alecthomas/kingpin.v2 to v2.4.0 (#14970) (37d0bb8).
deps: Update module k8s.io/apimachinery to v0.32.0 (#15398) (fada02c).
deps: Update module k8s.io/klog to v2 (#15046) (734caf5).
distributor: Fix data race in distributor tests (#15423) (b8168a8).
distributor: Return error 422 (Unprocessable Content) when push request does not contain any streams (#13706) (17bf32b).
docs: Correct typo in architecture name (amr64 -> arm64) (#15707) (068f81d).
docs: Improve docs for min and max table offsets (#14890) (fd9d332).
docs: Change "did received" to "did receive" in NOTES.txt (#15584) (9191eaf).
helm/loki: Add namespace to Loki templates for ruler and index-gateway (#14838) (733d99f).
helm/loki: Allow prefixing of rollout-group value and ingester name (#15063) (f4accc1).
helm: Add 
appProtocol: tcp to headless backend grpc ports (#14507) (400f5af).
helm: Add conditional to include ruler config only when enabled (#15385) (3badbb3).
helm: Add default wal dir to ruler config (#14920) (1a6751f).
helm: Fix statefulset templates to not show diffs in ArgoCD (#15192) (3c71c83).
helm: Split ingester HPA when zoneAwareReplication is enabled (#14565) (80e46f7).
helm: FIx Yaml templates incorrectly annotated (#15207) (acd4d11).
helm: Disable Helm test pod when test.enabled=false (#15346) (1fb4e2b).
ingester: data race in ingester test (#15465) (04f621e).
ingester: Add stream creation error logging (#15640) (f63c16f).
kafka: Ensure partition-reader starts up correctly (#14845) (b2f3d2e).
kafka: Fix incorrect error check in autoCommitLoop (#15149) (4b5925a).
kafka: Read from beginning if no committed offset (#14865) (1b6da11).
kafka: Use KafkaEndOffset instead of -1 (#15099) (2113526).
kafka: Use separate variable to track the consume offset (#15095) (161a192).
Keep the service_name label after discovery in push.ParseLokiRequest before calculating retention and passing to the usage tracker (#14960) (71d4d6a)
lambda-promtail: Correct S3 bucket owner label key to prevent retrieval errors (#14913) (85aaf63).
lambda-promtail: Remove s3 bucket ownership verification (#15383) (bf73014).
logcli: Check for errors before checking for 
exists when fetching data (#14780) (9e7d2f2).
logql: Allow comparison against zero bytes (#15217) (17f1972).
memberlist: Respect interface names config for advertise address (#13940) (751324c).
metric_aggregation: Fix duplicate metrics registration (#15142) (215f994).
metrics: Correctly propagate index stats to metrics.go log line (#14941) (ff111dc).
metrics: Enable metric aggregation in 
loki-local-config.yaml (#15302) (c96b750).
operator: Add log attribute for level to structured metadata (#14776) (036c131).
operator: Fix maximum OpenShift version (#14764) (cc496c6).
operator: Fix operator release pipeline warnings (#14817) (e707a3d).
operator: Fix RBAC permission for poddisruptionbudgets (#15107) (f00d07a).
operator: Publish images on docker hub upon release (#14786) (126ebed).
operator: Update kube-rbac-proxy to upstream registry (#14809) (568d22f).
otlp: Write protobuf status on error (#15097) (63a2442).
pattern ingester: Remove 
enabled field in loki-local-config.yaml (#15059) (7410f10).
promtail: Fix API level node filtering of Kubernetes SD to only fetch pod metadata for pods of the current node (#14267) (d2b6fd8).
promtail: Fix Windows forward event crash. #16155 (1c0d558).
querier: Fix goroutine leak in queryrange downstreamer (#15665) (5f476a3).
querier: Legacy label endpoint route (#14933) (8ce321e).
querier: Mitigate ingester race between Query & GetChunkIDs (#15178) (bd46e4c).
querier: Show correct setting in error message (#14842) (f5ae015).
queries: Bytes unit detection in detected fields (#15525) (8e260fe).
Remove unsafe pkg usage from util.mempool (#15428) (e6d82b9).
ruler: Fix loki ruler generator url left parameter url encoding (#15601) (e347eb7).
Shut down delete client on local rule-evaluator (#15345) (42469cc).
storage: Data race in chunk client hedging tests (#15466) (fad06ee).
storage: Do not retain copy of chunk while indexing a new chunk in tsdb while processing delete requests (#15541) (ff19955).
structured metadata: Structured metadata not always passed down for processing deletes (#15526) (a3808b6).
wal: Ensure successive WAL replays don't overwrite each other (#14848) (ec95ed1).

v3.5
Grafana Labs and the Loki team are excited to announce the release of Loki 3.5. Here's a summary of new enhancements and important fixes.
For a full list of all changes and fixes, refer to the CHANGELOG.
Features and enhancements
Key features in Loki 3.5.0 include the following:
Helm charts: Loki 3.5 includes multiple improvements to the Helm charts, including: Adding support for additional global variables in the Helm chart (#16062) (e5f31b3), adding the 
extraContainers property to the backend Loki pods (#16205) (528a1c6)[closes #16164], adding 
tpl() support back to read, write and backend pods (#16664) (af05c38), and allowing provisioner to be namespaced (#16091) (af85ecf).
storage: Add support for `SQLite' as a database for storing delete requests (#16437) (5b33e65) and when using SQLite, use the stored completion time to reduce the requests considered for querytime filtering (#16600) (3277321).
Docs: This release includes a heavily revised meta-monitoring section to support new monitoring requirements for Grafana Enterprise Logs customers.
Other improvements include the following:
distributor: Add simulated latency (#16733) (7119f6e)
distributor: Do not enforce labels vs agg metric stream (#16696) (25c0620)
distributor: Enforce limits in distributors (#16705) (bc2111e)
distributor: Update log level discovery from json to detect nested fields (#16026) (86dc47b)
distributor: Use stream labels instead of hash in rate limit reasons (#16738) (fdf8632)
distributors: Add dry-run mode for checking limits in distributors (#16754) (d6c499b)
kafka: tenant topics (#15977) (c258419)
ksonnet: Assert for bad 
max_query_lookback configuration (#16362) (073c94c)
ksonnet: Add no_schedule_constraints option to distributors (#16029) (9e9a09d)
lambda-promtail: Improve relabel configuration parsing and testing (#16100) (2587f34)
logical planning: Planner playground (#16403) (89f0ed7)
logql: Add LogQL Benchmark Suite (#16434) (1b57297)
operator: Add configuration option for dropping OTLP attributes (#15857) (bd1ea23)
operator: Add support for Swift TLS CA configuration (#15260) (62a72f6)
operator: Enable time-based stream-sharding (#16390) (1b4f1f5)
operator: Update Loki operand to v3.4.2 (#16360) (42f87d3)
storage: Add S3 chunk delimiter configuration to support MinIO running on Windows (#16319) (8429b0f)
storage: Add a configuration option for custom GCS endpoints (#16419) (f1ebd97)
Deprecations
One of the focuses of Loki 3.0 was cleaning up unused code and old features that had been previously deprecated but not removed. Loki 3.0 removed a number of previous deprecations and introduces some new deprecations. Some of the main areas with changes include:
Promtail is deprecated, as the code has been merged into Grafana Alloy. You can find migration documentation and a utility to convert your Promtail configuration to Alloy configuration in the Alloy documentation. Note that this deprecation does NOT include the lambda-promtail client.
Deprecated storage options including the deprecation of the BoltDB store.
Deprecated configuration options.
API endpoint deprecations.
To learn more about breaking changes in this release, refer to the Upgrade guide.
{{< docs/shared source="alloy" lookup="agent-deprecation.md" version="next" >}}
Upgrade Considerations
For important upgrade guidance, refer to the Upgrade Guide.
BREAKING CHANGE - ksonnet: Clean-up deprecated configurations (#16007).
BREAKING CHANGE - operator: Add configuration option for dropping OTLP attributes (#15857).
Bug fixes
3.5.12 (2026-03-12)
deps: Upgrade go to 1.25.8 (#21116) (3b305e3)
3.5.11 (2026-02-23)
alloc: Set a limit on preallocations (backport release-3.5.x) (#20919) (1906dc2).
3.5.10 (2026-02-11)
deps:  Update to go 1.25.7 (#20694) (#20752) (dd71399).
build UI deps (#20762) (6d1e9ea).
3.5.9 (2025-12-10)
deps: Bump x/crypto module (#20016) (b21123b).
deps: Update go version to 1.24.11 (#20200) (79d8922).
logger: correct log level verbosity in rule evaluation (#19519) (backport release-3.5.x) (#19857) (562d87d).
3.5.8 (2025-11-05)
deps: Update to Go version 1.24.9 (#19702) (cf33543).
3.5.7 (2025-10-13)
This release fixes a build issue in 3.5.6 where it was not built with Go 1.24.8 as expected, but Go 1.24.1.
build: Update rpm signature config (#19476) (#19483) (d5b382b).
build: Update loki build image (#19473) (#19474) (c70476a).
3.5.6 (2025-10-10)
deps: Remove CVE GHSA-2464-8j7c-4cjm (#19469) (f2a304a).
deps: Update module github.com/twmb/franz-go/plugin/kprom to v1.2.1 (release-3.5.x) (#18793) (bb7356c).
deps: Update to Go version to 1.24.8 (#19454) (#19462) (1b8b5dd).
3.5.5 (2025-09-11)
otlp: Backport fix to correctly calculate entry metadata size (#19163) (5aa8bd2).
3.5.4 (2025-09-04)
deps: Update to Go version 1.24.6 in release-3.5.x (#19096) (ce8d705).
docs: added instructions for how to upgrade zone-aware ingesters. (#18663) (e0de7da).
3.5.3 (2025-07-23)
deps: Update to Go version 1.24.5 (#18412) (2aa4680).
WAL: Handle WAL corruption properly on startup (backport release-3.5.x) (#18408) (5b8ee9a).
3.5.2 (2025-04-22)
ci: Update release code 3.5 (#18014) (b1b28b0).
deps: Update to Go version 1.24.2 (backport release-3.5.x) (#17805) (d0fbad0).
deps: Update to Go version 1.24.5 (backport release-3.5.x) (#18412) (4642c63).
frontend: Allow resolution of v6 addresses. (backport release-3.5.x) (#18261) (aed4610).
jsonparser: Fix possible JSON log line corruption caused by 
json parser on query path (backport release-3.5.x) (#18059) (546d456).
memberlist: Allow resolution of advertise address from IPv6  interfaces defined in common 
instance_interface_names.(backport release-3.5.x) (#18257) (6f878ad).
WAL: Handle WAL corruption properly on startup (backport release-3.5.x) (#18408) (5b8ee9a).
3.5.1 (2025-05-19)
build: Bump loki-build-image for docker-driver (backport release-3.5.x) (#17741) (d4e637c).
3.5 (2025-04-22)
approx_topk: Map approx_topk operation in all cases (#16131) (6d691ac)
blooms: Initialize bloom gateway client only once (#15994) (e271e22)
builds: Update crc64nvme to fix build issues with arm images (#16413) (80a616d)
ci: do not disable CGO in Makefile (#16271) (810dff9)
ci: Pass image tag details to logcli docker build (#16159) (f22527f)
ci: Provide correct build info for release builds (#15939) (625bdab)
compactor: remove unused variable in delete requests store code (#16241) (5e6d0b5)
dataobj: Fix shutdown race in dataobj consumer (#16157) (656b1fa)
dataobj: Fixes timerange matching for dataobj files (#16222) (5aa9e47)
dataobj: Fixes timerange predicate (#16245) (5195296)
deps: Update dependency @hookform/resolvers to v4.1.3 (main) (#16524) (2869e92)
deps: Update dependency @radix-ui/react-checkbox to v1.1.4 (main) (#16297) (85a6e88)
deps: Update dependency @radix-ui/react-collapsible to v1.1.3 (main) (#16298) (4a1225e)
deps: Update dependency @radix-ui/react-dialog to v1.1.6 (main) (#16301) (548c8c2)
deps: Update dependency @radix-ui/react-dropdown-menu to v2.1.6 (main) (#16303) (5fedc3c)
deps: Update dependency @radix-ui/react-hover-card to v1.1.6 (main) (#16304) (f11fd12)
deps: Update dependency @radix-ui/react-label to v2.1.2 (main) (#16305) (13bfb12)
deps: Update dependency @radix-ui/react-popover to v1.1.6 (main) (#16306) (abb3a94)
deps: Update dependency @radix-ui/react-progress to v1.1.2 (main) (#16307) (18210e9)
deps: Update dependency @radix-ui/react-scroll-area to v1.2.3 (main) (#16308) (9136ba3)
deps: Update dependency @radix-ui/react-select to v2.1.6 (main) (#16310) (1115a77)
deps: Update dependency @radix-ui/react-separator to v1.1.2 (main) (#16311) (aee6e88)
deps: Update dependency @radix-ui/react-slot to v1.1.2 (main) (#16313) (9b5f33c)
deps: Update dependency @radix-ui/react-switch to v1.1.3 (main) (#16314) (7fa049e)
deps: Update dependency @radix-ui/react-tabs to v1.1.3 (main) (#16316) (96b4e8f)
deps: Update dependency @radix-ui/react-toast to v1.2.6 (main) (#16320) (9b47b19)
deps: Update dependency @radix-ui/react-toggle to v1.1.2 (main) (#16323) (8fd8697)
deps: Update dependency @radix-ui/react-toggle-group to v1.1.2 (main) (#16326) (677faad)
deps: Update dependency @radix-ui/react-tooltip to v1.1.8 (main) (#16327) (c084144)
deps: Update dependency @tanstack/react-query to v5.66.5 (main) (#16328) (dbf0161)
deps: Update dependency @tanstack/react-query to v5.66.6 (main) (#16341) (24137bf)
deps: Update dependency @tanstack/react-query to v5.66.7 (main) (#16352) (877243c)
deps: Update dependency @tanstack/react-query to v5.66.9 (main) (#16421) (d8f0c06)
deps: Update dependency @tanstack/react-query to v5.67.1 (main) (#16561) (8641a02)
deps: Update dependency @tanstack/react-query to v5.67.2 (main) (#16621) (15c498c)
deps: Update dependency @tanstack/react-query to v5.67.3 (main) (#16685) (2935a74)
deps: Update dependency @tanstack/react-query to v5.68.0 (main) (#16757) (58840c5)
deps: Update dependency @tanstack/react-query to v5.69.0 (main) (#16808) (4161c76)
deps: Update dependency @tanstack/react-query-devtools to v5.66.5 (main) (#16331) (dbb1716)
deps: Update dependency @tanstack/react-query-devtools to v5.66.6 (main) (#16343) (2d707d9)
deps: Update dependency @tanstack/react-query-devtools to v5.66.7 (main) (#16353) (412f733)
deps: Update dependency @tanstack/react-query-devtools to v5.66.9 (main) (#16423) (48e00b5)
deps: Update dependency @tanstack/react-query-devtools to v5.67.1 (main) (#16563) (0f1dce8)
deps: Update dependency @tanstack/react-query-devtools to v5.67.2 (main) (#16622) (7276ae2)
deps: Update dependency @tanstack/react-query-devtools to v5.67.3 (main) (#16686) (029abea)
deps: Update dependency @tanstack/react-query-devtools to v5.68.0 (main) (#16756) (02a6800)
deps: Update dependency @tanstack/react-query-devtools to v5.69.0 (main) (#16810) (6109de3)
deps: Update dependency @types/lodash to v4.17.16 (main) (#16516) (7c9b44a)
deps: Update dependency @types/react-datepicker to v7 (main) (#16480) (1796269)
deps: Update dependency cmdk to v1.1.1 (main) (#16766) (06553d8)
deps: Update dependency date-fns to v4 (main) (#16481) (5b7f1f0)
deps: Update dependency lucide-react to ^0.475.0 (main) (#16344) (2f57d60)
deps: Update dependency lucide-react to ^0.476.0 (main) (#16455) (32fd332)
deps: Update dependency lucide-react to ^0.477.0 (main) (#16509) (f84f0c7)
deps: Update dependency lucide-react to ^0.478.0 (main) (#16623) (31d8648)
deps: Update dependency lucide-react to ^0.479.0 (main) (#16630) (1e1e7a9)
deps: Update dependency lucide-react to ^0.481.0 (main) (#16765) (a354eee)
deps: Update dependency lucide-react to ^0.482.0 (main) (#16767) (901320c)
deps: Update dependency lucide-react to ^0.483.0 (main) (#16811) (de6f0e9)
deps: Update dependency next-themes to v0.4.5 (main) (#16655) (2507888)
deps: Update dependency next-themes to v0.4.6 (main) (#16697) (c515a4e)
deps: Update dependency react to v19 (main) (#16000) (1b35cb6)
deps: Update dependency react-code-block to v1.1.3 (main) (#16428) (9691e94)
deps: Update dependency react-datepicker to v8.1.0 (main) (#16346) (4a674b2)
deps: Update dependency react-datepicker to v8.2.0 (main) (#16662) (7a70860)
deps: Update dependency react-datepicker to v8.2.1 (main) (#16749) (20eeecb)
deps: Update dependency react-icons to v5.5.0 (main) (#16456) (bce5f55)
deps: Update dependency react-router-dom to v6.29.0 (main) (#16347) (14ce844)
deps: Update dependency react-router-dom to v7 (main) (#16515) (976cbbb)
deps: Update dependency react-router-dom to v7.1.4 (main) (#16018) (f163e44)
deps: Update dependency react-router-dom to v7.1.5 (main) (#16046) (45a0c59)
deps: Update dependency react-router-dom to v7.3.0 (main) (#16615) (23acfcb)
deps: Update dependency tailwind-merge to v3 (main) (#16518) (c8f9474)
deps: Update dependency zod to v3.24.2 (main) (#16333) (ec5bef4)
deps: Update github.com/prometheus/prometheus (main) (#15950) (b37eefe)
deps: Update module cloud.google.com/go/pubsub to v1.46.0 (main) (#16019) (998da11)
deps: Update module cloud.google.com/go/pubsub to v1.47.0 (main) (#16047) (8f9f92f)
deps: Update module cloud.google.com/go/pubsub to v1.48.0 (main) (#16724) (b07bb72)
deps: Update module cloud.google.com/go/storage to v1.51.0 (main) (#16720) (b4abbf4)
deps: Update module github.com/aws/aws-sdk-go-v2 to v1.36.1 (main) (#16024) (542cff4)
deps: Update module github.com/aws/aws-sdk-go-v2 to v1.36.2 (main) (#16430) (ba9afd6)
deps: Update module github.com/aws/aws-sdk-go-v2/config to v1.29.3 (main) (#16022) (03f18ac)
deps: Update module github.com/aws/aws-sdk-go-v2/config to v1.29.4 (main) (#16042) (0bf0253)
deps: Update module github.com/aws/aws-sdk-go-v2/config to v1.29.5 (main) (#16093) (c4e916d)
deps: Update module github.com/aws/aws-sdk-go-v2/config to v1.29.6 (main) (#16123) (d28e627)
deps: Update module github.com/aws/aws-sdk-go-v2/config to v1.29.7 (main) (#16431) (ca41297)
deps: Update module github.com/aws/aws-sdk-go-v2/config to v1.29.8 (main) (#16503) (364e80e)
deps: Update module github.com/aws/aws-sdk-go-v2/config to v1.29.9 (main) (#16542) (2cb3b3a)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.75.0 (main) (#16003) (e3d1054)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.75.1 (main) (#16023) (d2b7d82)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.75.2 (main) (#16043) (01358cf)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.75.3 (main) (#16094) (1d6c0eb)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.75.4 (main) (#16124) (0c44e76)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.76.0 (main) (#16140) (f70984c)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.76.1 (main) (#16173) (a93a3ab)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.77.0 (main) (#16356) (879168b)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.77.1 (main) (#16433) (92bdb9d)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.78.0 (main) (#16513) (b7b5c76)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.78.1 (main) (#16548) (81aab19)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.78.2 (main) (#16694) (9a99859)
deps: Update module github.com/axiomhq/hyperloglog to v0.2.5 (main) (#16504) (dcda340)
deps: Update module github.com/azure/go-autorest/autorest to v0.11.30 (main) (#15987) (543b7f3)
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.216 (main) (#16177) (9690cad)
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.217 (main) (#16276) (b262a2b)
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.219 (main) (#16612) (ade0de6)
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.221 (main) (#16760) (e0e7133)
deps: Update module github.com/bmatcuk/doublestar/v4 to v4.8.1 (main) (#15973) (a8bd3a8)
deps: Update module github.com/charmbracelet/bubbletea to v1.3.4 (main) (#16730) (2f99d84)
deps: Update module github.com/charmbracelet/lipgloss to v1.1.0 (main) (#16742) (53ae070)
deps: Update module github.com/datadog/sketches-go to v1.4.7 (main) (#16436) (1907238)
deps: Update module github.com/docker/docker to v28 (main) (#16520) (722ce05)
deps: Update module github.com/fsouza/fake-gcs-server to v1.52.2 (main) (#16334) (20b9bd2)
deps: Update module github.com/golang/snappy to v1 (main) (#16620) (2306f0b)
deps: Update module github.com/google/go-cmp to v0.7.0 (main) (#16457) (4f7fd29)
deps: Update module github.com/grafana/loki/v3 to v3.4.0 (main) (#16217) (750fe66)
deps: Update module github.com/grafana/loki/v3 to v3.4.1 (main) (#16265) (cc49ebc)
deps: Update module github.com/grafana/loki/v3 to v3.4.2 (main) (#16335) (977fd54)
deps: Update module github.com/grpc-ecosystem/go-grpc-middleware/v2 to v2.3.0 (main) (#16357) (5fe1fa7)
deps: Update module github.com/grpc-ecosystem/go-grpc-middleware/v2 to v2.3.1 (main) (#16613) (9d8a310)
deps: Update module github.com/hashicorp/consul/api to v1.31.1 (main) (#16230) (70d3863)
deps: Update module github.com/hashicorp/consul/api to v1.31.2 (main) (#16361) (2f4f16d)
deps: Update module github.com/heroku/x to v0.4.3 (main) (#16204) (12e436a)
deps: Update module github.com/ibm/go-sdk-core/v5 to v5.19.0 (main) (#16647) (16e87a6)
deps: Update module github.com/ibm/ibm-cos-sdk-go to v1.12.2 (main) (#16728) (22cdf61)
deps: Update module github.com/ibm/sarama to v1.45.1 (main) (#16522) (bc9bbb8)
deps: Update module github.com/influxdata/telegraf to v1.33.2 (main) (#16179) (022b45c)
deps: Update module github.com/influxdata/telegraf to v1.33.3 (main) (#16444) (01c9117)
deps: Update module github.com/influxdata/telegraf to v1.34.0 (main) (#16669) (5d0e6fb)
deps: Update module github.com/klauspost/compress to v1.18.0 (main) (#16458) (6c0f67d)
deps: Update module github.com/minio/minio-go/v7 to v7.0.85 (main) (#16152) (be047cb)
deps: Update module github.com/minio/minio-go/v7 to v7.0.87 (main) (#16442) (c92e339)
deps: Update module github.com/minio/minio-go/v7 to v7.0.88 (main) (#16657) (bf99989)
deps: Update module github.com/parquet-go/parquet-go to v0.25.0 (main) (#16459) (4d65435)
deps: Update module github.com/prometheus/alertmanager to v0.28.1 (main) (#16634) (57c14ac)
deps: Update module github.com/prometheus/client_golang to v1.21.0 (main) (#16446) (c128aa8)
deps: Update module github.com/prometheus/client_golang to v1.21.1 (main) (#16552) (0a697f9)
deps: Update module github.com/prometheus/prometheus to v0.302.1 (main) (#16475) (9fe8b57)
deps: Update module github.com/prometheus/prometheus to v2 (main) (#16004) (dc01eb9)
deps: Update module github.com/prometheus/sigv4 to v0.1.2 (main) (#16447) (946497e)
deps: Update module github.com/redis/go-redis/v9 to v9.7.1 (main) (#16448) (2cc1873)
deps: Update module github.com/shirou/gopsutil/v4 to v4.25.1 (main) (#16049) (319bb0f)
deps: Update module github.com/shirou/gopsutil/v4 to v4.25.2 (main) (#16517) (fcf1518)
deps: Update module github.com/spf13/afero to v1.14.0 (main) (#16769) (342fd40)
deps: Update module github.com/tjhop/slog-gokit to v0.1.4 (main) (#16505) (c409a66)
deps: Update module go.etcd.io/bbolt to v1.4.0 (main) (#16107) (78becba)
deps: Update module go.opentelemetry.io/collector/pdata to v1.25.0 (main) (#16076) (2ec948c)
deps: Update module go.opentelemetry.io/collector/pdata to v1.26.0 (main) (#16359) (043617e)
deps: Update module go.opentelemetry.io/collector/pdata to v1.27.0 (main) (#16564) (2b99806)
deps: Update module go.opentelemetry.io/collector/pdata to v1.28.0 (main) (#16797) (65f90c7)
deps: Update module go.opentelemetry.io/otel to v1.35.0 (main) (#16580) (ec2c9dc)
deps: Update module go.opentelemetry.io/otel/trace to v1.35.0 (main) (#16581) (378af0c)
deps: Update module golang.org/x/crypto to v0.33.0 (main) (#16149) (0c20717)
deps: Update module golang.org/x/crypto to v0.35.0 (main) (#16452) (41c1cfc)
deps: Update module golang.org/x/crypto to v0.36.0 (main) (#16575) (e9f2ee1)
deps: Update module golang.org/x/net to v0.35.0 (main) (#16167) (f084f02)
deps: Update module golang.org/x/net to v0.36.0 (main) (#16567) (c84d7e9)
deps: Update module golang.org/x/net to v0.37.0 (main) (#16582) (133c884)
deps: Update module golang.org/x/oauth2 to v0.26.0 (main) (#16085) (9ddc756)
deps: Update module golang.org/x/oauth2 to v0.27.0 (main) (#16453) (b731680)
deps: Update module golang.org/x/oauth2 to v0.28.0 (main) (#16568) (f86c43c)
deps: Update module golang.org/x/sync to v0.11.0 (main) (#16086) (536dbc6)
deps: Update module golang.org/x/sync to v0.12.0 (main) (#16569) (9052cc2)
deps: Update module golang.org/x/sys to v0.30.0 (main) (#16087) (6d49d91)
deps: Update module golang.org/x/sys to v0.31.0 (main) (#16570) (173e75c)
deps: Update module golang.org/x/text to v0.22.0 (main) (#16090) (8c2406e)
deps: Update module golang.org/x/text to v0.23.0 (main) - abandoned (#16576) (97981a5)
deps: Update module golang.org/x/time to v0.10.0 (main) (#16089) (fe17cc0)
deps: Update module golang.org/x/time to v0.11.0 (main) (#16571) (a38bba9)
deps: Update module google.golang.org/api to v0.219.0 (main) (#15991) (d2e6d99)
deps: Update module google.golang.org/api to v0.220.0 (main) (#16115) (0528e40)
deps: Update module google.golang.org/api to v0.221.0 (main) (#16234) (a66faf5)
deps: Update module google.golang.org/api to v0.223.0 (main) (#16460) (861affe)
deps: Update module google.golang.org/api to v0.224.0 (main) (#16614) (7dfe4a5)
deps: Update module google.golang.org/api to v0.225.0 (main) (#16690) (b243d42)
deps: Update module google.golang.org/api to v0.226.0 (main) (#16752) (745bf67)
deps: Update module google.golang.org/grpc to v1.71.0 (main) (#16578) (f989bd0)
deps: Update module google.golang.org/protobuf to v1.36.5 (main) (#16130) (b1c1c5a)
deps: Update module k8s.io/apimachinery to v0.32.2 (main) (#16267) (abcd730)
deps: Update module k8s.io/apimachinery to v0.32.3 (main) (#16704) (0be4897)
distributor: Determine when all logs have been filtered (#16073) (f2bff20)
distributor: Fix blocked ingestion returned error when 260 (#16387) (3d6163a)
distributor: Increment UsageTracker.DiscardedBytesAdd() for ingestion policy discards (#16528) (a4c796d)
distributor: Remove colons from level detection (#16764) (865c43a)
distributor: Zeroed stream hash when checking limits (#16729) (ab0cbce)
docs: Add a note on docker configuration.md doc to explain accepted values for boolean type (#16299) (2071a50)
engine: Fix 
Walk() function implementation on various 
Expr implementations (#16033) (3888866)
frontend: Fix a panic in ServeHTTP where stream was nil (#16818) (ba7550a)
frontend: Fix a data race I added changing this to a map (#16814) (f4893ee)
helm: Change object_store.prefix to object_store.storage_prefix in helm (#16266) (b5ad4f0)
helm: Disable service monitor for nginx service (#12746) (0c38b94)
ksonnet: Revert "feat: Assert for bad 
max_query_lookback configuration (#16362)" (#16441) (42ddb68)
ksonnet: Scheduling constraints (#16045) (b45a31e)
ksonnet: Clean-up deprecated configurations (#16007) (76183fa)
logql: Adjust with offset in last, first and quantile over time queries. (#15915) (83339cb)
memchunk: Skip a test that currently is not used in our main path (#16474) (196cbaa)
nix: Build with go 1.24 (#16579) (f1ac3c4)
operator: Fix minimum available ingesters for 1x.pico size (#16035) (40cf074)
operator: Select non-zero delete worker count for all sizes (#16492) (1e5579a)
operator: Update maximum OpenShift version (#16443) (ddf3cfb)
operator: Update OTLP user guide to reflect change in LokiStack (#16057) (14e2c87)
operator: Update skipRange in OpenShift variant (#15984) (dfbe00c)
pattern: Correctly initialize overrides for pattern tee. (#16112) (8f38cf7)
policies: Make policy test non-flaky (#16158) (41d9c8b)
promtail: Remove flaky TestFileTarget_StopsTailersCleanly (#16473) (2ab63d2)
promtail: Fix Windows forward event crash (#16155) (82cfaea)
querier: Fix label values query when server.http_path_prefix is set (#15978) (65f5990)
querier: Only allow aggregated metric queries from logs drilldown app (#16670) (6c6410c)
queryrange: Properly handle context cancellation in Downstreamer (#16471) (5335a21)
Reverts the tailwindcss upgrade that was breaking builds (#16058) (bb76772)
ruler: Ensure test service shutdown is properly awaited (#16468) (8976709)
structured metadata: Export ExcludedMetadataLabels so it can be extended in GEL (#16083) (9d84a68)
tsdb: Export 
exportTSInSecs field in TSDB identifier to make sure correct gap calculation (#13116) (99d9f1c)

v3.6
Grafana Labs and the Loki team are excited to announce the release of Loki 3.6. Here's a summary of new enhancements and important fixes.
For a full list of all changes and fixes, refer to the CHANGELOG
{{< admonition type="note" >}}
See the deprecations section for important information about the Loki Helm charts.
{{< /admonition >}}
Features and enhancements
Key features in Loki 3.6.0 include the following:
Horizontally scalable compactor: The Compactor in Loki is a singleton service that compacts the index and applies retention, including processing delete requests. Because it can only be scaled vertically, it limits how many resources you can add to speed things up when the workload increases in bigger cells. This release adds a horizontally scalable compactor, to speed up processing large delete requests or catching up if there has been a lag in processing for some reason. Note that the horizontally scalable compactor delegates deletion work to the workers. Loki continues to perform compaction and apply retention in the main Compactor. For more information, refer to the documentation.
OTel: Loki has been refactored to use the OpenTelemetry tracing library instead of the OpenTracing library. (#17859). Note that Loki can still be configured using 
JAEGER_ env vars and still exports traces in 
JAEGER_ format.
meta-monitoring: Moves the meta-monitoring responsibilities from the Grafana meta-monitoring Helm chart to the widely supported Grafana Kubernetes Monitoring Helm chart. This was implemented in two PRs: Meta Monitoring Loki v2 (Part 1) - Telemetry Collection (#16665) (2d0bbfd) and Meta Monitoring Loki v2 (Part 2) - Loki Mixin's (#16962) (93360f6).
Helm charts: The Grafana Developer Advocates launched a pilot Community Helm Maintainers group at the end of June made up of Grafana Champions who expressed interest in helping maintain the Loki Helm chart. We have resumed regular releases of the Loki Helm chart and reduced the backlog of open Loki Helm chart pull requests from around 150 to less than 40. For information about recent changes, refer to the Helm chart CHANGELOG.
UI: We have moved the Loki UI to a Grafana Plugin. The static Operational UI is not going away, and all the APIs that serve it will remain in Loki. The JavaScript that serves the UI, however, has been moved to the Loki Operational UI Grafana plugin (#19390).
Docs: Published a third Cloud Deployment Guide, this one for Google Cloud Platform (#17503). Added links to three log related Learning paths to the Loki docs (#17956). Revised the LogQL query page (#16286) and added a LogQL video (#18111). Added a caution to the Labels topic and added a new topic with instructions how to modify the default OpenTelemetry resource attributes that Loki converts to labels (#17958). Updated the caching topic (#17420).
{{< admonition type="note" >}}
Note that when appropriate, documentation updates have been backported to previous releases.
{{< /admonition >}}
Other improvements include the following:
analytics: Change how analytics proxy works (#19098)
API: Add endpoint returning tenant configured limits (#17101) (6818c90)
API: Loki Improvement Document (LID) for query splitting (#16958) (66f0f0b)
compactor HS: Add a dashboard for monitoring deletion with horizontally scalable compactor (#18588) (10db8ab)
compactor HS: Add a job runner for processing of deletion jobs (#18058)
compactor HS: Add job queue and deletion job builder (#17843)
compactor HS: Add jsonnet for deploying horizontally scalable compactor (#18550) (5e28950)
compactor HS: Add support for applying storage updates after finishing processing of deletion manifest (#18294) (ba3c670)
compactor HS: Add support for worker for processing of jobs from the compactor's job queue (#18165) (d05c4bc)
compactor HS: Fixes and changes for getting horizontally scalable compactor to work well (#18526) (000f90d)
compactor HS: Refactor interfaces and build a link between delete requests manager and compactor for HS of compactor (#18326) (3646607)
compactor HS: Store index updates per processed segment in the manifest to the object storage (#18212) (8007e22)
compactor HS: Tune compactor worker config and add experimental feature warning to the docs (#18539) (5bf3f74)
compactor HS: Use streaming RPCs for interacting with job queue over grpc (#18125) (385e317)
compactor HS: Wire things up and add some metrics (#18421) (bd29320)
compactor HS: Add a metric to track the number of jobs left to conclude ongoing unit of work (#19506)
distributor: Add MaxRecvMsgSize config for uncompressed message size limits (#16915) (1d99f4d)
distributor: Log ingestion policy when log_push_request is set (#18712) (0ec787f)
distributor: Mark truncated log lines with identifier (#18262) (0ee8e76)
distributor: Stop pushing to the metadata topic from distributors (#17525) (e0f015c)
distributor: Remove tenant topic tee (#19082)
fluentbit: Update fluent/fluent-bit to v4 which adds support for structured metadata to the out_loki plugin (#17200) (f108e1c)
fluentd: Add 'compress gzip' setting to fluentd plugin (#18995)
helm: Add 
block_builder config to helm chart (#17451) (a9808a1)
helm: Add 
startupProbe to memcached(#19166)
helm: Add 
startupProbe to memcached exporter(#19165)
helm: Add configurable livenessProbe to single binary StatefulSet(#19072)
helm: Add dnsconfig to all missing resources(#18903)
helm: Add FOLDER_ANNOTATION logic for sidecar container in loki helm chart, making multi-tenant alerting easier (#13289) (833fcbc)
helm: Add health probes to admin-api (#18546) (358228a)
helm: Add health probes to bloom-builder (#18549)
helm: Add health probes to bloom-gateway(#18551)
helm: Add health probes to bloom-planner(#18552)
helm: Add health probes to gateway component(#18545)
helm: Add health probes to loki in backend(#18548)
helm: Add location snippet to nginx config (#18105) (5737751)
helm: Add missing revisionHistoryLimit (#18638) (d0e0d1a)
helm: Add more specificity to affinity and topology spread constraints(#19061)
helm: Add namespaceOverride(#18707)
helm: Add non-default-values.yaml to increase test coverage (#18360) (b0e04f4)
helm: Add option to use deployment type instead of daemonset in canary (#17364) (a6563cf)
helm: Add PDB for patternIngester (#17058) (7fad38d)
helm: Add readiness and liveness support for all memcached components (#18525) (9b54d9f)
helm: Add readiness probe for memcached (#15609) (6ed1461)
helm: Add replicas to loki-canary deployment (#19095)
helm: Add startupProbe to loki-sc-rules (#18547) (121e84e)
helm: Add support for custom alert rule annotations (#19193)
helm: Add tpl support for pattern_ingester and ingester_client (#16759) (901b2b0)
helm: Added trafficDistribution for service-distributor (#17742) (78f154f)
helm: Adds a flag for tokengen rbac components so that they can be disabled optionally (#15882) (10536b2)
helm: Allow configuring service types (#18708)
helm: Allow enabling user namespaces (#18661)
helm: Allow external memcached setup (#17432) (013aa17)
helm: Allow extraObjects to be defined as either a list or dictionary.(#13252)
helm: Allow passing tenant password hash instead of password (#17049) (20b97f7)
helm: Allow setting labels for ingesters (#18536) (ab426be)
helm: Enable rules sidecar for the ruler pods (#17231) (2c5ad02)
helm: Expose all storage configuration (#18509) (db5e021)
helm: Expose labels on volumeClaimTemplates (#13317) (cdefbbf)
helm: Expose Loki UI in pure ingress (#18400) (21f3d09)
helm: Helm issue templates (#18693) (e0ec815)
helm: Implementing support of l2 chunksCache memcache (#17556) (039e7c1)
helm: Integrate the enterprise-provisioner into Loki helm chart (#17616)
helm: Make access modes for persistence on all PVCs and StatefulSets editable (#13474)
helm: Only delete PVC on statefulset deletion, not scaling (#19097)
helm: Support bypass all s3/gcs/azure config in object_store (#17597) (d6c2a8e)
helm: Support separate loki ruler storage configuration (#18510)
helm: Support templated loki.operational_config (#17045) (078f353)
helm: Update Grafana Loki version to 3.5.3 (#18619) (907bb61)
helm: Update Loki version in charts (#18447) (e2770ae)
helm: Use named ports for memcached services targetports (#18990)
helm: Delete statefulset if volumeClaimTemplates changes (#19217)
helm: Added affinity property to the loki-canary DaemonSet (#17720)
ingester: Add ingester_chunks_flush_requests_total (#18696)
ingester: Add per policy overridable stream limits (#18994)
kafka: Support multiple Kafka clients in component (#17757)
kafka: Adds partition-ingester push latency histogram (#17385) (01cf36d)
kafka: Optionally enable Kafka histograms for read/write latency (#17089) (e962398)
kafka: Refactors kafka/client to improve metric collection (#17308) (ab65ac5)
kafka: Support consuming records from Kafka (#17691)
limits: Add ability to filter tenant limits endpoint by allowlist (#18521) (73de7b5)
logcli: Add delete commands to logcli (#18375) (f38950a)
logcli: Add 
--log.level parameter to help with diagnosing errors when using logcli. (#19136)
logs drilldown: Return metric query partial results for drilldown (#18286) (c0c2265)
logs drilldown: Drilldown config endpoint (#19028)
logs drilldown: Implement 
unwrap as a projection
(#19409)
logs drilldown: Persist patterns as aggregated metrics (#17737)
logs drilldown: Query persisted patterns (#17980)
loki-canary: Update loki canary to support batched log push (#17558)
metrics: Adds distributor lag counter to push.go (#18012)
metrics: Adds User-Agent to distributor push logging (#17764)
mixins: Allow disabling Thanos dashboard and fix dashboard's file name (#18749)
operator: Add alert LokiIngesterFlushFailureRateCritical (#18698)
operator: Add warning alert for when LokiStack is not getting ready (#19258)
operator: Allow users to configure virtual-host-style for S3 secrets (#17363) (c4dace5)
operator: Allow using resource requests as limits (#17484) (0b2bae8)
operator: Deploy network policies with LokiStack (#19099)
operator: Extend LokiStack authorization to support OTel Semantics (#16031)
operator: S3 validation reject endpoints that contain a URL path (#19356)
operator: Update Loki operand to v3.5.5 (#19187)
pattern ingester: Pattern ingester pushes detected level as structured metadata (#16924) (66033bb)
pattern ingester: Implement volume-based pattern filtering (#18838)
policies: Default ingestion policy mappings merged with per-tenant mappings (#18926)
thanos: Allow dashes in storage_prefix config (#16934) (c01d9c7)
3.6.4 (2025-01-21)
ruler: Add rules check for namespace and group (#20437) (#20463) (7733ab1).
3.6.5 (2025-02-05)
loki: Add loki health command (backport release-3.6.x) (#20590) (dfdbe2a)
3.6.16 (2026-08-27)
compactor: Add flag to ignore missing chunks during deletion of logs with line filter (#24232) (b87159f).
Deprecations
3.6.5 (2025-02-05)
Simple Scalable Deployment (SSD) mode is being deprecated. The deprecation notice was posted in the CHANGELOG for Helm charts version 6.52.0. The timeline for the deprecation is to be determined (TBD), but will happen before Loki 4.0 is released.
Loki Helm charts deprecations: The following Community maintained Helm charts have been deprecated:
LGTM-distributed
loki-canary
loki-distributed
loki-simple-scalable
{{< admonition type="note" >}}
In June 2025, Grafana recruited three Grafana Champions and launched a three month pilot program to maintain the Loki Helm chart. The pilot proved successful, attracting more Grafana champions, and led to the Champions asking to take over maintenance of the chart entirely. The grafana-community/helm-charts repository launched on January 30, 2026. Effective March 16, 2026, the Grafana Loki Helm chart will be forked to the new repository grafana-community/helm-charts. The motivation behind this change is to accelerate development and enable direct community maintenance, as outlined in grafana/helm-charts#3955. The Champions will attempt to make this transition as seamless as possible.
{{< /admonition >}}
3.6.0 (2025-11-17)
One of the focuses of Loki 3.0 was cleaning up unused code and old features that had been previously deprecated but not removed. Loki 3.0 removed a number of previous deprecations and introduces some new deprecations. Some of the main areas with changes include:
Grafana Agent - Long term support for Grafana Agent ended October 31, 2025. For information on how to migrate to Grafana Alloy, see the migration docs.
Promtail is deprecated, as the code has been merged into Grafana Alloy. Commercial support will end February 28, 2026. You can find migration documentation and a utility to convert your Promtail configuration to Alloy configuration in the Alloy documentation. Note that this deprecation does NOT include the lambda-promtail client.
Deprecated storage options including the deprecation of the BoltDB store.
Deprecated configuration options.
API endpoint deprecations.
To learn more about breaking changes in this release, refer to the Upgrade guide.
Upgrade Considerations
For important upgrade guidance, refer to the Upgrade Guide.
BREAKING CHANGE - Loki UI: The Loki UI has been moved to a Grafana Plugin. Enabling the UI in the helm chart will now only enable the APIs needed by the plugin, and will host them on the querier. The gateway will now forward all UI requests to the queriers.(#19390)
Bug fixes
3.6.19 (2026-10-20)
security: Bump Go to 1.26.10 [SECURITY] (release-3.6.x) (#25457) (de41836)
3.6.18 (2026-10-15)
deps: Update dependencies related to otel for 3.6 (#25180) (603de5a).
security/LOW/: Update module go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc to v1.45.0 (#24619) (3e7c0ed).
security/LOW/: Update module go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp to v1.45.0 (#24620) (ad5a87c).
security: Bump Go to 1.26.9 (#25193) (c969f5c).
3.6.17 (2026-09-09)
security/HIGH/pkg/push: Update module google.golang.org/grpc to v1.83.1 (#24330) (d8d78f6).
security/HIGH/pkg/push: Update module google.golang.org/grpc to v1.83.2 (#24444) (6d21d6b).
security/HIGH/: Update module google.golang.org/grpc to v1.83.1 (#24329) (ed9df58).
security/HIGH/: Update module google.golang.org/grpc to v1.83.2 (#24430) (ee0ef56).
security/MEDIUM/: Update module github.com/containerd/containerd/v2 to v2.0.12 (#24461) (7fa1224).
security/UNKNOWN/: Update module golang.org/x/crypto to v0.55.0 (#24256) (85b2d4f).
security/UNKNOWN/: Update module golang.org/x/crypto to v0.56.0 (#24336) (57bc922).
security/UNKNOWN/: Update module golang.org/x/mod to v0.40.0 (#23963) (c08bf30).
3.6.16 (2026-08-27)
security/UNKNOWN/: Update module go.etcd.io/etcd/client/pkg/v3 to v3.5.33 (#24062) (bf4b367).
storage: Pre-compute SHA-256 to avoid aws-chunked on PutObject (#24214) (0a29895).
3.6.15 (2026-08-05)
queryrange: Preserve sketch in MergeLabels (#23771) (20087cd)
3.6.14 (2026-08-04)
ci: Backport deb/rpm packaging fix to release-3.6.x (#23680) (#23722) (d7ee4de)
ingester: Backport WAL replay hang fix to release-3.6.x (#21176) (#23630) (355e2f3)
security/UNKNOWN/: Update module github.com/klauspost/compress to v1.18.7 (release-3.6.x) (#23614) (8308a12)
security/UNKNOWN/cmd/chunks-inspect: Update module github.com/klauspost/compress to v1.18.7 (release-3.6.x) (#23615) (a6cc2e5)
security/UNKNOWN/cmd/dataobj-inspect: Update module github.com/klauspost/compress to v1.18.7 (release-3.6.x) (#23616) (a71c910)
security/UNKNOWN/operator: Update module github.com/klauspost/compress to v1.18.7 (release-3.6.x) (#23618) (59970a9)
3.6.13 (2026-07-23)
ci: Fix zizmor findings for operator-images in Loki 3.6 (#22821) (050f742)
ci: Helm CI warning fix (#22606) (4bc2586)
deps: Update objstore to include fix for GCS Exists (#23381) (87383d8)
security/CRITICAL/clients/cmd/fluentd/docker: Update dependency fluentd to v1.19.3 (release-3.6.x) (#22694) (692f2b1)
security/HIGH/: Bump Go to 1.26.5 to address CVE-2026-39822 and CVE-2026-42505 (#23398) (fc478e1)
security/HIGH/: Update golang.org/x/net, golang.org/x/text and google.golang.org/grpc (70f75b7)
security/HIGH/pkg/push: Update module google.golang.org/grpc to v1.82.1 (70f75b7)
security/UNKNOWN/: Update module github.com/containerd/containerd/v2 to v2.0.10 (release-3.6.x) (#22479) (697bd83)
security/UNKNOWN/cmd/chunks-inspect: Update go toolchain directive to v1.25.12 (release-3.6.x) (#23131) (b327439)
security/UNKNOWN/pkg/push: Update module golang.org/x/net to v0.56.0 (70f75b7)
security/UNKNOWN/pkg/push: Update module golang.org/x/text to v0.39.0 (70f75b7)
security: Backport security updates to release-3.6.x (#23403) (70f75b7)
3.6.12 (2026-06-24)
deps: Update 
go.opentelemetry.io/otel/sdk to v1.43.0 (#21480) (47fb29e).
deps: Update 
golang.org/x/crypto to v0.52.0 (#22212) (424e5ff).
deps: Update 
golang.org/x/net to v0.55.0 (#22213) (90ae8f3).
deps: Update 
golang.org/x/sys to v0.44.0 (#22214) (49b9a22).
deps: Update 
golang.org/x/sys to v0.44.0 in cmd/dataobj-inspect (release-3.6.x) (#22215) (c696c66).
deps: Update 
golang.org/x/crypto to v0.52.0 in cmd/segment-inspect (release-3.6.x) (#22216) (807ffd7).
deps: Update 
golang.org/x/net to v0.55.0 in cmd/segment-inspect (release-3.6.x) (#22217) (7081875).
deps: Update 
golang.org/x/sys to v0.44.0 in cmd/segment-inspect (release-3.6.x) (#22218) (36deca1).
deps: Update 
golang.org/x/net to v0.55.0 in pkg/push (release-3.6.x) (#22224) (0c53ce1).
deps: Update go toolchain to v1.25.11 in cmd/chunks-inspect (release-3.6.x) (#22202) (90e12c9).
deps: Update go toolchain to v1.25.11 in cmd/segment-inspect (release-3.6.x) (#22203) (2c67a33).
security/HIGH: Update 
github.com/containerd/containerd/v2 to v2.0.9 (#22204) (ed38a68).
security/HIGH: Update 
github.com/prometheus/prometheus to v0.311.3 in cmd/segment-inspect (release-3.6.x) (#22205) (bfd37e4).
security/HIGH: Update 
go.opentelemetry.io/otel to v1.41.0 in cmd/segment-inspect (release-3.6.x) (#22206) (0ed7bce).
security/MEDIUM: Update 
github.com/grafana/loki/v3 to v3.6.4 in cmd/dataobj-inspect (release-3.6.x) (#22209) (d6117ed).
security/MEDIUM: Update 
github.com/grafana/loki/v3 to v3.6.4 in cmd/segment-inspect (release-3.6.x) (#22210) (186b9a9).
3.6.11 (2025-05-13)
deps: CVEs in release 3.7.x (#21771) (bb4c5d8)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.97.3 (#21457) (7bc9450).
ruler: Fix ruler panic related to unset validation scheme (#21401) (cf65729).
storage: Attach SHA-256 checksum on PutObject for Object Lock buckets (#21849) (7df13d9).
3.6.10 (2025-04-03)
ci Backporting #19989 Add build tags to Promtail build commands, into 3.6 (#21356) (0f56890).
3.6.9 (2025-04-01)
This release does not exist. We triggered the release pipeline, but it decided to play an April Fool's joke on us and did not generate the release.
3.6.8 (2025-03-25)
deps: Upgrade 
go.opentelemetry.io/otel/sdk from v1.38.0 to v1.40.0 (#21115) (d1ab148).
deps: Upgrade Go to 1.25.8 (release-3.6.x) (#21240) (10d2666).
deps: Upgrade Go used by querytee/promtail to 3.6 (#21244) (41a4e0c).
deps: Use different debian version for fluent-bit (#21247) (138c391).
deps: Update module github.com/buger/jsonparser to v1.1.2 (#21201) (3185466).
deps: Update module go.opentelemetry.io/otel/sdk to v1.40.0 (#20887) (d267ad3).
deps: Update module google.golang.org/grpc to v1.79.3 (#21193) (87dff41).
3.6.7 (2025-02-23)
alloc: Set a limit on preallocations (backport release-3.6.x) (#20920) (7e1daf3).
3.6.6 (2025-02-16)
deps: Update Alpine Docker tag to v3.23.3 (main) (#20630) (#20832) (fb20246).
3.6.5 (2025-02-05)
deps: Bump go version for 3.6.x (#20667) (b06b508)
logql: LineFilterLabelFilter.String() regexp correct delimiters (backport release-3.6.x) (#20649) (9d5fb5f)
3.6.4 (2025-01-21)
deps: Update dskit to resolve Otel conflict (#20368) (fb05a36).
distributor: Backport gzip fix release 3.6.x (#20514) (d805266).
3.6.3 (2025-12-11)
deps: Update to Go 1.25 (#20204).
storage: Do not override S3 region if already specified in configuration chain (backport release-3.6.x) (#20128) (38582ac).
storage: Fix regression in S3 client configuration (backport release-3.6.x) (#20118) (7d8176f).
3.6.2 (2025-11-25)
deps: Loki 3.6.1 CVE updates.
3.6.1 (2025-11-21)
docker: Missing permissions to start Docker (#19948) (48b507f).
docker: Set WORKDIR to root in Loki Dockerfiles (#19952) (6040a09).
3.6.0 (2025-11-17)
aggregated metrics: Hide 
__aggregated_metric__ in /series and /labels. This will alleviate confusion for people using a tool like logcli series --analyze-labels for cardinality analysis. (#14677) (4b0aaaa)
build: Update purgo package (#17807)
build: Remove busybox from dockerfile (#19502)
{{< admonition type="note" >}}
You cannot exec into the container with a shell anymore. For more information see the upgrade topic.
{{< /admonition >}}
build: Remove UI from docker build (#19425)
build: RPM signature config (#19476)
build: Update ckit (#18997)
chunkenc: Avoid memory allocations when loading symbolizer from a flushed chunk (#17953)
ci: Fix deprecation errors from staticcheck linter (#17210) (cfe3675)
ci: Fix lambda-promtail image build (#18443) (8f0ba2e)
ci: Fix workflow timeout added in wrong format (#18348) (53ba9c7)
ci: Read releaseLibRef from jsonnet.json lockfile (#18538) (26417b5)
compactor: Return request ID in response header (#19444)
compactor: Concurrent map access for processedSeries in delete requests manager (#17469) (32c5088)
compactor: Fix timeout and series progress marker for same requests with different shards (#17125) (288ec8c)
compactor: Move the check for duplicate series up the call stack to fix an issue when retention is enabled (#17663) (a5e95ba)
compactor: Wire up deletion series progress tracking (#17099) (db59d0e)
compactor: Do not try to merge the already consolidated delete requests while listing them (#18544) (8054076)
compactor: Ensure to return delete requests only for the requested tenant (#18589) (cdb190b)
compactor: Fix handling of duplicate delete requests (#18460) (255fce2)
compactor: Fix lint failure in retention.go (#17894)
compactor: Use the custom objectExists function to work around bug in Thanos object client (#18820)
compactor: Fix panic while applying storage updates when a whole chunk is deleted by line filters (#19262)
compactor: Set delete request batch to empty when we load to fail requests (#17810)
config_wrapper: Apply 
instance_enable_ipv6 from common to all components (#18254) (5bc5853)
deps: Update to Go 1.24.2 (#17804)
deps: Update go toolchain to 1.24.6 (#19449)
deps: Patch mapstructure dependency to remove CVE (#19447)
deps: Remove workspace files from ignore and regenerate vendor (#17234)
deps: Update dskit dependency (#18711) (19233d0)
deps: Update dskit to fix linux/arm build (#18715) (f39fc1e)
deps: Update dependency @hookform/resolvers to v5.2.2 (main) (#19276)
deps: Update dependency @radix-ui.react-checkbox to v1.3.3 (main) (#18836)
deps: Update dependency @radix-ui/react-collapsible to v1.1.8 (main) (#17382) (0a4aec6)
deps: Update dependency @radix-ui.react-dialog to v1.1.15 (main) (#18841)
deps: Update dependency @radix-ui.react-dropdown-menu to v2.1.16 (main) (#18845)
deps: Update dependency @radix-ui.react-hover-card to v1.1.15 (main)(#18849)
deps: Update dependency @radix-ui/react-label to v2.1.7 (main) (#17877)
deps: Update dependency @radix-ui.react-popover to v1.1.15 (main) (#18850)
deps: Update dependency @radix-ui/react-progress to v1.1.7 (main) (#17879)
deps: Update dependency @radix-ui.react-scroll-area to v1.2.10 (main) (#18852)
deps: Update dependency @radix-ui.react-select to v2.2.6 (main) (#18853)
deps: Update dependency @radix-ui/react-separator to v1.1.7 (main) (#17882)
deps: Update dependency @radix-ui/react-slot to v1.2.3 (main) (#17883)
deps: Update dependency @radix-ui.react-switch to v1.2.6 (main)(#18854)
deps: Update dependency @radix-ui/react-tabs to v1.1.9 (main) (#17389) (23a2f2d)
deps: Update dependency @radix-ui/react-toast to v1.2.7 (main) (#17148) (1a1cbb3
deps: Update dependency @radix-ui.react-toggle to v1.1.10 (main)(#18859)
deps: Update dependency @radix-ui.react-toggle-group to v1.1.11 (main) (#18860)
deps: Update dependency @radix-ui.react-tooltip to v1.2.8 (main) (#18862)
deps: Update dependency @tanstack.react-query to v5.85.5 (main) (#18962)
deps: Update dependency @tanstack/react-query-devtools to v5.85.5 (main) (#18963)
deps: Update dependency @types/lodash to v4.17.20 (main) (#18305) (90ca918)
deps: Update module cloud.google.com/go/bigtable to v1.40.1 (main) (#19337)
deps: Update module cloud.google.com/go/pubsub to v1.49.0 (main) (#17187) (cbd73d0)
deps: Update module cloud.google.com/go/storage to v1.57.0 (main) (#19312)
deps: Update module dario.cat/mergo to v1.0.2 (main) (#17651) (bb8a9d9)
deps: Update module github.com.prometheus/otlptranslator to v0.0.2 (main) (#18782)
deps: Update module github.com/alecthomas/chroma/v2 to v2.20.0 (main) (#18710) (c8e87dd)
deps: Update module github.com/alicebob/miniredis/v2 to v2.35.0 (main) (#18029)
deps: Update module github.com/apache/arrow-go/v18 to v18.4.0 (main) (#18519) (600fc9e)
deps: Update module github.com/aws/aws-lambda-go to v1.49.0 (main) (#18036)
deps: Update module github.com/aws/aws-sdk-go to v1.55.7 (main) (#17402) (06a803d)
deps: Update module github.com/aws/aws-sdk-go-v2 to v1.36.6 (main) (#18506) (1d51304)
deps: Update module github.com/aws/aws-sdk-go-v2/config to v1.29.18 (main) (#18507) (4ea7409)
deps: Update module github.com/aws/aws-sdk-go-v2/credentials to v1.18.16 (main) (#19352)
deps: Update module github.com/aws/aws-sdk-go-v2/service/dynamodb to v1.51.0 (main) (#19388)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.84.1 (main) (#18508) (4fdae62)
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.245 (main) (#19278)
deps: Update module github.com/bmatcuk/doublestar/v4 to v4.9.1 (main) (#18599) (17b377f)
deps: Update module github.com/charmbracelet/bubbles to v0.21.0 (main) (#17191) (3e9c07e)
deps: Update module github.com/charmbracelet/bubbletea to v1.3.10 (main) (#19279)
deps: Update module github.com/coder/quartz to v0.2.1 (main) (#17922) (3cb216c)
deps: Update module github.com/docker/docker to v28.5.0+incompatible (main) (#19414)
deps: Update module github.com/fsnotify/fsnotify to v1.9.0 (main) (#17192) (eb4b651)
deps: Update module github.com/fsouza/fake-gcs-server to v1.52.3 (main) (#18999)
deps: Update module github.com/go-logfmt/logfmt to v0.6.1 (main) (#19412)
deps: Update module github.com/grafana/loki/v3 to v3.5.5 (main) (#18444)
deps: Update module github.com/grpc-ecosystem/go-grpc-middleware/v2 to v2.3.2 (main) (#17652) (988857e)
deps: Update module github.com/hashicorp/consul/api to v1.32.1 (main) (#17653) (44b9b64)
deps: Update module github.com/heroku/x to v0.5.2 (main) (#18809)
deps: Update module github.com/ibm/go-sdk-core/v5 to v5.21.0 (main) (#18587) (86a31ec)
deps: Update module github.com/ibm/ibm-cos-sdk-go to v1.12.3 (main) (#19281)
deps: Update module github.com/ibm/sarama to v1.46.1 (main) (#19282)
deps: Update module github.com/minio/minio-go/v7 to v7.0.95 (main) (#18514) (70c176c)
deps: Update module github.com/ncw/swift/v2 to v2.0.4 (main) (#18011)
deps: Update module github.com/oklog/run to v1.2.0 (main) (#18263) (3011a6c)
deps: Update module github.com/opentracing-contrib/go-grpc to v0.1.2 (main) (#17164) (23b47ce)
deps: Update module github.com/oschwald/geoip2-golang to v1.13.0 (main) (#18354)
deps: Update module github.com/parquet-go/parquet-go to v0.25.1 (main) (#18013)
deps: Update module github.com/pressly/goose/v3 to v3.26.0 (main) (#19417)
deps: Update module github.com/prometheus/client_golang to v1.23.0 (main) (#18674) (07a627a)
deps: Update module github.com/prometheus/client_model to v0.6.2 (main) (#17171) (fa84382)
deps: Update module github.com/prometheus/common to v0.65.0 (main) (#18196) (3ad1077)
deps: Update module github.com/prometheus/prometheus to v0.304.2 (main) (#18248) (b2e1d7f)
deps: Update module github.com/prometheus/sigv4 to v0.2.1 (main) (#18454) (9ea365a)
deps: Update module github.com/redis/go-redis/v9 to v9.12.1 (main) (#18218)
deps: Update module github.com/shirou/gopsutil/v4 to v4.25.9 (main) (#19366)
deps: Update module github.com/sony/gobreaker/v2 to v2.3.0 (main) (#19418)
deps: Update module github.com/spf13/afero to v1.15.0 (main) (#19317)
deps: Update module github.com/stretchr/testify to v1.11.1 (main) (#19042)
deps: Update module github.com/twmb/franz-go to v1.19.5 (main) (#18039)
deps: Update module github.com/twmb/franz-go/pkg/kadm to v1.16.1 (main) (#18733)
deps: Update module github.com/twmb/franz-go/pkg/kmsg to v1.11.2 (main) (#17258) (7c47a3e)
deps: Update module github.com/twmb/franz-go/plugin/kotel to v1.6.0 (main) (#17194) (9491fbb)
deps: Update module github.com/twmb/franz-go/plugin/kprom to v1.3.0 (main) (#18596) (5405a83)
deps: Update module github.com/workiva/go-datastructures to v1.1.6 (main) (#19287)
deps: Update module go.etcd.io/bbolt to v1.4.3 (main) (#18973)
deps: Update module go.opentelemetry.io/collector/pdata to v1.43.0 (main) (#19419)
deps: Update module go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc to v0.63.0 (main) (#18455)
deps: Update module go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace to v0.63.0 (main) (#19324)
deps: Update module go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp to v0.63.0 (main) (#19327)
deps: Update module golang.org/x.crypto to v0.41.0 (main) (#18767)
deps: Update module golang.org/x.net to v0.43.0 (main) (#18769)
deps: Update module golang.org/x/crypto to v0.40.0 (main) (#18406) (8fa91c6)
deps: Update module golang.org/x/net to v0.45.0 (main) (#19420)
deps: Update module golang.org/x/oauth2 to v0.31.0 (main) (#19329)
deps: Update module golang.org/x/sync to v0.16.0 (main) (#18383) (6b1204a)
deps: Update module golang.org/x/sys to v0.35.0 (main) (#18759) (9c748e0)
deps: Update module golang.org/x/text to v0.28.0 (main) (#18762) (e6fb85c)
deps: Update module golang.org/x/time to v0.13.0 (main) (#19330)
deps: Update module google.golang.org/api to v0.252.0 (main) (#19421)
deps: Update module google.golang.org/grpc to v1.75.1 (main) (#19288)
deps: Update module google.golang.org/protobuf to v1.36.9 (main) (#19289)
deps: Update module k8s.io/apimachinery to v0.33.4 (main) (#18868)
deps: Update dependency lucide-react to ^0.544.0 (main) (#19310)
deps: Update dependency react to v19.1.0 (main) (#16949) (404fd13)
deps: Update dependency react-datepicker to v8.7.0 (main) (#18984)
deps: Update dependency react-dom to v19.1.0 (main) (#16950) (8d1dc9c)
deps: Update dependency react-hook-form to v7.62.0 (main) (#18704) (4bc9162)
deps: Update dependency react-router-dom to v7.8.2 (main) (#18971)
deps: Update dependency recharts to v2.15.4 (main) (#18187) (7593bf5)
deps: Update dependency tailwind-merge to v3.3.1 (main) (#18061)
deps: Update dependency zod to v3.25.76 (main) (#18359) (eb1e039)
deps: Update dependency zod to v4 (main) (#18392)
deps: Update module zombiezen.com/go/sqlite to v1.4.2 (main) (#17903)
distributor: Disable metadata topic writes (#17437) (46b2271)
distributor: Revert remove colons from level detection (#16999) (e678b61)
distributor: Short circuit level detection when already detected (#17028) (58aa00a)
distributor: Skip streams over limits in dry-run mode (#17114) (2cde9b1)
distributor: Prevent panic when ContentLength is negative in Distributor RequestBodyTooLarge metrics (#17054) (9e9f534)
distributor: Detect "Information" log level as "info" (#18195) (82263d3)
distributor: Correctly register allowed log level fields (#19261)
distributor: Extended detected level for debug and critical level (#18370)
docs: Update configuration.md (#17269) (70de523)
docs: Added instructions for how to upgrade zone-aware ingesters (#18658) (aff2cb0)
engine: Don't attempt to release batch reference on read for merge node (#18913)
engine: Fix bug in 
SortMerge which caused rows to be skipped due to incorrect sorting (#18334) (4fbe7c1)
engine: Fix out-of-bounds panic in SortMerge implementation (#17967)
frontend: Allow resolution of v6 addresses. (#18251) (8481b34)
helm: Add flush=true to preStop hook (#16063) (a375751)
helm: Add init container configuration for backend, bloom builder, distributor, query-frontend, query-scheduler, read, write. (#18709)
helm: Add livenessProbe to read pods in Simple Scalable mode (#17063)
helm: Add missing log deletion endpoint to native ingress paths (#14390) (6db8b1f)
helm: Add release namespace metadata to HorizontalPodAutoscaling that lack it (#18453) (faae40a)
helm: Add validation for loki.storage.bucketNames (#13781) (831c814)
helm: Admin-api should only be deployed in microservice mode (#13655)
helm: Allow loki to use hostPath volumes (#17680) (8e61a5b)
helm: Allow metrics networkpolicy only from namespace (#17555) (3a9b8e8)
helm: Avoid double zonal ingester scrapping by not scrapping headless svc (#19000)
helm: Create namespaced RBAC when sidecar is enabled (#16776) (7c9003e)
helm: Do not strip whitespace in target definitions for querier and read deployments (#19464)
helm: Ensure global.extraEnv and global.extraEnvFrom applied to all resources consistently (#16828) (ee479d0)
helm: Explicitly set registry for k8s-sidecar image (#19233)
helm: Fix global extra* values (#17020)
helm: Fix incorrect context references in Helm storage templates (#18740) (1f362c8)
helm: Fix PDB settings for chunksCache and resultsCache (#18321) (e03b9b9)
helm: Fix setting X-Scope-OrgID header (#18414) (2f461c9)
helm: Fix storage_config when use_thanos_objstore is set to true (#17024) (81866d7)
helm: Fixed statement logic to enable annotations (#17756)
helm: Gateway Ingester endpoints points to inexistent service when zone aware replication is enabled (#17362) (395a711)
helm: Make loki.storage.bucketNames are optional, if builtin minio is enabled. (#18653) (409ccbd)
helm: Missing S3 field in lokiStorageConfig templated value (#18791)
helm: Move loki-sc-rules to second position in containers (#17937) (6e72df3)
helm: Only validate the ruler key exists under loki.storage.bucketNames when an object storage bucket is being used (#18665) (0604477)
helm: Remove flaky enterprise test (#19362)
helm: Removing deprecated admin_api_directory to avoid config crash (#18731) (8cce03c)
helm: Roll back change to Thanos ruler storage (#18907)
helm: Un-deprecate several features in 
monitoring block (#19012)
helm: Update querier, read, and single binary target definitions for UI enablement (#19461)
helm: Use correct serviceName in zone-aware ingester statefulsets (#18558) (2706302)
helm: In helm chart fix indentation in nginx gateway config template handling (#18167) (4fe20a7)
helm: Loki chart fails to render proper YAML when add more than one extra manifest (#12911) (cf12656)
helm: Make helm use grpc for compactor address (#17454) (90003f6)
helm: Use gateway container port as nginx server port (#18774)
helm: Use strings in stead of integers for ports in CiliumNetworkPolicies (#19252)
helm: Websocket related proxy_set_header to locations back to resolve high CPU usage. (#18800)
helm: Use UDP/53 for DNS egress instead of named port (#19073)
helm: Add single-binary component to ingress NetworkPolicy (#19229)
helm: Canary only worked when gateway enabled (#16758)
helm: Enable ui in helm deployments (#17562)
helm: Update loki-helm-test image tag to latest commit (#19227)
ingester: Fix missing series in monolithic mode (#17067) (5625464)
json parser: Fix possible JSON log line corruption caused by 
json parser on query path (#18056)
json, structured metadata: Correctly handle escaping in json parsing API (#17068)
json, structured metadata: JSON-parsing cleanup (#17072) (2aed4c3)
kafka: Remove duplicated metric from Kafka producer (#18614) (a67a460)
kafka: Disable defaults address defaults (#17825)
kafka: Use cooperative active sticky load balancer (#19160)
limits: Adapt defaults and expose evict interval (#17808)
limits: Fix ingest-limits eviction metric (#17779)
limits: Read the consumer group and topic from the ingest-limits config (#17831)
limits: Set ready when all partitions ready (#18092) (a07bee5)
limits: Counter variables should not have total at the end ((#17982)
logql: Fix inconsistency with parsed field short circuiting (#17104) (88beefb)
logql: While validating logql expression, detect and validate expression in label_replace (#18470) (d379de5)
logql: Implement approx-topk function on querier (#17816)
logql: Reduce allocations for JSON and logfmt parser (#18637)
loki: Return defaults for non-existent tenants on applied limits endpoint (#17942)
memberlist: Allow resolution of advertise address from v6 interface (#18250) (8594d1c)
mixin: Fix typo in Loki Reads dashboard TSDB row (#18872)
mixins: Fix label generation for Loki logs dashboard (#17412) (da2b620)
objstore: Ruler not starting and incorrect mkdir path when Thanos Store is enabled (#16555) (3da9f2c)
oltp: Apply global otlp config to tenant config only when it is updated in the overrides (#19213)
operator: Fix type of maximum OpenShift version property (#18066) (f7c7dfa)
operator: Fix typo in docs regarding forcepathstyle (#17725)
operator: Update maximum OpenShift version (#17954)
operator: Update memberlist when ingester becomes unhealthy (#17026) (c63f0a9)
operator: Update webhook validator for alerts/rules (#17824)
operator: Updated AlertingRule sample to make it comply with the validation we apply (#18671)
operator: Upgrade OPA policy syntax for v1+ (#18795)
otlp: Calculate entry metadata size before adding resource/scope attributes (#17629) (2161c8c)
patterns: Pattern persistence feature flag (#18285) (a206324)
patterns: Limit volume and frequency of persisted patterns (#18362) (c690827)
patterns: Fix feature flag for enabling pattern persistence (#18216) (c167800)
push: Add guard clauses to prevent negative counter values (#17056) (9000de1)
push: Fix guard clauses to prevent error spam in logs (#17372) (d70b20e)
push: Fix push stats calculation (#19319)
querier: Fixes panic when filtering agg metrics from nil resp (#17662)
querier: Allow boolean numeric values in detected labels (#16997) (bfb935d)
querier: Fix integer overrun when calculating parallelism for very long time range queries (#17428) (82acbd5)
querier: Better fix integer overrun when calculating parallelism for very long time range queries (#17430) (d660af3)
querier: Revert "perf: Fix memory leak in cachedIterator (#17628)" (#18687) (0316740)
ruler: Correct log level verbosity in rule evaluation (#19519)
ruler: Return StatusBadRequest on multiple org IDs (#17850)
storage: Fix nil pointer dereference in Volume method of the composite store (#18064) (fc7c018)
storage: Use default config when building S3 client (backport k277) (#19559)
stream-generator: Split create/keep-alive streams routines (#17815)
structured metadata: Improve structured metadata label normalization performance (#17332) (87922da)
structured metadata: Unescape JSON structured metadata string values (#13919) (9629d07)
structured metadata: Ensure we do not Add duplicate structured metadata from stream labels and extracted map (#18523) (db72d63)
tests: Update kfake to fix tests and revert go-redis (#18958)
tools: Correct typo in appenPath function name (#17917)
tsdb: Avoid copying label values from tsdb unless required (#17077) (8e4b104)
ui: Fix UI build (#17717) (e01a2a9)
ui: Downgrade UI's @swc/core dep back to v1.13.5 (#19345)
WAL: Handle WAL corruption properly on startup (#18175) (1954f67)

v3.7
Grafana Labs and the Loki team are excited to announce the release of Loki 3.7. Here's a summary of new enhancements and important fixes.
For a full list of all changes and fixes, refer to the CHANGELOG
{{< admonition type="note" >}}
See the deprecations section for important information about the Loki Helm charts.
{{< /admonition >}}
Features and enhancements
Key features in Loki 3.7.0 include the following:
Helm charts: - Effective March 16, 2026, the Grafana Loki Helm chart has been be forked to the new repository grafana-community/helm-charts. The motivation behind this change is to accelerate development and enable direct community maintenance, as outlined in grafana/helm-charts#3955. The Grafana Community now owns maintenance of the Loki Helm Chart, while Grafana Labs continues to maintain the Helm Chart for Grafana Enterprise Logs (GEL). The Loki Helm Charts documentation has been updated to point to the new Grafana-Community repo.
Docs: The documentation now includes a new, comprehensive Troubleshooting section, documenting the Loki and Logs Drilldown error messages.
{{< admonition type="note" >}}
Note that when appropriate, documentation updates have been backported to previous releases.
{{< /admonition >}}
Other improvements include the following:
canary: Support passing arbitrary set of labels to use for the query (#17008) (993b3ae)
compactor: Add support for storing chunk deletion markers in object storage instead of local disk (#19689) (856c11d)
distributor: Add gauge to track in-flight bytes (#20091) (23ef8ec)
distributor: Add resolved policy to blocked and enforced label error (#19826) (48d13d1)
distributor: Add segmentation keys and resolver (#19927) (c853f2c)
distributor: Add support for UpdateRates RPC to distributors (#19918) (9018886)
distributor: Randomly distribute requests to the ingest-limits frontend (#19840) (1605a38)
distributor: Shuffle shard on tenant rate limit (#19990) (3904c2b)
distributor: Write to dataobj partitions based on segmentation key (#19946) (3a24f5d)
helm: 
nameOverride now passed through helm tpl function. (#19590) (7f56fd2)
helm: Add ability to toggle grpclb port for query frontend service (#19609) (9c4f022)
helm: Add startupProbe to distributor (#20073) (5b76589)
helm: Allow configuration of service trafficDistribution parameter (#19558) (55f95e3)
helm: Allow set topologySpreadConstraints on singleBinary (#19534) (265601f)
helm: Make loki-canary readinessProbe configurable via values.yaml (#19328) (7231766)
helm: Use fsGroupChangePolicy=OnRootMismatch to speed up pod starts (#13942) (c7cec3a)
index-gateway: Client side index gateway shuffle sharding (#20124) (326c7d1)
ingest-limits: Add UpdateRates RPC, update rates from the frontend, return no-op in the service (#19894) (e173cf4)
ingest-limits: Check partition state in parallel (#19884) (b8536aa)
ingester: Handle state change lock in prepare downscale (#20141) (de092da)
logcli: Allow custom headers to be passed (#20231) (c524203)
loki: Add loki health command (#20313) (ef69cfd)
lokitool: Add regex namespace filtering (#20209) (0c1561d)
operator: Add option to disable ingress (#19382) (9dc71a6)
querier: Ability to send query context for limit enforcement (#19900) (1a66d2d)
Key features in Loki 3.7.3 include the following:
logql: Add case-insensitive regex optimization (#20578) (f6a6a4f)
loki: Allow disabling cache for 
query_range requests (#21112) (1242aa5)
operator: Add option to customize gateway server cert (#20325) (3c8b9e8)
operator: Update metrics authentication to remove dependency on kube-rbac-proxy (#20853) (2de6e79)
Key features in Loki 3.7.7 include the following:
compactor: Add flag to ignore missing chunks during deletion of logs with line filter (#24233) (7a40404).
Deprecations
One of the focuses of Loki 3.0 was cleaning up unused code and old features that had been previously deprecated but not removed. Loki 3.0 removed a number of previous deprecations and introduces some new deprecations. Some of the main areas with changes include:
Promtail was deprecated in Loki 3.0 and has been removed as of Loki 3.7.3 (#21248). The code has been merged into Grafana Alloy. You can find migration documentation and a utility to convert your Promtail configuration to Alloy configuration in the Alloy documentation. Note that this removal does NOT include the lambda-promtail client.
Deprecated storage options including the deprecation of the BoltDB store.
Deprecated configuration options.
API endpoint deprecations.
To learn more about breaking changes in this release, refer to the Upgrade guide.
Upgrade Considerations
For important upgrade guidance, refer to the Upgrade Guide.
BREAKING CHANGE - engine: Make scheduler aware of total compute capacity (#19876)
BREAKING CHANGE - engine: Share worker threads across all scheduler connections (#20229)
BREAKING CHANGE - labels: parsed labels should not override structured metadata (#19991)
BREAKING CHANGE - operator: Switch default for OpenShift stream labels (#21001)
Bug fixes
3.7.9 (2026-10-20)
deps: Update github.com/thanos-io/objstore digest to 72d3b9f (release-3.7.x) (#25389) (e7d6809).
deps: Update go4.org/netipx digest to 3075585 (release-3.7.x) (#25390) (24be2f4).
deps: Update k8s.io/utils digest to cf1189d (release-3.7.x) (#25391) (a5218df).
ingester: Concurrent WAL 
Append() calls could lead to inconsistent record order and irrecoverable WAL state (#25125) (72aa93e).
logql: Keep entries while stream accumulator has free room (#24703) (4390fb1).
ruler: Limit request body size for ruler "CreateRuleGroup" API and LogQLAnalyzer API (#24838) (6c6e9dc).
security/LOW/: Update module go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc to v1.45.0 (#24614) (bab6de2).
security/LOW/: Update module go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp to v1.45.0 (#24615) (7aee5f1).
security/LOW/: Update module go.opentelemetry.io/otel/sdk to v1.45.0 (#24616) (b787d9a).
security/MEDIUM/: Update module github.com/containerd/containerd/v2 to v2.2.9 (#24806) (bafb1b8).
security: Bump Go to 1.26.10 (#25458) (d3b4a2f).
3.7.8 (2026-08-27)
security/HIGH/pkg/push: Update module google.golang.org/grpc to v1.83.2 (#24433) (4d5dc2b).
security/HIGH/: Update module google.golang.org/grpc to v1.83.1 (#24328) (0a01797).
security/HIGH/: Update module google.golang.org/grpc to v1.83.2 (#24429) (14b5da8).
security/MEDIUM/: Update module github.com/containerd/containerd/v2 to v2.2.8 (#24460) (3e917f2).
security/UNKNOWN/pkg/push: Update module golang.org/x/net to v0.56.0 (#23422) (41064da).
security/UNKNOWN/pkg/push: Update module google.golang.org/grpc to v1.83.1 (#23416) (cd8e76d).
security/UNKNOWN/: Update module golang.org/x/crypto to v0.56.0 (#24335) (47149cb).
3.7.7 (2026-08-27)
security/UNKNOWN/: Update module github.com/containerd/containerd/v2 to v2.2.5 (#24097) (07cd1f0).
security/UNKNOWN/: Update module go.etcd.io/etcd/client/pkg/v3 to v3.6.14 (#24061) (9d1c9ee).
security/UNKNOWN/: Update module golang.org/x/mod to v0.40.0 (#23962) (1d5e027).
storage: Pre-compute SHA-256 to avoid aws-chunked on PutObject (#24215) (3d6f245).
3.7.6 (2026-08-05)
queryrange: Preserve sketch in MergeLabels (#23770) (d48329d)
3.7.5 (2026-08-05)
ci: Build deb/rpm packages again in the release workflow (#23598) (6f49272)
ci: Disable docker plugin publishing on release-3.7.x (#23440) (6095f71)
ingester: Fix flush race in ingester (#23682) (ff152dc)
security/UNKNOWN/: Update module github.com/klauspost/compress to v1.18.7 (release-3.7.x) (#23610) (941b557)
security/UNKNOWN/: Update module golang.org/x/text to v0.39.0 (release-3.7.x) (#23418) (2347a9a)
security/UNKNOWN/: Update module google.golang.org/grpc to v1.82.1 (release-3.7.x) (#23405) (f290af0)
security/UNKNOWN/cmd/chunks-inspect: Update module github.com/klauspost/compress to v1.18.7 (release-3.7.x) (#23611) (11c4b29)
security/UNKNOWN/cmd/dataobj-inspect: Update module github.com/klauspost/compress to v1.18.7 (release-3.7.x) (#23612) (1e3108e)
security/UNKNOWN/cmd/dataobj-inspect: Update module go.opentelemetry.io/otel to v1.42.0 (release-3.7.x) (#23518) (28917d3)
security/UNKNOWN/cmd/dataobj-inspect: Update module golang.org/x/text to v0.39.0 (release-3.7.x) (#23419) (5d2b9d4)
security/UNKNOWN/operator: Update module github.com/klauspost/compress to v1.18.7 (release-3.7.x) (#23613) (8d71c59)
3.7.4 (2026-07-22)
ci: Fix zizmor findings for operator-images in Loki 3.7 (#22820) (abc1186)
ci: Helm CI warning fix (#22605) (7389428)
ci: Re-enable docker plugin publishing (#22818) (28f44a8)
compactor: Fix delete request when using Thanos objstore client with filesystem backend (#22811) (dd6fdb5)
deps: Update module charm.land/bubbletea/v2 to v2.0.7 (release-3.7.x) (#22768) (e11f935)
deps: Update module charm.land/lipgloss/v2 to v2.0.4 (release-3.7.x) (#22769) (10213c2)
deps: Update module github.com/aws/aws-sdk-go-v2/credentials to v1.19.24 (release-3.7.x) (#22776) (3daf198)
deps: Update module github.com/aws/smithy-go to v1.27.3 (release-3.7.x) (#22795) (1f6a191)
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.270 (release-3.7.x) (#22783) (103ccd8)
deps: Update module github.com/coder/quartz to v0.3.1 (release-3.7.x) (#22784) (3509e31)
deps: Update module github.com/IBM/ibm-cos-sdk-go to v1.14.1 (release-3.7.x) (#22785) (d0abe72)
deps: Update module github.com/klauspost/compress to v1.18.6 (release-3.7.x) (#22797) (327da7c)
deps: Update module github.com/pierrec/lz4/v4 to v4.1.27 (release-3.7.x) (#22798) (6b3297f)
deps: Update module github.com/shirou/gopsutil/v4 to v4.26.5 (release-3.7.x) (#22800) (cb7f17d)
security/CRITICAL/clients/cmd/fluentd/docker: Update dependency fluentd to v1.19.3 (release-3.7.x) (#22693) (6d8f1b5)
security/HIGH/: Bump Go to 1.26.5 to address CVE-2026-39822 and CVE-2026-42505 (#23393) (b318f28)
security/HIGH/: Update module github.com/apache/thrift to v0.24.0 (release-3.7.x) (#22767) (36eb5f7)
security/UNKNOWN/: Update module github.com/containerd/containerd/v2 to v2.0.10 (release-3.7.x) (#22478) (6fa6165)
security/UNKNOWN/pkg/push: Update module golang.org/x/net to v0.55.0 (release-3.7.x) (#22200) (dd7a124)
storage: Fix index filename issue with legacy S3 client and 
chunk_delimiter (#23165) (0eca83c)
3.7.3 (2026-06-24)
deps: Move to Go 1.26 (#20988) (4fb960c).
deps: Update go toolchain directive to v1.25.11 (#22192) (d5d7a54).
deps: Update golang.org/x/crypto to v0.52.0 (#22189) (6383bfe).
deps: Update golang.org/x/net to v0.55.0 (#22190) (3d93fb8).
deps: Update golang.org/x/sys to v0.44.0 (#22191) (3379538).
deps: Update module github.com/containerd/containerd/v2 to v2.0.9 (#22187) (365c7e9).
deps: Update module go.opentelemetry.io/otel to v1.41.0 (#22188) (dc8f4de).
distributor: Sanitize User-Agent to valid UTF-8 before using as Prometheus label value (#21162) (ed3cf08).
engine: Align range-query evaluation timestamps to the step grid (#20904) (28d03ec).
engine: Fix additional instances of writing/reading on a closed conn (#20982) (1f7cacf).
ingester: Duplicate lines with different structured metadata (#21039) (60a1567).
ingester: WAL replay hang when flush doesn't progress (#21176) (fb58ab8).
logql: Return error on way-too-long label string (#20937) (9c9660a).
operator: Fix incorrect image reference in OLM bundle (#20966) (84bc9bc).
operator: Properly handle the deprecated replicationFactor (#20935) (1b96366).
operator: Provide AWS region in environment variable for STS mode (#21136) (cb261d9).
operator: Set ingester minimum available pods based on replication factor (#19517) (efc63e8).
querier: Prevent goroutine leak in downstreamer on context cancellation (#21062) (2891919).
query-engine: Fix OR operations being silently dropped (#21204) (20ea556).
security: Mask all credentials (#20766) (f8d94cb).
storage: Broken get and replace for S3 client (#21163) (e6abd15).
3.7.2 (2026-05-13)
deps: CVEs in release 3.7.x (#21771) (bb4c5d8).
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.97.3 (#21457) (7bc9450).
ruler: Fix ruler panic related to unset validation scheme (backport release-3.7.x) (#21401) (cf65729).
storage: Attach SHA-256 checksum on PutObject for Object Lock buckets (#21849) (7df13d9).
3.7.1 (2026-03-27)
deps: Upgrade Go and gRPC versions on 3.7.x (#21282) (2c8fff2).
3.7.0 (2026-03-26)
cd: Add loki-image to needs (#19870) (a2c4ea6)
compactor: Compactor file descriptor leak (#20077) (0c3dd8c)
config: Migrate renovate config (#19436) (97745fe)
dataobj: Flush into multiple index objects when ErrBuilderFull (#19223) (32dbef9)
deps: Update dataobj-inspect transitive deps version (#19813) (5b212b7)
deps: Update module cloud.google.com/go/bigtable to v1.41.0 (main) (#20352) (6102309)
deps: Update module cloud.google.com/go/pubsub to v2 (main) (#19803) (d47dde3)
deps: Update module cloud.google.com/go/storage to v1.59.0 (main) (#20407) (5c71db6)
deps: Update module github.com/alecthomas/chroma/v2 to v2.22.0 (main) (#20409) (c64f044)
deps: Update module github.com/apache/arrow-go/v18 to v18.5.0 (main) (#20354) (d0861a1)
deps: Update module github.com/aws/aws-sdk-go-v2 to v1.40.1 (main) (#20137) (e106809)
deps: Update module github.com/aws/aws-sdk-go-v2/config to v1.32.7 (main) (#20401) (50ce71a)
deps: Update module github.com/aws/aws-sdk-go-v2/credentials to v1.19.7 (main) (#20402) (f20228d)
deps: Update module github.com/aws/aws-sdk-go-v2/service/dynamodb to v1.53.6 (main) (#20403) (31a870c)
deps: Update module github.com/aws/aws-sdk-go-v2/service/s3 to v1.95.1 (main) (#20406) (c7c1411)
deps: Update module github.com/aws/smithy-go to v1.24.0 (main) (#20117) (b0efa70)
deps: Update module github.com/axiomhq/hyperloglog to v0.2.6 (main) (#20341) (4469f82)
deps: Update module github.com/baidubce/bce-sdk-go to v0.9.256 (main) (#20342) (69e6254)
deps: Update module github.com/bits-and-blooms/bloom/v3 to v3.7.1 (main) (#19755) (af47e1f)
deps: Update module github.com/bmatcuk/doublestar/v4 to v4.9.2 (main) (#20418) (078dc94)
deps: Update module github.com/coder/quartz to v0.3.0 (main) (#19786) (6f784f9)
deps: Update module github.com/docker/docker to v28.5.2+incompatible (main) (#19756) (1007ee4)
deps: Update module github.com/gocql/gocql to v2 (main) (#19794) (898b6d2)
deps: Update module github.com/google/renameio/v2 to v2.0.2 (main) (#20411) (cc43074)
deps: Update module github.com/grafana/loki/v3 to v3.6.3 (main) (#20343) (d1ae7a1)
deps: Update module github.com/grpc-ecosystem/go-grpc-middleware/v2 to v2.3.3 (main) (#19758) (8133da9)
deps: Update module github.com/hashicorp/consul/api to v1.33.0 (main) (#19788) (e417259)
deps: Update module github.com/ibm/go-sdk-core/v5 to v5.21.2 (main) (#19988) (d8ab970)
deps: Update module github.com/ibm/ibm-cos-sdk-go to v1.13.0 (main) (#20364) (52d1d8d)
deps: Update module github.com/ibm/sarama to v1.46.3 (main) (#19760) (4a19787)
deps: Update module github.com/influxdata/telegraf to v1.37.0 (main) (#20356) (dc1e0ae)
deps: Update module github.com/klauspost/compress to v1.18.2 (main) (#20108) (f4f2b2a)
deps: Update module github.com/leodido/go-syslog/v4 to v4.3.0 (main) (#19416) (036387b)
deps: Update module github.com/minio/minio-go/v7 to v7.0.98 (main) (#20436) (cf89342)
deps: Update module github.com/ncw/swift/v2 to v2.0.5 (main) (#19764) (fa5e144)
deps: Update module github.com/oschwald/geoip2-golang to v2 (main) (#19799) (33eeab6)
deps: Update module github.com/oschwald/geoip2-golang/v2 to v2.1.0 (main) (#20357) (8853d71)
deps: Update module github.com/parquet-go/parquet-go to v0.27.0 (main) (#20426) (a283eac)
deps: Update module github.com/prometheus/alertmanager to v0.30.0 (main) (#20358) (f53a609)
deps: Update module github.com/prometheus/client_golang to v1.23.2 (main) (#19763) (8317f7e)
deps: Update module github.com/prometheus/common to v0.67.5 (main) (#20363) (aaacbf4)
deps: Update module github.com/prometheus/prometheus to v0.309.1 (main) (#20388) (bf79bcf)
deps: Update module github.com/prometheus/sigv4 to v0.4.0 (main) (#20386) (2f80526)
deps: Update module github.com/redis/go-redis/v9 to v9.17.2 (main) (#20116) (434a929)
deps: Update module github.com/schollz/progressbar/v3 to v3.19.0 (main) (#20365) (0b238bc)
deps: Update module github.com/shirou/gopsutil/v4 to v4.25.12 (main) (#20347) (0740eb8)
deps: Update module github.com/sirupsen/logrus to v1.9.4 (main) (#20447) (35c8df7)
deps: Update module github.com/sony/gobreaker/v2 to v2.4.0 (main) (#20366) (090ffd5)
deps: Update module github.com/tjhop/slog-gokit to v0.1.5 (main) (#19808) (615413e)
deps: Update module github.com/twmb/franz-go to v1.20.6 (main) (#20348) (5ee4fee)
deps: Update module github.com/twmb/franz-go/pkg/kadm to v1.17.1 (main) (#19790) (1dad0be)
deps: Update module github.com/twmb/franz-go/pkg/kmsg to v1.12.0 (main) (#19791) (f28c247)
deps: Update module github.com/workiva/go-datastructures to v1.1.7 (main) (#19766) (f5e0683)
deps: Update module github.com/xdg-go/scram to v1.2.0 (main) (#20046) (9e52320)
deps: Update module go.opentelemetry.io/collector/pdata to v1.49.0 (main) (#20371) (7d759f2)
deps: Update module go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc to v0.64.0 (main) (#20372) (9da1b1b)
deps: Update module go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace to v0.64.0 (main) (#20373) (719635a)
deps: Update module go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp to v0.64.0 (main) (#20374) (f1b3e1b)
deps: Update module go.opentelemetry.io/otel/sdk to v1.39.0 (main) (#20376) (95b82d6)
deps: Update module golang.org/x/crypto to v0.44.0 (main) (#19776) (c85c67a)
deps: Update module golang.org/x/net to v0.47.0 (main) (#19850) (5c422a6)
deps: Update module golang.org/x/oauth2 to v0.34.0 (main) (#20160) (ebf7b93)
deps: Update module golang.org/x/sync to v0.19.0 (main) (#20161) (817b9d8)
deps: Update module golang.org/x/sys to v0.40.0 (main) (#20378) (9538b20)
deps: Update module golang.org/x/text to v0.33.0 (main) (#20408) (9fd6733)
deps: Update module golang.org/x/time to v0.14.0 (main) (#19782) (74f68fa)
deps: Update module google.golang.org/api to v0.257.0 (main) (#20143) (5f1da75)
deps: Update module google.golang.org/grpc to v1.77.0 (main) (#19945) (f3213bb)
deps: Update module google.golang.org/protobuf to v1.36.11 (main) (#20349) (a80b52e)
deps: Update module k8s.io/apimachinery to v0.35.0 (main) (#20381) (731e067)
distributor: Avoid recalculating the segmentation key hash twice (#19961) (8b78f79)
distributor: Panic when no healthy instances found (#19998) (1c5dfed)
distributor: Track discarded entries and bytes when hitting stream limits using the ingest limits service (#20244) (9b0af7c)
docker: Missing permissions to start docker (#19947) (39d2bea)
docker: Set WORKDIR to root in loki Dockerfiles (#19941) (13f2b1a)
helm: Bump Helm deps, publish loki-helm-test w/ release (#19939) (7e4e34e)
helm: Add startup probe read (#19708) (bce87fb)
helm: Apply fix from #14126 to example (#20252) (716563a)
helm: Correct GEL provisioner tenant creation instructions (#20271) (9639e2e)
helm: Correct GEL version (#19657) (cd8b195)
helm: Do not mark loki.storage.bucketNames.chunks as required, if an s3 url is provided (#19873) (e9951bb)
helm: Do not mark loki.storage.bucketNames.chunks as required, if minio is used. (#19871) (eddd4f8)
helm: Do not mark loki.storage.bucketNames.ruler as required, if rulerConfig.storage.type is local (#19882) (f2f564a)
helm: Don't fail for missing bucket name, if local disk is used. (#19675) (ad0a29e)
helm: Don't fail for missing bucket name, if minio is enabled. (#19745) (cd0c578)
helm: Enable volumeAttributesClassName attributes for volumeClaimTemplates (#19719) (06da42a)
helm: Fix ingester-b volumeAttributesClassName templating (#20188) (d696f18)
helm: Fix rendering of dnsConfig for backend, read, write, single-binary and table-manager (#20013) (1cdb3c7)
helm: Respect global registry in sidecar image (#18246) (#19347) (79eae2c)
helm: Update Chart version in README.md (#19669) (94096b7)
helm: Update version (#19670) (b90ae22)
ingest-limits: Reduce FetchMaxBytes to 10MB (#19883) (82cfafd)
ingest-limits: Revoke partitions if lost (#20030) (1ac5d1f)
kafka: Deadlock on shutdown (#20384) (272a278)
logql: Parsed labels should not override structured metadata (#19991) (61f9367)
lokitool: Update ruler path and enable alternative TLS env variables (#19572) (d1ce5cb)
nomad: Nomad simple example (#19629) (17aec11)
operator: Change leader-election parameters (#19707) (86068cf)
operator: Do not deploy NetworkPolicies automatically on OCP 4.20 (#19680) (8df33ff)
operator: Return quickstart script to working condition and improve rootless usage (#19960) (397da27)
parser: Do not cache key conflicts results in intern set (#19984) (0a9b024)
querier: Support multi-tenant queries in Patterns API (#19809) (f609e27)
querier: Restrict start/end timestamp to requested range for scheduler (#20086) (448cc74)
querier: Set Content-Type header for JSON responses in serializeHTTPHandler (#19878) (019d6b4)
querylimits: Accept request limits over not initialized limits (#19891) (905eac8)
retry: Do not retry if any of multierrors is a client error (#19887) (9825137)
ruler: Validate remote write config (#19920) (e916944)
server: Return status bad request code for interval limit error (#19895) (f21f5d5)
storage: Do not override S3 region if already specified in configuration chain (#20127) (0046bfb)
storage: Fix regression in S3 client configuration (#20110) (d3f9532), closes #19908