1v1.15.4
2===
3
4This patch release fixes a difficult to encounter, but
5fatal-for-group-consuming bug.
6
7The sequence of events to trigger this bug:
8* OffsetCommit is issued before Heartbeat
9* The coordinator for the group needs to be loaded (so, likely, a previous `NOT_COORDINATOR` error was received)
10* OffsetCommit triggers the load
11* a second OffsetCommit happens while the first is still running, canceling the first OffsetCommit's context
12
13In this sequence of events, FindCoordinator will fail with `context.Canceled`
14and, importantly, also return that error to Heartbeat. In the guts of the
15client, a `context.Canceled` error _should_ only happen when a group is being
16left, so this error is recognized as a group-is-leaving error and the group
17management goroutine exits. Thus, the group is never rejoined.
18
19This likely requires a system to be overloaded to begin with, because
20FindCoordinator requests are usually very fast.
21
22The fix is to use the client context when issuing FindCoordinator, rather than
23the parent request. The parent request can still quit, but FindCoordinator
24continues. No parent request can affect any other waiting request.
25
26This patch also includes a dep bump for everything but klauspost/compress;
27klauspost/compress changed go.mod to require go1.19, while this repo still
28requires 1.18. v1.16 will change to require 1.19 and then this repo will bump
29klauspost/compress.
30
31There were multiple additions to the yet-unversioned kfake package, so that an
32advanced "test" could be written to trigger the behavior for this patch and
33then ensure it is fixed. To see the test, please check the comment on PR
34[650](https://github.com/twmb/franz-go/pull/650).
35
36- [`7d050fc`](https://github.com/twmb/franz-go/commit/7d050fc) kgo: do not cancel FindCoordinator if the parent context cancels
37
38v1.15.3
39===
40
41This patch release fixes one minor bug, reduces allocations on gzip and lz4
42decompression, and contains a behavior improvement when OffsetOutOfRange is
43received while consuming.
44
45For the bugfix: previously, if the client was using a fetch session (as is the
46default when consuming), and all partitions for a topic transfer to a different
47broker, the client would not properly unregister the topic from the prior
48broker's fetch session. This could result in more data being consumed and
49discarded than necessary (although, it's possible the broker just reset the
50fetch session anyway, I'm not entirely positive).
51
52- [`fdf371c`](https://github.com/twmb/franz-go/commit/fdf371c) use bytes buffer instead of ReadAll (thanks [@kalbhor](https://github.com/kalbhor)!)
53- [`e6ed69f`](https://github.com/twmb/franz-go/commit/e6ed69f) consuming: reset to nearest if we receive OOOR while fetching
54- [`1b6a721`](https://github.com/twmb/franz-go/commit/1b6a721) **bugfix** kgo source: use the proper topic-to-id map when forgetting topics
55
56v1.15.2
57===
58
59This patch release fixes two bugs and changes Mark functions to be no-ops when
60not using AutoCommitMarks to avoid confusion. This also includes a minor commit
61further improving the sticky balancer. See the commits for more details.
62
63- [`72778cb`](https://github.com/twmb/franz-go/commit/72778cb) **behavior change** kgo: no-op mark functions when not using AutoCommitMarks
64- [`e209bb6`](https://github.com/twmb/franz-go/commit/e209bb6) **bugfix** kgo: pin AddPartitionsToTxn to v3 when using one transaction
65- [`36b4437`](https://github.com/twmb/franz-go/commit/36b4437) sticky: further improvements
66- [`af5bc1f`](https://github.com/twmb/franz-go/commit/af5bc1f) **bugfix** kgo: be sure to use topics when other topics are paused
67
68v1.15.1
69===
70
71This patch release contains a bunch of internal improvements to kgo and
72includes a bugfix for a very hard to encounter logic race. Each improvement
73is a bit focused on a specific use case, so I recommend reading any relevant-to-you
74commit message below.
75
76As well, the kversion package now detects Kafka 3.6, and the kgo package now
77handles AddPartitionsToTxn v4 (however, you will probably not be issuing this
78request).
79
80Lastly, this release is paired with a minor kadm release, which adds the
81ErrMessage field CreateTopicsResponse and DeleteTopicsResponse, and,
82importantly, fixes a data race in the ApiVersions request.
83
84#### franz-go
85
86- [`2a3b6bd`](https://github.com/twmb/franz-go/commit/2a3b6bd) **improvement** kversion: detect 3.6
87- [`fe5a660`](https://github.com/twmb/franz-go/commit/fe5a660) **improvement** kgo: add sharding for AddPartitionsToTxn for KIP-890
88- [`b2ccc2f`](https://github.com/twmb/franz-go/commit/b2ccc2f) **improvement** kgo: reintroduce random broker iteration
89- [`54a7418`](https://github.com/twmb/franz-go/commit/54a7418) **improvement** kgo: allow PreTxnCommitFnContext to modify empty offsets
90- [`c013050`](https://github.com/twmb/franz-go/commit/c013050) **bugfix** kgo: avoid rare panic
91- [`0ecb52b`](https://github.com/twmb/franz-go/commit/0ecb52b) **improvement** kgo: do not rotate the consumer session when pausing topics/partitions
92- [`1429d47`](https://github.com/twmb/franz-go/commit/1429d47) **improvement** sticky balancer: try for better topic distribution among members
93
94#### kadm
95
96- [`1955938`](https://github.com/twmb/franz-go/commit/1955938) **bugfix** kadm: do not reuse ApiVersions in many concurrent requests
97- [`66974e8`](https://github.com/twmb/franz-go/commit/66974e8) **feature** kadm: include ErrMessage in topic response
98
99v1.15.0
100===
101
102This release comes 74 days (just over two months) since the last minor release.
103This mostly contains new features, with one relatively minor but important bug
104addressed (and one _very_ minor bug fixed).
105
106## Bug fixes
107
108* Fetch sessions now properly send forgotten topics if we forget the entire
109 topic (not just individual partitions while other partitions are still
110 consumed on the broker). For long-running clients where partitions move
111 around the cluster a bunch over time, this ensures we are not sending requests
112 with null topics / null topic IDs. See [#535](https://github.com/twmb/franz-go/issues/535)
113 for more details.
114* If the client talks to an http endpoint, we now properly detect the bytes
115 'HTTP' and send a more relevant error message. This previously existed, but
116 used the wrong int32 internally so 'HTTP' was not properly detected (thanks
117 [@alistairking](https://github.com/alistairking)!).
118
119## Features
120
121* `RecordReader` now supports %v{json} to parse json inputs
122* `RecordReader` now supports %v{}, an empty no-op formatting directive
123* Adds `PurgeTopicsFromProducing` and `PurgeTopicsFromConsuming` to purge _either_
124 the producing or consuming half of the client, if you are producing to and
125 consuming from the same topics in the same client.
126* The new `ConsiderMissingTopicDeletedAfter` option allows finer grained
127 control for how long a regexp-discovered topic can be missing on the cluster
128 before the topic is considered deleted (and then internally purged in the
129 client)
130* Adds `NewErrFetch` to create a single-error `Fetches`, for use in end-user
131 tests / stubs.
132* The new `MaxBufferedBytes` option can control how many bytes are buffered
133 when producing, an option similar to `MaxBufferedRecords`.
134* Adds `BufferedFetchBytes` and `BufferedProduceBytes` to return the total
135 bytes in records buffered (note this counts only keys, values, and headers).
136* Adds `PreTxnCommitFnContext` to allow custom `Metadata` annotation for
137 transactional commits.
138* Adds `LeaveGroupContext` to control how long leaving a group can take, as
139 well as to return any leave group error.
140
141## Relevant commits
142
143#### franz-go
144
145- [`4dcfb06`](https://github.com/twmb/franz-go/commit/4dcfb06) **feature** kgo: add LeaveGroupContext
146- [`910e91d`](https://github.com/twmb/franz-go/commit/910e91d) and [`60b601a`](https://github.com/twmb/franz-go/commit/60b601a) **feature** kgo: add PreTxnCommitFnContext
147- [`c80d6f4`](https://github.com/twmb/franz-go/commit/c80d6f4) **feature** kgo: add Buffered{Fetch,Produce}Bytes
148- [`304559f`](https://github.com/twmb/franz-go/commit/304559f) **feature** kgo: support MaxBufferedBytes
149- [`310a5da`](https://github.com/twmb/franz-go/commit/310a5da) **feature** kgo: add NewErrFetch
150- [`504a9d7`](https://github.com/twmb/franz-go/commit/504a9d7) **feature** kgo: expose ConsiderMissingTopicDeletedAfter
151- [`253e1a9`](https://github.com/twmb/franz-go/commit/253e1a9) **feature** kgo: add PurgeTopicsFrom{Producing,Consuming}
152- [`055e2d8`](https://github.com/twmb/franz-go/commit/055e2d8) **improvement** kgo record formatter: accept %v{} to be a no-op (plain read/format)
153- [`37edfb9`](https://github.com/twmb/franz-go/commit/37edfb9) **improvement** kgo.RecordReader: support %v{json} to read json values
154- [`8a9a459`](https://github.com/twmb/franz-go/commit/8a9a459) **bugfix** kgo: track topic IDs in the fetch session
155- [`9d25d3a`](https://github.com/twmb/franz-go/commit/9d25d3a) **bugfix** kgo: fix typo in parseReadSize to properly detect and warn about talking to HTTP endpoints (thanks [@alistairking](https://github.com/alistairking)!)
156
157#### kadm
158
159This release comes with a corresponding kadm release that contains a few
160behavior changes and improvements.
161
162- [`bfd07b2`](https://github.com/twmb/franz-go/commit/bfd07b2) kadm: fix occasionally empty topic/partitions in Lag
163- [`00ac608`](https://github.com/twmb/franz-go/commit/00ac608) kadm: change FetchOffsetsForTopics to only return requested topics by default
164
165v1.14.4
166===
167
168This small patch fixes kversion.VersionGuess to properly guess versions against
169zookeeper broker versions v2.7 through 3.4. See the commit for details.
170
171- [`5978156`](https://github.com/twmb/franz-go/commit/5978156) **bugfix** kversion: fix version detection for Kafka v2.7 through 3.4
172
173v1.14.3
174===
175
176This patch fixes regex consuming a deleted topic causing an unending internal
177loop of metadata reloading (trying to discover where the topic went).
178
179- [`627d39a`](https://github.com/twmb/franz-go/commit/627d39a) **bugfix** kgo: fix / improve handling deleted topics while regex consuming
180
181v1.14.2
182===
183
184This patch fixes an internal logic race that can be easily encountered when
185specifying exact offsets to consume from. If you encountered this bug, your
186consumer could just stop consuming for an indeterminite amount of time. This
187bug has existed for a _long_ time and relies on both the client being slow and
188the broker being fast to hit.
189
190- [`1f696ca`](https://github.com/twmb/franz-go/commit/1f696ca) **bugfix** kgo: avoid a consumer logic race where the consumer stops consuming
191
192v1.14.1
193===
194
195This patch release, quick on the heels of v1.14.0, fixes a race condition
196introduced in v1.14 in the `PauseFetchTopics` and `PauseFetchPartitions`
197functions, a second race condition that can occur when purging a topic, and
198fully addresses [#493][https://github.com/twmb/franz-go/issues/493] which was
199not completely addressed in v1.14.0.
200
201- [`8c785fa`](https://github.com/twmb/franz-go/commit/8c785fa) **bugfix** kgo: fix race between client closing and purging
202- [`dc5283e`](https://github.com/twmb/franz-go/commit/dc5283e) kgo: re-fix #493, supporting other buggy clients, and add a test
203- [`32ac27f`](https://github.com/twmb/franz-go/commit/32ac27f) **bugfix** kgo: ensure assignPartitions is locked when pausing topics/partitions
204
205v1.14.0
206===
207
208This release contains a few new APIs, one behavior change, and one minor bugfix.
209
210## Bug fixes
211
212Previously, `HookBrokerRead` and `HookBrokerE2E` could not be used at the same
213time. This has been fixed.
214
215## Behavior changes
216
217`PauseFetch{Topics,Partitions}` now causes the client to drop all buffered
218fetches and kill all in-flight fetch requests. Importantly, this also means
219that once you pause, it is no longer possible for what you paused to be
220returned while polling. Previously, the client made no attempt to clear
221internal buffers / in flight requests, meaning you could receive paused data
222for a while.
223
224Seed brokers now show up in logs as `seed_###` rather than `seed ###` (an
225underscore has been added).
226
227## Features
228
229* `kgo.Offset` now has an `EpochOffset` getter function that allows access
230 to the actual epoch and offset that are inside the opaque `Offset` type.
231* `AddConsumePartitions` allows adding individual partitions to consume, and
232 the new counterpart `RemoveConsumePartitions` allows removing individual
233 partitions from being consumed. Removing is different from purging, please
234 see the docs.
235* `KeepRetryableFetchErrors` bubbles up retryable errors to the end user that
236 are encountered while fetching. By default, these errors are stripped.
237* kversion now supports Kafka 3.5
238* kversion now supports version guessing against KRaft by default
239* `kgo.DialTLS` now exists to even more easily opt into TLS.
240* `kgo.Client.Opts` now exists to return the original options that were used
241 to configure the client, making initializing new clients easier.
242* `kgo.NodeName` returns a string form of a broker node name. Internally, seed
243 brokers use math.MinInt32 for node IDs, which shows up as massively negative
244 numbers in logs sometimes. `NodeName` can help convert that to `seed_<#>`.
245
246## Relevant commits
247
248- [`c3b083b`](https://github.com/twmb/franz-go/commit/c3b083b) **improvement** kgo: do not returned paused topics/partitions after pausing
249- [`e224e90`](https://github.com/twmb/franz-go/commit/e224e90) **bugfix** kgo: allow HookBrokerRead and HookBrokerE2E to both be called
250- [`875761a`](https://github.com/twmb/franz-go/commit/875761a) **feature** kgo Offset: add EpochOffset getter field
251- [`c5d0fc5`](https://github.com/twmb/franz-go/commit/c5d0fc5) kgo: add a debug log for stripping retryable errors from fetches
252- [`b45d663`](https://github.com/twmb/franz-go/commit/b45d663) kgo: add more context to opportunistic metadata loads while fetching
253- [`9dae366`](https://github.com/twmb/franz-go/commit/9dae366) kgo: allow retries on dial timeouts
254- [`00e4e76`](https://github.com/twmb/franz-go/commit/00e4e76) kgo: tolerate buggy v1 group member metadata
255- [`34c8b3d`](https://github.com/twmb/franz-go/commit/34c8b3d) **feature** kgo: add AddConsumePartitions and RemoveConsumePartitions
256- [`b5cafba`](https://github.com/twmb/franz-go/commit/b5cafba) sasl: validate non-empty user/pass/token
257- [`76d2e71`](https://github.com/twmb/franz-go/commit/76d2e71) **feature** kgo: add KeepRetryableFetchErrors
258- [`0df3ec0`](https://github.com/twmb/franz-go/commit/0df3ec0) kgo: fix new niche CI problem against Kafka 3.5
259- [`8ff1d0d`](https://github.com/twmb/franz-go/commit/8ff1d0d) **feature** pkg/kversion: attempt to guess KRaft by default as well
260- [`e92f5d9`](https://github.com/twmb/franz-go/commit/e92f5d9) **feature** pkg/kversion: detect v3.5
261- [`f1b923e`](https://github.com/twmb/franz-go/commit/f1b923e) **feature** kgo: add DialTLS option
262- [`9667967`](https://github.com/twmb/franz-go/commit/9667967) **feature** kgo.Client: add Opts to return the original opts, to allow seeding new clients
263- [`8e14928`](https://github.com/twmb/franz-go/commit/8e14928) **feature** kgo: add NodeName for easy user formatting of internal seed node IDs
264
265v1.13.6
266===
267
268This minor patch release allows an injected fake fetch to be returned if that
269is the only fetch ready to be returned. A fake fetch is a fetch that actually
270does not contain any data, but instead contains an error (and may contain an
271empty topic with a single 0 partition). Previously, a fake fetch could only be
272returned if actual data was also ready to be returned. Now, a fake fetch can
273always be returned. Fake fetches are most commonly experienced if you are
274unable to join a group -- the client injects a fake fetch to notify you, the
275client, that an error occurred. This patch actually allows you to consume that
276error.
277
278- [`5c87ce0`](https://github.com/twmb/franz-go/commit/5c87ce0) consumer: return from Poll if the only error is an injected fake
279
280v1.13.5
281===
282
283This tiny patch release relaxes SASL/PLAIN to ignore any server response. This
284patch is only useful if you are using this client against Tencent (at least as
285we known at this moment). A normal broker does not reply to a successful PLAIN
286auth with any data, but Tencent apparently does. Sarama and the Kafka client
287itself both seem to ignore extra data once auth is successful, and if a broker
288actually _rejected_ the auth then the broker would close the connection, so
289ignoring this data seems fine.
290
291- [`3addecc`](https://github.com/twmb/franz-go/commit/3addecc) sasl plain: ignore any challenge data
292
293v1.13.4
294===
295
296This bugfix release fixes a race condition when calling client.Close.
297This bug was introduced in commit [`e45cd72`](https://github.com/twmb/franz-go/commit/e45cd72) in
298release v1.13.0. If a metadata request discovers a new broker as the client is
299closing, there is a tiny window where the metadata goroutine can update a map
300that the Close function is also concurrently accessing. The commit in this
301release fixes that by guarding the Close access with a mutex.
302
303- [`ee3d7c1`](https://github.com/twmb/franz-go/commit/ee3d7c1) **bugfix** kgo: fix race condition in close (thanks [@matino](https://github.com/matino)
304
305v1.13.3
306===
307
308This minor patch release adds a few internal improvements and fixes no bugs.
309
310Release 1.13.2 patched commits from occuring between join and sync; this patch
311extends that to blocking LeaveGroup or Close if an active join and sync is
312occurring. This is necessary because revoke callbacks are called while leaving
313a group, and usually, you commit in a revoke callback.
314
315A few extra logs are added, some areas of code are better, and a
316context.Canceled error that could occasionally be returned in fetches when an
317internal consumer session changed is no longer returned.
318
319This release went through three+ straight days of looped integration testing on
320two laptops to track down extremely rare test failures; those have been tracked
321down.
322
323Lastly, if you use a regex consumer and delete topics, the client now
324internally purges and stops fetching those topics. Previously, any topic
325discovered while regex consuming was permanently consumed until you manually
326called `PurgeTopicsFromClient`.
327
328- [`bb66f24`](https://github.com/twmb/franz-go/commit/bb66f24) kgo: purge missing-from-meta topics while regex consuming
329- [`f72fdaf`](https://github.com/twmb/franz-go/commit/f72fdaf) kgo: always retry on NotLeader for sharded requests
330- [`682d1f8`](https://github.com/twmb/franz-go/commit/682d1f8) kgo: add info log when the client is throttled
331- [`88fa883`](https://github.com/twmb/franz-go/commit/88fa883) kgo: avoid counting pinReq version failures against retries
332- [`de53fda`](https://github.com/twmb/franz-go/commit/de53fda) kgo: add a bit more context to sharded logs, avoid info log on Close
333- [`7338bcf`](https://github.com/twmb/franz-go/commit/7338bcf) kgo: avoiding context.Canceled fetch from List/Epoch, improve testing&logs
334- [`055b349`](https://github.com/twmb/franz-go/commit/055b349) consumer: do not use the partition epoch when assigning offsets
335- [`d833f61`](https://github.com/twmb/franz-go/commit/d833f61) group consuming: block LeaveGroup between join&sync
336
337v1.13.2
338===
339
340This patch improves the behavior of committing offsets while a rebalance is
341happening.
342
343In Kafka, if a commit arrives on the broker between JoinGroup and SyncGroup,
344the commit is rejected with `REBALANCE_IN_PROGRESS`. If a commit is started
345before JoinGroup but arrives to the broker after SyncGroup, the commit is
346rejected with `INVALID_GENERATION`.
347
348This patch changes how committing offsets interacts with join&sync: a client
349that is joining the group will block OffsetCommit (preventing the first error),
350and issuing an OffsetCommit will block the client from entering JoinGroup
351(preventing the second error).
352
353Previously, you could occasionally encounter these errors with the
354cooperative-sticky group balancer because the standard behavior for eager
355balancers is to commit in OnPartitionsRevoked before all partitions are dropped
356at the end of a group session. You now should no longer encounter these errors
357during a commit at all with any balancer unless something is going quite wrong
358(massive client timeouts or broker hangs).
359
360This does mean that OffsetCommit may take longer if you happen to commit in
361while a client is joining and the join&sync takes a long time.
362
363- [`ee70930`](https://github.com/twmb/franz-go/commit/ee70930) kgo groups: block join&sync while a commit is inflight
364
365v1.13.1
366===
367
368This patch release fixes a bug where a producer could enter a deadlock if a
369topic is deleted and recreated very quickly while producing.
370
371- [`769e02f`](https://github.com/twmb/franz-go/commit/769e02f) producer: avoid deadlock when when quickly recreating a topic
372
373v1.13.0
374===
375
376This release contains a few new APIs, two rare bug fixes, updates to plugins,
377and changes the library to now require 1.18.
378
379## Go version
380
381This library has supported Go 1.15 since the beginning. There have been many
382useful features that this library has not been able to use because of continued
383backcompat for 1.15. There is really no reason to support such an old version
384of Go, and Go itself does not support releases prior to 1.18 -- and 1.18 is
385currently only supported for security backports. Switching to 1.18 allows this
386library to remove a few 1.15 / 1.16 backcompat files, and allows switching this
387library from `interface{}` to `any`.
388
389## Behavior changes
390
391If group consuming fails with an error that looks non-retryable, the error is
392now injected into polling as a fake errored fetch. Multiple people have ran
393into problems where their group consumers were failing due to ACLs or due to
394network issues, and it is hard to detect these failures: you either have to pay
395close attention to logs, or you have to hook into HookGroupManageError. Now,
396the error is injected into polling.
397
398## Bug fixes
399
400This release contains two bug fixes, one of which is very rare to encounter, and
401one of which is very easy to encounter but requires configuring the client in
402a way that (nearly) nobody does.
403
404Rare: If you were using EndAndBeginTransaction, there was an internal race that
405could result in a deadlock.
406
407Rare configuration: If you configured balancers manually, and you configured
408CooperativeSticky with any other eager balancer, then the client would
409internally _sometimes_ think it was eager consuming, and _sometimes_ think it
410was cooperative consuming. This would result in stuck partitions while
411consuming.
412
413## Features
414
415* HookClientClosed: A new hook that allows a callback when the client is closed.
416* HookProduceRecordPartitioned: A new hook that is called when a record's partition is chosen.
417* Client.ConfigValue: Returns the value for any configuration option.
418* Client.ConfigValues: Returns the values for any configuration option (for multiple value options, or for strings that are internally string pointers).
419* kadm: a few minor API improvements.
420* plugin/klogr: A new plugin that satisfies the go-logr/logr interfaces.
421* pkg/kfake: A new experimental package that will be added to over time, this mocks brokers and can be used in unit testing (only basic producing & consuming supported so far).
422
423## Relevant commits
424
425- [`1b229ce`](https://github.com/twmb/franz-go/commit/1b229ce) kgo: bugfix transaction ending & beginning
426- [`461d2ef`](https://github.com/twmb/franz-go/commit/461d2ef) kgo: add HookClientClosed and HookProduceRecordPartitioned
427- [`3a7f35e`](https://github.com/twmb/franz-go/commit/3a7f35e) kgo.Client: add UpdateFetchMaxBytes
428- [`b0fa1a0`](https://github.com/twmb/franz-go/commit/b0fa1a0) kgo.Client: add ConfigValue and ConfigValues
429- [`b487a15`](https://github.com/twmb/franz-go/commit/b487a15) kgo: inject the group lost error into polling as `*ErrGroupLost`
430- [`bc638b0`](https://github.com/twmb/franz-go/commit/bc638b0) plugin/kotel.Tracer: add KeyFormatter, only accept utf8 keys
431- [`a568b21`](https://github.com/twmb/franz-go/commit/a568b21) bugfix kgo: do not default to eager if there is any eager balancer
432- [`bf20ac0`](https://github.com/twmb/franz-go/commit/bf20ac0) plugin/kzap: support LogLevelNone
433- [`a9369be`](https://github.com/twmb/franz-go/commit/a9369be) kadm: add state config altering using the older AlterConfigs request
434- [`f5ddf71`](https://github.com/twmb/franz-go/commit/f5ddf71) kadm: add NumPartitions,ReplicationFactor,Configs to CreateTopicResponse
435- [`e45cd72`](https://github.com/twmb/franz-go/commit/e45cd72) consuming: close fetch sessions when closing the client
436- [`d8230ca`](https://github.com/twmb/franz-go/commit/d8230ca) plugin/klogr: support for go-logr
437- [`0f42e43`](https://github.com/twmb/franz-go/commit/0f42e43) check hooks and flatten slices of hooks
438
439v1.12.1
440===
441
442This patch adds back the pre-v1.11.7 behavior of returning `UNKNOWN_TOPIC_ID`
443when consuming. Patch v1.11.7 explicitly stripped this error because it was
444found during testing that Kafka 3.4 now returns the error occasionally when
445immediately consuming from a new topic (i.e. in tests), and the error is
446explicitly marked as retryable within the Kafka source code. However, with
447topic IDs, there is no way for a client to recover if the topic has been
448deleted and recreated. By returning the error, end users can notified that
449their client is in a fatal state and they should restart. In the future, I will
450add an option to consume recreated topics.
451
452- [`efe1cdb`](https://github.com/twmb/franz-go/commit/efe1cdb) consumer: return UnknownTopicID even though it is retryable
453
454v1.12.0
455===
456
457This release is tested against Kafka 3.4, and kversion can now properly version
458guess 3.4. There are two external kgo features in this release and there is one
459internal feature.
460
461Externally,
462
463* `Client.CommitMarkedOffsets` makes committing marked offsets a bit easier
464* `Client.UpdateSeedBrokers` allows for updating seed brokers, which can be useful on extremely long lived clients.
465
466Internally,
467
468* KIP-792 improves fencing of zombie group consumers using cooperative rebalancing
469
470- [`c5f86ea`](https://github.com/twmb/franz-go/commit/c5f86ea) **feature** kgo.Client: add `UpdateSeedBrokers(...string) error`
471- [`3e45339`](https://github.com/twmb/franz-go/commit/3e45339) **internal improvement** group balancer: support KIP-792
472- [`fe727f8`](https://github.com/twmb/franz-go/commit/fe727f8) **feature** kversion: cut Kafka 3.4
473- [`6751589`](https://github.com/twmb/franz-go/commit/6751589) **feature** pkg/kgo: add `Client.CommitMarkedOffsets` (thanks [@celrenheit](https://github.com/celrenheit)!)
474
475v1.11.7
476===
477
478The integration test introduced in v1.11.6 was frequently failing in CI. After
479some investigation, the panic is caused from `PurgeTopicsFromClient` _while_ a
480load of offsets was happening (i.e. consuming was being initialized). That
481race is specifically easily triggered in the integration test.
482
483This patch also relaxes the error stripping while consuming. While consuming,
484retryable errors are stripped. The errors that were stripped was a small
485specific set that was originally taken from Sarama (nearly 4yr ago). Kafka 3.4
486returns a new retryable error frequently if consuming against a topic
487immediately after the topic is created. This patch changes the error set to
488just strip all retryable errors, since the kerr package already indicates which
489errors are retryable. This is more compatible against the broker returning
490errors that are retryable.
491
492This is the last commit in the v1.11 series -- immediately after tagging this,
493I will be working on v1.12 which will officially test against Kafka 3.4.
494
495- [`17567b0..78a12c3`](https://github.com/twmb/franz-go/compare/17567b0..78a12c3) **bugfix** consumer: fix potential panic when calling PurgeTopicsFromClient
496- [`17567b0`](https://github.com/twmb/franz-go/commit/17567b0) **improvement** kgo: always strip retryable errors when consuming
497
498v1.11.6
499===
500
501v1.11.5 introduced a regression when:
502
503* Using a direct consumer (not a consumer group)
504* Using `ConsumePartitions`
505* Not specifying _all_ partitions in the topic
506
507With the above setup, 1.11.5 introduced a bug that would consume all partitions
508in the topic, i.e. all partitions in a topic that were not specified in
509`ConsumePartitions`.
510
511This patch restores the old behavior to only consume partitions that are
512requested. This patch also makes it possible to purge topics or partitions from
513the direct consumer, and makes it possible to re-add a topic. Lastly, three
514integration tests were added to ensure that the problems fixed in 1.11.5 and in
515this patch do not crop up again.
516
517* [`edd0985`](https://github.com/twmb/franz-go/commit/edd0985) **bugfix** kgo: patch ConsumePartitions regression from 1.11.5
518
519v1.11.5
520===
521
522v1.11.4 is retracted and because it actually requires a v2 release. When a
523breaking kmsg release must happen, v2 of franz-go will happen and I will make a
524change thorough enough such that major kmsg changes will not require a major
525franz-go change again.
526
527When this repo was initially written, the Kafka documentation for the wire
528serialization of a Kafka record was wrong and different from how things were
529actually implemented in the Java source. I used the documentation for my own
530implementation. The documentation was [later fixed](https://github.com/apache/kafka/commit/94ccd4d).
531
532What this means is that in this repo, `kmsg.Record.TimestampDelta` should be
533changed from an int32 to int64. Rather than release a major version of kmsg, I
534have added a `TimestampDelta64` field.
535
536The old 32 bit timestamp delta meant that this package could only represent
537records that had up to 24 days of a delta within a batch. Apparently, this may
538happen in compacted topics.
539
540More minor: previously, `AddConsumeTopics` did not work on direct consumers nor
541on a client that consumed nothing to begin with. These shortcomings have been
542addressed.
543
544* [`12e3c11`](https://github.com/twmb/franz-go/commit/12e3c11) **bugfix** franz-go: support 64 bit timestamp deltas
545* [`f613fb8`](https://github.com/twmb/franz-go/commit/f613fb8) **bugfix** pkg/kgo: patch AddConsumeTopics
546
547v1.11.3
548===
549
550This patch release fixes a panic that can occur when fetching offsets in the
551following scenario:
552
553* You are fetching offsets for a group that the client has not yet loaded
554 internally
555* The internal load's FindCoordinator request fails OR the group cannot be
556 loaded
557
558FindCoordinator usually does not fail outright because the request internally
559retries. As well, the group load is usually successful. Group loading only
560fails if you are unauthorized to describe the group or if the group coordinator
561is not loaded.
562
563The most common case to encounter this error is when you issue a group request
564against a new cluster. The first time a group request is seen, the group
565coordinator loads. While loading, group requests are failed with
566`COORDINATOR_LOAD_IN_PROGRESS` or some other similar error.
567
568* [`5289ef6`](https://github.com/twmb/franz-go/commit/5289ef6) **bugfix** kgo.Client: avoid panic in OffsetFetchRequest when coordinator is not loaded
569
570v1.11.2
571===
572
573This patch release fixes `HookFetchRecordUnbuffered` never being called if a
574hook also implemented `HookFetchRecordBuffered`. No existing plugin currently
575implements these hooks (though one will soon), so this patch is only relevant
576to you if you manually have added these hooks.
577
578* [`2a37df9`](https://github.com/twmb/franz-go/commit/2a37df9) **bugfix** kgo: patch HookFetchRecordUnbuffered
579
580
581v1.11.1
582===
583
584This patch release fixes a bug in `ConsumePreferringLagFn`. The code could
585panic if you:
586
587* Consumed from two+ topics
588* Two of the topics have a different amount of partitions
589* The single-partition topic has some lag, the topic with more partitions has
590 one partition with no lag, and another partition with _more_ lag than the
591 single-partition topic
592
593In this case, the code previously would create a non-existent partition to
594consume from for the single-partition topic and this would immediately result
595in a panic when the fetch request was built.
596
597See the commit for more details.
598
599* [`38f2ec6`](https://github.com/twmb/franz-go/commit/38f2ec6) **bugfix** pkg/kgo: bugfix ConsumePreferringLagFn
600
601v1.11.0
602===
603
604This is a small release containing two minor features and a few behavior
605improvements. The `MarkedOffsets` function allows for manually committing
606marked offsets if you override `OnPartitionsRevoked`, and
607`UnsafeAbortBufferedRecords` allows for forcefully dropping anything being
608produced (albeit with documented caveats and downsides)
609
610The client now guards against a broker that advertises FetchRequest v13+ (which
611_only_ uses TopicIDs) but does not actually return / use TopicIDs. If you have
612an old IBP configured, the broker will not use TopicIDs even if the broker
613indicates it should. The client will now pin fetching to a max version of 12 if
614a topic has no TopicID.
615
616The client now sets a record's `Topic` field earlier to `DefaultProduceTopic`,
617which allows the `Topic` field to be known present (or known non-present) in
618the `OnRecordBuffered` hook.
619
620Lastly, we now universally use Go 1.19 atomic types if compiled with 1.19+. Go
621uses compiler intrinsics to ensure proper int64 / uint64 alignment within
622structs for the atomic types; Go does not ensure plain int64 / uint64 are
623properly aligned. A lot of work previously went into ensuring alignment and
624having a GitHub workflow that ran `go vet` on qemu armv7 emulation, but
625apparently that was not comprehensive enough. Now, if you use a 32 bit arch, it
626is recommended to just compile with 1.19+.
627
628* [`d1b6897`](https://github.com/twmb/franz-go/commit/d1b6897) **feature** kgo: add UnsafeAbortBufferedRecords
629* [`d0c42ad`](https://github.com/twmb/franz-go/commit/d0c42ad) **improvement** kgo source: do not use fetch topic IDs if the broker returns no ID
630* [`cc3a355`](https://github.com/twmb/franz-go/commit/cc3a355) and [`a2c4bad`](https://github.com/twmb/franz-go/commit/a2c4bad) **improvement** kgo: universally switch to 1.19's atomics if on Go 1.19+
631* [`66e626f`](https://github.com/twmb/franz-go/commit/66e626f) producer: set Record.Topic earlier
632* [`3186e61`](https://github.com/twmb/franz-go/commit/3186e61) **feature** kgo: add MarkedOffsets function
633
634v1.10.4
635===
636
637This patch release fixes two bugs introduced with v1.10.0. These bugs are not
638encountered in when using the client to simply consume or produce. Only admin
639usages of the client may encounter the bugs this patch is fixing.
640
641v1.10.0 introduced support for batch offset fetching or coordinator finding.
642These changes introduced a bug where empty coordinator keys (i.e., group names
643or transactional IDs) would be stripped from requests, and then a field in a
644nil pointer could be accessed and panic the program. These changes also
645introduced a bug that did not properly mirror one field for batched
646`FindCoordinator` requests.
647
648- [`ca67da4`](https://github.com/twmb/franz-go/commit/ca67da4) **bugfix** kgo: fix batch coordinator fetching
649- [`c6f7f9a`](https://github.com/twmb/franz-go/commit/c6f7f9a) **bugfix** kgo: allow empty groups when finding coordinator / fetching offsets
650
651v1.10.3
652===
653
654This small patch release is another attempted fix at [#239](https://github.com/twmb/franz-go/issues/239).
655It is only possible to encounter this bug if a broker completely dies and never
656comes back, and you do not replace the broker (i.e., broker 3 dies and it is
657just gone forever).
658
659Previously, kgo would cache the broker controller until `NOT_CONTROLLER` is
660seen. We now clear it a bit more widely, but this is just extra defensive
661behavior: the controller is updated on every metadata request.
662
663Worse however, kgo previously cached group or transactional-id coordinators
664until `COORDINATOR_NOT_AVAILABLE`, `COORDINATOR_LOAD_IN_PROGRESS`, or
665`NOT_CONTROLLER` were seen. If the coordinator outright died and never comes
666back and is never replaced, all coordinator requests to that specific
667coordinator would fail.
668
669Now, if we fail to dial the coordinator or controller 3x in a row, we delete
670the coordinator or controller to force a reload on the next retry. We only do
671this for dial errors because any other error means we actually contacted the
672broker and it exists.
673
674Lastly, we change the default max produce record batch bytes from 1,000,000 to
6751,000,012, to exactly mirror Kafka's max.message.bytes.
676
677- [`e2e80bf`](https://github.com/twmb/franz-go/commit/e2e80bf) kgo: clear controller/coordinator caches on failed dials
678
679v1.10.2
680===
681
682This patch release contains one very minor bug fix, tightens a failure
683scenario, adds two missing errors to kerr, fixes a build constraint, and has a
684few internal style fixes from [@PleasingFungus][1.10.2:pf] (thanks!).
685
686[1.10.2:pf]: https://github.com/PleasingFungus
687
688The bug was introduced in v1.9.0 through a patch that fixed a potential spin
689loop. In fixing the spin loop, I inadvertently caused consumer fetch sessions
690to reset when there is no more data to consume. In your application, this would
691show up as more frequent metadata updates and up to 100ms of extra latency when
692there is new data to consume.
693
694The tightened failure scenario allows records to be failed in more cases.
695Previously, as soon as a record was added to a produce request internally, the
696record could not be failed until a produce response is received. This behavior
697exists for duplicate prevention. However, there is a period of time between a
698produce request being created and actually being written, and if an
699`AddPartitionsToTxn` request takes a long time and then fails, the produce
700request would never be written and the records could never be failed. The
701tightened failure scenario allows records to be failed all the way up until
702they are actually serialized and written.
703
704- [`d620765`](https://github.com/twmb/franz-go/commit/d620765) sink: tighten canFailFromLoadErrs
705- [`6c0abd1`](https://github.com/twmb/franz-go/commit/6c0abd1) **minor bugfix** source: avoid backoff / session reset when there is no consumed data
706- [`6ce8bdf`](https://github.com/twmb/franz-go/commit/6ce8bdf) kerr: add two missing errors to ErrorForCode
707- [PR #264](https://github.com/twmb/franz-go/pull/264) fix `isNetClosedErr` build constraints: franz-go was using the `strings.Contains` version, which was meant for Go 1.15 only (thanks [@PleasingFungus][1.10.2:pf]!)
708
709v1.10.1
710===
711
712This patch release contains minor kgo internal improvements, and enables a new
713API for kadm. This patch contains no bug fixes. This patch should help recover
714faster if a transaction coordinator goes down while producing.
715
716This is released in tandem with kadm v1.6.0, which contains a small kadm bugfix
717for `DeleteRecords`, `OffsetForLeaderEpoch`, and `DescribeProducers`, and
718(hopefully) finishes support for all current admin APIs.
719
720- [`56fcfb4`](https://github.com/twmb/franz-go/commit/56fcfb4) sink: log all aspects of wanting to / failing records
721- [`9ee5efa`](https://github.com/twmb/franz-go/commit/9ee5efa) sink: update metadata when AddPartitionsToTxn fails repeatedly
722- [`bc6810d`](https://github.com/twmb/franz-go/commit/bc6810d) broker: hide retryable errors *once*
723- [`83f0dbe`](https://github.com/twmb/franz-go/commit/83f0dbe) kgo: add support for sharding WriteTxnMarkers
724
725v1.10.0
726===
727
728This is a small release that contains one bug fix, one new feature, and
729improvements in log lines, and improvements to work around AWS MSK being a bit
730odd with SASL reauthentication.
731
732Previously, the client's sticky partitioners actually did not preserve
733stickiness because the client did not attach previous-partitions when rejoining
734the group. That is now fixed.
735
736The new feature, `ConsumePreferringLagFn`, allows you to have some advanced
737reordering of how to consume. The recommended way of using this option is
738`kgo.ConsumePreferringLagFn(kgo.PreferLagAt(50))`, which allows you to favor
739laggy partitions if the client is more than 50 records behind in the topic.
740
741The kadm module is also released with v1.4.0, which contains new APIs to find
742coordinators for groups or transactional IDs, and an API to fetch API versions
743for all brokers in the cluster.
744
745- [`a995b1b`](https://github.com/twmb/franz-go/commit/a995b1b) kgo broker: retry sasl auth failures during reauthentication
746- [`8ab8074`](https://github.com/twmb/franz-go/commit/8ab8074) kgo connection: always allow one request after SASL
747- [`dcfcacb`](https://github.com/twmb/franz-go/commit/dcfcacb) **bugfix** `{Cooperative,Sticky}Balancer`: bug fix lack of stickiness
748- [`76430a8`](https://github.com/twmb/franz-go/commit/76430a8) **feature** kgo: add `ConsumePreferringLagFn` to consume preferring laggy partitions
749- [`9ac6c97`](https://github.com/twmb/franz-go/commit/9ac6c97) **improvement** kgo: support forward & backward batch requests for FindCoordinator, OffsetFetch
750
751
752v1.9.1
753===
754
755This is a small patch release to work around two behavior problems, one with
756AWS and one with Redpanda. This is not an important bug release if you are
757using this library against Kafka itself.
758
759For AWS, AWS is unexpectedly expiring certain permissions before the SASL
760lifetime is up. This manifests as `GROUP_AUTHORIZATION_ERROR` while consuming.
761Previously, we the client would mark connections to reauthenticate when the
762connection was within 3s of SASL expiry. We now are more pessimistic and
763reauthenticate within 95% to 98% of the lifetime, with a 2s minimum. This is
764similar to the Java client, which has always used 85 to 95% of the SASL
765lifetime and has no minimum.
766
767For Redpanda, Redpanda's transaction support is nearly complete (v22.3 release
768imminent), but Redpanda can return `UNKNOWN_SERVER_ERROR` a bit more than Kafka
769does. These errors are being ironed out, but there is no harm in the client to
770pre-emptively handling these as retryable.
771
772- [`3ecaff2`](https://github.com/twmb/franz-go/commit/3ecaff2) kgo txn: handle `UNKNOWN_SERVER_ERROR` more widely
773- [`eb6e3b5`](https://github.com/twmb/franz-go/commit/eb6e3b5) kgo sasl reauth: be more pessimistic
774
775v1.9.0
776===
777
778This release contains one important bugfix (sequence number int32 overflow) for
779long-lived producers, one minor bugfix that allows this client to work on 32
780bit systems, and a few other small improvements.
781
782This project now has integration tests ran on every PR (and it is now forbidden
783to push directly to master). These integration tests run against Kraft (Kafka +
784Raft), which itself seems to not be 100% polished. A good amount of
785investigation went into hardening the client internals to not fail when Kraft
786is going sideways.
787
788This release also improves behavior when a consumer group leader using an
789instance ID restarts _and_ changes the topics it wants to consume from. See the
790KIP-814 commit for more details.
791
792It is now easier to setup a TLS dialer with a custom dial timeout, it is easier
793to detect if requests are failing due to missing SASL, and it is now possible
794to print attributes with `RecordFormatter`.
795
796Lastly, the corresponding kadm v1.3.0 release adds new LeaveGroup admin APIs.
797
798#### franz-go
799
800- [`b18341d`](https://github.com/twmb/franz-go/commit/b18341d) kgo: work around KIP-814 limitations
801- [`6cac810`](https://github.com/twmb/franz-go/commit/6cac810) kversions: bump Stable from 3.0 to 3.3
802- [PR #227](https://github.com/twmb/franz-go/pull/227) **bugfix** further sequence number overflows fix (thanks [@ladislavmacoun](https://github.com/ladislavmacoun)!)
803- [PR #223](https://github.com/twmb/franz-go/pull/223) add GitHub actions integration test (thanks [@mihaitodor](https://github.com/mihaitodor)!) and [PR #224](https://github.com/twmb/franz-go/pull/224) fixup kgo guts to work around new kraft failures
804- [`203a837`](https://github.com/twmb/franz-go/commit/203a837) franz-go: fix 32 bit alignment, fix a few lints
805- [`719c6f4`](https://github.com/twmb/franz-go/commit/719c6f4) kgo: avoid overflow on 32 bit systems
806- [`db5c159`](https://github.com/twmb/franz-go/commit/db5c159) **feature** kgo: add DialTimeout function, complementing DialTLSConfig
807- [`b4aebf4`](https://github.com/twmb/franz-go/commit/b4aebf4) kgo: add ErrFirstReadEOF, which unwraps to io.EOF
808- [`bbac68b`](https://github.com/twmb/franz-go/commit/bbac68b) RecordFormatter: support %a; formatter&reader: support 'bool'
809
810#### kadm
811
812- [`d3ee144`](https://github.com/twmb/franz-go/commit/d3ee144) kadm: add LeaveGroup api
813- [`7b8d404`](https://github.com/twmb/franz-go/commit/7b8d404) kadm: ListOffsetsAfterMill(future) should return end offsets
814
815v1.8.0
816===
817
818This feature release adds one new API in kgo, one new field on `kgo.Record`,
819and stabilizes `kversion.V3_3_0()`. There is also one minor bug fix:
820`Record.TimestampType()` previously did not correctly return 1 to indicate
821broker generated timestamp types.
822
823This release improves the behavior of the client if the entire cluster becomes
824unreachable. Previously, the client may not have been able to reload metadata
825to discover new endpoints; now, the client periodically tries a seed broker to
826issue randomly-routable requests to. This release also makes a few more
827internal errors retryable, which can help reduce some request failures.
828
829A few docs have been improved. Notably, the UniformBytesPartitioner contains a
830new line describing that it may have poor interaction with lingering. See the
831doc comment for more details as well as the linked Kafka issue. This library
832does not use lingering by default.
833
834- [`f35ef66`](https://github.com/twmb/franz-go/commit/f35ef66) make `errUnknown{Controller,Coordinator}` retryable, improve error wording
835- [`750bf54`](https://github.com/twmb/franz-go/commit/750bf54) kversion: stabilize v3.3
836- [`3e02574`](https://github.com/twmb/franz-go/commit/3e02574) kgo: occasionally use seed brokers when choosing "random" brokers
837- [`3ebd775`](https://github.com/twmb/franz-go/commit/3ebd775) UniformBytesPartitioner: note sub-optimal batches with lingering
838- [PR #206](https://github.com/twmb/franz-go/pull/206) **bugfix** Record.TimestampType function to correctly return 1 for broker-set timestamps (thanks [@JacobSMoller](https://github.com/JacobSMoller))
839- [PR #201](https://github.com/twmb/franz-go/pull/201) **feature** add Context field to Record to enable more end-user instrumentation (thanks [@yianni](https://github.com/yianni))
840- [PR #197](https://github.com/twmb/franz-go/pull/197) **feature** add ValidateOpts to validate options before client initialization (thanks [@dwagin](https://github.com/dwagin))
841
842v1.7.1
843===
844
845This release fixes two bugs, one that is obscure and unlikely to be ran into in
846most use cases, and one that is not as obsure and may be ran into in a racy
847scenario. It is recommended to upgrade.
848
849[`3191842`][171a] fixes a bug that could eventually lead to a completely
850stalled consumer. The problem can happen whenever an internal "consumer
851session" is stopped and restarted -- which happens on most rebalances, and
852happens whenever a partition moves from one broker to another. This logic race
853required no active fetches to be in flight nor buffered, and required a fetch
854to _just about_ be issued.
855
856[`0ca6478`][171b] fixes a complicated bug that could result in a panic. It
857requires the following:
858
859* using a single client to produce and consume to the
860* consuming from that topic first
861* producing to that topic after the first consume
862* the metadata load that is triggered from the produce fails with partition errors
863* the metadata load retry moves a previously-errored partition from one broker to another
864
865Any deviation from this sequence of events would not result in a panic. If the
866final step did not move the partition between brokers, the client would still
867be internally problematic, but there would be no visible problem (the partition
868would be produced to two brokers, the produce to the wrong broker would fail
869while the correct broker would succeed),
870
871[171a]: https://github.com/twmb/franz-go/commit/3191842a81033342e8d37a529bd0a1b3d190fd9f
872[171b]: https://github.com/twmb/franz-go/commit/0ca6478600c632deed4c7d65c13c3459d19071bd
873
874## Relevant commits
875
876- [`0ca6478`](https://github.com/twmb/franz-go/commit/0ca6478) kgo: avoid pointer reuse in metadata across producers & consumers
877- [`3191842`](https://github.com/twmb/franz-go/commit/3191842) consumer: bugfix fetch concurrency loop
878- [`5f24fae`](https://github.com/twmb/franz-go/commit/5f24fae) kgo: fix an incorrect log line, add another log line
879
880v1.7.0
881===
882
883This feature release adds a few new APIs to expose more information in the
884client or make some things simpler. There are also two minor bug fixes:
885manually committing a topic with no partitions would previously result in an
886error, and we previously did not properly load the partition leader epochs for
887a brief period of time when the client started which prevented truncation
888detection. See the commits below for more details.
889
890- [`3a229d9`](https://github.com/twmb/franz-go/commit/3a229d9) kgo.Fetches: add Err0
891- [`5dd3321`](https://github.com/twmb/franz-go/commit/5dd3321) kgo.EpochOffset: export Less, add docs
892- [`2b38ec5`](https://github.com/twmb/franz-go/commit/2b38ec5) kgo: add CloseAllowingRebalance
893- [`ac2f97b`](https://github.com/twmb/franz-go/commit/ac2f97b) kgo.Client: add GroupMetadata, ProducerID, PartitionLeader functions
894- [`9497cf3`](https://github.com/twmb/franz-go/commit/9497cf3) **minor bugfix** kgo client.go: bugfix -1 leader epoch for a few minutes
895- [`11e3277`](https://github.com/twmb/franz-go/commit/11e3277) **bugfix** consumer groups: do not commit empty topics
896- [`52126de`](https://github.com/twmb/franz-go/commit/52126de) kgo: add optional interface to close sasls on client close
897- [`#193`](https://github.com/twmb/franz-go/pull/193) **minor logging bugfix** Use correct error in debug log for request failure (thanks [@keitwb](https://github.com/keitwb))
898- [`#191`](https://github.com/twmb/franz-go/pull/191) Add Fetches.NumRecords(), Fetches.Empty() (thanks [@un000](https://github.com/un000))
899- [`669a761`](https://github.com/twmb/franz-go/commit/669a761) kerr: add two new errors
900
901Also recently and newly released:
902
903* [pkg/kmsg v1.2.0](https://github.com/twmb/franz-go/releases/tag/pkg%2Fkmsg%2Fv1.2.0), adding protocol support for new incoming KIPs.
904* [pkg/kadm v1.2.1](https://github.com/twmb/franz-go/releases/tag/pkg%2Fkadm%2Fv1.2.1), removing sorting from metadata response replica fields (v1.2 added new helpers)
905* [pkg/sasl/kerberos v1.1.0](https://github.com/twmb/franz-go/releases/tag/pkg%2Fsasl%2Fkerberos%2Fv1.1.0), making it easier to persist and shutdown the Kerberos client
906
907v1.6.0
908===
909
910This release contains two new APIs, a minor bugfix, and other notable
911improvements and stabilizations.
912
913The [kadm][160kadmdoc] package is now stabilized as a separate,
914[v1.0 module][160kadmtag]. This module will remain separate from the top level
915franz-go so that it can have independent minor releases, and has more freedom
916to have major version bumps if necessary. I took stabilization as an
917opportunity to break & fix some infrequently used APIs; this can be seen in the
918commit linked below.
919
920I have also created a new package for the schema registry, [sr][160srdoc]. This
921will likely live as a separate module for similar reasons to kmsg and kadm: to
922have more freedom to make major-version-changing breaking changes if Confluent
923decides to change some of their HTTP API. I expect to stabilize sr once I know
924of at least one use case that helps double check the API is alright.
925
926[160kadmdoc]: https://pkg.go.dev/github.com/twmb/franz-go/pkg/kadm
927[160kadmtag]: https://github.com/twmb/franz-go/releases/tag/pkg%2Fkadm%2Fv1.0.0
928[160srdoc]: https://pkg.go.dev/github.com/twmb/franz-go/pkg/sr
929
930## Bugfix
931
932Previously, if you consumed a topic that had record timestamps generated by the
933broker per `LogAppendTime`, this client would not set the timestamp properly
934when consumed. We now set the timestamp properly.
935
936## Improvements
937
938The client now allows the user to set a timestamp when producing: if a record's
939timestamp is non-zero, the client will not override it.
940
941The default partitioner has been changed to the new `UniformBytesPartitioner`.
942This is an improvement on the previous `StickyKeyPartitioner` in a few ways and
943can be read about in [KIP-794][KIP-794]. The implementation in this client is
944slightly different from KIP-794 in ways that are documented on the
945`UniformBytesPartitioner` itself. This new partitioner can result in more
946balanced load over a long period of producing to your brokers. Note: the new
947default partitioner partitions keys the _same_ as the old default partitioner,
948so this is not a behavior change.
949
950## Features
951
952* `UniformBytesPartitioner` exists, which can be used for more balanced
953 producing over long time periods. The default batch size switch is 64KiB, and
954you may want to tune this up or down if you produce in high or low throughput.
955
956* `kversion.V3_2_0()` now officially exists, and kversion correctly detects
957 v3.2 brokers.
958
959* kmsg has a new `UnsafeReadFrom` interface to help reduce garbage in some
960 advanced scenarios (requires kmsg@v1.1.0).
961
962[KIP-794]: https://wiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner
963
964## Relevant commits
965
966- [`b279658`](https://github.com/twmb/franz-go/commit/b279658) kadm: break APIs for 1.0
967- [`a23a076`](https://github.com/twmb/franz-go/commit/a23a076) **bugfix** source: properly set Timestamp for LogAppendTime records
968- [`653010d`](https://github.com/twmb/franz-go/commit/653010d) **improvement** producer: allow the user to set Timestamp
969- [`d53c0fe`](https://github.com/twmb/franz-go/commit/d53c0fe) **feature** kmsg: add new UnsafeReadFrom interface
970- [`5536ec1`](https://github.com/twmb/franz-go/commit/5536ec1) **feature** kversion: add `V3_2_0`
971- [`0f65bb1`](https://github.com/twmb/franz-go/commit/0f65bb1) **feature** kgo: expose UniformBytesPartitioner
972- [`82af4a1`](https://github.com/twmb/franz-go/commit/82af4a1) and followup commits: add package sr, providing a client for the schema registry
973
974v1.5.3
975===
976
977This bugfix release fixes a problem I noticed when reading Kafka source code
978last week. KIP-98 never specified what a sequence number wrapping meant when
979producing with idempotency enabled / with transactions, so I implemented this
980to wrap to negative. As it turns out, there is validation in the Kafka code
981that the sequence number must wrap to 0.
982
983Thus, using this client, if an application produced `2**31` records to a single
984partition with idempotent production or with transactions without restarting,
985this client would enter a fatal state.
986
987I'm not sure how common this use case is considering this bug has not been
988reported in 2 years, but this patch release is to ensure people do not
989encounter this bug after this point.
990
991I've tested this by producing >2.147bil records to a single-partition Kafka topic
992before this patch, which eventually results in `INVALID_RECORD` errors, and
993then producing >2.2 records to a single-partition Kafka topic after this
994patch. The patch allows the client to produce indefinitely.
995
996This release also includes a few commits that eliminate two external
997dependencies, and contains a patch in kadm to avoid a panic on nil input to a
998function.
999
1000- [`4e2fa3f`](https://github.com/twmb/franz-go/commit/4e2fa3f) sticky: vendor go-rbtree, drop external dep
1001- [`6c0756d`](https://github.com/twmb/franz-go/commit/6c0756d) drop go-cmp dep
1002- [`27880b4`](https://github.com/twmb/franz-go/commit/27880b4) **bugfix** sink: sequence number wrapping for EOS
1003- [`afc9017`](https://github.com/twmb/franz-go/commit/afc9017) kadm.CalculateGroupLag: avoid panic if input maps are nil
1004
1005v1.5.2
1006===
1007
1008This tiny release contains another fix for `RecordReader`, specifically for
1009regex parsing, and changes the behavior of `MarkCommitRecords` to its original
1010behavior.
1011
1012* For `RecordReader`, the regex type was not properly registered everywhere
1013 internally. This resulted in `%v{re#...#}\n` trying to parse a newline as
1014part of the regex, rather than as a delimiter. This is fixed and a test case
1015added. Note that this feature is niche _and_ new, so this is a very minor bug.
1016
1017* For `MarkCommitRecords`, v1.3.3 changed the behavior to allow rewinds. This
1018 is difficult to reason about and can result in bugs. More likely, you just
1019want to periodically mark to move forward; the behavior now is more in line
1020with what people expect.
1021
1022- [`ff5a3ed`](https://github.com/twmb/franz-go/commit/ff5a3ed) MarkCommitRecords: forbid rewinds
1023- [`41284b3`](https://github.com/twmb/franz-go/commit/41284b3) RecordReader: fix regex reading even more
1024
1025v1.5.1
1026===
1027
1028This release fixes a minor bug in `RecordReader`, and has a behavior change for
1029polling records (and a corresponding tiny new helper API for this).
1030
1031For the bugfix, `RecordReader` did not always properly return
1032`io.ErrUnexpectedEOF`. We now return it more properly and add tests for the
1033missing edge cases.
1034
1035For the behavior change, we now inject an error into `PollFetches` and
1036`PollRecords` if the user context is canceled. Previously, we would just quit
1037the poll and return. This change introduces a new corresponding function,
1038`Fetches.Err() error`. The thought here is that now we have more injected
1039errors, so `Fetches.Err` can make it easier to check various errors.
1040
1041- [`cbc8962`](https://github.com/twmb/franz-go/commit/cbc8962) **behavior change** Poll{Records,Fetches}: inject an error for canceled context
1042- [#163](https://github.com/twmb/franz-go/pull/163) docs: updates StickyBalancer godoc (thanks [@Streppel](https://github.com/Streppel))
1043- [`2018d20`](https://github.com/twmb/franz-go/commit/2018d20) **bugfix** RecordReader: properly return `io.ErrUnexpectedEOF`
1044- [#161](https://github.com/twmb/franz-go/pull/161) examples: avoid duplicated module name (thanks [@robsonpeixoto](https://github.com/robsonpeixoto))
1045
1046v1.5.0
1047===
1048
1049This release adds a few new APIs, has a few small behavior changes, and has one
1050"breaking" change.
1051
1052## Breaking changes
1053
1054The `kerberos` package is now a dedicated separate module. Rather than
1055requiring a major version bump, since this fix is entirely at the module level
1056for an almost entirely unused package, I figured it is _okayish_ to technically
1057break compatibility for the few usages of this package, when the fix can be
1058done entirely when `go get`ing.
1059
1060The [gokrb5](https://github.com/jcmturner/gokrb5) library, basically the only
1061library in the Go ecosystem that implements Kerberos, has a slightly [broken
1062license](https://github.com/jcmturner/gokrb5/issues/461). Organizations that
1063are sensitive to this were required to not use franz-go even if they did not
1064use Kerberos because franz-go pulls in a dependency on gokrb5.
1065
1066Now, with `kerberos` being a distinct and separate module, depending on
1067franz-go only will _not_ cause an indirect dependency on gokrb5.
1068
1069If your upgrade is broken by this change, run:
1070
1071```go
1072go get github.com/twmb/franz-go/pkg/sasl/kerberos@v1.0.0
1073go get github.com/twmb/franz-go@v1.5.0
1074```
1075
1076## Behavior changes
1077
1078* `UnknownTopicRetries` now allows -1 to signal disabling the option (meaning
1079 unlimited retries, rather than no retries). This follows the convention of
1080other options where -1 disables limits.
1081
1082## Improvements
1083
1084* Waiting for unknown topics while producing now takes into account both the
1085 produce context and aborting. Previously, the record context was only taken
1086into account _after_ a topic was loaded. The same is true for aborting buffered
1087records: previously, abort would hang until a topic was loaded.
1088
1089* New APIs are added to kmsg to deprecate the previous `Into` functions. The
1090 `Into` functions still exist and will not be removed until kadm is stabilized
1091(see [#141](https://github.com/twmb/franz-go/issues/141)).
1092
1093## Features
1094
1095* `ConsumeResetOffset` is now clearer, you can now use `NoResetOffset` with
1096 start _or_ end _or_ exact offsets, and there is now the very useful
1097`Offset.AfterMilli` function. Previously, `NoResetOffset` only allowed starting
1098consuming at the start and it was not obvious why. We keep the previous
1099default-to-start behavior, but we now allow modifying it. As well, `AfterMilli`
1100can be used to largely replace `AtEnd`. Odds are, you want to consume all
1101records after your program starts _even if_ new partitions are added to a
1102topic. Previously, if you added a partition to a topic, `AtEnd` would miss
1103records that were produced until the client refreshed metadata and discovered
1104the partition. Because of this, you were safer using `AtStart`, but this
1105unnecessarily forced you to consume everything on program start.
1106
1107* Custom group balancers can now return errors, you can now intercept commits
1108 to attach metadata, and you can now intercept offset fetches to read
1109metadata. Previously, none of this was possible. I considered metadata a bit
1110of a niche feature, but accessing it (as well as returning errors when
1111balancing) is required if you want to implement streams. New APIs now exist to
1112support the more advanced behavior: `PreCommitFnContext`, `OnOffsetsFetched`,
1113and `GroupMemberBalancerOrError`. As well, `BalancePlan.AsMemberIDMap` now
1114exists to provide access to a plan's underlying plan map. This did not exist
1115previously because I wanted to keep the type opaque for potential future
1116changes, but the odds of this are low and we can attempt forward compatibility
1117when the time arises.
1118
1119* `RecordReader` now supports regular expressions for text values.
1120
1121## Relevant commits
1122
1123- [`a2cbbf8`](https://github.com/twmb/franz-go/commit/a2cbbf8) go.{mod,sum}: go get -u ./...; go mod tidy
1124- [`ce7a84f`](https://github.com/twmb/franz-go/commit/ce7a84f) kerberos: split into dedicated module, p1
1125- [`e8e5c82`](https://github.com/twmb/franz-go/commit/e8e5c82) and [`744a60e`](https://github.com/twmb/franz-go/commit/744a60e) kgo: improve ConsumeResetOffset, NoResetOffset, add Offset.AfterMilli
1126- [`78fff0f`](https://github.com/twmb/franz-go/commit/78fff0f) and [`e8e5117`](https://github.com/twmb/franz-go/commit/e8e5117) and [`b457742`](https://github.com/twmb/franz-go/commit/b457742): add GroupMemberBalancerOrError
1127- [`b5256c7`](https://github.com/twmb/franz-go/commit/b5256c7) kadm: fix long standing poor API (Into fns)
1128- [`8148c55`](https://github.com/twmb/franz-go/commit/8148c55) BalancePlan: add AsMemberIDMap
1129- [`113a2c0`](https://github.com/twmb/franz-go/commit/113a2c0) add OnOffsetsFetched function to allow inspecting commit metadata
1130- [`0a4f2ec`](https://github.com/twmb/franz-go/commit/0a4f2ec) and [`cba9e26`](https://github.com/twmb/franz-go/commit/cba9e26) kgo: add PreCommitFnContext, enabling pre-commit interceptors for metadata
1131- [`42e5b57`](https://github.com/twmb/franz-go/commit/42e5b57) producer: allow a canceled context & aborting to quit unknown wait
1132- [`96d647a`](https://github.com/twmb/franz-go/commit/96d647a) UnknownTopicRetries: allow -1 to disable the option
1133- [`001c6d3`](https://github.com/twmb/franz-go/commit/001c6d3) RecordReader: support regular expressions for text values
1134
1135v1.4.2
1136===
1137
1138This release fixes a potential incremental fetch session spin loop /
1139undesirable behavior. This was not reported, but can happen if you use many
1140clients against your cluster.
1141
1142Previously, if a broker advertised that it supported consumer fetch sessions
1143but did not actually create any and returned "0" to signify no session was
1144created, the client would accept that 0 as a new fetch session. If the fetch
1145response returned no data and thus made no forward progress, the next fetch
1146request would include no partitions (believing a fetch session was created),
1147and the broker would again reply immediately with no data and no fetch session.
1148This would loop. Now, if the broker indicates no fetch session was created, we
1149immediately stop trying to create new fetch sessions and never try again.
1150
1151In practice, fetch sessions are rejected if and replied to with 0 if a new one
1152cannot be created. The default fetch session cache in Kafka is 1,000. If you
1153have more than 1,000 active clients (where brokers count as clients against
1154other brokers), you are at risk of this bug.
1155
1156This bug would manifest in clearly visible ways: higher cpu, no forward
1157progress while consuming. If you have not seen these, you have not experienced
1158the bug. However, it is recommended that all users upgrade to avoid it.
1159
1160This has two followup fixes to [`83b0a32`][83b0a32], one which fixes behavior
1161that broke `EndBeginTxnSafe`, and one which mirrors some of the logic
1162supporting `EndBeginTxnUnsafe` into `EndTransaction` itself. This also fixes a
1163very rare data race that _realistically_ would result in a new connection being
1164killed immediately (since at the CPU, reads/writes of pointers is atomic).
1165
1166- [`2faf459`](https://github.com/twmb/franz-go/commit/2faf459) **bugfix** broker: fix rare data race
1167- [`8f7c8cd`](https://github.com/twmb/franz-go/commit/8f7c8cd) **bugfix** EndBeginTxnUnsafe: partially back out of [`83b0a32`][83b0a32]
1168- [`85a680e`](https://github.com/twmb/franz-go/commit/85a680e) **bugfix** consuming: do not continually try to create fetch sessions
1169- [`2decd27`](https://github.com/twmb/franz-go/commit/2decd27) **bugfix** EndTransaction: mirror EndBeginTxnUnsafe logic
1170
1171[83b0a32]: https://github.com/twmb/franz-go/commit/83b0a32
1172
1173v1.4.1
1174===
1175
1176This release pins kmsg to its newly stable version v1.0.0, fixing a compilation
1177issue for anybody doing `go get -u`.
1178
1179The kmsg package was previously unversioned because Kafka sometimes changes the
1180protocol in such a way that breaks the API as chosen in kmsg (plain types, not
1181objects for everything). The most recent change in kmsg to match a recent type
1182rename in Kafka broke kgo because kgo depended on the old name. Again, this was
1183not pinned because franz-go did not depend on a specific version of kmsg. To
1184prevent this issue from happening again, we now pin to a stable kmsg version.
1185
1186There are also two small bugfixes and a few improvements. Previously,
1187`ProducerFenced` was marked as retriable, which could result in the client
1188internally entering a fatal state that the user was unaware of. This should now
1189be bubbled up. As well, there were a few subtle issues with `EndBeginTxnUnsafe`
1190that have been addressed.
1191
1192Notable commits & PRs:
1193
1194- [`83b0a32`](https://github.com/twmb/franz-go/commit/83b0a32) **bugfix** kgo: EndAndBeginTransaction w/ EndBeginTxnUnsafe: fix three issues
1195- [`bd1d43d`](https://github.com/twmb/franz-go/commit/bd1d43d) sink: small AddPartitionsToTxn improvements
1196- [`65ca0bd`](https://github.com/twmb/franz-go/commit/65ca0bd) **bugfix** kerr: ProducerFenced is not retriable
1197- [PR #148](https://github.com/twmb/franz-go/pull/148) lower `FetchMaxPartitionBytes` to 1MiB to be in line with the Kafka default (thanks [@jcsp](https://github.com/jcsp))
1198- [`806cf53`](https://github.com/twmb/franz-go/commit/806cf53) **feature** kmsg: add TextMarshaler/TextUnmarshaler to enums
1199- [`49f678d`](https://github.com/twmb/franz-go/commit/49f678d) update deps, pulling in klauspost/compress v1.15.1 which makes zstd encoding & decoding stateless
1200
1201v1.4.0
1202===
1203
1204This release adds a lot of new features and changes a few internal behaviors.
1205The new features have been tested, but it is possible that a bug slipped by—if
1206you see one, please open an issue and the bug can be fixed promptly.
1207
1208## Behavior changes
1209
1210* **Promises are now serialized**. Previously, promises were called at the end
1211 of handling produce requests. As well, errors that caused records to fail
1212independent of producing could fail whenever. Now, all promises are called in
1213one loop. Benchmarking showed that concurrent promises did not really help,
1214even in cases where the promises could be concurrent. As well, my guess is that
1215most people serialize promises, resulting in more complicated logic punted to
1216the users. Now with serializing promises, user code can be simpler.
1217
1218* The default `MetadataMinAge` has been lowered from 5s to 2.5s. Metadata
1219refreshes internally on retryable errors, 2.5s helps fail records for
1220non-existing topics quicker. Related, for sharded requests, we now cache topic
1221& partition metadata for the `MetadataMinAge`. This mostly benefits
1222`ListOffsets`, where usually a person may list both the start and end back to
1223back. We cannot cache indefinitely because a user may add partitions outside
1224this client, but 2.5s is still helpful especially for how infrequently sharded
1225requests are issued.
1226
1227* Group leaders now track topics that the leader is not interested in
1228 consuming. Previously, if leader A consumed only topic foo and member B only
1229bar, then leader A would not notice if partitions were added to bar. Now, the
1230leader tracks bar. This behavior change only affects groups where the members
1231consume non-overlapping topics.
1232
1233* Group joins & leaves now include a reason, as per KIP-800. This will be
1234 useful when Kafka 3.2 is released.
1235
1236* Transactions no longer log `CONCURRENT_TRANSACTIONS` errors at the info
1237 level. This was a noisy log that meant nothing and was non-actionable. We
1238still track this at the debug level.
1239
1240## Features
1241
1242A few new APIs and options have been added. These will be described shortly
1243here, and the commits are linked below.
1244
1245* `ConcurrentTransactionsBackoff`: a new option that allows configuring the
1246 backoff when starting a transaction runs into the `CONCURRENT_TRANSACTIONS`
1247error. Changing the backoff can decrease latency if Kafka is fast, but can
1248increase load on the cluster.
1249
1250* `MaxProduceRequestsInflightPerBroker`: a new option that allows changing the
1251 max inflight produce requests per broker _if_ you disable idempotency.
1252Idempotency has an upper bound of 5 requests; by default, disabling idempotency
1253sets the max inflight to 1.
1254
1255* `UnknownTopicRetries`: a new option that sets how many times a metadata load
1256 for a topic can return `UNKNOWN_TOPIC_OR_PARTITION` before all records
1257buffered for the topic are failed. As well, we now use this option more widely:
1258if a topic is loaded successfully and then later repeatedly experiences these
1259errors, records will be failed. Previously, this limit was internal and was
1260only applied before the topic was loaded successfully once.
1261
1262* `NoResetOffset`: a new special offset that can be used with
1263 `ConsumeResetOffset` to trigger the client to enter a fatal state if
1264`OffsetOutOfRange` is encountered.
1265
1266* `Client.PurgeTopicsFromClient`: a new API that allows for completely removing
1267a topic from the client. This can help if you regex consume and delete a topic,
1268or if you produce to random topics and then stop producing to some of them.
1269
1270* `Client.AddConsumeTopics`: a new API that enables you to consume from topics
1271 that you did not initially configure. This enables you to add more topics to
1272consume from without restarting the client; this works both both direct
1273consumers and group consumers.
1274
1275* `Client.TryProduce`: a new API that is a truly non-blocking produce. If the
1276 client has the maximum amount of records buffered, this function will
1277immediately fail a new promise with `ErrMaxBuffered`.
1278
1279* `Client.ForceMetadataRefresh`: a new API that allows you to manually trigger
1280a metadata refresh. This can be useful if you added partitions to a topic and
1281want to trigger a metadata refresh to load those partitions sooner than the
1282default `MetadataMaxAge` refresh interval.
1283
1284* `Client.EndAndBeginTransaction`: a new API that can be used to have higher
1285 throughput when producing transactionally. This API requires care; if you use
1286it, read the documentation for what it provides and any downsides.
1287
1288* `BlockRebalancesOnPoll` and `Client.AllowRebalance`: a new option and
1289 corresponding required API that allows for easier reasoning about when
1290rebalances can happen. This option can be greatly beneficial to users for
1291simplifying code, but has a risk around taking so long that your group member
1292is booted from the group. Two examples were added using these options.
1293
1294* `kversion.V3_1_0`: the kversion package now officially detects v3.1 and has
1295 an API for it.
1296
1297## Relevant commits
1298
1299- [PR #137](https://github.com/twmb/franz-go/pull/137) and [`c3fc8e0`](https://github.com/twmb/franz-go/commit/c3fc8e0): add two more goroutine per consumer examples (thanks [@JacobSMoller](https://github.com/JacobSMoller))
1300- [`cffbee7`](https://github.com/twmb/franz-go/commit/cffbee7) consumer: add BlockRebalancesOnPoll option, AllowRebalance (commit accidentally pluralized)
1301- [`39af436`](https://github.com/twmb/franz-go/commit/39af436) docs: add metrics-and-logging.md
1302- [`83dfa9d`](https://github.com/twmb/franz-go/commit/83dfa9d) client: add EndAndBeginTransaction
1303- [`d11066f`](https://github.com/twmb/franz-go/commit/d11066f) committing: internally retry on some errors when cooperative
1304- [`31f3f5f`](https://github.com/twmb/franz-go/commit/31f3f5f) producer: serialize promises
1305- [`e3ef142`](https://github.com/twmb/franz-go/commit/e3ef142) txn: move concurrent transactions log to debug level
1306- [`10ee8dd`](https://github.com/twmb/franz-go/commit/10ee8dd) group consuming: add reasons to JoinGroup, LeaveGroup per KIP-800
1307- [`0bfaf64`](https://github.com/twmb/franz-go/commit/0bfaf64) consumer group: track topics that the leader is not interested in
1308- [`e8495bb`](https://github.com/twmb/franz-go/commit/e8495bb) client: add ForceMetadataRefresh
1309- [`c763c9b`](https://github.com/twmb/franz-go/commit/c763c9b) consuming: add NoResetOffset
1310- [`4e0e1d7`](https://github.com/twmb/franz-go/commit/4e0e1d7) config: add UnknownTopicRetries option, use more widely
1311- [`7f58a97`](https://github.com/twmb/franz-go/commit/7f58a97) config: lower default MetadataMinAge to 2.5s
1312- [`e7bd28f`](https://github.com/twmb/franz-go/commit/e7bd28f) Client,GroupTransactSession: add TryProduce
1313- [`2a2cf66`](https://github.com/twmb/franz-go/commit/2a2cf66) consumer: add AddConsumeTopics
1314- [`d178e26`](https://github.com/twmb/franz-go/commit/d178e26) client: add PurgeTopicsFromClient
1315- [`336d2c9`](https://github.com/twmb/franz-go/commit/336d2c9) kgo: add ConcurrentTransactionsBackoff, MaxProduceRequestsInflightPerBroker
1316- [`fb04711`](https://github.com/twmb/franz-go/commit/fb04711) kversion: cut v3.1
1317
1318v1.3.5
1319===
1320
1321This patch release fixes a panic in `GroupTransactSession.End` and has three
1322behavior changes that are beneficial to users. The panic was introduced in
1323v1.3.0; if using a `GroupTransactSession`, it is recommended you upgrade to
1324this release.
1325
1326The next release aims to be v1.4.0, this release is a small one to address a
1327few issues before the much larger and feature filled v1.4 release.
1328
1329- [`010e8e1`](https://github.com/twmb/franz-go/commit/010e8e1) txn: fix panic in GroupTransactSession.End
1330- [`f9cd625`](https://github.com/twmb/franz-go/commit/f9cd625) consuming: handle exact offset consuming better
1331- [`2ab1978`](https://github.com/twmb/franz-go/commit/2ab1978) EndTransaction: return nil rather than an error if not in a transaction
1332- [`96bfe52`](https://github.com/twmb/franz-go/commit/96bfe52) broker: remove 5s minimum for sasl session lifetime
1333
1334v1.3.4
1335===
1336
1337This small patch release fixes a problem with
1338[`4f2e7fe3`](https://github.com/twmb/franz-go/commit/4f2e7fe3) which was meant
1339to address [#98](https://github.com/twmb/franz-go/issues/98). The fix was not
1340complete in that the fix would only trigger if a group member had partitions
1341added to it. We now rearrange the logic such that it occurs always. This bug
1342was found while making a change in the code in support of a new feature in the
1343v1.4 branch; this bug was not encountered in production.
1344
1345This also bumps the franz-go/pkg/kmsg dependency so that `JoinGroup.Reason` is
1346properly tagged as v9+.
1347
1348The next release will be v1.4, which is nearly ready to be merged into this
1349branch and tagged. Follow issue #135 for more details.
1350
1351- [`02560c7`](https://github.com/twmb/franz-go/commit/02560c7) consumer group: bugfix fetch offsets spanning rebalance
1352
1353v1.3.3
1354===
1355
1356This patch release contains two minor bug fixes and a few small behavior
1357changes. The upcoming v1.4 release will contain more changes to lint the entire
1358codebase and will have a few new options as features to configure the client
1359internals. There are a few features in development yet that I would like to
1360complete before tagging the next minor release.
1361
1362## Bug fixes
1363
1364Repeated integration testing resulted in a rare data race, and one other bug
1365was found by linting. For the race, if a group was left _or_ heartbeating
1366stopped _before_ offset fetching finished, then there would be a concurrent
1367double write to an error variable: one write would try to write an error from
1368the request being cut (which would be `context.Canceled`), and the other write
1369would write the same error, but directly from `ctx.Err`. Since both of these
1370are the same type pointer and data pointer, it is unlikely this race would
1371result in anything if it was ever encountered, and encountering it would be
1372rare.
1373
1374For the second bug, after this prior one, I wondered if any linter would have
1375caught this bug (the answer is no). However, in the process of heavily linting
1376the code base, a separate bug was found. This bug has **no impact**, but it is
1377good to fix. If you previously consumed and specified _exact_ offsets to
1378consume from, the internal `ListOffsets` request would use that offset as a
1379timestamp, rather than using -1 as I meant to internally. The whole point is
1380just to load the partition, so using a random number for a timestmap is just as
1381good as using -1, but we may as well use -1 to be proper.
1382
1383## Behavior changes
1384
1385Previously when producing, the buffer draining goroutine would sleep for 50ms
1386whenever it started. This was done to give high throughput producers a chance
1387to produce more records within the first batch, rather than the loop being so
1388fast that one record is in the first batch (and more in the others). This 50ms
1389sleep is a huge penalty to oneshot producers (producing one message at a time,
1390synchronously) and also was a slight penalty to high throughput producers
1391whenever the drain loop quit and needed to be resumed. We now eliminate this
1392sleep. This may result in more smaller batches, but definitely helps oneshot
1393producers. A linger can be used to avoid small batches, if needed.
1394
1395Previously, due to MS Azure improperly replying to produce requests when acks
1396were zero, a discard goroutine was added to drain the produce connection if the
1397client was configured to produce with no acks. Logic has been added to quit
1398this goroutine if nothing is read on that goroutine for 3x the connection
1399timeout overhead, which is well enough time that a response should be received
1400if the broker is ever going to send one.
1401
1402Previously, `MarkCommitRecords` would forbid rewinds and only allow advancing
1403offsets. The code now allows rewinds if you mark an early offset after you have
1404already marked a later offset. This brings the behavior in line with the
1405current `CommitOffsets`.
1406
1407Previously, if the client encountered `CONCURRENT_TRANSACTIONS` during transactions,
1408it would sleep for 100ms and then retry the relevant request. This sleep has been
1409dropped to 20ms, which should help latency when transacting quickly. The v1.4
1410release will allow this number to be configured with a new option.
1411
1412## Additions
1413
1414KIP-784 and KIP-814 are now supported (unreleased yet in Kafka). Support for
1415KIP-814 required bumping the franz-go's kmsg dep. Internally, only KIP-814
1416affects client behavior, but since this is unrelased, it is not changing any
1417behavior.
1418
1419## Relevant commits
1420
1421- [`b39ca31`](https://github.com/twmb/franz-go/commit/b39ca31) fetchOffsets: fix data race
1422- [`4156e9f`](https://github.com/twmb/franz-go/commit/4156e9f) kgo: fix one bug found by linting
1423- [`72760bf..ad991d8`](https://github.com/twmb/franz-go/compare/72760bf..ad991d8) kmsg, kversion, kgo: support KIP-814 (SkipAssignment in JoinGroupResponse)
1424- [`db9017a`](https://github.com/twmb/franz-go/commit/db9017a) broker: allow the acks==0 producing discard goroutine to die
1425- [`eefb1f3`](https://github.com/twmb/franz-go/commit/eefb1f3) consuming: update docs & simplify; `MarkCommitRecords`: allow rewinds
1426- [`8808b94`](https://github.com/twmb/franz-go/commit/8808b94) sink: remove 50ms wait on new drain loops
1427- [`a13f918`](https://github.com/twmb/franz-go/commit/a13f918) kmsg & kversion: add support for KIP-784 (ErrorCode in DescribeLogDirs response)
1428- [PR #133](https://github.com/twmb/franz-go/pull/133) - lower concurrent transactions retry to 20ms; configurability will be in the next release (thanks [@eduard-netsajev](https://github.com/eduard-netsajev))
1429
1430v1.3.2
1431===
1432
1433This patch fixes a bug of unclear severity related to transactions. Credit goes
1434to [@eduard-netsajev](https://github.com/eduard-netsajev) for finding this long
1435standing problem.
1436
1437In Kafka, if you try to start a transaction too soon after finishing the
1438previous one, Kafka may not actually have internally finished the prior
1439transaction yet and can return a `CONCURRENT_TRANSACTIONS` error. To work
1440around this, clients are expected to retry when they see this error (even
1441though it is marked as not retriable).
1442
1443This client does that properly, but unfortunately did not bubble up any _non_
1444`CONCURRENT_TRANSACTIONS` errors.
1445
1446From the code, it _appears_ as if in the worst case, this could have meant that
1447transactions invisibly looked like they were working and being used when they
1448actually were not. However, it's likely that other errors would be noticed
1449internally, and it's possible that if you encountered problems, the entire ETL
1450pipeline would stall anyway.
1451
1452All told, it's not entirely clear what the ramifications for this bug are, and
1453it is recommended that if you use transactions, you should update immediately.
1454
1455- [PR #131](https://github.com/twmb/franz-go/pull/131) - txns: don't ignore error in doWithConcurrentTransactions
1456
1457v1.3.1
1458===
1459
1460This small patch release fixes a leaked-goroutine problem after closing a
1461consuming client, and adds one config validation to hopefully reduce confusion.
1462
1463For the bug, if a fetch was buffered when you were closing the client, an
1464internal goroutine would stay alive. In normal cases where you have one client
1465in your program and you close it on program shutdown, this leak is not really
1466important. If your program recreates consuming clients often and stays alive,
1467the leaked goroutine could eventually result in unexpected memory consumption.
1468Now, the client internally unbuffers all fetches at the end of `Close`, which
1469allows the previously-leaking goroutine to exit.
1470
1471For the config validation, using `ConsumePartitions` and `ConsumeTopics` with
1472the same topic in both options would silently ignore the topic in
1473`ConsumePartitions`. Now, the client will return an error that using the same
1474topic in both options is invalid.
1475
1476- [`ea11266`](https://github.com/twmb/franz-go/commit/ea11266) config: add duplicately specified topic validation
1477- [`bb581f4`](https://github.com/twmb/franz-go/commit/bb581f4) **bugfix** client: unbuffer fetches on Close to allow mangeFetchConcurrency to quit
1478
1479v1.3.0
1480===
1481
1482This release contains three new features, a few behavior changes, and one minor
1483bugfix.
1484
1485For features, you can now adjust fetched offsets before they are used (thanks
1486@michaelwilner!), you can now "ping" your cluster to see if the client can
1487connect at all, and you can now use `SetOffsets` when consuming partitions
1488manually. As a somewhat of a feature-ish, producing no longer requires a
1489context, instead if a context is nil, `context.Background` is used (this was
1490added to allow more laziness when writing small unimportant files).
1491
1492The transactional behavior change is important: the documentation changes are
1493worth reading, and it is worth using a 2.5+ cluster along with the
1494`RequireStableFetchOffsets` option if possible. The metadata leader epoch
1495rewinding behavior change allows the client to continue in the event of odd
1496cluster issues.
1497
1498In kadm, we now return individual per-partition errors if partitions are not
1499included in OffsetCommit responses. The generated code now has a few more enums
1500(thanks [@weeco](https://github.com/weeco)!)
1501
1502Lastly, as a small bugfix, `client.Close()` did not properly stop seed brokers.
1503A previous commit split seed brokers and non-seed brokers internally into two
1504fields but did not add broker shutdown on the now-split seed broker field.
1505
1506- [`e0b520c`](https://github.com/twmb/franz-go/commit/e0b520c) **behavior change** kadm: set per-partition errors on missing offsets in CommitOffsets
1507- [`32425df`](https://github.com/twmb/franz-go/commit/32425df) **feature** client: add Ping method
1508- [`a059901`](https://github.com/twmb/franz-go/commit/a059901) **behavior change** txns: sleep 200ms on commit, preventing rebalance / new commit
1509- [`12eaa1e`](https://github.com/twmb/franz-go/commit/12eaa1e) **behavior change** metadata: allow leader epoch rewinds after 5 tries
1510- [`029e655`](https://github.com/twmb/franz-go/commit/029e655) **feature-ish** Produce{,Sync}: default to context.Background if no ctx is provided
1511- [`eb2cec3`](https://github.com/twmb/franz-go/commit/eb2cec3) **bugfix** client: stop seed brokers on client.Close
1512- [`2eae20d`](https://github.com/twmb/franz-go/commit/2eae20d) **feature** consumer: allow SetOffsets for direct partition consuming
1513- [pr #120](https://github.com/twmb/franz-go/pull/120) **feature** Add groupopt to swizzle offset assignments before consumption (thanks [@michaelwilner](https://github.com/michaelwilner)!)
1514
1515v1.2.6
1516===
1517
1518This patch release contains a behavior change to immediate metadata triggers to
1519better support some loading error conditions and drops the default
1520MetadataMinAge from 10s to 5s. For more details, see commit
1521[`8325ba7`][8325ba7] or issue [114][issue114].
1522
1523This release also contains a bugfix for using preferred read replicas.
1524Previously, if a replica was not the leader of any partition being consumed,
1525the client would not internally save knowledge of that replica and it could not
1526be consumed from. This would result in the client ping-ponging trying to fetch
1527from the leader and then erroring when the leader indicates the client should
1528consume from a replica the client does not know about. This is no longer the
1529case.
1530
1531Lastly, this patch fixes the behavior of rotating through partitions while
1532consuming. Previously, the client rotated through partitions to evaluate for
1533consuming, but then saved those partitions to nested maps which had
1534non-deterministic (and, due to how Go ranges, mostly not-that-random) range
1535semantics. The client now always rotates through partitions and topics in the
1536order the partitions are evaluated. The old behavior could theoretically lead
1537to more starvation than necessary while consuming. The new behavior should have
1538no starvation, but really due to the randomness of the prior behavior, the new
1539semantics may be a wash. However, the new semantics are the original intended
1540semantics.
1541
1542- [`bc391a3`](https://github.com/twmb/franz-go/commit/bc391a3) **improvement** source: rotate through topics/partitions as we fetch
1543- [`6bbdaa2`](https://github.com/twmb/franz-go/commit/6bbdaa2) **bugfix** client: create a sink & source for all partition replicas
1544- [`8325ba7`][8325ba7] **behavior change** metadata: minor changes, & wait a bit when looping for now triggers
1545- [`b8b7bd1`](https://github.com/twmb/franz-go/commit/b8b7bd1) **behavior change** RecordFormatter: always use UTC
1546- [`6ab9044`](https://github.com/twmb/franz-go/commit/6ab9044) kadm: return defined errors for On functions
1547
1548[8325ba7]: https://github.com/twmb/franz-go/commit/8325ba7
1549[issue114]: https://github.com/twmb/franz-go/issues/114
1550
1551v1.2.5
1552===
1553
1554This small patch release fixes one non-impacting bug in SCRAM authentication,
1555and allows more errors to be abort&retryable rather than fatal while
1556transactionally committing.
1557
1558For SCRAM, this client did not implement the client-final-reply completely
1559correctly: this client replied with just the `client nonce`, not the `client
1560nonce + server nonce`. Technically this was not to spec, but no broker today
1561enforces this final nonce correctly. However, the client has been fixed so that
1562if someday brokers do start enforcing the nonce correctly, this client will be
1563ready.
1564
1565For transactional committing, we can handle a few extra errors while committing
1566without entering a fatal state. Previously, `ILLEGAL_GENERATION` was handled:
1567this meant that a rebalance began and completed before the client's commit went
1568through. In this case, we just aborted the transaction and continued
1569successfully. We can do this same thing for `REBALANCE_IN_PROGRESS`, which is
1570similar to the prior error, as well as for errors that result from client
1571request retry limits.
1572
1573The integration tests now no longer depend on `kcl`, meaning you can simply `go
1574test` in the `kgo` package to test against your local brokers. Seeds can be
1575provided by specifying a `KGO_SEEDS` environment variable, otherwise the
1576default is 127.0.0.1:9092.
1577
1578Lastly, a few bugs have been fixed in the not-yet-stable,
1579currently-separate-module kadm package. If you use that package, you may have
1580already pulled in these fixes.
1581
1582- [`17dfae8`](https://github.com/twmb/franz-go/commit/17dfae8) go.{mod,sum}: update deps
1583- [`3b34db0`](https://github.com/twmb/franz-go/commit/3b34db0) txn test: remove increasing-from-0 strictness when producing
1584- [`d0a27f3`](https://github.com/twmb/franz-go/commit/d0a27f3) testing: remove dependency on kcl
1585- [`8edf934`](https://github.com/twmb/franz-go/commit/8edf934) txn: allow more commit errors to just trigger abort
1586- [`03c58cb`](https://github.com/twmb/franz-go/commit/03c58cb) scram: use c-nonce s-nonce, not just c-nonce, in client-reply-final
1587- [`8f34083`](https://github.com/twmb/franz-go/commit/8f34083) consumer group: avoid regex log if no topics were added/skipped
1588
1589v1.2.4
1590===
1591
1592This patch release fixes handling forgotten topics in fetch sessions, and
1593allows disabling fetch sessions. Previously, if a topic or partition was paused
1594with fetch sessions active, the partition would continually be fetched and
1595skipped behind the scenes. As well, the client now allows disabling fetch
1596sessions entirely if they are not desired.
1597
1598This also fixes one bug in the `RecordReader`: using an empty layout would mean
1599that any `io.Reader` would result in infinite messages.
1600
1601Lastly, some log messages have been improved.
1602
1603- [`db90100`](https://github.com/twmb/franz-go/commit/db90100) consuming: improve error messages
1604- [`101d6bd`](https://github.com/twmb/franz-go/commit/101d6bd) consuming: log added/skipped when consuming by regex
1605- [`b6759bc`](https://github.com/twmb/franz-go/commit/b6759bc) **improvement** consumer: allow disabling fetch sessions with a config opt
1606- [`7cd959c`](https://github.com/twmb/franz-go/commit/7cd959c) **bugfix** source: use forgotten topics for sessions
1607- [`cfb4a7f`](https://github.com/twmb/franz-go/commit/cfb4a7f) **bugfix** kgo: error if RecordReader layout is empty
1608
1609v1.2.3
1610===
1611
1612This patch fixes the client not supporting brokers that start from large node
1613IDs. The client previously assumed that brokers started at node ID 0 or 1 and
1614went upwards from there. Starting at node 1000 would not always work.
1615
1616This also adds two very tiny new APIs, `FetchTopic.Each{Partition,Record}`,
1617which are so tiny they are not worth a minor release.
1618
1619More work on kadm has been done: the package now supports ACLs.
1620
1621SASL no longer has a hard coded 30s write timeout; instead, we use the
1622RequestTimeoutOverhead same as everything else. The default produce request
1623timeout has been dropped to 10s from 30s, which is still ~10x more time than
1624Kafka usually needs at the p99.9 level. This will help speed up producing to
1625hung brokers. As well, the default backoff has been changed to 250ms-2.5s
1626rather than 100ms-1s. The old backoff retried too quickly in all cases.
1627
1628Logs for assigned partitions have been improved, as have logs for inner
1629metadata request / fetch request failures.
1630
1631- [`07a38bc`](https://github.com/twmb/franz-go/commit/07a38bc) **bugfix** client: support non-zero/one node IDs
1632- [`3cbaa5f`](https://github.com/twmb/franz-go/commit/3cbaa5f) add more context to metadata reloads on inner partition errors
1633- [`1bc1156`](https://github.com/twmb/franz-go/commit/1bc1156) **feature** FetchTopic: add EachRecord, Records helper methods
1634- [`0779837`](https://github.com/twmb/franz-go/commit/0779837) consuming, group: improve logging, simplify code
1635- [`d378b32`](https://github.com/twmb/franz-go/commit/d378b32) config: edit comment for FetchMaxWait (about Java setting) (thanks [@dwagin](https://github.com/dwagin)!)
1636- [`df80a52`](https://github.com/twmb/franz-go/commit/df80a52) **behavior change** config: drop ProduceRequestTimeout to 10s, doc more
1637- [`6912cfe`](https://github.com/twmb/franz-go/commit/6912cfe) **behavior change** config: change default backoff from 100ms-1s to 250ms-2.5s
1638- [`c197efd`](https://github.com/twmb/franz-go/commit/c197efd) **behavior change** client: remove 30s default write timeout for SASL
1639- [`10ff785`](https://github.com/twmb/franz-go/commit/10ff785) metadata: drop timer based info log, reword
1640
1641v1.2.2
1642===
1643
1644This patch release fixes a bug with cooperative group consuming. If a
1645cooperative group member rebalances before offset fetching returns, then it
1646will not resume offset fetching partitions it kept across the rebalance. It
1647will only offset fetch new partitions.
1648
1649This release fixes that by tracking a bit of fetch state across rebalances for
1650cooperative consumers. See the embedded comments in the commit for more
1651details.
1652
1653- [`4f2e7fe`](https://github.com/twmb/franz-go/commit/4f2e7fe) **bugfix** consumer group: fix cooperative rebalancing losing offset fetches
1654
1655
1656v1.2.1
1657===
1658
1659This patch release fixes a panic that can occur in the following sequence of
1660events:
1661
16621) a LeaveGroup occurs (which always happens in Close)
16632) the cluster moves a partition from one broker to another
16643) a metadata refresh occurs and sees the partition has moved
1665
1666Whenever a client closes, `Close` calls `LeaveGroup` (even if not in a group),
1667and there is a small window of time before the metadata loop quits.
1668
1669- [`864526a`](https://github.com/twmb/franz-go/commit/864526a) **bugfix** consuming: avoid setting cursors to nil on LeaveGroup
1670
1671
1672v1.2.0
1673===
1674
1675This release contains new formatting features and sets the stage for a new
1676admin administration package to be added in v1.3.0. For now, the `kadm` package
1677is in a separate unversioned module. The API is relatively stable, but I'd like
1678to play with it some more to figure out what needs changing or not. Any
1679external experimenting and feedback is welcome.
1680
1681There are two new types introduced in the kgo package: `RecordFormatter` and
1682`RecordReader`. Both of these extract and improve logic used in the
1683[`kcl`](github.com/twmb/kcl) repo. These are effectively powerful ways to
1684format records into bytes, and read bytes into records. They are not _exactly_
1685opposites of each other due to reasons mentioned in the docs. I expect
1686formatting to be more useful than reading, and formatting is actually
1687surprisingly fast if the formatter is reused. The one off `Record.AppendFormat`
1688builds a new formatter on each use and should only be used if needed
1689infrequently, because building the formatter is the slow part.
1690
1691I am considering removing the default of snappy compression for the v1.3
1692release. I generally find it useful: many people forget to add compression
1693which increases network costs and makes broker disks fill faster, and snappy
1694compression is cheap cpu-wise and fast. However, some people are confused when
1695their bandwidth metrics look lower than they expect, and snappy _does_ mean
1696throughput is not _as_ high as it could be. Removal of snappy by default may or
1697may not happen, but if anybody agrees or disagrees with it, please mention so
1698in an issue or on discord.
1699
1700There are a few minor behavior changes in this release that should make default
1701clients safer (lowering max buffered memory) and faster on request failures.
1702These behavior changes should not affect anything. As well, the `bench` utility
1703no longer uses snappy by default.
1704
1705Non-formatter/reader/kadm commits worth mentioning:
1706
1707- [`a8dbd2f`](https://github.com/twmb/franz-go/commit/a8dbd2f) broker: add context to responses that look like HTTP
1708- [`ec0d81f`](https://github.com/twmb/franz-go/commit/ec0d81f) broker: avoid non-debug logs on ErrClientClosed
1709- [`af4fce4`](https://github.com/twmb/franz-go/commit/af4fce4) **beahvior change** bench: use no compression by default, remove -no-compression flag
1710- [`d368d11`](https://github.com/twmb/franz-go/commit/d368d11) bench: allow custom certs with -ca-cert, -client-cert, -client-key
1711- [`fbf9239`](https://github.com/twmb/franz-go/commit/fbf9239) broker: add two new connection types, cxnGroup and cxnSlow
1712- [`2d6c1a8`](https://github.com/twmb/franz-go/commit/2d6c1a8) **behavior change** client: lower RequestTimeoutOverhead and RetryTimeout defaults
1713- [`9bcfc98`](https://github.com/twmb/franz-go/commit/9bcfc98) kgo: only run req/resp handling goroutines when needed
1714- [`b9b592e`](https://github.com/twmb/franz-go/commit/b9b592e) **behavior change** kgo: change default MaxBufferedRecords from unlimited to 10,000
1715- [`19f4e9b`](https://github.com/twmb/franz-go/commit/19f4e9b) and [`58bf74a`](https://github.com/twmb/franz-go/commit/58bf74a) client: collapse shard errors
1716- [`126778a`](https://github.com/twmb/franz-go/commit/126778a) client: avoid sharding to partitions with no leader
1717- [`254764a`](https://github.com/twmb/franz-go/commit/254764a) **behavior change** kversion: skip WriteTxnMarkers for version guessing
1718
1719v1.1.4
1720===
1721
1722This includes on more patch for the prior tag to fully make `InitProducerID`
1723retriable when it encounters a retriable error.
1724
1725- [`d623ffe`](https://github.com/twmb/franz-go/commit/d623ffe) errors: make errProducerIDLoadFail retriable
1726
1727
1728v1.1.3
1729===
1730
1731This patch allows `BrokerNotAvailable` to be retried. As well, it contains some
1732small improvements under the hood. When sharding certain requests, we now avoid
1733issuing requests to partitions that have no leader. We also better collapse
1734per-partition sharding error messages.
1735
1736Dial errors are now not retriable. Now, if a produce request fails because the
1737broker cannot be dialed, we refresh metadata. Previously, we just accepted the
1738error as retriable and tried producing again.
1739
1740The SoftwareVersion used in ApiVersions requests now includes your current kgo
1741version.
1742
1743This tag is based on a 1.1.x branch in the repo. There is some feature work
1744that is yet to be stabilized on the main branch.
1745
1746Notable commits:
1747
1748- [`747ab0c`](https://github.com/twmb/franz-go/commit/747ab0c) kerr: make BrokerNotAvailable retriable
1749- [`f494301`](https://github.com/twmb/franz-go/commit/f494301) errors: make dial errors non-retriable
1750- [`4a76861`](https://github.com/twmb/franz-go/commit/4a76861) config: default to kgo dep version in ApiVersions
1751
1752v1.1.2
1753===
1754
1755This patch release fixes processing the `LogAppendTime` timestamp for message
1756set v1 formatted records. Message set v1 records were used only during the
1757Kafka v0.10 releases.
1758
1759- [`5fee169..688d6fb`](https://github.com/twmb/franz-go/compare/5fee169..688d6fb): **bugfix** message set v1: fix bit 4 as the timestamp bit
1760
1761v1.1.1
1762===
1763
1764This patch release fixes a bug in `PollRecords` that could cause a panic during
1765cluster instability. This also defines Kafka 3.0 in kversion.
1766
1767- [`847aeb9`](https://github.com/twmb/franz-go/commit/847aeb9) **bugfix** consumer: do not buffer partitions we reload
1768- [`49f82b9`](https://github.com/twmb/franz-go/commit/49f82b9) kversion: cut 3.0
1769
1770v1.1.0
1771===
1772
1773This minor release contains a few useful features, an incredibly minor bugfix
1774(fixing an error message), and a behavior change around buffering produced
1775records for unknown topics. As well, all plugins have been tagged v1.0.0.
1776
1777- [`cf04997`](https://github.com/twmb/franz-go/commit/cf04997) errors: export ErrRecordRetries, ErrRecordTimeout
1778- [`ee8b12d`](https://github.com/twmb/franz-go/commit/ee8b12d) producer: fail topics that repeatedly fail to load after 5 tries
1779- [`f6a8c9a`](https://github.com/twmb/franz-go/commit/f6a8c9a) **very minor bugfix** ErrDataLoss: use the new offset **after** we create the error
1780- [`8216b7c`](https://github.com/twmb/franz-go/commit/8216b7c) and [`c153b9a`](https://github.com/twmb/franz-go/commit/c153b9a) **feature** kmsg: add EnumStrings(), ParseEnum() functions
1781- [`e44dde9`](https://github.com/twmb/franz-go/commit/e44dde9) add `goroutine_per_partition` consuming example
1782- [`cedffb7`](https://github.com/twmb/franz-go/commit/cedffb7) **feature** plugins: Add kzerolog logging adapter (thanks [@fsaintjacques](https://github.com/fsaintjacques)!)
1783- [`563e016`](https://github.com/twmb/franz-go/commit/563e016) **feature** FetchTopic: add EachPartition; FetchPartition: add EachRecord
1784- [`8f648e7`](https://github.com/twmb/franz-go/commit/8f648e7) consumer group: document actual behavior for on revoked / lost
1785
1786v1.0.0
1787===
1788
1789This is a significant release and is a commitment to not changing the API.
1790There is no change in this release from v0.11.1. This release has been a long
1791time coming!
1792
1793The kmsg package is a separate dedicated module and its API can still change if
1794Kafka changes the protocol in a way that breaks kmsg. However, the odds are
1795increasingly likely that kmsg will switch to bumping major versions as Kafka
1796makes incompatible changes, so that the module does not have breaking updates.
1797
1798v0.11.1
1799===
1800
1801This is a patch release to a bug introduced in v0.10.3 / v0.11.0.
1802
1803The intent is to tag v1.0 either Friday (tomorrow) or Monday.
1804
1805- [`1469495`](https://github.com/twmb/franz-go/commit/1469495) **bugfix** isRetriableBrokerErr: nil error is **not** retriable (thanks [@Neal](https://github.com/Neal)!)
1806- [`33635e2`](https://github.com/twmb/franz-go/commit/33635e2) **feature** consumer: add pausing / unpausing topics and partitions
1807- [`082db89`](https://github.com/twmb/franz-go/commit/082db89) **bugfix** add locking to g.uncommitted MarkCommitRecords (thanks [@vtolstov](https://github.com/vtolstov)!)
1808- [`3684df2`](https://github.com/twmb/franz-go/commit/3684df2) fetches: rename CollectRecords to Records
1809
1810v0.11.0
1811===
1812
1813This release is an rc2 for a v1 release, and contains breaking API changes
1814related to renaming some options. The point of this release is to allow any
1815chance for final API suggestions before stabilization.
1816
1817The following configuration options were changed:
1818
1819* `ConnTimeoutOverhead` to `RequestTimeoutOverhead`
1820* `CommitCallback` to `AutoCommitCallback`
1821* `On{Assigned,Revoked,Lost}` to `OnPartitions{Assigned,LostRevoked}`
1822* `BatchCompression` to `ProducerBatchCompression`
1823* `Linger` to `ProducerLinger`
1824* `BatchMaxBytes` to `ProducerBatchMaxBytes`
1825* `{,Stop}OnDataLoss` to `{,Stop}ProducerOnDataLossDetected`
1826* `RecordTimeout` to `RecordDeliveryTimeout`
1827
1828The point of these renames is to make the options a bit clearer as to their
1829purpose and to hopefully be slightly less confusing.
1830
1831**If you have any API suggestions that should be addressed before a 1.0 API
1832stabilization release, please open an issue.**
1833
1834- [`577c73a`](https://github.com/twmb/franz-go/commit/577c73a) **breaking**: rename RecordTimeout to RecordDeliveryTimeout
1835- [`76ae8f5`](https://github.com/twmb/franz-go/commit/76ae8f5) **breaking**: rename {,Stop}OnDataLoss to {,Stop}ProducerOnDataLossDetected
1836- [`6272a3b`](https://github.com/twmb/franz-go/commit/6272a3b) **breaking**: rename BatchMaxBytes to ProducerBatchMaxBytes
1837- [`9a76213`](https://github.com/twmb/franz-go/commit/9a76213) **breaking**: rename Linger to ProducerLinger
1838- [`3d354bc`](https://github.com/twmb/franz-go/commit/3d354bc) **breaking**: rename BatchCompression to ProducerBatchCompression
1839- [`8c3eb3c`](https://github.com/twmb/franz-go/commit/8c3eb3c) **breaking**: renaming OnXyz to OnPartitionsXyz
1840- [`525b2d2`](https://github.com/twmb/franz-go/commit/525b2d2) **breaking**: rename CommitCallback to AutoCommitCallback
1841- [`ac9fd1c`](https://github.com/twmb/franz-go/commit/ac9fd1c) docs: clarify Balancer is equivalent to Java's PartitionAssignor
1842- [`2109ed4`](https://github.com/twmb/franz-go/commit/2109ed4) **breaking**: rename ConnTimeoutOverhead to RequestTimeoutOverhead
1843
1844v0.10.3
1845===
1846
1847This is a small release intended to be a minor release for anybody that does
1848not want to update the API. The next release, v0.11.0, is going to contain
1849breaking config option changes. The point of the intermediate release is to
1850allow a few days for anybody to raise issues for better suggestions.
1851
1852- [`31ed46b`](https://github.com/twmb/franz-go/commit/31ed46b) **feature** consuming: add autocommitting marked records
1853- [`c973268`](https://github.com/twmb/franz-go/commit/c973268) **feature** fetches: add CollectRecords convenience function
1854- [`a478251`](https://github.com/twmb/franz-go/commit/a478251) **bugfix** source: advance offsets even if we return no records
1855- [`307c22e`](https://github.com/twmb/franz-go/commit/307c22e) **minor bugfix** client retriable req: fix err bug
1856- [`fcaaf3f`](https://github.com/twmb/franz-go/commit/fcaaf3f) breaking kmsg: update per protocol changes
1857- [`262afb4`](https://github.com/twmb/franz-go/commit/262afb4) **bugfix** offset commit: use -1 for RetentionTimeMillis
1858- [`c6df11d`](https://github.com/twmb/franz-go/commit/c6df11d) consumer: commit all dirty offsets when _entering_ poll
1859- [`e185676`](https://github.com/twmb/franz-go/commit/e185676) (minor) breaking change: `AllowedConcurrentFetches` => `MaxConcurrentFetches`
1860- [`d5e80b3`](https://github.com/twmb/franz-go/commit/d5e80b3) bump min go version to go1.15
1861
1862v0.10.2
1863===
1864
1865This release contains everything currently outstanding for a v1.0.0 release.
1866Primarily, this release addresses #57, meaning that by default, auto committing
1867no longer has the potential to lose records during crashes. It is now highly
1868recommended to have a manual `CommitUncommittedOffsets` before shutting down if
1869autocommitting, just to prevent any unnecessary duplicate messages.
1870
1871The aim is to let this release bake for a day / a few days / a week so that if
1872anybody tries it out, I can receive any bug reports.
1873
1874- [`31754a9`](https://github.com/twmb/franz-go/commit/31754a9) **feature-ish** compression: switch snappy to @klauspost's s2
1875- [`28bba43`](https://github.com/twmb/franz-go/commit/28bba43) autocommitting: only commit previously polled fetches
1876- [`4fb0de2`](https://github.com/twmb/franz-go/commit/4fb0de2) `isRetriableBrokerErr`: opt in `net.ErrClosed`, restructure
1877- [`0d01f74`](https://github.com/twmb/franz-go/commit/0d01f74) **feature** add `RoundRobinPartitioner`
1878- [`5be804d`](https://github.com/twmb/franz-go/commit/5be804d) `LeastBackupPartitioner`: fix, speedup
1879- [`733848b`](https://github.com/twmb/franz-go/commit/733848b) **bugfix** CommitRecords: commit next offset to move forward
1880- [`4c20135`](https://github.com/twmb/franz-go/commit/4c20135) **bugfix/simplification** sink: simplify decInflight for batches in req
1881
1882v0.10.1
1883===
1884
1885This fix contains an important bugfix for some behavior introduced in v0.9.1,
1886as well as extra debug logging messages, and a few other general improvements.
1887Some more errors are now retriable in more cases, and the client can now
1888tolerate some input seeds being invalid when others are not invalid.
1889
1890It is highly recommended that any use of v0.9.1 switch to v0.10.1. If on v0.9.1
1891and a produced batch has a retriable error, and then some cluster maintenance
1892causes the partition to move to a different broker, the client will be unable
1893to continue producing to the partition.
1894
1895- [`029f5e3`](https://github.com/twmb/franz-go/commit/029f5e3) **bugfix** sink: fix decInflight double decrement bug
1896- [`2cf62e2`](https://github.com/twmb/franz-go/commit/2cf62e2) sink: properly debug log the number of records written
1897- [`370c18e`](https://github.com/twmb/franz-go/commit/370c18e) and [`caadb8b`](https://github.com/twmb/franz-go/commit/caadb8b) **feature**: add LeastBackupPartitioner
1898- [`0fee54c`](https://github.com/twmb/franz-go/commit/0fee54c) and [`589c5e5`](https://github.com/twmb/franz-go/commit/589c5e5) add errUnknownBroker and io.EOF to be retriable errors in sink; add errUnknownBroker to be skippable in the client
1899- [`ec0c992`](https://github.com/twmb/franz-go/commit/ec0c992) client improvement: retry more when input brokers are invalid
1900
1901v0.10.0
1902===
1903
1904#62 now tracks the v1.0.0 stabilization status. This release contains
1905everything but addressing #57, which I would like to address before
1906stabilizing.
1907
1908After using this client some more in a different program, I've run into some
1909things that are better to break now. Particularly, a few options are awkward.
1910The goal is for this release to make these options less awkward.
1911
1912Three options have been renamed in this release, the fixes are very simple (and
1913ideally, if you used `RetryTimeout`, you may find the new `RetryTimeout` even
1914simpler).
1915
1916- [`c1d62a7`](https://github.com/twmb/franz-go/commit/c1d62a7) **feature** config: add DialTLSConfig option
1917- [`4e5eca8`](https://github.com/twmb/franz-go/commit/4e5eca8) **breaking**: rename ProduceRetries to RecordRetries
1918- [`12808d5`](https://github.com/twmb/franz-go/commit/12808d5) **breaking**: rename RetryBackoff to RetryBackoffFn
1919- [`8199f5b`](https://github.com/twmb/franz-go/commit/8199f5b) **breaking**: split RetryTimeout function
1920
1921v0.9.1
1922===
1923
1924v0.9.0 is to be stabilized, but I would like to address #57 first, which may
1925slightly change how autocommitting works. This intermediate release pulls in
1926some decent fixes.
1927
1928- [`fd889cc`](https://github.com/twmb/franz-go/commit/fd889cc) all: work around brokers that have inconsistent request versions
1929- [`f591593`](https://github.com/twmb/franz-go/commit/f591593) broker: permanently store the initial ApiVersions response
1930- [`8da4eaa`](https://github.com/twmb/franz-go/commit/8da4eaa) and [`37ecd21`](https://github.com/twmb/franz-go/commit/37ecd21) **feature** client: enable ipv6 support in seed brokers (thanks [@vtolstov](https://github.com/vtolstov) and [@SteadBytes](https://github.com/SteadBytes)!)
1931- [`aecaf27`](https://github.com/twmb/franz-go/commit/aecaf27) **bugfix** client: force a coordinator refresh if the coordinator is unknown
1932- [`f29fb7f`](https://github.com/twmb/franz-go/commit/f29fb7f) **bugfix** sink: fix out-of-order response handling across partition rebalances
1933- [`4fadcde`](https://github.com/twmb/franz-go/commit/4fadcde) **rare bugfix** consuming: fix logic race, simplify some logic
1934
1935v0.9.0
1936===
1937
1938This is a v1.0 release candidate. The only breaking change in this release is
1939that the kmsg package is now a dedicated module.
1940
1941### Why have a module for kmsg?
1942
1943The kmsg package is almost entirely autogenerated from the Kafka protocol.
1944Unfortunately, the Kafka protocol is mostly designed for an object oriented
1945language (i.e., Java), so some changes to the Kafka protocol that are
1946non-breaking are breaking in the kmsg package. In particular, the Kafka changes
1947a non-nullable field to a nullable field, the kmsg package must change from a
1948non-pointer to a pointer. This is a breaking change. Even more rarely,
1949sometimes Kafka changes a field's type by renaming its old version to
1950"SomethingOld" and having a new version take the place of "Something". I would
1951prefer the kmsg avoid "SomethingNew" to avoid deprecating the old field, so I
1952also choose to rename fields.
1953
1954One option is to have absolutely everything be a pointer in the kmsg package,
1955but this would create a lot of garbage and make using the API even worse. As
1956well, I could just add newly named fields, but this makes things worse as time
1957goes on, because the newer fields would be the ones with the worse names.
1958
1959In Go, if kmsg were stabilized, any protocol change would need to be a major
1960version bump. We cannot get away with minor bumps and a big doc comment saying
1961"deal with it", because we run into a transitive dependency issue: if user A
1962uses franz-go at v1.1.0, and they also use a library that uses franz-go at
1963v1.0.0, and there is an incompatible kmsg change, then the user's compilation
1964breaks.
1965
1966So, we choose to have a dedicated module for kmsg which is unversioned. This
1967avoids some transitive dependency issues.
1968
1969If you use the kmsg package directly, the only update you should need to do is:
1970
1971```
1972go get -u github.com/twmb/franz-go/pkg/kmsg@latest
1973```
1974
1975### Other potentially notable changes
1976
1977- [`f82e5c6`](https://github.com/twmb/franz-go/commit/f82e5c6) consumer: document that a nil / closed context is supported for polling
1978- [`8770662`](https://github.com/twmb/franz-go/commit/8770662) very minor bugfix for producing: only opt in to more than 1req if idempotent
1979- [`5c0a2c7`](https://github.com/twmb/franz-go/commit/5c0a2c7) bench: add -prometheus flag
1980
1981v0.8.7
1982===
1983
1984This release contains commits for upcoming Kafka protocol changes, new features
1985to gain insight into buffered records (and intercept & modify them before being
1986produced or consumed), a few minor other changes, and one minor breaking
1987change.
1988
1989The minor breaking change should not affect anybody due to its rather niche use
1990when better methods exist, but if it does, the fix is to delete the
1991`Partitions` field that was removed (the field is now embedded).
1992
1993This field was removed just for ease of use purposes: there was no reason to
1994have a separate named field in `FetchTopicPartition` for the `Partition`,
1995instead, we can just embed `FetchPartition` to make usage of the type much
1996simpler. Dropping this field basically makes this type much more appealing to
1997use.
1998
1999I may be releasing v0.9.0 shortly, with plans to potentially split the kmsg
2000package into a dedicated module. More details on that if this happens.
2001
2002### Breaking change
2003
2004- [`ffc94ea`](https://github.com/twmb/franz-go/commit/ffc94ea) **minor breaking change**: embed `FetchPartition` in `FetchTopicPartition` rather than have a named `Partition` field
2005
2006### Features
2007
2008- [`1bb70a5`](https://github.com/twmb/franz-go/commit/1bb70a5) kprom: clarify seed ids, add two new metrics
2009- [`916fc10`](https://github.com/twmb/franz-go/commit/916fc10) client: add four new hooks to provide buffer/unbuffer interceptors
2010- [`1e74109`](https://github.com/twmb/franz-go/commit/1e74109) hooks: add HookNewClient
2011- [`3256518`](https://github.com/twmb/franz-go/commit/3256518) client: add Buffered{Produce,Fetch}Records methods
2012- [`ebf2f07`](https://github.com/twmb/franz-go/commit/ebf2f07) support KIP-516 for Fetch (topic IDs in fetch requests)
2013- [`e5e37fc`](https://github.com/twmb/franz-go/commit/e5e37fc) and [`3a3cc06`](https://github.com/twmb/franz-go/commit/3a3cc06) support KIP-709: batched OffsetFetchRequest in a forward-compatible way (and minor group sharded request redux)
2014- [`eaf9ebe`](https://github.com/twmb/franz-go/commit/eaf9ebe) kprom: use Registerer and Gatherer interface instead of concrete registry type.
2015- [`10e3f44`](https://github.com/twmb/franz-go/commit/10e3f44) and [`4fdc7e0`](https://github.com/twmb/franz-go/commit/4fdc7e0) support KIP-699 (batch FindCoordinator requests)
2016
2017### Minor other changes
2018
2019- [`d9b4fbe`](https://github.com/twmb/franz-go/commit/d9b4fbe) consumer group: add "group" to all log messages
2020- [`f70db15`](https://github.com/twmb/franz-go/commit/f70db15) broker: use ErrClientClosed more properly for internal requests that use the client context
2021- [`8f1e732`](https://github.com/twmb/franz-go/commit/8f1e732) producer: use ErrClientClosed properly if waiting to produce when the client is closed
2022- [`ee918d9`](https://github.com/twmb/franz-go/commit/ee918d9) sink: drop the drain loop sleep from 5ms to 50microsec
2023- [`e48c03c`](https://github.com/twmb/franz-go/commit/e48c03c) client: log "seed #" rather than a large negative for seed brokers
2024
2025v0.8.6
2026===
2027
2028
2029- [`7ec0a45`](https://github.com/twmb/franz-go/commit/7ec0a45) **bugfix** consumer group: bugfix regex consuming (from v0.8.0+)
2030- [`91fe77b`](https://github.com/twmb/franz-go/commit/91fe77b) bench: faster record value generation, add -static-record flag
2031- [`f95859e`](https://github.com/twmb/franz-go/commit/f95859e) generated code: always return typed value if present
2032- [`db5fca5`](https://github.com/twmb/franz-go/commit/db5fca5) support KIP-734
2033- [`ae7182b`](https://github.com/twmb/franz-go/commit/ae7182b) group balancing: address KAFKA-12898
2034
2035This small tag fixes a bug in group consuming via regex, adds a new
2036`-static-record` flag to the benchmark example, and has a few other very minor
2037changes (see commits).
2038
2039Also, thank you @Neal for fixing some typos and eliminating a useless goroutine
2040in the benchmarks!
2041
2042v0.8.5
2043===
2044
2045- [`2732fb8`](https://github.com/twmb/franz-go/commit/2732fb8) **bugfix** consumer: bugfix consuming of specific partitions if no topics are specified
2046
2047If `ConsumePartitions` was used without `ConsumeTopics`, the consumer would not
2048start. This was missed in the conversion of starting the consumer after
2049initializing a client to initializing the consumer _with_ the client.
2050
2051v0.8.4
2052===
2053
2054- [`e0346a2`](https://github.com/twmb/franz-go/commit/e0346a2) **very minor bugfix** consumer group: set defaultCommitCallback
2055- [`5047b31`](https://github.com/twmb/franz-go/commit/5047b31) and [`05a6b8a`](https://github.com/twmb/franz-go/commit/05a6b8a) bench: add -linger, -disable-idempotency, -log-level, -compression, -batch-max-bytes options
2056
2057This is a small release containing a very minor bugfix and more flags on the
2058benchmark program. The bugfix basically re-adds a missing commit callback if
2059you are not specifying your own, and the default commit callback just logs
2060errors. So in short, this release ensures you see errors if there are any and
2061you are using the default commit callback.
2062
2063v0.8.3
2064===
2065
2066- [`053911b`](https://github.com/twmb/franz-go/commit/053911b) plugins: fix module declarations
2067
2068This is a patch release on the prior commit to fix the path declarations in the
2069new plugin modules.
2070
2071v0.8.2
2072===
2073
2074- [`65a0ed1`](https://github.com/twmb/franz-go/commit/65a0ed1) add pluggable kgmetrics, kprom, kzap packages & examples
2075
2076This release immediately follows the prior release so that the plugin packages
2077can refer to v0.8.1, which contains the (hopefully) final API breakages. This
2078allows Go's version resolution to ensure users of these plugins use the latest
2079franz-go package.
2080
2081These new plugins should make it very easy to integrate prometheus or
2082go-metrics or zap with kgo.
2083
2084v0.8.1
2085===
2086
2087This release contains a few features, one minor bugfix, two minor breaking
2088API changes, and a few minor behavior changes.
2089
2090This release will immediately be followed with a v0.8.2 release that contains
2091new examples using our new hooks added in this release.
2092
2093### Breakage
2094
2095One of the breaking API changes is followup from the prior release: all
2096broker-specific hooks were meant to be renamed to `OnBrokerXyz`, but I missed
2097`OnThrottle`. This is a rather niche hook so ideally this does not break
2098many/any users.
2099
2100The other breaking API change is to make an API consistent: we have `Produce`
2101and `ProduceSync`, and now we have `CommitRecords` and
2102`CommitUncommittedOffsets` in addition to the original `CommitOffsets`, so
2103`BlockingCommitOffsets` should be renamed to `CommitOffsetsSync`. This has the
2104added benefit that now all committing functions will be next to each other in
2105the documentation. One benefit with breaking this API is that it may help
2106notify users of the much more easily used `CommitUncommittedOffsets`.
2107
2108### Features
2109
2110- Two new partitioners have been added, one of which allows you to set the
2111 `Partition` field on a record before producing and have the client use that
2112field for partitioning.
2113
2114- Two new hooks have been added that allow tracking some metrics per-batch
2115 while producing and consuming. More per-batch metrics can be added to the
2116hooks later if necessary, because the hooks take a struct that can be expanded
2117upon.
2118
2119- The client now returns an injected `ErrClientClosed` fetch partition when
2120 polling if the client has been closed, and `Fetches` now contains a helper
2121`IsClientClosed` function. This can be used to break out of a poll loop on
2122shutdown.
2123
2124### Behavior changes
2125
2126- The client will no longer add empty fetches to be polled. If fetch sessions
2127 are disabled, or in certain other cases, Kafka replies to a fetch requests
2128with the requested topics and partitions, but no records. The client would
2129process these partitions and add them to be polled. Now, the client will avoid
2130adding empty fetches (unless they contain an error), meaning polling should
2131always have fetches that contain either records or an error.
2132
2133- When using sharded requests, the client no longer issues split pieces of the
2134 requests to partitions that are currently erroring. Previously, if a request
2135needed to go to the partition leader, but the leader was offline, the client
2136would choose a random broker to send the request to. The request was expected
2137to fail, but the failure error would be retriable, at which point we would
2138reload metadata and hope the initial partition leader error would be resolved.
2139We now just avoid this try-and-fail-and-hope loop, instead erroring the split
2140piece immediately.
2141
2142### Examples
2143
2144This contains one more example, [`examples/group_consuming`](./examples/group_consuming),
2145which demonstrates how to consume as a group and commit in three different ways,
2146and describes the downsides of autocommitting.
2147
2148### Changes
2149
2150- [`fa1fd35`](https://github.com/twmb/franz-go/commit/fa1fd35) **feature** consuming: add HookFetchBatchRead
2151- [`9810427`](https://github.com/twmb/franz-go/commit/9810427) **feature** producing: add HookProduceBatchWritten
2152- [`20e5912`](https://github.com/twmb/franz-go/commit/20e5912) **breaking api** hook: rename OnThrottle => OnBrokerThrottle
2153- [`a1d7506`](https://github.com/twmb/franz-go/commit/a1d7506) examples: add group consumer example, with three different commit styles
2154- [`058f692`](https://github.com/twmb/franz-go/commit/058f692) behavior change, consuming: only add fetch if it has records or errors
2155- [`d9649df`](https://github.com/twmb/franz-go/commit/d9649df) **feature** fetches: add IsClientClosed helper
2156- [`bc0add3`](https://github.com/twmb/franz-go/commit/bc0add3) consumer: inject ErrClientClosing when polling if the client is closed
2157- [`f50b320`](https://github.com/twmb/franz-go/commit/f50b320) client: make public ErrClientClosed
2158- [`8b7b43e`](https://github.com/twmb/franz-go/commit/8b7b43e) behavior change, client: avoid issuing requests to shards that we know are erroring
2159- [`96cb1c2`](https://github.com/twmb/franz-go/commit/96cb1c2) **bugfix** fix ACLResourcePatternType: add ANY
2160- [`8cf3e5a`](https://github.com/twmb/franz-go/commit/8cf3e5a) **breaking api** rename BlockingCommitOffsets to CommitOffsetsSync
2161- [`2092b4c`](https://github.com/twmb/franz-go/commit/2092b4c) and [`922f4b8`](https://github.com/twmb/franz-go/commit/922f4b8) **feature** add CommitRecords and CommitUncommittedOffsets
2162- [`6808a55`](https://github.com/twmb/franz-go/commit/6808a55) **feature** add BasicConsistentPartitioner / ManualPartitioner
2163
2164v0.8.0
2165===
2166
2167This is a **major breaking release**. This release is intended to be a release
2168candidate for a v1.0.0 tag; thus, I wanted to nail down all API breakage now to
2169help prevent future breakage and to ensure a path towards 1.0 stabilization. I
2170do not expect major API changes after this tag, and I intend to release v1.0.0
2171within a month of this tag.
2172
2173## Why the breakage?
2174
2175It never felt "right" that to consume, you needed to first create a client, and
2176then assign something to consume. One large reason that `AssignXyz` existed was
2177so that you could reassign what was being consumed at runtime. In fact, you
2178could consume from a group, then leave and switch to direct partition
2179consuming, and then switch back to consuming from a different group. This
2180flexibility was unnecessary, and assigning after the client was initialized was
2181awkward. Worse, allowing these reassignments necessitated extreme care to not
2182have race conditions or deadlocks. This was the source of many bugs in the
2183past, and while they have since been ironed out, we may as well just remove
2184them as a possibility while on the path towards v1.0.
2185
2186Because we have this one major breakage already, I decided it was a good time
2187to clean up the other not-so-great aspects of the code.
2188
2189- All current hooks have been renamed, because `OnE2E` being dedicated to
2190 brokers does not leave room for a different `E2E` hook in the future that is
2191not specific to brokers. Instead, if we namespace the current broker hooks, we
2192can namespace future hooks as well, and nothing will look weird. More
2193beneficially, in user code, the hook will be more self describing.
2194
2195- One now redundant option has been removed (`BrokerConnDeadRetries`), and one
2196 useless option has been removed (`DisableClientID`).
2197
2198Moving consumer options to the client required changing the names of some
2199consumer options, merging some duplicated group / direct options, adding
2200`*Client` as an argument to some callbacks, and cleaning up some consume APIs.
2201
2202From there on, the breakages get more minor: `AutoTopicCreation` now is
2203`AllowAutoTopicCreation`, and `ProduceTopic` is now `DefaultProduceTopic`.
2204
2205## Upgrade considerations
2206
2207Due to the number of breaking changes, upgrading may _look_ more difficult than
2208it actually is. I've updated every example in this repo and all code usage
2209in my corresponding [`kcl`](github.com/twmb/kcl) repo, these updates were
2210completed relatively quickly.
2211
2212## tl;dr of upgrading
2213
2214The simpler fixes:
2215
2216- Change `AutoTopicCreate` to `AllowAutoTopicCreate`
2217- Change `ProduceTopic` to `DefaultProduceTopic`
2218- Remove `BrokerConnDeadRetries` and `DisableClientID` if you used them (unlikely)
2219- Add `Broker` in any hook (`OnConnect` => `OnBrokerConnect`, etc)
2220
2221If directly consuming, perform the following changes to options and move the options to `NewClient`:
2222
2223- Drop the offset argument from `ConsumeTopics`
2224- Move `ConsumePartitions`
2225- Change `ConsumeTopicsRegex` to `ConsumeRegex`
2226- Delete `AssignPartitions`
2227
2228If group consuming, perform the following changes to options and move the options to `NewClient`:
2229
2230- Add `ConsumerGroup` with the group argument you used in `AssignGroup`
2231- Change `GroupTopics` to `ConsumeTopics`
2232- Add a `*Client` argument to any of `OnAssigned`, `OnRevoked`, `OnLost`, and `CommitCallback`
2233
2234If using a group transaction session, perform the above group changes, and use `NewGroupTransactSession`, rather than `NewClient`.
2235
2236## Changes
2237
2238- [`6a048db`](https://github.com/twmb/franz-go/commit/6a048db) **breaking API** hooks: namespace all hooks with Broker
2239- [`8498383`](https://github.com/twmb/franz-go/commit/8498383) **breaking API** client: large breaking change for consuming APIs
2240- [`45004f8`](https://github.com/twmb/franz-go/commit/45004f8) **breaking API** config: rename ProduceTopic to DefaultProduceTopic, doc changes
2241- [`aa849a1`](https://github.com/twmb/franz-go/commit/aa849a1) **breaking API** options: prefix AutoTopicCreation with Allow
2242- [`be6adf5`](https://github.com/twmb/franz-go/commit/be6adf5) **breaking API** client: remove DisableClientID option
2243- [`68b1a04`](https://github.com/twmb/franz-go/commit/68b1a04) **breaking API** client: remove BrokerConnDeadRetries option; drop retries to 20
2244- [`88e131d`](https://github.com/twmb/franz-go/commit/88e131d) **bugfix** kerberos: fix off-by-one in asn1LengthBytes (but it appears this is still not fully working)
2245- [`20e0f66`](https://github.com/twmb/franz-go/commit/20e0f66) **feature** Fetches: add EachError, clarifying documentation
2246- [`085ad30`](https://github.com/twmb/franz-go/commit/085ad30) metadata: limit retries, bump produce load errors on failure
2247- [`b26489f`](https://github.com/twmb/franz-go/commit/b26489f) config: change default non-produce retries from unlimited to 30 (later commit just above changes down to 20)
2248
2249v0.7.9
2250===
2251
2252- [`5231902`](https://github.com/twmb/franz-go/commit/5231902) **bugfix** patch on prior commit
2253
2254If I could yank tags, I would.
2255
2256v0.7.8
2257===
2258
2259- [`b7cb533`](https://github.com/twmb/franz-go/commit/b7cb533) **bugfix** allow any `*os.SyscallError` to be retriable
2260
2261_This_ should be the last v0.7 release. This is a small bugfix to allow much
2262more retrying of failing requests, particularly around failed dials, which is
2263much more resilient to restarting brokers.
2264
2265v0.7.7
2266===
2267
2268- [`afa1209`](https://github.com/twmb/franz-go/commit/afa1209) txn: detect a fatal txnal client when beginning transactions
2269- [`5576dce`](https://github.com/twmb/franz-go/commit/5576dce) benchmarks: add comparisons to confluent-kafka-go & sarama
2270- [`d848174`](https://github.com/twmb/franz-go/commit/d848174) examples: add benchmarking example
2271- [`fec2a18`](https://github.com/twmb/franz-go/commit/fec2a18) client: fix request buffer pool, add promisedNumberedRecord pool
2272- [`a0d712e`](https://github.com/twmb/franz-go/commit/a0d712e) transactions: small wording changes in docs
2273- [`bad47ba`](https://github.com/twmb/franz-go/commit/bad47ba) and [`a9691bd`](https://github.com/twmb/franz-go/commit/a9691bd) **feature** hooks: add HookBrokerE2E
2274
2275This is a small release with one useful new hook, a few minor updates /
2276internal fixes, and no bug fixes.
2277
2278This now properly pools request buffers, which will reduce garbage when
2279producing, and re-adds pooling slices that records are appended to before
2280flushing. This latter pool is less important, but can help.
2281
2282This now adds one more chance to recover a transactional client, which also
2283gives the user a chance to things are fatally failed when beginning
2284transactions.
2285
2286Finally, this adds a benchmarking example and comparisons to sarama /
2287confluent-kafka-go. To say the least, the numbers are favorable.
2288
2289This is likely the last release of the v0.7 series, the next change will be a
2290few breaking API changes that should hopefully simplify initializing a
2291consumer.
2292
2293v0.7.6
2294===
2295
2296This is a small release that adds defaults for any `kmsg.Request` that has a
2297`TimeoutMillis` field that previously did not have a default.
2298
2299This also changes how the `TimeoutMillis` field is specified for generating,
2300and now all documentation around it is consistent.
2301
2302Lastly, any field that has a default now has that default documented.
2303
2304v0.7.5
2305===
2306
2307This commit adds support for session tokens and user agents in `AWS_MSK_IAM`,
2308as well as adds an [example](./examples/sasl/aws_msk_iam) for how to use
2309`AWS_MSK_IAM`.
2310
2311v0.7.4
2312===
2313
2314- [`467e739`](https://github.com/twmb/franz-go/commit/467e739) FirstErrPromise: change semantics
2315
2316This is a small release to change the semantics of `FirstErrPromise`: now, it
2317uses a `sync.WaitGroup` on every `Promise` and will wait until all records have
2318been published / aborted before returning from `Err`. This also adds an
2319`AbortingFirstErrPromise` that automatically aborts all records if any promise
2320finishes with an error.
2321
2322v0.7.3
2323===
2324
2325- [`30c4ba3`](https://github.com/twmb/franz-go/commit/30c4ba3) **feature** sasl: add support for `AWS_MSK_IAM`
2326
2327This is a small release dedicated to adding support for `AWS_MSK_IAM` sasl.
2328
2329v0.7.2
2330===
2331
2332- [`522c9e2`](https://github.com/twmb/franz-go/commit/522c9e2) **bugfix** consumer group: use `JoinGroupResponse.Protocol`, not `SyncGroupResponse.Protocol`
2333
2334This is a small bugfix release; the `Protocol` field in `SyncGroupResponse` was
2335added in 2.5.0, and my integration tests did not catch this because I usually
2336test against the latest releases. All `JoinGroupResponse` versions have the
2337protocol that was chosen, so we use that field.
2338
2339v0.7.1
2340===
2341
2342- [`98f74d1`](https://github.com/twmb/franz-go/commit/98f74d1) README: note that we actually do support KIP-533
2343- [`528f007`](https://github.com/twmb/franz-go/commit/528f007) and [`ef9a16a`](https://github.com/twmb/franz-go/commit/ef9a16a) and [`d0cc729`](https://github.com/twmb/franz-go/commit/d0cc729) **bugfix** client: avoid caching invalid coordinators; allow retries
2344
2345This is a small bugfix release: previously, if FindCoordinator returned a node
2346ID of -1, we would permanently cache that. -1 is returned when the load has an
2347error, which we were not checking, but even still, we should not cache the
2348response if we do not know of the broker.
2349
2350Now, we will only cache successful loads.
2351
2352(Also, I noticed that I _do_ have an existing configuration knob for the retry
2353timeout, so that is now "Supported" in the KIP section, making all KIPs
2354supported).
2355
2356v0.7.0
2357===
2358
2359This is a big release, and it warrants finally switching to 0.7.0. There are a
2360few small breaking changes in this release, a lot of new features, a good few
2361bug fixes, and some other general changes. The documentation has been
2362overhauled, and there now exists an example of hooking in to prometheus metrics
2363as well as an example for EOS.
2364
2365Most of the new features are quality of life improvements; I recommend taking a
2366look at them to see where they might simplify your code.
2367
2368## Upgrade considerations
2369
2370Besides the two breaking changes just below, one bug fix may affect how you if
2371you are using a group consumer with autocommitting. If autocommitting, the
2372consumer should always have been issuing a blocking commit when leaving the
2373group to commit the final consumed records. A bug existed such that this commit
2374would never actually be issued. That bug has been fixed, so now, if you rely on
2375autocommitting, the client closing may take a bit longer due to the blocking
2376commit.
2377
2378## Breaking changes
2379
2380There are two breaking changes in this release, one of which likely will go
2381unnoticed. First, to allow all hooks to display next to each other in godoc,
2382the interfaces have had their trailing "Hook" moved to a leading "Hook".
2383Second and definitely noticeable, record producing no longer returns an error.
2384The original error return was not really that intuitive and could lead to bugs
2385if not understood entirely, so it is much simpler to just always call the
2386promise.
2387
2388- [`215f76f`](https://github.com/twmb/franz-go/commit/215f76f) small breaking API: prefix hook interfaces with "Hook"
2389- [`c045366`](https://github.com/twmb/franz-go/commit/c045366) producer: drop error return from Produce (breaking API change)
2390
2391## Features
2392
2393- [`c83d5ba`](https://github.com/twmb/franz-go/commit/c83d5ba) generate: support parsing and encoding unknown tags
2394- [`6a9eb0b`](https://github.com/twmb/franz-go/commit/6a9eb0b) kmsg: add Tags opaque type; ReadTags helper
2395- [`9de3959`](https://github.com/twmb/franz-go/commit/9de3959) add support for KIP-568 (force rebalance)
2396- [`d38ac84`](https://github.com/twmb/franz-go/commit/d38ac84) add HookGroupManageError
2397- [`0bfa547`](https://github.com/twmb/franz-go/commit/0bfa547) consumer group: add CommitCallback option
2398- [`231d0e4`](https://github.com/twmb/franz-go/commit/231d0e4) fetches: add EachRecord
2399- [`aea185e`](https://github.com/twmb/franz-go/commit/aea185e) add FirstErrPromise
2400- [`780d168`](https://github.com/twmb/franz-go/commit/780d168) Record: add helper constructors; allocation avoiders w/ unsafe
2401- [`55be413`](https://github.com/twmb/franz-go/commit/55be413) producer feature: allow a default Topic to produce to
2402- [`e05002b`](https://github.com/twmb/franz-go/commit/e05002b) and [`1b69836`](https://github.com/twmb/franz-go/commit/1b69836) consumer group: export APIs allow custom balancers
2403- [`6db1c39`](https://github.com/twmb/franz-go/commit/6db1c39) Fetches: add EachTopic helper
2404- [`b983d63`](https://github.com/twmb/franz-go/commit/b983d6), [`7c9f591`](https://github.com/twmb/franz-go/commit/7c9f59), [`3ad8fc7`](https://github.com/twmb/franz-go/commit/3ad8fc), and [`3ad8fc7`](https://github.com/twmb/franz-go/commit/3ad8fc7) producer: add ProduceSync
2405
2406## Bug fixes
2407
2408- [`45cb9df`](https://github.com/twmb/franz-go/commit/45cb9df) consumer: fix SetOffsets bug
2409- [`46cfcb7`](https://github.com/twmb/franz-go/commit/46cfcb7) group: fix blocking commit on leave; potential deadlock
2410- [`1aaa1ef`](https://github.com/twmb/franz-go/commit/1aaa1ef) consumer: fix data race in handleListOrEpochResults
2411- [`d1341ae`](https://github.com/twmb/franz-go/commit/d1341ae) sticky: fix extreme edge case for complex balancing
2412- [`9ada82d`](https://github.com/twmb/franz-go/commit/9ada82d) sink: create producerID *BEFORE* produce request (partial revert of dc44d10b)
2413- [`5475f6b`](https://github.com/twmb/franz-go/commit/5475f6b) sink: bugfix firstTimestamp
2414- [`2c473c4`](https://github.com/twmb/franz-go/commit/2c473c4) client: add OffsetDeleteRequest to handleCoordinatorReq
2415- [`bf5b74c`](https://github.com/twmb/franz-go/commit/bf5b74c) and 3ad8fc7 broker: avoid reaping produce cxn on no reads when acks == 0
2416- [`43a0009`](https://github.com/twmb/franz-go/commit/43a0009) sink w/ no acks: debug log needs to be before finishBatch
2417- [`8cf9eb9`](https://github.com/twmb/franz-go/commit/8cf9eb9) sink: bugfix panic on acks=0
2418
2419## General
2420
2421- [`939cba2`](https://github.com/twmb/franz-go/commit/939cba2) txns: make even safer (& drop default txn timeout to 40s)
2422- [`faaecd2`](https://github.com/twmb/franz-go/commit/faaecd2) implement KIP-735 (bump session timeout to 45s)
2423- [`3a95ec8`](https://github.com/twmb/franz-go/commit/3a95ec8) GroupTxnSession.End: document no error is worth retrying
2424- [`c5a47ea`](https://github.com/twmb/franz-go/commit/c5a47ea) GroupTransactSession.End: retry with abort on OperationNotAttempted
2425- [`6398677`](https://github.com/twmb/franz-go/commit/6398677) EndTxn: avoid creating a producer ID if nothing was produced
2426- [`c7c08fb`](https://github.com/twmb/franz-go/commit/c7c08fb) txnal producer: work around KAFKA-12671, take 2
2427- [`9585e1d`](https://github.com/twmb/franz-go/commit/9585e1d) FetchesRecordIter: avoid mutating input fetches
2428
2429v0.6.14
2430===
2431
2432- [`dc44d10`](https://github.com/twmb/franz-go/commit/dc44d10) sink: call producerID after creating the produce request
2433- [`ce113d5`](https://github.com/twmb/franz-go/commit/ce113d5) **bugfix** producer: fix potential lingering recBuf issue
2434- [`19d57dc`](https://github.com/twmb/franz-go/commit/19d57dc) **bugfix** metadata: do not nil cursors/records pointers ever
2435- [`e324b56`](https://github.com/twmb/franz-go/commit/e324b56) producing: evaluate whether a batch should fail before and after
2436
2437This is a small bugfix release for v0.6.13, which would panic if a user was
2438producing to and consuming from the same topic within a single client.
2439
2440At the same time, there was a highly-unlikely-to-be-experienced bug where
2441orphaned recBuf pointers could linger in a sink through a certain sequence
2442of events (see the producer bugfix commit for more details).
2443
2444This also now avoids initializing a producer ID when consuming only. For code
2445terseness during the producer redux, I moved the createReq portion in sink to
2446below the producerID call. We actually call createReq when there are no records
2447ever produced: a metadata update adding a recBuf to a sink triggers a drain,
2448which then evaluates the recBuf and sees there is nothing to produce. This
2449triggered drain was initializing a producer ID unnecessesarily. We now create
2450the request and see if there is anything to flush before initializing the
2451producer ID, and we now document the reason for having producerID after
2452createReq so that I do not switch the order in the future again.
2453
2454Lastly, as a feature enhancement, this unifies the logic that fails buffered
2455records before producing or after producing. The context can now be used to
2456cancel records after producing, and record timeouts / retries can be evaluated
2457before producing. The order of evaluation is first the context, then the record
2458timeout, and lastly the number of tries.
2459
2460v0.6.13
2461===
2462
2463- [`6ae76d0`](https://github.com/twmb/franz-go/commit/6ae76d0): **feature** producer: allow using the passed in context to cancel records
2464- [`2b9b8ca`](https://github.com/twmb/franz-go/commit/2b9b8ca): **bugfix** sink: reset needSeqReset
2465- [`314de8e`](https://github.com/twmb/franz-go/commit/314de8e): **feature** producer: allow records to fail when idempotency is enabled
2466- [`bf956f4`](https://github.com/twmb/franz-go/commit/bf956f4): **feature** producer: delete knowledge of a topic if we never load it
2467- [`83ecd8a`](https://github.com/twmb/franz-go/commit/83ecd8a): **bugfix** producer unknown wait: retry on retriable errors
2468- [`a3c2a5c`](https://github.com/twmb/franz-go/commit/a3c2a5c): consumer: backoff when list offsets or load epoch has any err (avoid spinloop)
2469- [`9c27589`](https://github.com/twmb/franz-go/commit/9c27589): zstd compression: add more options (switch to minimal memory usage)
2470- [`0554ad5`](https://github.com/twmb/franz-go/commit/0554ad5): producer: retry logic changes
2471
2472This release focuses on restructing some code to allow deleting topics from the
2473client and, most importantly, to allow records to fail even with idempotency.
2474
2475For deleting records, this is a minimal gain that will really only benefit a
2476user reassigning consuming topics, or for producers to a topic that does not
2477exist. Still, nifty, and for an EOS user, produce and consume topics will no
2478longer be next to each other (which may make scanning partitions to produce or
2479consume quicker).
2480
2481For allowing records to have limited retries / be canceled with a context, this
2482is an important feature that gives users more control of aborting the work they
2483tried. As noted in the config docs, we cannot always safely abort records, but
2484for the most part, we can. The only real unsafe time is when we have written a
2485produce request but have not received a response. If we ever get a response for
2486the first batch in a record buffer (or if we do not write the batch ever), then
2487we can safely fail the records. We do not need to worry about the other
2488concurrent requests because if the first batch errors, the others will error
2489and we know we can fail records based off the first's status, and if the first
2490batch does not error, we will just finish the records (and not error them).
2491
2492v0.6.12
2493===
2494
2495This is a small release corresponding with Kafka's 2.8.0 release.
2496A few small things were fixed when trying out `kcl misc probe-versions`
2497with the new code, and with that, we are able to determine exactly
2498where to cut 2.8.0 from tip in the kversion package.
2499
2500Lastly, this also converts topic IDs that are introduced in this release
2501from `[2]uint64` to `[16]byte`. A raw blob of bytes is easier to reason
2502about and affords us avoiding worrying about endianness.
2503
2504v0.6.11
2505===
2506- [`46138f7`](https://github.com/twmb/franz-go/commit/46138f7): **feature** client: add ConnIdleTimeout option && connection reaping (further fix to this in later commit)
2507- [`26c1ea2`](https://github.com/twmb/franz-go/commit/26c1ea2): generate: return "Unknown" for unknown NameForKey
2508- [`557e15f`](https://github.com/twmb/franz-go/commit/557e15f): **module change** update module deps; bump lz4 to v4. This dep update will require go-getting the new v4 module.
2509- [`10b743e`](https://github.com/twmb/franz-go/commit/10b743e): producer: work around KAFKA-12671
2510- [`89bee85`](https://github.com/twmb/franz-go/commit/89bee85): **bugfix** consumer: fix potential slowReloads problem
2511- [`111f922`](https://github.com/twmb/franz-go/commit/111f922): consumer: avoid bubbling up retriable broker errors
2512- [`fea3195`](https://github.com/twmb/franz-go/commit/fea3195): **bugfix** client: fix fetchBrokerMetadata
2513- [`6b64728`](https://github.com/twmb/franz-go/commit/6b64728): **breaking API**: error redux -- this makes private many named errors; realistically this is a minor breaking change
2514- [`b2a0578`](https://github.com/twmb/franz-go/commit/b2a0578): sink: allow concurrent produces (this was lost in a prior release; we now again allow 5 concurrent produce requests per broker!)
2515- [`ebc8ee2`](https://github.com/twmb/franz-go/commit/ebc8ee2): **bugfix / improvements** producer: guts overhaul, fixing sequence number issues, and allowing aborting records when aborting transactions
2516- [`39caca6`](https://github.com/twmb/franz-go/commit/39caca6): Poll{Records,Fetches}: quit if client is closed
2517- [`56b8308`](https://github.com/twmb/franz-go/commit/56b8308): **bugfix** client: fix loadCoordinators
2518- [`d4fe91d`](https://github.com/twmb/franz-go/commit/d4fe91d): **bugfix** source: more properly ignore truncated partitions
2519- [`71c6109`](https://github.com/twmb/franz-go/commit/71c6109): **bugfix** consumer group: map needs to be one block lower
2520- [`009e1ba`](https://github.com/twmb/franz-go/commit/009e1ba): consumer: retry reloading offsets on non-retriable errors
2521- [`6318b15`](https://github.com/twmb/franz-go/commit/6318b15): **bugfix** group consumer: revoke all on LeaveGroup, properly blocking commit
2522- [`b876c09`](https://github.com/twmb/franz-go/commit/b876c09): **bugfix** kversion: KIP-392 landed in Kafka 2.3, not 2.2
2523- [`1c5af12`](https://github.com/twmb/franz-go/commit/1c5af12): kversion: support guessing with skipped keys, guessing raft
2524
2525This is an important release with multiple bug fixes. All important commits
2526are noted above; less important ones are elided.
2527
2528The producer code has been overhauled to more accurately manage sequence
2529numbers. Particularly, we cannot just reset sequence numbers whenever. A
2530sequence number reset is a really big deal, and so we should avoid it at all
2531costs unless absolutely necessary.
2532
2533By more accurately tracking sequence numbers, we are able to abort buffered
2534records when aborting a transact session. The simple approach we take here is
2535to reset the producer ID after the abort is completed.
2536
2537The most major consumer bug was one that incorrectly tracked uncommitted
2538offsets when multiple topics were consumed in a single group member. The other
2539bugs would be infrequently hit through some more niche edge cases / timing
2540issues.
2541
2542The lz4 module was updated, which will require a go mod edit. This was done
2543because major-version-bumping is pretty undiscoverable with mod updates, and it
2544turns out I had always used v1 instead of v4.
2545
2546Sometime in the past, a broker/producer bugfix accidentally lost the ability to
2547send 5 concurrent produce request at once. That ability is readded in this
2548release, but this can occasionally trigger a subtle problem within Kafka while
2549transactionally producing. For more details, see [KAFKA-12671](https://issues.apache.org/jira/browse/KAFKA-12671).
2550The client attempts to work around this problem by sleeping for 1s if it detects
2551the problem has a potential of happening when aborting buffered records. Realistically,
2552this problem should not happen, but the one way to easily detect it is to look
2553for a stuck LastStableOffset.
2554
2555A `ConnIdleTimeout` option was added that allows for reaping idle connections.
2556The logic takes a relatively easy approach, meaning idle connections may take
2557up to 2x the idle timeout to be noticed, but that should not be problematic.
2558This will help prevent the problem of Kafka closing connections and the client
2559only noticing when it goes to eventually write on the connection again. This
2560will also help avoid sockets in the kernel being stuck in `CLOSE_WAIT`.
2561
2562Finally, after being inspired from [KAFKA-12675](https://issues.apache.org/jira/browse/KAFKA-12671),
2563I have significantly optimized the sticky assignor, as well as added other
2564improvements for the cooperative adjusting of group balancer plans.
2565
2566v0.6.10
2567===
2568
2569- [`04f8e12`](https://github.com/twmb/franz-go/commit/04f8e12): update debug logging (broker and consumer group)
2570- [`62e2e24`](https://github.com/twmb/franz-go/commit/62e2e24): add definitions & code for control record keys and values
2571- [`e168855`](https://github.com/twmb/franz-go/commit/e168855): **bugfix** sticky: fix and drastically simplify isComplex detection
2572- [`bd5d5ad`](https://github.com/twmb/franz-go/commit/bd5d5ad): **bugfix** consumer: properly shut down manageFetchConcurrency
2573- [`a670bc7`](https://github.com/twmb/franz-go/commit/a670bc7): group balancer: debug => info logging; handle join better
2574- [`d74bbc3`](https://github.com/twmb/franz-go/commit/d74bbc3): **bugfix** group: avoid concurrently setting err; use better log message
2575- [`ca32e19`](https://github.com/twmb/franz-go/commit/ca32e19): **feature** consumer: add PollRecords to poll a limited number of records
2576- [`2ce2def`](https://github.com/twmb/franz-go/commit/2ce2def): txn: change what we set offsets to on revoke / error
2577- [`582335d`](https://github.com/twmb/franz-go/commit/582335d): txn: add PollFetches and Produce to GroupTransactSession
2578- [`c8cd120`](https://github.com/twmb/franz-go/commit/c8cd120): txn: document that canceling contexts is not recommended
2579- [`bef2311`](https://github.com/twmb/franz-go/commit/bef2311): enforce KIP-98 idempotency config rules: acks=all, retries > 0
2580- [`2dc7d3f`](https://github.com/twmb/franz-go/commit/2dc7d3f): **bugfix** client.EndTransaction: if offsets were added to transaction, ensure we commit
2581- [`938651e`](https://github.com/twmb/franz-go/commit/938651e): **bugfix** consumer & consumer group: small redux, bug fixes
2582- [`b531098`](https://github.com/twmb/franz-go/commit/b531098): **feature** add support for ListTransactions, minor DescribeTransactions change
2583- [`3e6bcc3`](https://github.com/twmb/franz-go/commit/3e6bcc3): **feature** client: add option to disable idempotent writes
2584- [`5f50891`](https://github.com/twmb/franz-go/commit/5f50891): **feature**: kgo.Fetches: add EachPartition callback iterator
2585- [`05de8e6`](https://github.com/twmb/franz-go/commit/05de8e6): **bugfix** use proper keyvals for join group error
2586- [`8178f2c`](https://github.com/twmb/franz-go/commit/8178f2c): group assigning: add debug logs for balancing plans
2587- [`e038916`](https://github.com/twmb/franz-go/commit/e038916): **bugfix** range balancer: use correct members for div/rem
2588- [`e806126`](https://github.com/twmb/franz-go/commit/e806126): **bugfix** (minor) broker: shift sizeBuf 1 left by 8 for guessing tls version
2589- [`cd5e7fe`](https://github.com/twmb/franz-go/commit/cd5e7fe): client: rewrite `*kmsg.ProduceRequest`'s Acks, TimeoutMillis
2590- [`8fab998`](https://github.com/twmb/franz-go/commit/8fab998): kgo: use a discard goroutine when produce acks is 0
2591
2592This is an important release that fixes various bugs in the consumer code and
2593adds a few nice-to-have features. Particularly, this includes a minor redux of
2594the consumer group code that audited for correctness, and includes a few
2595transactional consumer fixes.
2596
2597There is a known problem in sequence numbers in the producer code that will be
2598hit if any consumer resets its sequence numbers. This happens if a consumer
2599hits a produce timeout or hits the retry limit, and may happen if
2600`AbortBufferedRecords` is called. A fix for this is being worked on now and
2601will be in the next release.
2602
2603Most commits since v0.6.9 are noted above, minus a few that are left out for
2604being very minor documentation updates or something similar.
2605
2606v0.6.9
2607===
2608
2609- [`0ca274c`](https://github.com/twmb/franz-go/commit/0ca274c): **bugfix** consumer group: fix race where we did not wait for a revoke to finish if the group closed
2610- [`0cdc2b6`](https://github.com/twmb/franz-go/commit/0cdc2b6): **bugfix** consumer: process FetchResponse's RecordBatches correctly (versions after v0.11.0 can use message sets)
2611- [`bcb330b`](https://github.com/twmb/franz-go/commit/bcb330b): kerr: add TypedErrorForCode and four new errors
2612- [`31a9bbc`](https://github.com/twmb/franz-go/commit/31a9bbc): **bugfix** (that is not hit by general consumers): FetchResponse's DivergingEpoch.EndOffset is an int64, not int32
2613
2614This release fixes some problems with consuming. As it turns out, message sets,
2615which are unused after v0.11.0, are indeed returned with higher FetchResponse
2616versions. Thus, we cannot rely on the FetchResponse version alone to decide
2617whether to decode as a message set or a record batch, instead we must check the
2618magic field.
2619
2620The `DivergingEpoch.EndOffset` bug likely would not be noticed by general
2621consumers, as this is a field set by leaders to followers and is not useful to
2622nor set for clients. As well, it's only used for the not-yet-finalized raft work.
2623
2624I noticed a race while looping through integration tests in `-race` mode, this
2625was an easy fix. Practically, the race can only be encountered when leaving a
2626group if your revoke is slow and the timing races exactly with the manage loop
2627also revoking. This would mean your onRevoke could be called twice. Now, that
2628will not happen. This bug itself is basically a reminder to go through and
2629audit the consumer group code, one of the last chunks of code I intend to
2630potentially redux internally.
2631
2632v0.6.8
2633===
2634
2635- [`8a42602`](https://github.com/twmb/franz-go/commit/8a42602): **bugfix** kversion: avoid panic on a key with value of max int16 (32767)
2636
2637This is a small bugfix release; Confluent replies with odd keys sometimes,
2638turns out this time it was max int16. That showed that the parentheses was in
2639the wrong place for SetMaxKeyVersions.
2640
2641This also has one more commit that change the String output a little bit to use
2642"Unknown" instead of an empty string on unknown keys.
2643
2644v0.6.7
2645===
2646
2647- [`2bea568`](https://github.com/twmb/franz-go/commit/2bea568): **bugfix** producer: fix producing with NoAck
2648- [`d1ecc7b`](https://github.com/twmb/franz-go/commit/d1ecc7b): On invalidly broker large reads, guess if the problem is a tls alert on a plaintext connection
2649
2650This is a small bugfix release to fix producing with no acks. As it turns out,
2651Kafka truly sends _no_ response when producing with no acks. Previously, we
2652would produce, then hang until the read timeout, and then think the entire
2653request failed. We would retry in perpetuity until the request retry limit hit,
2654and then we would fail all records in a batch.
2655
2656This fixes that by immediately finishing all promises for produce requests if
2657we wrote successfully.
2658
2659v0.6.6
2660===
2661
2662- [`21ddc56`](https://github.com/twmb/franz-go/commit/21ddc56): kgo: sort `RequestSharded` by broker metadata before returning
2663- [`4979b52`](https://github.com/twmb/franz-go/commit/4979b52): **bugfix** consumer: ensure we advance when consuming compacted topics
2664
2665This is a small bugfix release to fix a stuck-consumer bug on compacted topics.
2666
2667v0.6.5
2668===
2669
2670- [`d5b9365`](https://github.com/twmb/franz-go/commit/d5b9365): configuration: clamp max partition bytes to max bytes to work around faulty providers
2671- [`c7dbafb`](https://github.com/twmb/franz-go/commit/c7dbafb): **bugfix** consumer: kill the session if the response is less than version 7
2672- [`f57fc76`](https://github.com/twmb/franz-go/commit/f57fc76): **bugfix** producer: handle ErrBrokerTooOld in doInitProducerID, allowing the producer to work for 0.10.0 through 0.11.0
2673
2674This is a small release to ensure that we can make progress when producing to
2675Kafka clusters v0.10.0 through v0.11.0 (as well as that we can consume from
2676them if the consume has a partition error).
2677
2678v0.6.4
2679===
2680
2681- [`802bf74`](https://github.com/twmb/franz-go/commit/802bf74): **bugfix** kgo: fix three races
2682- [`1e5c11d`](https://github.com/twmb/franz-go/commit/1e5c11d): kgo: Favor non-seeds when selecting brokers for requests that go to a random broker
2683- [`4509d41`](https://github.com/twmb/franz-go/commit/4509d41): kgo: Add `AllowedConcurrentFetches` option to allow bounding the maximum possible memory consumed by the client
2684- [pr #22](https://github.com/twmb/franz-go/pull/22): Add transactional producer / consumer example (thanks [@dcrodman](https://github.com/dcrodman)!)
2685- [`6a041a8`](https://github.com/twmb/franz-go/commit/6a041a8): Add explicit Client.LeaveGroup method
2686- [`fe7d976`](https://github.com/twmb/franz-go/commit/fe7d976): KIP-500 / KIP-631: add support for admin-level request DecommissionBroker
2687- [`ab66776`](https://github.com/twmb/franz-go/commit/ab66776): KIP-500 / KIP-631: add support for broker-only requests BrokerRegistration and BrokerHeartbeat
2688- [`31b1df7`](https://github.com/twmb/franz-go/commit/31b1df7): KIP-664: add support for DescribeProducers
2689- [`368bb21`](https://github.com/twmb/franz-go/commit/368bb21): **breaking kmsg protocol changes**: add support for KIP-516 CreateTopics and DeleteTopics bumps
2690- [`b50282e`](https://github.com/twmb/franz-go/commit/b50282e): kerr: ReplicaNotAvailable is retriable; add UnknownTopicID
2691- [`c05572d`](https://github.com/twmb/franz-go/commit/c05572d): kmsg: change (Request|Response)ForKey to use proper NewPtr functions
2692- [`360a4dc`](https://github.com/twmb/franz-go/commit/360a4dc): client: actually use ConnTimeoutOverhead properly; bump base to 20s
2693- [`0ff08da`](https://github.com/twmb/franz-go/commit/0ff08da): Allow list offsets v0 to work (for Kafka v0.10.0 and before)
2694- [`59c935c` through `c7caea1`](https://github.com/twmb/franz-go/compare/59c935c..c7caea1): fix fetch session bugs
2695- [pr #4](https://github.com/twmb/franz-go/pull/4): Redesign readme (thanks [@weeco](https://github.com/weeco)!)
2696
2697Of note, this fixes three races, fixes fetch session bugs and has small
2698breaking protocol changes in kmsg.
2699
2700For the three races, two of them are obvious in hindsight, but one makes no
2701sense and may not belong in franz-go itself. The fix for the third is to remove
2702a line that we do not need to do anyway.
2703
2704For fetch sessions, sessions were not reset properly in the face of context
2705cancelations or across consumer topic reassignments. It was possible for fetch
2706sessions to get out of sync and never recover. This is fixed by resetting fetch
2707sessions in all appropriate places.
2708
2709For kmsg protocol changes, DeleteTopics changed Topics from a list of strings
2710(topics to delete) to a list of structs where you could delete by topic or by
2711topic uuid. The Java code coincidentally historically used TopicNames, so it
2712did not need to break the protocol to add this new struct under the Topics
2713name. For lack of confusion, we change Topics to TopicNames and introduce the
2714struct under Topics.
2715
2716v0.6.3
2717===
2718
2719- [`db09137`](https://github.com/twmb/franz-go/commit/db09137): Add support for new DescribeCluster API (KIP-700); followup commit (`ab5bdd3`) deprecates `Include<*>AuthorizedOperations` in the metadata request.
2720- [`3866e0c`](https://github.com/twmb/franz-go/commit/3866e0c): Set Record.Attrs properly before calling Produce callback hook.
2721- [`05346db`](https://github.com/twmb/franz-go/commit/05346db) and [`3d3787e`](https://github.com/twmb/franz-go/commit/3d3787e): kgo: breaking change for kversions breaking change two commits ago (see two lines below)
2722- [`b921c14`](https://github.com/twmb/franz-go/commit/b921c14): doc fix: v0.10.0 changed produce from MessageSet v0 to MessageSet v1
2723- [`33a8b26`](https://github.com/twmb/franz-go/commit/33a8b26): Breaking pointerification change to kversions.Versions; introduce VersionGuess
2724
2725Two small breaking changes related to kversions.Versions in this; switching to
2726pointers now makes the struct future compatible; if it grows, we avoid the
2727concern of passing around a large struct on the stack.
2728
2729Contains one small fix related to KIP-360 w.r.t. the idempotent producer: when
2730we encounter OutOfOrderSequenceNumber, we need to reset _all_ partition's
2731sequence numbers, not just the failing one. This is now done by resetting
2732everything the next time the producer ID is loaded. The related Kafka ticket is
2733KAFKA-12152.
2734
2735
2736v0.6.2
2737===
2738
2739- [`9761889`](https://github.com/twmb/franz-go/commit/9761889): bump dependencies
2740- [`d07538d`](https://github.com/twmb/franz-go/commit/d07538d) Allow clients to cancel requests with the passed context; fix potential hanging requests
2741- [`303186a`](https://github.com/twmb/franz-go/commit/303186a) Add BrokerThrottleHook (thanks [@akesle](https://github.com/akesle))
2742- [`63d3a60`](https://github.com/twmb/franz-go/commit/63d3a60) and previous commits: Use new & better kmsg.ThrottleResponse; this removes the usage of reflect to check for a ThrottleMillis field
2743- [`b68f112`](https://github.com/twmb/franz-go/commit/b68f112) kerr: add description to error string
2744- [`7cef63c`](https://github.com/twmb/franz-go/commit/7cef63c) config: strengthen validation
2745- [`faac3cc`](https://github.com/twmb/franz-go/commit/faac3cc) producing: fail large messages more correctly (bugfix)
2746- [`bc1f2d1`](https://github.com/twmb/franz-go/commit/bc1f2d1) and [`2493ae7`](https://github.com/twmb/franz-go/commit/2493ae7): kversion: switchup Versions API, finalize 2.7.0
2747- [`991e7f3`](https://github.com/twmb/franz-go/commit/991e7f3): Add support for KIP-630's new FetchSnapshot API
2748- [`de22e10`](https://github.com/twmb/franz-go/commit/de22e10): and [`9805d36`](https://github.com/twmb/franz-go/commit/9805d36) and [`c1d9ff4`](https://github.com/twmb/franz-go/commit/c1d9ff4): Add Uuid support and some definitions for KIP-516 (topic IDs)
2749- [`539b06c`](https://github.com/twmb/franz-go/commit/539b06c): Completely rewrite the consumer.go & source.go logic and fix sink & source broker pointer issues
2750- [`385cecb`](https://github.com/twmb/franz-go/commit/385cecb): Adds a warning hint if the connection dies on the first read without sasl
2751- [`71cda7b`](https://github.com/twmb/franz-go/commit/71cda7b): Bump another batch of requests to be flexible (per KAFKA-10729 / 85f94d50271)
2752- [`9d1238b8e`](https://github.com/twmb/franz-go/commit/9d1238b8ee28ad032e0bc9aa4891e0fbcdd27a63) and [`8a75a8091`](https://github.com/twmb/franz-go/commit/8a75a80914500a7f9c9e0edc4e5aefde327adf45): Fix misleading doc comments on DescribeGroupsRequest and FindCoordinatorResponse
2753- [`7f30228b8`](https://github.com/twmb/franz-go/commit/7f30228b87d4f883ba5c344048201dfb4d90336e): Adds FetchMinBytes option
2754- [`56ab90c72`](https://github.com/twmb/franz-go/commit/56ab90c7273be5ccc40dc38ff669b36995c505ce): Support EnvelopeRequest & DefaultPrincipalBuilder for KIP-590
2755- [`37d6868fc`](https://github.com/twmb/franz-go/commit/37d6868fc474813a6fb1e814b1a7fd87cc34d8ee): Add producer ID and epoch to the Record struct
2756- [`9467a951d`](https://github.com/twmb/franz-go/commit/9467a951d71e511c28180239336093225082896c): Add explicit sharded requests and retriable broker specific requests (followup commit settles API naming; later commits fix small problems in initial implementation)
2757- [`b94b15549`](https://github.com/twmb/franz-go/commit/b94b155497b4216808236aa393f06e7dccd772ed): Add kmsg constants for enums
2758- [`7ac5aaff8`](https://github.com/twmb/franz-go/commit/7ac5aaff8cea693e6a57533c8e18cca75c17ecc0): Add a MinVersions option
2759- [`3b06d558b`](https://github.com/twmb/franz-go/commit/3b06d558ba7453a191f434ce5654853322742859): Add a max broker read bytes option
2760- [`4c5cf8422`](https://github.com/twmb/franz-go/commit/4c5cf84223401eb1f5f6a78671dc04ff51407bd3): Account for throttling in responses
2761
2762The primary commit of this release is `539b06c`, which completely rewrites the
2763consumer.go and source.go files to be more easily understandable. This commit
2764also fixes a bug related to the `sink` and `source` broker pointers; if the
2765broker changed but kept the same node ID, the `sink` and `source` would not
2766have their old pointers (to the old broker) updated.
2767
2768There were some small breaking API changes in the `kmsg` package, most notably,
2769the metadata request changed one field from a `string` to a `*string` due to
2770Kafka switching the field from non-nullable to nullable. As specified in the
2771`kmsg` docs, API compatibility is not guaranteed due to Kafka being able to
2772switch things like this (and I favor small breakages like this rather than
2773wrapping every type into a struct with a `null` field).
2774
2775The `kversion` package has had its `Versions` type completely changed. The new
2776format should be future compatible for new changes. The current change was
2777required due to Kafka's `ApiVersions` having a gap in keys that it responds
2778with in 2.7.0.
2779
2780Config validation has been strengthened quite a bit.
2781
2782Includes a few other minor, not too noteworthy changes.
2783
2784v0.6.1
2785===
2786
2787- [`0ddc468b2`](https://github.com/twmb/franz-go/commit/0ddc468b2ceca4fc0e42206c397f6d7ccd74bc4b): Workaround for two minor EventHubs issues.
2788
2789This is a small release to patch around two minor EventHubs protocol serializing issues.
2790
2791v0.6.0
2792===
2793
2794- [`0024a93a4`](https://github.com/twmb/franz-go/commit/0024a93a4d386318869fe029260f3e23a602c03e): Fix bug introduced in pkg/kgo/source.go
2795- [`ee58bc706`](https://github.com/twmb/franz-go/commit/ee58bc7065e50a40207a730afc6dfc14b47b1da1): Avoid noise when closing the client if we were not consuming
2796- [`6e0b042e3`](https://github.com/twmb/franz-go/commit/6e0b042e3ad2281b82030375257f0c200bc511c3): Pin the client by default to the latest stable API versions
2797
2798This release also changes the name to franz-go. After sitting on frang for a
2799week, ultimately I was missing the fact that the name related to Kafka. `franz`
2800is an easy enough mental map, and then this brings back the `-go` suffix.
2801
2802v0.5.0
2803===
2804
2805- [`9dc80fd29`](https://github.com/twmb/franz-go/commit/9dc80fd2978e8fb792f669ec6595cc5dadfc9fa5): Add hooks, which enable metrics to be layered in.
2806- [`a68d13bcb`](https://github.com/twmb/franz-go/commit/a68d13bcb4931870c9ca57abc085aa937b47373a): Set SoftwareName & SoftwareVersion on ApiVersions request if missing
2807- [`5fb0a3831`](https://github.com/twmb/franz-go/commit/5fb0a3831f56e444497131674b582113344704db): Fix SASL Authenticate
2808- [`85243c5c5`](https://github.com/twmb/franz-go/commit/85243c5c5aee1e23f6d360efdc82480fd83fcfc3): Breaking API change for enums in ACL & config managing request / response types
2809- [`4bc453bc2`](https://github.com/twmb/franz-go/commit/4bc453bc237bd534947cebdc5bc72f8bd7e855d9): Add NewPtr functions for generated structs in kmsg
2810- [`dcaf32daa`](https://github.com/twmb/franz-go/commit/dcaf32daa16fdc20e1c9cf91847b74342d13e36f): Generate RequestWith function for all requests in kmsg
2811- [`c27b41c3e`](https://github.com/twmb/franz-go/commit/c27b41c3e85df89651d9634cd024a2114f273cbd): Add Requestor interface to kmsg package
2812- [`2262f3a8f`](https://github.com/twmb/franz-go/commit/2262f3a8f90db2f459203b8ae665b089c19901d2): Add New functions for generated structs in kmsg
2813- [`7fc3a701e`](https://github.com/twmb/franz-go/commit/7fc3a701ec2155417cc02d2535510fcb5c9cdb6d): Add UpdateFeatures for 2.7.0 (KIP-584)
2814- [`f09690186`](https://github.com/twmb/franz-go/commit/f09690186ce2697f78626ba06e7ccfb586f1c7ee): Add shortcuts to get sasl.Mechanisms from sasl structs
2815- [PR #12](https://github.com/twmb/franz-go/pull/12): Add clarifying wording on NewClient (thanks [@weeco](https://github.com/weeco)!)
2816- [`9687df7ad`](https://github.com/twmb/franz-go/commit/9687df7ad71c552c61eea86376ebf3a9f6bff09e): Breaking API change for kgo.Dialer
2817- [`aca99bcf1`](https://github.com/twmb/franz-go/commit/aca99bcf19e741850378adbfe64c62b009340d7d): New APIs for KIP-497 (alter isr)
2818- [`a5f66899f`](https://github.com/twmb/franz-go/commit/a5f66899f6b492de37d689566d34869f50744717): New APIs for KIP-595 (raft)
2819- [`cd499f4ea`](https://github.com/twmb/franz-go/commit/cd499f4eaceaa0bd73452b5737c7713fe2b60ca9): Fetch request changes for KIP-595 (raft)
2820- [`7fd87da1c`](https://github.com/twmb/franz-go/commit/7fd87da1c8562943b095a54b5bc6258e2d2bdc6c): Support defaults in generated code (and a few commits after adding defaults)
2821- [`74c7f57e8`](https://github.com/twmb/franz-go/commit/74c7f57e81dd45033e9eec3bacbc4dca75fef83d): Support for KIP-584 (ApiVersions changes for min/max features)
2822
2823Multiple generator changes were made to better handle tags, add support for
2824enums, and parse many files in a `definitions` directory. Minor changes were
2825made in kgo to use the new `RequestWith` function. More debug & warn log lines
2826were added around opening a connection and initializing sasl.
2827
2828This release also changed the name of the repo from "kafka-go" to "frang" to
2829avoid potential trademark issues down the road.
2830
2831v0.4.9
2832======
2833
2834Typo fix for the prior release (in 2.7.0 APIs).
2835
2836v0.4.8
2837======
2838
2839- [`2e235164d`](https://github.com/twmb/franz-go/commit/2e235164daca64cafc715642582b13424e494e5c): Further prepare for KIP-588 (recover on txn timed out)
2840- [`60de0dafb`](https://github.com/twmb/franz-go/commit/60de0dafbeb7bf72996ad5f24690820962d2f584): Further audit for KIP-360 correctness (recoverable producer id)
2841- [`2eb27c653`](https://github.com/twmb/franz-go/commit/2eb27c653a4cdb815bf366894a2b87a3555ee50b): client.Request: shard DeleteRecords, {AlterReplica,Describe}LogDirs
2842- [`ebadfdae1`](https://github.com/twmb/franz-go/commit/ebadfdae1d975a5a937a1f1d67fd909b728c7386): Add support for KIP-554 (SCRAM APIs)
2843- [`4ea4c4297`](https://github.com/twmb/franz-go/commit/4ea4c4297e0402fcf37ef913f4a161203ff83dd4): Un-admin a lot of requests and fix OffsetDelete to be a group request
2844
2845v0.4.7
2846======
2847
2848- [`598261505`](https://github.com/twmb/franz-go/commit/598261505033d0255c37dc06b9b6c1112818a1be): Breaking API change in kmsg to add validating & erroring when reading records/messages
2849
2850v0.4.6
2851======
2852
2853- Introduces the CHANGELOG file
2854- [`ff1dc467e`](https://github.com/twmb/franz-go/commit/ff1dc467e32b6be41656c1f3bc57cb4d45e32a0c): Breaking API change in kmsg to support future non-breaking changes
2855- [`4eb594044`](https://github.com/twmb/franz-go/commit/4eb594044cfc611b75352530f7122c596b15764c): Correctly copies over Version/ThrottleMillis when merging ListGroups
View as plain text