CHANGELOG.md 7.92 KB
Newer Older
1
2
3
# Changelog
All notable changes to this project will be documented in this file.

4
## Unreleased
5
6
- ``deque-scan-back`` was returning wrong cursor to signal end of iteration: ``next:0``
while it should have been ``0``.
7
8
- A race condition was sometimes causing elections to fail spuriously, making
the election of a stable leader to require slightly more rounds than it should have.
9
- Implementation of health indicators through ``QUARKDB-HEALTH`` command.
10
11
- Added support for RESPv3 push types, activated on a per-client basis through
``ACTIVATE-PUSH-TYPES`` command.
12
- Add convenience command ``DEQUE-CLEAR``.
13
14
- Add support for ``MATCHLOC`` in ``LHSCAN``, used to filter out results based
on locality hint.
15
16
17
18
- Protection for a strange case of corruption which brought down a development
test cluster. (last-applied jumped ahead of commit-index by 1024, causing all
writes to stall). From now on, similar kind of corruption should only take out
a single node, and not spread to the entire cluster.
19
20
- Add command ``RAFT-JOURNAL-SCAN`` to make searching through the contents of the
raft journal easier.
21
- ``KEYS`` is now implemented in terms of ``SCAN``, making prefix matching of the
22
keyspace just as efficient as with ``SCAN``. (Note: The use of ``KEYS`` is still
23
generally discouraged due to potentially huge response size)
24
25
- Add ``RECOVERY-SCAN`` command for scanning through complete keyspace, including
internal rocksdb keys.
26
- Add tool ``quarkdb-sst-inspect`` to allow low-level inspection of SST files.
27
- Removed unused tool ``quarkdb-scrub``.
28

29
30
31
## 0.3.8 (2019-05-27)
- Prevent elections from hanging on the TCP timeout when one of the member hosts
is dropping packets, which could bring down an otherwise healthy cluster.
32
33
- Prevent crashing when ``LHSCAN`` is provided with a cursor missing the field
component.
34
35
- Make request statistics available through ``command-stats`` command.
- Addition of configuration file path to ``quarkdb-info``.
36
- Print simple error message when the given path to quarkdb-create already exists,
37
instead of a stacktrace.
38

39
40
41
42
43
## 0.3.7 (2019-04-24)
- Heavy use of lease commands could cause performance degradation and latency
spikes, due to accumulation of long-lived deletion tombstones on expiration
events. Starting from this release, such tombstones should disappear much
more quickly without accumulating.
44
45
46
- Fixed regression introduced in 0.3.6, which made it impossible to create new
bulkload nodes. (Note: to workaround, delete directory ``/path/to/bulkload/current/state-machine``
right after running ``quarkdb-create``)
47
48
49
50
51
- Addition of ``LEASE-GET-PENDING-EXPIRATION-EVENTS`` command to list all
pending lease expiration events, and ``RAW-SCAN-TOMBSTONES`` to inspect tombstones
as an aid in debugging.
- Expanding a cluster is now easier, no need to pass ``--nodes`` to quarkdb-create
when creating a new node for an established cluster.
52

53
54
## 0.3.6 (2019-03-21)
- Improved memory management and recycling, putting less pressure on the global
55
56
57
58
memory allocator.
- Addition of pub/sub support, commands implemented: ``PUBLISH``, ``SUBSCRIBE``,
  ``PSUBSCRIBE``, ``UNSUBSCRIBE``, ``PUNSUBSCRIBE``
- Addition of ``--steal-state-machine`` flag to ``quarkdb-create`` to make
59
  transition out of bulkload mode easier and less error-prone.
Georgios Bitzes's avatar
Georgios Bitzes committed
60
- Updated rocksdb to v5.18.3.
61

62
63
64
65
66
67
## 0.3.5 (2018-11-28)
- Updated rocksdb dependency to v5.17.2.
- Improved output of backup command ``raft-checkpoint``. The generated directory
can be used directly to spin-up a full QuarkDB node, without manual tinkering.
Command aliased to ``quarkdb-checkpoint``.
- Removed flood of ``attempting connection to .. `` messages when some node is unavailable.
68
69
- Light refactoring, more widespread use of ``std::string_view``, which paves the
way for certain performance optimizations in the future.
70
- Fixed several flaky tests.
Georgios Bitzes's avatar
Georgios Bitzes committed
71

72
## 0.3.4 (2018-10-09)
73
- Updated rocksdb dependency to v5.15.10.
Georgios Bitzes's avatar
Georgios Bitzes committed
74
- Added `TYPE` command.
75
76
- A read-only MULTI immediatelly after a read-write MULTI could
  cause the cluster to crash.
77
78
79
- Added command `LHSCAN` for scanning through a locality hash.
- Added convenience command in recovery mode for performing forced membership updates.
- It's now possible to issue one-off commands from the recovery tool, without setting up a server.
80

81
82
83
84
85
86
87
88
## 0.3.3 (2018-09-14)

### Added
- Ability to run QuarkDB in raft mode with only a single node. This allows starting
with a single node, and growing the cluster in the future if needed, which is
cumbersome to do in standalone mode.
- Added `quarkdb-version` command, which simply returns the current quarkdb version.
- Commands `deque-scan-back`, `deque-trim-front`.
89
90
91
92
93
94
95
96
97

### Changed
- Dropped list commands, as the underlying implementation makes it impossible to support all list commands found in official redis. (most notably `linsert`)

  Still, the old implementation makes for an excellent deque. Renamed list commands `lpush`, `lpop`, `rpush`, `rpop`, `rlen` to `deque-push-front`, `deque-pop-front`, `deque-push-back`, `deque-pop-back`, `deque-len`. The underlying data format did not change, only the command names.

  This makes it possible to implement lists properly in the future.

  No-one has been using the list operations, which gives us the opportunity to change the command names.
98
- Minor code and performance improvements.
99

100
## 0.3.2 (2018-08-14)
101
102
103
104
105
106
107
108
### Added
- Protection against 1-way network partitions, in which a cluster node
  is able to establish TCP connections to others, but the rest cannot do the same.

  This resulted in cluster disruption as the affected node would not be receiving
  heartbeats, but could still repeatedly attempt to get elected.

  From now on, a node which has been vetoed will abstain from starting election
109
110
111
  rounds until it has received fresh heartbeats since receiving that veto. This
  will prevent a 1-way network partitioned node from causing extensive cluster
  disruption.
112

113
114
115
### Fixed
- A couple of minor memory leaks.

Georgios Bitzes's avatar
Georgios Bitzes committed
116
## 0.3.1 (2018-08-03)
117
118
119
### Added
- Command `hclone` for creating identical copies of entire hashes.

120
121
122
### Fixed
- An `EXEC` when not inside a `MULTI` would cause a crash.

123
124
125
126
127
128
## 0.2.9 (2018-07-16)
### Added
- Commands `convert-string-to-int`, `convert-int-to-string` to convert between
  binary-string-encoded integers and human-readable-ASCII encoding. Meant for
  interactive use only, to make life easier during low-level debugging when
  needing to edit low-level rocksdb keys, where binary-encoded integers are used.
129

130
131
132
### Changed
- Refactoring of transactions, we no longer pack / unpack a transaction into a single request within the same node, saving
  CPU cycles.
133
- Explicitly block zero-sized strings when parsing the redis protocol, print appropriate warning.
134
135
136
137
138
139
140

### Fixed
- In certain cases, such as when redirecting or reporting unavailability for pipelined writes, fewer
  responses might be provided than expected, causing the client connection to hang. This did not affect
  QClient when redirects are active, as it would shut the connection down and retry upon reception of
  the first such response.

141
## 0.2.8 (2018-07-04)
142
143
144
145
146
147
148
149
### Added
- Support for leases, which can be used as locks with timeouts, allowing QuarkDB to serve as a distributed lock manager.
- Commands `lease-acquire`, `lease-get`, `lease-release`.

### Changed
- A newly elected leader now stalls writers in addition to readers, until its leadership marker entry in the raft journal has been committed and applied.

### Fixed
150
- Sockets and threads from closed connections were not being cleaned due to misunderstanding how XRootD handles connection shutdown. Each connection would hog a socket and a deadlocked thread on the server forever. (oops)
151
152
153
154
155
156
157
158
- A particularly rare race condition was able to trigger an assertion in the Raft subsystem, causing the current cluster leader to crash.

## 0.2.7 (2018-06-22)
### Added
- Updated rocksdb dependency to 5.13.4.

### Fixed
- Certain unlikely sequences of pipelined writes were able to trigger an assertion and bring a cluster down, when part of a transaction. Without that assertion, the commands would have left ghost key-value pairs in the rocksdb keyspace.