Git log: commit 005fbb6bb6fffafc7f077e704039c3c9b25786e3
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Thu Jan 27 11:45:43 2022 +0300
ci: add workflow to check commits on pull request
This commit adds a new script, tools/check-commits. The script takes git
revisions in the same format as git-rev-list (actually, it feeds all its
arguments to git-rev-list) and runs some checks on each of the commits.
For example, to check the head commit, run:
./tools/check-commits -1 HEAD
To check the last five commits, run:
./tools/check-commits HEAD~5..HEAD
Currently, there are only two checks, but in future we may add more
checks, e.g. check diffs for trailing spaces:
- The commit message contains a documentation request. Can be
suppressed with NO_DOC=<reason> in the commit message.
- A new changelog entry is added to changelog/unreleased by the commit.
Can be suppressed with NO_CHANGELOG=<reason> in the commit message.
This commit also adds a new workflow triggered on pull request,
lint/commits. The workflow runs the tools/check-commits script on all
the commits added by the pull request. The workflow doesn't run on push,
because it's problematic to figure out what commits are new on a branch.
Besides, we don't want to run it on push to release branches, because
it's a pure dev workflow.
Example output:
Checking commit a33f3cc7f1bf6020de95d144d63050894d2d0716 PASS
Checking commit 6f29f9d7725331d74d5e9a57a4922b1c3801ad50 FAIL
SHA: 6f29f9d7725331d74d5e9a57a4922b1c3801ad50
SUBJECT: iproto: introduce graceful shutdown protocol
ERROR:
Changelog not found in changelog/unreleased. If this commit
doesn't require changelog, please add NO_CHANGELOG=<reason>
to the commit message.
Checking commit fbc25aae895f46f9b078a0fce0ed47c651ad2f5e FAIL
SHA: fbc25aae895f46f9b078a0fce0ed47c651ad2f5e
SUBJECT: Update small submodule
ERROR:
Missing documentation request ('@TarantoolBot document' not
found in the commit message). If this commit doesn't need to
be documented, please add NO_DOC=<reason> to the commit
message.
ERROR:
Changelog not found in changelog/unreleased. If this commit
doesn't require changelog, please add NO_CHANGELOG=<reason>
to the commit message.
NO_DOC=ci
NO_CHANGELOG=ci
commit eadadbe8d8c5318a34c6883da9018ffedb1297d0
Author: Yaroslav Lobankov <y.lobankov@tarantool.org>
Date: Wed Jan 26 20:43:16 2022 +0300
ci: drop using testing docker image from GitLab
Now the tarantool/testing:debian-stretch image from Docker Hub is used.
Closes tarantool/tarantool-qa#136
commit ae4802774853b88b9950574b40f3e241939e0d5f
Author: Nick Volynkin <nick.volynkin@gmail.com>
Date: Wed Jan 26 17:11:39 2022 +0700
build: update URL for submodule third_party/luafun
commit a33f3cc7f1bf6020de95d144d63050894d2d0716
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Wed Jan 12 15:14:43 2022 +0300
net.box: support graceful shutdown protocol
Closes #5924
@TarantoolBot document
Title: Document graceful shutdown of net.box connections
If a Tarantool server supports the IPROTO graceful shutdown protocol
(`connection.peer_protocol_features` contains `graceful_shutdown` or
`peer_protocol_version` is >= 4), then when the server is asked to exit
(`os.exit()` is called on the server or `SIGTERM` signal is received),
it won't terminate net.box connections immediately. Instead here's what
will happen:
1. The server stops accepting new connections and sends a special
'shutdown' packet to all connection that support the graceful
shutdown protocol.
2. Upon receiving a 'shutdown' packet, a net.box connection executes
shutdown triggers. The triggers are installed by `on_shutdown()`
method of a net.box connection. The method follows the same protocol
as `on_connect()`, `on_disconnect()`, and `on_schema_reload()`.
Triggers are executed asynchronously in a new fiber. The connection
remains active while triggers are running so a trigger callback may
send new requests over the net.box connection.
3. After shutdown triggers have returned, the connection is switched to
`graceful_shutdown` state, in which all new requests fail with an
error. The connection will remain in this state until all requests
have been completed.
4. Once all in-progress requests have completed, the connection is
closed and switched to `error` or `error_reconnect` state, depending
on whether `reconnect_after` option is set.
5. Once all connections that support the graceful shutdown protocol are
closed, the server exits.
Note, the graceful shutdown protocol is best-effort: there's no
guarantee that the server doesn't exit before all active connections
are gracefully closed; the server may still exit on timeout or just be
killed. The timeout is configured by `box.ctl.set_on_shutdown_timeout()`
on a server.
Please also update the net.box state machine diagram:
```
initial -> auth -> fetch_schema <-> active
fetch_schema, active -> graceful_shutdown
(any state, on error) -> error_reconnect -> auth -> ...
\
-> error
(any state, but 'error') -> closed
```
commit 6f29f9d7725331d74d5e9a57a4922b1c3801ad50
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Tue Jan 11 18:31:27 2022 +0300
iproto: introduce graceful shutdown protocol
This commit adds the graceful shutdown feature to the IPROTO protocol.
Net.box is patched by the next commit, which also adds tests.
Part of #5924
@TarantoolBot document
Title: Document IPROTO graceful shutdown
A new IPROTO request type was introduced - `IPROTO_SHUTDOWN`, code 77.
When asked to shut down (`os.exit()` is called or `SIGTERM` signal is
received), a server stops accepting new connections and sends a packet
of this type to each of its clients that support the graceful shutdown
feature (see below how a server figures out if a client supports the
feature). The server won't exit until all the clients that were sent the
packets close connections. If all the clients don't close connections
within the shutdown timeout, the server will exit anyway. The default
shutdown timeout is 3 seconds, and it can be configured with
`box.ctl.set_on_shutdown_timeout()`, which also determines the timeout
of `box.ctl.on_shutdown()` triggers.
An `IPROTO_SHUTDOWN` packet doesn't have any keys in its headers (not
even sync number or schema version) nor a body. A client isn't supposed
to reply to an `IPROTO_SHUTDOWN` packet. Instead it's supposed to close
its connection as soon as possible. A client may wait for pending
requests to complete and even send new requests after receiving an
`IPROTO_SHUTDOWN` packet. The server will serve them as usual until it
exits on timeout.
Clients that support the graceful shutdown feature are supposed to set
the `IPROTO_FEATURE_GRACEFUL_SHUTDOWN` feature (bit 4) when sending an
`IPROTO_ID` request to a server. Servers that support the feature set
the same bit in reply to an `IPROTO_ID` request. Introduction of this
feature bumped the IPROTO protocol version up to 4.
commit fbc25aae895f46f9b078a0fce0ed47c651ad2f5e
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Mon Jan 17 17:46:43 2022 +0300
Update small submodule
Needed to fix rlist_foreach_entry ASAN runtime error in case
the rlist member is aligned.
commit b115dd9837c9c386db7fa8b0d8d89be6aea3ac1f
Author: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Date: Fri Jan 14 01:00:22 2022 +0100
election: activate raft split vote handling
Raft needs to know cluster size in order to detect and handle
split vote. The patch uses registered server count as cluster
size.
It is not documented nor has a changelog file because this is an
optimization. Can't be observed except in logs or with a watch.
Closes #5285
commit 3165e01b2ecc2a977e63c12c31c2ace5e523c411
Author: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Date: Fri Jan 14 00:59:19 2022 +0100
raft: introduce split vote detection
Split vote is a situation when during election nobody can win and
will not win in this term for sure. Not a single node could get
enough votes. For example, with 4 nodes one could get 2 votes and
other also 2 votes. Nobody will get quorum 3 in this term.
The patch makes raft able to notice that situation and speed up
the term bump. It is not bumped immediately though, because nodes
might do that simultaneously and will get the split vote again
after voting for self. There is a random delay. But it is just max
10% of election timeout, so it should speed up split vote
resolution on 90% at least.
Part of #5285
commit 6f05b0cecb1b796c44c32dc83df7462c76062f75
Author: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Date: Wed Jan 12 23:51:42 2022 +0100
raft: track all votes, even not own
To detect split vote a node needs to see that number of free votes
is not enough for anyone to win even if it gets them all.
Hence every node needs to count votes for all other nodes.
The patch makes raft store votes in a bit more complicated manner
than a simple bitmap for just the current instance.
Part of #5285
commit e51c61ae174ada09ea7b26b27e78fa059117e7b2
Author: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Date: Wed Jan 19 01:37:10 2022 +0100
raft: fix ev_timer.at incorrect usage
ev_timer.at was used as timeout. But after ev_timer_start() it
turns into the deadline - totally different value.
The patch makes sure ev_timer.at is not used in raft at all.
To test that the fakeev subsystem is patched to start its time not
from 0. Otherwise ev_timer.at often really matched the timeout
even for an active timer.
commit 82757e55e78885002a111fa2032b85ee840af443
Author: Vladislav Shpilevoy <v.shpilevoy@tarantool.org>
Date: Sat Jan 15 00:06:02 2022 +0100
raft: fix crash on election_timeout reconfig
It used to crash if done during election on a node voted for
anybody, it is a candidate, it doesn't know a leader yet, but has
a WAL write in progress.
Thus it could only happen if the term was bumped by a message from
a non-leader node and wasn't flushed to the disk yet.
The patch makes the reconfig check if there is a WAL write in
progress. Then don't do anything.
Could also check for volatile vote instead of persistent, but it
would create the same problem for the case when started writing
vote for self and didn't finish yet. Reconfig would crash.
commit d3d04b49d299fefda5b802831d6c7a90d66ca0e5
Author: artembo <artembo@me.com>
Date: Tue Jan 11 08:23:41 2022 +0300
github-ci: add fedora-35 workflows
Add GitHub Actions workflows for Fedora 35
Closes: #6692
commit 2b975e499a46086940af685852aa44ad4f4f520b
Author: Serge Petrenko <sergepetrenko@tarantool.org>
Date: Mon Dec 20 14:40:04 2021 +0300
test: fix replication/errinj potential flakiness
The test has a function wait_repl() which tries to wait until the data
is replicated to the other node for a tiny timeout (0.2 seconds)
This didn't cause issues in the past, but it became an issue after
applier in thread introduction.
Let's use the default test_run:wait_cond() approach instead of a custom
one. This way we increase waiting timeout (making the test pass with a
higher chance).
Follow-up #6329
commit d552a1ed22359a69261eb0ad6148b660b4ffae71
Author: Serge Petrenko <sergepetrenko@tarantool.org>
Date: Fri Dec 17 10:35:45 2021 +0300
test: fix replication/gh-5806-xlog-cleanup potential flakiness
Follow-up #6329
commit dacbf708f4a4cf0a130a4733982b69955c611718
Author: Serge Petrenko <sergepetrenko@tarantool.org>
Date: Mon Dec 6 04:56:16 2021 +0300
Introduce applier thread
It's reported that in master-slave setup replicas often have higher CPU
usage than their replication masters. Moreover, it's easy for a replica
to start falling behind the master.
The reason for that is the additional work load on replica's tx thread as
compared to master. While master typically splits request processing into 2
threads: iproto, which decodes the requests, and tx, which applies them,
replica performs both tasks in the tx thread.
This is due to replication architecture: replica handles master
connection by a single fiber in the tx thread. The fiber first decodes the
incoming requests and then applies them.
Teach replicas to decode the incoming replication stream in a separate
thread. This way tx thread doesn't waste processing time on row
decoding.
Each applier thread may serve several appliers, and the total number of
applier threads is controlled by a new configuration option -
`replication_threads`, with default value `1` (meaning, a single
thread is spawned to handle all the appliers.
Closes #6329
@TarantoolBot document
Title: New configuration option - `replication_threads`
It's now possible to specify how many threads will be spawned to decode
the incoming replication stream.
The default value is '1', meaning a single thread will handle all
incoming replication streams.
You may set the value to anything from 1 to 1000. When there are
multiple replication threads, connections to serve are evenly
distributed between the threads.
commit a7028dde8589a8119389d66e5dc55760b3af0019
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Thu Dec 23 18:52:02 2021 +0300
Add SSL iostream stub
This commit adds SSL context to iostream_ctx and a new kind of error for
SSL connections.
The SSL context is created if the target URI has transport=ssl
parameter. If transport=plain or absent, it will be set to NULL and
plain streams will be created as before. For other values of the
transport parameter, an error is raised. Note, this is just a stub -
an attempt to create an SSL context always currently fails with
"SSL not available" error. It is supposed to be implemented in
EE build.
The new kind of error is currently only used for decoding errors from
MsgPack and never raised. It will be raised by the actual implementation
of SSL stream.
commit 2734785ebec9a39ad58156390029e07e33012078
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Tue Dec 28 13:58:18 2021 +0300
box: make cfg value comparison more thorough
Use table.equals for comparing table-valued configuration parameters
instead of ipairs. Needed to trigger reconfiguraiton if the new URI
differs from the old one only by parameters, e.g.:
box.cfg{listen = {uri = 3301, params = {transport = 'plain'}}}
box.cfg{listen = {uri = 3301, params = {transport = 'ssl'}}}
No test is added, because currently there's no URI parameters handled
by box. The next commit adds the 'transport' parameter and a test that
wouldn't pass without this commit.
commit 26482e906a1f54ac37b893f31d51af08f48839f5
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Thu Dec 23 18:10:17 2021 +0300
iostream: introduce context and use it for creating streams
IO stream context (iostream_ctx) stores information needed to construct
an iostream object. A context is created from URI and mode, which is
either server or client. Currently, both parameters are ignored and a
newly created context always constructs plain streams. However, in
future we will create SSL streams for certain URIs. That's when we'll
need the mode (SSL connection is asymmetrical).
commit 0c4e6374d3673e1bf7c5c2d638d02223ec9b1bb9
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Thu Jan 13 11:06:28 2022 +0300
iostream: rename ctx to data
In SSL implementation iostream::ctx would point to a SSL object, while
a SSL_CTX object would be used for creating streams. Let's rename ctx to
data to avoid confusion.
commit a07f9143a7f2c1ac5840d5629b36dcd32ab0ad99
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Thu Dec 23 17:24:58 2021 +0300
box: inline cfg_get_replication
The function is used only once, in box_sync_replication. Inlining it
makes the code more robust - it allows us to throw from applier_new.
commit 7122f0f563dc9b83fc1b5201bbc28f1093011525
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Thu Dec 23 15:44:40 2021 +0300
evio: pass iostream instead of fd (and uri) to on_accept callback
Currently, we always create a plain stream, but soon there will be an
option to create an encrypted stream so let's pass iostream, which will
encapsulate this information, instead of fd to the accept callback. The
iostream will be created by evio in accordance with the settings passed
in uri parameters so uri won't be needed in the accept callback.
commit efee67ca96a3945e5dd069658ef9441357007fac
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Thu Dec 23 15:07:06 2021 +0300
coio: drop coio_service
It's been many years since it was last used. I'm tired of patching it
whenever I need to update evio_service. Besides, it decreases code
coverage. So let's finally drop it. After all, it's just a few lines
of trivial code that can be rewritten from scratch any time we need.
commit 34a7cfa8e57c328be9c6683cdc9f7dea8b8a7905
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Thu Dec 23 15:03:31 2021 +0300
Rename iostream_create to plain_iostream_create
The function creates a plain iostream so let's rename it accordingly.
commit 23a7ce80fb0a931d38a37741b2c84cee367aff68
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Fri Dec 24 13:10:00 2021 +0300
net.box: allow to pass table uri to connect
There are two acceptable ways of calling `connect` - as a method of
the net.box module or as a function defined in the module:
```lua
local net = require('net.box')
net.connect(...)
net:connect(...)
```
We determine which one is used by checking the type of the first
argument: if it's a table, we assume that the function is called as
a method. However, the first argument can be a table itself - in case
it's a URI given in the extended format.
Let's fix this check by comparing the first argument against the net.box
module. Let's also add a test checking all possible combinations of
`connect` arguments.
Follow-up 0badb4130deac46d6651cdcecb6c6d7ee401a96a ("net.box: pass uri
instead of host and port to state machine").
commit 197088c3af6439b2e2e6f94d97f5aed728a8c6a0
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Tue Jan 18 15:28:08 2022 +0300
test: fix flaky engine/errinj_ddl test
The commit fixes the following test failure:
```
[011] engine/errinj_ddl.test.lua memtx [ fail ]
[011]
[011] Test failed! Result content mismatch:
[011] --- engine/errinj_ddl.result Tue Jan 18 15:28:21 2022
[011] +++ var/rejects/engine/errinj_ddl.reject Tue Jan 18 15:28:26 2022
[011] @@ -343,7 +343,7 @@
[011] s:create_index('sk', {parts = {2, 'unsigned'}}) -- must fail
[011] ---
[011] - error: Duplicate key exists in unique index "sk" in space "test" with old tuple
[011] - - [101, 101, "xxxxxxxxxxxxxxxx"] and new tuple - [100, 101]
[011] + - [100, 101] and new tuple - [101, 101, "xxxxxxxxxxxxxxxx"]
[011] ...
[011] ch:get()
[011] ---
```
The test is inherently racy: a conflicting tuple may be inserted to the
new index either by the index build procedure or by the test fiber doing
DML in the background. The error messages will disagree regarding what
tuple should be considered old and which one new. Let's match the error
message explicitly.
The failure was introduced by d11fb3061e15faf4e0eb5375fb8056b4e64348ae
("box: change ER_TUPLE_FOUND message") which enhanced error messages
with conflicting tuples.
commit 41fdd2a8889342750e31d1d85dba24df4e550c33
Author: Georgiy Lebedev <g.lebedev@tarantool.org>
Date: Mon Jan 17 14:22:22 2022 +0300
core: fix testing tuple field count overflow
Testing tuple field count overflow handling requires creating a severely
large tuple with box.schema.FIELD_MAX (INT32_MAX) fields: introduce an
error injection for testing this corner case.
Fixes #6684
commit 91d8fef59bb55f608ae0739ddefe8dac7c160fe5
Author: Kirill Yukhin <kyukhin@tarantool.org>
Date: Mon Jan 17 20:03:34 2022 +0300
test: bump new test-run version
* Enable parallel invocation of fragile tests
commit 5cd399b7779a204ffd60fd8d5ec4946cfc64236c
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Fri Jan 14 17:04:08 2022 +0300
test: fix flaky vinyl/gh-4810-dump-during-index-build test
The commit fixes the following test failure:
```
[082] vinyl/gh-4810-dump-during-index-build.test.lua Test timeout of 310 secs reached [ fail ]
[082]
[082] Test failed! Result content mismatch:
[082] --- vinyl/gh-4810-dump-during-index-build.result Thu Dec 9 05:31:17 2021
[082] +++ /build/usr/src/debug/tarantool-2.10.0~beta1.324.dev/test/var/rejects/vinyl/gh-4810-dump-during-index-build.reject Thu Dec 9 06:51:03 2021
[082] @@ -117,34 +117,3 @@
[082] for i = 1, ch:size() do
[082] ch:get()
[082] end;
[082] - | ---
[082] - | ...
[082] -
...
```
The test hangs waiting for the test fibers to exit. There are two test
fibers - one builds an index, another populates the test space. The
latter uses pcall so it always returns. The one that builds an index,
however, doesn't. The problem is index build may fail because it builds
a unique index while the fiber populating the space may insert
non-unique values. Fix this by building a non-unique index instead,
which should never fail. To reproduce the issue the test checks is fixed
one can build any index, unique or non-unique, so it should be fine.
Closes #5508
commit cc6c328dd458c2b2b12e4892697f412c2ab86ad1
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Fri Jan 14 15:59:28 2022 +0300
test: fix flaky vinyl/gh test failure
The commit fixes the following test failure:
```
[005] vinyl/gh.test.lua [ fail ]
[005]
[005] Test failed! Result content mismatch:
[005] --- vinyl/gh.result Mon Dec 13 15:03:45 2021
[005] +++ /root/actions-runner/_work/tarantool/tarantool/test/var/rejects/vinyl/gh.reject Fri Dec 17 10:41:24 2021
[005] @@ -716,7 +716,7 @@
[005] ...
[005] test_run:wait_cond(function() return finished == 2 end)
[005] ---
[005] -- true
[005] +- false
[005] ...
[005] s:drop()
[005] ---
```
The reason of the failure is that the fiber doing checkpoints fails,
because a checkpoint may be already running by the checkpoint daemon.
Invoke box.snapshot() under pcall to make the test more robust.
Part of #5141
commit 7f8c549b55c75c8f3fd6f0ce7898c5096b64ed05
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Fri Jan 14 15:31:09 2022 +0300
test: fix flaky vinyl/deferred_delete test
The commit fixes the following test failure:
```
[019] vinyl/deferred_delete.test.lua [ fail ]
[019]
[019] Test failed! Result content mismatch:
[019] --- vinyl/deferred_delete.result Tue Jan 11 11:10:22 2022
[019] +++ /build/usr/src/debug/tarantool-2.10.0~beta2.37.dev/test/var/rejects/vinyl/deferred_delete.reject Fri Jan 14 11:45:26 2022
[019] @@ -964,7 +964,7 @@
[019] ...
[019] sk:stat().disk.dump.count -- 1
[019] ---
[019] -- 1
[019] +- 0
[019] ...
[019] sk:stat().rows - dummy_rows -- 120 old REPLACEs + 120 new REPLACEs + 120 deferred DELETEs
[019] ---
```
The test checks that compaction of a primary index triggers dump of
secondary indexes of the same space, because it generates deferred
DELETE statements. There's no guarantee that by the time compaction
completes, secondary index dump have been completed as well, because
compaction may ignore the memory quota (it uses vy_quota_force_use in
vy_deferred_delete_on_replace). Make the check more robust by using
wait_cond.
Follow-up #5089
commit 8c913a104044da87c807d046215b7ba4f4324bcd
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Fri Jan 14 15:28:26 2022 +0300
test: use wait_cond in vinyl/deferred_delete test
It's better than hand-written busy-wait.
commit cd9fd77eb6e578049cae1b1def6deafef45fd717
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Fri Jan 14 14:45:06 2022 +0300
test: fix flaky vinyl/gc test
The commit fixes the following test failure:
```
[013] vinyl/gc.test.lua [ fail ]
[013]
[013] Test failed! Result content mismatch:
[013] --- vinyl/gc.result Fri Dec 24 12:27:33 2021
[013] +++ /build/usr/src/debug/tarantool-2.10.0~beta2.18.dev/test/var/rejects/vinyl/gc.reject Thu Dec 30 10:29:29 2021
[013] @@ -102,7 +102,7 @@
[013] ...
[013] check_files_number(2)
[013] ---
[013] -- true
[013] +- null
[013] ...
[013] -- All records should have been purged from the log by now
[013] -- so we should only keep the previous log file.
```
The reason of the failure is that vylog files are deleted asynchronously
(`box.snapshot()` doesn't wait for `unlink` to complete) since commit
8e429f4b7a335ff318ca86b1147b194d1ede9c02 ("wal: remove old xlog files
asynchronously"). So to fix the test, we just need to make the test wait
for garbage collection to complete.
Follow-up #5383
commit 68c29688c0d8e0eb23b4b2e72acc425ee37eb22c
Author: mechanik20051988 <mechanik20051988@tarantool.org>
Date: Wed Dec 22 16:25:05 2021 +0300
small: fix assertion during slab allocation
When `slab_get` is called from `region_alloc` or `ibuf_alloc` first
of all we try to calculate order of appropriate slab. But there is
no check that requested size with slab meta is <= UINT32_MAX, that
leads to assertion failer in `slab_order` function. There is no
need for this assertion we should return `cache->order_max + 1`
for this case.
Closes #6726
commit f0eb2ca3f919e16a51e180a9adcfe447ef812213
Author: Serge Petrenko <sergepetrenko@tarantool.org>
Date: Fri Oct 8 08:18:14 2021 +0300
replication: set default synchro quorum value to majority
This required some test refactoring for tests which rely on quorum being
"1" by default.
Closes #5966
Prerequisite #6471
commit 1fa40d8a4255f887e56aaf52fd78ff90410b9288
Author: Yaroslav Lobankov <y.lobankov@tarantool.org>
Date: Tue Jan 11 18:04:37 2022 +0300
ci: make osx workflow names consistent
This patch brings consistency to OSX workflow names with other workflow
names.
The following changes were made:
* Arch suffix changed (arm64 -> aarch64)
* Minor OS version removed (11_2 -> 11)
* Specific options moved to the end of the workflow name
(osx_debug_xyz.yml -> osx_xyz_debug.yml)
Closes tarantool/tarantool-qa#149
commit 50c83f74441a6fab94e03c84682bb94613ba3699
Author: Alexander Turenko <alexander.turenko@tarantool.org>
Date: Tue Jan 4 00:31:59 2022 +0300
gen-release-notes: group sections case insensitively
Now `## bugfix/luajit` and `## bugfix/LuaJIT` entries will form one
section.
commit df5d69e61fe13154eb6c85de4ec491f3bf0f5f11
Author: Alexander Turenko <alexander.turenko@tarantool.org>
Date: Mon Jan 3 23:58:16 2022 +0300
gen-release-notes: prettify only lowercased headers
Now one can write either `## bugfix/luajit` or `## bugfix/LuaJIT`. The
latter will NOT be transformed into `### Luajit` anymore. Both variants
now give `### LuaJIT` section inside `## Bugs fixed` section.
See `REDEFINITIONS` variable inside the script to understand how well
known headers (such as `## <...>/LuaJIT`) are prettified.
There is a problem with section grouping, when headers are written in
lower/title/mixed case. It'll be resolved in a next commit.
commit b9c022f2f7a7c9c3c160a864ef548e3745877bc4
Author: Alexander Turenko <alexander.turenko@tarantool.org>
Date: Mon Jan 3 22:50:03 2022 +0300
gen-release-notes: handle no changelog entries
We run the script in CI as linter (see PR #6701), so it should handle
lack of any changelog entries gracefully.
commit 8973b9682a83015a4c37546aabf1b822c795e8db
Author: Alexander Turenko <alexander.turenko@tarantool.org>
Date: Mon Jan 3 22:36:47 2022 +0300
gen-release-notes: handle no features/no bugfixes
Previously the script mistakely requires at least one feature and at
least one bugfix. However it is quite natural to have only bugfixes in a
bugfix release. Moreover, we added the script into CI as linter (see
PR #6701), so it should work even when we just start filling release
notes.
commit 06cd93739487182c5b568df0923abf1e819df547
Author: Alexander Turenko <alexander.turenko@tarantool.org>
Date: Mon Jan 3 21:58:55 2022 +0300
gen-release-notes: keep changelogs/unreleased dir
Git does not store empty directories, but it is convenient for us to
have this directory always.
commit 094c46b8f5e039f66d78704874f7286aa1ea1254
Author: Sergey Bronnikov <sergeyb@tarantool.org>
Date: Wed Jan 12 18:32:05 2022 +0300
ci: run fuzzing on tags and release branches too
commit dc9c21177f67834858708246e1bea0476ea36905
Author: Sergey Bronnikov <sergeyb@tarantool.org>
Date: Mon Jan 10 15:46:51 2022 +0300
ci: run fuzzing on PR when the 'full-ci' label is set
Patch allows to run fuzzing in PR where label 'full-ci' is set.
Complememts a patch in commit "ci: enable fuzzing for *-full-ci
branches" (0115aab066db3757617c7b21ec9d1a3ae3801318).
Follows up #6630
commit 91ea03247904cf9664808edf5b2d784b9dddf10a
Author: Sergey Bronnikov <sergeyb@tarantool.org>
Date: Mon Jan 10 15:45:52 2022 +0300
ci: enable fuzzing workflow
Fuzzing workflow has been disabled due to warnings (treated by errors)
on compilation with Clang 14 that is used as a default compiler in
OSS-Fuzz infrastructure and missed libunwind in a test env where fuzzers
run. Compilation warnings were fixed in a previous commit and problem
with missed libunwind library has been resolved by patch to OSS-Fuzz
[1].
Follows up #6681
1. https://github.com/google/oss-fuzz/pull/7110
commit 78827e322ded8052ee74cbd60f233c9602ee2997
Author: Sergey Bronnikov <sergeyb@tarantool.org>
Date: Mon Jan 10 14:53:29 2022 +0300
build: fix warnings on Clang 13+
Part of #6681
commit c9d186082b423fa4da322e5fb9c717e6cb102a76
Author: Kirill Yukhin <kyukhin@tarantool.org>
Date: Thu Dec 23 10:50:00 2021 +0300
github-ci: add ubuntu implish (21.10) workflow
Add ubuntu-impish workflow, which runs on push and pull-requests.
Closes #6566
commit 1b46becbe695786d599a4f75699cd71558d10ad0
Author: Yaroslav Lobankov <y.lobankov@tarantool.org>
Date: Tue Jan 11 13:27:55 2022 +0300
ci: mark 'unicode_de__phonebook_s3' as unstable
The test for the 'unicode_de__phonebook_s3' collation from
sql-tap/collation_unicode.test.lua fail if the ICU version >= 70.1.
So let's temporarily mark it as unstable until the issue is resolved.
See for more details tarantool/tarantool#6695.
commit 6f867e1b4d4b744d23fa09d2eceba2069d6d0f0c
Author: Pavel Balaev <balaev@tarantool.org>
Date: Tue Jan 4 20:14:48 2022 +0300
box: fix error code numbers in comment
commit 2b1dbe921d004b6118afecc493a12c65c9edc9bc
Author: Kirill Yukhin <kyukhin@tarantool.org>
Date: Tue Jan 4 11:49:05 2022 +0300
github-ci: disable fuzzing
Fuzzing is disabled until #6681 is resolved.
commit 0115aab066db3757617c7b21ec9d1a3ae3801318
Author: Sergey Bronnikov <sergeyb@tarantool.org>
Date: Thu Dec 30 14:17:46 2021 +0300
ci: enable fuzzing for *-full-ci branches
Now it is not possible to run fuzzing on CI without merge to master
branch. Last time we missed [1] a broken compilation on merging new
fuzzers to master [2]. This patch enables fuzzing for branches with
postfix 'full-ci'.
1. https://github.com/tarantool/tarantool/pull/6757
2. https://github.com/tarantool/tarantool/pull/6627
commit 46b2ec56ffd67c433afa68044beb1e3d123a128e
Author: Alexey Vishnyakov <vishnya@ispras.ru>
Date: Tue Nov 30 15:45:08 2021 +0300
test: Add swim proto fuzz corpus.
commit fd0c5d85ef06e43712a5ef280773c4fe57cb709d
Author: Alexey Vishnyakov <vishnya@ispras.ru>
Date: Fri Nov 19 16:31:14 2021 +0300
test: Add swim proto fuzz targets.
commit ee94aa69606c95e2364a1f18e7d2724aa5d38450
Author: Sergey Kaplun <skaplun@tarantool.org>
Date: Wed Dec 29 13:51:35 2021 +0300
test: fix dynamic modules loading on macOS
Since the auxiliary libraries are built as dynamically loaded modules on
macOS instead of shared libraries as it is done on Linux and BSD,
another environment variable should be used to guide `ffi.load()` while
searching the extension. Hence the paths are set in test need to be set
to `DYLD_LIBRARY_PATH` variable instead of `LD_LIBRARY_PATH` on macOS.
commit d011dffd6e7f768656357afadbc61e10fb3d4f81
Author: Kirill Yukhin <kyukhin@tarantool.org>
Date: Fri Dec 24 10:38:17 2021 +0300
github-ci: introduce macOS ver. 12 testing on amd64
New workflow was copy-and-pasted from macOS 11 workflow
and patched obviously.
NB: it seems like app-tap/popen test is constantly failing
on macOS ver. 12. Disable until it is fixed.
Part of #6739
commit b3c1b09d0acacd021004e56c4f9ca58103d37007
Author: Kirill Yukhin <kyukhin@tarantool.org>
Date: Fri Dec 24 10:35:45 2021 +0300
github-ci: remove strange digit from OSX 11 workflow
I have no idea what this trailing zero stands for.
Drop it.
commit 280267227b1e0f9a1694aae574547e4565a6d6b0
Author: Kirill Yukhin <kyukhin@tarantool.org>
Date: Tue Dec 28 14:48:35 2021 +0300
github-ci: remove macOS ver. 10.15
We support current and previous version of macOS.
So let's remove 10.15 from CI.
commit eb8e7218ab9bf833343e37b752aaea45febb2749
Author: Kirill Yukhin <kyukhin@tarantool.org>
Date: Fri Dec 24 09:17:36 2021 +0300
github-ci: move LTO testing on macOS to version 11
We're about to stop CI for macOS version 10.15. But
seems (for now) we still interested in LTO on mscOS.
So, move LTO testing to macOS version 11.
Part of #6739
commit e493b777dd5de305a6cd4b6e773905d629618cc4
Author: Yaroslav Lobankov <y.lobankov@tarantool.org>
Date: Wed Oct 27 01:55:45 2021 +0300
ci: add integration check for memcached module
This patch extends the 'integration.yml' workflow and adds a new
workflow call for running tests to verify integration between tarantool
and the memcached module.
Part of #5265
Part of #6056
Closes #6563
commit bd67539511ed0701bb29c087bf701e48c081db61
Author: Aleksandr Lyapunov <alyapunov@tarantool.org>
Date: Mon Dec 20 14:32:08 2021 +0300
lib: insignificant change to calm down coverity checker
Now scan.coverity.com reports "Overrunning buffer pointed to by
&map of 4 bytes by passing it to a function which accesses it at
byte offset 7" in bit_iterator_init call.
Add an unit test that verifies that bit iterator works correctly
with small size bitmaps (like uint32_t).
Change condition a bit hoping that it will calm down the checker.
No functional changes.
commit e2d6c260516c30d33c816b7a6c6c1696e55a39b2
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Tue Dec 28 14:04:46 2021 +0300
test: disable #6198 test until #6684 is fixed
It eats way too much memory and is killed by OOM on my laptop,
which has 8 GB of RAM.
commit 8196ceb20d14c764ea277daa99432c171ec34da5
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Fri Dec 24 13:46:49 2021 +0300
Ignore EE tests
commit e2916691d7781058a4d013b5ec07996bd74272bf
Author: Yaroslav Lobankov <y.lobankov@tarantool.org>
Date: Mon Dec 27 16:53:40 2021 +0400
cd: deploy source packages with the right version
For now, source packages are deployed to our repositories with the wrong
version. It is due to the actions/checkout bug with not preserving tag
annotations [1].
This issue was fixed from our side by adding a workaround [2] to the
.github/actions/environment action. But for some reason, invocation of
this action was not added to the 'source.yml' workflow. This patch fixes
that.
[1] https://github.com/actions/checkout/issues/290
[2] https://github.com/tarantool/tarantool/pull/6197
Closes tarantool/tarantool-qa#146
commit 634f59c7f8ad41d9a7cee016a7bc9e2c4d039998
Author: Serge Petrenko <sergepetrenko@tarantool.org>
Date: Wed Dec 22 12:29:05 2021 +0300
recovery: panic in case of recovery and replicaset vclock mismatch
We assume that no one touches the instance's WALs, once it has taken the
wal_dir_lock. This is not the case when upgrading from an old setup
(running tarantool 1.7.3-6 or less). Such nodes either take a lock on
snap dir, which may be different from wal dir, or don't take the lock at
all.
So, it's possible that during upgrade an old node is not stopped
properly before a new node is started in the same data directory.
The old node might even write some extra data to WAL during new node's
startup.
This is obviously bad and leads to multiple issues. For example, new node
might start local recovery, scan the WALs and set replicaset.vclock to
some value {1 : 5}. While the node recovers WALs they are appended by the old
node up to vclock {1 : 10}.
The node finishes local recovery with replicaset vclock {1 : 5}, but
data recovered up to vclock {1 : 10}.
The node will use the now outdated replicaset vclock to subscribe to
remote peers (leading to replication breaking due to duplicate keys
found), to initialize WAL (leading to new xlogs appearing with duplicate
LSNs). There might be a number of other issues we just haven't stumbled
upon.
Let's prevent situations like that and panic as soon as we see that the
initially scanned vclock (replicaset vclock) differs from actually
recovered vclock.
Closes #6709
commit dc19be406e36bbfa5761a64c2fb53f7c3dd346b1
Author: Kirill Yukhin <kyukhin@tarantool.org>
Date: Fri Dec 24 15:12:18 2021 +0300
github-ci: Remove Ubuntu 20.10 for arm64
Patch 19a161b removed Ubuntu 20.10 from CI/CD for amd64 only.
Let's do it for arm64 as well.
commit 19a161b5f675d628dad2f376fc05b86ea7a81bc1
Author: Kirill Yukhin <kyukhin@tarantool.org>
Date: Fri Dec 24 07:18:31 2021 +0300
github-ci: Remove Ubuntu 20.10
Ubuntu 20.10 reached end-of-life few months ago,
remove it from CI/CD.
commit 743ed86373eed67da6c1ff710016983de245f080
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Tue Dec 21 17:11:19 2021 +0300
evio: store uri in evio_service_entry
The uri is passed to the accept callback. Currently, it's unused,
but in future we will use uri parameters to configure connections
(e.g. setup encryption).
commit 8b1aef56a09884781ec141054c0a3936948304be
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Tue Dec 21 16:32:41 2021 +0300
uri: add copy constructor
While we are at it, let's also add a test case for the move constructor.
commit 0badb4130deac46d6651cdcecb6c6d7ee401a96a
Author: Vladimir Davydov <vdavydov@tarantool.org>
Date: Tue Dec 21 14:09:35 2021 +0300
net.box: pass uri instead of host and port to state machine
We are planning to pass some extra connection parameters in uri
(e.g. encryption options).
