pgcopydb (0.17-1)
[PTS] [DDPO]
COMMITS: VCS has seen 10 commits since the v0.17 tag
- Git: https://github.com/dimitri/pgcopydb.git
-
- Branch: main
- Path: debian/changelog
- Repo size: 23130112
- Browser: https://github.com/dimitri/pgcopydb
- Last scan: 2024-12-03 17:40:07+00
- Next scan: 2024-12-04 19:26:00+00
- Debian changelog in Git:
pgcopydb (0.17-1) unstable; urgency=medium
* Official 0.17 release of pgcopydb
-- Dimitri Fontaine <dim@tapoueh.org> Wed, 07 Aug 2024 15:17:18 +0200
- This branch is 10 commits ahead of tag v0.17
- Git log:
commit 8361a93a0db42c8683c9979b411b5b13c74582d4
Author: Arunprasad Rajkumar <ar.arunprasad@gmail.com>
Date: Fri Oct 4 16:23:54 2024 +0530
Fix invalid UPDATE stmt when none of the columns change (#883)
We need at least two columns to be able to skip a column
if the value is the same in the old and new rows.
Otherwise, we would end up with an invalid UPDATE statement
like below:
```
UPDATE table SET WHERE "id" = 1;
```
Usually, the above could happen when REPLICA IDENTITY is set
to FULL, and the UPDATE statement executed with the same
values as the old ones.
For e.g.
```
UPDATE table SET "id" = 1 WHERE "id" = 1;
```
Solution: Skip the update when all columns in SET clause is equal to the WHERE clause.
Signed-off-by: Arunprasad Rajkumar <ar.arunprasad@gmail.com>
commit 903162691a3a1cbd6c8edbe65312090cf834eabe
Author: Arunprasad Rajkumar <ar.arunprasad@gmail.com>
Date: Thu Oct 3 19:43:40 2024 +0530
Tests are broken after PG17 release (#886)
We do not pin the postgresql-client causing the test failures after PG17
release due to the issue discussed here in the PG mailing list[1].
This commit pins the postgresql-client version as per the docker env variable PGVERSION.
[1] https://postgrespro.com/list/thread-id/2693550
Signed-off-by: Arunprasad Rajkumar <ar.arunprasad@gmail.com>
commit 0f79684606bcb23d9edabd867ec3f4260b4008a7
Author: Hanefi Onaldi <Hanefi.Onaldi@microsoft.com>
Date: Wed Sep 18 17:08:52 2024 +0300
Remove unused functions and structs (#882)
* Remove unused functions and structs
Co-authored-by: Hanefi Onaldi <hanefi@citusdata.com>
* Reintroduce one declaration that should not be removed
---------
Co-authored-by: Cem Eliguzel <cemeliguzel@microsoft.com>
commit 38b2c8f91a773a0ae58885380f002f46c1d81a17
Author: Dimitri Fontaine <dim@tapoueh.org>
Date: Wed Sep 11 17:14:00 2024 +0200
Fix docker compose external volume (#860)
* Fix docker-compose volume sharing between services.
In unit tests where we use both an inject and a test services, we need the
inject service to access the same pgcopydb work directory as the test
service. For that we use an external docker volume.
The previous implementation of that idea was found to be wrong, this is
fixing it by properly using docker APIs for shared "external" volumes.
* Attempt to fix github action errors.
On my local environment docker volume create turned out to not be necessary,
which is good news because we can (and probably want to) keep the volumes
around between test rounds. A cleanup script has even been implemented to
restart fresh.
On GitHub actions, it looks like we might need to first create the docker
volume to then be able to use it. The error message is not very helpful (it
just says: docker-compose: no such file or directory), but it's worth trying
that.
* Add `make attach` to tests using a docker volume.
This runs an interactive docker container (bash) with the volume mounted for
exploration / debug / inspect.
* Refrain from re-assigning the PATH variable in Makefiles.
* The make fix-volume operation does not need an interactive terminal.
* Fix `docker compose version` command.
commit ac1c1930b06210cc3f68975757075d76cf942932
Author: Hanefi Onaldi <Hanefi.Onaldi@microsoft.com>
Date: Thu Sep 5 14:49:41 2024 +0300
test_decoding transform: skip toast columns with unchanged values (#877)
test_decoding plugin may not include the value of a TOAST column if it
has not changed. In that case, instead of the value of the column the
string "unchanged-toast-datum" is included in the output. In that case,
we should remove the column from the range table entry list in the
generated SQL queries instead of including it and setting that column to
the value "unchanged-toast-datum".
There is one strange case that if a table has only toast columns, and an
update query that does not update any value is executed, we end up
filtering all the range table entries and skip executing any sql queries
as it will be no-op.
commit 36ab63c7107fd4b56ad92a30bd4982f9573c63de
Author: Arunprasad Rajkumar <ar.arunprasad@gmail.com>
Date: Thu Sep 5 15:47:56 2024 +0530
Fix deadlock during pipeline sync (#880)
* Fix deadlock during pipeline sync
Prior to the fix, `pgsql_sync_pipeline` implementation calls
`PQpipelineSync` followed by single `PQconsumeInput` and reads the
results using `PQgetResult` until we get `PGRES_PIPELINE_SYNC`.
However, `PQgetResult` would block when there are not enough data already consumed
according to[1],
> Note that PQgetResult will block only if a command is active and the
necessary response data has not yet been read by PQconsumeInput.
The default read buffer for libpq connection is 16K. When the result
size exceeds 16K[2], the `PQgetResult` would block forever unless we
consume for input using `PQconsumeInput`.
This commit attempts to fix the problem by reintroducing the socket read
readiness, but also uses `PQisBusy` to decide whether to consume input
using `PQconsumeInput`.
The the new implementation consumes the data in a loop until it
receives the `PGRES_PIPELINE_SYNC` on the condition `PQisBusy(conn) == 1` or the connection's socket read readiness. When the the command is not busy i.e. `PQisBusy(conn) == 0`, the results will be read. The changes are inspired from the libpq pipeline integration test [3].
[1] https://www.postgresql.org/docs/current/libpq-async.html#LIBPQ-PQGETRESULT
[2] https://github.com/postgres/postgres/blob/a68159ff2b32f290b1136e2940470d50b8491301/src/interfaces/libpq/fe-connect.c#L4616
[3] https://github.com/postgres/postgres/blob/a68159ff2b32f290b1136e2940470d50b8491301/src/test/modules/libpq_pipeline/libpq_pipeline.c#L1967-L2023
Signed-off-by: Arunprasad Rajkumar <ar.arunprasad@gmail.com>
* Add test
Signed-off-by: Arunprasad Rajkumar <ar.arunprasad@gmail.com>
---------
Signed-off-by: Arunprasad Rajkumar <ar.arunprasad@gmail.com>
commit a53c0903d412f580789a321bc789fdec9cea53c8
Author: Hanefi Onaldi <Hanefi.Onaldi@microsoft.com>
Date: Mon Sep 2 16:17:46 2024 +0300
Fix bug in passing logSQL from spec to context (#878)
* Pass logSQL from spec to context
* Remove unused function argument
Co-authored-by: Cem Eliguzel <cemeliguzel@microsoft.com>
commit 575b7ddcf24175cd15f18a57b0f28bebdd14eae1
Author: Hanefi Onaldi <Hanefi.Onaldi@microsoft.com>
Date: Mon Sep 2 13:01:46 2024 +0300
Increase max allowed chars in Archive TOC lines (#867)
* Testcase with a function with a long signature
* Increase max allowed chars in Archive TOC lines
The maximum line length for an Archive TOC entry is currently limited to
1024, which is too short for some objects such as functions and
procedures with a high number of parameters.
Increase the maximum line length to 8 kilobytes, which should be enough
for all objects.
* Fix tests
* Improve jq scripts
* Revert "Fix tests"
This reverts commit ab267434057a7aa5ebb84554af205ee1d46580b4.
commit 6bdcbee620e5142a8e89b4382ea61a9c84189fa1
Author: Aditi Kesarwani <104135148+Kesarwani-aditi@users.noreply.github.com>
Date: Thu Aug 8 18:39:34 2024 +0530
Add support for the timescaledb extension. (#849)
This extension requires specific pre and post pg_restore steps to be
implemented, and also seems to require pgcopydb to use a superuser role to
be able to fetch the data and use pg_dump.
Co-authored-by: Aditi Kesarwani <aditi.kesarwani2022@gmail.com>
Co-authored-by: VaibhaveS <vaibhavedavey@gmail.com>
Co-authored-by: Arunprasad Rajkumar <ar.arunprasad@gmail.com>
commit 64ca602343a30cdb04f37a76ce749e1b40a0ecae
Author: Dimitri Fontaine <dim@tapoueh.org>
Date: Wed Aug 7 16:12:33 2024 +0200
Change the version string in the docs setup. (#865)