[PGDay.Seoul 2020] PostgreSQL 13 New Features

PGDay.Seoul 2020 :
PostgreSQL 13
New features
https://99designs.com/illustrations/contests/postgresql-version-release-artwork-1029738

About me
Daniel lee
PostgreSQL Korea
MySQL Korea User Group

PostgreSQL 13 Release Notes by Community
PostgreSQL 13 contains many new features and enhancements, including:
• Space savings and performance gains from de-duplication of B-tree index entries
• Improved performance for queries that use aggregates or partitioned tables
• Better query planning when using extended statistics
• Parallelized vacuuming of indexes
• Incremental sorting
Migration to Version 13
A dump/restore using pg_dumpall or use of pg_upgrade or logical replication is required for those wishing to migrate data from any previous release. See Section 18.6 for general information on migrating to new major releases.
Version 13 contains a number of changes that may affectcompatibility with previous releases. Observe the following incompatibilities:
•Change SIMILAR TO ... ESCAPE NULL to return NULL (Tom Lane)
This new behavior matches the SQL specification. Previously a null ESCAPE value was taken to mean using the default escape string (abackslash character). This also applies to substring(text FROM pattern ESCAPE text). The previous behavior has been retained in old views by keeping the original function unchanged.
•Make json[b]_to_tsvector() fully check the spelling of its string option (Dominik Czarnota)
•Change the way non-default effective_io_concurrency values affectconcurrency (Thomas Munro)
Previously, this value was adjusted before setting the number of concurrentrequests. The value is now used directly. Conversion of old values to new ones can be done using:
SELECT round(sum(OLDVALUE / n::float)) AS newvalue FROM generate_series(1, OLDVALUE) s(n);
•Prevent display of auxiliary processes in pg_stat_ssl and pg_stat_gssapi system views (Euler Taveira)
Queries that join these views to pg_stat_activity and wish to see auxiliary processes will need to use left joins.
•Rename various wait events to improve consistency (Fujii Masao, Tom Lane)
•Fix ALTER FOREIGN TABLE ... RENAME COLUMN to returna more appropriate command tag (Fujii Masao)
Previously it returned ALTER TABLE; now it returns ALTER FOREIGN TABLE.
•Fix ALTER MATERIALIZED VIEW ... RENAME COLUMN to return a more appropriate command tag (Fujii Masao)
Previously it returned ALTER TABLE; now it returns ALTER MATERIALIZED VIEW.
•Rename configuration parameter wal_keep_segments to wal_keep_size (Fujii Masao)
This determines how much WAL to retain for standby servers. It is specified in megabytes, rather than number of files as with the old parameter. If you previously used wal_keep_segments, the following formula will give you an approximately equivalent setting:
wal_keep_size = wal_keep_segments * wal_segment_size (typically 16MB)
•Remove support for defining operator classes using pre-PostgreSQL 8.0 syntax (Daniel Gustafsson)
•Remove support for defining foreign key constraints using pre-PostgreSQL 7.3 syntax (Daniel Gustafsson)
•Remove support for "opaque" pseudo-types used by pre-PostgreSQL 7.3 servers (Daniel Gustafsson)
•Remove support for upgrading unpackaged (pre-9.1) extensions (Tom Lane)
The FROM option of CREATE EXTENSION is no longer supported. Any installations still using unpackaged extensions should upgrade them to a packaged version before updating to PostgreSQL 13.
•Remove support for posixrules files in the timezone database (Tom Lane)
IANA's timezone group has deprecated this feature, meaning that it will gradually disappear fromsystems' timezone databases over the next few years. Rather than have a behavioral change appear unexpectedly with a timezone data update, we have removed PostgreSQL's support for this feature as of version 13. This affectsonly the behavior of POSIX-style time zone specifications that lack an explicit daylight savings transition rule; formerly the transition rule could be determined by installing a custom posixrules file, but now it is hard-wired. The recommended fix for a
•In ltree, when an lquery pattern contains adjacent asterisks with braces, e.g., *{2}.*{3},properly interpret that as *{5} (Nikita Glukhov)
•Fix pageinspect's bt_metap() to return more appropriate data types that are less likely to overflow (Peter Geoghegan)
E.2.3. Changes
Below you will find a detailed account of the changes between PostgreSQL 13 and the previous major release.
E.2.3.1. Server
E.2.3.1.1. Partitioning
•Allow pruning of partitions to happen in more cases (Yuzuko Hosoya, Amit Langote, Álvaro Herrera)
•Allow partitionwise joins to happen in more cases (Ashutosh Bapat, Etsuro Fujita, Amit Langote, Tom Lane)
For example, partitionwise joins can now happen between partitioned tables even when their partition bounds do not match exactly.
•Support row-level BEFORE triggers on partitioned tables (Álvaro Herrera)
However, such a trigger is not allowed to change which partition is the destination.
•Allow partitioned tables to be logically replicated via publications (Amit Langote)
Previously, partitions had to be replicated individually. Now a partitioned table can be published explicitly, causing all its partitions to be published automatically. Addition/removal of a partition causes it to be likewise added to or removed from the publication. The CREATE PUBLICATION option publish_via_partition_root controls whether changes to partitions are published as their own changes or their parent's.
•Allow logical replication into partitioned tables on subscribers (Amit Langote)
Previously, subscribers could only receive rows into non-partitioned tables.
•Allow whole-row variables (that is, table.*) to be used in partitioning expressions (Amit Langote)
E.2.3.1.2. Indexes
•More efficiently store duplicates in B-tree indexes (Anastasia Lubennikova, Peter Geoghegan)
This allows efficientB-tree indexing of low-cardinality columns by storing duplicate keys only once. Users upgrading with pg_upgrade will need to use REINDEX to make an existing index use this feature.
•Allow GiST and SP-GiST indexes on box columns to support ORDER BY box <-> point queries (Nikita Glukhov)
•Allow GIN indexes to more efficiently handle ! (NOT) clauses in tsquery searches (Nikita Glukhov, Alexander Korotkov, Tom Lane, Julien Rouhaud)
•Allow index operator classes to take parameters (Nikita Glukhov)
•Allow CREATE INDEX to specify the GiST signature length and maximum number of integer ranges (Nikita Glukhov)
Indexes created on four and eight-byte integer array, tsvector, pg_trgm, ltree, and hstore columns can now control these GiST index parameters, rather than using the defaults.
•Prevent indexes that use non-default collations frombeing added as a table's unique or primary key constraint (Tom Lane)
The index's collation must match that of the underlying column, but ALTER TABLE previously failed to check this.
E.2.3.1.3. Optimizer
•Improve the optimizer's selectivity estimation for containment/match operators (Tom Lane)
•Allow setting the statistics target for extended statistics (Tomas Vondra)
This is controlled with the new command option ALTER STATISTICS ... SET STATISTICS. Previously this was computed based on more general statistics target settings.
•Allow use of multiple extended statistics objects in a single query (Tomas Vondra)
•Allow use of extended statistics objects for OR clauses and IN/ANY constant lists (PierreDucroquet, Tomas Vondra)
•Allow functions in FROM clauses to be pulled up (inlined) if they evaluate to constants (Alexander Kuzmenkov, Aleksandr Parfenov)
E.2.3.1.4. General Performance
•Implement incremental sorting (James Coleman, Alexander Korotkov, Tomas Vondra)
If an intermediate query result is known to be sorted by one or more leading keys of a required sort ordering, the additional sorting can be done considering only the remaining keys, if the rows are sorted in batches that have equal leading keys.
If necessary, this can be controlled using enable_incremental_sort.
•Improve the performanceof sorting inet values (Brandur Leach)
•Allow hash aggregation to use disk storage for large aggregation result sets (Jeff Davis)
Previously, hash aggregation was avoided if it was expected to use more than work_mem memory. Now, a hash aggregation plan can be chosen despite that. The hash table will be spilled to disk if it exceeds work_mem times hash_mem_multiplier.
This behavior is normally preferable to the old behavior, in which once hash aggregation had been chosen, the hash table would be kept in memory no matter how large it got — which could be very large if the planner had misestimated. If necessary, behavior similar to that can be obtained by increasing hash_mem_multiplier.
•Allow inserts, not only updates and deletes, to trigger vacuuming activity in autovacuum (Laurenz Albe, Darafei Praliaskouski)
Previously, insert-only activity would trigger auto-analyze but not auto-vacuum, on the grounds that there could not be any dead tuples to remove. However, a vacuum scan has other useful side-effects such as setting page-all-visible bits, which improves the efficiency of index-only scans. Also, allowing an insert-only table to receive periodic vacuuming helps to spread out the work of “freezing” old tuples, so that there is not suddenly a large amount of freezing work to do when the entire table reaches the anti-wraparound threshold all at once.
If necessary, this behavior can be adjusted with the new parameters autovacuum_vacuum_insert_threshold and autovacuum_vacuum_insert_scale_factor, or the equivalent table storage options.
•Add maintenance_io_concurrency parameter to control I/Oconcurrency for maintenance operations (Thomas Munro)
•Allow WAL writes to be skipped during a transaction that creates or rewrites a relation, if wal_level is minimal (Kyotaro Horiguchi)
Relations larger than wal_skip_threshold will have their files fsync'ed rather than generating WAL. Previously this was done only for COPY operations, but the implementation had a bug that could cause data loss during crashrecovery.
•Improve performancewhen replaying DROP DATABASE commands when many tablespaces are in use (Fujii Masao)
•Improve performancefor truncation of very large relations (Kirk Jamison)
•Improve retrieval of the leading bytes of TOAST'ed values (Binguo Bao, Andrey Borodin)
Previously, compressed out-of-line TOAST values were fully fetched even when it's known that only some leading bytes are needed. Now, only enough data to produce the result is fetched.
•Improve performanceof LISTEN/NOTIFY (Martijn van Oosterhout, Tom Lane)
•Speed up conversions of integers to text (David Fetter)
•Reduce memory usage for query strings and extension scripts that contain many SQL statements (Amit Langote)
E.2.3.1.5. Monitoring
•Allow EXPLAIN, auto_explain, autovacuum, and pg_stat_statements to track WAL usage statistics (Kirill Bychik, Julien Rouhaud)
•Allow a sample of SQL statements, rather than all statements, to be logged (Adrien Nayrat)
A log_statement_sample_rate fractionof those statements taking more than log_min_duration_sample duration will be logged.
•Add the backend type to csvlog and optionally log_line_prefix log output (Peter Eisentraut)
•Improve control of prepared statement parameter logging (Alexey Bashtanov, Álvaro Herrera)
The GUC setting log_parameter_max_length controls the maximum length of parameter values output during logging of non-error statements, while log_parameter_max_length_on_error does the same for logging of statements with errors. Previously, prepared statement parameters were never logged during errors.
•Allow function call backtraces to be logged after errors(Peter Eisentraut, Álvaro Herrera)
The new parameter backtrace_functions specifies which C functions should generate backtraces on error.
•Make vacuum buffer counters 64-bits wide to avoid overflow (Álvaro Herrera)
E.2.3.1.6. System Views
•Add leader_pid to pg_stat_activity to report a parallel worker's leader process (Julien Rouhaud)
•Add system view pg_stat_progress_basebackup to report the progress of streaming base backups (Fujii Masao)
•Add system view pg_stat_progress_analyze to report ANALYZE progress (Álvaro Herrera, Tatsuro Yamada, Vinayak Pokale)
•Add system view pg_shmem_allocations to display shared memory usage (Andres Freund, Robert Haas)
•Add system view pg_stat_slru to monitor internal SLRU caches (Tomas Vondra)
•Allow track_activity_query_size to be set as high as 1MB (Vyacheslav Makarov)
The previous maximum was 100kB.
E.2.3.1.7. Wait Events
•Report a wait event while creating a DSM segment with posix_fallocate() (Thomas Munro)
•Add wait event VacuumDelay to reporton cost-based vacuum delay (Justin Pryzby)
•Add wait events for WAL archive and recovery pause (Fujii Masao)
The new events are BackupWaitWalArchive and RecoveryPause.
•Add wait events RecoveryConflictSnapshot and RecoveryConflictTablespace to monitor recovery conflicts (Masahiko Sawada)
•Improve performanceof wait events on BSD-based systems (Thomas Munro)
E.2.3.1.8. Authentication
•Allow only superusers to view the ssl_passphrase_command setting (Insung Moon)
This was changed as a security precaution.
•Change the server's default minimum TLS version for encrypted connections from1.0 to 1.2 (Peter Eisentraut)
This choice can be controlled by ssl_min_protocol_version.
E.2.3.2. Streaming Replication And Recovery
•Allow streaming replication configuration settings to be changed by reload (Sergei Kornilov)
Previously, a server restart was required to change primary_conninfo and primary_slot_name.
•Allow WAL receivers to use a temporary replication slot when a permanent one is not specified (Peter Eisentraut, Sergei Kornilov)
This behavior can be enabled using wal_receiver_create_temp_slot.
•Allow WALstorage for replication slots to be limited by max_slot_wal_keep_size (Kyotaro Horiguchi)
Replication slots that would require exceeding this value are marked invalid.
•Allow standby promotion to cancel any requested pause (Fujii Masao)
Previously, promotion could not happen while the standby was in paused state.
•Generate an error if recovery does not reach the specified recovery target (Leif Gunnar Erlandsen, Peter Eisentraut)
Previously, a standby would promote itself upon reaching the end of WAL, even if the target was not reached.
•Allow control over how much memory is used by logical decoding before it is spilled to disk (Tomas Vondra, Dilip Kumar, Amit Kapila)
This is controlled by logical_decoding_work_mem.
•Allow recovery to continue even if invalid pages are referenced by WAL (Fujii Masao)
This is enabled using ignore_invalid_pages.
E.2.3.3. UtilityCommands
•Allow VACUUM to process a table's indexes in parallel (Masahiko Sawada, Amit Kapila)
The new PARALLEL option controls this.
•Allow FETCH FIRST to use WITH TIES to returnany additional rows that match the last result row (Surafel Temesgen)
•Report planning-time buffer usage in EXPLAIN's BUFFER output (Julien Rouhaud)
•Make CREATE TABLE LIKE propagate a CHECK constraint's NO INHERIT property to the created table (Ildar Musin, Chris Travers)
•When using LOCK TABLE on a partitioned table, do not checkpermissions on the child tables (Amit Langote)
•Allow OVERRIDING USER VALUE on inserts into identity columns (Dean Rasheed)
•Add ALTER TABLE ... DROP EXPRESSION to allow removing the GENERATED property froma column (Peter Eisentraut)
•Fix bugs in multi-step ALTER TABLE commands (Tom Lane)
IF NOT EXISTS clauses now work as expected, in that derived actions (such as index creation) do not execute if the column already exists. Also, certain cases of combining related actions into one ALTER TABLE now work when they did not before.
•Add ALTER VIEW syntax to rename view columns (Fujii Masao)
Renaming view columns was already possible, but one had to write ALTER TABLE RENAME COLUMN, which is confusing.
•Add ALTER TYPE options to modify a base type's TOAST properties and support functions (Tomas Vondra, Tom Lane)
•Add CREATE DATABASE LOCALE option (Peter Eisentraut)
This combines the existing options LC_COLLATE and LC_CTYPE into a single option.
•Allow DROP DATABASE to disconnect sessions using the target database, allowing the drop to succeed (Pavel Stehule, Amit Kapila)
This is enabled by the FORCE option.
•Add structure member tg_updatedcols to allow C-language update triggers to know which column(s) were updated (Peter Eisentraut)
E.2.3.4. Data Types
•Add polymorphic data types for use by functions requiring compatible arguments (Pavel Stehule)
The new data types are anycompatible, anycompatiblearray, anycompatiblenonarray, and anycompatiblerange.
•Add SQL data type xid8 to expose FullTransactionId (Thomas Munro)
The existing xid data type is only four bytes so it does not provide the transaction epoch.
•Add data type regcollation and associated functions, to represent OIDs of collation objects (Julien Rouhaud)
•Use the glibc version in some cases as a collation version identifier (Thomas Munro)
If the glibc version changes, a warning will be issued about possible corruption of collation-dependent indexes.
•Add support for collation versions on Windows (Thomas Munro)
•Allow ROW expressions to have their members extracted with suffixnotation (Tom Lane)
For example, (ROW(4, 5.0)).f1 now returns 4.
E.2.3.5. Functions
•Add alternate version of jsonb_set() with improved NULL handling (Andrew Dunstan)
The new function, jsonb_set_lax(), handles a NULL new value by either setting the specified key to a JSON null, deleting the key, raising an exception, or returning the jsonb value unmodified, as requested.
•Add jsonpath .datetime() method (Nikita Glukhov, Teodor Sigaev, Oleg Bartunov, Alexander Korotkov)
This function allows JSON values to be converted to timestamps, which can then be processed in jsonpath expressions. This change also adds jsonpath functions that support time-zone-aware output.
•Add SQL functions NORMALIZE() to normalize Unicode strings, and IS NORMALIZED to check for normalization (Peter Eisentraut)
•Add min() and max() aggregates for pg_lsn (Fabrízio de Royes Mello)
These are particularly useful in monitoring queries.
•Allow Unicode escapes, e.g., E'unnnn' or U&'nnnn', to specify any character available in the database encoding, even when the database encoding is not UTF-8 (Tom Lane)
•Allow to_date() and to_timestamp() to recognize non-English month/day names (Juan José Santamaría Flecha, Tom Lane)
The names recognized are the same as those output by to_char() with the same format patterns.
•Add datetime format patterns FF1 – FF6 to specify input or output of 1 to 6 fractional-second digits (Alexander Korotkov, Nikita Glukhov, Teodor Sigaev, Oleg Bartunov)
These patterns can be used by to_char(), to_timestamp(), and jsonpath's .datetime().
•Add SSSSS datetime format pattern as an SQL-standard alias for SSSS (Nikita Glukhov, Alexander Korotkov)
•Add function gen_random_uuid() to generate version-4 UUIDs (Peter Eisentraut)
Previously UUID generation functions were only available in the external modules uuid-ossp and pgcrypto.
•Add greatest-common-denominator (gcd) and least-common-multiple (lcm) functions (Vik Fearing)
•Improve the performanceand accuracy of the numeric type's square root (sqrt) and natural log (ln) functions (Dean Rasheed)
•Add function min_scale() that returns the number of digits to the right of the decimal point that are required to represent a numeric value with full accuracy (Pavel Stehule)
•Add function trim_scale() to reduce the scale of a numeric value by removing trailing zeros (Pavel Stehule)
•Add commutators of distance operators (Nikita Glukhov)
For example, previously only point <-> line was supported, now line <-> point works too.
•Create xid8 versions of all transaction ID functions (Thomas Munro)
The old xid-based functions still exist, for backward compatibility.
•Allow get_bit() and set_bit() to set bits beyond the first256MB of a bytea value (Movead Li)
•Allow advisory-lock functions to be used in some parallel operations (Tom Lane)
•Add the ability to remove an object's dependency on an extension (Álvaro Herrera)
The object can be a function, materialized view, index, or trigger. The syntax is ALTER .. NO DEPENDS ON.
E.2.3.6. PL/PgSQL
•Improve performanceof simple PL/pgSQL expressions (Tom Lane, Amit Langote)
•Improve performanceof PL/pgSQL functions that use immutable expressions (Konstantin Knizhnik)
E.2.3.7. Client Interfaces
•Allow libpq clients to require channel binding for encrypted connections (Jeff Davis)
Using the libpq connection parameter channel_binding forcesthe other end of the TLS connection to prove it knows the user's password. This prevents man-in-the-middle attacks.
•Add libpq connection parameters to control the minimum and maximum TLS version allowed for an encrypted connection (Daniel Gustafsson)
The settings are ssl_min_protocol_version and ssl_max_protocol_version. By default, the minimum TLS version is 1.2 (this represents a behavioral change from previous releases).
•Allow use of passwords to unlock client certificates(Craig Ringer, Andrew Dunstan)
This is enabled by libpq's sslpassword connection parameter.
•Allow libpq to use DER-encoded client certificates(Craig Ringer, Andrew Dunstan)
•Fix ecpg's EXEC SQL elif directive to work correctly (Tom Lane)
Previously it behaved the same as endif followed by ifdef, so that a successful previous branch of the same if construct did not prevent expansion of the elif branch or following branches.
E.2.3.8. Client Applications
E.2.3.8.1. psql
•Add transaction status (%x) to psql's default prompts (Vik Fearing)
•Allow the secondary psql prompt to be blank but the same width as the primary prompt (Thomas Munro)
This is accomplished by setting PROMPT2 to %w.
•Allow psql's gand gx commands to change pset output options for the duration of that single command (Tom Lane)
This feature allows syntax like g (expand=on), which is equivalent to gx.
•Add psql commands to display operator classes and operator families (Sergey Cherkashin, Nikita Glukhov, Alexander Korotkov)
The new commands are dAc, dAf, dAo, and dAp.
•Show table persistence in psql's dt+ and related commands (David Fetter)
In verbose mode, the table/index/view shows if the object is permanent, temporary, or unlogged.
•Improve output of psql's d for TOAST tables (Justin Pryzby)
•Fix redisplay after psql's e command (Tom Lane)
When exiting the editor, if the query doesn't end with a semicolon or g, the query buffer contents will now be displayed.
•Add warn command to psql (David Fetter)
This is like echo except that the text is sent to stderr instead of stdout.
•Add the PostgreSQL home page to command-line --help output (Peter Eisentraut)
E.2.3.8.2. pgbench
•Allow pgbench to partition its “accounts” table (Fabien Coelho)
This allows performancetesting of partitioning.
•Add pgbench command aset, which behaves like gset, but for multiple queries (Fabien Coelho)
•Allow pgbench to generate its initial data server-side, rather than client-side (Fabien Coelho)
•Allow pgbench to show script contents using option --show-script (Fabien Coelho)
E.2.3.9. Server Applications
•Generate backup manifests for base backups, and verify them (Robert Haas)
A new tool pg_verifybackup can verify backups.
•Have pg_basebackup estimate the total backup size by default (Fujii Masao)
This computation allows pg_stat_progress_basebackup to show progress. If that is not needed, it can be disabled by using the --no-estimate-size option. Previously, this computation happened only if the --progress option was used.
•Add an option to pg_rewind to configure standbys (Paul Guo, Jimmy Yih, Ashwin Agrawal)
This matches pg_basebackup's --write-recovery-conf option.
•Allow pg_rewind to use the target cluster's restore_command to retrieve needed WAL (Alexey Kondratov)
This is enabled using the -c/--restore-target-wal option.
•Have pg_rewind automatically run crash recovery before rewinding (Paul Guo, Jimmy Yih, Ashwin Agrawal)
This can be disabled by using --no-ensure-shutdown.
•Increase the PREPARE TRANSACTION-related information reported by pg_waldump (Fujii Masao)
•Add pg_waldump option --quiet to suppress non-error output (Andres Freund, Robert Haas)
•Add pg_dump option --include-foreign-data to dump data from foreign servers (Luis Carril)
•Allow vacuum commands run by vacuumdb to operate in parallel mode (Masahiko Sawada)
This is enabled with the new --parallel option.
•Allow reindexdb to operate in parallel (Julien Rouhaud)
Parallel mode is enabled with the new --jobs option.
•Allow dropdb to disconnect sessions using the target database, allowing the drop to succeed (Pavel Stehule)
This is enabled with the -f option.
•Remove --adduser and --no-adduser fromcreateuser (Alexander Lakhin)
The long-supported preferredoptions for this are called --superuser and --no-superuser.
•Use the directory of the pg_upgrade program as the default --new-bindir setting when running pg_upgrade (Daniel Gustafsson)
E.2.3.10. Documentation
•Add a glossary to the documentation (Corey Huinker, Jürgen Purtz, Roger Harkavy, Álvaro Herrera)
•Reformat tables containing function and operator information for better clarity (Tom Lane)
•Upgrade to use DocBook 4.5 (Peter Eisentraut)
E.2.3.11. Source Code
•Add support for building on Visual Studio 2019 (Haribabu Kommi)
•Add build support for MSYS2 (Peter Eisentraut)
•Add compare_exchange and fetch_add assembly language code for Power PC compilers (Noah Misch)
•Update Snowball stemmer dictionaries used by full text search (Panagiotis Mavrogiorgos)
This adds Greek stemming and improves Danish and French stemming.
•Remove support for Windows 2000 (Michael Paquier)
•Remove support for non-ELF BSD systems (Peter Eisentraut)
•Remove support for Python versions 2.5.X and earlier (Peter Eisentraut)
•Remove support for OpenSSL 0.9.8 and 1.0.0 (Michael Paquier)
•Remove configure options --disable-float8-byval and --disable-float4-byval (Peter Eisentraut)
These were needed for compatibility with some version-zero C functions, but those are no longer supported.
•Pass the query string to planner hook functions (Pascal Legrand, Julien Rouhaud)
•Add TRUNCATE command hook (Yuli Khodorkovskiy)
•Add TLS init hook (Andrew Dunstan)
•Allow building with no predefined Unix-domain socket directory (Peter Eisentraut)
•Reduce the probability of SysV resource key collision on Unix platforms (Tom Lane)
•Use operating system functions to reliably erase memory that contains sensitive information (Peter Eisentraut)
For example, this is used for clearing passwords stored in memory.
•Add headerscheck scriptto test C header-file compatibility (Tom Lane)
•Implement internal lists as arrays, rather than a chain of cells (Tom Lane)
This improves performance for queries that access many objects.
•Change the API for TS_execute() (Tom Lane, Pavel Borisov)
TS_execute callbacks must now provide ternary (yes/no/maybe) logic. Calculating NOT queries accurately is now the default.
E.2.3.12. Additional Modules
•Allow extensions to be specified as trusted (Tom Lane)
Such extensions can be installed in a database by users with database-level CREATE privileges, even if they are not superusers. This change also removes the pg_pltemplate system catalog.
•Allow non-superusers to connect to postgres_fdw foreign servers without using a password (Craig Ringer)
Specifically, allow a superuser to set password_required to false for a user mapping. Care must still be taken to prevent non-superusers from using superuser credentials to connect to the foreign server.
•Allow postgres_fdw to use certificateauthentication (Craig Ringer)
Different users can use differentcertificates.
•Allow sepgsql to control access to the TRUNCATE command (Yuli Khodorkovskiy)
•Add extension bool_plperl which transforms SQL booleans to/fromPL/Perl booleans (Ivan Panchenko)
•Have pg_stat_statements treat SELECT ... FOR UPDATE commands as distinct from those without FOR UPDATE (Andrew Gierth, Vik Fearing)
•Allow pg_stat_statements to optionally track the planning time of statements (Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao)
Previously only execution time was tracked.
•Overhaul ltree's lquery syntax to treat NOT (!) morelogically (Filip Rembialkowski, Tom Lane, Nikita Glukhov)
Also allow non-* queries to use a numeric range ({}) of matches.
•Add support for binary I/Oof ltree, lquery, and ltxtquery types (Nino Floris)
•Add an option to dict_int to ignore the sign of integers (Jeff Janes)
•Add adminpack function pg_file_sync() to allow fsync'ing a file (Fujii Masao)
•Add pageinspect functions to output t_infomask/t_infomask2 values in human-readable format (Craig Ringer, Sawada Masahiko, Michael Paquier)
•Add B-tree index de-duplication processing columns to pageinspect output (Peter Geoghegan)

Cloud SQL now supports PostgreSQL 13
PostgreSQL 13 introduces performance improvements across the board, including enhanced partitioning capabilities,
increased index and vacuum efficiency, and better extended monitoring. Here are some highlights of what’s new:
• Additional partitioning and pruning cases support : As part of the continuous improvements of partitioned tables in the last two PostgreSQL versions, new cases of partition
pruning and direct joins have been introduced, including joins between partitioned tables when their partition bounds do not match exactly. In addition, BEFORE triggers on
partitioned tables are now supported.
• Incremental sorting : Sorting is a performance-intensive task, so every improvement in this area can make a difference. Now PostgreSQL 13 introduces incremental sorting,
which leverages early-stage sorts of a query and sorts only the incremental unsorted fields, increasing the chances the sorted block will fit in memory and by that, improving
performance.
• Efficient hash aggregation : In previous versions, it was decided in the planning stage whether hash aggregation functionality could be used, based on whether the hash
table fits in memory. With the new version, hash aggregation can be determined based on cost analysis, regardless of space in memory.
• B-tree index : now works more efficiently, thanks to storage space reduction enabled by removing duplicate values.
• Vacuuming : Vacuuming is an essential operation for database health and performance, especially for demanding and critical workloads. It reclaims storage occupied by dead
tuples and catalogues it in the visibility map for future use. In PostgreSQL 13, performance improvements and enhanced automations are being introduced:
• Faster vacuum : Parallel vacuuming of multiple indexes reduces vacuuming execution time.
• Autovacuum : Autovacuum can now be triggered by inserts (in addition to the existing update and delete commands), ensuring the visibility map is updating in time. This
allows better tuning of freezing tuples while they are still in buffer cache.
• Monitoring capabilities : WAL usage visibility in EXPLAIN, enhanced logging options, new system views for monitoring shared memory and LRU buffer usage, and more.
• WITH TIES addition to FETCH FIRST : To ease paging, simplify processing and reduce number of statements, FETCH FIRST WITH TIES returns any additional rows that tie
for the last place in the result set according to the ORDER BY clause.

SRA OSS, Inc. PostgreSQL 13 検証レポート ( 검증보고서 )
主な追加機能 ( 주요 추가 기능 )
• B-Treeインデックスのパフォーマンスを向上させる ( B-Tree 인덱스의 성능 향상 )
• 新しい実行計画 ( 새로운 실행 계획 )
• パーティションテーブル機能の拡張 ( 파티션 테이블 기능의 확장 )
• 並列VACUUM ( 병렬 VACUUM )
• 進捗レポートの表示拡張 ( 진척 보고서보기 확장 )
• pgbenchの拡張 ( pgbench 의 확장 )
• pg_rewindの拡張 ( pg_rewind 의 확장 )
• SQLの機能を追加 ( SQL 기능 추가 )
この他にも、機能追加や変更がたくさんあります。すべての変更点の一覧は、PostgreSQL13文書のリリースノートに記
載されています。
( 이 외에도, 기능 추가나 변경이 많이 있습니다. 모든 변경 사항 목록은 PostgreSQL 13 문서의 릴리스 노트에 기재되어
있습니다. )

What’s New in PostgreSQL 13 by EDB
• Vacuum up to 3X faster
• Avoid man-in-the-middle attacks for passwords
• Reduce errors in production
• Verify full backups
• Join partitioned tables easily
• Logical replication for partitioned tables
• De-duplication of B-tree indexes
• Other security enhancements

First Experience with PostgreSQL 13 Beta 1 by Alicloud
New Features in PostgreSQL 13
• Functionality
+ B-tree indexes – Improvements in handling the duplicate data which help to shrink index size and improve lookup speed, particularly for indexes that contain repeated values.
+ Incremental sorting capabilities –To get better planning and performance gains.
+ Improved partitioning functionality – To improve overall query execution time.
+ Added features like FETCH FIRST WITH TIES– To make more convenience while writing queries.
• Administration
+ Introduction of the concept of a “trusted extension”, which allows for a superuser to specify extensions that a user can install in their database so long as they have a CREATE
+ The ability for the VACUUM command to process indexes in parallel.
+ Added parallelism for the reindex db
+ Tracking of WAL usage statistics and the progress of streaming base backups and ANALYZE
• Security
+ Several new parameters included in the connection library libpq to help secure connections.
+ Introduction of the channel_binding connection parameters, which lets a client specify that they want to require the channel binding functionality as part of SCRAM.
+ Added support for DER encoded certificates
+ Permits using the ssl password parameter while specifying passwords in TLS certificate.
• Other Highlights
+ Improvement inoperability on Windows, as now users who run PostgreSQL on Windows now have the option to connect over UNIX domain sockets.
+ Addition of glossary of terms in documentation to help people familiarize themselves with both PostgreSQL and general database concepts.
+ pgbench utility supports the ability to partition its “accounts” table, making it easier to benchmark workloads that contain partitions.
+ Inclusion of warn command in psql which is similar to the echo command in terms of output data, except warn sends it to stderr.
• Summary
Although PostgreSQL 13 did not introduce the planned TDE and zheap features, it still has many eye-catching features, including deduplication for B-tree indexes, parallel VACUUM, and
disk usage by hash aggregation. Moreover, you can use a new tool called pg_verifybackup to verify the integrity of a backup and use pg_rewind to configure standby PostgreSQL instances.
pg_rewind can also use the restore_command of the target instance to fetch needed write-ahead logs. If you are interested, you can download the source code, compile it, analyze the
implementation of the features you are interested in. You may get better ideas.

PostgreSQL 13 New Features With Examples (GA) by HPE Japan’s noriyoshi shinoda
• https://twitter.com/nori_shinoda
1.5. The Question, comment, and Responsibility
The contents of this document are not an official opinion of Hewlett Packard Enterprise Japan Co, Ltd. The
author and affiliation company do not take any responsibility for the problem caused by the mistake of
contents. If you have any comments for this document, please contact Noriyoshi Shinoda
(noriyoshi.shinoda@hpe.com) Hewlett Packard Enterprise Japan Co, Ltd.

PostgreSQL 13 New Features With Examples (GA) by HPE Japan’s noriyoshi shinoda
New Features Summary ............... 7
2.1. Improve analytic query performance.. 7
2.2. Improve reliability ................ 7
2.3. Improved maintainability....... 8
2.4. Preparing for future new features....... 8
2.5. Incompatibility ................... 10
2.5.1. configure....... 10
2.5.2. createuser..................... 10
2.5.3. CSV log format ............ 10
2.5.4. Extension..................... 10
2.5.5. opaque...........11
2.5.6. Require version of OpenSSL .........11
2.5.7. pg_regress.....................11
2.5.8. psql default prompt........11
2.5.9. to_date/to_timestamp................... 12
2.5.10. Promotion during recovery ......... 12
2.5.11. Partition key ............... 12
2.5.12. Wait Event.................. 12
2.5.13. SIMILAR TO ESCAPE.............. 13
3. New Feature Detail.................... 14
3.1. Architecture......... 14
3.1.1. Modified catalogs ......... 14
3.1.2. Data types .................... 19
3.1.2. Data types .................... 19
3.1.3. Disk-based hash aggregation ........ 20
3.1.4. Incremental sort............ 21
3.1.5. Backup manifests.......... 22
3.1.6. Partitioned table............ 24
3.1.7. Log output for Autovacuum.......... 27
3.1.8. Wait events................... 27
3.1.9. libpq connection string................. 29
3.1.10. libpq functions............ 30
3.1.11. Hook........... 30
3.1.12. Column trigger ........... 30
3.1.13. Local connection key ................. 30
3.1.14. Trusted Extension ....... 31
3.1.15. Replication slot........... 33
3.1.16. Text search ................. 33
3.2. SQL statement .................... 35
3.2.1. ALTER NO DEPENDS ON.......... 35
3.2.2. ALTER STATISTICS SET
STATISTICS ...... 35
3.2.3. ALTER TABLE ............ 36
3.2.4. ALTER TYPE .............. 38
3.2.5. ALTER VIEW.............. 39
3.2.6. CREATE DATABASE ................. 40
3.2.7. CREATE INDEX.......... 41
3.2.8. CREATE TABLE ......... 42
3.2.9. CREATE TABLESPACE ............. 42
3.2.10. DROP DATABASE FORCE....... 43
3.2.11. EXPLAIN ANALYZE................ 43
3.2.12. INSERT ..................... 47
3.2.13. JSON.......... 48
3.2.14. MAX/MIN pg_lsn....... 48
3.2.15. ROW .......... 48
3.2.16. SELECT FETCH FIRST WITH TIES .
49
3.2.17. VACUUM PARALLEL.............. 49
3.2.18. Operator <->............... 50
3.2.19. Functions ................... 50
3.3. Configuration parameters................ 55
3.3.1. Added parameters ......... 55
3.3.2. Changed parameters...... 58
3.3.3. Parameters with default values
changed........ 58
3.4. Utilities ............... 59
3.4.1. dropdb .......... 59
3.4.2. pg_basebackup ............. 59
3.4.3. pg_dump....... 60
3.4.4. pg_rewind.................... 62
3.4.5. pg_verifybackup........... 63
3.4.6. pg_waldump................. 66
3.4.7. psql .............. 66
3.4.8. reindexdb..................... 68
3.4.9. vacuumdb .................... 69
3.4.10. Other .......... 69
3.5. Contrib modules.................. 70
3.5.1. adminpack.................... 70
3.5.2. auto_explain................. 70
3.5.3. dict_int ......... 71
3.5.4. ltree .............. 72
3.5.5. pageinspect .................. 72
3.5.6. pg_stat_statements........ 74
3.5.7. postgres_fdw................ 76
3.5.8. bool_plperl................... 77
URL List ........ 79
Change History.............. 80

B-Tree vs B+Tree Structure
B+tree
Type Tree (data structure)
Invented 1972
TAOCP ?!
Time complexity in big O notation
Algorithm Average Worst case
Space O(n) O(n)
Search O(log n) O(log n + log L)
Insert O(log n) O(M*log n + log L)
Delete O(log n) O(M*log n + log L)
B-tree
Type Tree (data structure)
Invented 1970
Rudolf Bayer, Edward M. McCreight
Time complexity in big O notation
Algorithm Average Worst case
Space O(n) O(n)
Search O(log n) O(log n)
Insert O(log n) O(log n)
Delete O(log n) O(log n)

Deduplication in B-Tree Indexes
$ CREATE TABLE tbl ( col TEXT COLLATE “C” );
$ CREATE INDEX CONCURRENTLY idx_yes_dedup ON tbl(col) WITH (deduplicate_items=on); -- default
$ ALTER INDEX CONCURRENTLY idx_no_dedup ON tbl(col) SET (deduplicate_items=off);
Note that deduplication is deemed unsafe and cannot be used in the following cases involving semantically significant differences among equal datums:
• text, varchar, and char cannot use deduplication when a nondeterministic collation is used. Case and accent differences must be preserved among equal datums.
• numeric cannot use deduplication. Numeric display scale must be preserved among equal datums.
• jsonb cannot use deduplication, since the jsonb B-Tree operator class uses numeric internally.
• float4 and float8 cannot use deduplication. These types have distinct representations for -0 and 0, which are nevertheless considered equal. This difference must
be preserved.
There is one further implementation-level restriction that may be lifted in a future version of PostgreSQL:
• Container types (such as composite types, arrays, or range types) cannot use deduplication.
There is one further implementation-level restriction that applies regardless of the operator class or collation used:
• INCLUDE indexes can never use deduplication.
• pg_upgrade 🡺 REINDEX

Smaller Indexes with B-Tree Deduplication

Partitioning Improvements in PostgreSQL 13

Partitioning Improvements in PostgreSQL 13
• Improve cases where pruning of partitions can happen (Yuzuko Hosoya, Amit Langote, Á lvaro Herrera)
• Allow partition wise joins to happen in more cases (Ashutosh Bapat, Etsuro Fujita, Amit Langote, Tom Lane) – For example, partition
wise joins can now happen between partitioned tables even when their partition bounds do not match exactly.
• Allow BEFORE row-level triggers on partitioned tables (Á lvaro Herrera) – These triggers cannot change which partition is the destination.
• Allow partitioned tables to be logically replicated via publications (Amit Langote)
• Allow logical replication into partitioned tables on subscribers (Amit Langote) – Previously, subscribers could only receive rows into non-
partitioned tables.
• Allow ROW values to be used as partitioning expressions (Amit Langote)

• Postgres multi-version concurrency control means you need to run VACUUM regularly (usually you can rely on the autovacuum process, though it may need some
tuning). In Postgres 13, one notable improvement is that multiple indexes for a single table can be vacuumed in parallel. This can lead to big performance
improvements in VACUUM work. Parallel VACUUM is the default and can be controlled with the PARALLEL option:
# VACUUM (PARALLEL 2, VERBOSE) queries;
Parallel VACUUM occurs when the following is true:
+ Sufficient parallel workers are available, based on the system-wide limit set by max_parallel_maintenance_workers (defaults to 2)
+ There are multiple indexes on the table (one index can be processed by one worker at a time)
+ Index types support it (all built-in index types support parallelism to some extent)
+ The indexes are large enough to exceed min_parallel_index_scan_size (defaults to 512 kB)
• Be aware that parallel VACUUM is currently not supported for autovacuum. This new feature is intended for use in manual VACUUM runs that need to
complete quickly, such as when insufficient autovacuum tuning has lead to an imminent TXID wraparound, and you need to intervene to fix it.
• On that note, an important autovacuum improvement in Postgres 13 is that the autovacuum background process can now be triggered by INSERT statements for
append-only tables. The main purpose of VACUUM is to clean up old versions of updated and deleted rows, but it is also essential to set pages as all-visible for
MVCC bookkeeping. All-visible pages allow index-only scans to avoid checking visibility status row-by-row, making them faster.
• We make extensive use of append-only tables at pganalyze for our timeseries data, and this improvement will make our lives considerably easier, avoiding the
occasional manual VACUUM run on these tables. This new behavior can be controlled by the autovacuum_vacuum_insert_threshold and
autovacuum_vacuum_insert_scale_factor variables.
Parallel VACUUM & Better Support for Append-only Workloads

Incremental Sorting
• Sorting data is a common database task, and Postgres has a number of features to avoid unnecessary work here. For
example, if you have a B-Tree index on a column, and you query your table ordered by that column, it can just scan that
index in order to get sorted data.
• In Postgres 13, this is improved to handle partially sorted data. If you have an index on (a, b) (or the data is already
sorted by (a, b) for another reason), and you issue a query to order by (a, b, c), Postgres understands that the input
data is already partially sorted, and can avoid re-sorting the whole dataset. This is especially useful if you have a LIMIT in
your query, since this can avoid even more work.
# SET enable_incremental_sort TO on;
# EXPLAIN (ANALYZE) SELECT * FROM t_log WHERE dt > ' 2020-05-25 14:10' ORDER BY dt, severity, message LIMIT 1000;
QUERY PLAN
---------------------------------------------------------------------------
Limit (cost=0.38..74.06 rows=1000 width=46)
(actual time=0.031..0.599 rows=1000 loops=1)
-> Incremental Sort (cost=0.38..735.25 rows=9974 width=46) (actual time=0.030..0.534 rows=1000 loops=1) Sort Key: dt, severity, message
Presorted Key: dt
Full-sort Groups: 32 Sort Method: quicksort Average Memory: 27kB Peak Memory: 27kB
…

Extended Statistics Improvements in Postgres 13
• Postgres 10 introduced the concept of extended statistics. Postgres keeps some statistics about the "shape" of your data to ensure it
can plan queries efficiently, but the statistics kept by default cannot track things like inter-column dependencies. Extended statistics were
introduced to address that: These are database objects (like indexes) that you create manually with CREATE STATISTICS to give the
query planner more information for more specific situations. These would be expensive for Postgres to determine automatically, but armed
with an understanding of the semantics of your schema, you can provide that additional info. Used carefully, this can lead to massive
performance improvements. ( How we used Postgres extended statistics to achieve a 3000x speedup )
• Postgres 13 brings a number of small but important improvements to extended statistics, including support for using them with OR clauses
and in IN/ANY constant lists, allowing consideration of multiple extended statistics objects in planning a query, and support for setting a
statistics target for extended statistics:
• Like with the regular statistics target, this is a trade-off between additional planning time (and longer ANALYZE runs), versus having
more precise plans. We recommend using this in a targeted manner using EXPLAIN plans to confirm plan changes.
# ALTER STATISTICS name OWNER TO { new_owner | CURRENT_USER | SESSION_USER }
# ALTER STATISTICS name RENAME TO new_name
# ALTER STATISTICS name SET SCHEMA new_schema
# ALTER STATISTICS name SET STATISTICS new_target

Execution plan creation and WAL output tracking
WAL Usage Stats
• The write-ahead log (WAL) ensures your data stays consistent in the event of a crash, even mid-write. Consistency is a fundamental property of databases—it
ensures your transaction either committed or did not commit; you don't have to worry about in-between states. But on a busy system, WAL writes can often be a
bottleneck. To help diagnose this, Postgres 13 includes more information on WAL usage from your queries.
• EXPLAIN now supports information about WAL records generated during execution:
# EXPLAIN (ANALYZE, WAL) DELETE FROM users;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------
Delete on users (cost=0.00..5409.00 rows=100000 width=6) (actual time=108.910..108.911 rows=0 loops=1)
WAL: records=100000 fpi=741 bytes=11425721
-> Seq Scan on users (cost=0.00..5409.00 rows=100000 width=6) (actual time=8.519..51.850 rows=100000 loops=1)
…
# SELECT query, calls, wal_records, wal_fpi, wal_bytes FROM pg_stat_statements ORDER BY wal_records DESC LIMIT 1;
-[ RECORD 1 ]---------------------------------------------------------------------------------------------
query | CREATE TEMPORARY TABLE upsert_data (server_id uuid NOT NULL, backend_id uuid NOT NULL,
| query_start timestamp NOT NULL, query_fingerprint bytea NOT NULL, query_text text NOT NULL)
calls | 7974948
wal_records | 966960816
wal_fpi | 1018412
wal_bytes | 100086092097

[PGDay.Seoul 2020] PostgreSQL 13 New Features

More Related Content

What's hot (20)

Similar to [PGDay.Seoul 2020] PostgreSQL 13 New Features (20)

More from hyeongchae lee (12)

Recently uploaded (20)

[PGDay.Seoul 2020] PostgreSQL 13 New Features