Doc

Backup and Restore

backup

As with everything that contains valuable data, PostgreSQL databases should be backed up regularly. While the procedure is essentially simple, it is important to have a clear understanding of the underlying techniques and assumptions.

There are three fundamentally different approaches to backing up PostgreSQL data:

Each has its own strengths and weaknesses; each is discussed in turn in the following sections.

SQL Dump

The idea behind this dump method is to generate a file with SQL commands that, when fed back to the server, will recreate the database in the same state as it was at the time of the dump. PostgreSQL provides the utility program ??? for this purpose. The basic usage of this command is: pg_dump <dbname> > <dumpfile> As you see, pg_dump writes its result to the standard output. We will see below how this can be useful. While the above command creates a text file, pg_dump can create files in other formats that allow for parallelism and more fine-grained control of object restoration.

pg_dump is a regular PostgreSQL client application (albeit a particularly clever one). This means that you can perform this backup procedure from any remote host that has access to the database. But remember that pg_dump does not operate with special permissions. In particular, it must have read access to all tables that you want to back up, so in order to back up the entire database you almost always have to run it as a database superuser. (If you do not have sufficient privileges to back up the entire database, you can still back up portions of the database to which you do have access using options such as -n schema or -t table.)

To specify which database server pg_dump should contact, use the command line options -h host and -p port. The default host is the local host or whatever your PGHOST environment variable specifies. Similarly, the default port is indicated by the PGPORT environment variable or, failing that, by the compiled-in default. (Conveniently, the server will normally have the same compiled-in default.)

Like any other PostgreSQL client application, pg_dump will by default connect with the database user name that is equal to the current operating system user name. To override this, either specify the -U option or set the environment variable PGUSER. Remember that pg_dump connections are subject to the normal client authentication mechanisms (which are described in ???).

An important advantage of pg_dump over the other backup methods described later is that pg_dump's output can generally be re-loaded into newer versions of PostgreSQL, whereas file-level backups and continuous archiving are both extremely server-version-specific. pg_dump is also the only method that will work when transferring a database to a different machine architecture, such as going from a 32-bit to a 64-bit server.

Dumps created by pg_dump are internally consistent, meaning, the dump represents a snapshot of the database at the time pg_dump began running. pg_dump does not block other operations on the database while it is working. (Exceptions are those operations that need to operate with an exclusive lock, such as most forms of ALTER TABLE.)

Restoring the Dump

Text files created by pg_dump are intended to be read by the psql program using its default settings. The general command form to restore a text dump is psql -X <dbname> < <dumpfile> where <dumpfile> is the file output by the pg_dump command. The database <dbname> will not be created by this command, so you must create it yourself from template0 before executing psql (e.g., with createdb -T template0 dbname). To ensure psql runs with its default settings, use the -X (--no-psqlrc) option. psql supports options similar to pg_dump for specifying the database server to connect to and the user name to use. See the ??? reference page for more information.

Non-text file dumps should be restored using the ??? utility.

Before restoring an SQL dump, all the users who own objects or were granted permissions on objects in the dumped database must already exist. If they do not, the restore will fail to recreate the objects with the original ownership and/or permissions. (Sometimes this is what you want, but usually it is not.)

By default, the psql script will continue to execute after an SQL error is encountered. You might wish to run psql with the ON_ERROR_STOP variable set to alter that behavior and have psql exit with an exit status of 3 if an SQL error occurs:

psql -X --set ON_ERROR_STOP=on dbname < dumpfile

Either way, you will only have a partially restored database. Alternatively, you can specify that the whole dump should be restored as a single transaction, so the restore is either fully completed or fully rolled back. This mode can be specified by passing the -1 or --single-transaction command-line options to psql. When using this mode, be aware that even a minor error can rollback a restore that has already run for many hours. However, that might still be preferable to manually cleaning up a complex database after a partially restored dump.

The ability of pg_dump and psql to write to or read from pipes makes it possible to dump a database directly from one server to another, for example:

pg_dump -h host1 dbname | psql -X -h host2 dbname

[!IMPORTANT] The dumps produced by pg_dump are relative to template0. This means that any languages, procedures, etc. added via template1 will also be dumped by pg_dump. As a result, when restoring, if you are using a customized template1, you must create the empty database from template0, as in the example above.

After restoring a backup, it is wise to run ANALYZE on each database so the query optimizer has useful statistics; see ??? and ??? for more information. For more advice on how to load large amounts of data into PostgreSQL efficiently, refer to ???.

Using pg_dumpall

pg_dump dumps only a single database at a time, and it does not dump information about roles or tablespaces (because those are cluster-wide rather than per-database). To support convenient dumping of the entire contents of a database cluster, the ??? program is provided. pg_dumpall backs up each database in a given cluster, and also preserves cluster-wide data such as role and tablespace definitions. The basic usage of this command is: pg_dumpall > <dumpfile> The resulting dump can be restored with psql: psql -X -f <dumpfile> postgres (Actually, you can specify any existing database name to start from, but if you are loading into an empty cluster then postgres should usually be used.) It is always necessary to have database superuser access when restoring a pg_dumpall dump, as that is required to restore the role and tablespace information. If you use tablespaces, make sure that the tablespace paths in the dump are appropriate for the new installation.

pg_dumpall works by emitting commands to re-create roles, tablespaces, and empty databases, then invoking pg_dump for each database. This means that while each database will be internally consistent, the snapshots of different databases are not synchronized.

Cluster-wide data can be dumped alone using the pg_dumpall --globals-only option. This is necessary to fully backup the cluster if running the pg_dump command on individual databases.

Handling Large Databases

Some operating systems have maximum file size limits that cause problems when creating large pg_dump output files. Fortunately, pg_dump can write to the standard output, so you can use standard Unix tools to work around this potential problem. There are several possible methods:

Use compressed dumps.

You can use your favorite compression program, for example gzip:

pg_dump dbname | gzip > filename.gz

Reload with:

gunzip -c filename.gz | psql dbname

or:

cat filename.gz | gunzip | psql dbname

Use split.

The split command allows you to split the output into smaller files that are acceptable in size to the underlying file system. For example, to make 2 gigabyte chunks:

pg_dump dbname | split -b 2G - filename

Reload with:

cat filename* | psql dbname

If using GNU split, it is possible to use it and gzip together:

pg_dump dbname | split -b 2G --filter='gzip > $FILE.gz'

It can be restored using zcat.

Use pg_dump's custom dump format.

If PostgreSQL was built on a system with the zlib compression library installed, the custom dump format will compress data as it writes it to the output file. This will produce dump file sizes similar to using gzip, but it has the added advantage that tables can be restored selectively. The following command dumps a database using the custom dump format:

pg_dump -Fc dbname > filename

A custom-format dump is not a script for psql, but instead must be restored with pg_restore, for example:

pg_restore -d dbname filename

See the ??? and ??? reference pages for details.

For very large databases, you might need to combine split with one of the other two approaches.

Use pg_dump's parallel dump feature.

To speed up the dump of a large database, you can use pg_dump's parallel mode. This will dump multiple tables at the same time. You can control the degree of parallelism with the -j parameter. Parallel dumps are only supported for the "directory" archive format.

pg_dump -j num -F d -f out.dir dbname

You can use pg_restore -j to restore a dump in parallel. This will work for any archive of either the "custom" or the "directory" archive mode, whether or not it has been created with pg_dump -j.

File System Level Backup

An alternative backup strategy is to directly copy the files that PostgreSQL uses to store the data in the database; ??? explains where these files are located. You can use whatever method you prefer for doing file system backups; for example:

tar -cf backup.tar /usr/local/pgsql/data

There are two restrictions, however, which make this method impractical, or at least inferior to the pg_dump method:

  1. The database server must be shut down in order to get a usable backup. Half-way measures such as disallowing all connections will not work (in part because tar and similar tools do not take an atomic snapshot of the state of the file system, but also because of internal buffering within the server). Information about stopping the server can be found in ???. Needless to say, you also need to shut down the server before restoring the data.

  2. If you have dug into the details of the file system layout of the database, you might be tempted to try to back up or restore only certain individual tables or databases from their respective files or directories. This will not work because the information contained in these files is not usable without the commit log files, pg_xact/*, which contain the commit status of all transactions. A table file is only usable with this information. Of course it is also impossible to restore only a table and the associated pg_xact data because that would render all other tables in the database cluster useless. So file system backups only work for complete backup and restoration of an entire database cluster.

An alternative file-system backup approach is to make a “consistent snapshot” of the data directory, if the file system supports that functionality (and you are willing to trust that it is implemented correctly). The typical procedure is to make a “frozen snapshot” of the volume containing the database, then copy the whole data directory (not just parts, see above) from the snapshot to a backup device, then release the frozen snapshot. This will work even while the database server is running. However, a backup created in this way saves the database files in a state as if the database server was not properly shut down; therefore, when you start the database server on the backed-up data, it will think the previous server instance crashed and will replay the WAL log. This is not a problem; just be aware of it (and be sure to include the WAL files in your backup). You can perform a CHECKPOINT before taking the snapshot to reduce recovery time.

If your database is spread across multiple file systems, there might not be any way to obtain exactly-simultaneous frozen snapshots of all the volumes. For example, if your data files and WAL log are on different disks, or if tablespaces are on different file systems, it might not be possible to use snapshot backup because the snapshots must be simultaneous. Read your file system documentation very carefully before trusting the consistent-snapshot technique in such situations.

If simultaneous snapshots are not possible, one option is to shut down the database server long enough to establish all the frozen snapshots. Another option is to perform a continuous archiving base backup (Making a Base Backup) because such backups are immune to file system changes during the backup. This requires enabling continuous archiving just during the backup process; restore is done using continuous archive recovery (Recovering Using a Continuous Archive Backup).

Another option is to use rsync to perform a file system backup. This is done by first running rsync while the database server is running, then shutting down the database server long enough to do an rsync --checksum. (--checksum is necessary because rsync only has file modification-time granularity of one second.) The second rsync will be quicker than the first, because it has relatively little data to transfer, and the end result will be consistent because the server was down. This method allows a file system backup to be performed with minimal downtime.

Note that a file system backup will typically be larger than an SQL dump. (pg_dump does not need to dump the contents of indexes for example, just the commands to recreate them.) However, taking a file system backup might be faster.

Continuous Archiving and Point-in-Time Recovery (PITR)

continuous archiving

point-in-time recovery

PITR

At all times, PostgreSQL maintains a write ahead log (WAL) in the pg_wal/ subdirectory of the cluster's data directory. The log records every change made to the database's data files. This log exists primarily for crash-safety purposes: if the system crashes, the database can be restored to consistency by “replaying” the log entries made since the last checkpoint. However, the existence of the log makes it possible to use a third strategy for backing up databases: we can combine a file-system-level backup with backup of the WAL files. If recovery is needed, we restore the file system backup and then replay from the backed-up WAL files to bring the system to a current state. This approach is more complex to administer than either of the previous approaches, but it has some significant benefits:

[!NOTE] pg_dump and pg_dumpall do not produce file-system-level backups and cannot be used as part of a continuous-archiving solution. Such dumps are logical and do not contain enough information to be used by WAL replay.

As with the plain file-system-backup technique, this method can only support restoration of an entire database cluster, not a subset. Also, it requires a lot of archival storage: the base backup might be bulky, and a busy system will generate many megabytes of WAL traffic that have to be archived. Still, it is the preferred backup technique in many situations where high reliability is needed.

To recover successfully using continuous archiving (also called “online backup” by many database vendors), you need a continuous sequence of archived WAL files that extends back at least as far as the start time of your backup. So to get started, you should set up and test your procedure for archiving WAL files before you take your first base backup. Accordingly, we first discuss the mechanics of archiving WAL files.

Setting Up WAL Archiving

In an abstract sense, a running PostgreSQL system produces an indefinitely long sequence of WAL records. The system physically divides this sequence into WAL segment files, which are normally 16MB apiece (although the segment size can be altered during initdb). The segment files are given numeric names that reflect their position in the abstract WAL sequence. When not using WAL archiving, the system normally creates just a few segment files and then “recycles” them by renaming no-longer-needed segment files to higher segment numbers. It's assumed that segment files whose contents precede the last checkpoint are no longer of interest and can be recycled.

When archiving WAL data, we need to capture the contents of each segment file once it is filled, and save that data somewhere before the segment file is recycled for reuse. Depending on the application and the available hardware, there could be many different ways of “saving the data somewhere”: we could copy the segment files to an NFS-mounted directory on another machine, write them onto a tape drive (ensuring that you have a way of identifying the original name of each file), or batch them together and burn them onto CDs, or something else entirely. To provide the database administrator with flexibility, PostgreSQL tries not to make any assumptions about how the archiving will be done. Instead, PostgreSQL lets the administrator specify a shell command or an archive library to be executed to copy a completed segment file to wherever it needs to go. This could be as simple as a shell command that uses cp, or it could invoke a complex C function — it's all up to you.

To enable WAL archiving, set the ??? configuration parameter to replica or higher, ??? to on, specify the shell command to use in the ??? configuration parameter or specify the library to use in the ??? configuration parameter. In practice these settings will always be placed in the postgresql.conf file.

In archive_command, %p is replaced by the path name of the file to archive, while %f is replaced by only the file name. (The path name is relative to the current working directory, i.e., the cluster's data directory.) Use %% if you need to embed an actual % character in the command. The simplest useful command is something like:

archive_command = 'test ! -f "/mnt/server/archivedir/%f" && cp "%p" "/mnt/server/archivedir/%f"'  # Unix
archive_command = 'copy "%p" "C:\\server\\archivedir\\%f"'  # Windows

which will copy archivable WAL segments to the directory /mnt/server/archivedir. (This is an example, not a recommendation, and might not work on all platforms.) After the %p and %f parameters have been replaced, the actual command executed might look like this:

test ! -f /mnt/server/archivedir/00000001000000A900000065 && cp pg_wal/00000001000000A900000065 /mnt/server/archivedir/00000001000000A900000065

A similar command will be generated for each new file to be archived.

The archive command will be executed under the ownership of the same user that the PostgreSQL server is running as. Since the series of WAL files being archived contains effectively everything in your database, you will want to be sure that the archived data is protected from prying eyes; for example, archive into a directory that does not have group or world read access.

It is important that the archive command return zero exit status if and only if it succeeds. Upon getting a zero result, PostgreSQL will assume that the file has been successfully archived, and will remove or recycle it. However, a nonzero status tells PostgreSQL that the file was not archived; it will try again periodically until it succeeds.

Another way to archive is to use a custom archive module as the archive_library. Since such modules are written in C, creating your own may require considerably more effort than writing a shell command. However, archive modules can be more performant than archiving via shell, and they will have access to many useful server resources. For more information about archive modules, see ???.

When the archive command is terminated by a signal (other than SIGTERM that is used as part of a server shutdown) or an error by the shell with an exit status greater than 125 (such as command not found), or if the archive function emits an ERROR or FATAL, the archiver process aborts and gets restarted by the postmaster. In such cases, the failure is not reported in ???.

Archive commands and libraries should generally be designed to refuse to overwrite any pre-existing archive file. This is an important safety feature to preserve the integrity of your archive in case of administrator error (such as sending the output of two different servers to the same archive directory). It is advisable to test your proposed archive library to ensure that it does not overwrite an existing file.

In rare cases, PostgreSQL may attempt to re-archive a WAL file that was previously archived. For example, if the system crashes before the server makes a durable record of archival success, the server will attempt to archive the file again after restarting (provided archiving is still enabled). When an archive command or library encounters a pre-existing file, it should return a zero status or true, respectively, if the WAL file has identical contents to the pre-existing archive and the pre-existing archive is fully persisted to storage. If a pre-existing file contains different contents than the WAL file being archived, the archive command or library must return a nonzero status or false, respectively.

The example command above for Unix avoids overwriting a pre-existing archive by including a separate test step. On some Unix platforms, cp has switches such as -i that can be used to do the same thing less verbosely, but you should not rely on these without verifying that the right exit status is returned. (In particular, GNU cp will return status zero when -i is used and the target file already exists, which is not the desired behavior.)

While designing your archiving setup, consider what will happen if the archive command or library fails repeatedly because some aspect requires operator intervention or the archive runs out of space. For example, this could occur if you write to tape without an autochanger; when the tape fills, nothing further can be archived until the tape is swapped. You should ensure that any error condition or request to a human operator is reported appropriately so that the situation can be resolved reasonably quickly. The pg_wal/ directory will continue to fill with WAL segment files until the situation is resolved. (If the file system containing pg_wal/ fills up, PostgreSQL will do a PANIC shutdown. No committed transactions will be lost, but the database will remain offline until you free some space.)

The speed of the archive command or library is unimportant as long as it can keep up with the average rate at which your server generates WAL data. Normal operation continues even if the archiving process falls a little behind. If archiving falls significantly behind, this will increase the amount of data that would be lost in the event of a disaster. It will also mean that the pg_wal/ directory will contain large numbers of not-yet-archived segment files, which could eventually exceed available disk space. You are advised to monitor the archiving process to ensure that it is working as you intend.

In writing your archive command or library, you should assume that the file names to be archived can be up to 64 characters long and can contain any combination of ASCII letters, digits, and dots. It is not necessary to preserve the original relative path (%p) but it is necessary to preserve the file name (%f).

Note that although WAL archiving will allow you to restore any modifications made to the data in your PostgreSQL database, it will not restore changes made to configuration files (that is, postgresql.conf, pg_hba.conf and pg_ident.conf), since those are edited manually rather than through SQL operations. You might wish to keep the configuration files in a location that will be backed up by your regular file system backup procedures. See ??? for how to relocate the configuration files.

The archive command or function is only invoked on completed WAL segments. Hence, if your server generates only little WAL traffic (or has slack periods where it does so), there could be a long delay between the completion of a transaction and its safe recording in archive storage. To put a limit on how old unarchived data can be, you can set ??? to force the server to switch to a new WAL segment file at least that often. Note that archived files that are archived early due to a forced switch are still the same length as completely full files. It is therefore unwise to set a very short archive_timeout — it will bloat your archive storage. archive_timeout settings of a minute or so are usually reasonable.

Also, you can force a segment switch manually with pg_switch_wal if you want to ensure that a just-finished transaction is archived as soon as possible. Other utility functions related to WAL management are listed in ???.

When wal_level is minimal some SQL commands are optimized to avoid WAL logging, as described in ???. If archiving or streaming replication were turned on during execution of one of these statements, WAL would not contain enough information for archive recovery. (Crash recovery is unaffected.) For this reason, wal_level can only be changed at server start. However, archive_command and archive_library can be changed with a configuration file reload. If you are archiving via shell and wish to temporarily stop archiving, one way to do it is to set archive_command to the empty string (''). This will cause WAL files to accumulate in pg_wal/ until a working archive_command is re-established.

Making a Base Backup

The easiest way to perform a base backup is to use the ??? tool. It can create a base backup either as regular files or as a tar archive. If more flexibility than ??? can provide is required, you can also make a base backup using the low level API (see Making a Base Backup Using the Low Level API).

It is not necessary to be concerned about the amount of time it takes to make a base backup. However, if you normally run the server with full_page_writes disabled, you might notice a drop in performance while the backup runs since full_page_writes is effectively forced on during backup mode.

To make use of the backup, you will need to keep all the WAL segment files generated during and after the file system backup. To aid you in doing this, the base backup process creates a backup history file that is immediately stored into the WAL archive area. This file is named after the first WAL segment file that you need for the file system backup. For example, if the starting WAL file is 0000000100001234000055CD the backup history file will be named something like 0000000100001234000055CD.007C9330.backup. (The second part of the file name stands for an exact position within the WAL file, and can ordinarily be ignored.) Once you have safely archived the file system backup and the WAL segment files used during the backup (as specified in the backup history file), all archived WAL segments with names numerically less are no longer needed to recover the file system backup and can be deleted. However, you should consider keeping several backup sets to be absolutely certain that you can recover your data.

The backup history file is just a small text file. It contains the label string you gave to ???, as well as the starting and ending times and WAL segments of the backup. If you used the label to identify the associated dump file, then the archived history file is enough to tell you which dump file to restore.

Since you have to keep around all the archived WAL files back to your last base backup, the interval between base backups should usually be chosen based on how much storage you want to expend on archived WAL files. You should also consider how long you are prepared to spend recovering, if recovery should be necessary — the system will have to replay all those WAL segments, and that could take awhile if it has been a long time since the last base backup.

Making an Incremental Backup

You can use ??? to take an incremental backup by specifying the --incremental option. You must supply, as an argument to --incremental, the backup manifest to an earlier backup from the same server. In the resulting backup, non-relation files will be included in their entirety, but some relation files may be replaced by smaller incremental files which contain only the blocks which have been changed since the earlier backup and enough metadata to reconstruct the current version of the file.

To figure out which blocks need to be backed up, the server uses WAL summaries, which are stored in the data directory, inside the directory pg_wal/summaries. If the required summary files are not present, an attempt to take an incremental backup will fail. The summaries present in this directory must cover all LSNs from the start LSN of the prior backup to the start LSN of the current backup. Since the server looks for WAL summaries just after establishing the start LSN of the current backup, the necessary summary files probably won't be instantly present on disk, but the server will wait for any missing files to show up. This also helps if the WAL summarization process has fallen behind. However, if the necessary files have already been removed, or if the WAL summarizer doesn't catch up quickly enough, the incremental backup will fail.

When restoring an incremental backup, it will be necessary to have not only the incremental backup itself but also all earlier backups that are required to supply the blocks omitted from the incremental backup. See ??? for further information about this requirement. Note that there are restrictions on the use of pg_combinebackup when the checksum status of the cluster has been changed; see pg_combinebackup limitations.

Note that all of the requirements for making use of a full backup also apply to an incremental backup. For instance, you still need all of the WAL segment files generated during and after the file system backup, and any relevant WAL history files. And you still need to create a recovery.signal (or standby.signal) and perform recovery, as described in Recovering Using a Continuous Archive Backup. The requirement to have earlier backups available at restore time and to use pg_combinebackup is an additional requirement on top of everything else. Keep in mind that PostgreSQL has no built-in mechanism to figure out which backups are still needed as a basis for restoring later incremental backups. You must keep track of the relationships between your full and incremental backups on your own, and be certain not to remove earlier backups if they might be needed when restoring later incremental backups.

Incremental backups typically only make sense for relatively large databases where a significant portion of the data does not change, or only changes slowly. For a small database, it's simpler to ignore the existence of incremental backups and simply take full backups, which are simpler to manage. For a large database all of which is heavily modified, incremental backups won't be much smaller than full backups.

An incremental backup is only possible if replay would begin from a later checkpoint than for the previous backup upon which it depends. If you take the incremental backup on the primary, this condition is always satisfied, because each backup triggers a new checkpoint. On a standby, replay begins from the most recent restartpoint. Therefore, an incremental backup of a standby server can fail if there has been very little activity since the previous backup, since no new restartpoint might have been created.

Making a Base Backup Using the Low Level API

Instead of taking a full or incremental base backup using ???, you can take a base backup using the low-level API. This procedure contains a few more steps than the pg_basebackup method, but is relatively simple. It is very important that these steps are executed in sequence, and that the success of a step is verified before proceeding to the next step.

Multiple backups are able to be run concurrently (both those started using this backup API and those started using ???).

  1. Ensure that WAL archiving is enabled and working.

  2. Connect to the server (it does not matter which database) as a user with rights to run pg_backup_start (superuser, or a user who has been granted EXECUTE on the function) and issue the command:

    SELECT pg_backup_start(label => 'label', fast => false);

    where label is any string you want to use to uniquely identify this backup operation. The connection calling pg_backup_start must be maintained until the end of the backup, or the backup will be automatically aborted.

    Online backups are always started at the beginning of a checkpoint. By default, pg_backup_start will wait for the next regularly scheduled checkpoint to complete, which may take a long time (see the configuration parameters ??? and ???). This is usually preferable as it minimizes the impact on the running system. If you want to start the backup as soon as possible, pass true as the second parameter to pg_backup_start and it will request a fast checkpoint, which will finish as fast as possible using as much I/O as possible.

  3. Perform the backup, using any convenient file-system-backup tool such as tar or cpio (not pg_dump or pg_dumpall). It is neither necessary nor desirable to stop normal operation of the database while you do this. See Backing Up the Data Directory for things to consider during this backup.

  4. In the same connection as before, issue the command:

    SELECT * FROM pg_backup_stop(wait_for_archive => true);

    This terminates backup mode. On a primary, it also performs an automatic switch to the next WAL segment. On a standby, it is not possible to automatically switch WAL segments, so you may wish to run pg_switch_wal on the primary to perform a manual switch. The reason for the switch is to arrange for the last WAL segment file written during the backup interval to be ready to archive.

    pg_backup_stop will return one row with three values. The second of these fields should be written to a file named backup_label in the root directory of the backup. The third field should be written to a file named tablespace_map unless the field is empty. These files are vital to the backup working and must be written byte for byte without modification, which may require opening the file in binary mode.

  5. Once the WAL segment files active during the backup are archived, you are done. The file identified by pg_backup_stop's first return value is the last segment that is required to form a complete set of backup files. On a primary, if archive_mode is enabled and the wait_for_archive parameter is true, pg_backup_stop does not return until the last segment has been archived. On a standby, archive_mode must be always in order for pg_backup_stop to wait. Archiving of these files happens automatically since you have already configured archive_command or archive_library. In most cases this happens quickly, but you are advised to monitor your archive system to ensure there are no delays. If the archive process has fallen behind because of failures of the archive command or library, it will keep retrying until the archive succeeds and the backup is complete. If you wish to place a time limit on the execution of pg_backup_stop, set an appropriate statement_timeout value, but make note that if pg_backup_stop terminates because of this your backup may not be valid.

    If the backup process monitors and ensures that all WAL segment files required for the backup are successfully archived then the wait_for_archive parameter (which defaults to true) can be set to false to have pg_backup_stop return as soon as the stop backup record is written to the WAL. By default, pg_backup_stop will wait until all WAL has been archived, which can take some time. This option must be used with caution: if WAL archiving is not monitored correctly then the backup might not include all of the WAL files and will therefore be incomplete and not able to be restored.

Backing Up the Data Directory

Some file system backup tools emit warnings or errors if the files they are trying to copy change while the copy proceeds. When taking a base backup of an active database, this situation is normal and not an error. However, you need to ensure that you can distinguish complaints of this sort from real errors. For example, some versions of rsync return a separate exit code for “vanished source files”, and you can write a driver script to accept this exit code as a non-error case. Also, some versions of GNU tar return an error code indistinguishable from a fatal error if a file was truncated while tar was copying it. Fortunately, GNU tar versions 1.16 and later exit with 1 if a file was changed during the backup, and 2 for other errors. With GNU tar version 1.23 and later, you can use the warning options --warning=no-file-changed --warning=no-file-removed to hide the related warning messages.

Be certain that your backup includes all of the files under the database cluster directory (e.g., /usr/local/pgsql/data). If you are using tablespaces that do not reside underneath this directory, be careful to include them as well (and be sure that your backup archives symbolic links as links, otherwise the restore will corrupt your tablespaces).

You should, however, omit from the backup the files within the cluster's pg_wal/ subdirectory. This slight adjustment is worthwhile because it reduces the risk of mistakes when restoring. This is easy to arrange if pg_wal/ is a symbolic link pointing to someplace outside the cluster directory, which is a common setup anyway for performance reasons. You might also want to exclude postmaster.pid and postmaster.opts, which record information about the running postmaster, not about the postmaster which will eventually use this backup. (These files can confuse pg_ctl.)

It is often a good idea to also omit from the backup the files within the cluster's pg_replslot/ directory, so that replication slots that exist on the primary do not become part of the backup. Otherwise, the subsequent use of the backup to create a standby may result in indefinite retention of WAL files on the standby, and possibly bloat on the primary if hot standby feedback is enabled, because the clients that are using those replication slots will still be connecting to and updating the slots on the primary, not the standby. Even if the backup is only intended for use in creating a new primary, copying the replication slots isn't expected to be particularly useful, since the contents of those slots will likely be badly out of date by the time the new primary comes on line.

The contents of the directories pg_dynshmem/, pg_notify/, pg_serial/, pg_snapshots/, pg_stat_tmp/, and pg_subtrans/ (but not the directories themselves) can be omitted from the backup as they will be initialized on postmaster startup.

Any file or directory beginning with pgsql_tmp can be omitted from the backup. These files are removed on postmaster start and the directories will be recreated as needed.

pg_internal.init files can be omitted from the backup whenever a file of that name is found. These files contain relation cache data that is always rebuilt when recovering.

The backup label file includes the label string you gave to pg_backup_start, as well as the time at which pg_backup_start was run, and the name of the starting WAL file. In case of confusion it is therefore possible to look inside a backup file and determine exactly which backup session the dump file came from. The tablespace map file includes the symbolic link names as they exist in the directory pg_tblspc/ and the full path of each symbolic link. These files are not merely for your information; their presence and contents are critical to the proper operation of the system's recovery process.

It is also possible to make a backup while the server is stopped. In this case, you obviously cannot use pg_backup_start or pg_backup_stop, and you will therefore be left to your own devices to keep track of which backup is which and how far back the associated WAL files go. It is generally better to follow the continuous archiving procedure above.

Recovering Using a Continuous Archive Backup

Okay, the worst has happened and you need to recover from your backup. Here is the procedure:

  1. Stop the server, if it's running.

  2. If you have the space to do so, copy the whole cluster data directory and any tablespaces to a temporary location in case you need them later. Note that this precaution will require that you have enough free space on your system to hold two copies of your existing database. If you do not have enough space, you should at least save the contents of the cluster's pg_wal subdirectory, as it might contain WAL files which were not archived before the system went down.

  3. Remove all existing files and subdirectories under the cluster data directory and under the root directories of any tablespaces you are using.

  4. If you're restoring a full backup, you can restore the database files directly into the target directories. Be sure that they are restored with the right ownership (the database system user, not root!) and with the right permissions. If you are using tablespaces, you should verify that the symbolic links in pg_tblspc/ were correctly restored.

  5. If you're restoring an incremental backup, you'll need to restore the incremental backup and all earlier backups upon which it directly or indirectly depends to the machine where you are performing the restore. These backups will need to be placed in separate directories, not the target directories where you want the running server to end up. Once this is done, use ??? to pull data from the full backup and all of the subsequent incremental backups and write out a synthetic full backup to the target directories. As above, verify that permissions and tablespace links are correct.

  6. Remove any files present in pg_wal/; these came from the file system backup and are therefore probably obsolete rather than current. If you didn't archive pg_wal/ at all, then recreate it with proper permissions, being careful to ensure that you re-establish it as a symbolic link if you had it set up that way before.

  7. If you have unarchived WAL segment files that you saved in step 2, copy them into pg_wal/. (It is best to copy them, not move them, so you still have the unmodified files if a problem occurs and you have to start over.)

  8. Set recovery configuration settings in postgresql.conf (see ???) and create a file recovery.signal in the cluster data directory. You might also want to temporarily modify pg_hba.conf to prevent ordinary users from connecting until you are sure the recovery was successful.

  9. Start the server. The server will go into recovery mode and proceed to read through the archived WAL files it needs. Should the recovery be terminated because of an external error, the server can simply be restarted and it will continue recovery. Upon completion of the recovery process, the server will remove recovery.signal (to prevent accidentally re-entering recovery mode later) and then commence normal database operations.

  10. Inspect the contents of the database to ensure you have recovered to the desired state. If not, return to step 1. If all is well, allow your users to connect by restoring pg_hba.conf to normal.

The key part of all this is to set up a recovery configuration that describes how you want to recover and how far the recovery should run. The one thing that you absolutely must specify is the restore_command, which tells PostgreSQL how to retrieve archived WAL file segments. Like the archive_command, this is a shell command string. It can contain %f, which is replaced by the name of the desired WAL file, and %p, which is replaced by the path name to copy the WAL file to. (The path name is relative to the current working directory, i.e., the cluster's data directory.) Write %% if you need to embed an actual % character in the command. The simplest useful command is something like:

restore_command = 'cp "/mnt/server/archivedir/%f" "%p"'

which will copy previously archived WAL segments from the directory /mnt/server/archivedir. Of course, you can use something much more complicated, perhaps even a shell script that requests the operator to mount an appropriate tape.

It is important that the command return nonzero exit status on failure. The command will be called requesting files that are not present in the archive; it must return nonzero when so asked. This is not an error condition. An exception is that if the command was terminated by a signal (other than SIGTERM, which is used as part of a database server shutdown) or an error by the shell (such as command not found), then recovery will abort and the server will not start up.

Not all of the requested files will be WAL segment files; you should also expect requests for files with a suffix of .history. Also be aware that the base name of the %p path will be different from %f; do not expect them to be interchangeable.

WAL segments that cannot be found in the archive will be sought in pg_wal/; this allows use of recent un-archived segments. However, segments that are available from the archive will be used in preference to files in pg_wal/.

Normally, recovery will proceed through all available WAL segments, thereby restoring the database to the current point in time (or as close as possible given the available WAL segments). Therefore, a normal recovery will end with a “file not found” message, the exact text of the error message depending upon your choice of restore_command. You may also see an error message at the start of recovery for a file named something like 00000001.history. This is also normal and does not indicate a problem in simple recovery situations; see Timelines for discussion.

If you want to recover to some previous point in time (say, right before the junior DBA dropped your main transaction table), just specify the required stopping point. You can specify the stop point, known as the “recovery target”, either by date/time, named restore point or by completion of a specific transaction ID. As of this writing only the date/time and named restore point options are very usable, since there are no tools to help you identify with any accuracy which transaction ID to use.

[!NOTE] The stop point must be after the ending time of the base backup, i.e., the end time of pg_backup_stop. You cannot use a base backup to recover to a time when that backup was in progress. (To recover to such a time, you must go back to your previous base backup and roll forward from there.)

If recovery finds corrupted WAL data, recovery will halt at that point and the server will not start. In such a case the recovery process could be re-run from the beginning, specifying a “recovery target” before the point of corruption so that recovery can complete normally. If recovery fails for an external reason, such as a system crash or if the WAL archive has become inaccessible, then the recovery can simply be restarted and it will restart almost from where it failed. Recovery restart works much like checkpointing in normal operation: the server periodically forces all its state to disk, and then updates the pg_control file to indicate that the already-processed WAL data need not be scanned again.

Timelines

timelines

The ability to restore the database to a previous point in time creates some complexities that are akin to science-fiction stories about time travel and parallel universes. For example, in the original history of the database, suppose you dropped a critical table at 5:15PM on Tuesday evening, but didn't realize your mistake until Wednesday noon. Unfazed, you get out your backup, restore to the point-in-time 5:14PM Tuesday evening, and are up and running. In this history of the database universe, you never dropped the table. But suppose you later realize this wasn't such a great idea, and would like to return to sometime Wednesday morning in the original history. You won't be able to if, while your database was up-and-running, it overwrote some of the WAL segment files that led up to the time you now wish you could get back to. Thus, to avoid this, you need to distinguish the series of WAL records generated after you've done a point-in-time recovery from those that were generated in the original database history.

To deal with this problem, PostgreSQL has a notion of timelines. Whenever an archive recovery completes, a new timeline is created to identify the series of WAL records generated after that recovery. The timeline ID number is part of WAL segment file names so a new timeline does not overwrite the WAL data generated by previous timelines. For example, in the WAL file name 0000000100001234000055CD, the leading 00000001 is the timeline ID in hexadecimal. (Note that in other contexts, such as server log messages, timeline IDs are usually printed in decimal.)

It is in fact possible to archive many different timelines. While that might seem like a useless feature, it's often a lifesaver. Consider the situation where you aren't quite sure what point-in-time to recover to, and so have to do several point-in-time recoveries by trial and error until you find the best place to branch off from the old history. Without timelines this process would soon generate an unmanageable mess. With timelines, you can recover to any prior state, including states in timeline branches that you abandoned earlier.

Every time a new timeline is created, PostgreSQL creates a “timeline history” file that shows which timeline it branched off from and when. These history files are necessary to allow the system to pick the right WAL segment files when recovering from an archive that contains multiple timelines. Therefore, they are archived into the WAL archive area just like WAL segment files. The history files are just small text files, so it's cheap and appropriate to keep them around indefinitely (unlike the segment files which are large). You can, if you like, add comments to a history file to record your own notes about how and why this particular timeline was created. Such comments will be especially valuable when you have a thicket of different timelines as a result of experimentation.

The default behavior of recovery is to recover to the latest timeline found in the archive. If you wish to recover to the timeline that was current when the base backup was taken or into a specific child timeline (that is, you want to return to some state that was itself generated after a recovery attempt), you need to specify current or the target timeline ID in ???. You cannot recover into timelines that branched off earlier than the base backup.

Tips and Examples

Some tips for configuring continuous archiving are given here.

Standalone Hot Backups

It is possible to use PostgreSQL's backup facilities to produce standalone hot backups. These are backups that cannot be used for point-in-time recovery, yet are typically much faster to backup and restore than pg_dump dumps. (They are also much larger than pg_dump dumps, so in some cases the speed advantage might be negated.)

As with base backups, the easiest way to produce a standalone hot backup is to use the ??? tool. If you include the -X parameter when calling it, all the write-ahead log required to use the backup will be included in the backup automatically, and no special action is required to restore the backup.

Compressed Archive Logs

If archive storage size is a concern, you can use gzip to compress the archive files:

archive_command = 'gzip < "%p" > "/mnt/server/archivedir/%f.gz"'

You will then need to use gunzip during recovery:

restore_command = 'gunzip < "/mnt/server/archivedir/%f.gz" > "%p"'

archive_command Scripts

Many people choose to use scripts to define their archive_command, so that their postgresql.conf entry looks very simple:

archive_command = 'local_backup_script.sh "%p" "%f"'

Using a separate script file is advisable any time you want to use more than a single command in the archiving process. This allows all complexity to be managed within the script, which can be written in a popular scripting language such as bash or perl.

Examples of requirements that might be solved within a script include:

[!TIP] When using an archive_command script, it's desirable to enable ???. Any messages written to stderr from the script will then appear in the database server log, allowing complex configurations to be diagnosed easily if they fail.

Caveats

At this writing, there are several limitations of the continuous archiving technique. These will probably be fixed in future releases:

It should also be noted that the default WAL format is fairly bulky since it includes many disk page snapshots. These page snapshots are designed to support crash recovery, since we might need to fix partially-written disk pages. Depending on your system hardware and software, the risk of partial writes might be small enough to ignore, in which case you can significantly reduce the total volume of archived WAL files by turning off page snapshots using the ??? parameter. (Read the notes and warnings in ??? before you do so.) Turning off page snapshots does not prevent use of the WAL for PITR operations. An area for future development is to compress archived WAL data by removing unnecessary page copies even when full_page_writes is on. In the meantime, administrators might wish to reduce the number of page snapshots included in WAL by increasing the checkpoint interval parameters as much as feasible.

Client Authentication

client authentication

When a client application connects to the database server, it specifies which PostgreSQL database user name it wants to connect as, much the same way one logs into a Unix computer as a particular user. Within the SQL environment the active database user name determines access privileges to database objects — see ??? for more information. Therefore, it is essential to restrict which database users can connect.

[!NOTE] As explained in ???, PostgreSQL actually does privilege management in terms of “roles”. In this chapter, we consistently use database user to mean “role with the LOGIN privilege”.

Authentication is the process by which the database server establishes the identity of the client, and by extension determines whether the client application (or the user who runs the client application) is permitted to connect with the database user name that was requested.

PostgreSQL offers a number of different client authentication methods. The method used to authenticate a particular client connection can be selected on the basis of (client) host address, database, and user.

PostgreSQL database user names are logically separate from user names of the operating system in which the server runs. If all the users of a particular server also have accounts on the server's machine, it makes sense to assign database user names that match their operating system user names. However, a server that accepts remote connections might have many database users who have no local operating system account, and in such cases there need be no connection between database user names and OS user names.

The pg_hba.conf File

pg_hba.conf

Client authentication is controlled by a configuration file, which traditionally is named pg_hba.conf and is stored in the database cluster's data directory. (HBA stands for host-based authentication.) A default pg_hba.conf file is installed when the data directory is initialized by ???. It is possible to place the authentication configuration file elsewhere, however; see the ??? configuration parameter.

The pg_hba.conf file is read on start-up and when the main server process receives a SIGHUP signal. If you edit the file on an active system, you will need to signal the postmaster (using pg_ctl reload, calling the SQL function pg_reload_conf(), or using kill -HUP) to make it re-read the file.

[!NOTE] The preceding statement is not true on Microsoft Windows: there, any changes in the pg_hba.conf file are immediately applied by subsequent new connections.

The system view pg_hba_file_rules can be helpful for pre-testing changes to the pg_hba.conf file, or for diagnosing problems if loading of the file did not have the desired effects. Rows in the view with non-null error fields indicate problems in the corresponding lines of the file.

The general format of the pg_hba.conf file is a set of records, one per line. Blank lines are ignored, as is any text after the # comment character. A record can be continued onto the next line by ending the line with a backslash. (Backslashes are not special except at the end of a line.) A record is made up of a number of fields which are separated by spaces and/or tabs. Fields can contain white space if the field value is double-quoted. Quoting one of the keywords in a database, user, or address field (e.g., all or replication) makes the word lose its special meaning, and just match a database, user, or host with that name. Backslash line continuation applies even within quoted text or comments.

Each authentication record specifies a connection type, a client IP address range (if relevant for the connection type), a database name, a user name, and the authentication method to be used for connections matching these parameters. The first record with a matching connection type, client address, requested database, and user name is used to perform authentication. There is no “fall-through” or “backup”: if one record is chosen and the authentication fails, subsequent records are not considered. If no record matches, access is denied.

Each record can be an include directive or an authentication record. Include directives specify files that can be included, that contain additional records. The records will be inserted in place of the include directives. Include directives only contain two fields: include, include_if_exists or include_dir directive and the file or directory to be included. The file or directory can be a relative or absolute path, and can be double-quoted. For the include_dir form, all files not starting with a . and ending with .conf will be included. Multiple files within an include directory are processed in file name order (according to C locale rules, i.e., numbers before letters, and uppercase letters before lowercase ones).

A record can have several formats: local <database> <user> <auth-method> [<auth-options>] host <database> <user> <address> <auth-method> [<auth-options>] hostssl <database> <user> <address> <auth-method> [<auth-options>] hostnossl <database> <user> <address> <auth-method> [<auth-options>] hostgssenc <database> <user> <address> <auth-method> [<auth-options>] hostnogssenc <database> <user> <address> <auth-method> [<auth-options>] host <database> <user> <IP-address> <IP-mask> <auth-method> [<auth-options>] hostssl <database> <user> <IP-address> <IP-mask> <auth-method> [<auth-options>] hostnossl <database> <user> <IP-address> <IP-mask> <auth-method> [<auth-options>] hostgssenc <database> <user> <IP-address> <IP-mask> <auth-method> [<auth-options>] hostnogssenc <database> <user> <IP-address> <IP-mask> <auth-method> [<auth-options>] include <file> include_if_exists <file> include_dir <directory> The meaning of the fields is as follows:

local
This record matches connection attempts using Unix-domain sockets. Without a record of this type, Unix-domain socket connections are disallowed.

host
This record matches connection attempts made using TCP/IP. host records match SSL or non-SSL connection attempts as well as GSSAPI encrypted or non-GSSAPI encrypted connection attempts.

[!NOTE] Remote TCP/IP connections will not be possible unless the server is started with an appropriate value for the ??? configuration parameter, since the default behavior is to listen for TCP/IP connections only on the local loopback address localhost.

hostssl
This record matches connection attempts made using TCP/IP, but only when the connection is made with SSL encryption.

To make use of this option the server must be built with SSL support. Furthermore, SSL must be enabled by setting the ??? configuration parameter (see ??? for more information). Otherwise, the hostssl record is ignored except for logging a warning that it cannot match any connections.

hostnossl
This record type has the opposite behavior of hostssl; it only matches connection attempts made over TCP/IP that do not use SSL.

hostgssenc
This record matches connection attempts made using TCP/IP, but only when the connection is made with GSSAPI encryption.

To make use of this option the server must be built with GSSAPI support. Otherwise, the hostgssenc record is ignored except for logging a warning that it cannot match any connections.

hostnogssenc
This record type has the opposite behavior of hostgssenc; it only matches connection attempts made over TCP/IP that do not use GSSAPI encryption.

<database>
Specifies which database name(s) this record matches. The value all specifies that it matches all databases. The value sameuser specifies that the record matches if the requested database has the same name as the requested user. The value samerole specifies that the requested user must be a member of the role with the same name as the requested database. (samegroup is an obsolete but still accepted spelling of samerole.) Superusers are not considered to be members of a role for the purposes of samerole unless they are explicitly members of the role, directly or indirectly, and not just by virtue of being a superuser. The value replication specifies that the record matches if a physical replication connection is requested, however, it doesn't match with logical replication connections. Note that physical replication connections do not specify any particular database whereas logical replication connections do specify it. Otherwise, this is the name of a specific PostgreSQL database or a regular expression. Multiple database names and/or regular expressions can be supplied by separating them with commas.

If the database name starts with a slash (/), the remainder of the name is treated as a regular expression. (See ??? for details of PostgreSQL's regular expression syntax.)

A separate file containing database names and/or regular expressions can be specified by preceding the file name with @.

<user>
Specifies which database user name(s) this record matches. The value all specifies that it matches all users. Otherwise, this is either the name of a specific database user, a regular expression when starting with a slash (/), or a group name preceded by +. (Recall that there is no real distinction between users and groups in PostgreSQL; a + mark really means “match any of the roles that are directly or indirectly members of this role”, while a name without a + mark matches only that specific role.) For this purpose, a superuser is only considered to be a member of a role if they are explicitly a member of the role, directly or indirectly, and not just by virtue of being a superuser. Multiple user names and/or regular expressions can be supplied by separating them with commas.

If the user name starts with a slash (/), the remainder of the name is treated as a regular expression. (See ??? for details of PostgreSQL's regular expression syntax.)

A separate file containing user names and/or regular expressions can be specified by preceding the file name with @.

<address>
Specifies the client machine address(es) that this record matches. This field can contain either a host name, an IP address range, or one of the special key words mentioned below.

An IP address range is specified using standard numeric notation for the range's starting address, then a slash (/) and a CIDR mask length. The mask length indicates the number of high-order bits of the client IP address that must match. Bits to the right of this should be zero in the given IP address. There must not be any white space between the IP address, the /, and the CIDR mask length.

Typical examples of an IPv4 address range specified this way are 172.20.143.89/32 for a single host, or 172.20.143.0/24 for a small network, or 10.6.0.0/16 for a larger one. An IPv6 address range might look like ::1/128 for a single host (in this case the IPv6 loopback address) or fe80::7a31:c1ff:0000:0000/96 for a small network. 0.0.0.0/0 represents all IPv4 addresses, and ::0/0 represents all IPv6 addresses. To specify a single host, use a mask length of 32 for IPv4 or 128 for IPv6. In a network address, do not omit trailing zeroes.

An entry given in IPv4 format will match only IPv4 connections, and an entry given in IPv6 format will match only IPv6 connections, even if the represented address is in the IPv4-in-IPv6 range.

You can also write all to match any IP address, samehost to match any of the server's own IP addresses, or samenet to match any address in any subnet that the server is directly connected to.

If a host name is specified (anything that is not an IP address range or a special key word is treated as a host name), that name is compared with the result of a reverse name resolution of the client's IP address (e.g., reverse DNS lookup, if DNS is used). Host name comparisons are case insensitive. If there is a match, then a forward name resolution (e.g., forward DNS lookup) is performed on the host name to check whether any of the addresses it resolves to are equal to the client's IP address. If both directions match, then the entry is considered to match. (The host name that is used in pg_hba.conf should be the one that address-to-name resolution of the client's IP address returns, otherwise the line won't be matched. Some host name databases allow associating an IP address with multiple host names, but the operating system will only return one host name when asked to resolve an IP address.)

A host name specification that starts with a dot (.) matches a suffix of the actual host name. So .example.com would match foo.example.com (but not just example.com).

When host names are specified in pg_hba.conf, you should make sure that name resolution is reasonably fast. It can be of advantage to set up a local name resolution cache such as nscd. Also, you may wish to enable the configuration parameter log_hostname to see the client's host name instead of the IP address in the log.

These fields do not apply to local records.

[!NOTE] Users sometimes wonder why host names are handled in this seemingly complicated way, with two name resolutions including a reverse lookup of the client's IP address. This complicates use of the feature in case the client's reverse DNS entry is not set up or yields some undesirable host name. It is done primarily for efficiency: this way, a connection attempt requires at most two resolver lookups, one reverse and one forward. If there is a resolver problem with some address, it becomes only that client's problem. A hypothetical alternative implementation that only did forward lookups would have to resolve every host name mentioned in pg_hba.conf during every connection attempt. That could be quite slow if many names are listed. And if there is a resolver problem with one of the host names, it becomes everyone's problem.

Also, a reverse lookup is necessary to implement the suffix matching feature, because the actual client host name needs to be known in order to match it against the pattern.

Note that this behavior is consistent with other popular implementations of host name-based access control, such as the Apache HTTP Server and TCP Wrappers.

<IP-address>; <IP-mask>
These two fields can be used as an alternative to the <IP-address>/<mask-length> notation. Instead of specifying the mask length, the actual mask is specified in a separate column. For example, 255.0.0.0 represents an IPv4 CIDR mask length of 8, and 255.255.255.255 represents a CIDR mask length of 32.

These fields do not apply to local records.

<auth-method>
Specifies the authentication method to use when a connection matches this record. The possible choices are summarized here; details are in Authentication Methods. All the options are lower case and treated case sensitively, so even acronyms like ldap must be specified as lower case.

trust
Allow the connection unconditionally. This method allows anyone that can connect to the PostgreSQL database server to login as any PostgreSQL user they wish, without the need for a password or any other authentication. See Trust Authentication for details.

reject
Reject the connection unconditionally. This is useful for “filtering out” certain hosts from a group, for example a reject line could block a specific host from connecting, while a later line allows the remaining hosts in a specific network to connect.

scram-sha-256
Perform SCRAM-SHA-256 authentication to verify the user's password. See Password Authentication for details.

md5
Perform SCRAM-SHA-256 or MD5 authentication to verify the user's password. See Password Authentication for details.

[!WARNING] Support for MD5-encrypted passwords is deprecated and will be removed in a future release of PostgreSQL. Refer to Password Authentication for details about migrating to another password type.

password
Require the client to supply an unencrypted password for authentication. Since the password is sent in clear text over the network, this should not be used on untrusted networks. See Password Authentication for details.

gss
Use GSSAPI to authenticate the user. This is only available for TCP/IP connections. See GSSAPI Authentication for details. It can be used in conjunction with GSSAPI encryption.

sspi
Use SSPI to authenticate the user. This is only available on Windows. See SSPI Authentication for details.

ident
Obtain the operating system user name of the client by contacting the ident server on the client and check if it matches the requested database user name. Ident authentication can only be used on TCP/IP connections. When specified for local connections, peer authentication will be used instead. See Ident Authentication for details.

peer
Obtain the client's operating system user name from the operating system and check if it matches the requested database user name. This is only available for local connections. See Peer Authentication for details.

ldap
Authenticate using an LDAP server. See LDAP Authentication for details.

cert
Authenticate using SSL client certificates. See Certificate Authentication for details.

pam
Authenticate using the Pluggable Authentication Modules (PAM) service provided by the operating system. See PAM Authentication for details.

bsd
Authenticate using the BSD Authentication service provided by the operating system. See BSD Authentication for details.

oauth
Authorize and optionally authenticate using a third-party OAuth 2.0 identity provider. See OAuth Authorization/Authentication for details.

<auth-options>
After the <auth-method> field, there can be field(s) of the form <name>=<value> that specify options for the authentication method. Details about which options are available for which authentication methods appear below.

In addition to the method-specific options listed below, there is a method-independent authentication option clientcert, which can be specified in any hostssl record. This option can be set to verify-ca or verify-full. Both options require the client to present a valid (trusted) SSL certificate, while verify-full additionally enforces that the cn (Common Name) in the certificate matches the username or an applicable mapping. This behavior is similar to the cert authentication method (see Certificate Authentication) but enables pairing the verification of client certificates with any authentication method that supports hostssl entries.

On any record using client certificate authentication (i.e. one using the cert authentication method or one using the clientcert option), you can specify which part of the client certificate credentials to match using the clientname option. This option can have one of two values. If you specify clientname=CN, which is the default, the username is matched against the certificate's Common Name (CN). If instead you specify clientname=DN the username is matched against the entire Distinguished Name (DN) of the certificate. This option is probably best used in conjunction with a username map. The comparison is done with the DN in RFC 2253 format. To see the DN of a client certificate in this format, do

openssl x509 -in myclient.crt -noout -subject -nameopt RFC2253 | sed "s/^subject=//"

Care needs to be taken when using this option, especially when using regular expression matching against the DN.

include
This line will be replaced by the contents of the given file.

include_if_exists
This line will be replaced by the content of the given file if the file exists. Otherwise, a message is logged to indicate that the file has been skipped.

include_dir
This line will be replaced by the contents of all the files found in the directory, if they don't start with a . and end with .conf, processed in file name order (according to C locale rules, i.e., numbers before letters, and uppercase letters before lowercase ones).

Files included by @ constructs are read as lists of names, which can be separated by either whitespace or commas. Comments are introduced by #, just as in pg_hba.conf, and nested @ constructs are allowed. Unless the file name following @ is an absolute path, it is taken to be relative to the directory containing the referencing file.

Since the pg_hba.conf records are examined sequentially for each connection attempt, the order of the records is significant. Typically, earlier records will have tight connection match parameters and weaker authentication methods, while later records will have looser match parameters and stronger authentication methods. For example, one might wish to use trust authentication for local TCP/IP connections but require a password for remote TCP/IP connections. In this case a record specifying trust authentication for connections from 127.0.0.1 would appear before a record specifying password authentication for a wider range of allowed client IP addresses.

[!TIP] To connect to a particular database, a user must not only pass the pg_hba.conf checks, but must have the CONNECT privilege for the database. If you wish to restrict which users can connect to which databases, it's usually easier to control this by granting/revoking CONNECT privilege than to put the rules in pg_hba.conf entries.

Some examples of pg_hba.conf entries are shown in example_title. See the next section for details on the different authentication methods.

Example pg_hba.conf Entries

# Allow any user on the local system to connect to any database with
# any database user name using Unix-domain sockets (the default for local
# connections).
#
# TYPE  DATABASE        USER            ADDRESS                 METHOD
local   all             all                                     trust

# The same using local loopback TCP/IP connections.
#
# TYPE  DATABASE        USER            ADDRESS                 METHOD
host    all             all             127.0.0.1/32            trust

# The same as the previous line, but using a separate netmask column
#
# TYPE  DATABASE        USER            IP-ADDRESS      IP-MASK             METHOD
host    all             all             127.0.0.1       255.255.255.255     trust

# The same over IPv6.
#
# TYPE  DATABASE        USER            ADDRESS                 METHOD
host    all             all             ::1/128                 trust

# The same using a host name (would typically cover both IPv4 and IPv6).
#
# TYPE  DATABASE        USER            ADDRESS                 METHOD
host    all             all             localhost               trust

# The same using a regular expression for DATABASE, that allows connection
# to any databases with a name beginning with "db" and finishing with a
# number using two to four digits (like "db1234" or "db12").
#
# TYPE  DATABASE        USER            ADDRESS                 METHOD
host    "/^db\d{2,4}$"  all             localhost               trust

# Allow any user from any host with IP address 192.168.93.x to connect
# to database "postgres" as the same user name that ident reports for
# the connection (typically the operating system user name).
#
# TYPE  DATABASE        USER            ADDRESS                 METHOD
host    postgres        all             192.168.93.0/24         ident

# Allow any user from host 192.168.12.10 to connect to database
# "postgres" if the user's password is correctly supplied.
#
# TYPE  DATABASE        USER            ADDRESS                 METHOD
host    postgres        all             192.168.12.10/32        scram-sha-256

# Allow any user from hosts in the example.com domain to connect to
# any database if the user's password is correctly supplied.
#
# Require SCRAM authentication for most users, but make an exception
# for user 'mike', who uses an older client that doesn't support SCRAM
# authentication.
#
# TYPE  DATABASE        USER            ADDRESS                 METHOD
host    all             mike            .example.com            md5
host    all             all             .example.com            scram-sha-256

# In the absence of preceding "host" lines, these three lines will
# reject all connections from 192.168.54.1 (since that entry will be
# matched first), but allow GSSAPI-encrypted connections from anywhere else
# on the Internet.  The zero mask causes no bits of the host IP address to
# be considered, so it matches any host.  Unencrypted GSSAPI connections
# (which "fall through" to the third line since "hostgssenc" only matches
# encrypted GSSAPI connections) are allowed, but only from 192.168.12.10.
#
# TYPE  DATABASE        USER            ADDRESS                 METHOD
host    all             all             192.168.54.1/32         reject
hostgssenc all          all             0.0.0.0/0               gss
host    all             all             192.168.12.10/32        gss

# Allow users from 192.168.x.x hosts to connect to any database, if
# they pass the ident check.  If, for example, ident says the user is
# "bryanh" and he requests to connect as PostgreSQL user "guest1", the
# connection is allowed if there is an entry in pg_ident.conf for map
# "omicron" that says "bryanh" is allowed to connect as "guest1".
#
# TYPE  DATABASE        USER            ADDRESS                 METHOD
host    all             all             192.168.0.0/16          ident map=omicron

# If these are the only four lines for local connections, they will
# allow local users to connect only to their own databases (databases
# with the same name as their database user name) except for users whose
# name end with "helpdesk", administrators and members of role "support",
# who can connect to all databases.  The file $PGDATA/admins contains a
# list of names of administrators.  Passwords are required in all cases.
#
# TYPE  DATABASE        USER            ADDRESS                 METHOD
local   sameuser        all                                     scram-sha-256
local   all             /^.*helpdesk$                           scram-sha-256
local   all             @admins                                 scram-sha-256
local   all             +support                                scram-sha-256

# The last two lines above can be combined into a single line:
local   all             @admins,+support                        scram-sha-256

# The database column can also use lists and file names:
local   db1,db2,@demodbs  all                                   scram-sha-256

User Name Maps

User name maps

When using an external authentication system such as Ident or GSSAPI, the name of the operating system user that initiated the connection might not be the same as the database user (role) that is to be used. In this case, a user name map can be applied to map the operating system user name to a database user. To use user name mapping, specify map=<map-name> in the options field in pg_hba.conf. This option is supported for all authentication methods that receive external user names. Since different mappings might be needed for different connections, the name of the map to be used is specified in the <map-name> parameter in pg_hba.conf to indicate which map to use for each individual connection.

User name maps are defined in the ident map file, which by default is named pg_ident.conf and is stored in the cluster's data directory. (It is possible to place the map file elsewhere, however; see the ??? configuration parameter.) The ident map file contains lines of the general forms: <map-name> <system-username> <database-username> <include> <file> <include_if_exists> <file> <include_dir> <directory> Comments, whitespace and line continuations are handled in the same way as in pg_hba.conf. The <map-name> is an arbitrary name that will be used to refer to this mapping in pg_hba.conf. The other two fields specify an operating system user name and a matching database user name. The same <map-name> can be used repeatedly to specify multiple user-mappings within a single map.

As for pg_hba.conf, the lines in this file can be include directives, following the same rules.

The pg_ident.conf file is read on start-up and when the main server process receives a SIGHUP signal. If you edit the file on an active system, you will need to signal the postmaster (using pg_ctl reload, calling the SQL function pg_reload_conf(), or using kill -HUP) to make it re-read the file.

The system view pg_ident_file_mappings can be helpful for pre-testing changes to the pg_ident.conf file, or for diagnosing problems if loading of the file did not have the desired effects. Rows in the view with non-null error fields indicate problems in the corresponding lines of the file.

There is no restriction regarding how many database users a given operating system user can correspond to, nor vice versa. Thus, entries in a map should be thought of as meaning “this operating system user is allowed to connect as this database user”, rather than implying that they are equivalent. The connection will be allowed if there is any map entry that pairs the user name obtained from the external authentication system with the database user name that the user has requested to connect as. The value all can be used as the <database-username> to specify that if the <system-username> matches, then this user is allowed to log in as any of the existing database users. Quoting all makes the keyword lose its special meaning.

If the <database-username> begins with a + character, then the operating system user can login as any user belonging to that role, similarly to how user names beginning with + are treated in pg_hba.conf. Thus, a + mark means “match any of the roles that are directly or indirectly members of this role”, while a name without a + mark matches only that specific role. Quoting a username starting with a + makes the + lose its special meaning.

If the <system-username> field starts with a slash (/), the remainder of the field is treated as a regular expression. (See ??? for details of PostgreSQL's regular expression syntax.) The regular expression can include a single capture, or parenthesized subexpression. The portion of the system user name that matched the capture can then be referenced in the <database-username> field as \1 (backslash-one). This allows the mapping of multiple user names in a single line, which is particularly useful for simple syntax substitutions. For example, these entries

mymap   /^(.*)@mydomain\.com$      \1
mymap   /^(.*)@otherdomain\.com$   guest

will remove the domain part for users with system user names that end with @mydomain.com, and allow any user whose system name ends with @otherdomain.com to log in as guest. Quoting a <database-username> containing \1 does not make \1 lose its special meaning.

If the <database-username> field starts with a slash (/), the remainder of the field is treated as a regular expression. When the <database-username> field is a regular expression, it is not possible to use \1 within it to refer to a capture from the <system-username> field.

[!TIP] Keep in mind that by default, a regular expression can match just part of a string. It's usually wise to use ^ and $, as shown in the above example, to force the match to be to the entire system user name.

A pg_ident.conf file that could be used in conjunction with the pg_hba.conf file in example_title is shown in example_title. In this example, anyone logged in to a machine on the 192.168 network that does not have the operating system user name bryanh, ann, or robert would not be granted access. Unix user robert would only be allowed access when he tries to connect as PostgreSQL user bob, not as robert or anyone else. ann would only be allowed to connect as ann. User bryanh would be allowed to connect as either bryanh or as guest1.

An Example pg_ident.conf File

# MAPNAME       SYSTEM-USERNAME         PG-USERNAME

omicron         bryanh                  bryanh
omicron         ann                     ann
# bob has user name robert on these machines
omicron         robert                  bob
# bryanh can also connect as guest1
omicron         bryanh                  guest1

Authentication Methods

PostgreSQL provides various methods for authenticating users:

Peer authentication is usually recommendable for local connections, though trust authentication might be sufficient in some circumstances. Password authentication is the easiest choice for remote connections. All the other options require some kind of external security infrastructure (usually an authentication server or a certificate authority for issuing SSL certificates), or are platform-specific.

The following sections describe each of these authentication methods in more detail.

Trust Authentication

When trust authentication is specified, PostgreSQL assumes that anyone who can connect to the server is authorized to access the database with whatever database user name they specify (even superuser names). Of course, restrictions made in the database and user columns still apply. This method should only be used when there is adequate operating-system-level protection on connections to the server.

trust authentication is appropriate and very convenient for local connections on a single-user workstation. It is usually not appropriate by itself on a multiuser machine. However, you might be able to use trust even on a multiuser machine, if you restrict access to the server's Unix-domain socket file using file-system permissions. To do this, set the unix_socket_permissions (and possibly unix_socket_group) configuration parameters as described in ???. Or you could set the unix_socket_directories configuration parameter to place the socket file in a suitably restricted directory.

Setting file-system permissions only helps for Unix-socket connections. Local TCP/IP connections are not restricted by file-system permissions. Therefore, if you want to use file-system permissions for local security, remove the host ... 127.0.0.1 ... line from pg_hba.conf, or change it to a non-trust authentication method.

trust authentication is only suitable for TCP/IP connections if you trust every user on every machine that is allowed to connect to the server by the pg_hba.conf lines that specify trust. It is seldom reasonable to use trust for any TCP/IP connections other than those from localhost (127.0.0.1).

Password Authentication

MD5

SCRAM

password

authentication

There are several password-based authentication methods. These methods operate similarly but differ in how the users' passwords are stored on the server and how the password provided by a client is sent across the connection.

scram-sha-256
The method scram-sha-256 performs SCRAM-SHA-256 authentication, as described in RFC 7677. It is a challenge-response scheme that prevents password sniffing on untrusted connections and supports storing passwords on the server in a cryptographically hashed form that is thought to be secure.

This is the most secure of the currently provided methods, but it is not supported by older client libraries.

md5
The method md5 uses a custom less secure challenge-response mechanism. It prevents password sniffing and avoids storing passwords on the server in plain text but provides no protection if an attacker manages to steal the password hash from the server. Also, the MD5 hash algorithm is nowadays no longer considered secure against determined attacks.

To ease transition from the md5 method to the newer SCRAM method, if md5 is specified as a method in pg_hba.conf but the user's password on the server is encrypted for SCRAM (see below), then SCRAM-based authentication will automatically be chosen instead.

[!WARNING] Support for MD5-encrypted passwords is deprecated and will be removed in a future release of PostgreSQL. Refer to the text below for details about migrating to another password type.

password
The method password sends the password in clear-text and is therefore vulnerable to password “sniffing” attacks. It should always be avoided if possible. If the connection is protected by SSL encryption then password can be used safely, though. (Though SSL certificate authentication might be a better choice if one is depending on using SSL).

PostgreSQL database passwords are separate from operating system user passwords. The password for each database user is stored in the pg_authid system catalog. Passwords can be managed with the SQL commands ??? and ???, e.g., CREATE ROLE foo WITH LOGIN PASSWORD 'secret', or the psql command \password. If no password has been set up for a user, the stored password is null and password authentication will always fail for that user.

The availability of the different password-based authentication methods depends on how a user's password on the server is encrypted (or hashed, more accurately). This is controlled by the configuration parameter ??? at the time the password is set. If a password was encrypted using the scram-sha-256 setting, then it can be used for the authentication methods scram-sha-256 and password (but password transmission will be in plain text in the latter case). The authentication method specification md5 will automatically switch to using the scram-sha-256 method in this case, as explained above, so it will also work. If a password was encrypted using the md5 setting, then it can be used only for the md5 and password authentication method specifications (again, with the password transmitted in plain text in the latter case). (Previous PostgreSQL releases supported storing the password on the server in plain text. This is no longer possible.) To check the currently stored password hashes, see the system catalog pg_authid.

To upgrade an existing installation from md5 to scram-sha-256, after having ensured that all client libraries in use are new enough to support SCRAM, set password_encryption = 'scram-sha-256' in postgresql.conf, make all users set new passwords, and change the authentication method specifications in pg_hba.conf to scram-sha-256.

GSSAPI Authentication

GSSAPI

GSSAPI is an industry-standard protocol for secure authentication defined in RFC 2743. PostgreSQL supports GSSAPI for authentication, communications encryption, or both. GSSAPI provides automatic authentication (single sign-on) for systems that support it. The authentication itself is secure. If GSSAPI encryption or SSL encryption is used, the data sent along the database connection will be encrypted; otherwise, it will not.

GSSAPI support has to be enabled when PostgreSQL is built; see ??? for more information.

When GSSAPI uses Kerberos, it uses a standard service principal (authentication identity) name in the format servicename/hostname@realm. The principal name used by a particular installation is not encoded in the PostgreSQL server in any way; rather it is specified in the keytab file that the server reads to determine its identity. If multiple principals are listed in the keytab file, the server will accept any one of them. The server's realm name is the preferred realm specified in the Kerberos configuration file(s) accessible to the server.

When connecting, the client must know the principal name of the server it intends to connect to. The <servicename> part of the principal is ordinarily postgres, but another value can be selected via libpq's ??? connection parameter. The <hostname> part is the fully qualified host name that libpq is told to connect to. The realm name is the preferred realm specified in the Kerberos configuration file(s) accessible to the client.

The client will also have a principal name for its own identity (and it must have a valid ticket for this principal). To use GSSAPI for authentication, the client principal must be associated with a PostgreSQL database user name. The pg_ident.conf configuration file can be used to map principals to user names; for example, pgusername@realm could be mapped to just pgusername. Alternatively, you can use the full username@realm principal as the role name in PostgreSQL without any mapping.

PostgreSQL also supports mapping client principals to user names by just stripping the realm from the principal. This method is supported for backwards compatibility and is strongly discouraged as it is then impossible to distinguish different users with the same user name but coming from different realms. To enable this, set include_realm to 0. For simple single-realm installations, doing that combined with setting the krb_realm parameter (which checks that the principal's realm matches exactly what is in the krb_realm parameter) is still secure; but this is a less capable approach compared to specifying an explicit mapping in pg_ident.conf.

The location of the server's keytab file is specified by the ??? configuration parameter. For security reasons, it is recommended to use a separate keytab just for the PostgreSQL server rather than allowing the server to read the system keytab file. Make sure that your server keytab file is readable (and preferably only readable, not writable) by the PostgreSQL server account. (See also ???.)

The keytab file is generated using the Kerberos software; see the Kerberos documentation for details. The following example shows doing this using the kadmin tool of MIT Kerberos:

kadmin% addprinc -randkey postgres/server.my.domain.org
kadmin% ktadd -k krb5.keytab postgres/server.my.domain.org

The following authentication options are supported for the GSSAPI authentication method:

include_realm
If set to 0, the realm name from the authenticated user principal is stripped off before being passed through the user name mapping (User Name Maps). This is discouraged and is primarily available for backwards compatibility, as it is not secure in multi-realm environments unless krb_realm is also used. It is recommended to leave include_realm set to the default (1) and to provide an explicit mapping in pg_ident.conf to convert principal names to PostgreSQL user names.

map
Allows mapping from client principals to database user names. See User Name Maps for details. For a GSSAPI/Kerberos principal, such as username@EXAMPLE.COM (or, less commonly, username/hostbased@EXAMPLE.COM), the user name used for mapping is username@EXAMPLE.COM (or username/hostbased@EXAMPLE.COM, respectively), unless include_realm has been set to 0, in which case username (or username/hostbased) is what is seen as the system user name when mapping.

krb_realm
Sets the realm to match user principal names against. If this parameter is set, only users of that realm will be accepted. If it is not set, users of any realm can connect, subject to whatever user name mapping is done.

In addition to these settings, which can be different for different pg_hba.conf entries, there is the server-wide ??? configuration parameter. If that is set to true, client principals are matched to user map entries case-insensitively. krb_realm, if set, is also matched case-insensitively.

SSPI Authentication

SSPI

SSPI is a Windows technology for secure authentication with single sign-on. PostgreSQL will use SSPI in negotiate mode, which will use Kerberos when possible and automatically fall back to NTLM in other cases. SSPI and GSSAPI interoperate as clients and servers, e.g., an SSPI client can authenticate to an GSSAPI server. It is recommended to use SSPI on Windows clients and servers and GSSAPI on non-Windows platforms.

When using Kerberos authentication, SSPI works the same way GSSAPI does; see GSSAPI Authentication for details.

The following configuration options are supported for SSPI:

include_realm
If set to 0, the realm name from the authenticated user principal is stripped off before being passed through the user name mapping (User Name Maps). This is discouraged and is primarily available for backwards compatibility, as it is not secure in multi-realm environments unless krb_realm is also used. It is recommended to leave include_realm set to the default (1) and to provide an explicit mapping in pg_ident.conf to convert principal names to PostgreSQL user names.

compat_realm
If set to 1, the domain's SAM-compatible name (also known as the NetBIOS name) is used for the include_realm option. This is the default. If set to 0, the true realm name from the Kerberos user principal name is used.

Do not disable this option unless your server runs under a domain account (this includes virtual service accounts on a domain member system) and all clients authenticating through SSPI are also using domain accounts, or authentication will fail.

upn_username
If this option is enabled along with compat_realm, the user name from the Kerberos UPN is used for authentication. If it is disabled (the default), the SAM-compatible user name is used. By default, these two names are identical for new user accounts.

Note that libpq uses the SAM-compatible name if no explicit user name is specified. If you use libpq or a driver based on it, you should leave this option disabled or explicitly specify user name in the connection string.

map
Allows for mapping between system and database user names. See User Name Maps for details. For an SSPI/Kerberos principal, such as username@EXAMPLE.COM (or, less commonly, username/hostbased@EXAMPLE.COM), the user name used for mapping is username@EXAMPLE.COM (or username/hostbased@EXAMPLE.COM, respectively), unless include_realm has been set to 0, in which case username (or username/hostbased) is what is seen as the system user name when mapping.

krb_realm
Sets the realm to match user principal names against. If this parameter is set, only users of that realm will be accepted. If it is not set, users of any realm can connect, subject to whatever user name mapping is done.

Ident Authentication

ident

The ident authentication method works by obtaining the client's operating system user name from an ident server and using it as the allowed database user name (with an optional user name mapping). This is only supported on TCP/IP connections.

[!NOTE] When ident is specified for a local (non-TCP/IP) connection, peer authentication (see Peer Authentication) will be used instead.

The following configuration options are supported for ident:

map
Allows for mapping between system and database user names. See User Name Maps for details.

The “Identification Protocol” is described in RFC 1413. Virtually every Unix-like operating system ships with an ident server that listens on TCP port 113 by default. The basic functionality of an ident server is to answer questions like “What user initiated the connection that goes out of your port <X> and connects to my port <Y>?”. Since PostgreSQL knows both <X> and <Y> when a physical connection is established, it can interrogate the ident server on the host of the connecting client and can theoretically determine the operating system user for any given connection.

The drawback of this procedure is that it depends on the integrity of the client: if the client machine is untrusted or compromised, an attacker could run just about any program on port 113 and return any user name they choose. This authentication method is therefore only appropriate for closed networks where each client machine is under tight control and where the database and system administrators operate in close contact. In other words, you must trust the machine running the ident server. Heed the warning:

The Identification Protocol is not intended as an authorization or access control protocol.

— RFC 1413

Some ident servers have a nonstandard option that causes the returned user name to be encrypted, using a key that only the originating machine's administrator knows. This option must not be used when using the ident server with PostgreSQL, since PostgreSQL does not have any way to decrypt the returned string to determine the actual user name.

Peer Authentication

peer

The peer authentication method works by obtaining the client's operating system user name from the kernel and using it as the allowed database user name (with optional user name mapping). This method is only supported on local connections.

The following configuration options are supported for peer:

map
Allows for mapping between system and database user names. See User Name Maps for details.

Peer authentication is only available on operating systems providing the getpeereid() function, the SO_PEERCRED socket parameter, or similar mechanisms. Currently that includes Linux, most flavors of BSD including macOS, and Solaris.

LDAP Authentication

LDAP

This authentication method operates similarly to password except that it uses LDAP as the password verification method. LDAP is used only to validate the user name/password pairs. Therefore the user must already exist in the database before LDAP can be used for authentication.

LDAP authentication can operate in two modes. In the first mode, which we will call the simple bind mode, the server will bind to the distinguished name constructed as <prefix> <username> <suffix>. Typically, the <prefix> parameter is used to specify cn=, or <DOMAIN>\ in an Active Directory environment. <suffix> is used to specify the remaining part of the DN in a non-Active Directory environment.

In the second mode, which we will call the search+bind mode, the server first binds to the LDAP directory with a fixed user name and password, specified with <ldapbinddn> and <ldapbindpasswd>, and performs a search for the user trying to log in to the database. If no user and password is configured, an anonymous bind will be attempted to the directory. The search will be performed over the subtree at <ldapbasedn>, and will try to do an exact match of the attribute specified in <ldapsearchattribute>. Once the user has been found in this search, the server re-binds to the directory as this user, using the password specified by the client, to verify that the login is correct. This mode is the same as that used by LDAP authentication schemes in other software, such as Apache mod_authnz_ldap and pam_ldap. This method allows for significantly more flexibility in where the user objects are located in the directory, but will cause two additional requests to the LDAP server to be made.

The following configuration options are used in both modes:

ldapserver
Names or IP addresses of LDAP servers to connect to. Multiple servers may be specified, separated by spaces.

ldapport
Port number on LDAP server to connect to. If no port is specified, the LDAP library's default port setting will be used.

ldapscheme
Set to ldaps to use LDAPS. This is a non-standard way of using LDAP over SSL, supported by some LDAP server implementations. See also the ldaptls option for an alternative.

ldaptls
Set to 1 to make the connection between PostgreSQL and the LDAP server use TLS encryption. This uses the StartTLS operation per RFC 4513. See also the ldapscheme option for an alternative.

Note that using ldapscheme or ldaptls only encrypts the traffic between the PostgreSQL server and the LDAP server. The connection between the PostgreSQL server and the PostgreSQL client will still be unencrypted unless SSL is used there as well.

The following options are used in simple bind mode only:

ldapprefix
String to prepend to the user name when forming the DN to bind as, when doing simple bind authentication.

ldapsuffix
String to append to the user name when forming the DN to bind as, when doing simple bind authentication.

The following options are used in search+bind mode only:

ldapbasedn
Root DN to begin the search for the user in, when doing search+bind authentication.

ldapbinddn
DN of user to bind to the directory with to perform the search when doing search+bind authentication.

ldapbindpasswd
Password for user to bind to the directory with to perform the search when doing search+bind authentication.

ldapsearchattribute
Attribute to match against the user name in the search when doing search+bind authentication. If no attribute is specified, the uid attribute will be used.

ldapsearchfilter
The search filter to use when doing search+bind authentication. Occurrences of $username will be replaced with the user name. This allows for more flexible search filters than ldapsearchattribute.

The following option may be used as an alternative way to write some of the above LDAP options in a more compact and standard form:

ldapurl
An RFC 4516 LDAP URL. The format is ldap[s]://<host>[:<port>]/<basedn>[?[<attribute>][?[<scope>][?[<filter>]]]] <scope> must be one of base, one, sub, typically the last. (The default is base, which is normally not useful in this application.) <attribute> can nominate a single attribute, in which case it is used as a value for ldapsearchattribute. If <attribute> is empty then <filter> can be used as a value for ldapsearchfilter.

The URL scheme ldaps chooses the LDAPS method for making LDAP connections over SSL, equivalent to using ldapscheme=ldaps. To use encrypted LDAP connections using the StartTLS operation, use the normal URL scheme ldap and specify the ldaptls option in addition to ldapurl.

For non-anonymous binds, ldapbinddn and ldapbindpasswd must be specified as separate options.

LDAP URLs are currently only supported with OpenLDAP, not on Windows.

It is an error to mix configuration options for simple bind with options for search+bind. To use ldapurl in simple bind mode, the URL must not contain a basedn or query elements.

When using search+bind mode, the search can be performed using a single attribute specified with ldapsearchattribute, or using a custom search filter specified with ldapsearchfilter. Specifying ldapsearchattribute=foo is equivalent to specifying ldapsearchfilter="(foo=$username)". If neither option is specified the default is ldapsearchattribute=uid.

If PostgreSQL was compiled with OpenLDAP as the LDAP client library, the ldapserver setting may be omitted. In that case, a list of host names and ports is looked up via RFC 2782 DNS SRV records. The name _ldap._tcp.DOMAIN is looked up, where DOMAIN is extracted from ldapbasedn.

Here is an example for a simple-bind LDAP configuration:

host ... ldap ldapserver=ldap.example.net ldapprefix="cn=" ldapsuffix=", dc=example, dc=net"

When a connection to the database server as database user someuser is requested, PostgreSQL will attempt to bind to the LDAP server using the DN cn=someuser, dc=example, dc=net and the password provided by the client. If that connection succeeds, the database access is granted.

Here is a different simple-bind configuration, which uses the LDAPS scheme and a custom port number, written as a URL:

host ... ldap ldapurl="ldaps://ldap.example.net:49151" ldapprefix="cn=" ldapsuffix=", dc=example, dc=net"

This is slightly more compact than specifying ldapserver, ldapscheme, and ldapport separately.

Here is an example for a search+bind configuration:

host ... ldap ldapserver=ldap.example.net ldapbasedn="dc=example, dc=net" ldapsearchattribute=uid

When a connection to the database server as database user someuser is requested, PostgreSQL will attempt to bind anonymously (since ldapbinddn was not specified) to the LDAP server, perform a search for (uid=someuser) under the specified base DN. If an entry is found, it will then attempt to bind using that found information and the password supplied by the client. If that second bind succeeds, the database access is granted.

Here is the same search+bind configuration written as a URL:

host ... ldap ldapurl="ldap://ldap.example.net/dc=example,dc=net?uid?sub"

Some other software that supports authentication against LDAP uses the same URL format, so it will be easier to share the configuration.

Here is an example for a search+bind configuration that uses ldapsearchfilter instead of ldapsearchattribute to allow authentication by user ID or email address:

host ... ldap ldapserver=ldap.example.net ldapbasedn="dc=example, dc=net" ldapsearchfilter="(|(uid=$username)(mail=$username))"

Here is an example for a search+bind configuration that uses DNS SRV discovery to find the host name(s) and port(s) for the LDAP service for the domain name example.net:

host ... ldap ldapbasedn="dc=example,dc=net"

[!TIP] Since LDAP often uses commas and spaces to separate the different parts of a DN, it is often necessary to use double-quoted parameter values when configuring LDAP options, as shown in the examples.

Certificate Authentication

Certificate

This authentication method uses SSL client certificates to perform authentication. It is therefore only available for SSL connections; see ??? for SSL configuration instructions. When using this authentication method, the server will require that the client provide a valid, trusted certificate. No password prompt will be sent to the client. The cn (Common Name) attribute of the certificate will be compared to the requested database user name, and if they match the login will be allowed. User name mapping can be used to allow cn to be different from the database user name.

The following configuration options are supported for SSL certificate authentication:

map
Allows for mapping between system and database user names. See User Name Maps for details.

It is redundant to use the clientcert option with cert authentication because cert authentication is effectively trust authentication with clientcert=verify-full.

PAM Authentication

PAM

This authentication method operates similarly to password except that it uses PAM (Pluggable Authentication Modules) as the authentication mechanism. The default PAM service name is postgresql. PAM is used only to validate user name/password pairs and optionally the connected remote host name or IP address. Therefore the user must already exist in the database before PAM can be used for authentication. For more information about PAM, please read the Linux-PAM Page.

The following configuration options are supported for PAM:

pamservice
PAM service name.

pam_use_hostname
Determines whether the remote IP address or the host name is provided to PAM modules through the PAM_RHOST item. By default, the IP address is used. Set this option to 1 to use the resolved host name instead. Host name resolution can lead to login delays. (Most PAM configurations don't use this information, so it is only necessary to consider this setting if a PAM configuration was specifically created to make use of it.)

[!NOTE] If PAM is set up to read /etc/shadow, authentication will fail because the PostgreSQL server is started by a non-root user. However, this is not an issue when PAM is configured to use LDAP or other authentication methods.

BSD Authentication

BSD Authentication

This authentication method operates similarly to password except that it uses BSD Authentication to verify the password. BSD Authentication is used only to validate user name/password pairs. Therefore the user's role must already exist in the database before BSD Authentication can be used for authentication. The BSD Authentication framework is currently only available on OpenBSD.

BSD Authentication in PostgreSQL uses the auth-postgresql login type and authenticates with the postgresql login class if that's defined in login.conf. By default that login class does not exist, and PostgreSQL will use the default login class.

[!NOTE] To use BSD Authentication, the PostgreSQL user account (that is, the operating system user running the server) must first be added to the auth group. The auth group exists by default on OpenBSD systems.

OAuth Authorization/Authentication

OAuth Authorization/Authentication

OAuth 2.0 is an industry-standard framework, defined in RFC 6749, to enable third-party applications to obtain limited access to a protected resource. OAuth client support has to be enabled when PostgreSQL is built, see ??? for more information.

This documentation uses the following terminology when discussing the OAuth ecosystem:

Resource Owner (or End User)
The user or system who owns protected resources and can grant access to them. This documentation also uses the term end user when the resource owner is a person. When you use psql to connect to the database using OAuth, you are the resource owner/end user.

Client
The system which accesses the protected resources using access tokens. Applications using libpq, such as psql, are the OAuth clients when connecting to a PostgreSQL cluster.

Resource Server
The system hosting the protected resources which are accessed by the client. The PostgreSQL cluster being connected to is the resource server.

Provider
The organization, product vendor, or other entity which develops and/or administers the OAuth authorization servers and clients for a given application. Different providers typically choose different implementation details for their OAuth systems; a client of one provider is not generally guaranteed to have access to the servers of another.

This use of the term "provider" is not standard, but it seems to be in wide use colloquially. (It should not be confused with OpenID's similar term "Identity Provider". While the implementation of OAuth in PostgreSQL is intended to be interoperable and compatible with OpenID Connect/OIDC, it is not itself an OIDC client and does not require its use.)

Authorization Server
The system which receives requests from, and issues access tokens to, the client after the authenticated resource owner has given approval. PostgreSQL does not provide an authorization server; it is the responsibility of the OAuth provider.

Issuer
An identifier for an authorization server, printed as an https:// URL, which provides a trusted "namespace" for OAuth clients and applications. The issuer identifier allows a single authorization server to talk to the clients of mutually untrusting entities, as long as they maintain separate issuers.

[!NOTE] For small deployments, there may not be a meaningful distinction between the "provider", "authorization server", and "issuer". However, for more complicated setups, there may be a one-to-many (or many-to-many) relationship: a provider may rent out multiple issuer identifiers to separate tenants, then provide multiple authorization servers, possibly with different supported feature sets, to interact with their clients.

PostgreSQL supports bearer tokens, defined in RFC 6750, which are a type of access token used with OAuth 2.0 where the token is an opaque string. The format of the access token is implementation specific and is chosen by each authorization server.

The following configuration options are supported for OAuth:

issuer
An HTTPS URL which is either the exact issuer identifier of the authorization server, as defined by its discovery document, or a well-known URI that points directly to that discovery document. This parameter is required.

When an OAuth client connects to the server, a URL for the discovery document will be constructed using the issuer identifier. By default, this URL uses the conventions of OpenID Connect Discovery: the path /.well-known/openid-configuration will be appended to the end of the issuer identifier. Alternatively, if the issuer contains a /.well-known/ path segment, that URL will be provided to the client as-is.

[!WARNING] The OAuth client in libpq requires the server's issuer setting to exactly match the issuer identifier which is provided in the discovery document, which must in turn match the client's ??? setting. No variations in case or formatting are permitted.

scope
A space-separated list of the OAuth scopes needed for the server to both authorize the client and authenticate the user. Appropriate values are determined by the authorization server and the OAuth validation module used (see ??? for more information on validators). This parameter is required.

validator
The library to use for validating bearer tokens. If given, the name must exactly match one of the libraries listed in ???. This parameter is optional unless oauth_validator_libraries contains more than one library, in which case it is required.

validator.option
Validator modules may define additional configuration options for oauth HBA entries. These validator-specific options are accessible via the validator.* "namespace". For example, a module may register the validator.foo and validator.bar options and define their effects on authentication.

The name, syntax, and behavior of each <option> are not determined by PostgreSQL; consult the documentation for the validator module in use.

[!WARNING] A limitation of the current implementation is that unrecognized <option> names will not be caught until connection time. A pg_ctl reload will succeed, but matching connections will fail:

LOG:  connection received: host=[local]
WARNING:  unrecognized authentication option name: "validator.bad"
DETAIL:  The installed validator module ("my_validator") did not define an option named "bad".
HINT:  All OAuth connections matching this line will fail. Correct the option and reload the server configuration.
CONTEXT:  line 2 of configuration file "data/pg_hba.conf"

Use caution when making changes to validator-specific HBA options in production systems.

map
Allows for mapping between OAuth identity provider and database user names. See User Name Maps for details. If a map is not specified, the user name associated with the token (as determined by the OAuth validator) must exactly match the role name being requested. This parameter is optional.

delegate_ident_mapping
An advanced option which is not intended for common use.

When set to 1, standard user mapping with pg_ident.conf is skipped, and the OAuth validator takes full responsibility for mapping end user identities to database roles. If the validator authorizes the token, the server trusts that the user is allowed to connect under the requested role, and the connection is allowed to proceed regardless of the authentication status of the user.

This parameter is incompatible with map.

[!WARNING] delegate_ident_mapping provides additional flexibility in the design of the authentication system, but it also requires careful implementation of the OAuth validator, which must determine whether the provided token carries sufficient end-user privileges in addition to the standard checks required of all validators. Use with caution.

Authentication Problems

Authentication failures and related problems generally manifest themselves through error messages like the following:

FATAL:  no pg_hba.conf entry for host "123.123.123.123", user "andym", database "testdb"

This is what you are most likely to get if you succeed in contacting the server, but it does not want to talk to you. As the message suggests, the server refused the connection request because it found no matching entry in its pg_hba.conf configuration file.

FATAL:  password authentication failed for user "andym"

Messages like this indicate that you contacted the server, and it is willing to talk to you, but not until you pass the authorization method specified in the pg_hba.conf file. Check the password you are providing, or check your Kerberos or ident software if the complaint mentions one of those authentication types.

FATAL:  user "andym" does not exist

The indicated database user name was not found.

FATAL:  database "testdb" does not exist

The database you are trying to connect to does not exist. Note that if you do not specify a database name, it defaults to the database user name.

[!TIP] The server log might contain more information about an authentication failure than is reported to the client. If you are confused about the reason for a failure, check the server log.

Data Manipulation

The previous chapter discussed how to create tables and other structures to hold your data. Now it is time to fill the tables with data. This chapter covers how to insert, update, and delete table data. The chapter after this will finally explain how to extract your long-lost data from the database.

Inserting Data

inserting

INSERT

When a table is created, it contains no data. The first thing to do before a database can be of much use is to insert data. Data is inserted one row at a time. You can also insert more than one row in a single command, but it is not possible to insert something that is not a complete row. Even if you know only some column values, a complete row must be created.

To create a new row, use the ??? command. The command requires the table name and column values. For example, consider the products table from ???:

CREATE TABLE products (
    product_no integer,
    name text,
    price numeric
);

An example command to insert a row would be:

INSERT INTO products VALUES (1, 'Cheese', 9.99);

The data values are listed in the order in which the columns appear in the table, separated by commas. Usually, the data values will be literals (constants), but scalar expressions are also allowed.

The above syntax has the drawback that you need to know the order of the columns in the table. To avoid this you can also list the columns explicitly. For example, both of the following commands have the same effect as the one above:

INSERT INTO products (product_no, name, price) VALUES (1, 'Cheese', 9.99);
INSERT INTO products (name, price, product_no) VALUES ('Cheese', 9.99, 1);

Many users consider it good practice to always list the column names.

If you don't have values for all the columns, you can omit some of them. In that case, the columns will be filled with their default values. For example:

INSERT INTO products (product_no, name) VALUES (1, 'Cheese');
INSERT INTO products VALUES (1, 'Cheese');

The second form is a PostgreSQL extension. It fills the columns from the left with as many values as are given, and the rest will be defaulted.

For clarity, you can also request default values explicitly, for individual columns or for the entire row:

INSERT INTO products (product_no, name, price) VALUES (1, 'Cheese', DEFAULT);
INSERT INTO products DEFAULT VALUES;

You can insert multiple rows in a single command:

INSERT INTO products (product_no, name, price) VALUES
    (1, 'Cheese', 9.99),
    (2, 'Bread', 1.99),
    (3, 'Milk', 2.99);

It is also possible to insert the result of a query (which might be no rows, one row, or many rows):

INSERT INTO products (product_no, name, price)
  SELECT product_no, name, price FROM new_products
    WHERE release_date = 'today';

This provides the full power of the SQL query mechanism (???) for computing the rows to be inserted.

[!TIP] When inserting a lot of data at the same time, consider using the ??? command. It is not as flexible as the ??? command, but is more efficient. Refer to ??? for more information on improving bulk loading performance.

Updating Data

updating

UPDATE

The modification of data that is already in the database is referred to as updating. You can update individual rows, all the rows in a table, or a subset of all rows. Each column can be updated separately; the other columns are not affected.

To update existing rows, use the ??? command. This requires three pieces of information:

  1. The name of the table and column to update
  2. The new value of the column
  3. Which row(s) to update

Recall from ??? that SQL does not, in general, provide a unique identifier for rows. Therefore it is not always possible to directly specify which row to update. Instead, you specify which conditions a row must meet in order to be updated. Only if you have a primary key in the table (independent of whether you declared it or not) can you reliably address individual rows by choosing a condition that matches the primary key. Graphical database access tools rely on this fact to allow you to update rows individually.

For example, this command updates all products that have a price of 5 to have a price of 10:

UPDATE products SET price = 10 WHERE price = 5;

This might cause zero, one, or many rows to be updated. It is not an error to attempt an update that does not match any rows.

Let's look at that command in detail. First is the key word UPDATE followed by the table name. As usual, the table name can be schema-qualified, otherwise it is looked up in the path. Next is the key word SET followed by the column name, an equal sign, and the new column value. The new column value can be any scalar expression, not just a constant. For example, if you want to raise the price of all products by 10% you could use:

UPDATE products SET price = price * 1.10;

As you see, the expression for the new value can refer to the existing value(s) in the row. We also left out the WHERE clause. If it is omitted, it means that all rows in the table are updated. If it is present, only those rows that match the WHERE condition are updated. Note that the equals sign in the SET clause is an assignment while the one in the WHERE clause is a comparison, but this does not create any ambiguity. Of course, the WHERE condition does not have to be an equality test. Many other operators are available (see ???). But the expression needs to evaluate to a Boolean result.

You can update more than one column in an UPDATE command by listing more than one assignment in the SET clause. For example:

UPDATE mytable SET a = 5, b = 3, c = 1 WHERE a > 0;

Deleting Data

deleting

DELETE

So far we have explained how to add data to tables and how to change data. What remains is to discuss how to remove data that is no longer needed. Just as adding data is only possible in whole rows, you can only remove entire rows from a table. In the previous section we explained that SQL does not provide a way to directly address individual rows. Therefore, removing rows can only be done by specifying conditions that the rows to be removed have to match. If you have a primary key in the table then you can specify the exact row. But you can also remove groups of rows matching a condition, or you can remove all rows in the table at once.

You use the ??? command to remove rows; the syntax is very similar to the ??? command. For instance, to remove all rows from the products table that have a price of 10, use:

DELETE FROM products WHERE price = 10;

If you simply write:

DELETE FROM products;

then all rows in the table will be deleted! Caveat programmer.

Returning Data from Modified Rows

RETURNING

INSERT

RETURNING

UPDATE

RETURNING

DELETE

RETURNING

MERGE

RETURNING

Sometimes it is useful to obtain data from modified rows while they are being manipulated. The INSERT, UPDATE, DELETE, and MERGE commands all have an optional RETURNING clause that supports this. Use of RETURNING avoids performing an extra database query to collect the data, and is especially valuable when it would otherwise be difficult to identify the modified rows reliably.

The allowed contents of a RETURNING clause are the same as a SELECT command's output list (see ???). It can contain column names of the command's target table, or value expressions using those columns. A common shorthand is RETURNING *, which selects all columns of the target table in order.

In an INSERT, the default data available to RETURNING is the row as it was inserted. This is not so useful in trivial inserts, since it would just repeat the data provided by the client. But it can be very handy when relying on computed default values. For example, when using a serial column to provide unique identifiers, RETURNING can return the ID assigned to a new row:

CREATE TABLE users (firstname text, lastname text, id serial PRIMARY KEY);

INSERT INTO users (firstname, lastname) VALUES ('Joe', 'Cool') RETURNING id;

The RETURNING clause is also very useful with INSERT ... SELECT.

In an UPDATE, the default data available to RETURNING is the new content of the modified row. For example:

UPDATE products SET price = price * 1.10
  WHERE price <= 99.99
  RETURNING name, price AS new_price;

In a DELETE, the default data available to RETURNING is the content of the deleted row. For example:

DELETE FROM products
  WHERE obsoletion_date = 'today'
  RETURNING *;

In a MERGE, the default data available to RETURNING is the content of the source row plus the content of the inserted, updated, or deleted target row. Since it is quite common for the source and target to have many of the same columns, specifying RETURNING * can lead to a lot of duplicated columns, so it is often more useful to qualify it so as to return just the source or target row. For example:

MERGE INTO products p USING new_products n ON p.product_no = n.product_no
  WHEN NOT MATCHED THEN INSERT VALUES (n.product_no, n.name, n.price)
  WHEN MATCHED THEN UPDATE SET name = n.name, price = n.price
  RETURNING p.*;

In each of these commands, it is also possible to explicitly return the old and new content of the modified row. For example:

UPDATE products SET price = price * 1.10
  WHERE price <= 99.99
  RETURNING name, old.price AS old_price, new.price AS new_price,
            new.price - old.price AS price_change;

In this example, writing new.price is the same as just writing price, but it makes the meaning clearer.

This syntax for returning old and new values is available in INSERT, UPDATE, DELETE, and MERGE commands, but typically old values will be NULL for an INSERT, and new values will be NULL for a DELETE. However, there are situations where it can still be useful for those commands. For example, in an INSERT with an ON CONFLICT DO SELECT/UPDATE clause, the old values will be non-NULL for conflicting rows. Similarly, if a DELETE is turned into an UPDATE by a rewrite rule, the new values may be non-NULL.

If there are triggers (???) on the target table, the data available to RETURNING is the row as modified by the triggers. Thus, inspecting columns computed by triggers is another common use-case for RETURNING.

A Brief History of PostgreSQL

history

of PostgreSQL

The object-relational database management system now known as PostgreSQL is derived from the POSTGRES package written at the University of California at Berkeley. With decades of development behind it, PostgreSQL is now the most advanced open-source database available anywhere.

Another take on the history presented here can be found in Dr. Joe Hellerstein's paper “Looking Back at Postgres” ???.

The Berkeley POSTGRES Project

POSTGRES

The POSTGRES project, led by Professor Michael Stonebraker, was sponsored by the Defense Advanced Research Projects Agency (DARPA), the Army Research Office (ARO), the National Science Foundation (NSF), and ESL, Inc. The implementation of POSTGRES began in 1986. The initial concepts for the system were presented in ???, and the definition of the initial data model appeared in ???. The design of the rule system at that time was described in ???. The rationale and architecture of the storage manager were detailed in ???.

POSTGRES has undergone several major releases since then. The first “demoware” system became operational in 1987 and was shown at the 1988 ACM-SIGMOD Conference. Version 1, described in ???, was released to a few external users in June 1989. In response to a critique of the first rule system (???), the rule system was redesigned (???), and Version 2 was released in June 1990 with the new rule system. Version 3 appeared in 1991 and added support for multiple storage managers, an improved query executor, and a rewritten rule system. For the most part, subsequent releases until Postgres95 (see below) focused on portability and reliability.

POSTGRES has been used to implement many different research and production applications. These include: a financial data analysis system, a jet engine performance monitoring package, an asteroid tracking database, a medical information database, and several geographic information systems. POSTGRES has also been used as an educational tool at several universities. Finally, Illustra Information Technologies (later merged into Informix, which is now owned by IBM) picked up the code and commercialized it. In late 1992, POSTGRES became the primary data manager for the Sequoia 2000 scientific computing project described in ???.

The size of the external user community nearly doubled during 1993. It became increasingly obvious that maintenance of the prototype code and support was taking up large amounts of time that should have been devoted to database research. In an effort to reduce this support burden, the Berkeley POSTGRES project officially ended with Version 4.2.

Postgres95

Postgres95

In 1994, Andrew Yu and Jolly Chen added an SQL language interpreter to POSTGRES. Under a new name, Postgres95 was subsequently released to the web to find its own way in the world as an open-source descendant of the original POSTGRES Berkeley code.

Postgres95 code was completely ANSI C and trimmed in size by 25%. Many internal changes improved performance and maintainability. Postgres95 release 1.0.x ran about 30–50% faster on the Wisconsin Benchmark compared to POSTGRES, Version 4.2. Apart from bug fixes, the following were the major enhancements:

PostgreSQL

By 1996, it became clear that the name “Postgres95” would not stand the test of time. We chose a new name, PostgreSQL, to reflect the relationship between the original POSTGRES and the more recent versions with SQL capability. At the same time, we set the version numbering to start at 6.0, putting the numbers back into the sequence originally begun by the Berkeley POSTGRES project.

Postgres is still considered an official project name, both because of tradition and because people find it easier to pronounce Postgres than PostgreSQL.

The emphasis during development of Postgres95 was on identifying and understanding existing problems in the server code. With PostgreSQL, the emphasis has shifted to augmenting features and capabilities, although work continues in all areas.

Details about what has happened in each PostgreSQL release since then can be found at https://www.postgresql.org/docs/release/.

Indexes

index

Indexes are a common way to enhance database performance. An index allows the database server to find and retrieve specific rows much faster than it could do without an index. But indexes also add overhead to the database system as a whole, so they should be used sensibly.

Introduction

Suppose we have a table similar to this:

CREATE TABLE test1 (
    id integer,
    content varchar
);

and the application issues many queries of the form:

SELECT content FROM test1 WHERE id = constant;

With no advance preparation, the system would have to scan the entire test1 table, row by row, to find all matching entries. If there are many rows in test1 and only a few rows (perhaps zero or one) that would be returned by such a query, this is clearly an inefficient method. But if the system has been instructed to maintain an index on the id column, it can use a more efficient method for locating matching rows. For instance, it might only have to walk a few levels deep into a search tree.

A similar approach is used in most non-fiction books: terms and concepts that are frequently looked up by readers are collected in an alphabetic index at the end of the book. The interested reader can scan the index relatively quickly and flip to the appropriate page(s), rather than having to read the entire book to find the material of interest. Just as it is the task of the author to anticipate the items that readers are likely to look up, it is the task of the database programmer to foresee which indexes will be useful.

The following command can be used to create an index on the id column, as discussed:

CREATE INDEX test1_id_index ON test1 (id);

The name test1_id_index can be chosen freely, but you should pick something that enables you to remember later what the index was for.

To remove an index, use the DROP INDEX command. Indexes can be added to and removed from tables at any time.

Once an index is created, no further intervention is required: the system will update the index when the table is modified, and it will use the index in queries when it thinks doing so would be more efficient than a sequential table scan. But you might have to run the ANALYZE command regularly to update statistics to allow the query planner to make educated decisions. See ??? for information about how to find out whether an index is used and when and why the planner might choose not to use an index.

Indexes can also benefit UPDATE and DELETE commands with search conditions. Indexes can moreover be used in join searches. Thus, an index defined on a column that is part of a join condition can also significantly speed up queries with joins.

In general, PostgreSQL indexes can be used to optimize queries that contain one or more WHERE or JOIN clauses of the form <indexed-column> <indexable-operator> <comparison-value> Here, the <indexed-column> is whatever column or expression the index has been defined on. The <indexable-operator> is an operator that is a member of the index's operator class for the indexed column. (More details about that appear below.) And the <comparison-value> can be any expression that is not volatile and does not reference the index's table.

In some cases the query planner can extract an indexable clause of this form from another SQL construct. A simple example is that if the original clause was <comparison-value> <operator> <indexed-column> then it can be flipped around into indexable form if the original <operator> has a commutator operator that is a member of the index's operator class.

Creating an index on a large table can take a long time. By default, PostgreSQL allows reads (SELECT statements) to occur on the table in parallel with index creation, but writes (INSERT, UPDATE, DELETE) are blocked until the index build is finished. In production environments this is often unacceptable. It is possible to allow writes to occur in parallel with index creation, but there are several caveats to be aware of — for more information see ???.

After an index is created, the system has to keep it synchronized with the table. This adds overhead to data manipulation operations. Indexes can also prevent the creation of heap-only tuples. Therefore indexes that are seldom or never used in queries should be removed.

Index Types

PostgreSQL provides several index types: B-tree, Hash, GiST, SP-GiST, GIN, BRIN, and the extension bloom. Each index type uses a different algorithm that is best suited to different types of indexable clauses. By default, the CREATE INDEX command creates B-tree indexes, which fit the most common situations. The other index types are selected by writing the keyword USING followed by the index type name. For example, to create a Hash index:

CREATE INDEX name ON table USING HASH (column);

B-Tree

index

B-Tree

B-Tree

index

B-trees can handle equality and range queries on data that can be sorted into some ordering. In particular, the PostgreSQL query planner will consider using a B-tree index whenever an indexed column is involved in a comparison using one of these operators: <   <=   =   >=   > Constructs equivalent to combinations of these operators, such as BETWEEN and IN, can also be implemented with a B-tree index search. Also, an IS NULL or IS NOT NULL condition on an index column can be used with a B-tree index.

The optimizer can also use a B-tree index for queries involving the pattern matching operators LIKE and ~ if the pattern is a constant and is anchored to the beginning of the string — for example, col LIKE 'foo%' or col ~ '^foo', but not col LIKE '%bar'. However, if your database does not use the C locale you will need to create the index with a special operator class to support indexing of pattern-matching queries; see Operator Classes and Operator Families below. It is also possible to use B-tree indexes for ILIKE and ~*, but only if the pattern starts with non-alphabetic characters, i.e., characters that are not affected by upper/lower case conversion.

B-tree indexes can also be used to retrieve data in sorted order. This is not always faster than a simple scan and sort, but it is often helpful.

Hash

index

hash

hash

index

Hash indexes store a 32-bit hash code derived from the value of the indexed column. Hence, such indexes can only handle simple equality comparisons. The query planner will consider using a hash index whenever an indexed column is involved in a comparison using the equal operator: =

GiST

index

GiST

GiST

index

GiST indexes are not a single kind of index, but rather an infrastructure within which many different indexing strategies can be implemented. Accordingly, the particular operators with which a GiST index can be used vary depending on the indexing strategy (the operator class). As an example, the standard distribution of PostgreSQL includes GiST operator classes for several two-dimensional geometric data types, which support indexed queries using these operators: <<   &<   &>   >>   <<|   &<|   |&>   |>>   @>   <@   ~=   && (See ??? for the meaning of these operators.) The GiST operator classes included in the standard distribution are documented in ???. Many other GiST operator classes are available in the contrib collection or as separate projects. For more information see ???.

GiST indexes are also capable of optimizing “nearest-neighbor” searches, such as

SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10;

which finds the ten places closest to a given target point. The ability to do this is again dependent on the particular operator class being used. In ???, operators that can be used in this way are listed in the column “Ordering Operators”.

SP-GiST

index

SP-GiST

SP-GiST

index

SP-GiST indexes, like GiST indexes, offer an infrastructure that supports various kinds of searches. SP-GiST permits implementation of a wide range of different non-balanced disk-based data structures, such as quadtrees, k-d trees, and radix trees (tries). As an example, the standard distribution of PostgreSQL includes SP-GiST operator classes for two-dimensional points, which support indexed queries using these operators: <<   >>   ~=   <@   <<|   |>> (See ??? for the meaning of these operators.) The SP-GiST operator classes included in the standard distribution are documented in ???. For more information see ???.

Like GiST, SP-GiST supports “nearest-neighbor” searches. For SP-GiST operator classes that support distance ordering, the corresponding operator is listed in the “Ordering Operators” column in ???.

GIN

index

GIN

GIN

index

GIN indexes are “inverted indexes” which are appropriate for data values that contain multiple component values, such as arrays. An inverted index contains a separate entry for each component value, and can efficiently handle queries that test for the presence of specific component values.

Like GiST and SP-GiST, GIN can support many different user-defined indexing strategies, and the particular operators with which a GIN index can be used vary depending on the indexing strategy. As an example, the standard distribution of PostgreSQL includes a GIN operator class for arrays, which supports indexed queries using these operators: <@   @>   =   && (See ??? for the meaning of these operators.) The GIN operator classes included in the standard distribution are documented in ???. Many other GIN operator classes are available in the contrib collection or as separate projects. For more information see ???.

BRIN

index

BRIN

BRIN

index

BRIN indexes (a shorthand for Block Range INdexes) store summaries about the values stored in consecutive physical block ranges of a table. Thus, they are most effective for columns whose values are well-correlated with the physical order of the table rows. Like GiST, SP-GiST and GIN, BRIN can support many different indexing strategies, and the particular operators with which a BRIN index can be used vary depending on the indexing strategy. For data types that have a linear sort order, the indexed data corresponds to the minimum and maximum values of the values in the column for each block range. This supports indexed queries using these operators: <   <=   =   >=   > The BRIN operator classes included in the standard distribution are documented in ???. For more information see ???.

Multicolumn Indexes

index

multicolumn

An index can be defined on more than one column of a table. For example, if you have a table of this form:

CREATE TABLE test2 (
  major int,
  minor int,
  name varchar
);

(say, you keep your /dev directory in a database...) and you frequently issue queries like:

SELECT name FROM test2 WHERE major = constant AND minor = constant;

then it might be appropriate to define an index on the columns major and minor together, e.g.:

CREATE INDEX test2_mm_idx ON test2 (major, minor);

Currently, only the B-tree, GiST, GIN, and BRIN index types support multiple-key-column indexes. Whether there can be multiple key columns is independent of whether INCLUDE columns can be added to the index. Indexes can have up to 32 columns, including INCLUDE columns. (This limit can be altered when building PostgreSQL; see the file pg_config_manual.h.)

A multicolumn B-tree index can be used with query conditions that involve any subset of the index's columns, but the index is most efficient when there are constraints on the leading (leftmost) columns. The exact rule is that equality constraints on leading columns, plus any inequality constraints on the first column that does not have an equality constraint, will always be used to limit the portion of the index that is scanned. Constraints on columns to the right of these columns are checked in the index, so they'll always save visits to the table proper, but they do not necessarily reduce the portion of the index that has to be scanned. If a B-tree index scan can apply the skip scan optimization effectively, it will apply every column constraint when navigating through the index via repeated index searches. This can reduce the portion of the index that has to be read, even though one or more columns (prior to the least significant index column from the query predicate) lacks a conventional equality constraint. Skip scan works by generating a dynamic equality constraint internally, that matches every possible value in an index column (though only given a column that lacks an equality constraint that comes from the query predicate, and only when the generated constraint can be used in conjunction with a later column constraint from the query predicate).

For example, given an index on (x, y), and a query condition WHERE y = 7700, a B-tree index scan might be able to apply the skip scan optimization. This generally happens when the query planner expects that repeated WHERE x = N AND y = 7700 searches for every possible value of N (or for every x value that is actually stored in the index) is the fastest possible approach, given the available indexes on the table. This approach is generally only taken when there are so few distinct x values that the planner expects the scan to skip over most of the index (because most of its leaf pages cannot possibly contain relevant tuples). If there are many distinct x values, then the entire index will have to be scanned, so in most cases the planner will prefer a sequential table scan over using the index.

The skip scan optimization can also be applied selectively, during B-tree scans that have at least some useful constraints from the query predicate. For example, given an index on (a, b, c) and a query condition WHERE a = 5 AND b >= 42 AND c < 77, the index might have to be scanned from the first entry with a = 5 and b = 42 up through the last entry with a = 5. Index entries with c >= 77 will never need to be filtered at the table level, but it may or may not be profitable to skip over them within the index. When skipping takes place, the scan starts a new index search to reposition itself from the end of the current a = 5 and b = N grouping (i.e. from the position in the index where the first tuple a = 5 AND b = N AND c >= 77 appears), to the start of the next such grouping (i.e. the position in the index where the first tuple a = 5 AND b = N + 1 appears).

A multicolumn GiST index can be used with query conditions that involve any subset of the index's columns. Conditions on additional columns restrict the entries returned by the index, but the condition on the first column is the most important one for determining how much of the index needs to be scanned. A GiST index will be relatively ineffective if its first column has only a few distinct values, even if there are many distinct values in additional columns.

A multicolumn GIN index can be used with query conditions that involve any subset of the index's columns. Unlike B-tree or GiST, index search effectiveness is the same regardless of which index column(s) the query conditions use.

A multicolumn BRIN index can be used with query conditions that involve any subset of the index's columns. Like GIN and unlike B-tree or GiST, index search effectiveness is the same regardless of which index column(s) the query conditions use. The only reason to have multiple BRIN indexes instead of one multicolumn BRIN index on a single table is to have a different pages_per_range storage parameter.

Of course, each column must be used with operators appropriate to the index type; clauses that involve other operators will not be considered.

Multicolumn indexes should be used sparingly. In most situations, an index on a single column is sufficient and saves space and time. Indexes with more than three columns are unlikely to be helpful unless the usage of the table is extremely stylized. See also Combining Multiple Indexes and Index-Only Scans and Covering Indexes for some discussion of the merits of different index configurations.

Indexes and ORDER BY

index

and

ORDER BY

In addition to simply finding the rows to be returned by a query, an index may be able to deliver them in a specific sorted order. This allows a query's ORDER BY specification to be honored without a separate sorting step. Of the index types currently supported by PostgreSQL, only B-tree can produce sorted output — the other index types return matching rows in an unspecified, implementation-dependent order.

The planner will consider satisfying an ORDER BY specification either by scanning an available index that matches the specification, or by scanning the table in physical order and doing an explicit sort. For a query that requires scanning a large fraction of the table, an explicit sort is likely to be faster than using an index because it requires less disk I/O due to following a sequential access pattern. Indexes are more useful when only a few rows need be fetched. An important special case is ORDER BY in combination with LIMIT <n>: an explicit sort will have to process all the data to identify the first <n> rows, but if there is an index matching the ORDER BY, the first <n> rows can be retrieved directly, without scanning the remainder at all.

By default, B-tree indexes store their entries in ascending order with nulls last (table TID is treated as a tiebreaker column among otherwise equal entries). This means that a forward scan of an index on column x produces output satisfying ORDER BY x (or more verbosely, ORDER BY x ASC NULLS LAST). The index can also be scanned backward, producing output satisfying ORDER BY x DESC (or more verbosely, ORDER BY x DESC NULLS FIRST, since NULLS FIRST is the default for ORDER BY DESC).

You can adjust the ordering of a B-tree index by including the options ASC, DESC, NULLS FIRST, and/or NULLS LAST when creating the index; for example:

CREATE INDEX test2_info_nulls_low ON test2 (info NULLS FIRST);
CREATE INDEX test3_desc_index ON test3 (id DESC NULLS LAST);

An index stored in ascending order with nulls first can satisfy either ORDER BY x ASC NULLS FIRST or ORDER BY x DESC NULLS LAST depending on which direction it is scanned in.

You might wonder why bother providing all four options, when two options together with the possibility of backward scan would cover all the variants of ORDER BY. In single-column indexes the options are indeed redundant, but in multicolumn indexes they can be useful. Consider a two-column index on (x, y): this can satisfy ORDER BY x, y if we scan forward, or ORDER BY x DESC, y DESC if we scan backward. But it might be that the application frequently needs to use ORDER BY x ASC, y DESC. There is no way to get that ordering from a plain index, but it is possible if the index is defined as (x ASC, y DESC) or (x DESC, y ASC).

Obviously, indexes with non-default sort orderings are a fairly specialized feature, but sometimes they can produce tremendous speedups for certain queries. Whether it's worth maintaining such an index depends on how often you use queries that require a special sort ordering.

Combining Multiple Indexes

index

combining multiple indexes

bitmap scan

A single index scan can only use query clauses that use the index's columns with operators of its operator class and are joined with AND. For example, given an index on (a, b) a query condition like WHERE a = 5 AND b = 6 could use the index, but a query like WHERE a = 5 OR b = 6 could not directly use the index.

Fortunately, PostgreSQL has the ability to combine multiple indexes (including multiple uses of the same index) to handle cases that cannot be implemented by single index scans. The system can form AND and OR conditions across several index scans. For example, a query like WHERE x = 42 OR x = 47 OR x = 53 OR x = 99 could be broken down into four separate scans of an index on x, each scan using one of the query clauses. The results of these scans are then ORed together to produce the result. Another example is that if we have separate indexes on x and y, one possible implementation of a query like WHERE x = 5 AND y = 6 is to use each index with the appropriate query clause and then AND together the index results to identify the result rows.

To combine multiple indexes, the system scans each needed index and prepares a bitmap in memory giving the locations of table rows that are reported as matching that index's conditions. The bitmaps are then ANDed and ORed together as needed by the query. Finally, the actual table rows are visited and returned. The table rows are visited in physical order, because that is how the bitmap is laid out; this means that any ordering of the original indexes is lost, and so a separate sort step will be needed if the query has an ORDER BY clause. For this reason, and because each additional index scan adds extra time, the planner will sometimes choose to use a simple index scan even though additional indexes are available that could have been used as well.

In all but the simplest applications, there are various combinations of indexes that might be useful, and the database developer must make trade-offs to decide which indexes to provide. Sometimes multicolumn indexes are best, but sometimes it's better to create separate indexes and rely on the index-combination feature. For example, if your workload includes a mix of queries that sometimes involve only column x, sometimes only column y, and sometimes both columns, you might choose to create two separate indexes on x and y, relying on index combination to process the queries that use both columns. You could also create a multicolumn index on (x, y). This index would typically be more efficient than index combination for queries involving both columns, but as discussed in Multicolumn Indexes, it would be less useful for queries involving only y. Just how useful will depend on how effective the B-tree index skip scan optimization is; if x has no more than several hundred distinct values, skip scan will make searches for specific y values execute reasonably efficiently. A combination of a multicolumn index on (x, y) and a separate index on y might also serve reasonably well. For queries involving only x, the multicolumn index could be used, though it would be larger and hence slower than an index on x alone. The last alternative is to create all three indexes, but this is probably only reasonable if the table is searched much more often than it is updated and all three types of query are common. If one of the types of query is much less common than the others, you'd probably settle for creating just the two indexes that best match the common types.

Unique Indexes

index

unique

Indexes can also be used to enforce uniqueness of a column's value, or the uniqueness of the combined values of more than one column. CREATE UNIQUE INDEX <name> ON <table> (<column> [, ...]) [NULLS [NOT] DISTINCT]; Currently, only B-tree indexes can be declared unique.

When an index is declared unique, multiple table rows with equal indexed values are not allowed. By default, null values in a unique column are not considered equal, allowing multiple nulls in the column. The NULLS NOT DISTINCT option modifies this and causes the index to treat nulls as equal. A multicolumn unique index will only reject cases where all indexed columns are equal in multiple rows.

PostgreSQL automatically creates a unique index when a unique constraint or primary key is defined for a table. The index covers the columns that make up the primary key or unique constraint (a multicolumn index, if appropriate), and is the mechanism that enforces the constraint.

[!NOTE] There's no need to manually create indexes on unique columns; doing so would just duplicate the automatically-created index.

Indexes on Expressions

index

on expressions

An index column need not be just a column of the underlying table, but can be a function or scalar expression computed from one or more columns of the table. This feature is useful to obtain fast access to tables based on the results of computations.

For example, a common way to do case-insensitive comparisons is to use the lower function:

SELECT * FROM test1 WHERE lower(col1) = 'value';

This query can use an index if one has been defined on the result of the lower(col1) function:

CREATE INDEX test1_lower_col1_idx ON test1 (lower(col1));

If we were to declare this index UNIQUE, it would prevent creation of rows whose col1 values differ only in case, as well as rows whose col1 values are actually identical. Thus, indexes on expressions can be used to enforce constraints that are not definable as simple unique constraints.

As another example, if one often does queries like:

SELECT * FROM people WHERE (first_name || ' ' || last_name) = 'John Smith';

then it might be worth creating an index like this:

CREATE INDEX people_names ON people ((first_name || ' ' || last_name));

The syntax of the CREATE INDEX command normally requires writing parentheses around index expressions, as shown in the second example. The parentheses can be omitted when the expression is just a function call, as in the first example.

Index expressions are relatively expensive to maintain, because the derived expression(s) must be computed for each row insertion and non-HOT update. However, the index expressions are not recomputed during an indexed search, since they are already stored in the index. In both examples above, the system sees the query as just WHERE indexedcolumn = 'constant' and so the speed of the search is equivalent to any other simple index query. Thus, indexes on expressions are useful when retrieval speed is more important than insertion and update speed.

Partial Indexes

index

partial

A partial index is an index built over a subset of a table; the subset is defined by a conditional expression (called the predicate of the partial index). The index contains entries only for those table rows that satisfy the predicate. Partial indexes are a specialized feature, but there are several situations in which they are useful.

One major reason for using a partial index is to avoid indexing common values. Since a query searching for a common value (one that accounts for more than a few percent of all the table rows) will not use the index anyway, there is no point in keeping those rows in the index at all. This reduces the size of the index, which will speed up those queries that do use the index. It will also speed up many table update operations because the index does not need to be updated in all cases. example_title shows a possible application of this idea.

Setting up a Partial Index to Exclude Common Values

Suppose you are storing web server access logs in a database. Most accesses originate from the IP address range of your organization but some are from elsewhere (say, employees on dial-up connections). If your searches by IP are primarily for outside accesses, you probably do not need to index the IP range that corresponds to your organization's subnet.

Assume a table like this:

CREATE TABLE access_log (
    url varchar,
    client_ip inet,
    ...
);

To create a partial index that suits our example, use a command such as this:

CREATE INDEX access_log_client_ip_ix ON access_log (client_ip)
WHERE NOT (client_ip > inet '192.168.100.0' AND
           client_ip < inet '192.168.100.255');

A typical query that can use this index would be:

SELECT *
FROM access_log
WHERE url = '/index.html' AND client_ip = inet '212.78.10.32';

Here the query's IP address is covered by the partial index. The following query cannot use the partial index, as it uses an IP address that is excluded from the index:

SELECT *
FROM access_log
WHERE url = '/index.html' AND client_ip = inet '192.168.100.23';

Observe that this kind of partial index requires that the common values be predetermined, so such partial indexes are best used for data distributions that do not change. Such indexes can be recreated occasionally to adjust for new data distributions, but this adds maintenance effort.

Another possible use for a partial index is to exclude values from the index that the typical query workload is not interested in; this is shown in example_title. This results in the same advantages as listed above, but it prevents the “uninteresting” values from being accessed via that index, even if an index scan might be profitable in that case. Obviously, setting up partial indexes for this kind of scenario will require a lot of care and experimentation.

Setting up a Partial Index to Exclude Uninteresting Values

If you have a table that contains both billed and unbilled orders, where the unbilled orders take up a small fraction of the total table and yet those are the most-accessed rows, you can improve performance by creating an index on just the unbilled rows. The command to create the index would look like this:

CREATE INDEX orders_unbilled_index ON orders (order_nr)
    WHERE billed IS NOT TRUE;

A possible query to use this index would be:

SELECT * FROM orders WHERE billed IS NOT TRUE AND order_nr < 10000;

However, the index can also be used in queries that do not involve order_nr at all, e.g.:

SELECT * FROM orders WHERE billed IS NOT TRUE AND amount > 5000.00;

This is not as efficient as a partial index on the amount column would be, since the system has to scan the entire index. Yet, if there are relatively few unbilled orders, using this partial index just to find the unbilled orders could be a win.

Note that this query cannot use this index:

SELECT * FROM orders WHERE order_nr = 3501;

The order 3501 might be among the billed or unbilled orders.

example_title also illustrates that the indexed column and the column used in the predicate do not need to match. PostgreSQL supports partial indexes with arbitrary predicates, so long as only columns of the table being indexed are involved. However, keep in mind that the predicate must match the conditions used in the queries that are supposed to benefit from the index. To be precise, a partial index can be used in a query only if the system can recognize that the WHERE condition of the query mathematically implies the predicate of the index. PostgreSQL does not have a sophisticated theorem prover that can recognize mathematically equivalent expressions that are written in different forms. (Not only is such a general theorem prover extremely difficult to create, it would probably be too slow to be of any real use.) The system can recognize simple inequality implications, for example “x < 1” implies “x < 2”; otherwise the predicate condition must exactly match part of the query's WHERE condition or the index will not be recognized as usable. Matching takes place at query planning time, not at run time. As a result, parameterized query clauses do not work with a partial index. For example a prepared query with a parameter might specify “x < ?” which will never imply “x < 2” for all possible values of the parameter.

A third possible use for partial indexes does not require the index to be used in queries at all. The idea here is to create a unique index over a subset of a table, as in example_title. This enforces uniqueness among the rows that satisfy the index predicate, without constraining those that do not.

Setting up a Partial Unique Index

Suppose that we have a table describing test outcomes. We wish to ensure that there is only one “successful” entry for a given subject and target combination, but there might be any number of “unsuccessful” entries. Here is one way to do it:

CREATE TABLE tests (
    subject text,
    target text,
    success boolean,
    ...
);

CREATE UNIQUE INDEX tests_success_constraint ON tests (subject, target)
    WHERE success;

This is a particularly efficient approach when there are few successful tests and many unsuccessful ones. It is also possible to allow only one null in a column by creating a unique partial index with an IS NULL restriction.

Finally, a partial index can also be used to override the system's query plan choices. Also, data sets with peculiar distributions might cause the system to use an index when it really should not. In that case the index can be set up so that it is not available for the offending query. Normally, PostgreSQL makes reasonable choices about index usage (e.g., it avoids them when retrieving common values, so the earlier example really only saves index size, it is not required to avoid index usage), and grossly incorrect plan choices are cause for a bug report.

Keep in mind that setting up a partial index indicates that you know at least as much as the query planner knows, in particular you know when an index might be profitable. Forming this knowledge requires experience and understanding of how indexes in PostgreSQL work. In most cases, the advantage of a partial index over a regular index will be minimal. There are cases where they are quite counterproductive, as in example_title.

Do Not Use Partial Indexes as a Substitute for Partitioning

You might be tempted to create a large set of non-overlapping partial indexes, for example

CREATE INDEX mytable_cat_1 ON mytable (data) WHERE category = 1;
CREATE INDEX mytable_cat_2 ON mytable (data) WHERE category = 2;
CREATE INDEX mytable_cat_3 ON mytable (data) WHERE category = 3;
...
CREATE INDEX mytable_cat_N ON mytable (data) WHERE category = N;

This is a bad idea! Almost certainly, you'll be better off with a single non-partial index, declared like

CREATE INDEX mytable_cat_data ON mytable (category, data);

(Put the category column first, for the reasons described in Multicolumn Indexes.) While a search in this larger index might have to descend through a couple more tree levels than a search in a smaller index, that's almost certainly going to be cheaper than the planner effort needed to select the appropriate one of the partial indexes. The core of the problem is that the system does not understand the relationship among the partial indexes, and will laboriously test each one to see if it's applicable to the current query.

If your table is large enough that a single index really is a bad idea, you should look into using partitioning instead (see ???). With that mechanism, the system does understand that the tables and indexes are non-overlapping, so far better performance is possible.

More information about partial indexes can be found in ???, ???, and ???.

Index-Only Scans and Covering Indexes

index

index-only scans

index-only scan

index

covering

covering index

All indexes in PostgreSQL are secondary indexes, meaning that each index is stored separately from the table's main data area (which is called the table's heap in PostgreSQL terminology). This means that in an ordinary index scan, each row retrieval requires fetching data from both the index and the heap. Furthermore, while the index entries that match a given indexable WHERE condition are usually close together in the index, the table rows they reference might be anywhere in the heap. The heap-access portion of an index scan thus involves a lot of random access into the heap, which can be slow, particularly on traditional rotating media. (As described in Combining Multiple Indexes, bitmap scans try to alleviate this cost by doing the heap accesses in sorted order, but that only goes so far.)

To solve this performance problem, PostgreSQL supports index-only scans, which can answer queries from an index alone without any heap access. The basic idea is to return values directly out of each index entry instead of consulting the associated heap entry. There are two fundamental restrictions on when this method can be used:

  1. The index type must support index-only scans. B-tree indexes always do. GiST and SP-GiST indexes support index-only scans for some operator classes but not others. Other index types have no support. The underlying requirement is that the index must physically store, or else be able to reconstruct, the original data value for each index entry. As a counterexample, GIN indexes cannot support index-only scans because each index entry typically holds only part of the original data value.

  2. The query must reference only columns stored in the index. For example, given an index on columns x and y of a table that also has a column z, these queries could use index-only scans:

    SELECT x, y FROM tab WHERE x = 'key';
SELECT x FROM tab WHERE x = 'key' AND y < 42;

    but these queries could not:

    SELECT x, z FROM tab WHERE x = 'key';
SELECT x FROM tab WHERE x = 'key' AND z < 42;

    (Expression indexes and partial indexes complicate this rule, as discussed below.)

If these two fundamental requirements are met, then all the data values required by the query are available from the index, so an index-only scan is physically possible. But there is an additional requirement for any table scan in PostgreSQL: it must verify that each retrieved row be “visible” to the query's MVCC snapshot, as discussed in ???. Visibility information is not stored in index entries, only in heap entries; so at first glance it would seem that every row retrieval would require a heap access anyway. And this is indeed the case, if the table row has been modified recently. However, for seldom-changing data there is a way around this problem. PostgreSQL tracks, for each page in a table's heap, whether all rows stored in that page are old enough to be visible to all current and future transactions. This information is stored in a bit in the table's visibility map. An index-only scan, after finding a candidate index entry, checks the visibility map bit for the corresponding heap page. If it's set, the row is known visible and so the data can be returned with no further work. If it's not set, the heap entry must be visited to find out whether it's visible, so no performance advantage is gained over a standard index scan. Even in the successful case, this approach trades visibility map accesses for heap accesses; but since the visibility map is four orders of magnitude smaller than the heap it describes, far less physical I/O is needed to access it. In most situations the visibility map remains cached in memory all the time.

In short, while an index-only scan is possible given the two fundamental requirements, it will be a win only if a significant fraction of the table's heap pages have their all-visible map bits set. But tables in which a large fraction of the rows are unchanging are common enough to make this type of scan very useful in practice.

To make effective use of the index-only scan feature, you might choose to create a covering index, which is an index specifically designed to include the columns needed by a particular type of query that you run frequently. Since queries typically need to retrieve more columns than just the ones they search on, PostgreSQL allows you to create an index in which some columns are just “payload” and are not part of the search key. This is done by adding an INCLUDE clause listing the extra columns. For example, if you commonly run queries like

SELECT y FROM tab WHERE x = 'key';

the traditional approach to speeding up such queries would be to create an index on x only. However, an index defined as

CREATE INDEX tab_x_y ON tab(x) INCLUDE (y);

could handle these queries as index-only scans, because y can be obtained from the index without visiting the heap.

Because column y is not part of the index's search key, it does not have to be of a data type that the index can handle; it's merely stored in the index and is not interpreted by the index machinery. Also, if the index is a unique index, that is

CREATE UNIQUE INDEX tab_x_y ON tab(x) INCLUDE (y);

the uniqueness condition applies to just column x, not to the combination of x and y. (An INCLUDE clause can also be written in UNIQUE and PRIMARY KEY constraints, providing alternative syntax for setting up an index like this.)

It's wise to be conservative about adding non-key payload columns to an index, especially wide columns. If an index tuple exceeds the maximum size allowed for the index type, data insertion will fail. In any case, non-key columns duplicate data from the index's table and bloat the size of the index, thus potentially slowing searches. And remember that there is little point in including payload columns in an index unless the table changes slowly enough that an index-only scan is likely to not need to access the heap. If the heap tuple must be visited anyway, it costs nothing more to get the column's value from there. Other restrictions are that expressions are not currently supported as included columns, and that only B-tree, GiST and SP-GiST indexes currently support included columns.

Before PostgreSQL had the INCLUDE feature, people sometimes made covering indexes by writing the payload columns as ordinary index columns, that is writing

CREATE INDEX tab_x_y ON tab(x, y);

even though they had no intention of ever using y as part of a WHERE clause. This works fine as long as the extra columns are trailing columns; making them be leading columns is unwise for the reasons explained in Multicolumn Indexes. However, this method doesn't support the case where you want the index to enforce uniqueness on the key column(s).

Suffix truncation always removes non-key columns from upper B-Tree levels. As payload columns, they are never used to guide index scans. The truncation process also removes one or more trailing key column(s) when the remaining prefix of key column(s) happens to be sufficient to describe tuples on the lowest B-Tree level. In practice, covering indexes without an INCLUDE clause often avoid storing columns that are effectively payload in the upper levels. However, explicitly defining payload columns as non-key columns reliably keeps the tuples in upper levels small.

In principle, index-only scans can be used with expression indexes. For example, given an index on f(x) where x is a table column, it should be possible to execute

SELECT f(x) FROM tab WHERE f(x) < 1;

as an index-only scan; and this is very attractive if f() is an expensive-to-compute function. However, PostgreSQL's planner is currently not very smart about such cases. It considers a query to be potentially executable by index-only scan only when all columns needed by the query are available from the index. In this example, x is not needed except in the context f(x), but the planner does not notice that and concludes that an index-only scan is not possible. If an index-only scan seems sufficiently worthwhile, this can be worked around by adding x as an included column, for example

CREATE INDEX tab_f_x ON tab (f(x)) INCLUDE (x);

An additional caveat, if the goal is to avoid recalculating f(x), is that the planner won't necessarily match uses of f(x) that aren't in indexable WHERE clauses to the index column. It will usually get this right in simple queries such as shown above, but not in queries that involve joins. These deficiencies may be remedied in future versions of PostgreSQL.

Partial indexes also have interesting interactions with index-only scans. Consider the partial index shown in example_title:

CREATE UNIQUE INDEX tests_success_constraint ON tests (subject, target)
    WHERE success;

In principle, we could do an index-only scan on this index to satisfy a query like

SELECT target FROM tests WHERE subject = 'some-subject' AND success;

But there's a problem: the WHERE clause refers to success which is not available as a result column of the index. Nonetheless, an index-only scan is possible because the plan does not need to recheck that part of the WHERE clause at run time: all entries found in the index necessarily have success = true so this need not be explicitly checked in the plan. PostgreSQL versions 9.6 and later will recognize such cases and allow index-only scans to be generated, but older versions will not.

Operator Classes and Operator Families

operator class

operator family

An index definition can specify an operator class for each column of an index. CREATE INDEX <name> ON <table> (<column> <opclass> [ ( <opclass_options> ) ] [<sort options>] [, ...]); The operator class identifies the operators to be used by the index for that column. For example, a B-tree index on the type int4 would use the int4_ops class; this operator class includes comparison functions for values of type int4. In practice the default operator class for the column's data type is usually sufficient. The main reason for having operator classes is that for some data types, there could be more than one meaningful index behavior. For example, we might want to sort a complex-number data type either by absolute value or by real part. We could do this by defining two operator classes for the data type and then selecting the proper class when making an index. The operator class determines the basic sort ordering (which can then be modified by adding sort options COLLATE, ASC/DESC and/or NULLS FIRST/NULLS LAST).

There are also some built-in operator classes besides the default ones:

The following query shows all defined operator classes:

SELECT am.amname AS index_method,
       opc.opcname AS opclass_name,
       opc.opcintype::regtype AS indexed_type,
       opc.opcdefault AS is_default
    FROM pg_am am, pg_opclass opc
    WHERE opc.opcmethod = am.oid
    ORDER BY index_method, opclass_name;

An operator class is actually just a subset of a larger structure called an operator family. In cases where several data types have similar behaviors, it is frequently useful to define cross-data-type operators and allow these to work with indexes. To do this, the operator classes for each of the types must be grouped into the same operator family. The cross-type operators are members of the family, but are not associated with any single class within the family.

This expanded version of the previous query shows the operator family each operator class belongs to:

SELECT am.amname AS index_method,
       opc.opcname AS opclass_name,
       opf.opfname AS opfamily_name,
       opc.opcintype::regtype AS indexed_type,
       opc.opcdefault AS is_default
    FROM pg_am am, pg_opclass opc, pg_opfamily opf
    WHERE opc.opcmethod = am.oid AND
          opc.opcfamily = opf.oid
    ORDER BY index_method, opclass_name;

This query shows all defined operator families and all the operators included in each family:

SELECT am.amname AS index_method,
       opf.opfname AS opfamily_name,
       amop.amopopr::regoperator AS opfamily_operator
    FROM pg_am am, pg_opfamily opf, pg_amop amop
    WHERE opf.opfmethod = am.oid AND
          amop.amopfamily = opf.oid
    ORDER BY index_method, opfamily_name, opfamily_operator;

[!TIP] ??? has commands \dAc, \dAf, and \dAo, which provide slightly more sophisticated versions of these queries.

Indexes and Collations

An index can support only one collation per index column. If multiple collations are of interest, multiple indexes may be needed.

Consider these statements:

CREATE TABLE test1c (
    id integer,
    content varchar COLLATE "x"
);

CREATE INDEX test1c_content_index ON test1c (content);

The index automatically uses the collation of the underlying column. So a query of the form

SELECT * FROM test1c WHERE content > constant;

could use the index, because the comparison will by default use the collation of the column. However, this index cannot accelerate queries that involve some other collation. So if queries of the form, say,

SELECT * FROM test1c WHERE content > constant COLLATE "y";

are also of interest, an additional index could be created that supports the "y" collation, like this:

CREATE INDEX test1c_content_y_index ON test1c (content COLLATE "y");

Examining Index Usage

index

examining usage

Although indexes in PostgreSQL do not need maintenance or tuning, it is still important to check which indexes are actually used by the real-life query workload. Examining index usage for an individual query is done with the ??? command; its application for this purpose is illustrated in ???. It is also possible to gather overall statistics about index usage in a running server, as described in ???.

It is difficult to formulate a general procedure for determining which indexes to create. There are a number of typical cases that have been shown in the examples throughout the previous sections. A good deal of experimentation is often necessary. The rest of this section gives some tips for that:

legal

2026

1996–2026

The PostgreSQL Global Development Group

PostgreSQL Database Management System (also known as Postgres, formerly known as Postgres95)

Permission to use, copy, modify, and distribute this software and its documentation for any purpose, without fee, and without a written agreement is hereby granted, provided that the above copyright notice and this paragraph and the following two paragraphs appear in all copies.

IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN “AS-IS” BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.

Routine Database Maintenance Tasks

maintenance

routine maintenance

PostgreSQL, like any database software, requires that certain tasks be performed regularly to achieve optimum performance. The tasks discussed here are required, but they are repetitive in nature and can easily be automated using standard tools such as cron scripts or Windows' Task Scheduler. It is the database administrator's responsibility to set up appropriate scripts, and to check that they execute successfully.

One obvious maintenance task is the creation of backup copies of the data on a regular schedule. Without a recent backup, you have no chance of recovery after a catastrophe (disk failure, fire, mistakenly dropping a critical table, etc.). The backup and recovery mechanisms available in PostgreSQL are discussed at length in ???.

The other main category of maintenance task is periodic “vacuuming” of the database. This activity is discussed in Routine Vacuuming. Closely related to this is updating the statistics that will be used by the query planner, as discussed in Updating Planner Statistics.

Another task that might need periodic attention is log file management. This is discussed in Log File Maintenance.

check_postgres is available for monitoring database health and reporting unusual conditions. check_postgres integrates with Nagios and MRTG, but can be run standalone too.

PostgreSQL is low-maintenance compared to some other database management systems. Nonetheless, appropriate attention to these tasks will go far towards ensuring a pleasant and productive experience with the system.

Routine Vacuuming

vacuum

PostgreSQL databases require periodic maintenance known as vacuuming. For many installations, it is sufficient to let vacuuming be performed by the autovacuum daemon, which is described in The Autovacuum Daemon. You might need to adjust the autovacuuming parameters described there to obtain best results for your situation. Some database administrators will want to supplement or replace the daemon's activities with manually-managed VACUUM commands, which typically are executed according to a schedule by cron or Task Scheduler scripts. To set up manually-managed vacuuming properly, it is essential to understand the issues discussed in the next few subsections. Administrators who rely on autovacuuming may still wish to skim this material to help them understand and adjust autovacuuming.

Vacuuming Basics

PostgreSQL's VACUUM command has to process each table on a regular basis for several reasons:

  1. To recover or reuse disk space occupied by updated or deleted rows.

  2. To update data statistics used by the PostgreSQL query planner.

  3. To update the visibility map, which speeds up index-only scans.

  4. To protect against loss of very old data due to transaction ID wraparound or multixact ID wraparound.

Each of these reasons dictates performing VACUUM operations of varying frequency and scope, as explained in the following subsections.

There are two variants of VACUUM: standard VACUUM and VACUUM FULL. VACUUM FULL can reclaim more disk space but runs much more slowly. Also, the standard form of VACUUM can run in parallel with production database operations. (Commands such as SELECT, INSERT, UPDATE, and DELETE will continue to function normally, though you will not be able to modify the definition of a table with commands such as ALTER TABLE while it is being vacuumed.) VACUUM FULL requires an ACCESS EXCLUSIVE lock on the table it is working on, and therefore cannot be done in parallel with other use of the table. Generally, therefore, administrators should strive to use standard VACUUM and avoid VACUUM FULL.

VACUUM creates a substantial amount of I/O traffic, which can cause poor performance for other active sessions. There are configuration parameters that can be adjusted to reduce the performance impact of background vacuuming — see ???.

Recovering Disk Space

disk space

In PostgreSQL, an UPDATE or DELETE of a row does not immediately remove the old version of the row. This approach is necessary to gain the benefits of multiversion concurrency control (MVCC, see ???): the row version must not be deleted while it is still potentially visible to other transactions. But eventually, an outdated or deleted row version is no longer of interest to any transaction. The space it occupies must then be reclaimed for reuse by new rows, to avoid unbounded growth of disk space requirements. This is done by running VACUUM.

The standard form of VACUUM removes dead row versions in tables and indexes and marks the space available for future reuse. However, it will not return the space to the operating system, except in the special case where one or more pages at the end of a table become entirely free and an exclusive table lock can be easily obtained. In contrast, table-rewriting commands such as VACUUM FULL, CLUSTER, and REPACK actively compact tables by writing a complete new version of the table file without the dead space left by old row versions. This minimizes the size of the table, but can take a long time. It also requires extra disk space for the new copy of the table, until the operation completes.

The usual goal of routine vacuuming is to do standard VACUUMs often enough to avoid needing VACUUM FULL. The autovacuum daemon attempts to work this way, and in fact will never issue VACUUM FULL. In this approach, the idea is not to keep tables at their minimum size, but to maintain steady-state usage of disk space: each table occupies space equivalent to its minimum size plus however much space gets used up between vacuum runs. Although table-rewriting compaction operations such as VACUUM FULL, CLUSTER, and REPACK can be used to shrink a table back to its minimum size and return the disk space to the operating system, there is not much point in this if the table will just grow again in the future. Thus, moderately-frequent standard VACUUM runs are a better approach than infrequent use of such operations for maintaining heavily-updated tables.

Some administrators prefer to schedule vacuuming themselves, for example doing all the work at night when load is low. The difficulty with doing vacuuming according to a fixed schedule is that if a table has an unexpected spike in update activity, it may get bloated to the point that a table-rewriting compaction operation such as VACUUM FULL or REPACK may be needed to reclaim space. Using the autovacuum daemon alleviates this problem, since the daemon schedules vacuuming dynamically in response to update activity. It is unwise to disable the daemon completely unless you have an extremely predictable workload. One possible compromise is to set the daemon's parameters so that it will only react to unusually heavy update activity, thus keeping things from getting out of hand, while scheduled VACUUMs are expected to do the bulk of the work when the load is typical.

For those not using autovacuum, a typical approach is to schedule a database-wide VACUUM once a day during a low-usage period, supplemented by more frequent vacuuming of heavily-updated tables as necessary. (Some installations with extremely high update rates vacuum their busiest tables as often as once every few minutes.) If you have multiple databases in a cluster, don't forget to VACUUM each one; the program ??? might be helpful.

[!TIP] Plain VACUUM may not be satisfactory when a table contains large numbers of dead row versions as a result of massive update or delete activity. If you have such a table and you need to reclaim the excess disk space it occupies, you will need to use VACUUM FULL, CLUSTER, REPACK, or one of the table-rewriting variants of ALTER TABLE. These commands rewrite an entire new copy of the table and build new indexes for it. All these options require an ACCESS EXCLUSIVE lock, except that REPACK can be run with CONCURRENTLY, in which case the ACCESS EXCLUSIVE lock is held only while swapping the table and index files. Note that they also temporarily use extra disk space approximately equal to the size of the table, since the old copies of the table and indexes can't be released until the new ones are complete.

[!TIP] If you have a table whose entire contents are deleted on a periodic basis, consider doing it with TRUNCATE rather than using DELETE followed by VACUUM. TRUNCATE removes the entire content of the table immediately, without requiring a subsequent VACUUM or VACUUM FULL to reclaim the now-unused disk space. The disadvantage is that strict MVCC semantics are violated.

Updating Planner Statistics

statistics

of the planner

ANALYZE

The PostgreSQL query planner relies on statistical information about the contents of tables in order to generate good plans for queries. These statistics are gathered by the ANALYZE command, which can be invoked by itself or as an optional step in VACUUM. It is important to have reasonably accurate statistics, otherwise poor choices of plans might degrade database performance.

The autovacuum daemon, if enabled, will automatically issue ANALYZE commands whenever the content of a table has changed sufficiently. However, administrators might prefer to rely on manually-scheduled ANALYZE operations, particularly if it is known that update activity on a table will not affect the statistics of “interesting” columns. The daemon schedules ANALYZE strictly as a function of the number of rows inserted or updated; it has no knowledge of whether that will lead to meaningful statistical changes.

Tuples changed in partitions and inheritance children do not trigger analyze on the parent table. If the parent table is empty or rarely changed, it may never be processed by autovacuum, and the statistics for the inheritance tree as a whole won't be collected. It is necessary to run ANALYZE on the parent table manually in order to keep the statistics up to date.

As with vacuuming for space recovery, frequent updates of statistics are more useful for heavily-updated tables than for seldom-updated ones. But even for a heavily-updated table, there might be no need for statistics updates if the statistical distribution of the data is not changing much. A simple rule of thumb is to think about how much the minimum and maximum values of the columns in the table change. For example, a timestamp column that contains the time of row update will have a constantly-increasing maximum value as rows are added and updated; such a column will probably need more frequent statistics updates than, say, a column containing URLs for pages accessed on a website. The URL column might receive changes just as often, but the statistical distribution of its values probably changes relatively slowly.

It is possible to run ANALYZE on specific tables and even just specific columns of a table, so the flexibility exists to update some statistics more frequently than others if your application requires it. In practice, however, it is usually best to just analyze the entire database, because it is a fast operation. ANALYZE uses a statistically random sampling of the rows of a table rather than reading every single row.

[!TIP] Although per-column tweaking of ANALYZE frequency might not be very productive, you might find it worthwhile to do per-column adjustment of the level of detail of the statistics collected by ANALYZE. Columns that are heavily used in WHERE clauses and have highly irregular data distributions might require a finer-grain data histogram than other columns. See ALTER TABLE SET STATISTICS, or change the database-wide default using the ??? configuration parameter.

Also, by default there is limited information available about the selectivity of functions. However, if you create a statistics object or an expression index that uses a function call, useful statistics will be gathered about the function, which can greatly improve query plans that use the expression index.

[!TIP] The autovacuum daemon does not issue ANALYZE commands for foreign tables, since it has no means of determining how often that might be useful. If your queries require statistics on foreign tables for proper planning, it's a good idea to run manually-managed ANALYZE commands on those tables on a suitable schedule.

[!TIP] The autovacuum daemon does not issue ANALYZE commands for partitioned tables. Inheritance parents will only be analyzed if the parent itself is changed - changes to child tables do not trigger autoanalyze on the parent table. If your queries require statistics on parent tables for proper planning, it is necessary to periodically run a manual ANALYZE on those tables to keep the statistics up to date.

Updating the Visibility Map

Vacuum maintains a visibility map for each table to keep track of which pages contain only tuples that are known to be visible to all active transactions (and all future transactions, until the page is again modified). This has two purposes. First, vacuum itself can skip such pages on the next run, since there is nothing to clean up.

Second, it allows PostgreSQL to answer some queries using only the index, without reference to the underlying table. Since PostgreSQL indexes don't contain tuple visibility information, a normal index scan fetches the heap tuple for each matching index entry, to check whether it should be seen by the current transaction. An index-only scan, on the other hand, checks the visibility map first. If it's known that all tuples on the page are visible, the heap fetch can be skipped. This is most useful on large data sets where the visibility map can prevent disk accesses. The visibility map is vastly smaller than the heap, so it can easily be cached even when the heap is very large.

Preventing Transaction ID Wraparound Failures

transaction ID

wraparound

wraparound

of transaction IDs

PostgreSQL's MVCC transaction semantics depend on being able to compare transaction ID (XID) numbers: a row version with an insertion XID greater than the current transaction's XID is “in the future” and should not be visible to the current transaction. But since transaction IDs have limited size (32 bits) a cluster that runs for a long time (more than 4 billion transactions) would suffer transaction ID wraparound: the XID counter wraps around to zero, and all of a sudden transactions that were in the past appear to be in the future — which means their output become invisible. In short, catastrophic data loss. (Actually the data is still there, but that's cold comfort if you cannot get at it.) To avoid this, it is necessary to vacuum every table in every database at least once every two billion transactions.

The reason that periodic vacuuming solves the problem is that VACUUM will mark rows as frozen, indicating that they were inserted by a transaction that committed sufficiently far in the past that the effects of the inserting transaction are certain to be visible to all current and future transactions. Normal XIDs are compared using modulo-232 arithmetic. This means that for every normal XID, there are two billion XIDs that are “older” and two billion that are “newer”; another way to say it is that the normal XID space is circular with no endpoint. Therefore, once a row version has been created with a particular normal XID, the row version will appear to be “in the past” for the next two billion transactions, no matter which normal XID we are talking about. If the row version still exists after more than two billion transactions, it will suddenly appear to be in the future. To prevent this, PostgreSQL reserves a special XID, FrozenTransactionId, which does not follow the normal XID comparison rules and is always considered older than every normal XID. Frozen row versions are treated as if the inserting XID were FrozenTransactionId, so that they will appear to be “in the past” to all normal transactions regardless of wraparound issues, and so such row versions will be valid until deleted, no matter how long that is.

[!NOTE] In PostgreSQL versions before 9.4, freezing was implemented by actually replacing a row's insertion XID with FrozenTransactionId, which was visible in the row's xmin system column. Newer versions just set a flag bit, preserving the row's original xmin for possible forensic use. However, rows with xmin equal to FrozenTransactionId (2) may still be found in databases pg_upgrade'd from pre-9.4 versions.

Also, system catalogs may contain rows with xmin equal to BootstrapTransactionId (1), indicating that they were inserted during the first phase of initdb. Like FrozenTransactionId, this special XID is treated as older than every normal XID.

??? controls how old an XID value has to be before rows bearing that XID will be frozen. Increasing this setting may avoid unnecessary work if the rows that would otherwise be frozen will soon be modified again, but decreasing this setting increases the number of transactions that can elapse before the table must be vacuumed again.

VACUUM uses the visibility map to determine which pages of a table must be scanned. Normally, it will skip pages that don't have any dead row versions even if those pages might still have row versions with old XID values. Therefore, normal VACUUMs won't always freeze every old row version in the table. When that happens, VACUUM will eventually need to perform an aggressive vacuum, which will freeze all eligible unfrozen XID and MXID values, including those from all-visible but not all-frozen pages.

If a table is building up a backlog of all-visible but not all-frozen pages, a normal vacuum may choose to scan skippable pages in an effort to freeze them. Doing so decreases the number of pages the next aggressive vacuum must scan. These are referred to as eagerly scanned pages. Eager scanning can be tuned to attempt to freeze more all-visible pages by increasing ???. Even if eager scanning has kept the number of all-visible but not all-frozen pages to a minimum, most tables still require periodic aggressive vacuuming. However, any pages successfully eager frozen may be skipped during an aggressive vacuum, so eager freezing may minimize the overhead of aggressive vacuums.

??? controls when a table is aggressively vacuumed. All all-visible but not all-frozen pages are scanned if the number of transactions that have passed since the last such scan is greater than vacuum_freeze_table_age minus vacuum_freeze_min_age. Setting vacuum_freeze_table_age to 0 forces VACUUM to always use its aggressive strategy.

The maximum time that a table can go unvacuumed is two billion transactions minus the vacuum_freeze_min_age value at the time of the last aggressive vacuum. If it were to go unvacuumed for longer than that, data loss could result. To ensure that this does not happen, autovacuum is invoked on any table that might contain unfrozen rows with XIDs older than the age specified by the configuration parameter ???. (This will happen even if autovacuum is disabled.)

This implies that if a table is not otherwise vacuumed, autovacuum will be invoked on it approximately once every autovacuum_freeze_max_age minus vacuum_freeze_min_age transactions. For tables that are regularly vacuumed for space reclamation purposes, this is of little importance. However, for static tables (including tables that receive inserts, but no updates or deletes), there is no need to vacuum for space reclamation, so it can be useful to try to maximize the interval between forced autovacuums on very large static tables. Obviously one can do this either by increasing autovacuum_freeze_max_age or decreasing vacuum_freeze_min_age.

The effective maximum for vacuum_freeze_table_age is 0.95 * autovacuum_freeze_max_age; a setting higher than that will be capped to the maximum. A value higher than autovacuum_freeze_max_age wouldn't make sense because an anti-wraparound autovacuum would be triggered at that point anyway, and the 0.95 multiplier leaves some breathing room to run a manual VACUUM before that happens. As a rule of thumb, vacuum_freeze_table_age should be set to a value somewhat below autovacuum_freeze_max_age, leaving enough gap so that a regularly scheduled VACUUM or an autovacuum triggered by normal delete and update activity is run in that window. Setting it too close could lead to anti-wraparound autovacuums, even though the table was recently vacuumed to reclaim space, whereas lower values lead to more frequent aggressive vacuuming.

The sole disadvantage of increasing autovacuum_freeze_max_age (and vacuum_freeze_table_age along with it) is that the pg_xact and pg_commit_ts subdirectories of the database cluster will take more space, because it must store the commit status and (if track_commit_timestamp is enabled) timestamp of all transactions back to the autovacuum_freeze_max_age horizon. The commit status uses two bits per transaction, so if autovacuum_freeze_max_age is set to its maximum allowed value of two billion, pg_xact can be expected to grow to about half a gigabyte and pg_commit_ts to about 20GB. If this is trivial compared to your total database size, setting autovacuum_freeze_max_age to its maximum allowed value is recommended. Otherwise, set it depending on what you are willing to allow for pg_xact and pg_commit_ts storage. (The default, 200 million transactions, translates to about 50MB of pg_xact storage and about 2GB of pg_commit_ts storage.)

One disadvantage of decreasing vacuum_freeze_min_age is that it might cause VACUUM to do useless work: freezing a row version is a waste of time if the row is modified soon thereafter (causing it to acquire a new XID). So the setting should be large enough that rows are not frozen until they are unlikely to change any more.

To track the age of the oldest unfrozen XIDs in a database, VACUUM stores XID statistics in the system tables pg_class and pg_database. In particular, the relfrozenxid column of a table's pg_class row contains the oldest remaining unfrozen XID at the end of the most recent VACUUM that successfully advanced relfrozenxid (typically the most recent aggressive VACUUM). Similarly, the datfrozenxid column of a database's pg_database row is a lower bound on the unfrozen XIDs appearing in that database — it is just the minimum of the per-table relfrozenxid values within the database. A convenient way to examine this information is to execute queries such as:

SELECT c.oid::regclass AS table_name,
       greatest(age(c.relfrozenxid), age(t.relfrozenxid)) AS age
FROM pg_class c
LEFT JOIN pg_class t ON c.reltoastrelid = t.oid
WHERE c.relkind IN ('r', 'm');

SELECT datname, age(datfrozenxid) FROM pg_database;

The age column measures the number of transactions from the cutoff XID to the current transaction's XID.

[!TIP] When the VACUUM command's VERBOSE parameter is specified, VACUUM prints various statistics about the table. This includes information about how relfrozenxid and relminmxid advanced, and the number of newly frozen pages. The same details appear in the server log when autovacuum logging (controlled by ???) reports on a VACUUM operation executed by autovacuum.

While VACUUM scans mostly pages that have been modified since the last vacuum, it may also eagerly scan some all-visible but not all-frozen pages in an attempt to freeze them, but the relfrozenxid will only be advanced when every page of the table that might contain unfrozen XIDs is scanned. This happens when relfrozenxid is more than vacuum_freeze_table_age transactions old, when VACUUM's FREEZE option is used, or when all pages that are not already all-frozen happen to require vacuuming to remove dead row versions. When VACUUM scans every page in the table that is not already all-frozen, it should set age(relfrozenxid) to a value just a little more than the vacuum_freeze_min_age setting that was used (more by the number of transactions started since the VACUUM started). VACUUM will set relfrozenxid to the oldest XID that remains in the table, so it's possible that the final value will be much more recent than strictly required. If no relfrozenxid-advancing VACUUM is issued on the table until autovacuum_freeze_max_age is reached, an autovacuum will soon be forced for the table.

If for some reason autovacuum fails to clear old XIDs from a table, the system will begin to emit warning messages like this when the database's oldest XIDs reach one hundred million transactions from the wraparound point:

WARNING:  database "mydb" must be vacuumed within 99985967 transactions
DETAIL:  Approximately 4.66% of transaction ID space remains before wraparound.
HINT:  To avoid transaction ID assignment failures, execute a database-wide VACUUM in that database.

(A manual VACUUM should fix the problem, as suggested by the hint; but note that the VACUUM should be performed by a superuser, else it will fail to process system catalogs, which prevent it from being able to advance the database's datfrozenxid.) If these warnings are ignored, the system will refuse to assign new XIDs once there are fewer than three million transactions left until wraparound:

ERROR:  database is not accepting commands that assign new transaction IDs to avoid wraparound data loss in database "mydb"
HINT:  Execute a database-wide VACUUM in that database.

In this condition any transactions already in progress can continue, but only read-only transactions can be started. Operations that modify database records or truncate relations will fail. The VACUUM command can still be run normally. Note that, contrary to what was sometimes recommended in earlier releases, it is not necessary or desirable to stop the postmaster or enter single user-mode in order to restore normal operation. Instead, follow these steps:

  1. Resolve old prepared transactions. You can find these by checking pg_prepared_xacts for rows where age(transactionid) is large. Such transactions should be committed or rolled back.

  2. End long-running open transactions. You can find these by checking pg_stat_activity for rows where age(backend_xid) or age(backend_xmin) is large. Such transactions should be committed or rolled back, or the session can be terminated using pg_terminate_backend.

  3. Drop any old replication slots. Use pg_replication_slots to find slots where age(xmin) or age(catalog_xmin) is large. In many cases, such slots were created for replication to servers that no longer exist, or that have been down for a long time. If you drop a slot for a server that still exists and might still try to connect to that slot, that replica may need to be rebuilt.

  4. Execute VACUUM in the target database. A database-wide VACUUM is simplest; to reduce the time required, it is also possible to issue manual VACUUM commands on the tables where relminxid is oldest. Do not use VACUUM FULL in this scenario, because it requires an XID and will therefore fail, except in super-user mode, where it will instead consume an XID and thus increase the risk of transaction ID wraparound. Do not use VACUUM FREEZE either, because it will do more than the minimum amount of work required to restore normal operation.

  5. Once normal operation is restored, ensure that autovacuum is properly configured in the target database in order to avoid future problems.

[!NOTE] In earlier versions, it was sometimes necessary to stop the postmaster and VACUUM the database in a single-user mode. In typical scenarios, this is no longer necessary, and should be avoided whenever possible, since it involves taking the system down. It is also riskier, since it disables transaction ID wraparound safeguards that are designed to prevent data loss. The only reason to use single-user mode in this scenario is if you wish to TRUNCATE or DROP unneeded tables to avoid needing to VACUUM them. The three-million-transaction safety margin exists to let the administrator do this. See the ??? reference page for details about using single-user mode.

Multixacts and Wraparound

MultiXactId

wraparound

of multixact IDs

Multixact IDs (MXIDs) are used to support row locking by multiple transactions. Since there is only limited space in a tuple header to store lock information, that information is encoded as a “multiple transaction ID”, or multixact ID for short, whenever there is more than one transaction concurrently locking a row. Information about which transaction IDs are included in any particular multixact ID is stored separately in the pg_multixact subdirectory, and only the multixact ID appears in the xmax field in the tuple header. Like transaction IDs, multixact IDs are implemented as a 32-bit counter and corresponding storage, all of which requires careful aging management, storage cleanup, and wraparound handling. There is a separate storage area which holds the list of members in each multixact, which cannot wrap around but must also be managed to keep disk usage under control. The system function pg_get_multixact_members() described in ??? can be used to examine the transaction IDs associated with a multixact ID.

Whenever VACUUM scans any part of a table, it will replace any multixact ID it encounters which is older than ??? by a different value, which can be the zero value, a single transaction ID, or a newer multixact ID. For each table, pg_class.relminmxid stores the oldest possible multixact ID still appearing in any tuple of that table. If this value is older than ???, an aggressive vacuum is forced. As discussed in the previous section, an aggressive vacuum means that only those pages which are known to be all-frozen will be skipped. mxid_age() can be used on pg_class.relminmxid to find its age.

Aggressive VACUUMs, regardless of what causes them, are guaranteed to be able to advance the table's relminmxid. Eventually, as all tables in all databases are scanned and their oldest multixact values are advanced, on-disk storage for older multixacts can be removed.

As a safety device, an aggressive vacuum scan will occur for any table whose multixact-age is greater than ???. The aggressive scans will occur even if autovacuum is nominally disabled. Also, if the number of multixact member entries created exceeds approximately 2 billion entries (occupying roughly 10GB in the pg_multixact/members directory), aggressive vacuum scans will occur more often for all tables, starting with those that have the oldest multixact-age.

The pg_get_multixact_stats() function described in ??? provides a way to monitor multixact allocation and usage patterns in real time, for example:

=# SELECT *, pg_size_pretty(members_size) members_size_pretty
     FROM pg_catalog.pg_get_multixact_stats();
 num_mxids | num_members | members_size | oldest_multixact | members_size_pretty
-----------+-------------+--------------+------------------+---------------------
 311740299 |  2785241176 |  13926205880 |                2 | 13 GB
(1 row)

This output shows a system with significant multixact activity: about 312 million multixact IDs and about 2.8 billion member entries consuming 13 GB of storage space. A spike in num_mxids might indicate multiple sessions running UPDATE statements with foreign key checks, concurrent SELECT FOR SHARE operations, or frequent use of savepoints causing lock contention. If oldest_multixact value remains unchanged while num_members grows, it could indicate that long-running transactions are preventing cleanup, or autovacuum is not keeping up with the workload.

Similar to the XID case, if autovacuum fails to clear old MXIDs from a table, the system will begin to emit warning messages when the database's oldest MXIDs reach one hundred million transactions from the wraparound point. And, just as in the XID case, if these warnings are ignored, the system will refuse to generate new MXIDs once there are fewer than three million left until wraparound.

Normal operation when MXIDs are exhausted can be restored in much the same way as when XIDs are exhausted. Follow the same steps in the previous section, but with the following differences:

  1. Running transactions and prepared transactions can be ignored if there is no chance that they might appear in a multixact.

  2. Unlike transaction ID wraparound, replication slots do not directly hold back multixact cleanup. Dropping stale replication slots is therefore not usually relevant to resolving multixact ID wraparound problems.

  3. MXID information is not directly visible in system views such as pg_stat_activity; however, looking for old XIDs is still a good way of determining which transactions are causing MXID wraparound problems.

  4. XID exhaustion will block all write transactions, but MXID exhaustion will only block a subset of write transactions, specifically those that involve row locks that require an MXID.

The Autovacuum Daemon

autovacuum

general information

PostgreSQL has an optional but highly recommended feature called autovacuum, whose purpose is to automate the execution of VACUUM and ANALYZE commands. When enabled, autovacuum checks for tables that have had a large number of inserted, updated or deleted tuples. These checks use the statistics collection facility; therefore, autovacuum cannot be used unless ??? is set to true. In the default configuration, autovacuuming is enabled and the related configuration parameters are appropriately set.

The “autovacuum daemon” actually consists of multiple processes. There is a persistent daemon process, called the autovacuum launcher, which is in charge of starting autovacuum worker processes for all databases. The launcher will distribute the work across time, attempting to start one worker within each database every ??? seconds. (Therefore, if the installation has <N> databases, a new worker will be launched every autovacuum_naptime/<N> seconds.) A maximum of ??? worker processes are allowed to run at the same time. If there are more than autovacuum_max_workers databases to be processed, the next database will be processed as soon as the first worker finishes. Each worker process will check each table within its database and execute VACUUM and/or ANALYZE as needed. ??? and ??? can be set to monitor autovacuum workers' activity.

If several large tables all become eligible for vacuuming in a short amount of time, all autovacuum workers might become occupied with vacuuming those tables for a long period. This would result in other tables and databases not being vacuumed until a worker becomes available. There is no limit on how many workers might be in a single database, but workers do try to avoid repeating work that has already been done by other workers. Note that the number of running workers does not count towards ??? or ??? limits.

Tables whose relfrozenxid value is more than ??? transactions old are always vacuumed (this also applies to those tables whose freeze max age has been modified via storage parameters; see below). Otherwise, if the number of tuples obsoleted since the last VACUUM exceeds the “vacuum threshold”, the table is vacuumed. The vacuum threshold is defined as:

vacuum threshold = Minimum(vacuum max threshold, vacuum base threshold + vacuum scale factor * number of tuples)

where the vacuum max threshold is ???, the vacuum base threshold is ???, the vacuum scale factor is ???, and the number of tuples is pg_class.reltuples.

The table is also vacuumed if the number of tuples inserted since the last vacuum has exceeded the defined insert threshold, which is defined as:

vacuum insert threshold = vacuum base insert threshold + vacuum insert scale factor * number of tuples * percent of table not frozen

where the vacuum insert base threshold is ???, the vacuum insert scale factor is ???, the number of tuples is pg_class.reltuples, and the percent of the table not frozen is 1 - pg_class.relallfrozen / pg_class.relpages. Such vacuums may allow portions of the table to be marked as all visible and also allow tuples to be frozen, which can reduce the work required in subsequent vacuums. For tables which receive INSERT operations but no or almost no UPDATE/DELETE operations, it may be beneficial to lower the table's ??? as this may allow tuples to be frozen by earlier vacuums. The number of obsolete tuples and the number of inserted tuples are obtained from the cumulative statistics system; it is an eventually-consistent count updated by each UPDATE, DELETE and INSERT operation. If the relfrozenxid value of the table is more than vacuum_freeze_table_age transactions old, an aggressive vacuum is performed to freeze old tuples and advance relfrozenxid.

For analyze, a similar condition is used: the threshold, defined as:

analyze threshold = analyze base threshold + analyze scale factor * number of tuples

is compared to the total number of tuples inserted, updated, or deleted since the last ANALYZE.

Partitioned tables do not directly store tuples and consequently are not processed by autovacuum. (Autovacuum does process table partitions just like other tables.) Unfortunately, this means that autovacuum does not run ANALYZE on partitioned tables, and this can cause suboptimal plans for queries that reference partitioned table statistics. You can work around this problem by manually running ANALYZE on partitioned tables when they are first populated, and again whenever the distribution of data in their partitions changes significantly.

Temporary tables cannot be accessed by autovacuum. Therefore, appropriate vacuum and analyze operations should be performed via session SQL commands.

The default thresholds and scale factors are taken from postgresql.conf, but it is possible to override them (and many other autovacuum control parameters) on a per-table basis; see ??? for more information. If a setting has been changed via a table's storage parameters, that value is used when processing that table; otherwise the global settings are used. See ??? for more details on the global settings.

When multiple workers are running, the autovacuum cost delay parameters (see ???) are “balanced” among all the running workers, so that the total I/O impact on the system is the same regardless of the number of workers actually running. However, any workers processing tables whose per-table autovacuum_vacuum_cost_delay or autovacuum_vacuum_cost_limit storage parameters have been set are not considered in the balancing algorithm. Parallel workers launched for Parallel Vacuum are using the same cost delay parameters as the leader worker. If any of these parameters are changed in the leader worker, it will propagate the new parameter values to all of its parallel workers.

Autovacuum workers generally don't block other commands. If a process attempts to acquire a lock that conflicts with the SHARE UPDATE EXCLUSIVE lock held by autovacuum, lock acquisition will interrupt the autovacuum. For conflicting lock modes, see ???. However, if the autovacuum is running to prevent transaction ID wraparound (i.e., the autovacuum query name in the pg_stat_activity view ends with (to prevent wraparound) or the started_by column in the pg_stat_progress_vacuum view shows autovacuum_wraparound value), the autovacuum is not automatically interrupted.

[!WARNING] Regularly running commands that acquire locks conflicting with a SHARE UPDATE EXCLUSIVE lock (e.g., ANALYZE) can effectively prevent autovacuums from ever completing.

Autovacuum Prioritization

Autovacuum decides what to process in two steps: first it chooses a database, then it chooses the tables within that database. The autovacuum launcher process prioritizes databases at risk of transaction ID or multixact ID wraparound, else it chooses the database processed least recently. As an exception, it skips databases with no connections or no activity since the last statistics reset, unless at risk of wraparound.

Within a database, the autovacuum worker process builds a list of tables that require vacuum or analyze and sorts them using a scoring system. It scores each table by taking the maximum value of several component scores representing various criteria important to vacuum or analyze. Those components are as follows:

To revert to the prioritization strategy used before PostgreSQL 19 (i.e., the order the tables are listed in the pg_class system catalog), set all of the aforementioned "weight" parameters to 0.0. Otherwise, these "weight" parameters are multiplied to their respective component scores. For example, raising ??? to 2.0 effectively doubles the analyze component score.

The pg_stat_autovacuum_scores view shows the current scores of all tables in the current database.

Parallel Vacuum

VACUUM can perform index vacuuming and index cleanup phases in parallel using background workers (for the details of each vacuum phase, please refer to ???). The degree of parallelism is determined by the number of indexes on the relation that support parallel vacuum. For manual VACUUM, this is limited by the PARALLEL option if specified, which is further capped by ???. For autovacuum, it is limited by the table's ??? if specified, which is further capped by ??? parameter. Please note that it is not guaranteed that the number of parallel workers that was calculated will be used during execution. It is possible for a vacuum to run with fewer workers than specified, or even with no workers at all.

An index can participate in parallel vacuum if and only if the size of the index is more than ???. Only one worker can be used per index. So parallel workers are launched only when there are at least 2 indexes in the table. Workers for vacuum are launched before the start of each phase and exit at the end of the phase. These behaviors might change in a future release.

Routine Reindexing

reindex

In some situations it is worthwhile to rebuild indexes periodically with the ??? command or a series of individual rebuilding steps.

B-tree index pages that have become completely empty are reclaimed for re-use. However, there is still a possibility of inefficient use of space: if all but a few index keys on a page have been deleted, the page remains allocated. Therefore, a usage pattern in which most, but not all, keys in each range are eventually deleted will see poor use of space. For such usage patterns, periodic reindexing is recommended.

The potential for bloat in non-B-tree indexes has not been well researched. It is a good idea to periodically monitor the index's physical size when using any non-B-tree index type.

Also, for B-tree indexes, a freshly-constructed index is slightly faster to access than one that has been updated many times because logically adjacent pages are usually also physically adjacent in a newly built index. (This consideration does not apply to non-B-tree indexes.) It might be worthwhile to reindex periodically just to improve access speed.

??? can be used safely and easily in all cases. This command requires an ACCESS EXCLUSIVE lock by default, hence it is often preferable to execute it with its CONCURRENTLY option, which requires only a SHARE UPDATE EXCLUSIVE lock.

Log File Maintenance

server log

log file maintenance

It is a good idea to save the database server's log output somewhere, rather than just discarding it via /dev/null. The log output is invaluable when diagnosing problems.

[!NOTE] The server log can contain sensitive information and needs to be protected, no matter how or where it is stored, or the destination to which it is routed. For example, some DDL statements might contain plaintext passwords or other authentication details. Logged statements at the ERROR level might show the SQL source code for applications and might also contain some parts of data rows. Recording data, events and related information is the intended function of this facility, so this is not a leakage or a bug. Please ensure the server logs are visible only to appropriately authorized people.

Log output tends to be voluminous (especially at higher debug levels) so you won't want to save it indefinitely. You need to rotate the log files so that new log files are started and old ones removed after a reasonable period of time.

If you simply direct the stderr of postgres into a file, you will have log output, but the only way to truncate the log file is to stop and restart the server. This might be acceptable if you are using PostgreSQL in a development environment, but few production servers would find this behavior acceptable.

A better approach is to send the server's stderr output to some type of log rotation program. There is a built-in log rotation facility, which you can use by setting the configuration parameter logging_collector to true in postgresql.conf. The control parameters for this program are described in ???. You can also use this approach to capture the log data in machine readable CSV (comma-separated values) format.

Alternatively, you might prefer to use an external log rotation program if you have one that you are already using with other server software. For example, the rotatelogs tool included in the Apache distribution can be used with PostgreSQL. One way to do this is to pipe the server's stderr output to the desired program. If you start the server with pg_ctl, then stderr is already redirected to stdout, so you just need a pipe command, for example:

pg_ctl start | rotatelogs /var/log/pgsql_log 86400

You can combine these approaches by setting up logrotate to collect log files produced by PostgreSQL built-in logging collector. In this case, the logging collector defines the names and location of the log files, while logrotate periodically archives these files. When initiating log rotation, logrotate must ensure that the application sends further output to the new file. This is commonly done with a postrotate script that sends a SIGHUP signal to the application, which then reopens the log file. In PostgreSQL, you can run pg_ctl with the logrotate option instead. When the server receives this command, the server either switches to a new log file or reopens the existing file, depending on the logging configuration (see ???).

[!NOTE] When using static log file names, the server might fail to reopen the log file if the max open file limit is reached or a file table overflow occurs. In this case, log messages are sent to the old log file until a successful log rotation. If logrotate is configured to compress the log file and delete it, the server may lose the messages logged in this time frame. To avoid this issue, you can configure the logging collector to dynamically assign log file names and use a prerotate script to ignore open log files.

Another production-grade approach to managing log output is to send it to syslog and let syslog deal with file rotation. To do this, set the configuration parameter log_destination to syslog (to log to syslog only) in postgresql.conf. Then you can send a SIGHUP signal to the syslog daemon whenever you want to force it to start writing a new log file. If you want to automate log rotation, the logrotate program can be configured to work with log files from syslog.

On many systems, however, syslog is not very reliable, particularly with large log messages; it might truncate or drop messages just when you need them the most. Also, on Linux, syslog will flush each message to disk, yielding poor performance. (You can use a “-” at the start of the file name in the syslog configuration file to disable syncing.)

Note that all the solutions described above take care of starting new log files at configurable intervals, but they do not handle deletion of old, no-longer-useful log files. You will probably want to set up a batch job to periodically delete old log files. Another possibility is to configure the rotation program so that old log files are overwritten cyclically.

pgBadger is an external project that does sophisticated log file analysis. check_postgres provides Nagios alerts when important messages appear in the log files, as well as detection of many other extraordinary conditions.

Managing Databases

database

Every instance of a running PostgreSQL server manages one or more databases. Databases are therefore the topmost hierarchical level for organizing SQL objects (“database objects”). This chapter describes the properties of databases, and how to create, manage, and destroy them.

Overview

schema

A small number of objects, like role, database, and tablespace names, are defined at the cluster level and stored in the pg_global tablespace. Inside the cluster are multiple databases, which are isolated from each other but can access cluster-level objects. Inside each database are multiple schemas, which contain objects like tables and functions. So the full hierarchy is: cluster, database, schema, table (or some other kind of object, such as a function).

When connecting to the database server, a client must specify the database name in its connection request. It is not possible to access more than one database per connection. However, clients can open multiple connections to the same database, or different databases. Database-level security has two components: access control (see ???), managed at the connection level, and authorization control (see ???), managed via the grant system. Foreign data wrappers (see ???) allow for objects within one database to act as proxies for objects in other database or clusters. The older dblink module (see ???) provides a similar capability. By default, all users can connect to all databases using all connection methods.

If one PostgreSQL server cluster is planned to contain unrelated projects or users that should be, for the most part, unaware of each other, it is recommended to put them into separate databases and adjust authorizations and access controls accordingly. If the projects or users are interrelated, and thus should be able to use each other's resources, they should be put in the same database but probably into separate schemas; this provides a modular structure with namespace isolation and authorization control. More information about managing schemas is in ???.

While multiple databases can be created within a single cluster, it is advised to consider carefully whether the benefits outweigh the risks and limitations. In particular, the impact that having a shared WAL (see ???) has on backup and recovery options. While individual databases in the cluster are isolated when considered from the user's perspective, they are closely bound from the database administrator's point-of-view.

Databases are created with the CREATE DATABASE command (see Creating a Database) and destroyed with the DROP DATABASE command (see Destroying a Database). To determine the set of existing databases, examine the pg_database system catalog, for example SELECT datname FROM pg_database; The ??? program's \l meta-command and -l command-line option are also useful for listing the existing databases.

[!NOTE] The SQL standard calls databases “catalogs”, but there is no difference in practice.

Creating a Database

CREATE DATABASE

In order to create a database, the PostgreSQL server must be up and running (see ???).

Databases are created with the SQL command ???: CREATE DATABASE <name>; where <name> follows the usual rules for SQL identifiers. The current role automatically becomes the owner of the new database. It is the privilege of the owner of a database to remove it later (which also removes all the objects in it, even if they have a different owner).

The creation of databases is a restricted operation. See ??? for how to grant permission.

Since you need to be connected to the database server in order to execute the CREATE DATABASE command, the question remains how the first database at any given site can be created. The first database is always created by the initdb command when the data storage area is initialized. (See ???.) This database is called postgres. So to create the first “ordinary” database you can connect to postgres.

Two additional databases, template1 and template0, are also created during database cluster initialization. Whenever a new database is created within the cluster, template1 is essentially cloned. This means that any changes you make in template1 are propagated to all subsequently created databases. Because of this, avoid creating objects in template1 unless you want them propagated to every newly created database. template0 is meant as a pristine copy of the original contents of template1. It can be cloned instead of template1 when it is important to make a database without any such site-local additions. More details appear in Template Databases.

As a convenience, there is a program you can execute from the shell to create new databases, createdb. createdb <dbname> createdb does no magic. It connects to the postgres database and issues the CREATE DATABASE command, exactly as described above. The ??? reference page contains the invocation details. Note that createdb without any arguments will create a database with the current user name.

[!NOTE] ??? contains information about how to restrict who can connect to a given database.

Sometimes you want to create a database for someone else, and have them become the owner of the new database, so they can configure and manage it themselves. To achieve that, use one of the following commands:

CREATE DATABASE dbname OWNER rolename;

from the SQL environment, or:

createdb -O rolename dbname

from the shell. Only the superuser is allowed to create a database for someone else (that is, for a role you are not a member of).

Template Databases

CREATE DATABASE actually works by copying an existing database. By default, it copies the standard system database named template1. Thus that database is the “template” from which new databases are made. If you add objects to template1, these objects will be copied into subsequently created user databases. This behavior allows site-local modifications to the standard set of objects in databases. For example, if you install the procedural language PL/Perl in template1, it will automatically be available in user databases without any extra action being taken when those databases are created.

However, CREATE DATABASE does not copy database-level GRANT permissions attached to the source database. The new database has default database-level permissions.

There is a second standard system database named template0. This database contains the same data as the initial contents of template1, that is, only the standard objects predefined by your version of PostgreSQL. template0 should never be changed after the database cluster has been initialized. By instructing CREATE DATABASE to copy template0 instead of template1, you can create a “pristine” user database (one where no user-defined objects exist and where the system objects have not been altered) that contains none of the site-local additions in template1. This is particularly handy when restoring a pg_dump dump: the dump script should be restored in a pristine database to ensure that one recreates the correct contents of the dumped database, without conflicting with objects that might have been added to template1 later on.

Another common reason for copying template0 instead of template1 is that new encoding and locale settings can be specified when copying template0, whereas a copy of template1 must use the same settings it does. This is because template1 might contain encoding-specific or locale-specific data, while template0 is known not to.

To create a database by copying template0, use:

CREATE DATABASE dbname TEMPLATE template0;

from the SQL environment, or:

createdb -T template0 dbname

from the shell.

It is possible to create additional template databases, and indeed one can copy any database in a cluster by specifying its name as the template for CREATE DATABASE. It is important to understand, however, that this is not (yet) intended as a general-purpose “COPY DATABASE” facility. The principal limitation is that no other sessions can be connected to the source database while it is being copied. CREATE DATABASE will fail if any other connection exists when it starts; during the copy operation, new connections to the source database are prevented.

Two useful flags exist in pg_database for each database: the columns datistemplate and datallowconn. datistemplate can be set to indicate that a database is intended as a template for CREATE DATABASE. If this flag is set, the database can be cloned by any user with CREATEDB privileges; if it is not set, only superusers and the owner of the database can clone it. If datallowconn is false, then no new connections to that database will be allowed (but existing sessions are not terminated simply by setting the flag false). The template0 database is normally marked datallowconn = false to prevent its modification. Both template0 and template1 should always be marked with datistemplate = true.

[!NOTE] template1 and template0 do not have any special status beyond the fact that the name template1 is the default source database name for CREATE DATABASE. For example, one could drop template1 and recreate it from template0 without any ill effects. This course of action might be advisable if one has carelessly added a bunch of junk in template1. (To delete template1, it must have pg_database.datistemplate = false.)

The postgres database is also created when a database cluster is initialized. This database is meant as a default database for users and applications to connect to. It is simply a copy of template1 and can be dropped and recreated if necessary.

Database Configuration

Recall from ??? that the PostgreSQL server provides a large number of run-time configuration variables. You can set database-specific default values for many of these settings.

For example, if for some reason you want to disable the GEQO optimizer for a given database, you'd ordinarily have to either disable it for all databases or make sure that every connecting client is careful to issue SET geqo TO off. To make this setting the default within a particular database, you can execute the command:

ALTER DATABASE mydb SET geqo TO off;

This will save the setting (but not set it immediately). In subsequent connections to this database it will appear as though SET geqo TO off; had been executed just before the session started. Note that users can still alter this setting during their sessions; it will only be the default. To undo any such setting, use ALTER DATABASE dbname RESET varname.

Destroying a Database

Databases are destroyed with the command ???: DROP DATABASE <name>; Only the owner of the database, or a superuser, can drop a database. Dropping a database removes all objects that were contained within the database. The destruction of a database cannot be undone.

You cannot execute the DROP DATABASE command while connected to the victim database. You can, however, be connected to any other database, including the template1 database. template1 would be the only option for dropping the last user database of a given cluster.

For convenience, there is also a shell program to drop databases, ???: dropdb <dbname> (Unlike createdb, it is not the default action to drop the database with the current user name.)

Tablespaces

tablespace

Tablespaces in PostgreSQL allow database administrators to define locations in the file system where the files representing database objects can be stored. Once created, a tablespace can be referred to by name when creating database objects.

By using tablespaces, an administrator can control the disk layout of a PostgreSQL installation. This is useful in at least two ways. First, if the partition or volume on which the cluster was initialized runs out of space and cannot be extended, a tablespace can be created on a different partition and used until the system can be reconfigured.

Second, tablespaces allow an administrator to use knowledge of the usage pattern of database objects to optimize performance. For example, an index which is very heavily used can be placed on a very fast, highly available disk, such as an expensive solid state device. At the same time a table storing archived data which is rarely used or not performance critical could be stored on a less expensive, slower disk system.

[!WARNING] Even though located outside the main PostgreSQL data directory, tablespaces are an integral part of the database cluster and cannot be treated as an autonomous collection of data files. They are dependent on metadata contained in the main data directory, and therefore cannot be attached to a different database cluster or backed up individually. Similarly, if you lose a tablespace (file deletion, disk failure, etc.), the database cluster might become unreadable or unable to start. Placing a tablespace on a temporary file system like a RAM disk risks the reliability of the entire cluster.

To define a tablespace, use the ??? command, for example::

CREATE TABLESPACE fastspace LOCATION '/ssd1/postgresql/data';

The location must be an existing, empty directory that is owned by the PostgreSQL operating system user. All objects subsequently created within the tablespace will be stored in files underneath this directory. The location must not be on removable or transient storage, as the cluster might fail to function if the tablespace is missing or lost.

[!NOTE] There is usually not much point in making more than one tablespace per logical file system, since you cannot control the location of individual files within a logical file system. However, PostgreSQL does not enforce any such limitation, and indeed it is not directly aware of the file system boundaries on your system. It just stores files in the directories you tell it to use.

Creation of the tablespace itself must be done as a database superuser, but after that you can allow ordinary database users to use it. To do that, grant them the CREATE privilege on it.

Tables, indexes, and entire databases can be assigned to particular tablespaces. To do so, a user with the CREATE privilege on a given tablespace must pass the tablespace name as a parameter to the relevant command. For example, the following creates a table in the tablespace space1:

CREATE TABLE foo(i int) TABLESPACE space1;

Alternatively, use the ??? parameter:

SET default_tablespace = space1;
CREATE TABLE foo(i int);

When default_tablespace is set to anything but an empty string, it supplies an implicit TABLESPACE clause for CREATE TABLE and CREATE INDEX commands that do not have an explicit one.

There is also a ??? parameter, which determines the placement of temporary tables and indexes, as well as temporary files that are used for purposes such as sorting large data sets. This can be a list of tablespace names, rather than only one, so that the load associated with temporary objects can be spread over multiple tablespaces. A random member of the list is picked each time a temporary object is to be created.

The tablespace associated with a database is used to store the system catalogs of that database. Furthermore, it is the default tablespace used for tables, indexes, and temporary files created within the database, if no TABLESPACE clause is given and no other selection is specified by default_tablespace or temp_tablespaces (as appropriate). If a database is created without specifying a tablespace for it, it uses the same tablespace as the template database it is copied from.

Two tablespaces are automatically created when the database cluster is initialized. The pg_global tablespace is used only for shared system catalogs. The pg_default tablespace is the default tablespace of the template1 and template0 databases (and, therefore, will be the default tablespace for other databases as well, unless overridden by a TABLESPACE clause in CREATE DATABASE).

Once created, a tablespace can be used from any database, provided the requesting user has sufficient privilege. This means that a tablespace cannot be dropped until all objects in all databases using the tablespace have been removed.

To remove an empty tablespace, use the ??? command.

To determine the set of existing tablespaces, examine the pg_tablespace system catalog, for example SELECT spcname, spcowner::regrole, pg_tablespace_location(oid) FROM pg_tablespace; It is possible to find which databases use which tablespaces; see ???. The ??? program's \db meta-command is also useful for listing the existing tablespaces.

The directory $PGDATA/pg_tblspc contains symbolic links that point to each of the non-built-in tablespaces defined in the cluster. Although not recommended, it is possible to adjust the tablespace layout by hand by redefining these links. Under no circumstances perform this operation while the server is running.

Concurrency Control

concurrency

This chapter describes the behavior of the PostgreSQL database system when two or more sessions try to access the same data at the same time. The goals in that situation are to allow efficient access for all sessions while maintaining strict data integrity. Every developer of database applications should be familiar with the topics covered in this chapter.

Introduction

Multiversion Concurrency Control

MVCC

Serializable Snapshot Isolation

SSI

PostgreSQL provides a rich set of tools for developers to manage concurrent access to data. Internally, data consistency is maintained by using a multiversion model (Multiversion Concurrency Control, MVCC). This means that each SQL statement sees a snapshot of data (a database version) as it was some time ago, regardless of the current state of the underlying data. This prevents statements from viewing inconsistent data produced by concurrent transactions performing updates on the same data rows, providing transaction isolation for each database session. MVCC, by eschewing the locking methodologies of traditional database systems, minimizes lock contention in order to allow for reasonable performance in multiuser environments.

The main advantage of using the MVCC model of concurrency control rather than locking is that in MVCC locks acquired for querying (reading) data do not conflict with locks acquired for writing data, and so reading never blocks writing and writing never blocks reading. PostgreSQL maintains this guarantee even when providing the strictest level of transaction isolation through the use of an innovative Serializable Snapshot Isolation (SSI) level.

Table- and row-level locking facilities are also available in PostgreSQL for applications which don't generally need full transaction isolation and prefer to explicitly manage particular points of conflict. However, proper use of MVCC will generally provide better performance than locks. In addition, application-defined advisory locks provide a mechanism for acquiring locks that are not tied to a single transaction.

Transaction Isolation

transaction isolation

The SQL standard defines four levels of transaction isolation. The most strict is Serializable, which is defined by the standard in a paragraph which says that any concurrent execution of a set of Serializable transactions is guaranteed to produce the same effect as running them one at a time in some order. The other three levels are defined in terms of phenomena, resulting from interaction between concurrent transactions, which must not occur at each level. The standard notes that due to the definition of Serializable, none of these phenomena are possible at that level. (This is hardly surprising -- if the effect of the transactions must be consistent with having been run one at a time, how could you see any phenomena caused by interactions?)

The phenomena which are prohibited at various levels are:

dirty read
A transaction reads data written by a concurrent uncommitted transaction.

nonrepeatable read
A transaction re-reads data it has previously read and finds that data has been modified by another transaction (that committed since the initial read).

phantom read
A transaction re-executes a query returning a set of rows that satisfy a search condition and finds that the set of rows satisfying the condition has changed due to another recently-committed transaction.

serialization anomaly
The result of successfully committing a group of transactions is inconsistent with all possible orderings of running those transactions one at a time.

The SQL standard and PostgreSQL-implemented transaction isolation levels are described in Transaction Isolation Levels.

Isolation Level Dirty Read Nonrepeatable Read Phantom Read Serialization Anomaly
Read uncommitted Allowed, but not in PG Possible Possible Possible
Read committed Not possible Possible Possible Possible
Repeatable read Not possible Not possible Allowed, but not in PG Possible
Serializable Not possible Not possible Not possible Not possible

Transaction Isolation Levels {#mvcc-isolevel-table}

In PostgreSQL, you can request any of the four standard transaction isolation levels, but internally only three distinct isolation levels are implemented, i.e., PostgreSQL's Read Uncommitted mode behaves like Read Committed. This is because it is the only sensible way to map the standard isolation levels to PostgreSQL's multiversion concurrency control architecture.

The table also shows that PostgreSQL's Repeatable Read implementation does not allow phantom reads. This is acceptable under the SQL standard because the standard specifies which anomalies must not occur at certain isolation levels; higher guarantees are acceptable. The behavior of the available isolation levels is detailed in the following subsections.

To set the transaction isolation level of a transaction, use the command ???.

[!IMPORTANT] Some PostgreSQL data types and functions have special rules regarding transactional behavior. In particular, changes made to a sequence (and therefore the counter of a column declared using serial) are immediately visible to all other transactions and are not rolled back if the transaction that made the changes aborts. See ??? and ???.

Read Committed Isolation Level

transaction isolation level

read committed

read committed

Read Committed is the default isolation level in PostgreSQL. When a transaction uses this isolation level, a SELECT query (without a FOR UPDATE/SHARE clause) sees only data committed before the query began; it never sees either uncommitted data or changes committed by concurrent transactions during the query's execution. In effect, a SELECT query sees a snapshot of the database as of the instant the query begins to run. However, SELECT does see the effects of previous updates executed within its own transaction, even though they are not yet committed. Also note that two successive SELECT commands can see different data, even though they are within a single transaction, if other transactions commit changes after the first SELECT starts and before the second SELECT starts.

UPDATE, DELETE, SELECT FOR UPDATE, and SELECT FOR SHARE commands behave the same as SELECT in terms of searching for target rows: they will only find target rows that were committed as of the command start time. However, such a target row might have already been updated (or deleted or locked) by another concurrent transaction by the time it is found. In this case, the would-be updater will wait for the first updating transaction to commit or roll back (if it is still in progress). If the first updater rolls back, then its effects are negated and the second updater can proceed with updating the originally found row. If the first updater commits, the second updater will ignore the row if the first updater deleted it, otherwise it will attempt to apply its operation to the updated version of the row. The search condition of the command (the WHERE clause) is re-evaluated to see if the updated version of the row still matches the search condition. If so, the second updater proceeds with its operation using the updated version of the row. In the case of SELECT FOR UPDATE and SELECT FOR SHARE, this means it is the updated version of the row that is locked and returned to the client.

INSERT with an ON CONFLICT DO UPDATE clause behaves similarly. In Read Committed mode, each row proposed for insertion will either insert or update. Unless there are unrelated errors, one of those two outcomes is guaranteed. If a conflict originates in another transaction whose effects are not yet visible to the INSERT, the UPDATE clause will affect that row, even though possibly no version of that row is conventionally visible to the command.

INSERT with an ON CONFLICT DO SELECT clause behaves similarly to ON CONFLICT DO UPDATE. In Read Committed mode, each row proposed for insertion is guaranteed to either insert or return the conflicting row (unless there are unrelated errors). If a conflict originates in another transaction whose effects are not yet visible to the INSERT, the command will wait for that transaction to commit or roll back, then return the conflicting row if it was committed (even though that row was not visible when the command started).

INSERT with an ON CONFLICT DO NOTHING clause may have insertion not proceed for a row due to the outcome of another transaction whose effects are not visible to the INSERT snapshot. Again, this is only the case in Read Committed mode.

MERGE allows the user to specify various combinations of INSERT, UPDATE and DELETE subcommands. A MERGE command with both INSERT and UPDATE subcommands looks similar to INSERT with an ON CONFLICT DO UPDATE clause but does not guarantee that either INSERT or UPDATE will occur. If MERGE attempts an UPDATE or DELETE and the row is concurrently updated but the join condition still passes for the current target and the current source tuple, then MERGE will behave the same as the UPDATE or DELETE commands and perform its action on the updated version of the row. However, because MERGE can specify several actions and they can be conditional, the conditions for each action are re-evaluated on the updated version of the row, starting from the first action, even if the action that had originally matched appears later in the list of actions. On the other hand, if the row is concurrently updated so that the join condition fails, then MERGE will evaluate the command's NOT MATCHED BY SOURCE and NOT MATCHED [BY TARGET] actions next, and execute the first one of each kind that succeeds. If the row is concurrently deleted, then MERGE will evaluate the command's NOT MATCHED [BY TARGET] actions, and execute the first one that succeeds. If MERGE attempts an INSERT and a unique index is present and a duplicate row is concurrently inserted, then a uniqueness violation error is raised; MERGE does not attempt to avoid such errors by restarting evaluation of MATCHED conditions.

Because of the above rules, it is possible for an updating command to see an inconsistent snapshot: it can see the effects of concurrent updating commands on the same rows it is trying to update, but it does not see effects of those commands on other rows in the database. This behavior makes Read Committed mode unsuitable for commands that involve complex search conditions; however, it is just right for simpler cases. For example, consider transferring $100 from one account to another:

BEGIN;
UPDATE accounts SET balance = balance + 100.00 WHERE acctnum = 12345;
UPDATE accounts SET balance = balance - 100.00 WHERE acctnum = 7534;
COMMIT;

If another transaction concurrently tries to change the balance of account 7534, we clearly want the second statement to start with the updated version of the account's row. Because each command is affecting only a predetermined row, letting it see the updated version of the row does not create any troublesome inconsistency.

More complex usage can produce undesirable results in Read Committed mode. For example, consider a DELETE command operating on data that is being both added and removed from its restriction criteria by another command, e.g., assume website is a two-row table with website.hits equaling 9 and 10:

BEGIN;
UPDATE website SET hits = hits + 1;
-- run from another session:  DELETE FROM website WHERE hits = 10;
COMMIT;

The DELETE will have no effect even though there is a website.hits = 10 row before and after the UPDATE. This occurs because the pre-update row value 9 is skipped, and when the UPDATE completes and DELETE obtains a lock, the new row value is no longer 10 but 11, which no longer matches the criteria.

Because Read Committed mode starts each command with a new snapshot that includes all transactions committed up to that instant, subsequent commands in the same transaction will see the effects of the committed concurrent transaction in any case. The point at issue above is whether or not a single command sees an absolutely consistent view of the database.

The partial transaction isolation provided by Read Committed mode is adequate for many applications, and this mode is fast and simple to use; however, it is not sufficient for all cases. Applications that do complex queries and updates might require a more rigorously consistent view of the database than Read Committed mode provides.

Repeatable Read Isolation Level

transaction isolation level

repeatable read

repeatable read

The Repeatable Read isolation level only sees data committed before the transaction began; it never sees either uncommitted data or changes committed by concurrent transactions during the transaction's execution. (However, each query does see the effects of previous updates executed within its own transaction, even though they are not yet committed.) This is a stronger guarantee than is required by the SQL standard for this isolation level, and prevents all of the phenomena described in Transaction Isolation Levels except for serialization anomalies. As mentioned above, this is specifically allowed by the standard, which only describes the minimum protections each isolation level must provide.

This level is different from Read Committed in that a query in a repeatable read transaction sees a snapshot as of the start of the first non-transaction-control statement in the transaction, not as of the start of the current statement within the transaction. Thus, successive SELECT commands within a single transaction see the same data, i.e., they do not see changes made by other transactions that committed after their own transaction started.

Applications using this level must be prepared to retry transactions due to serialization failures.

UPDATE, DELETE, MERGE, SELECT FOR UPDATE, and SELECT FOR SHARE commands behave the same as SELECT in terms of searching for target rows: they will only find target rows that were committed as of the transaction start time. However, such a target row might have already been updated (or deleted or locked) by another concurrent transaction by the time it is found. In this case, the repeatable read transaction will wait for the first updating transaction to commit or roll back (if it is still in progress). If the first updater rolls back, then its effects are negated and the repeatable read transaction can proceed with updating the originally found row. But if the first updater commits (and actually updated or deleted the row, not just locked it) then the repeatable read transaction will be rolled back with the message

ERROR:  could not serialize access due to concurrent update

because a repeatable read transaction cannot modify or lock rows changed by other transactions after the repeatable read transaction began.

When an application receives this error message, it should abort the current transaction and retry the whole transaction from the beginning. The second time through, the transaction will see the previously-committed change as part of its initial view of the database, so there is no logical conflict in using the new version of the row as the starting point for the new transaction's update.

Note that only updating transactions might need to be retried; read-only transactions will never have serialization conflicts.

The Repeatable Read mode provides a rigorous guarantee that each transaction sees a completely stable view of the database. However, this view will not necessarily always be consistent with some serial (one at a time) execution of concurrent transactions of the same level. For example, even a read-only transaction at this level may see a control record updated to show that a batch has been completed but not see one of the detail records which is logically part of the batch because it read an earlier revision of the control record. Attempts to enforce business rules by transactions running at this isolation level are not likely to work correctly without careful use of explicit locks to block conflicting transactions.

The Repeatable Read isolation level is implemented using a technique known in academic database literature and in some other database products as Snapshot Isolation. Differences in behavior and performance may be observed when compared with systems that use a traditional locking technique that reduces concurrency. Some other systems may even offer Repeatable Read and Snapshot Isolation as distinct isolation levels with different behavior. The permitted phenomena that distinguish the two techniques were not formalized by database researchers until after the SQL standard was developed, and are outside the scope of this manual. For a full treatment, please see ???.

[!NOTE] Prior to PostgreSQL version 9.1, a request for the Serializable transaction isolation level provided exactly the same behavior described here. To retain the legacy Serializable behavior, Repeatable Read should now be requested.

Serializable Isolation Level

transaction isolation level

serializable

serializable

predicate locking

serialization anomaly

The Serializable isolation level provides the strictest transaction isolation. This level emulates serial transaction execution for all committed transactions; as if transactions had been executed one after another, serially, rather than concurrently. However, like the Repeatable Read level, applications using this level must be prepared to retry transactions due to serialization failures. In fact, this isolation level works exactly the same as Repeatable Read except that it also monitors for conditions which could make execution of a concurrent set of serializable transactions behave in a manner inconsistent with all possible serial (one at a time) executions of those transactions. This monitoring does not introduce any blocking beyond that present in repeatable read, but there is some overhead to the monitoring, and detection of the conditions which could cause a serialization anomaly will trigger a serialization failure.

As an example, consider a table mytab, initially containing:

 class | value
-------+-------
     1 |    10
     1 |    20
     2 |   100
     2 |   200

Suppose that serializable transaction A computes:

SELECT SUM(value) FROM mytab WHERE class = 1;

and then inserts the result (30) as the value in a new row with class= 2. Concurrently, serializable transaction B computes:

SELECT SUM(value) FROM mytab WHERE class = 2;

and obtains the result 300, which it inserts in a new row with class= 1. Then both transactions try to commit. If either transaction were running at the Repeatable Read isolation level, both would be allowed to commit; but since there is no serial order of execution consistent with the result, using Serializable transactions will allow one transaction to commit and will roll the other back with this message:

ERROR:  could not serialize access due to read/write dependencies among transactions

This is because if A had executed before B, B would have computed the sum 330, not 300, and similarly the other order would have resulted in a different sum computed by A.

When relying on Serializable transactions to prevent anomalies, it is important that any data read from a permanent user table not be considered valid until the transaction which read it has successfully committed. This is true even for read-only transactions, except that data read within a deferrable read-only transaction is known to be valid as soon as it is read, because such a transaction waits until it can acquire a snapshot guaranteed to be free from such problems before starting to read any data. In all other cases applications must not depend on results read during a transaction that later aborted; instead, they should retry the transaction until it succeeds.

To guarantee true serializability PostgreSQL uses predicate locking, which means that it keeps locks which allow it to determine when a write would have had an impact on the result of a previous read from a concurrent transaction, had it run first. In PostgreSQL these locks do not cause any blocking and therefore can not play any part in causing a deadlock. They are used to identify and flag dependencies among concurrent Serializable transactions which in certain combinations can lead to serialization anomalies. In contrast, a Read Committed or Repeatable Read transaction which wants to ensure data consistency may need to take out a lock on an entire table, which could block other users attempting to use that table, or it may use SELECT FOR UPDATE or SELECT FOR SHARE which not only can block other transactions but cause disk access.

Predicate locks in PostgreSQL, like in most other database systems, are based on data actually accessed by a transaction. These will show up in the pg_locks system view with a mode of SIReadLock. The particular locks acquired during execution of a query will depend on the plan used by the query, and multiple finer-grained locks (e.g., tuple locks) may be combined into fewer coarser-grained locks (e.g., page locks) during the course of the transaction to prevent exhaustion of the memory used to track the locks. A READ ONLY transaction may be able to release its SIRead locks before completion, if it detects that no conflicts can still occur which could lead to a serialization anomaly. In fact, READ ONLY transactions will often be able to establish that fact at startup and avoid taking any predicate locks. If you explicitly request a SERIALIZABLE READ ONLY DEFERRABLE transaction, it will block until it can establish this fact. (This is the only case where Serializable transactions block but Repeatable Read transactions don't.) On the other hand, SIRead locks often need to be kept past transaction commit, until overlapping read write transactions complete.

Consistent use of Serializable transactions can simplify development. The guarantee that any set of successfully committed concurrent Serializable transactions will have the same effect as if they were run one at a time means that if you can demonstrate that a single transaction, as written, will do the right thing when run by itself, you can have confidence that it will do the right thing in any mix of Serializable transactions, even without any information about what those other transactions might do, or it will not successfully commit. It is important that an environment which uses this technique have a generalized way of handling serialization failures (which always return with an SQLSTATE value of '40001'), because it will be very hard to predict exactly which transactions might contribute to the read/write dependencies and need to be rolled back to prevent serialization anomalies. The monitoring of read/write dependencies has a cost, as does the restart of transactions which are terminated with a serialization failure, but balanced against the cost and blocking involved in use of explicit locks and SELECT FOR UPDATE or SELECT FOR SHARE, Serializable transactions are the best performance choice for some environments.

While PostgreSQL's Serializable transaction isolation level only allows concurrent transactions to commit if it can prove there is a serial order of execution that would produce the same effect, it doesn't always prevent errors from being raised that would not occur in true serial execution. In particular, it is possible to see unique constraint violations caused by conflicts with overlapping Serializable transactions even after explicitly checking that the key isn't present before attempting to insert it. This can be avoided by making sure that all Serializable transactions that insert potentially conflicting keys explicitly check if they can do so first. For example, imagine an application that asks the user for a new key and then checks that it doesn't exist already by trying to select it first, or generates a new key by selecting the maximum existing key and adding one. If some Serializable transactions insert new keys directly without following this protocol, unique constraints violations might be reported even in cases where they could not occur in a serial execution of the concurrent transactions.

For optimal performance when relying on Serializable transactions for concurrency control, these issues should be considered:

The Serializable isolation level is implemented using a technique known in academic database literature as Serializable Snapshot Isolation, which builds on Snapshot Isolation by adding checks for serialization anomalies. Some differences in behavior and performance may be observed when compared with other systems that use a traditional locking technique. Please see ??? for detailed information.

Explicit Locking

lock

PostgreSQL provides various lock modes to control concurrent access to data in tables. These modes can be used for application-controlled locking in situations where MVCC does not give the desired behavior. Also, most PostgreSQL commands automatically acquire locks of appropriate modes to ensure that referenced tables are not dropped or modified in incompatible ways while the command executes. (For example, TRUNCATE cannot safely be executed concurrently with other operations on the same table, so it obtains an ACCESS EXCLUSIVE lock on the table to enforce that.)

To examine a list of the currently outstanding locks in a database server, use the pg_locks system view. For more information on monitoring the status of the lock manager subsystem, refer to ???.

Table-Level Locks

LOCK

The list below shows the available lock modes and the contexts in which they are used automatically by PostgreSQL. You can also acquire any of these locks explicitly with the command ???. Remember that all of these lock modes are table-level locks, even if the name contains the word “row”; the names of the lock modes are historical. To some extent the names reflect the typical usage of each lock mode — but the semantics are all the same. The only real difference between one lock mode and another is the set of lock modes with which each conflicts (see Conflicting Lock Modes). Two transactions cannot hold locks of conflicting modes on the same table at the same time. (However, a transaction never conflicts with itself. For example, it might acquire ACCESS EXCLUSIVE lock and later acquire ACCESS SHARE lock on the same table.) Non-conflicting lock modes can be held concurrently by many transactions. Notice in particular that some lock modes are self-conflicting (for example, an ACCESS EXCLUSIVE lock cannot be held by more than one transaction at a time) while others are not self-conflicting (for example, an ACCESS SHARE lock can be held by multiple transactions).

ACCESS SHARE (AccessShareLock)
Conflicts with the ACCESS EXCLUSIVE lock mode only.

The SELECT command acquires a lock of this mode on referenced tables. In general, any query that only reads a table and does not modify it will acquire this lock mode.

ROW SHARE (RowShareLock)
Conflicts with the EXCLUSIVE and ACCESS EXCLUSIVE lock modes.

The SELECT command acquires a lock of this mode on all tables on which one of the FOR UPDATE, FOR NO KEY UPDATE, FOR SHARE, or FOR KEY SHARE options is specified (in addition to ACCESS SHARE locks on any other tables that are referenced without any explicit FOR ... locking option).

ROW EXCLUSIVE (RowExclusiveLock)
Conflicts with the SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, and ACCESS EXCLUSIVE lock modes.

The commands UPDATE, DELETE, INSERT, and MERGE acquire this lock mode on the target table (in addition to ACCESS SHARE locks on any other referenced tables). In general, this lock mode will be acquired by any command that modifies data in a table.

SHARE UPDATE EXCLUSIVE (ShareUpdateExclusiveLock)
Conflicts with the SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, and ACCESS EXCLUSIVE lock modes. This mode protects a table against concurrent schema changes and VACUUM runs.

Acquired by VACUUM (without FULL), ANALYZE, CREATE INDEX CONCURRENTLY, CREATE STATISTICS, COMMENT ON, REINDEX CONCURRENTLY, and certain ALTER INDEX and ALTER TABLE variants (for full details see the documentation of these commands).

SHARE (ShareLock)
Conflicts with the ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE ROW EXCLUSIVE, EXCLUSIVE, and ACCESS EXCLUSIVE lock modes. This mode protects a table against concurrent data changes.

Acquired by CREATE INDEX (without CONCURRENTLY).

SHARE ROW EXCLUSIVE (ShareRowExclusiveLock)
Conflicts with the ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, and ACCESS EXCLUSIVE lock modes. This mode protects a table against concurrent data changes, and is self-exclusive so that only one session can hold it at a time.

Acquired by CREATE TRIGGER and some forms of ALTER TABLE.

EXCLUSIVE (ExclusiveLock)
Conflicts with the ROW SHARE, ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, and ACCESS EXCLUSIVE lock modes. This mode allows only concurrent ACCESS SHARE locks, i.e., only reads from the table can proceed in parallel with a transaction holding this lock mode.

Acquired by REFRESH MATERIALIZED VIEW CONCURRENTLY.

ACCESS EXCLUSIVE (AccessExclusiveLock)
Conflicts with locks of all modes (ACCESS SHARE, ROW SHARE, ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, and ACCESS EXCLUSIVE). This mode guarantees that the holder is the only transaction accessing the table in any way.

Acquired by the DROP TABLE, TRUNCATE, REINDEX, CLUSTER, VACUUM FULL, and REFRESH MATERIALIZED VIEW (without CONCURRENTLY) commands. REPACK also acquires this lock mode. When run with CONCURRENTLY, REPACK acquires it only while swapping the table and index files. Many forms of ALTER INDEX and ALTER TABLE also acquire a lock at this level. This is also the default lock mode for LOCK TABLE statements that do not specify a mode explicitly.

[!TIP] Only an ACCESS EXCLUSIVE lock blocks a SELECT (without FOR UPDATE/SHARE) statement.

Once acquired, a lock is normally held until the end of the transaction. But if a lock is acquired after establishing a savepoint, the lock is released immediately if the savepoint is rolled back to. This is consistent with the principle that ROLLBACK cancels all effects of the commands since the savepoint. The same holds for locks acquired within a PL/pgSQL exception block: an error escape from the block releases locks acquired within it.

Conflicting Lock Modes
Requested Lock Mode Existing Lock Mode
ACCESS SHARE ROW SHARE ROW EXCL. SHARE UPDATE EXCL. SHARE SHARE ROW EXCL. EXCL. ACCESS EXCL.
ACCESS SHARE X
ROW SHARE X X
ROW EXCL. X X X X
SHARE UPDATE EXCL. X X X X X
SHARE X X X X X
SHARE ROW EXCL. X X X X X X
EXCL. X X X X X X X
ACCESS EXCL. X X X X X X X X

Fast-Path Locking

fast-path locking

Internally, PostgreSQL can record some table-level locks using a fast-path locking mechanism instead of the main lock table. This reduces the overhead of acquiring and releasing locks that rarely conflict. It is an implementation optimization and does not change lock semantics.

Fast-path locking can be used only for eligible relation locks in the weak table-level lock modes ACCESS SHARE (AccessShareLock), ROW SHARE (RowShareLock), and ROW EXCLUSIVE (RowExclusiveLock). It does not apply to shared relations (those visible across all databases, such as pg_authid). Other lock types, and stronger lock modes on relations, always go through the main lock table. Even for eligible locks, fast-path is used only when a per-backend slot is available; the number of slots is derived from ???. When no slot is available, the lock is acquired via the main lock table instead.

Acquiring a lock via the main lock table is considerably more expensive than the fast path, and under heavy concurrent lock activity can become a point of contention (observable as the LockManager wait event).

Row-Level Locks

In addition to table-level locks, there are row-level locks, which are listed as below with the contexts in which they are used automatically by PostgreSQL. See Conflicting Row-Level Locks for a complete table of row-level lock conflicts. Note that a transaction can hold conflicting locks on the same row, even in different subtransactions; but other than that, two transactions can never hold conflicting locks on the same row. Row-level locks do not affect data querying; they block only writers and lockers to the same row. Row-level locks are released at transaction end or during savepoint rollback, just like table-level locks.

FOR UPDATE
FOR UPDATE causes the rows retrieved by the SELECT statement to be locked as though for update. This prevents them from being locked, modified or deleted by other transactions until the current transaction ends. That is, other transactions that attempt UPDATE, DELETE, SELECT FOR UPDATE, SELECT FOR NO KEY UPDATE, SELECT FOR SHARE or SELECT FOR KEY SHARE of these rows will be blocked until the current transaction ends; conversely, SELECT FOR UPDATE will wait for a concurrent transaction that has run any of those commands on the same row, and will then lock and return the updated row (or no row, if the row was deleted). Within a REPEATABLE READ or SERIALIZABLE transaction, however, an error will be thrown if a row to be locked has changed since the transaction started. For further discussion see Data Consistency Checks at the Application Level.

The FOR UPDATE lock mode is also acquired by any DELETE on a row, and also by an UPDATE that modifies the values of certain columns. Currently, the set of columns considered for the UPDATE case are those that have a unique index on them that can be used in a foreign key (so partial indexes and expressional indexes are not considered), but this may change in the future.

FOR NO KEY UPDATE
Behaves similarly to FOR UPDATE, except that the lock acquired is weaker: this lock will not block SELECT FOR KEY SHARE commands that attempt to acquire a lock on the same rows. This lock mode is also acquired by any UPDATE that does not acquire a FOR UPDATE lock.

FOR SHARE
Behaves similarly to FOR NO KEY UPDATE, except that it acquires a shared lock rather than exclusive lock on each retrieved row. A shared lock blocks other transactions from performing UPDATE, DELETE, SELECT FOR UPDATE or SELECT FOR NO KEY UPDATE on these rows, but it does not prevent them from performing SELECT FOR SHARE or SELECT FOR KEY SHARE.

FOR KEY SHARE
Behaves similarly to FOR SHARE, except that the lock is weaker: SELECT FOR UPDATE is blocked, but not SELECT FOR NO KEY UPDATE. A key-shared lock blocks other transactions from performing DELETE or any UPDATE that changes the key values, but not other UPDATE, and neither does it prevent SELECT FOR NO KEY UPDATE, SELECT FOR SHARE, or SELECT FOR KEY SHARE.

PostgreSQL doesn't remember any information about modified rows in memory, so there is no limit on the number of rows locked at one time. However, locking a row might cause a disk write, e.g., SELECT FOR UPDATE modifies selected rows to mark them locked, and so will result in disk writes.

Conflicting Row-Level Locks
Requested Lock Mode Current Lock Mode
FOR KEY SHARE FOR SHARE FOR NO KEY UPDATE FOR UPDATE
FOR KEY SHARE X
FOR SHARE X X
FOR NO KEY UPDATE X X X
FOR UPDATE X X X X

Page-Level Locks

In addition to table and row locks, page-level share/exclusive locks are used to control read/write access to table pages in the shared buffer pool. These locks are released immediately after a row is fetched or updated. Application developers normally need not be concerned with page-level locks, but they are mentioned here for completeness.

Deadlocks

deadlock

The use of explicit locking can increase the likelihood of deadlocks, wherein two (or more) transactions each hold locks that the other wants. For example, if transaction 1 acquires an exclusive lock on table A and then tries to acquire an exclusive lock on table B, while transaction 2 has already exclusive-locked table B and now wants an exclusive lock on table A, then neither one can proceed. PostgreSQL automatically detects deadlock situations and resolves them by aborting one of the transactions involved, allowing the other(s) to complete. (Exactly which transaction will be aborted is difficult to predict and should not be relied upon.)

Note that deadlocks can also occur as the result of row-level locks (and thus, they can occur even if explicit locking is not used). Consider the case in which two concurrent transactions modify a table. The first transaction executes:

UPDATE accounts SET balance = balance + 100.00 WHERE acctnum = 11111;

This acquires a row-level lock on the row with the specified account number. Then, the second transaction executes:

UPDATE accounts SET balance = balance + 100.00 WHERE acctnum = 22222;
UPDATE accounts SET balance = balance - 100.00 WHERE acctnum = 11111;

The first UPDATE statement successfully acquires a row-level lock on the specified row, so it succeeds in updating that row. However, the second UPDATE statement finds that the row it is attempting to update has already been locked, so it waits for the transaction that acquired the lock to complete. Transaction two is now waiting on transaction one to complete before it continues execution. Now, transaction one executes:

UPDATE accounts SET balance = balance - 100.00 WHERE acctnum = 22222;

Transaction one attempts to acquire a row-level lock on the specified row, but it cannot: transaction two already holds such a lock. So it waits for transaction two to complete. Thus, transaction one is blocked on transaction two, and transaction two is blocked on transaction one: a deadlock condition. PostgreSQL will detect this situation and abort one of the transactions.

The best defense against deadlocks is generally to avoid them by being certain that all applications using a database acquire locks on multiple objects in a consistent order. In the example above, if both transactions had updated the rows in the same order, no deadlock would have occurred. One should also ensure that the first lock acquired on an object in a transaction is the most restrictive mode that will be needed for that object. If it is not feasible to verify this in advance, then deadlocks can be handled on-the-fly by retrying transactions that abort due to deadlocks.

So long as no deadlock situation is detected, a transaction seeking either a table-level or row-level lock will wait indefinitely for conflicting locks to be released. This means it is a bad idea for applications to hold transactions open for long periods of time (e.g., while waiting for user input).

Advisory Locks

advisory lock

lock

advisory

PostgreSQL provides a means for creating locks that have application-defined meanings. These are called advisory locks, because the system does not enforce their use — it is up to the application to use them correctly. Advisory locks can be useful for locking strategies that are an awkward fit for the MVCC model. For example, a common use of advisory locks is to emulate pessimistic locking strategies typical of so-called “flat file” data management systems. While a flag stored in a table could be used for the same purpose, advisory locks are faster, avoid table bloat, and are automatically cleaned up by the server at the end of the session.

There are two ways to acquire an advisory lock in PostgreSQL: at session level or at transaction level. Once acquired at session level, an advisory lock is held until explicitly released or the session ends. Unlike standard lock requests, session-level advisory lock requests do not honor transaction semantics: a lock acquired during a transaction that is later rolled back will still be held following the rollback, and likewise an unlock is effective even if the calling transaction fails later. A lock can be acquired multiple times by its owning process; for each completed lock request there must be a corresponding unlock request before the lock is actually released. Transaction-level lock requests, on the other hand, behave more like regular lock requests: they are automatically released at the end of the transaction, and there is no explicit unlock operation. This behavior is often more convenient than the session-level behavior for short-term usage of an advisory lock. Session-level and transaction-level lock requests for the same advisory lock identifier will block each other in the expected way. If a session already holds a given advisory lock, additional requests by it will always succeed, even if other sessions are awaiting the lock; this statement is true regardless of whether the existing lock hold and new request are at session level or transaction level.

Like all locks in PostgreSQL, a complete list of advisory locks currently held by any session can be found in the pg_locks system view.

Both advisory locks and regular locks are stored in a shared memory pool whose size is defined by the configuration variables ??? and ???. Care must be taken not to exhaust this memory or the server will be unable to grant any locks at all. This imposes an upper limit on the number of advisory locks grantable by the server, typically in the tens to hundreds of thousands depending on how the server is configured.

In certain cases using advisory locking methods, especially in queries involving explicit ordering and LIMIT clauses, care must be taken to control the locks acquired because of the order in which SQL expressions are evaluated. For example:

SELECT pg_advisory_lock(id) FROM foo WHERE id = 12345; -- ok
SELECT pg_advisory_lock(id) FROM foo WHERE id > 12345 LIMIT 100; -- danger!
SELECT pg_advisory_lock(q.id) FROM
(
  SELECT id FROM foo WHERE id > 12345 LIMIT 100
) q; -- ok

In the above queries, the second form is dangerous because the LIMIT is not guaranteed to be applied before the locking function is executed. This might cause some locks to be acquired that the application was not expecting, and hence would fail to release (until it ends the session). From the point of view of the application, such locks would be dangling, although still viewable in pg_locks.

The functions provided to manipulate advisory locks are described in ???.

Data Consistency Checks at the Application Level

It is very difficult to enforce business rules regarding data integrity using Read Committed transactions because the view of the data is shifting with each statement, and even a single statement may not restrict itself to the statement's snapshot if a write conflict occurs.

While a Repeatable Read transaction has a stable view of the data throughout its execution, there is a subtle issue with using MVCC snapshots for data consistency checks, involving something known as read/write conflicts. If one transaction writes data and a concurrent transaction attempts to read the same data (whether before or after the write), it cannot see the work of the other transaction. The reader then appears to have executed first regardless of which started first or which committed first. If that is as far as it goes, there is no problem, but if the reader also writes data which is read by a concurrent transaction there is now a transaction which appears to have run before either of the previously mentioned transactions. If the transaction which appears to have executed last actually commits first, it is very easy for a cycle to appear in a graph of the order of execution of the transactions. When such a cycle appears, integrity checks will not work correctly without some help.

As mentioned in Serializable Isolation Level, Serializable transactions are just Repeatable Read transactions which add nonblocking monitoring for dangerous patterns of read/write conflicts. When a pattern is detected which could cause a cycle in the apparent order of execution, one of the transactions involved is rolled back to break the cycle.

Enforcing Consistency with Serializable Transactions

If the Serializable transaction isolation level is used for all writes and for all reads which need a consistent view of the data, no other effort is required to ensure consistency. Software from other environments which is written to use serializable transactions to ensure consistency should “just work” in this regard in PostgreSQL.

When using this technique, it will avoid creating an unnecessary burden for application programmers if the application software goes through a framework which automatically retries transactions which are rolled back with a serialization failure. It may be a good idea to set default_transaction_isolation to serializable. It would also be wise to take some action to ensure that no other transaction isolation level is used, either inadvertently or to subvert integrity checks, through checks of the transaction isolation level in triggers.

See Serializable Isolation Level for performance suggestions.

[!WARNING] This level of integrity protection using Serializable transactions does not yet extend to hot standby mode (???) or logical replicas. Because of that, those using hot standby or logical replication may want to use Repeatable Read and explicit locking on the primary.

Enforcing Consistency with Explicit Blocking Locks

When non-serializable writes are possible, to ensure the current validity of a row and protect it against concurrent updates one must use SELECT FOR UPDATE, SELECT FOR SHARE, or an appropriate LOCK TABLE statement. (SELECT FOR UPDATE and SELECT FOR SHARE lock just the returned rows against concurrent updates, while LOCK TABLE locks the whole table.) This should be taken into account when porting applications to PostgreSQL from other environments.

Also of note to those converting from other environments is the fact that SELECT FOR UPDATE does not ensure that a concurrent transaction will not update or delete a selected row. To do that in PostgreSQL you must actually update the row, even if no values need to be changed. SELECT FOR UPDATE temporarily blocks other transactions from acquiring the same lock or executing an UPDATE or DELETE which would affect the locked row, but once the transaction holding this lock commits or rolls back, a blocked transaction will proceed with the conflicting operation unless an actual UPDATE of the row was performed while the lock was held.

Global validity checks require extra thought under non-serializable MVCC. For example, a banking application might wish to check that the sum of all credits in one table equals the sum of debits in another table, when both tables are being actively updated. Comparing the results of two successive SELECT sum(...) commands will not work reliably in Read Committed mode, since the second query will likely include the results of transactions not counted by the first. Doing the two sums in a single repeatable read transaction will give an accurate picture of only the effects of transactions that committed before the repeatable read transaction started — but one might legitimately wonder whether the answer is still relevant by the time it is delivered. If the repeatable read transaction itself applied some changes before trying to make the consistency check, the usefulness of the check becomes even more debatable, since now it includes some but not all post-transaction-start changes. In such cases a careful person might wish to lock all tables needed for the check, in order to get an indisputable picture of current reality. A SHARE mode (or higher) lock guarantees that there are no uncommitted changes in the locked table, other than those of the current transaction.

Note also that if one is relying on explicit locking to prevent concurrent changes, one should either use Read Committed mode, or in Repeatable Read mode be careful to obtain locks before performing queries. A lock obtained by a repeatable read transaction guarantees that no other transactions modifying the table are still running, but if the snapshot seen by the transaction predates obtaining the lock, it might predate some now-committed changes in the table. A repeatable read transaction's snapshot is actually frozen at the start of its first query or data-modification command (SELECT, INSERT, UPDATE, DELETE, or MERGE), so it is possible to obtain locks explicitly before the snapshot is frozen.

Serialization Failure Handling

serialization failure

retryable error

Both Repeatable Read and Serializable isolation levels can produce errors that are designed to prevent serialization anomalies. As previously stated, applications using these levels must be prepared to retry transactions that fail due to serialization errors. Such an error's message text will vary according to the precise circumstances, but it will always have the SQLSTATE code 40001 (serialization_failure).

It may also be advisable to retry deadlock failures. These have the SQLSTATE code 40P01 (deadlock_detected).

In some cases it is also appropriate to retry unique-key failures, which have SQLSTATE code 23505 (unique_violation), and exclusion constraint failures, which have SQLSTATE code 23P01 (exclusion_violation). For example, if the application selects a new value for a primary key column after inspecting the currently stored keys, it could get a unique-key failure because another application instance selected the same new key concurrently. This is effectively a serialization failure, but the server will not detect it as such because it cannot “see” the connection between the inserted value and the previous reads. There are also some corner cases in which the server will issue a unique-key or exclusion constraint error even though in principle it has enough information to determine that a serialization problem is the underlying cause. While it's recommendable to just retry serialization_failure errors unconditionally, more care is needed when retrying these other error codes, since they might represent persistent error conditions rather than transient failures.

It is important to retry the complete transaction, including all logic that decides which SQL to issue and/or which values to use. Therefore, PostgreSQL does not offer an automatic retry facility, since it cannot do so with any guarantee of correctness.

Transaction retry does not guarantee that the retried transaction will complete; multiple retries may be needed. In cases with very high contention, it is possible that completion of a transaction may take many attempts. In cases involving a conflicting prepared transaction, it may not be possible to make progress until the prepared transaction commits or rolls back.

Caveats

Some commands, currently only TRUNCATE, the table-rewriting forms of ALTER TABLE and REPACK with the CONCURRENTLY option, are not MVCC-safe. This means that after the truncation or rewrite commits, the table will appear empty to concurrent transactions, if they are using a snapshot taken before the command committed. This will only be an issue for a transaction that did not access the table in question before the command started — any transaction that has done so would hold at least an ACCESS SHARE table lock, which would block the truncating or rewriting command until that transaction completes. So these commands will not cause any apparent inconsistency in the table contents for successive queries on the target table, but they could cause visible inconsistency between the contents of the target table and other tables in the database.

Support for the Serializable transaction isolation level has not yet been added to hot standby replication targets (described in ???). The strictest isolation level currently supported in hot standby mode is Repeatable Read. While performing all permanent database writes within Serializable transactions on the primary will ensure that all standbys will eventually reach a consistent state, a Repeatable Read transaction run on the standby can sometimes see a transient state that is inconsistent with any serial execution of the transactions on the primary.

Internal access to the system catalogs is not done using the isolation level of the current transaction. This means that newly created database objects such as tables are visible to concurrent Repeatable Read and Serializable transactions, even though the rows they contain are not. In contrast, queries that explicitly examine the system catalogs don't see rows representing concurrently created database objects, in the higher isolation levels.

Locking and Indexes

index

locks

Though PostgreSQL provides nonblocking read/write access to table data, nonblocking read/write access is not currently offered for every index access method implemented in PostgreSQL. The various index types are handled as follows:

B-tree, GiST and SP-GiST indexes
Short-term share/exclusive page-level locks are used for read/write access. Locks are released immediately after each index row is fetched or inserted. These index types provide the highest concurrency without deadlock conditions.

Hash indexes
Share/exclusive hash-bucket-level locks are used for read/write access. Locks are released after the whole bucket is processed. Bucket-level locks provide better concurrency than index-level ones, but deadlock is possible since the locks are held longer than one index operation.

GIN indexes
Short-term share/exclusive page-level locks are used for read/write access. Locks are released immediately after each index row is fetched or inserted. But note that insertion of a GIN-indexed value usually produces several index key insertions per row, so GIN might do substantial work for a single value's insertion.

Currently, B-tree indexes offer the best performance for concurrent applications; since they also have more features than hash indexes, they are the recommended index type for concurrent applications that need to index scalar data. When dealing with non-scalar data, B-trees are not useful, and GiST, SP-GiST or GIN indexes should be used instead.

Conventions

The following conventions are used in the synopsis of a command: brackets ([ and ]) indicate optional parts. Braces ({ and }) and vertical lines (|) indicate that you must choose one alternative. Dots (...) mean that the preceding element can be repeated. All other symbols, including parentheses, should be taken literally.

Where it enhances the clarity, SQL commands are preceded by the prompt =>, and shell commands are preceded by the prompt $. Normally, prompts are not shown, though.

An administrator is generally a person who is in charge of installing and running the server. A user could be anyone who is using, or wants to use, any part of the PostgreSQL system. These terms should not be interpreted too narrowly; this book does not have fixed presumptions about system administration procedures.

Queries

query

SELECT

The previous chapters explained how to create tables, how to fill them with data, and how to manipulate that data. Now we finally discuss how to retrieve the data from the database.

Overview

The process of retrieving or the command to retrieve data from a database is called a query. In SQL the SELECT command is used to specify queries. The general syntax of the SELECT command is [WITH <with_queries>] SELECT <select_list> FROM <table_expression> [<sort_specification>] The following sections describe the details of the select list, the table expression, and the sort specification. WITH queries are treated last since they are an advanced feature.

A simple kind of query has the form:

SELECT * FROM table1;

Assuming that there is a table called table1, this command would retrieve all rows and all user-defined columns from table1. (The method of retrieval depends on the client application. For example, the psql program will display an ASCII-art table on the screen, while client libraries will offer functions to extract individual values from the query result.) The select list specification * means all columns that the table expression happens to provide. A select list can also select a subset of the available columns or make calculations using the columns. For example, if table1 has columns named a, b, and c (and perhaps others) you can make the following query:

SELECT a, b + c FROM table1;

(assuming that b and c are of a numerical data type). See Select Lists for more details.

FROM table1 is a simple kind of table expression: it reads just one table. In general, table expressions can be complex constructs of base tables, joins, and subqueries. But you can also omit the table expression entirely and use the SELECT command as a calculator:

SELECT 3 * 4;

This is more useful if the expressions in the select list return varying results. For example, you could call a function this way:

SELECT random();

Table Expressions

table expression

A table expression computes a table. The table expression contains a FROM clause that is optionally followed by WHERE, GROUP BY, and HAVING clauses. Trivial table expressions simply refer to a table on disk, a so-called base table, but more complex expressions can be used to modify or combine base tables in various ways.

The optional WHERE, GROUP BY, and HAVING clauses in the table expression specify a pipeline of successive transformations performed on the table derived in the FROM clause. All these transformations produce a virtual table that provides the rows that are passed to the select list to compute the output rows of the query.

The FROM Clause

The FROM clause derives a table from one or more other tables given in a comma-separated table reference list. FROM <table_reference> [, <table_reference> [, ...]] A table reference can be a table name (possibly schema-qualified), or a derived table such as a subquery, a JOIN construct, or complex combinations of these. If more than one table reference is listed in the FROM clause, the tables are cross-joined (that is, the Cartesian product of their rows is formed; see below). The result of the FROM list is an intermediate virtual table that can then be subject to transformations by the WHERE, GROUP BY, and HAVING clauses and is finally the result of the overall table expression.

ONLY

When a table reference names a table that is the parent of a table inheritance hierarchy, the table reference produces rows of not only that table but all of its descendant tables, unless the key word ONLY precedes the table name. However, the reference produces only the columns that appear in the named table — any columns added in subtables are ignored.

You can write * after the table name to explicitly specify that descendant tables are included, which is the opposite of ONLY. There is no real reason to write * any more, because searching descendant tables has been the default behavior since PostgreSQL version 10. However, it remains supported for compatibility with older releases.

Joined Tables

join

A joined table is a table derived from two other (real or derived) tables according to the rules of the particular join type. Inner, outer, and cross-joins are available. The general syntax of a joined table is <T1> <join_type> <T2> [<join_condition>] Joins of all types can be chained together, or nested: either or both <T1> and <T2> can be joined tables. Parentheses can be used around JOIN clauses to control the join order. In the absence of parentheses, JOIN clauses nest left-to-right.

Cross join
T1

CROSS JOIN

T2

For every possible combination of rows from <T1> and <T2> (i.e., a Cartesian product), the joined table will contain a row consisting of all columns in <T1> followed by all columns in <T2>. If the tables have N and M rows respectively, the joined table will have N * M rows.

FROM T1 CROSS JOIN T2 is equivalent to FROM T1 INNER JOIN T2 ON TRUE (see below). It is also equivalent to FROM T1, T2.

[!NOTE] This latter equivalence does not hold exactly when more than two tables appear, because JOIN binds more tightly than comma. For example FROM T1 CROSS JOIN T2 INNER JOIN T3 ON condition is not the same as FROM T1, T2 INNER JOIN T3 ON condition because the <condition> can reference <T1> in the first case but not the second.

Qualified joins
T1

{

INNER

| { LEFT | RIGHT | FULL }

OUTER

} JOIN

T2

ON

boolean_expression

T1

{

INNER

| { LEFT | RIGHT | FULL }

OUTER

} JOIN

T2

USING (

join column list

)

T1

NATURAL {

INNER

| { LEFT | RIGHT | FULL }

OUTER

} JOIN

T2

The words INNER and OUTER are optional in all forms. INNER is the default; LEFT, RIGHT, and FULL imply an outer join.

The join condition is specified in the ON or USING clause, or implicitly by the word NATURAL. The join condition determines which rows from the two source tables are considered to “match”, as explained in detail below.

The possible types of qualified join are:

INNER JOIN
For each row R1 of T1, the joined table has a row for each row in T2 that satisfies the join condition with R1.

LEFT OUTER JOIN
First, an inner join is performed. Then, for each row in T1 that does not satisfy the join condition with any row in T2, a joined row is added with null values in columns of T2. Thus, the joined table always has at least one row for each row in T1.

RIGHT OUTER JOIN
First, an inner join is performed. Then, for each row in T2 that does not satisfy the join condition with any row in T1, a joined row is added with null values in columns of T1. This is the converse of a left join: the result table will always have a row for each row in T2.

FULL OUTER JOIN
First, an inner join is performed. Then, for each row in T1 that does not satisfy the join condition with any row in T2, a joined row is added with null values in columns of T2. Also, for each row of T2 that does not satisfy the join condition with any row in T1, a joined row with null values in the columns of T1 is added.

The ON clause is the most general kind of join condition: it takes a Boolean value expression of the same kind as is used in a WHERE clause. A pair of rows from <T1> and <T2> match if the ON expression evaluates to true.

The USING clause is a shorthand that allows you to take advantage of the specific situation where both sides of the join use the same name for the joining column(s). It takes a comma-separated list of the shared column names and forms a join condition that includes an equality comparison for each one. For example, joining <T1> and <T2> with USING (a, b) produces the join condition ON T1.a = T2.a AND T1.b = T2.b.

Furthermore, the output of JOIN USING suppresses redundant columns: there is no need to print both of the matched columns, since they must have equal values. While JOIN ON produces all columns from <T1> followed by all columns from <T2>, JOIN USING produces one output column for each of the listed column pairs (in the listed order), followed by any remaining columns from <T1>, followed by any remaining columns from <T2>.

Finally, NATURAL is a shorthand form of USING: it forms a USING list consisting of all column names that appear in both input tables. As with USING, these columns appear only once in the output table. If there are no common column names, NATURAL JOIN behaves like CROSS JOIN.

[!NOTE] USING is reasonably safe from column changes in the joined relations since only the listed columns are combined. NATURAL is considerably more risky since any schema changes to either relation that cause a new matching column name to be present will cause the join to combine that new column as well.

To put this together, assume we have tables t1:

 num | name
-----+------
   1 | a
   2 | b
   3 | c

and t2:

 num | value
-----+-------
   1 | xxx
   3 | yyy
   5 | zzz

then we get the following results for the various joins:

=> SELECT * FROM t1 CROSS JOIN t2;
 num | name | num | value
-----+------+-----+-------
   1 | a    |   1 | xxx
   1 | a    |   3 | yyy
   1 | a    |   5 | zzz
   2 | b    |   1 | xxx
   2 | b    |   3 | yyy
   2 | b    |   5 | zzz
   3 | c    |   1 | xxx
   3 | c    |   3 | yyy
   3 | c    |   5 | zzz
(9 rows)

=> SELECT * FROM t1 INNER JOIN t2 ON t1.num = t2.num;
 num | name | num | value
-----+------+-----+-------
   1 | a    |   1 | xxx
   3 | c    |   3 | yyy
(2 rows)

=> SELECT * FROM t1 INNER JOIN t2 USING (num);
 num | name | value
-----+------+-------
   1 | a    | xxx
   3 | c    | yyy
(2 rows)

=> SELECT * FROM t1 NATURAL INNER JOIN t2;
 num | name | value
-----+------+-------
   1 | a    | xxx
   3 | c    | yyy
(2 rows)

=> SELECT * FROM t1 LEFT JOIN t2 ON t1.num = t2.num;
 num | name | num | value
-----+------+-----+-------
   1 | a    |   1 | xxx
   2 | b    |     |
   3 | c    |   3 | yyy
(3 rows)

=> SELECT * FROM t1 LEFT JOIN t2 USING (num);
 num | name | value
-----+------+-------
   1 | a    | xxx
   2 | b    |
   3 | c    | yyy
(3 rows)

=> SELECT * FROM t1 RIGHT JOIN t2 ON t1.num = t2.num;
 num | name | num | value
-----+------+-----+-------
   1 | a    |   1 | xxx
   3 | c    |   3 | yyy
     |      |   5 | zzz
(3 rows)

=> SELECT * FROM t1 FULL JOIN t2 ON t1.num = t2.num;
 num | name | num | value
-----+------+-----+-------
   1 | a    |   1 | xxx
   2 | b    |     |
   3 | c    |   3 | yyy
     |      |   5 | zzz
(4 rows)

The join condition specified with ON can also contain conditions that do not relate directly to the join. This can prove useful for some queries but needs to be thought out carefully. For example:

=> SELECT * FROM t1 LEFT JOIN t2 ON t1.num = t2.num AND t2.value = 'xxx';
 num | name | num | value
-----+------+-----+-------
   1 | a    |   1 | xxx
   2 | b    |     |
   3 | c    |     |
(3 rows)

Notice that placing the restriction in the WHERE clause produces a different result:

=> SELECT * FROM t1 LEFT JOIN t2 ON t1.num = t2.num WHERE t2.value = 'xxx';
 num | name | num | value
-----+------+-----+-------
   1 | a    |   1 | xxx
(1 row)

This is because a restriction placed in the ON clause is processed before the join, while a restriction placed in the WHERE clause is processed after the join. That does not matter with inner joins, but it matters a lot with outer joins.

Table and Column Aliases

alias

in the FROM clause

label

alias

A temporary name can be given to tables and complex table references to be used for references to the derived table in the rest of the query. This is called a table alias.

To create a table alias, write FROM <table_reference> AS <alias> or FROM <table_reference> <alias> The AS key word is optional noise. <alias> can be any identifier.

A typical application of table aliases is to assign short identifiers to long table names to keep the join clauses readable. For example:

SELECT * FROM some_very_long_table_name s JOIN another_fairly_long_name a ON s.id = a.num;

The alias becomes the new name of the table reference so far as the current query is concerned — it is not allowed to refer to the table by the original name elsewhere in the query. Thus, this is not valid:

SELECT * FROM my_table AS m WHERE my_table.a > 5;    -- wrong

Table aliases are mainly for notational convenience, but it is necessary to use them when joining a table to itself, e.g.:

SELECT * FROM people AS mother JOIN people AS child ON mother.id = child.mother_id;

Parentheses are used to resolve ambiguities. In the following example, the first statement assigns the alias b to the second instance of my_table, but the second statement assigns the alias to the result of the join:

SELECT * FROM my_table AS a CROSS JOIN my_table AS b ...
SELECT * FROM (my_table AS a CROSS JOIN my_table) AS b ...

Another form of table aliasing gives temporary names to the columns of the table, as well as the table itself: FROM <table_reference> [AS] <alias> ( <column1> [, <column2> [, ...]] ) If fewer column aliases are specified than the actual table has columns, the remaining columns are not renamed. This syntax is especially useful for self-joins or subqueries.

When an alias is applied to the output of a JOIN clause, the alias hides the original name(s) within the JOIN. For example:

SELECT a.* FROM my_table AS a JOIN your_table AS b ON ...

is valid SQL, but:

SELECT a.* FROM (my_table AS a JOIN your_table AS b ON ...) AS c

is not valid; the table alias a is not visible outside the alias c.

Subqueries

subquery

Subqueries specifying a derived table must be enclosed in parentheses. They may be assigned a table alias name, and optionally column alias names (as in Table and Column Aliases). For example:

FROM (SELECT * FROM table1) AS alias_name

This example is equivalent to FROM table1 AS alias_name. More interesting cases, which cannot be reduced to a plain join, arise when the subquery involves grouping or aggregation.

A subquery can also be a VALUES list:

FROM (VALUES ('anne', 'smith'), ('bob', 'jones'), ('joe', 'blow'))
     AS names(first, last)

Again, a table alias is optional. Assigning alias names to the columns of the VALUES list is optional, but is good practice. For more information see Lists.

According to the SQL standard, a table alias name must be supplied for a subquery. PostgreSQL allows AS and the alias to be omitted, but writing one is good practice in SQL code that might be ported to another system.

Table Functions

table function

function

in the FROM clause

Table functions are functions that produce a set of rows, made up of either base data types (scalar types) or composite data types (table rows). They are used like a table, view, or subquery in the FROM clause of a query. Columns returned by table functions can be included in SELECT, JOIN, or WHERE clauses in the same manner as columns of a table, view, or subquery.

Table functions may also be combined using the ROWS FROM syntax, with the results returned in parallel columns; the number of result rows in this case is that of the largest function result, with smaller results padded with null values to match.

function_call

WITH ORDINALITY

AS

table_alias

(

column_alias

, ...

)

ROWS FROM(

function_call

, ...

)

WITH ORDINALITY

AS

table_alias

(

column_alias

, ...

)

If the WITH ORDINALITY clause is specified, an additional column of type bigint will be added to the function result columns. This column numbers the rows of the function result set, starting from 1. (This is a generalization of the SQL-standard syntax for UNNEST ... WITH ORDINALITY.) By default, the ordinal column is called ordinality, but a different column name can be assigned to it using an AS clause.

The special table function UNNEST may be called with any number of array parameters, and it returns a corresponding number of columns, as if UNNEST (???) had been called on each parameter separately and combined using the ROWS FROM construct.

UNNEST(

array_expression

, ...

)

WITH ORDINALITY

AS

table_alias

(

column_alias

, ...

)

If no <table_alias> is specified, the function name is used as the table name; in the case of a ROWS FROM() construct, the first function's name is used.

If column aliases are not supplied, then for a function returning a base data type, the column name is also the same as the function name. For a function returning a composite type, the result columns get the names of the individual attributes of the type.

Some examples:

CREATE TABLE foo (fooid int, foosubid int, fooname text);

CREATE FUNCTION getfoo(int) RETURNS SETOF foo AS $$
    SELECT * FROM foo WHERE fooid = $1;
$$ LANGUAGE SQL;

SELECT * FROM getfoo(1) AS t1;

SELECT * FROM foo
    WHERE foosubid IN (
                        SELECT foosubid
                        FROM getfoo(foo.fooid) z
                        WHERE z.fooid = foo.fooid
                      );

CREATE VIEW vw_getfoo AS SELECT * FROM getfoo(1);

SELECT * FROM vw_getfoo;

In some cases it is useful to define table functions that can return different column sets depending on how they are invoked. To support this, the table function can be declared as returning the pseudo-type record with no OUT parameters. When such a function is used in a query, the expected row structure must be specified in the query itself, so that the system can know how to parse and plan the query. This syntax looks like:

function_call

AS

alias

(

column_definition

, ...

)

function_call

AS

alias

(

column_definition

, ...

) ROWS FROM( ...

function_call

AS (

column_definition

, ...

)

, ...

)

When not using the ROWS FROM() syntax, the <column_definition> list replaces the column alias list that could otherwise be attached to the FROM item; the names in the column definitions serve as column aliases. When using the ROWS FROM() syntax, a <column_definition> list can be attached to each member function separately; or if there is only one member function and no WITH ORDINALITY clause, a <column_definition> list can be written in place of a column alias list following ROWS FROM().

Consider this example:

The ??? function (part of the ??? module) executes a remote query. It is declared to return record since it might be used for any kind of query. The actual column set must be specified in the calling query so that the parser knows, for example, what * should expand to.

This example uses ROWS FROM:

SELECT *
FROM ROWS FROM
    (
        json_to_recordset('[{"a":40,"b":"foo"},{"a":"100","b":"bar"}]')
            AS (a INTEGER, b TEXT),
        generate_series(1, 3)
    ) AS x (p, q, s)
ORDER BY p;

  p  |  q  | s
-----+-----+---
  40 | foo | 1
 100 | bar | 2
     |     | 3

It joins two functions into a single FROM target. json_to_recordset() is instructed to return two columns, the first integer and the second text. The result of generate_series() is used directly. The ORDER BY clause sorts the column values as integers.

LATERAL Subqueries

LATERAL

in the FROM clause

Subqueries appearing in FROM can be preceded by the key word LATERAL. This allows them to reference columns provided by preceding FROM items. (Without LATERAL, each subquery is evaluated independently and so cannot cross-reference any other FROM item.)

Table functions appearing in FROM can also be preceded by the key word LATERAL, but for functions the key word is optional; the function's arguments can contain references to columns provided by preceding FROM items in any case.

A LATERAL item can appear at the top level in the FROM list, or within a JOIN tree. In the latter case it can also refer to any items that are on the left-hand side of a JOIN that it is on the right-hand side of.

When a FROM item contains LATERAL cross-references, evaluation proceeds as follows: for each row of the FROM item providing the cross-referenced column(s), or set of rows of multiple FROM items providing the columns, the LATERAL item is evaluated using that row or row set's values of the columns. The resulting row(s) are joined as usual with the rows they were computed from. This is repeated for each row or set of rows from the column source table(s).

A trivial example of LATERAL is

SELECT * FROM foo, LATERAL (SELECT * FROM bar WHERE bar.id = foo.bar_id) ss;

This is not especially useful since it has exactly the same result as the more conventional

SELECT * FROM foo, bar WHERE bar.id = foo.bar_id;

LATERAL is primarily useful when the cross-referenced column is necessary for computing the row(s) to be joined. A common application is providing an argument value for a set-returning function. For example, supposing that vertices(polygon) returns the set of vertices of a polygon, we could identify close-together vertices of polygons stored in a table with:

SELECT p1.id, p2.id, v1, v2
FROM polygons p1, polygons p2,
     LATERAL vertices(p1.poly) v1,
     LATERAL vertices(p2.poly) v2
WHERE (v1 <-> v2) < 10 AND p1.id != p2.id;

This query could also be written

SELECT p1.id, p2.id, v1, v2
FROM polygons p1 CROSS JOIN LATERAL vertices(p1.poly) v1,
     polygons p2 CROSS JOIN LATERAL vertices(p2.poly) v2
WHERE (v1 <-> v2) < 10 AND p1.id != p2.id;

or in several other equivalent formulations. (As already mentioned, the LATERAL key word is unnecessary in this example, but we use it for clarity.)

It is often particularly handy to LEFT JOIN to a LATERAL subquery, so that source rows will appear in the result even if the LATERAL subquery produces no rows for them. For example, if get_product_names() returns the names of products made by a manufacturer, but some manufacturers in our table currently produce no products, we could find out which ones those are like this:

SELECT m.name
FROM manufacturers m LEFT JOIN LATERAL get_product_names(m.id) pname ON true
WHERE pname IS NULL;

The WHERE Clause

WHERE

The syntax of the WHERE clause is WHERE <search_condition> where <search_condition> is any value expression (see ???) that returns a value of type boolean.

After the processing of the FROM clause is done, each row of the derived virtual table is checked against the search condition. If the result of the condition is true, the row is kept in the output table, otherwise (i.e., if the result is false or null) it is discarded. The search condition typically references at least one column of the table generated in the FROM clause; this is not required, but otherwise the WHERE clause will be fairly useless.

[!NOTE] The join condition of an inner join can be written either in the WHERE clause or in the JOIN clause. For example, these table expressions are equivalent:

FROM a, b WHERE a.id = b.id AND b.val > 5

and:

FROM a INNER JOIN b ON (a.id = b.id) WHERE b.val > 5

or perhaps even:

FROM a NATURAL JOIN b WHERE b.val > 5

Which one of these you use is mainly a matter of style. The JOIN syntax in the FROM clause is probably not as portable to other SQL database management systems, even though it is in the SQL standard. For outer joins there is no choice: they must be done in the FROM clause. The ON or USING clause of an outer join is not equivalent to a WHERE condition, because it results in the addition of rows (for unmatched input rows) as well as the removal of rows in the final result.

Here are some examples of WHERE clauses:

SELECT ... FROM fdt WHERE c1 > 5

SELECT ... FROM fdt WHERE c1 IN (1, 2, 3)

SELECT ... FROM fdt WHERE c1 IN (SELECT c1 FROM t2)

SELECT ... FROM fdt WHERE c1 IN (SELECT c3 FROM t2 WHERE c2 = fdt.c1 + 10)

SELECT ... FROM fdt WHERE c1 BETWEEN (SELECT c3 FROM t2 WHERE c2 = fdt.c1 + 10) AND 100

SELECT ... FROM fdt WHERE EXISTS (SELECT c1 FROM t2 WHERE c2 > fdt.c1)

fdt is the table derived in the FROM clause. Rows that do not meet the search condition of the WHERE clause are eliminated from fdt. Notice the use of scalar subqueries as value expressions. Just like any other query, the subqueries can employ complex table expressions. Notice also how fdt is referenced in the subqueries. Qualifying c1 as fdt.c1 is only necessary if c1 is also the name of a column in the derived input table of the subquery. But qualifying the column name adds clarity even when it is not needed. This example shows how the column naming scope of an outer query extends into its inner queries.

The GROUP BY and HAVING Clauses

GROUP BY

grouping

After passing the WHERE filter, the derived input table might be subject to grouping, using the GROUP BY clause, and elimination of group rows using the HAVING clause.

SELECT

select_list

FROM ...

WHERE ...

GROUP BY

grouping_column_reference

,

grouping_column_reference

...

The GROUP BY clause is used to group together those rows in a table that have the same values in all the columns listed. The order in which the columns are listed does not matter. The effect is to combine each set of rows having common values into one group row that represents all rows in the group. This is done to eliminate redundancy in the output and/or compute aggregates that apply to these groups. For instance:

=> SELECT * FROM test1;
 x | y
---+---
 a | 3
 c | 2
 b | 5
 a | 1
(4 rows)

=> SELECT x FROM test1 GROUP BY x;
 x
---
 a
 b
 c
(3 rows)

In the second query, we could not have written SELECT * FROM test1 GROUP BY x, because there is no single value for the column y that could be associated with each group. The grouped-by columns can be referenced in the select list since they have a single value in each group.

In general, if a table is grouped, columns that are not listed in GROUP BY cannot be referenced except in aggregate expressions. An example with aggregate expressions is:

=> SELECT x, sum(y) FROM test1 GROUP BY x;
 x | sum
---+-----
 a |   4
 b |   5
 c |   2
(3 rows)

Here sum is an aggregate function that computes a single value over the entire group. More information about the available aggregate functions can be found in ???.

[!TIP] Grouping without aggregate expressions effectively calculates the set of distinct values in a column. This can also be achieved using the DISTINCT clause (see ).

Here is another example: it calculates the total sales for each product (rather than the total sales of all products):

SELECT product_id, p.name, (sum(s.units) * p.price) AS sales
    FROM products p LEFT JOIN sales s USING (product_id)
    GROUP BY product_id, p.name, p.price;

In this example, the columns product_id, p.name, and p.price must be in the GROUP BY clause since they are referenced in the query select list (but see below). The column s.units does not have to be in the GROUP BY list since it is only used in an aggregate expression (sum(...)), which represents the sales of a product. For each product, the query returns a summary row about all sales of the product.

functional dependency

If the products table is set up so that, say, product_id is the primary key, then it would be enough to group by product_id in the above example, since name and price would be functionally dependent on the product ID, and so there would be no ambiguity about which name and price value to return for each product ID group.

In strict SQL, GROUP BY can only group by columns of the source table, but PostgreSQL extends this to also allow GROUP BY to group by columns in the select list. Grouping by value expressions instead of simple column names is also allowed (but GROUP BY expressions cannot contain aggregate functions or window functions).

HAVING

If a table has been grouped using GROUP BY, but only certain groups are of interest, the HAVING clause can be used, much like a WHERE clause, to eliminate groups from the result. The syntax is: SELECT <select_list> FROM ... [WHERE ...] GROUP BY ... HAVING <boolean_expression> Expressions in the HAVING clause can refer both to grouped expressions and to ungrouped expressions (which necessarily involve an aggregate function).

Example:

=> SELECT x, sum(y) FROM test1 GROUP BY x HAVING sum(y) > 3;
 x | sum
---+-----
 a |   4
 b |   5
(2 rows)

=> SELECT x, sum(y) FROM test1 GROUP BY x HAVING x < 'c';
 x | sum
---+-----
 a |   4
 b |   5
(2 rows)

Again, a more realistic example:

SELECT product_id, p.name, (sum(s.units) * (p.price - p.cost)) AS profit
    FROM products p LEFT JOIN sales s USING (product_id)
    WHERE s.date > CURRENT_DATE - INTERVAL '4 weeks'
    GROUP BY product_id, p.name, p.price, p.cost
    HAVING sum(p.price * s.units) > 5000;

In the example above, the WHERE clause is selecting rows by a column that is not grouped (the expression is only true for sales during the last four weeks), while the HAVING clause restricts the output to groups with total gross sales over 5000. Note that the aggregate expressions do not necessarily need to be the same in all parts of the query.

If a query contains aggregate function calls, but no GROUP BY clause, grouping still occurs: the result is a single group row (or perhaps no rows at all, if the single row is then eliminated by HAVING). The same is true if it contains a HAVING clause, even without any aggregate function calls or GROUP BY clause.

GROUPING SETS, CUBE, and ROLLUP

GROUPING SETS

CUBE

ROLLUP

More complex grouping operations than those described above are possible using the concept of grouping sets. The data selected by the FROM and WHERE clauses is grouped separately by each specified grouping set, aggregates computed for each group just as for simple GROUP BY clauses, and then the results returned. For example:

=> SELECT * FROM items_sold;
 brand | size | sales
-------+------+-------
 Foo   | L    |  10
 Foo   | M    |  20
 Bar   | M    |  15
 Bar   | L    |  5
(4 rows)

=> SELECT brand, size, sum(sales) FROM items_sold GROUP BY GROUPING SETS ((brand), (size), ());
 brand | size | sum
-------+------+-----
 Foo   |      |  30
 Bar   |      |  20
       | L    |  15
       | M    |  35
       |      |  50
(5 rows)

Each sublist of GROUPING SETS may specify zero or more columns or expressions and is interpreted the same way as though it were directly in the GROUP BY clause. An empty grouping set means that all rows are aggregated down to a single group (which is output even if no input rows were present), as described above for the case of aggregate functions with no GROUP BY clause.

References to the grouping columns or expressions are replaced by null values in result rows for grouping sets in which those columns do not appear. To distinguish which grouping a particular output row resulted from, see ???.

A shorthand notation is provided for specifying two common types of grouping set. A clause of the form

ROLLUP ( e1, e2, e3, ... )

represents the given list of expressions and all prefixes of the list including the empty list; thus it is equivalent to

GROUPING SETS (
    ( e1, e2, e3, ... ),
    ...
    ( e1, e2 ),
    ( e1 ),
    ( )
)

This is commonly used for analysis over hierarchical data; e.g., total salary by department, division, and company-wide total.

A clause of the form

CUBE ( e1, e2, ... )

represents the given list and all of its possible subsets (i.e., the power set). Thus

CUBE ( a, b, c )

is equivalent to

GROUPING SETS (
    ( a, b, c ),
    ( a, b    ),
    ( a,    c ),
    ( a       ),
    (    b, c ),
    (    b    ),
    (       c ),
    (         )
)

The individual elements of a CUBE or ROLLUP clause may be either individual expressions, or sublists of elements in parentheses. In the latter case, the sublists are treated as single units for the purposes of generating the individual grouping sets. For example:

CUBE ( (a, b), (c, d) )

is equivalent to

GROUPING SETS (
    ( a, b, c, d ),
    ( a, b       ),
    (       c, d ),
    (            )
)

and

ROLLUP ( a, (b, c), d )

is equivalent to

GROUPING SETS (
    ( a, b, c, d ),
    ( a, b, c    ),
    ( a          ),
    (            )
)

The CUBE and ROLLUP constructs can be used either directly in the GROUP BY clause, or nested inside a GROUPING SETS clause. If one GROUPING SETS clause is nested inside another, the effect is the same as if all the elements of the inner clause had been written directly in the outer clause.

If multiple grouping items are specified in a single GROUP BY clause, then the final list of grouping sets is the Cartesian product of the individual items. For example:

GROUP BY a, CUBE (b, c), GROUPING SETS ((d), (e))

is equivalent to

GROUP BY GROUPING SETS (
    (a, b, c, d), (a, b, c, e),
    (a, b, d),    (a, b, e),
    (a, c, d),    (a, c, e),
    (a, d),       (a, e)
)

When specifying multiple grouping items together, the final set of grouping sets might contain duplicates. For example:

GROUP BY ROLLUP (a, b), ROLLUP (a, c)

is equivalent to

GROUP BY GROUPING SETS (
    (a, b, c),
    (a, b),
    (a, b),
    (a, c),
    (a),
    (a),
    (a, c),
    (a),
    ()
)

If these duplicates are undesirable, they can be removed using the DISTINCT clause directly on the GROUP BY. Therefore:

GROUP BY DISTINCT ROLLUP (a, b), ROLLUP (a, c)

is equivalent to

GROUP BY GROUPING SETS (
    (a, b, c),
    (a, b),
    (a, c),
    (a),
    ()
)

This is not the same as using SELECT DISTINCT because the output rows may still contain duplicates. If any of the ungrouped columns contains NULL, it will be indistinguishable from the NULL used when that same column is grouped.

[!NOTE] The construct (a, b) is normally recognized in expressions as a row constructor. Within the GROUP BY clause, this does not apply at the top levels of expressions, and (a, b) is parsed as a list of expressions as described above. If for some reason you need a row constructor in a grouping expression, use ROW(a, b).

Window Function Processing

window function

order of execution

If the query contains any window functions (see ???, ??? and ???), these functions are evaluated after any grouping, aggregation, and HAVING filtering is performed. That is, if the query uses any aggregates, GROUP BY, or HAVING, then the rows seen by the window functions are the group rows instead of the original table rows from FROM/WHERE.

When multiple window functions are used, all the window functions having equivalent PARTITION BY and ORDER BY clauses in their window definitions are guaranteed to see the same ordering of the input rows, even if the ORDER BY does not uniquely determine the ordering. However, no guarantees are made about the evaluation of functions having different PARTITION BY or ORDER BY specifications. (In such cases a sort step is typically required between the passes of window function evaluations, and the sort is not guaranteed to preserve ordering of rows that its ORDER BY sees as equivalent.)

Currently, window functions always require presorted data, and so the query output will be ordered according to one or another of the window functions' PARTITION BY/ORDER BY clauses. It is not recommended to rely on this, however. Use an explicit top-level ORDER BY clause if you want to be sure the results are sorted in a particular way.

Select Lists

SELECT

select list

As shown in the previous section, the table expression in the SELECT command constructs an intermediate virtual table by possibly combining tables, views, eliminating rows, grouping, etc. This table is finally passed on to processing by the select list. The select list determines which columns of the intermediate table are actually output.

Select-List Items

*

The simplest kind of select list is * which emits all columns that the table expression produces. Otherwise, a select list is a comma-separated list of value expressions (as defined in ???). For instance, it could be a list of column names:

SELECT a, b, c FROM ...

The columns names a, b, and c are either the actual names of the columns of tables referenced in the FROM clause, or the aliases given to them as explained in Table and Column Aliases. The name space available in the select list is the same as in the WHERE clause, unless grouping is used, in which case it is the same as in the HAVING clause.

If more than one table has a column of the same name, the table name must also be given, as in:

SELECT tbl1.a, tbl2.a, tbl1.b FROM ...

When working with multiple tables, it can also be useful to ask for all the columns of a particular table:

SELECT tbl1.*, tbl2.a FROM ...

See ??? for more about the <table_name>.* notation.

If an arbitrary value expression is used in the select list, it conceptually adds a new virtual column to the returned table. The value expression is evaluated once for each result row, with the row's values substituted for any column references. But the expressions in the select list do not have to reference any columns in the table expression of the FROM clause; they can be constant arithmetic expressions, for instance.

Column Labels

alias

in the select list

The entries in the select list can be assigned names for subsequent processing, such as for use in an ORDER BY clause or for display by the client application. For example:

SELECT a AS value, b + c AS sum FROM ...

If no output column name is specified using AS, the system assigns a default column name. For simple column references, this is the name of the referenced column. For function calls, this is the name of the function. For complex expressions, the system will generate a generic name.

The AS key word is usually optional, but in some cases where the desired column name matches a PostgreSQL key word, you must write AS or double-quote the column name in order to avoid ambiguity. (??? shows which key words require AS to be used as a column label.) For example, FROM is one such key word, so this does not work:

SELECT a from, b + c AS sum FROM ...

but either of these do:

SELECT a AS from, b + c AS sum FROM ...
SELECT a "from", b + c AS sum FROM ...

For greatest safety against possible future key word additions, it is recommended that you always either write AS or double-quote the output column name.

[!NOTE] The naming of output columns here is different from that done in the FROM clause (see Table and Column Aliases). It is possible to rename the same column twice, but the name assigned in the select list is the one that will be passed on.

DISTINCT

ALL

SELECT ALL

DISTINCT

SELECT DISTINCT

duplicates

After the select list has been processed, the result table can optionally be subject to the elimination of duplicate rows. The DISTINCT key word is written directly after SELECT to specify this: SELECT DISTINCT <select_list> ... (Instead of DISTINCT the key word ALL can be used to specify the default behavior of retaining all rows.)

null value

in DISTINCT

Obviously, two rows are considered distinct if they differ in at least one column value. Null values are considered equal in this comparison.

Alternatively, an arbitrary expression can determine what rows are to be considered distinct: SELECT DISTINCT ON (<expression> [, <expression> ...]) <select_list> ... Here <expression> is an arbitrary value expression that is evaluated for all rows. A set of rows for which all the expressions are equal are considered duplicates, and only the first row of the set is kept in the output. Note that the “first row” of a set is unpredictable unless the query is sorted on enough columns to guarantee a unique ordering of the rows arriving at the DISTINCT filter. (DISTINCT ON processing occurs after ORDER BY sorting.)

The DISTINCT ON clause is not part of the SQL standard and is sometimes considered bad style because of the potentially indeterminate nature of its results. With judicious use of GROUP BY and subqueries in FROM, this construct can be avoided, but it is often the most convenient alternative.

Combining Queries (UNION, INTERSECT, EXCEPT)

UNION

INTERSECT

EXCEPT

set union

set intersection

set difference

set operation

The results of two queries can be combined using the set operations union, intersection, and difference. The syntax is <query1> UNION [ALL] <query2> <query1> INTERSECT [ALL] <query2> <query1> EXCEPT [ALL] <query2> where <query1> and <query2> are queries that can use any of the features discussed up to this point.

UNION effectively appends the result of <query2> to the result of <query1> (although there is no guarantee that this is the order in which the rows are actually returned). Furthermore, it eliminates duplicate rows from its result, in the same way as DISTINCT, unless UNION ALL is used.

INTERSECT returns all rows that are both in the result of <query1> and in the result of <query2>. Duplicate rows are eliminated unless INTERSECT ALL is used.

EXCEPT returns all rows that are in the result of <query1> but not in the result of <query2>. (This is sometimes called the difference between two queries.) Again, duplicates are eliminated unless EXCEPT ALL is used.

In order to calculate the union, intersection, or difference of two queries, the two queries must be “union compatible”, which means that they return the same number of columns and the corresponding columns have compatible data types, as described in ???.

Set operations can be combined, for example <query1> UNION <query2> EXCEPT <query3> which is equivalent to (<query1> UNION <query2>) EXCEPT <query3> As shown here, you can use parentheses to control the order of evaluation. Without parentheses, UNION and EXCEPT associate left-to-right, but INTERSECT binds more tightly than those two operators. Thus <query1> UNION <query2> INTERSECT <query3> means <query1> UNION (<query2> INTERSECT <query3>) You can also surround an individual <query> with parentheses. This is important if the <query> needs to use any of the clauses discussed in following sections, such as LIMIT. Without parentheses, you'll get a syntax error, or else the clause will be understood as applying to the output of the set operation rather than one of its inputs. For example, SELECT a FROM b UNION SELECT x FROM y LIMIT 10 is accepted, but it means (SELECT a FROM b UNION SELECT x FROM y) LIMIT 10 not SELECT a FROM b UNION (SELECT x FROM y LIMIT 10)

Sorting Rows (ORDER BY)

sorting

ORDER BY

After a query has produced an output table (after the select list has been processed) it can optionally be sorted. If sorting is not chosen, the rows will be returned in an unspecified order. The actual order in that case will depend on the scan and join plan types and the order on disk, but it must not be relied on. A particular output ordering can only be guaranteed if the sort step is explicitly chosen.

The ORDER BY clause specifies the sort order: SELECT <select_list> FROM <table_expression> ORDER BY <sort_expression1> [ASC | DESC] [NULLS { FIRST | LAST }] [, <sort_expression2> [ASC | DESC] [NULLS { FIRST | LAST }] ...] The sort expression(s) can be any expression that would be valid in the query's select list. An example is:

SELECT a, b FROM table1 ORDER BY a + b, c;

When more than one expression is specified, the later values are used to sort rows that are equal according to the earlier values. Each expression can be followed by an optional ASC or DESC keyword to set the sort direction to ascending or descending. ASC order is the default. Ascending order puts smaller values first, where “smaller” is defined in terms of the < operator. Similarly, descending order is determined with the > operator. [^1]

The NULLS FIRST and NULLS LAST options can be used to determine whether nulls appear before or after non-null values in the sort ordering. By default, null values sort as if larger than any non-null value; that is, NULLS FIRST is the default for DESC order, and NULLS LAST otherwise.

Note that the ordering options are considered independently for each sort column. For example ORDER BY x, y DESC means ORDER BY x ASC, y DESC, which is not the same as ORDER BY x DESC, y DESC.

A <sort_expression> can also be the column label or number of an output column, as in:

SELECT a + b AS sum, c FROM table1 ORDER BY sum;
SELECT a, max(b) FROM table1 GROUP BY a ORDER BY 1;

both of which sort by the first output column. Note that an output column name has to stand alone, that is, it cannot be used in an expression — for example, this is not correct:

SELECT a + b AS sum, c FROM table1 ORDER BY sum + c;          -- wrong

This restriction is made to reduce ambiguity. There is still ambiguity if an ORDER BY item is a simple name that could match either an output column name or a column from the table expression. The output column is used in such cases. This would only cause confusion if you use AS to rename an output column to match some other table column's name.

ORDER BY can be applied to the result of a UNION, INTERSECT, or EXCEPT combination, but in this case it is only permitted to sort by output column names or numbers, not by expressions.

LIMIT and OFFSET

LIMIT

OFFSET

LIMIT and OFFSET allow you to retrieve just a portion of the rows that are generated by the rest of the query: SELECT <select_list> FROM <table_expression> [ORDER BY ...] [LIMIT { <count> | ALL }] [OFFSET <start>]

If a limit count is given, no more than that many rows will be returned (but possibly fewer, if the query itself yields fewer rows). LIMIT ALL is the same as omitting the LIMIT clause, as is LIMIT with a NULL argument.

OFFSET says to skip that many rows before beginning to return rows. OFFSET 0 is the same as omitting the OFFSET clause, as is OFFSET with a NULL argument.

If both OFFSET and LIMIT appear, then OFFSET rows are skipped before starting to count the LIMIT rows that are returned.

When using LIMIT, it is important to use an ORDER BY clause that constrains the result rows into a unique order. Otherwise you will get an unpredictable subset of the query's rows. You might be asking for the tenth through twentieth rows, but tenth through twentieth in what ordering? The ordering is unknown, unless you specified ORDER BY.

The query optimizer takes LIMIT into account when generating query plans, so you are very likely to get different plans (yielding different row orders) depending on what you give for LIMIT and OFFSET. Thus, using different LIMIT/OFFSET values to select different subsets of a query result will give inconsistent results unless you enforce a predictable result ordering with ORDER BY. This is not a bug; it is an inherent consequence of the fact that SQL does not promise to deliver the results of a query in any particular order unless ORDER BY is used to constrain the order.

The rows skipped by an OFFSET clause still have to be computed inside the server; therefore a large OFFSET might be inefficient.

VALUES Lists

VALUES

VALUES provides a way to generate a “constant table” that can be used in a query without having to actually create and populate a table on-disk. The syntax is VALUES ( <expression> [, ...] ) [, ...] Each parenthesized list of expressions generates a row in the table. The lists must all have the same number of elements (i.e., the number of columns in the table), and corresponding entries in each list must have compatible data types. The actual data type assigned to each column of the result is determined using the same rules as for UNION (see ???).

As an example:

VALUES (1, 'one'), (2, 'two'), (3, 'three');

will return a table of two columns and three rows. It's effectively equivalent to:

SELECT 1 AS column1, 'one' AS column2
UNION ALL
SELECT 2, 'two'
UNION ALL
SELECT 3, 'three';

By default, PostgreSQL assigns the names column1, column2, etc. to the columns of a VALUES table. The column names are not specified by the SQL standard and different database systems do it differently, so it's usually better to override the default names with a table alias list, like this:

=> SELECT * FROM (VALUES (1, 'one'), (2, 'two'), (3, 'three')) AS t (num,letter);
 num | letter
-----+--------
   1 | one
   2 | two
   3 | three
(3 rows)

Syntactically, VALUES followed by expression lists is treated as equivalent to: SELECT <select_list> FROM <table_expression> and can appear anywhere a SELECT can. For example, you can use it as part of a UNION, or attach a <sort_specification> (ORDER BY, LIMIT, and/or OFFSET) to it. VALUES is most commonly used as the data source in an INSERT command, and next most commonly as a subquery.

For more information see ???.

WITH Queries (Common Table Expressions)

WITH

in SELECT

common table expression

WITH

WITH provides a way to write auxiliary statements for use in a larger query. These statements, which are often referred to as Common Table Expressions or CTEs, can be thought of as defining temporary tables that exist just for one query. Each auxiliary statement in a WITH clause can be a SELECT, INSERT, UPDATE, DELETE, or MERGE; and the WITH clause itself is attached to a primary statement that can also be a SELECT, INSERT, UPDATE, DELETE, or MERGE.

SELECT in WITH

The basic value of SELECT in WITH is to break down complicated queries into simpler parts. An example is:

WITH regional_sales AS (
    SELECT region, SUM(amount) AS total_sales
    FROM orders
    GROUP BY region
), top_regions AS (
    SELECT region
    FROM regional_sales
    WHERE total_sales > (SELECT SUM(total_sales)/10 FROM regional_sales)
)
SELECT region,
       product,
       SUM(quantity) AS product_units,
       SUM(amount) AS product_sales
FROM orders
WHERE region IN (SELECT region FROM top_regions)
GROUP BY region, product;

which displays per-product sales totals in only the top sales regions. The WITH clause defines two auxiliary statements named regional_sales and top_regions, where the output of regional_sales is used in top_regions and the output of top_regions is used in the primary SELECT query. This example could have been written without WITH, but we'd have needed two levels of nested sub-SELECTs. It's a bit easier to follow this way.

Recursive Queries

The optional RECURSIVE modifier changes WITH from a mere syntactic convenience into a feature that accomplishes things not otherwise possible in standard SQL. Using RECURSIVE, a WITH query can refer to its own output. A very simple example is this query to sum the integers from 1 through 100:

WITH RECURSIVE t(n) AS (
    VALUES (1)
  UNION ALL
    SELECT n+1 FROM t WHERE n < 100
)
SELECT sum(n) FROM t;

The general form of a recursive WITH query is always a non-recursive term, then UNION (or UNION ALL), then a recursive term, where only the recursive term can contain a reference to the query's own output. Such a query is executed as follows:

  1. Evaluate the non-recursive term. For UNION (but not UNION ALL), discard duplicate rows. Include all remaining rows in the result of the recursive query, and also place them in a temporary working table.

  2. So long as the working table is not empty, repeat these steps:

    1. Evaluate the recursive term, substituting the current contents of the working table for the recursive self-reference. For UNION (but not UNION ALL), discard duplicate rows and rows that duplicate any previous result row. Include all remaining rows in the result of the recursive query, and also place them in a temporary intermediate table.

    2. Replace the contents of the working table with the contents of the intermediate table, then empty the intermediate table.

[!NOTE] While RECURSIVE allows queries to be specified recursively, internally such queries are evaluated iteratively.

In the example above, the working table has just a single row in each step, and it takes on the values from 1 through 100 in successive steps. In the 100th step, there is no output because of the WHERE clause, and so the query terminates.

Recursive queries are typically used to deal with hierarchical or tree-structured data. A useful example is this query to find all the direct and indirect sub-parts of a product, given only a table that shows immediate inclusions:

WITH RECURSIVE included_parts(sub_part, part, quantity) AS (
    SELECT sub_part, part, quantity FROM parts WHERE part = 'our_product'
  UNION ALL
    SELECT p.sub_part, p.part, p.quantity * pr.quantity
    FROM included_parts pr, parts p
    WHERE p.part = pr.sub_part
)
SELECT sub_part, SUM(quantity) AS total_quantity
FROM included_parts
GROUP BY sub_part

Search Order

When computing a tree traversal using a recursive query, you might want to order the results in either depth-first or breadth-first order. This can be done by computing an ordering column alongside the other data columns and using that to sort the results at the end. Note that this does not actually control in which order the query evaluation visits the rows; that is as always in SQL implementation-dependent. This approach merely provides a convenient way to order the results afterwards.

To create a depth-first order, we compute for each result row an array of rows that we have visited so far. For example, consider the following query that searches a table tree using a link field:

WITH RECURSIVE search_tree(id, link, data) AS (
    SELECT t.id, t.link, t.data
    FROM tree t
  UNION ALL
    SELECT t.id, t.link, t.data
    FROM tree t, search_tree st
    WHERE t.id = st.link
)
SELECT * FROM search_tree;

To add depth-first ordering information, you can write this:

WITH RECURSIVE search_tree(id, link, data, path) AS (
    SELECT t.id, t.link, t.data, ARRAY[t.id]
    FROM tree t
  UNION ALL
    SELECT t.id, t.link, t.data, path || t.id
    FROM tree t, search_tree st
    WHERE t.id = st.link
)
SELECT * FROM search_tree ORDER BY path;

In the general case where more than one field needs to be used to identify a row, use an array of rows. For example, if we needed to track fields f1 and f2:

WITH RECURSIVE search_tree(id, link, data, path) AS (
    SELECT t.id, t.link, t.data, ARRAY[ROW(t.f1, t.f2)]
    FROM tree t
  UNION ALL
    SELECT t.id, t.link, t.data, path || ROW(t.f1, t.f2)
    FROM tree t, search_tree st
    WHERE t.id = st.link
)
SELECT * FROM search_tree ORDER BY path;

[!TIP] Omit the ROW() syntax in the common case where only one field needs to be tracked. This allows a simple array rather than a composite-type array to be used, gaining efficiency.

To create a breadth-first order, you can add a column that tracks the depth of the search, for example:

WITH RECURSIVE search_tree(id, link, data, depth) AS (
    SELECT t.id, t.link, t.data, 0
    FROM tree t
  UNION ALL
    SELECT t.id, t.link, t.data, depth + 1
    FROM tree t, search_tree st
    WHERE t.id = st.link
)
SELECT * FROM search_tree ORDER BY depth;

To get a stable sort, add data columns as secondary sorting columns.

[!TIP] The recursive query evaluation algorithm produces its output in breadth-first search order. However, this is an implementation detail and it is perhaps unsound to rely on it. The order of the rows within each level is certainly undefined, so some explicit ordering might be desired in any case.

There is built-in syntax to compute a depth- or breadth-first sort column. For example:

WITH RECURSIVE search_tree(id, link, data) AS (
    SELECT t.id, t.link, t.data
    FROM tree t
  UNION ALL
    SELECT t.id, t.link, t.data
    FROM tree t, search_tree st
    WHERE t.id = st.link
) SEARCH DEPTH FIRST BY id SET ordercol
SELECT * FROM search_tree ORDER BY ordercol;

WITH RECURSIVE search_tree(id, link, data) AS (
    SELECT t.id, t.link, t.data
    FROM tree t
  UNION ALL
    SELECT t.id, t.link, t.data
    FROM tree t, search_tree st
    WHERE t.id = st.link
) SEARCH BREADTH FIRST BY id SET ordercol
SELECT * FROM search_tree ORDER BY ordercol;

This syntax is internally expanded to something similar to the above hand-written forms. The SEARCH clause specifies whether depth- or breadth first search is wanted, the list of columns to track for sorting, and a column name that will contain the result data that can be used for sorting. That column will implicitly be added to the output rows of the CTE.

Cycle Detection

When working with recursive queries it is important to be sure that the recursive part of the query will eventually return no tuples, or else the query will loop indefinitely. Sometimes, using UNION instead of UNION ALL can accomplish this by discarding rows that duplicate previous output rows. However, often a cycle does not involve output rows that are completely duplicate: it may be necessary to check just one or a few fields to see if the same point has been reached before. The standard method for handling such situations is to compute an array of the already-visited values. For example, consider again the following query that searches a table graph using a link field:

WITH RECURSIVE search_graph(id, link, data, depth) AS (
    SELECT g.id, g.link, g.data, 0
    FROM graph g
  UNION ALL
    SELECT g.id, g.link, g.data, sg.depth + 1
    FROM graph g, search_graph sg
    WHERE g.id = sg.link
)
SELECT * FROM search_graph;

This query will loop if the link relationships contain cycles. Because we require a “depth” output, just changing UNION ALL to UNION would not eliminate the looping. Instead we need to recognize whether we have reached the same row again while following a particular path of links. We add two columns is_cycle and path to the loop-prone query:

WITH RECURSIVE search_graph(id, link, data, depth, is_cycle, path) AS (
    SELECT g.id, g.link, g.data, 0,
      false,
      ARRAY[g.id]
    FROM graph g
  UNION ALL
    SELECT g.id, g.link, g.data, sg.depth + 1,
      g.id = ANY(path),
      path || g.id
    FROM graph g, search_graph sg
    WHERE g.id = sg.link AND NOT is_cycle
)
SELECT * FROM search_graph;

Aside from preventing cycles, the array value is often useful in its own right as representing the “path” taken to reach any particular row.

In the general case where more than one field needs to be checked to recognize a cycle, use an array of rows. For example, if we needed to compare fields f1 and f2:

WITH RECURSIVE search_graph(id, link, data, depth, is_cycle, path) AS (
    SELECT g.id, g.link, g.data, 0,
      false,
      ARRAY[ROW(g.f1, g.f2)]
    FROM graph g
  UNION ALL
    SELECT g.id, g.link, g.data, sg.depth + 1,
      ROW(g.f1, g.f2) = ANY(path),
      path || ROW(g.f1, g.f2)
    FROM graph g, search_graph sg
    WHERE g.id = sg.link AND NOT is_cycle
)
SELECT * FROM search_graph;

[!TIP] Omit the ROW() syntax in the common case where only one field needs to be checked to recognize a cycle. This allows a simple array rather than a composite-type array to be used, gaining efficiency.

There is built-in syntax to simplify cycle detection. The above query can also be written like this:

WITH RECURSIVE search_graph(id, link, data, depth) AS (
    SELECT g.id, g.link, g.data, 1
    FROM graph g
  UNION ALL
    SELECT g.id, g.link, g.data, sg.depth + 1
    FROM graph g, search_graph sg
    WHERE g.id = sg.link
) CYCLE id SET is_cycle USING path
SELECT * FROM search_graph;

and it will be internally rewritten to the above form. The CYCLE clause specifies first the list of columns to track for cycle detection, then a column name that will show whether a cycle has been detected, and finally the name of another column that will track the path. The cycle and path columns will implicitly be added to the output rows of the CTE.

[!TIP] The cycle path column is computed in the same way as the depth-first ordering column show in the previous section. A query can have both a SEARCH and a CYCLE clause, but a depth-first search specification and a cycle detection specification would create redundant computations, so it's more efficient to just use the CYCLE clause and order by the path column. If breadth-first ordering is wanted, then specifying both SEARCH and CYCLE can be useful.

A helpful trick for testing queries when you are not certain if they might loop is to place a LIMIT in the parent query. For example, this query would loop forever without the LIMIT:

WITH RECURSIVE t(n) AS (
    SELECT 1
  UNION ALL
    SELECT n+1 FROM t
)
SELECT n FROM t LIMIT 100;

This works because PostgreSQL's implementation evaluates only as many rows of a WITH query as are actually fetched by the parent query. Using this trick in production is not recommended, because other systems might work differently. Also, it usually won't work if you make the outer query sort the recursive query's results or join them to some other table, because in such cases the outer query will usually try to fetch all of the WITH query's output anyway.

Common Table Expression Materialization

A useful property of WITH queries is that they are normally evaluated only once per execution of the parent query, even if they are referred to more than once by the parent query or sibling WITH queries. Thus, expensive calculations that are needed in multiple places can be placed within a WITH query to avoid redundant work. Another possible application is to prevent unwanted multiple evaluations of functions with side-effects. However, the other side of this coin is that the optimizer is not able to push restrictions from the parent query down into a multiply-referenced WITH query, since that might affect all uses of the WITH query's output when it should affect only one. The multiply-referenced WITH query will be evaluated as written, without suppression of rows that the parent query might discard afterwards. (But, as mentioned above, evaluation might stop early if the reference(s) to the query demand only a limited number of rows.)

However, if a WITH query is non-recursive and side-effect-free (that is, it is a SELECT containing no volatile functions) then it can be folded into the parent query, allowing joint optimization of the two query levels. By default, this happens if the parent query references the WITH query just once, but not if it references the WITH query more than once. You can override that decision by specifying MATERIALIZED to force separate calculation of the WITH query, or by specifying NOT MATERIALIZED to force it to be merged into the parent query. The latter choice risks duplicate computation of the WITH query, but it can still give a net savings if each usage of the WITH query needs only a small part of the WITH query's full output.

A simple example of these rules is

WITH w AS (
    SELECT * FROM big_table
)
SELECT * FROM w WHERE key = 123;

This WITH query will be folded, producing the same execution plan as

SELECT * FROM big_table WHERE key = 123;

In particular, if there's an index on key, it will probably be used to fetch just the rows having key = 123. On the other hand, in

WITH w AS (
    SELECT * FROM big_table
)
SELECT * FROM w AS w1 JOIN w AS w2 ON w1.key = w2.ref
WHERE w2.key = 123;

the WITH query will be materialized, producing a temporary copy of big_table that is then joined with itself — without benefit of any index. This query will be executed much more efficiently if written as

WITH w AS NOT MATERIALIZED (
    SELECT * FROM big_table
)
SELECT * FROM w AS w1 JOIN w AS w2 ON w1.key = w2.ref
WHERE w2.key = 123;

so that the parent query's restrictions can be applied directly to scans of big_table.

An example where NOT MATERIALIZED could be undesirable is

WITH w AS (
    SELECT key, very_expensive_function(val) AS f FROM some_table
)
SELECT * FROM w AS w1 JOIN w AS w2 ON w1.f = w2.f;

Here, materialization of the WITH query ensures that very_expensive_function is evaluated only once per table row, not twice.

The examples above only show WITH being used with SELECT, but it can be attached in the same way to INSERT, UPDATE, DELETE, or MERGE. In each case it effectively provides temporary table(s) that can be referred to in the main command.

Data-Modifying Statements in WITH

You can use data-modifying statements (INSERT, UPDATE, DELETE, or MERGE) in WITH. This allows you to perform several different operations in the same query. An example is:

WITH moved_rows AS (
    DELETE FROM products
    WHERE
        "date" >= '2010-10-01' AND
        "date" < '2010-11-01'
    RETURNING *
)
INSERT INTO products_log
SELECT * FROM moved_rows;

This query effectively moves rows from products to products_log. The DELETE in WITH deletes the specified rows from products, returning their contents by means of its RETURNING clause; and then the primary query reads that output and inserts it into products_log.

A fine point of the above example is that the WITH clause is attached to the INSERT, not the sub-SELECT within the INSERT. This is necessary because data-modifying statements are only allowed in WITH clauses that are attached to the top-level statement. However, normal WITH visibility rules apply, so it is possible to refer to the WITH statement's output from the sub-SELECT.

Data-modifying statements in WITH usually have RETURNING clauses (see ???), as shown in the example above. It is the output of the RETURNING clause, not the target table of the data-modifying statement, that forms the temporary table that can be referred to by the rest of the query. If a data-modifying statement in WITH lacks a RETURNING clause, then it forms no temporary table and cannot be referred to in the rest of the query. Such a statement will be executed nonetheless. A not-particularly-useful example is:

WITH t AS (
    DELETE FROM foo
)
DELETE FROM bar;

This example would remove all rows from tables foo and bar. The number of affected rows reported to the client would only include rows removed from bar.

Recursive self-references in data-modifying statements are not allowed. In some cases it is possible to work around this limitation by referring to the output of a recursive WITH, for example:

WITH RECURSIVE included_parts(sub_part, part) AS (
    SELECT sub_part, part FROM parts WHERE part = 'our_product'
  UNION ALL
    SELECT p.sub_part, p.part
    FROM included_parts pr, parts p
    WHERE p.part = pr.sub_part
)
DELETE FROM parts
  WHERE part IN (SELECT part FROM included_parts);

This query would remove all direct and indirect subparts of a product.

Data-modifying statements in WITH are executed exactly once, and always to completion, independently of whether the primary query reads all (or indeed any) of their output. Notice that this is different from the rule for SELECT in WITH: as stated in the previous section, execution of a SELECT is carried only as far as the primary query demands its output.

The sub-statements in WITH are executed concurrently with each other and with the main query. Therefore, when using data-modifying statements in WITH, the order in which the specified updates actually happen is unpredictable. All the statements are executed with the same snapshot (see ???), so they cannot “see” one another's effects on the target tables. This alleviates the effects of the unpredictability of the actual order of row updates, and means that RETURNING data is the only way to communicate changes between different WITH sub-statements and the main query. An example of this is that in

WITH t AS (
    UPDATE products SET price = price * 1.05
    RETURNING *
)
SELECT * FROM products;

the outer SELECT would return the original prices before the action of the UPDATE, while in

WITH t AS (
    UPDATE products SET price = price * 1.05
    RETURNING *
)
SELECT * FROM t;

the outer SELECT would return the updated data.

Trying to update the same row twice in a single statement is not supported. Only one of the modifications takes place, but it is not easy (and sometimes not possible) to reliably predict which one. This also applies to deleting a row that was already updated in the same statement: only the update is performed. Therefore you should generally avoid trying to modify a single row twice in a single statement. In particular avoid writing WITH sub-statements that could affect the same rows changed by the main statement or a sibling sub-statement. The effects of such a statement will not be predictable.

At present, any table used as the target of a data-modifying statement in WITH must not have a conditional rule, nor an ALSO rule, nor an INSTEAD rule that expands to multiple statements.

[^1]: Actually, PostgreSQL uses the default B-tree operator class for the expression's data type to determine the sort ordering for ASC and DESC. Conventionally, data types will be set up so that the < and > operators correspond to this sort ordering, but a user-defined data type's designer could choose to do something different.

abort

ABORT

ABORT

7

SQL - Language Statements

ABORT

abort the current transaction

ABORT [ WORK | TRANSACTION ] [ AND [ NO ] CHAIN ]

Description

ABORT rolls back the current transaction and causes all the updates made by the transaction to be discarded. This command is identical in behavior to the standard SQL command ROLLBACK, and is present only for historical reasons.

Parameters

WORK; TRANSACTION
Optional key words. They have no effect.

AND CHAIN
If AND CHAIN is specified, a new transaction is immediately started with the same transaction characteristics (see SET TRANSACTION) as the just finished one. Otherwise, no new transaction is started.

Notes

Use COMMIT to successfully terminate a transaction.

Issuing ABORT outside of a transaction block emits a warning and otherwise has no effect.

Examples

To abort all changes:

ABORT;

Compatibility

This command is a PostgreSQL extension present for historical reasons. ROLLBACK is the equivalent standard SQL command.

See Also

alter_aggregate

ALTER AGGREGATE

ALTER AGGREGATE

7

SQL - Language Statements

ALTER AGGREGATE

change the definition of an aggregate function

ALTER AGGREGATE

name

(

aggregate_signature

) RENAME TO

new_name

ALTER AGGREGATE

name

(

aggregate_signature

) OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER AGGREGATE

name

(

aggregate_signature

) SET SCHEMA

new_schema

where

aggregate_signature

is:

* | [

argmode

] [

argname

]

argtype

[ , ... ] | [ [

argmode

] [

argname

]

argtype

[ , ... ] ] ORDER BY [

argmode

] [

argname

]

argtype

[ , ... ]

Description

ALTER AGGREGATE changes the definition of an aggregate function.

You must own the aggregate function to use ALTER AGGREGATE. To change the schema of an aggregate function, you must also have CREATE privilege on the new schema. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the aggregate function's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the aggregate function. However, a superuser can alter ownership of any aggregate function anyway.)

Parameters

<name>
The name (optionally schema-qualified) of an existing aggregate function.

<argmode>
The mode of an argument: IN or VARIADIC. If omitted, the default is IN.

<argname>
The name of an argument. Note that ALTER AGGREGATE does not actually pay any attention to argument names, since only the argument data types are needed to determine the aggregate function's identity.

<argtype>
An input data type on which the aggregate function operates. To reference a zero-argument aggregate function, write * in place of the list of argument specifications. To reference an ordered-set aggregate function, write ORDER BY between the direct and aggregated argument specifications.

<new_name>
The new name of the aggregate function.

<new_owner>
The new owner of the aggregate function.

<new_schema>
The new schema for the aggregate function.

Notes

ALTER AGGREGATE deals only with generic properties of an aggregate, such as its name. To change the aggregate-specific properties of an aggregate such as its support functions, replace its definition entirely with CREATE OR REPLACE AGGREGATE.

Examples

To rename the aggregate function myavg for type integer to my_average:

ALTER AGGREGATE myavg(integer) RENAME TO my_average;

To change the owner of the aggregate function myavg for type integer to joe:

ALTER AGGREGATE myavg(integer) OWNER TO joe;

To move the ordered-set aggregate mypercentile with direct argument of type float8 and aggregated argument of type integer into schema myschema:

ALTER AGGREGATE mypercentile(float8 ORDER BY integer) SET SCHEMA myschema;

This will work too:

ALTER AGGREGATE mypercentile(float8, integer) SET SCHEMA myschema;

Compatibility

There is no ALTER AGGREGATE statement in the SQL standard.

See Also

alter_collation

ALTER COLLATION

ALTER COLLATION

7

SQL - Language Statements

ALTER COLLATION

change the definition of a collation

ALTER COLLATION

name

REFRESH VERSION ALTER COLLATION

name

RENAME TO

new_name

ALTER COLLATION

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER COLLATION

name

SET SCHEMA

new_schema

Description

ALTER COLLATION changes the definition of a collation.

You must own the collation to use ALTER COLLATION. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the collation's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the collation. However, a superuser can alter ownership of any collation anyway.)

Parameters

<name>
The name (optionally schema-qualified) of an existing collation.

<new_name>
The new name of the collation.

<new_owner>
The new owner of the collation.

<new_schema>
The new schema for the collation.

REFRESH VERSION
Update the collation's version. See Notes below.

Notes

When a collation object is created, the provider-specific version of the collation is recorded in the system catalog. When the collation is used, the current version is checked against the recorded version, and a warning is issued when there is a mismatch, for example:

WARNING:  collation "xx-x-icu" has version mismatch
DETAIL:  The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
HINT:  Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.

A change in collation definitions can lead to corrupt indexes and other problems because the database system relies on stored objects having a certain sort order. Generally, this should be avoided, but it can happen in legitimate circumstances, such as when upgrading the operating system to a new major version or when using pg_upgrade to upgrade to server binaries linked with a newer version of ICU. When this happens, all objects depending on the collation should be rebuilt, for example, using REINDEX. When that is done, the collation version can be refreshed using the command ALTER COLLATION ... REFRESH VERSION. This will update the system catalog to record the current collation version and will make the warning go away. Note that this does not actually check whether all affected objects have been rebuilt correctly.

When using collations provided by libc, version information is recorded on systems using the GNU C library (most Linux systems), FreeBSD and Windows. When using collations provided by ICU, the version information is provided by the ICU library and is available on all platforms.

[!NOTE] When using the GNU C library for collations, the C library's version is used as a proxy for the collation version. Many Linux distributions change collation definitions only when upgrading the C library, but this approach is imperfect as maintainers are free to back-port newer collation definitions to older C library releases.

When using Windows for collations, version information is only available for collations defined with BCP 47 language tags such as en-US.

For the database default collation, there is an analogous command ALTER DATABASE ... REFRESH COLLATION VERSION.

The following query can be used to identify all collations in the current database that need to be refreshed and the objects that depend on them:

SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",
       pg_describe_object(classid, objid, objsubid) AS "Object"
  FROM pg_depend d JOIN pg_collation c
       ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid
  WHERE c.collversion <> pg_collation_actual_version(c.oid)
  ORDER BY 1, 2;

Examples

To rename the collation de_DE to german:

ALTER COLLATION "de_DE" RENAME TO german;

To change the owner of the collation en_US to joe:

ALTER COLLATION "en_US" OWNER TO joe;

Compatibility

There is no ALTER COLLATION statement in the SQL standard.

See Also

alter_conversion

ALTER CONVERSION

ALTER CONVERSION

7

SQL - Language Statements

ALTER CONVERSION

change the definition of a conversion

ALTER CONVERSION

name

RENAME TO

new_name

ALTER CONVERSION

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER CONVERSION

name

SET SCHEMA

new_schema

Description

ALTER CONVERSION changes the definition of a conversion.

You must own the conversion to use ALTER CONVERSION. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the conversion's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the conversion. However, a superuser can alter ownership of any conversion anyway.)

Parameters

<name>
The name (optionally schema-qualified) of an existing conversion.

<new_name>
The new name of the conversion.

<new_owner>
The new owner of the conversion.

<new_schema>
The new schema for the conversion.

Examples

To rename the conversion iso_8859_1_to_utf8 to latin1_to_unicode:

ALTER CONVERSION iso_8859_1_to_utf8 RENAME TO latin1_to_unicode;

To change the owner of the conversion iso_8859_1_to_utf8 to joe:

ALTER CONVERSION iso_8859_1_to_utf8 OWNER TO joe;

Compatibility

There is no ALTER CONVERSION statement in the SQL standard.

See Also

alter_database

ALTER DATABASE

ALTER DATABASE

7

SQL - Language Statements

ALTER DATABASE

change a database

ALTER DATABASE

name

[ [ WITH ]

option

[ ... ] ]

where

option

can be:

ALLOW_CONNECTIONS

allowconn

CONNECTION LIMIT

connlimit

IS_TEMPLATE

istemplate

ALTER DATABASE

name

RENAME TO

new_name

ALTER DATABASE

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER DATABASE

name

SET TABLESPACE

new_tablespace

ALTER DATABASE

name

REFRESH COLLATION VERSION ALTER DATABASE

name

SET

configuration_parameter

{ TO | = } {

value

| DEFAULT } ALTER DATABASE

name

SET

configuration_parameter

FROM CURRENT ALTER DATABASE

name

RESET

configuration_parameter

ALTER DATABASE

name

RESET ALL

Description

ALTER DATABASE changes the attributes of a database.

The first form changes certain per-database settings. (See below for details.) Only the database owner or a superuser can change these settings.

The second form changes the name of the database. Only the database owner or a superuser can rename a database; non-superuser owners must also have the CREATEDB privilege. The current database cannot be renamed. (Connect to a different database if you need to do that.)

The third form changes the owner of the database. To alter the owner, you must be able to SET ROLE to the new owning role, and you must have the CREATEDB privilege. (Note that superusers have all these privileges automatically.)

The fourth form changes the default tablespace of the database. Only the database owner or a superuser can do this; you must also have create privilege for the new tablespace. This command physically moves any tables or indexes in the database's old default tablespace to the new tablespace. The new default tablespace must be empty for this database, and no one can be connected to the database. Tables and indexes in non-default tablespaces are unaffected. The method used to copy files to the new tablespace is affected by the ??? setting.

The remaining forms change the session default for a run-time configuration variable for a PostgreSQL database. Whenever a new session is subsequently started in that database, the specified value becomes the session default value. The database-specific default overrides whatever setting is present in postgresql.conf or has been received from the postgres command line. Only the database owner or a superuser can change the session defaults for a database. Certain variables cannot be set this way, or can only be set by a superuser.

Parameters

<name>
The name of the database whose attributes are to be altered.

<allowconn>
If false then no one can connect to this database.

<connlimit>
How many concurrent connections can be made to this database. -1 means no limit.

<istemplate>
If true, then this database can be cloned by any user with CREATEDB privileges; if false, then only superusers or the owner of the database can clone it.

<new_name>
The new name of the database.

<new_owner>
The new owner of the database.

<new_tablespace>
The new default tablespace of the database.

This form of the command cannot be executed inside a transaction block.

REFRESH COLLATION VERSION
Update the database collation version. See ??? for background.

<configuration_parameter>; <value>
Set this database's session default for the specified configuration parameter to the given value. If <value> is DEFAULT or, equivalently, RESET is used, the database-specific setting is removed, so the system-wide default setting will be inherited in new sessions. Use RESET ALL to clear all database-specific settings. SET FROM CURRENT saves the session's current value of the parameter as the database-specific value.

See ??? and ??? for more information about allowed parameter names and values.

Notes

It is also possible to tie a session default to a specific role rather than to a database; see ???. Role-specific settings override database-specific ones if there is a conflict.

Examples

To disable index scans by default in the database test:

ALTER DATABASE test SET enable_indexscan TO off;

Compatibility

The ALTER DATABASE statement is a PostgreSQL extension.

See Also

alter_default_privileges

ALTER DEFAULT PRIVILEGES

ALTER DEFAULT PRIVILEGES

7

SQL - Language Statements

ALTER DEFAULT PRIVILEGES

define default access privileges

ALTER DEFAULT PRIVILEGES [ FOR { ROLE | USER }

target_role

[, ...] ] [ IN SCHEMA

schema_name

[, ...] ]

abbreviated_grant_or_revoke

where

abbreviated_grant_or_revoke

is one of:

GRANT { { SELECT | INSERT | UPDATE | DELETE | TRUNCATE | REFERENCES | TRIGGER | MAINTAIN } [, ...] | ALL [ PRIVILEGES ] } ON TABLES TO { [ GROUP ]

role_name

| PUBLIC } [, ...] [ WITH GRANT OPTION ] GRANT { { USAGE | SELECT | UPDATE } [, ...] | ALL [ PRIVILEGES ] } ON SEQUENCES TO { [ GROUP ]

role_name

| PUBLIC } [, ...] [ WITH GRANT OPTION ] GRANT { EXECUTE | ALL [ PRIVILEGES ] } ON { FUNCTIONS | ROUTINES } TO { [ GROUP ]

role_name

| PUBLIC } [, ...] [ WITH GRANT OPTION ] GRANT { USAGE | ALL [ PRIVILEGES ] } ON TYPES TO { [ GROUP ]

role_name

| PUBLIC } [, ...] [ WITH GRANT OPTION ] GRANT { { USAGE | CREATE } [, ...] | ALL [ PRIVILEGES ] } ON SCHEMAS TO { [ GROUP ]

role_name

| PUBLIC } [, ...] [ WITH GRANT OPTION ] GRANT { { SELECT | UPDATE } [, ...] | ALL [ PRIVILEGES ] } ON LARGE OBJECTS TO { [ GROUP ]

role_name

| PUBLIC } [, ...] [ WITH GRANT OPTION ] REVOKE [ GRANT OPTION FOR ] { { SELECT | INSERT | UPDATE | DELETE | TRUNCATE | REFERENCES | TRIGGER | MAINTAIN } [, ...] | ALL [ PRIVILEGES ] } ON TABLES FROM { [ GROUP ]

role_name

| PUBLIC } [, ...] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { { USAGE | SELECT | UPDATE } [, ...] | ALL [ PRIVILEGES ] } ON SEQUENCES FROM { [ GROUP ]

role_name

| PUBLIC } [, ...] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { EXECUTE | ALL [ PRIVILEGES ] } ON { FUNCTIONS | ROUTINES } FROM { [ GROUP ]

role_name

| PUBLIC } [, ...] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { USAGE | ALL [ PRIVILEGES ] } ON TYPES FROM { [ GROUP ]

role_name

| PUBLIC } [, ...] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { { USAGE | CREATE } [, ...] | ALL [ PRIVILEGES ] } ON SCHEMAS FROM { [ GROUP ]

role_name

| PUBLIC } [, ...] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { { SELECT | UPDATE } [, ...] | ALL [ PRIVILEGES ] } ON LARGE OBJECTS FROM { [ GROUP ]

role_name

| PUBLIC } [, ...] [ CASCADE | RESTRICT ]

Description

ALTER DEFAULT PRIVILEGES allows you to set the privileges that will be applied to objects created in the future. (It does not affect privileges assigned to already-existing objects.) Privileges can be set globally (i.e., for all objects created in the current database), or just for objects created in specified schemas.

While you can change your own default privileges and the defaults of roles that you are a member of, at object creation time, new object permissions are only affected by the default privileges of the current role, and are not inherited from any roles in which the current role is a member.

As explained in ???, the default privileges for any object type normally grant all grantable permissions to the object owner, and may grant some privileges to PUBLIC as well. However, this behavior can be changed by altering the global default privileges with ALTER DEFAULT PRIVILEGES.

Currently, only the privileges for schemas, tables (including views and foreign tables), sequences, functions, types (including domains), and large objects can be altered. For this command, functions include aggregates and procedures. The words FUNCTIONS and ROUTINES are equivalent in this command. (ROUTINES is preferred going forward as the standard term for functions and procedures taken together. In earlier PostgreSQL releases, only the word FUNCTIONS was allowed. It is not possible to set default privileges for functions and procedures separately.)

Default privileges that are specified per-schema are added to whatever the global default privileges are for the particular object type. This means you cannot revoke privileges per-schema if they are granted globally (either by default, or according to a previous ALTER DEFAULT PRIVILEGES command that did not specify a schema). Per-schema REVOKE is only useful to reverse the effects of a previous per-schema GRANT.

Parameters

<target_role>
Change default privileges for objects created by the <target_role>, or the current role if unspecified.

<schema_name>
The name of an existing schema. If specified, the default privileges are altered for objects later created in that schema. If IN SCHEMA is omitted, the global default privileges are altered. IN SCHEMA is not allowed when setting privileges for schemas and large objects, since schemas can't be nested and large objects don't belong to a schema.

<role_name>
The name of an existing role to grant or revoke privileges for. This parameter, and all the other parameters in <abbreviated_grant_or_revoke>, act as described under ??? or ???, except that one is setting permissions for a whole class of objects rather than specific named objects.

Notes

Use ???'s \ddp command to obtain information about existing assignments of default privileges. The meaning of the privilege display is the same as explained for \dp in ???.

If you wish to drop a role for which the default privileges have been altered, it is necessary to reverse the changes in its default privileges or use DROP OWNED BY to get rid of the default privileges entry for the role.

Examples

Grant SELECT privilege to everyone for all tables (and views) you subsequently create in schema myschema, and allow role webuser to INSERT into them too:

ALTER DEFAULT PRIVILEGES IN SCHEMA myschema GRANT SELECT ON TABLES TO PUBLIC;
ALTER DEFAULT PRIVILEGES IN SCHEMA myschema GRANT INSERT ON TABLES TO webuser;

Undo the above, so that subsequently-created tables won't have any more permissions than normal:

ALTER DEFAULT PRIVILEGES IN SCHEMA myschema REVOKE SELECT ON TABLES FROM PUBLIC;
ALTER DEFAULT PRIVILEGES IN SCHEMA myschema REVOKE INSERT ON TABLES FROM webuser;

Remove the public EXECUTE permission that is normally granted on functions, for all functions subsequently created by role admin:

ALTER DEFAULT PRIVILEGES FOR ROLE admin REVOKE EXECUTE ON FUNCTIONS FROM PUBLIC;

Note however that you cannot accomplish that effect with a command limited to a single schema. This command has no effect, unless it is undoing a matching GRANT:

ALTER DEFAULT PRIVILEGES IN SCHEMA public REVOKE EXECUTE ON FUNCTIONS FROM PUBLIC;

That's because per-schema default privileges can only add privileges to the global setting, not remove privileges granted by it.

Compatibility

There is no ALTER DEFAULT PRIVILEGES statement in the SQL standard.

See Also

alter_domain

ALTER DOMAIN

ALTER DOMAIN

7

SQL - Language Statements

ALTER DOMAIN

change the definition of a domain

ALTER DOMAIN

name

{ SET DEFAULT

expression

| DROP DEFAULT } ALTER DOMAIN

name

{ SET | DROP } NOT NULL ALTER DOMAIN

name

ADD

domain_constraint

[ NOT VALID ] ALTER DOMAIN

name

DROP CONSTRAINT [ IF EXISTS ]

constraint_name

[ RESTRICT | CASCADE ] ALTER DOMAIN

name

RENAME CONSTRAINT

constraint_name

TO

new_constraint_name

ALTER DOMAIN

name

VALIDATE CONSTRAINT

constraint_name

ALTER DOMAIN

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER DOMAIN

name

RENAME TO

new_name

ALTER DOMAIN

name

SET SCHEMA

new_schema

where

domain_constraint

is:

[ CONSTRAINT

constraint_name

] { NOT NULL | CHECK (

expression

) }

Description

ALTER DOMAIN changes the definition of an existing domain. There are several sub-forms:

SET/DROP DEFAULT
These forms set or remove the default value for a domain. Note that defaults only apply to subsequent INSERT commands; they do not affect rows already in a table using the domain.

SET/DROP NOT NULL
These forms change whether a domain is marked to allow NULL values or to reject NULL values. You can only SET NOT NULL when the columns using the domain contain no null values.

ADD domain_constraint [ NOT VALID ]
This form adds a new constraint to a domain. When a new constraint is added to a domain, all columns using that domain will be checked against the newly added constraint. These checks can be suppressed by adding the new constraint using the NOT VALID option; the constraint can later be made valid using ALTER DOMAIN ... VALIDATE CONSTRAINT. Newly inserted or updated rows are always checked against all constraints, even those marked NOT VALID. NOT VALID is only accepted for CHECK constraints.

DROP CONSTRAINT [ IF EXISTS ]
This form drops constraints on a domain. If IF EXISTS is specified and the constraint does not exist, no error is thrown. In this case a notice is issued instead.

RENAME CONSTRAINT
This form changes the name of a constraint on a domain.

VALIDATE CONSTRAINT
This form validates a constraint previously added as NOT VALID, that is, it verifies that all values in table columns of the domain type satisfy the specified constraint.

OWNER
This form changes the owner of the domain to the specified user.

RENAME
This form changes the name of the domain.

SET SCHEMA
This form changes the schema of the domain. Any constraints associated with the domain are moved into the new schema as well.

You must own the domain to use ALTER DOMAIN. To change the schema of a domain, you must also have CREATE privilege on the new schema. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the domain's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the domain. However, a superuser can alter ownership of any domain anyway.)

Parameters

<name>
The name (possibly schema-qualified) of an existing domain to alter.

<domain_constraint>
New domain constraint for the domain.

<constraint_name>
Name of an existing constraint to drop or rename.

NOT VALID
Do not verify existing stored data for constraint validity.

CASCADE
Automatically drop objects that depend on the constraint, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the constraint if there are any dependent objects. This is the default behavior.

<new_name>
The new name for the domain.

<new_constraint_name>
The new name for the constraint.

<new_owner>
The user name of the new owner of the domain.

<new_schema>
The new schema for the domain.

Notes

Although ALTER DOMAIN ADD CONSTRAINT attempts to verify that existing stored data satisfies the new constraint, this check is not bulletproof, because the command cannot “see” table rows that are newly inserted or updated and not yet committed. If there is a hazard that concurrent operations might insert bad data, the way to proceed is to add the constraint using the NOT VALID option, commit that command, wait until all transactions started before that commit have finished, and then issue ALTER DOMAIN VALIDATE CONSTRAINT to search for data violating the constraint. This method is reliable because once the constraint is committed, all new transactions are guaranteed to enforce it against new values of the domain type.

Currently, ALTER DOMAIN ADD CONSTRAINT, ALTER DOMAIN VALIDATE CONSTRAINT, and ALTER DOMAIN SET NOT NULL will fail if the named domain or any derived domain is used within a container-type column (a composite, array, or range column) in any table in the database. They should eventually be improved to be able to verify the new constraint for such nested values.

Examples

To add a NOT NULL constraint to a domain:

ALTER DOMAIN zipcode SET NOT NULL;

To remove a NOT NULL constraint from a domain:

ALTER DOMAIN zipcode DROP NOT NULL;

To add a check constraint to a domain:

ALTER DOMAIN zipcode ADD CONSTRAINT zipchk CHECK (char_length(VALUE) = 5);

To remove a check constraint from a domain:

ALTER DOMAIN zipcode DROP CONSTRAINT zipchk;

To rename a check constraint on a domain:

ALTER DOMAIN zipcode RENAME CONSTRAINT zipchk TO zip_check;

To move the domain into a different schema:

ALTER DOMAIN zipcode SET SCHEMA customers;

Compatibility

ALTER DOMAIN conforms to the SQL standard, except for the OWNER, RENAME, SET SCHEMA, and VALIDATE CONSTRAINT variants, which are PostgreSQL extensions. The NOT VALID clause of the ADD CONSTRAINT variant is also a PostgreSQL extension.

See Also

alter_event_trigger

ALTER EVENT TRIGGER

ALTER EVENT TRIGGER

7

SQL - Language Statements

ALTER EVENT TRIGGER

change the definition of an event trigger

ALTER EVENT TRIGGER

name

DISABLE ALTER EVENT TRIGGER

name

ENABLE [ REPLICA | ALWAYS ] ALTER EVENT TRIGGER

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER EVENT TRIGGER

name

RENAME TO

new_name

Description

ALTER EVENT TRIGGER changes properties of an existing event trigger.

You must be superuser to alter an event trigger.

Parameters

<name>
The name of an existing trigger to alter.

<new_owner>
The user name of the new owner of the event trigger.

<new_name>
The new name of the event trigger.

DISABLE/ENABLE [ REPLICA | ALWAYS ]
These forms configure the firing of event triggers. A disabled trigger is still known to the system, but is not executed when its triggering event occurs. See also ???.

Compatibility

There is no ALTER EVENT TRIGGER statement in the SQL standard.

See Also

alter_extension

ALTER EXTENSION

ALTER EXTENSION

7

SQL - Language Statements

ALTER EXTENSION

change the definition of an extension

ALTER EXTENSION

name

UPDATE [ TO

new_version

] ALTER EXTENSION

name

SET SCHEMA

new_schema

ALTER EXTENSION

name

ADD

member_object

ALTER EXTENSION

name

DROP

member_object

where

member_object

is:

ACCESS METHOD

object_name

| AGGREGATE

aggregate_name

(

aggregate_signature

) | CAST (

source_type

AS

target_type

) | COLLATION

object_name

| CONVERSION

object_name

| DOMAIN

object_name

| EVENT TRIGGER

object_name

| FOREIGN DATA WRAPPER

object_name

| FOREIGN TABLE

object_name

| FUNCTION

function_name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] | MATERIALIZED VIEW

object_name

| OPERATOR

operator_name

(

left_type

,

right_type

) | OPERATOR CLASS

object_name

USING

index_method

| OPERATOR FAMILY

object_name

USING

index_method

| [ PROCEDURAL ] LANGUAGE

object_name

| PROCEDURE

procedure_name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] | ROUTINE

routine_name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] | SCHEMA

object_name

| SEQUENCE

object_name

| SERVER

object_name

| TABLE

object_name

| TEXT SEARCH CONFIGURATION

object_name

| TEXT SEARCH DICTIONARY

object_name

| TEXT SEARCH PARSER

object_name

| TEXT SEARCH TEMPLATE

object_name

| TRANSFORM FOR

type_name

LANGUAGE

lang_name

| TYPE

object_name

| VIEW

object_name

and

aggregate_signature

is:

* | [

argmode

] [

argname

]

argtype

[ , ... ] | [ [

argmode

] [

argname

]

argtype

[ , ... ] ] ORDER BY [

argmode

] [

argname

]

argtype

[ , ... ]

Description

ALTER EXTENSION changes the definition of an installed extension. There are several subforms:

UPDATE
This form updates the extension to a newer version. The extension must supply a suitable update script (or series of scripts) that can modify the currently-installed version into the requested version.

SET SCHEMA
This form moves the extension's objects into another schema. The extension has to be relocatable for this command to succeed.

ADD member_object
This form adds an existing object to the extension. This is mainly useful in extension update scripts. The object will subsequently be treated as a member of the extension; notably, it can only be dropped by dropping the extension.

DROP member_object
This form removes a member object from the extension. This is mainly useful in extension update scripts. The object is not dropped, only disassociated from the extension.

See ??? for more information about these operations.

You must own the extension to use ALTER EXTENSION. The ADD/DROP forms require ownership of the added/dropped object as well.

Parameters

<name>
The name of an installed extension.

<new_version>
The desired new version of the extension. This can be written as either an identifier or a string literal. If not specified, ALTER EXTENSION UPDATE attempts to update to whatever is shown as the default version in the extension's control file.

<new_schema>
The new schema for the extension.

<object_name>; <aggregate_name>; <function_name>; <operator_name>; <procedure_name>; <routine_name>
The name of an object to be added to or removed from the extension. Names of tables, aggregates, domains, foreign tables, functions, operators, operator classes, operator families, procedures, routines, sequences, text search objects, types, and views can be schema-qualified.

<source_type>
The name of the source data type of the cast.

<target_type>
The name of the target data type of the cast.

<argmode>
The mode of a function, procedure, or aggregate argument: IN, OUT, INOUT, or VARIADIC. If omitted, the default is IN. Note that ALTER EXTENSION does not actually pay any attention to OUT arguments, since only the input arguments are needed to determine the function's identity. So it is sufficient to list the IN, INOUT, and VARIADIC arguments.

<argname>
The name of a function, procedure, or aggregate argument. Note that ALTER EXTENSION does not actually pay any attention to argument names, since only the argument data types are needed to determine the function's identity.

<argtype>
The data type of a function, procedure, or aggregate argument.

<left_type>; <right_type>
The data type(s) of the operator's arguments (optionally schema-qualified). Write NONE for the missing argument of a prefix operator.

PROCEDURAL
This is a noise word.

<type_name>
The name of the data type of the transform.

<lang_name>
The name of the language of the transform.

Examples

To update the hstore extension to version 2.0:

ALTER EXTENSION hstore UPDATE TO '2.0';

To change the schema of the hstore extension to utils:

ALTER EXTENSION hstore SET SCHEMA utils;

To add an existing function to the hstore extension:

ALTER EXTENSION hstore ADD FUNCTION populate_record(anyelement, hstore);

Compatibility

ALTER EXTENSION is a PostgreSQL extension.

See Also

alter_foreign_data_wrapper

ALTER FOREIGN DATA WRAPPER

ALTER FOREIGN DATA WRAPPER

7

SQL - Language Statements

ALTER FOREIGN DATA WRAPPER

change the definition of a foreign-data wrapper

ALTER FOREIGN DATA WRAPPER

name

[ HANDLER

handler_function

| NO HANDLER ] [ VALIDATOR

validator_function

| NO VALIDATOR ] [ CONNECTION

connection_function

| NO CONNECTION ] [ OPTIONS ( [ ADD | SET | DROP ]

option

['

value

'] [, ... ]) ] ALTER FOREIGN DATA WRAPPER

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER FOREIGN DATA WRAPPER

name

RENAME TO

new_name

Description

ALTER FOREIGN DATA WRAPPER changes the definition of a foreign-data wrapper. The first form of the command changes the support functions or the generic options of the foreign-data wrapper (at least one clause is required). The second form changes the owner of the foreign-data wrapper.

Only superusers can alter foreign-data wrappers. Additionally, only superusers can own foreign-data wrappers.

Parameters

<name>
The name of an existing foreign-data wrapper.

HANDLER handler_function
Specifies a new handler function for the foreign-data wrapper.

NO HANDLER
This is used to specify that the foreign-data wrapper should no longer have a handler function.

Note that foreign tables that use a foreign-data wrapper with no handler cannot be accessed.

VALIDATOR validator_function
Specifies a new validator function for the foreign-data wrapper.

Note that it is possible that pre-existing options of the foreign-data wrapper, or of dependent servers, user mappings, or foreign tables, are invalid according to the new validator. PostgreSQL does not check for this. It is up to the user to make sure that these options are correct before using the modified foreign-data wrapper. However, any options specified in this ALTER FOREIGN DATA WRAPPER command will be checked using the new validator.

NO VALIDATOR
This is used to specify that the foreign-data wrapper should no longer have a validator function.

CONNECTION connection_function
Specifies a new connection function for the foreign-data wrapper.

NO CONNECTION
This is used to specify that the foreign-data wrapper should no longer have a connection function.

OPTIONS ( [ ADD | SET | DROP ] option ['value'] [, ... ] )
Change options for the foreign-data wrapper. ADD, SET, and DROP specify the action to be performed. ADD is assumed if no operation is explicitly specified. Option names must be unique; names and values are also validated using the foreign data wrapper's validator function, if any.

<new_owner>
The user name of the new owner of the foreign-data wrapper.

<new_name>
The new name for the foreign-data wrapper.

Examples

Change a foreign-data wrapper dbi, add option foo, drop bar:

ALTER FOREIGN DATA WRAPPER dbi OPTIONS (ADD foo '1', DROP bar);

Change the foreign-data wrapper dbi validator to bob.myvalidator:

ALTER FOREIGN DATA WRAPPER dbi VALIDATOR bob.myvalidator;

Compatibility

ALTER FOREIGN DATA WRAPPER conforms to ISO/IEC 9075-9 (SQL/MED), except that the HANDLER, VALIDATOR, OWNER TO, and RENAME clauses are extensions.

See Also

alter_foreign_table

ALTER FOREIGN TABLE

ALTER FOREIGN TABLE

7

SQL - Language Statements

ALTER FOREIGN TABLE

change the definition of a foreign table

ALTER FOREIGN TABLE [ IF EXISTS ] [ ONLY ]

name

[ * ]

action

[, ... ] ALTER FOREIGN TABLE [ IF EXISTS ] [ ONLY ]

name

[ * ] RENAME [ COLUMN ]

column_name

TO

new_column_name

ALTER FOREIGN TABLE [ IF EXISTS ]

name

RENAME TO

new_name

ALTER FOREIGN TABLE [ IF EXISTS ]

name

SET SCHEMA

new_schema

where

action

is one of:

ADD [ COLUMN ] [ IF NOT EXISTS ]

column_name

data_type

[ COLLATE

collation

] [

column_constraint

[ ... ] ] DROP [ COLUMN ] [ IF EXISTS ]

column_name

[ RESTRICT | CASCADE ] ALTER [ COLUMN ]

column_name

[ SET DATA ] TYPE

data_type

[ COLLATE

collation

] ALTER [ COLUMN ]

column_name

SET DEFAULT

expression

ALTER [ COLUMN ]

column_name

DROP DEFAULT ALTER [ COLUMN ]

column_name

{ SET | DROP } NOT NULL ALTER [ COLUMN ]

column_name

SET STATISTICS

integer

ALTER [ COLUMN ]

column_name

SET (

attribute_option

=

value

[, ... ] ) ALTER [ COLUMN ]

column_name

RESET (

attribute_option

[, ... ] ) ALTER [ COLUMN ]

column_name

SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN | DEFAULT } ALTER [ COLUMN ]

column_name

OPTIONS ( [ ADD | SET | DROP ]

option

['

value

'] [, ... ]) ADD

table_constraint

[ NOT VALID ] VALIDATE CONSTRAINT

constraint_name

DROP CONSTRAINT [ IF EXISTS ]

constraint_name

[ RESTRICT | CASCADE ] DISABLE TRIGGER [

trigger_name

| ALL | USER ] ENABLE TRIGGER [

trigger_name

| ALL | USER ] ENABLE REPLICA TRIGGER

trigger_name

ENABLE ALWAYS TRIGGER

trigger_name

SET WITHOUT OIDS INHERIT

parent_table

NO INHERIT

parent_table

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } OPTIONS ( [ ADD | SET | DROP ]

option

['

value

'] [, ... ])

Description

ALTER FOREIGN TABLE changes the definition of an existing foreign table. There are several subforms:

ADD [ COLUMN ] [ IF NOT EXISTS ]
This form adds a new column to the foreign table, using the same syntax as CREATE FOREIGN TABLE. If IF NOT EXISTS is specified and a column already exists with this name, no error is thrown. Unlike the case when adding a column to a regular table, nothing happens to the underlying storage: this action simply declares that some new column is now accessible through the foreign table.

DROP [ COLUMN ] [ IF EXISTS ]
This form drops a column from a foreign table. You will need to say CASCADE if anything outside the table depends on the column; for example, views. If IF EXISTS is specified and the column does not exist, no error is thrown. In this case a notice is issued instead.

SET DATA TYPE
This form changes the type of a column of a foreign table. Again, this has no effect on any underlying storage: this action simply changes the type that PostgreSQL believes the column to have.

SET/DROP DEFAULT
These forms set or remove the default value for a column. Default values only apply in subsequent INSERT or UPDATE commands; they do not cause rows already in the table to change.

SET/DROP NOT NULL
Mark a column as allowing, or not allowing, null values.

SET STATISTICS
This form sets the per-column statistics-gathering target for subsequent ANALYZE operations. See the similar form of ALTER TABLE for more details.

SET ( attribute_option = value [, ... ] ); RESET ( attribute_option [, ... ] )
This form sets or resets per-attribute options. See the similar form of ALTER TABLE for more details.

SET STORAGE
This form sets the storage mode for a column. See the similar form of ALTER TABLE for more details. Note that the storage mode has no effect unless the table's foreign-data wrapper chooses to pay attention to it.

ADD table_constraint [ NOT VALID ]
This form adds a new constraint to a foreign table, using the same syntax as CREATE FOREIGN TABLE. Currently only CHECK and NOT NULL constraints are supported.

Unlike the case when adding a constraint to a regular table, nothing is done to verify the constraint is correct; rather, this action simply declares that some new condition should be assumed to hold for all rows in the foreign table. (See the discussion in CREATE FOREIGN TABLE.) If the constraint is marked NOT VALID (allowed only for the CHECK case), then it isn't assumed to hold, but is only recorded for possible future use.

VALIDATE CONSTRAINT
This form marks as valid a constraint that was previously marked as NOT VALID. No action is taken to verify the constraint, but future queries will assume that it holds.

DROP CONSTRAINT [ IF EXISTS ]
This form drops the specified constraint on a foreign table. If IF EXISTS is specified and the constraint does not exist, no error is thrown. In this case a notice is issued instead.

DISABLE/ENABLE [ REPLICA | ALWAYS ] TRIGGER
These forms configure the firing of trigger(s) belonging to the foreign table. See the similar form of ALTER TABLE for more details.

SET WITHOUT OIDS
Backward compatibility syntax for removing the oid system column. As oid system columns cannot be added anymore, this never has an effect.

INHERIT parent_table
This form adds the target foreign table as a new child of the specified parent table. See the similar form of ALTER TABLE for more details.

NO INHERIT parent_table
This form removes the target foreign table from the list of children of the specified parent table.

OWNER
This form changes the owner of the foreign table to the specified user.

OPTIONS ( [ ADD | SET | DROP ] option ['value'] [, ... ] )
Change options for the foreign table or one of its columns. ADD, SET, and DROP specify the action to be performed. ADD is assumed if no operation is explicitly specified. Duplicate option names are not allowed (although it's OK for a table option and a column option to have the same name). Option names and values are also validated using the foreign data wrapper library.

RENAME
The RENAME forms change the name of a foreign table or the name of an individual column in a foreign table.

SET SCHEMA
This form moves the foreign table into another schema.

All the actions except RENAME and SET SCHEMA can be combined into a list of multiple alterations to apply in parallel. For example, it is possible to add several columns and/or alter the type of several columns in a single command.

If the command is written as ALTER FOREIGN TABLE IF EXISTS ... and the foreign table does not exist, no error is thrown. A notice is issued in this case.

You must own the table to use ALTER FOREIGN TABLE. To change the schema of a foreign table, you must also have CREATE privilege on the new schema. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the table's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the table. However, a superuser can alter ownership of any table anyway.) To add a column or alter a column type, you must also have USAGE privilege on the data type.

Parameters

<name>
The name (possibly schema-qualified) of an existing foreign table to alter. If ONLY is specified before the table name, only that table is altered. If ONLY is not specified, the table and all its descendant tables (if any) are altered. Optionally, * can be specified after the table name to explicitly indicate that descendant tables are included.

<column_name>
Name of a new or existing column.

<new_column_name>
New name for an existing column.

<new_name>
New name for the table.

<data_type>
Data type of the new column, or new data type for an existing column.

<table_constraint>
New table constraint for the foreign table.

<constraint_name>
Name of an existing constraint to drop.

CASCADE
Automatically drop objects that depend on the dropped column or constraint (for example, views referencing the column), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the column or constraint if there are any dependent objects. This is the default behavior.

<trigger_name>
Name of a single trigger to disable or enable.

ALL
Disable or enable all triggers belonging to the foreign table. (This requires superuser privilege if any of the triggers are internally generated triggers. The core system does not add such triggers to foreign tables, but add-on code could do so.)

USER
Disable or enable all triggers belonging to the foreign table except for internally generated triggers.

<parent_table>
A parent table to associate or de-associate with this foreign table.

<new_owner>
The user name of the new owner of the table.

<new_schema>
The name of the schema to which the table will be moved.

Notes

The key word COLUMN is noise and can be omitted.

Consistency with the foreign server is not checked when a column is added or removed with ADD COLUMN or DROP COLUMN, a NOT NULL or CHECK constraint is added, or a column type is changed with SET DATA TYPE. It is the user's responsibility to ensure that the table definition matches the remote side.

Refer to CREATE FOREIGN TABLE for a further description of valid parameters.

Examples

To mark a column as not-null:

ALTER FOREIGN TABLE distributors ALTER COLUMN street SET NOT NULL;

To change options of a foreign table:

ALTER FOREIGN TABLE myschema.distributors OPTIONS (ADD opt1 'value', SET opt2 'value2', DROP opt3);

Compatibility

The forms ADD, DROP, and SET DATA TYPE conform with the SQL standard. The other forms are PostgreSQL extensions of the SQL standard. Also, the ability to specify more than one manipulation in a single ALTER FOREIGN TABLE command is an extension.

ALTER FOREIGN TABLE DROP COLUMN can be used to drop the only column of a foreign table, leaving a zero-column table. This is an extension of SQL, which disallows zero-column foreign tables.

See Also

alter_function

ALTER FUNCTION

ALTER FUNCTION

7

SQL - Language Statements

ALTER FUNCTION

change the definition of a function

ALTER FUNCTION

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ]

action

[ ... ] [ RESTRICT ] ALTER FUNCTION

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] RENAME TO

new_name

ALTER FUNCTION

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER FUNCTION

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] SET SCHEMA

new_schema

ALTER FUNCTION

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] [ NO ] DEPENDS ON EXTENSION

extension_name

where

action

is one of:

CALLED ON NULL INPUT | RETURNS NULL ON NULL INPUT | STRICT IMMUTABLE | STABLE | VOLATILE [ NOT ] LEAKPROOF [ EXTERNAL ] SECURITY INVOKER | [ EXTERNAL ] SECURITY DEFINER PARALLEL { UNSAFE | RESTRICTED | SAFE } COST

execution_cost

ROWS

result_rows

SUPPORT

support_function

SET

configuration_parameter

{ TO | = } {

value

| DEFAULT } SET

configuration_parameter

FROM CURRENT RESET

configuration_parameter

RESET ALL

Description

ALTER FUNCTION changes the definition of a function.

You must own the function to use ALTER FUNCTION. To change a function's schema, you must also have CREATE privilege on the new schema. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the function's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the function. However, a superuser can alter ownership of any function anyway.)

Parameters

<name>
The name (optionally schema-qualified) of an existing function. If no argument list is specified, the name must be unique in its schema.

<argmode>
The mode of an argument: IN, OUT, INOUT, or VARIADIC. If omitted, the default is IN. Note that ALTER FUNCTION does not actually pay any attention to OUT arguments, since only the input arguments are needed to determine the function's identity. So it is sufficient to list the IN, INOUT, and VARIADIC arguments.

<argname>
The name of an argument. Note that ALTER FUNCTION does not actually pay any attention to argument names, since only the argument data types are needed to determine the function's identity.

<argtype>
The data type(s) of the function's arguments (optionally schema-qualified), if any.

<new_name>
The new name of the function.

<new_owner>
The new owner of the function. Note that if the function is marked SECURITY DEFINER, it will subsequently execute as the new owner.

<new_schema>
The new schema for the function.

DEPENDS ON EXTENSION extension_name; NO DEPENDS ON EXTENSION extension_name
This form marks the function as dependent on the extension, or no longer dependent on that extension if NO is specified. A function that's marked as dependent on an extension is dropped when the extension is dropped, even if CASCADE is not specified. A function can depend upon multiple extensions, and will be dropped when any one of those extensions is dropped.

CALLED ON NULL INPUT; RETURNS NULL ON NULL INPUT; STRICT
CALLED ON NULL INPUT changes the function so that it will be invoked when some or all of its arguments are null. RETURNS NULL ON NULL INPUT or STRICT changes the function so that it is not invoked if any of its arguments are null; instead, a null result is assumed automatically. See ??? for more information.

IMMUTABLE; STABLE; VOLATILE
Change the volatility of the function to the specified setting. See ??? for details.

EXTERNAL SECURITY INVOKER; EXTERNAL SECURITY DEFINER
Change whether the function is a security definer or not. The key word EXTERNAL is ignored for SQL conformance. See ??? for more information about this capability.

PARALLEL
Change whether the function is deemed safe for parallelism. See ??? for details.

LEAKPROOF
Change whether the function is considered leakproof or not. See ??? for more information about this capability.

COST <execution_cost>
Change the estimated execution cost of the function. See ??? for more information.

ROWS <result_rows>
Change the estimated number of rows returned by a set-returning function. See ??? for more information.

SUPPORT <support_function>
Set or change the planner support function to use for this function. See ??? for details. You must be superuser to use this option.

This option cannot be used to remove the support function altogether, since it must name a new support function. Use CREATE OR REPLACE FUNCTION if you need to do that.

<configuration_parameter>; <value>
Add or change the assignment to be made to a configuration parameter when the function is called. If <value> is DEFAULT or, equivalently, RESET is used, the function-local setting is removed, so that the function executes with the value present in its environment. Use RESET ALL to clear all function-local settings. SET FROM CURRENT saves the value of the parameter that is current when ALTER FUNCTION is executed as the value to be applied when the function is entered.

See ??? and ??? for more information about allowed parameter names and values.

RESTRICT
Ignored for conformance with the SQL standard.

Examples

To rename the function sqrt for type integer to square_root:

ALTER FUNCTION sqrt(integer) RENAME TO square_root;

To change the owner of the function sqrt for type integer to joe:

ALTER FUNCTION sqrt(integer) OWNER TO joe;

To change the schema of the function sqrt for type integer to maths:

ALTER FUNCTION sqrt(integer) SET SCHEMA maths;

To mark the function sqrt for type integer as being dependent on the extension mathlib:

ALTER FUNCTION sqrt(integer) DEPENDS ON EXTENSION mathlib;

ALTER FUNCTION check_password(text) SET search_path = admin, pg_temp;

To disable automatic setting of search_path for a function:

ALTER FUNCTION check_password(text) RESET search_path;

The function will now execute with whatever search path is used by its caller.

Compatibility

This statement is partially compatible with the ALTER FUNCTION statement in the SQL standard. The standard allows more properties of a function to be modified, but does not provide the ability to rename a function, make a function a security definer, attach configuration parameter values to a function, or change the owner, schema, or volatility of a function. The standard also requires the RESTRICT key word, which is optional in PostgreSQL.

See Also

alter_group

ALTER GROUP

ALTER GROUP

7

SQL - Language Statements

ALTER GROUP

change role name or membership

ALTER GROUP

role_specification

ADD USER

user_name

[, ... ] ALTER GROUP

role_specification

DROP USER

user_name

[, ... ]

where

role_specification

can be:

role_name

| CURRENT_ROLE | CURRENT_USER | SESSION_USER ALTER GROUP

group_name

RENAME TO

new_name

Description

ALTER GROUP changes the attributes of a user group. This is an obsolete command, though still accepted for backwards compatibility, because groups (and users too) have been superseded by the more general concept of roles.

The first two variants add users to a group or remove them from a group. (Any role can play the part of either a “user” or a “group” for this purpose.) These variants are effectively equivalent to granting or revoking membership in the role named as the “group”; so the preferred way to do this is to use GRANT or REVOKE. Note that GRANT and REVOKE have additional options which are not available with this command, such as the ability to grant and revoke ADMIN OPTION, and the ability to specify the grantor.

The third variant changes the name of the group. This is exactly equivalent to renaming the role with ALTER ROLE.

Parameters

<group_name>
The name of the group (role) to modify.

<user_name>
Users (roles) that are to be added to or removed from the group. The users must already exist; ALTER GROUP does not create or drop users.

<new_name>
The new name of the group.

Examples

Add users to a group:

ALTER GROUP staff ADD USER karl, john;

Remove a user from a group:

ALTER GROUP workers DROP USER beth;

Compatibility

There is no ALTER GROUP statement in the SQL standard.

See Also

alter_index

ALTER INDEX

ALTER INDEX

7

SQL - Language Statements

ALTER INDEX

change the definition of an index

ALTER INDEX [ IF EXISTS ]

name

RENAME TO

new_name

ALTER INDEX [ IF EXISTS ]

name

SET TABLESPACE

tablespace_name

ALTER INDEX

name

ATTACH PARTITION

index_name

ALTER INDEX

name

[ NO ] DEPENDS ON EXTENSION

extension_name

ALTER INDEX [ IF EXISTS ]

name

SET (

storage_parameter

[=

value

] [, ... ] ) ALTER INDEX [ IF EXISTS ]

name

RESET (

storage_parameter

[, ... ] ) ALTER INDEX [ IF EXISTS ]

name

ALTER [ COLUMN ]

column_number

SET STATISTICS

integer

ALTER INDEX ALL IN TABLESPACE

name

[ OWNED BY

role_name

[, ... ] ] SET TABLESPACE

new_tablespace

[ NOWAIT ]

Description

ALTER INDEX changes the definition of an existing index. There are several subforms described below. Note that the lock level required may differ for each subform. An ACCESS EXCLUSIVE lock is held unless explicitly noted. When multiple subcommands are listed, the lock held will be the strictest one required from any subcommand.

RENAME
The RENAME form changes the name of the index. If the index is associated with a table constraint (either UNIQUE, PRIMARY KEY, or EXCLUDE), the constraint is renamed as well. There is no effect on the stored data.

Renaming an index acquires a SHARE UPDATE EXCLUSIVE lock.

SET TABLESPACE
This form changes the index's tablespace to the specified tablespace and moves the data file(s) associated with the index to the new tablespace. To change the tablespace of an index, you must own the index and have CREATE privilege on the new tablespace. All indexes in the current database in a tablespace can be moved by using the ALL IN TABLESPACE form, which will lock all indexes to be moved and then move each one. This form also supports OWNED BY, which will only move indexes owned by the roles specified. If the NOWAIT option is specified then the command will fail if it is unable to acquire all of the locks required immediately. Note that system catalogs will not be moved by this command, use ALTER DATABASE or explicit ALTER INDEX invocations instead if desired. See also CREATE TABLESPACE.

ATTACH PARTITION index_name
Causes the named index (possibly schema-qualified) to become attached to the altered index. The named index must be on a partition of the table containing the index being altered, and have an equivalent definition. An attached index cannot be dropped by itself, and will automatically be dropped if its parent index is dropped.

If the named index is already attached to the altered index, the command will attempt to validate the parent index if the parent is currently invalid.

DEPENDS ON EXTENSION extension_name; NO DEPENDS ON EXTENSION extension_name
This form marks the index as dependent on the extension, or no longer dependent on that extension if NO is specified. An index that's marked as dependent on an extension is automatically dropped when the extension is dropped.

SET ( storage_parameter [= value] [, ... ] )
This form changes one or more index-method-specific storage parameters for the index. See CREATE INDEX for details on the available parameters. Note that the index contents will not be modified immediately by this command; depending on the parameter you might need to rebuild the index with REINDEX to get the desired effects.

RESET ( storage_parameter [, ... ] )
This form resets one or more index-method-specific storage parameters to their defaults. As with SET, a REINDEX might be needed to update the index entirely.

ALTER [ COLUMN ] column_number SET STATISTICS integer
This form sets the per-column statistics-gathering target for subsequent ANALYZE operations, though can be used only on index columns that are defined as an expression. Since expressions lack a unique name, we refer to them using the ordinal number of the index column. The target can be set in the range 0 to 10000; alternatively, set it to -1 to revert to using the system default statistics target (???). For more information on the use of statistics by the PostgreSQL query planner, refer to ???.

Parameters

IF EXISTS
Do not throw an error if the index does not exist. A notice is issued in this case.

<column_number>
The ordinal number refers to the ordinal (left-to-right) position of the index column.

<name>
The name (possibly schema-qualified) of an existing index to alter.

<new_name>
The new name for the index.

<tablespace_name>
The tablespace to which the index will be moved.

<extension_name>
The name of the extension that the index is to depend on.

<storage_parameter>
The name of an index-method-specific storage parameter.

<value>
The new value for an index-method-specific storage parameter. This might be a number or a word depending on the parameter.

Notes

These operations are also possible using ALTER TABLE. ALTER INDEX is in fact just an alias for the forms of ALTER TABLE that apply to indexes.

There was formerly an ALTER INDEX OWNER variant, but this is now ignored (with a warning). An index cannot have an owner different from its table's owner. Changing the table's owner automatically changes the index as well.

Changing any part of a system catalog index is not permitted.

Examples

To rename an existing index:

ALTER INDEX distributors RENAME TO suppliers;

To move an index to a different tablespace:

ALTER INDEX distributors SET TABLESPACE fasttablespace;

To change an index's fill factor (assuming that the index method supports it):

ALTER INDEX distributors SET (fillfactor = 75);
REINDEX INDEX distributors;

Set the statistics-gathering target for an expression index:

CREATE INDEX coord_idx ON measured (x, y, (z + t));
ALTER INDEX coord_idx ALTER COLUMN 3 SET STATISTICS 1000;

Compatibility

ALTER INDEX is a PostgreSQL extension.

See Also

alter_language

ALTER LANGUAGE

ALTER LANGUAGE

7

SQL - Language Statements

ALTER LANGUAGE

change the definition of a procedural language

ALTER [ PROCEDURAL ] LANGUAGE

name

RENAME TO

new_name

ALTER [ PROCEDURAL ] LANGUAGE

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER }

Description

ALTER LANGUAGE changes the definition of a procedural language. The only functionality is to rename the language or assign a new owner. You must be superuser or owner of the language to use ALTER LANGUAGE.

Parameters

<name>
Name of a language

<new_name>
The new name of the language

<new_owner>
The new owner of the language

Compatibility

There is no ALTER LANGUAGE statement in the SQL standard.

See Also

alter_large_object

ALTER LARGE OBJECT

ALTER LARGE OBJECT

7

SQL - Language Statements

ALTER LARGE OBJECT

change the definition of a large object

ALTER LARGE OBJECT

large_object_oid

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER }

Description

ALTER LARGE OBJECT changes the definition of a large object.

You must own the large object to use ALTER LARGE OBJECT. To alter the owner, you must also be able to SET ROLE to the new owning role. (However, a superuser can alter any large object anyway.) Currently, the only functionality is to assign a new owner, so both restrictions always apply.

Parameters

<large_object_oid>
OID of the large object to be altered

<new_owner>
The new owner of the large object

Compatibility

There is no ALTER LARGE OBJECT statement in the SQL standard.

See Also

alter_materialized_view

ALTER MATERIALIZED VIEW

ALTER MATERIALIZED VIEW

7

SQL - Language Statements

ALTER MATERIALIZED VIEW

change the definition of a materialized view

ALTER MATERIALIZED VIEW [ IF EXISTS ]

name

action

[, ... ] ALTER MATERIALIZED VIEW

name

[ NO ] DEPENDS ON EXTENSION

extension_name

ALTER MATERIALIZED VIEW [ IF EXISTS ]

name

RENAME [ COLUMN ]

column_name

TO

new_column_name

ALTER MATERIALIZED VIEW [ IF EXISTS ]

name

RENAME TO

new_name

ALTER MATERIALIZED VIEW [ IF EXISTS ]

name

SET SCHEMA

new_schema

ALTER MATERIALIZED VIEW ALL IN TABLESPACE

name

[ OWNED BY

role_name

[, ... ] ] SET TABLESPACE

new_tablespace

[ NOWAIT ]

where

action

is one of:

ALTER [ COLUMN ]

column_name

SET STATISTICS

integer

ALTER [ COLUMN ]

column_name

SET (

attribute_option

=

value

[, ... ] ) ALTER [ COLUMN ]

column_name

RESET (

attribute_option

[, ... ] ) ALTER [ COLUMN ]

column_name

SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN | DEFAULT } ALTER [ COLUMN ]

column_name

SET COMPRESSION

compression_method

CLUSTER ON

index_name

SET WITHOUT CLUSTER SET ACCESS METHOD

new_access_method

SET TABLESPACE

new_tablespace

SET (

storage_parameter

[=

value

] [, ... ] ) RESET (

storage_parameter

[, ... ] ) OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER }

Description

ALTER MATERIALIZED VIEW changes various auxiliary properties of an existing materialized view.

You must own the materialized view to use ALTER MATERIALIZED VIEW. To change a materialized view's schema, you must also have CREATE privilege on the new schema. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the materialized view's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the materialized view. However, a superuser can alter ownership of any view anyway.)

The statement subforms and actions available for ALTER MATERIALIZED VIEW are a subset of those available for ALTER TABLE, and have the same meaning when used for materialized views. See the descriptions for ALTER TABLE for details.

Parameters

<name>
The name (optionally schema-qualified) of an existing materialized view.

<column_name>
Name of an existing column.

<extension_name>
The name of the extension that the materialized view is to depend on (or no longer dependent on, if NO is specified). A materialized view that's marked as dependent on an extension is automatically dropped when the extension is dropped.

<new_column_name>
New name for an existing column.

<new_owner>
The user name of the new owner of the materialized view.

<new_name>
The new name for the materialized view.

<new_schema>
The new schema for the materialized view.

Examples

To rename the materialized view foo to bar:

ALTER MATERIALIZED VIEW foo RENAME TO bar;

Compatibility

ALTER MATERIALIZED VIEW is a PostgreSQL extension.

See Also

alter_opclass

ALTER OPERATOR CLASS

ALTER OPERATOR CLASS

7

SQL - Language Statements

ALTER OPERATOR CLASS

change the definition of an operator class

ALTER OPERATOR CLASS

name

USING

index_method

RENAME TO

new_name

ALTER OPERATOR CLASS

name

USING

index_method

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER OPERATOR CLASS

name

USING

index_method

SET SCHEMA

new_schema

Description

ALTER OPERATOR CLASS changes the definition of an operator class.

You must own the operator class to use ALTER OPERATOR CLASS. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the operator class's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the operator class. However, a superuser can alter ownership of any operator class anyway.)

Parameters

<name>
The name (optionally schema-qualified) of an existing operator class.

<index_method>
The name of the index method this operator class is for.

<new_name>
The new name of the operator class.

<new_owner>
The new owner of the operator class.

<new_schema>
The new schema for the operator class.

Compatibility

There is no ALTER OPERATOR CLASS statement in the SQL standard.

See Also

alter_operator

ALTER OPERATOR

ALTER OPERATOR

7

SQL - Language Statements

ALTER OPERATOR

change the definition of an operator

ALTER OPERATOR

name

( {

left_type

| NONE } ,

right_type

) OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER OPERATOR

name

( {

left_type

| NONE } ,

right_type

) SET SCHEMA

new_schema

ALTER OPERATOR

name

( {

left_type

| NONE } ,

right_type

) SET ( { RESTRICT = {

res_proc

| NONE } | JOIN = {

join_proc

| NONE } | COMMUTATOR =

com_op

| NEGATOR =

neg_op

| HASHES | MERGES } [, ... ] )

Description

ALTER OPERATOR changes the definition of an operator.

You must own the operator to use ALTER OPERATOR. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the operator's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the operator. However, a superuser can alter ownership of any operator anyway.)

Parameters

<name>
The name (optionally schema-qualified) of an existing operator.

<left_type>
The data type of the operator's left operand; write NONE if the operator has no left operand.

<right_type>
The data type of the operator's right operand.

<new_owner>
The new owner of the operator.

<new_schema>
The new schema for the operator.

<res_proc>
The restriction selectivity estimator function for this operator; write NONE to remove existing selectivity estimator.

<join_proc>
The join selectivity estimator function for this operator; write NONE to remove existing selectivity estimator.

<com_op>
The commutator of this operator. Can only be changed if the operator does not have an existing commutator.

<neg_op>
The negator of this operator. Can only be changed if the operator does not have an existing negator.

HASHES
Indicates this operator can support a hash join. Can only be enabled and not disabled.

MERGES
Indicates this operator can support a merge join. Can only be enabled and not disabled.

Notes

Refer to ??? and ??? for further information.

Since commutators come in pairs that are commutators of each other, ALTER OPERATOR SET COMMUTATOR will also set the commutator of the <com_op> to be the target operator. Likewise, ALTER OPERATOR SET NEGATOR will also set the negator of the <neg_op> to be the target operator. Therefore, you must own the commutator or negator operator as well as the target operator.

Examples

Change the owner of a custom operator a @@ b for type text:

ALTER OPERATOR @@ (text, text) OWNER TO joe;

Change the restriction and join selectivity estimator functions of a custom operator a && b for type int[]:

ALTER OPERATOR && (int[], int[]) SET (RESTRICT = _int_contsel, JOIN = _int_contjoinsel);

Mark the && operator as being its own commutator:

ALTER OPERATOR && (int[], int[]) SET (COMMUTATOR = &&);

Compatibility

There is no ALTER OPERATOR statement in the SQL standard.

See Also

alter_opfamily

ALTER OPERATOR FAMILY

ALTER OPERATOR FAMILY

7

SQL - Language Statements

ALTER OPERATOR FAMILY

change the definition of an operator family

ALTER OPERATOR FAMILY

name

USING

index_method

ADD { OPERATOR

strategy_number

operator_name

(

op_type

,

op_type

) [ FOR SEARCH | FOR ORDER BY

sort_family_name

] | FUNCTION

support_number

[ (

op_type

[ ,

op_type

] ) ]

function_name

[ (

argument_type

[, ...] ) ] } [, ... ] ALTER OPERATOR FAMILY

name

USING

index_method

DROP { OPERATOR

strategy_number

(

op_type

[ ,

op_type

] ) | FUNCTION

support_number

(

op_type

[ ,

op_type

] ) } [, ... ] ALTER OPERATOR FAMILY

name

USING

index_method

RENAME TO

new_name

ALTER OPERATOR FAMILY

name

USING

index_method

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER OPERATOR FAMILY

name

USING

index_method

SET SCHEMA

new_schema

Description

ALTER OPERATOR FAMILY changes the definition of an operator family. You can add operators and support functions to the family, remove them from the family, or change the family's name or owner.

When operators and support functions are added to a family with ALTER OPERATOR FAMILY, they are not part of any specific operator class within the family, but are just “loose” within the family. This indicates that these operators and functions are compatible with the family's semantics, but are not required for correct functioning of any specific index. (Operators and functions that are so required should be declared as part of an operator class, instead; see ???.) PostgreSQL will allow loose members of a family to be dropped from the family at any time, but members of an operator class cannot be dropped without dropping the whole class and any indexes that depend on it. Typically, single-data-type operators and functions are part of operator classes because they are needed to support an index on that specific data type, while cross-data-type operators and functions are made loose members of the family.

You must be a superuser to use ALTER OPERATOR FAMILY. (This restriction is made because an erroneous operator family definition could confuse or even crash the server.)

ALTER OPERATOR FAMILY does not presently check whether the operator family definition includes all the operators and functions required by the index method, nor whether the operators and functions form a self-consistent set. It is the user's responsibility to define a valid operator family.

Refer to ??? for further information.

Parameters

<name>
The name (optionally schema-qualified) of an existing operator family.

<index_method>
The name of the index method this operator family is for.

<strategy_number>
The index method's strategy number for an operator associated with the operator family.

<operator_name>
The name (optionally schema-qualified) of an operator associated with the operator family.

<op_type>
In an OPERATOR clause, the operand data type(s) of the operator, or NONE to signify a prefix operator. Unlike the comparable syntax in CREATE OPERATOR CLASS, the operand data types must always be specified.

In an ADD FUNCTION clause, the operand data type(s) the function is intended to support, if different from the input data type(s) of the function. For B-tree comparison functions and hash functions it is not necessary to specify <op_type> since the function's input data type(s) are always the correct ones to use. For B-tree sort support functions, B-Tree equal image functions, and all functions in GiST, SP-GiST and GIN operator classes, it is necessary to specify the operand data type(s) the function is to be used with.

In a DROP FUNCTION clause, the operand data type(s) the function is intended to support must be specified.

<sort_family_name>
The name (optionally schema-qualified) of an existing btree operator family that describes the sort ordering associated with an ordering operator.

If neither FOR SEARCH nor FOR ORDER BY is specified, FOR SEARCH is the default.

<support_number>
The index method's support function number for a function associated with the operator family.

<function_name>
The name (optionally schema-qualified) of a function that is an index method support function for the operator family. If no argument list is specified, the name must be unique in its schema.

<argument_type>
The parameter data type(s) of the function.

<new_name>
The new name of the operator family.

<new_owner>
The new owner of the operator family.

<new_schema>
The new schema for the operator family.

The OPERATOR and FUNCTION clauses can appear in any order.

Notes

Notice that the DROP syntax only specifies the “slot” in the operator family, by strategy or support number and input data type(s). The name of the operator or function occupying the slot is not mentioned. Also, for DROP FUNCTION the type(s) to specify are the input data type(s) the function is intended to support; for GiST, SP-GiST and GIN indexes this might have nothing to do with the actual input argument types of the function.

Because the index machinery does not check access permissions on functions before using them, including a function or operator in an operator family is tantamount to granting public execute permission on it. This is usually not an issue for the sorts of functions that are useful in an operator family.

The operators should not be defined by SQL functions. An SQL function is likely to be inlined into the calling query, which will prevent the optimizer from recognizing that the query matches an index.

Examples

The following example command adds cross-data-type operators and support functions to an operator family that already contains B-tree operator classes for data types int4 and int2.

ALTER OPERATOR FAMILY integer_ops USING btree ADD

  -- int4 vs int2
  OPERATOR 1 < (int4, int2) ,
  OPERATOR 2 <= (int4, int2) ,
  OPERATOR 3 = (int4, int2) ,
  OPERATOR 4 >= (int4, int2) ,
  OPERATOR 5 > (int4, int2) ,
  FUNCTION 1 btint42cmp(int4, int2) ,

  -- int2 vs int4
  OPERATOR 1 < (int2, int4) ,
  OPERATOR 2 <= (int2, int4) ,
  OPERATOR 3 = (int2, int4) ,
  OPERATOR 4 >= (int2, int4) ,
  OPERATOR 5 > (int2, int4) ,
  FUNCTION 1 btint24cmp(int2, int4) ;

To remove these entries again:

ALTER OPERATOR FAMILY integer_ops USING btree DROP

  -- int4 vs int2
  OPERATOR 1 (int4, int2) ,
  OPERATOR 2 (int4, int2) ,
  OPERATOR 3 (int4, int2) ,
  OPERATOR 4 (int4, int2) ,
  OPERATOR 5 (int4, int2) ,
  FUNCTION 1 (int4, int2) ,

  -- int2 vs int4
  OPERATOR 1 (int2, int4) ,
  OPERATOR 2 (int2, int4) ,
  OPERATOR 3 (int2, int4) ,
  OPERATOR 4 (int2, int4) ,
  OPERATOR 5 (int2, int4) ,
  FUNCTION 1 (int2, int4) ;

Compatibility

There is no ALTER OPERATOR FAMILY statement in the SQL standard.

See Also

alter_policy

ALTER POLICY

ALTER POLICY

7

SQL - Language Statements

ALTER POLICY

change the definition of a row-level security policy

ALTER POLICY

name

ON

table_name

RENAME TO

new_name

ALTER POLICY

name

ON

table_name

[ TO {

role_name

| PUBLIC | CURRENT_ROLE | CURRENT_USER | SESSION_USER } [, ...] ] [ USING (

using_expression

) ] [ WITH CHECK (

check_expression

) ]

Description

ALTER POLICY changes the definition of an existing row-level security policy. Note that ALTER POLICY only allows the set of roles to which the policy applies and the USING and WITH CHECK expressions to be modified. To change other properties of a policy, such as the command to which it applies or whether it is permissive or restrictive, the policy must be dropped and recreated.

To use ALTER POLICY, you must own the table that the policy applies to.

In the second form of ALTER POLICY, the role list, <using_expression>, and <check_expression> are replaced independently if specified. When one of those clauses is omitted, the corresponding part of the policy is unchanged.

Parameters

<name>
The name of an existing policy to alter.

<table_name>
The name (optionally schema-qualified) of the table that the policy is on.

<new_name>
The new name for the policy.

<role_name>
The role(s) to which the policy applies. Multiple roles can be specified at one time. To apply the policy to all roles, use PUBLIC.

<using_expression>
The USING expression for the policy. See ??? for details.

<check_expression>
The WITH CHECK expression for the policy. See ??? for details.

Compatibility

ALTER POLICY is a PostgreSQL extension.

See Also

alter_procedure

ALTER PROCEDURE

ALTER PROCEDURE

7

SQL - Language Statements

ALTER PROCEDURE

change the definition of a procedure

ALTER PROCEDURE

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ]

action

[ ... ] [ RESTRICT ] ALTER PROCEDURE

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] RENAME TO

new_name

ALTER PROCEDURE

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER PROCEDURE

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] SET SCHEMA

new_schema

ALTER PROCEDURE

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] [ NO ] DEPENDS ON EXTENSION

extension_name

where

action

is one of:

[ EXTERNAL ] SECURITY INVOKER | [ EXTERNAL ] SECURITY DEFINER SET

configuration_parameter

{ TO | = } {

value

| DEFAULT } SET

configuration_parameter

FROM CURRENT RESET

configuration_parameter

RESET ALL

Description

ALTER PROCEDURE changes the definition of a procedure.

You must own the procedure to use ALTER PROCEDURE. To change a procedure's schema, you must also have CREATE privilege on the new schema. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the procedure's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the procedure. However, a superuser can alter ownership of any procedure anyway.)

Parameters

<name>
The name (optionally schema-qualified) of an existing procedure. If no argument list is specified, the name must be unique in its schema.

<argmode>
The mode of an argument: IN, OUT, INOUT, or VARIADIC. If omitted, the default is IN.

<argname>
The name of an argument. Note that ALTER PROCEDURE does not actually pay any attention to argument names, since only the argument data types are used to determine the procedure's identity.

<argtype>
The data type(s) of the procedure's arguments (optionally schema-qualified), if any. See ??? for the details of how the procedure is looked up using the argument data type(s).

<new_name>
The new name of the procedure.

<new_owner>
The new owner of the procedure. Note that if the procedure is marked SECURITY DEFINER, it will subsequently execute as the new owner.

<new_schema>
The new schema for the procedure.

<extension_name>
This form marks the procedure as dependent on the extension, or no longer dependent on the extension if NO is specified. A procedure that's marked as dependent on an extension is dropped when the extension is dropped, even if cascade is not specified. A procedure can depend upon multiple extensions, and will be dropped when any one of those extensions is dropped.

EXTERNAL SECURITY INVOKER; EXTERNAL SECURITY DEFINER
Change whether the procedure is a security definer or not. The key word EXTERNAL is ignored for SQL conformance. See ??? for more information about this capability.

<configuration_parameter>; <value>
Add or change the assignment to be made to a configuration parameter when the procedure is called. If <value> is DEFAULT or, equivalently, RESET is used, the procedure-local setting is removed, so that the procedure executes with the value present in its environment. Use RESET ALL to clear all procedure-local settings. SET FROM CURRENT saves the value of the parameter that is current when ALTER PROCEDURE is executed as the value to be applied when the procedure is entered.

See ??? and ??? for more information about allowed parameter names and values.

RESTRICT
Ignored for conformance with the SQL standard.

Examples

To rename the procedure insert_data with two arguments of type integer to insert_record:

ALTER PROCEDURE insert_data(integer, integer) RENAME TO insert_record;

To change the owner of the procedure insert_data with two arguments of type integer to joe:

ALTER PROCEDURE insert_data(integer, integer) OWNER TO joe;

To change the schema of the procedure insert_data with two arguments of type integer to accounting:

ALTER PROCEDURE insert_data(integer, integer) SET SCHEMA accounting;

To mark the procedure insert_data(integer, integer) as being dependent on the extension myext:

ALTER PROCEDURE insert_data(integer, integer) DEPENDS ON EXTENSION myext;

ALTER PROCEDURE check_password(text) SET search_path = admin, pg_temp;

To disable automatic setting of search_path for a procedure:

ALTER PROCEDURE check_password(text) RESET search_path;

The procedure will now execute with whatever search path is used by its caller.

Compatibility

This statement is partially compatible with the ALTER PROCEDURE statement in the SQL standard. The standard allows more properties of a procedure to be modified, but does not provide the ability to rename a procedure, make a procedure a security definer, attach configuration parameter values to a procedure, or change the owner, schema, or volatility of a procedure. The standard also requires the RESTRICT key word, which is optional in PostgreSQL.

See Also

alter_publication

ALTER PUBLICATION

ALTER PUBLICATION

7

SQL - Language Statements

ALTER PUBLICATION

change the definition of a publication

ALTER PUBLICATION

name

ADD

publication_object

[, ...] ALTER PUBLICATION

name

DROP

publication_drop_object

[, ...] ALTER PUBLICATION

name

SET {

publication_object

[, ...] |

publication_all_object

[, ... ] } ALTER PUBLICATION

name

SET (

publication_parameter

[=

value

] [, ... ] ) ALTER PUBLICATION

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER PUBLICATION

name

RENAME TO

new_name

where

publication_object

is one of:

TABLE

table_and_columns

[, ... ] TABLES IN SCHEMA {

schema_name

| CURRENT_SCHEMA } [, ... ]

and

publication_all_object

is one of:

ALL TABLES [ EXCEPT (

except_table_object

[, ... ] ) ] ALL SEQUENCES

and

publication_drop_object

is one of:

TABLE

table_object

[, ... ] TABLES IN SCHEMA {

schema_name

| CURRENT_SCHEMA } [, ... ]

and

table_and_columns

is:

table_object

[ (

column_name

[, ... ] ) ] [ WHERE (

expression

) ]

and

except_table_object

is:

TABLE

table_object

[, ... ]

and

table_object

is:

[ ONLY ]

table_name

[ * ]

Description

The command ALTER PUBLICATION can change the attributes of a publication.

The first two variants modify which tables/schemas are part of the publication. The ADD and DROP clauses will add and remove one or more tables/schemas from the publication.

The third variant either modifies the included tables/schemas or marks the publication as FOR ALL SEQUENCES or FOR ALL TABLES, optionally using EXCEPT to exclude specific tables. The SET ALL TABLES clause can transform an empty publication, or one defined for ALL SEQUENCES (or both ALL TABLES and ALL SEQUENCES), into a publication defined for ALL TABLES. Likewise, SET ALL SEQUENCES can convert an empty publication, or one defined for ALL TABLES (or both ALL TABLES and ALL SEQUENCES), into a publication defined for ALL SEQUENCES. In addition, SET ALL TABLES can be used to update the tables specified in the EXCEPT clause of a FOR ALL TABLES publication. If EXCEPT is specified with a list of tables, the existing exclusion list is replaced with the specified tables. If EXCEPT is omitted, the existing exclusion list is cleared. The SET clause, when used with a publication defined with FOR TABLE or FOR TABLES IN SCHEMA, replaces the list of tables/schemas in the publication with the specified list; the existing tables or schemas that were present in the publication will be removed.

Note that adding, dropping, or setting tables or schemas in a publication that is already subscribed to will require an ALTER SUBSCRIPTION ... REFRESH PUBLICATION action on the subscribing side in order to become effective. Likewise altering a publication to set ALL TABLES, to change the EXCEPT list, or to set ALL SEQUENCES also requires the subscriber to refresh the publication. Note also that DROP TABLES IN SCHEMA will not drop any schema tables that were specified using FOR TABLE/ ADD TABLE.

The fourth variant of this command listed in the synopsis can change all of the publication properties specified in ???. Properties not mentioned in the command retain their previous settings.

The remaining variants change the owner and the name of the publication.

You must own the publication to use ALTER PUBLICATION. Adding a table to a publication additionally requires owning that table. The ADD TABLES IN SCHEMA, SET TABLES IN SCHEMA, SET ALL TABLES, and SET ALL SEQUENCES to a publication requires the invoking user to be a superuser. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the database. Also, the new owner of a FOR TABLES IN SCHEMA or FOR ALL TABLES or FOR ALL SEQUENCES publication must be a superuser. However, a superuser can change the ownership of a publication regardless of these restrictions.

Adding/Setting any schema when the publication also publishes a table with a column list, and vice versa is not supported.

Parameters

<name>
The name of an existing publication whose definition is to be altered.

<table_name>
Name of an existing table. If ONLY is specified before the <table_name>, only that table is affected. If ONLY is not specified, the table and all its descendant tables (if any) are affected. Optionally, * can be specified after the <table_name> to explicitly indicate that descendant tables are included.

Optionally, a column list can be specified. See ??? for details. Note that a subscription having several publications in which the same table has been published with different column lists is not supported. See ??? for details of potential problems when altering column lists.

If the optional WHERE clause is specified, rows for which the <expression> evaluates to false or null will not be published. Note that parentheses are required around the expression. The <expression> is evaluated with the role used for the replication connection.

<column_name>
Name of an existing column of <table_name>.

<schema_name>
Name of an existing schema.

SET ( publication_parameter [= value] [, ... ] )
This clause alters publication parameters originally set by ???. See there for more information. This clause is not applicable to sequences.

[!CAUTION] Altering the publish_via_partition_root parameter can lead to data loss or duplication at the subscriber because it changes the identity and schema of the published tables. Note this happens only when a partition root table is specified as the replication target.

This problem can be avoided by refraining from modifying partition leaf tables after the ALTER PUBLICATION ... SET until the ALTER SUBSCRIPTION ... REFRESH PUBLICATION is executed and by only refreshing using the copy_data = off option.

<new_owner>
The user name of the new owner of the publication.

<new_name>
The new name for the publication.

Examples

Change the publication to publish only deletes and updates:

ALTER PUBLICATION noinsert SET (publish = 'update, delete');

Add some tables to the publication:

ALTER PUBLICATION mypublication ADD TABLE users (user_id, firstname), departments;

Change the set of columns published for a table:

ALTER PUBLICATION mypublication SET TABLE users (user_id, firstname, lastname), TABLE departments;

Replace the table list in the publication's EXCEPT clause:

ALTER PUBLICATION mypublication SET ALL TABLES EXCEPT (TABLE users, departments);

Reset the publication to be a FOR ALL TABLES publication with no excluded tables:

ALTER PUBLICATION mypublication SET ALL TABLES;

Add schemas marketing and sales to the publication sales_publication:

ALTER PUBLICATION sales_publication ADD TABLES IN SCHEMA marketing, sales;

Add tables users, departments and schema production to the publication production_publication:

ALTER PUBLICATION production_publication ADD TABLE users, departments, TABLES IN SCHEMA production;

Compatibility

ALTER PUBLICATION is a PostgreSQL extension.

See Also

alter_role

ALTER ROLE

ALTER ROLE

7

SQL - Language Statements

ALTER ROLE

change a database role

ALTER ROLE

role_specification

[ WITH ]

option

[ ... ]

where

option

can be:

SUPERUSER | NOSUPERUSER | CREATEDB | NOCREATEDB | CREATEROLE | NOCREATEROLE | INHERIT | NOINHERIT | LOGIN | NOLOGIN | REPLICATION | NOREPLICATION | BYPASSRLS | NOBYPASSRLS | CONNECTION LIMIT

connlimit

| [ ENCRYPTED ] PASSWORD '

password

' | PASSWORD NULL | VALID UNTIL '

timestamp

' ALTER ROLE

name

RENAME TO

new_name

ALTER ROLE {

role_specification

| ALL } [ IN DATABASE

database_name

] SET

configuration_parameter

{ TO | = } {

value

| DEFAULT } ALTER ROLE {

role_specification

| ALL } [ IN DATABASE

database_name

] SET

configuration_parameter

FROM CURRENT ALTER ROLE {

role_specification

| ALL } [ IN DATABASE

database_name

] RESET

configuration_parameter

ALTER ROLE {

role_specification

| ALL } [ IN DATABASE

database_name

] RESET ALL

where

role_specification

can be:

role_name

| CURRENT_ROLE | CURRENT_USER | SESSION_USER

Description

ALTER ROLE changes the attributes of a PostgreSQL role.

The first variant of this command listed in the synopsis can change many of the role attributes that can be specified in CREATE ROLE. (All the possible attributes are covered, except that there are no options for adding or removing memberships; use GRANT and REVOKE for that.) Attributes not mentioned in the command retain their previous settings. Database superusers can change any of these settings for any role, except for changing the SUPERUSER property for the bootstrap superuser. Non-superuser roles having CREATEROLE privilege can change most of these properties, but only for non-superuser and non-replication roles for which they have been granted ADMIN OPTION. Non-superusers cannot change the SUPERUSER property and can change the CREATEDB, REPLICATION, and BYPASSRLS properties only if they possess the corresponding property themselves. Ordinary roles can only change their own password.

The second variant changes the name of the role. Database superusers can rename any role. Roles having CREATEROLE privilege can rename non-superuser roles for which they have been granted ADMIN OPTION. The current session user cannot be renamed. (Connect as a different user if you need to do that.) Because MD5-encrypted passwords use the role name as cryptographic salt, renaming a role clears its password if the password is MD5-encrypted.

The remaining variants change a role's session default for a configuration variable, either for all databases or, when the IN DATABASE clause is specified, only for sessions in the named database. If ALL is specified instead of a role name, this changes the setting for all roles. Using ALL with IN DATABASE is effectively the same as using the command ALTER DATABASE ... SET ....

Whenever the role subsequently starts a new session, the specified value becomes the session default, overriding whatever setting is present in postgresql.conf or has been received from the postgres command line. This only happens at login time; executing SET ROLE or SET SESSION AUTHORIZATION does not cause new configuration values to be set. Settings set for all databases are overridden by database-specific settings attached to a role. Settings for specific databases or specific roles override settings for all roles.

Superusers can change anyone's session defaults. Roles having CREATEROLE privilege can change defaults for non-superuser roles for which they have been granted ADMIN OPTION. Ordinary roles can only set defaults for themselves. Certain configuration variables cannot be set this way, or can only be set if a superuser issues the command. Only superusers can change a setting for all roles in all databases.

Parameters

<name>
The name of the role whose attributes are to be altered.

CURRENT_ROLE; CURRENT_USER
Alter the current user instead of an explicitly identified role.

SESSION_USER
Alter the current session user instead of an explicitly identified role.

SUPERUSER; NOSUPERUSER; CREATEDB; NOCREATEDB; CREATEROLE; NOCREATEROLE; INHERIT; NOINHERIT; LOGIN; NOLOGIN; REPLICATION; NOREPLICATION; BYPASSRLS; NOBYPASSRLS; CONNECTION LIMIT <connlimit>; [ ENCRYPTED ] PASSWORD '<password>'; PASSWORD NULL; VALID UNTIL '<timestamp>'
These clauses alter attributes originally set by CREATE ROLE. For more information, see the CREATE ROLE reference page.

<new_name>
The new name of the role.

<database_name>
The name of the database the configuration variable should be set in.

<configuration_parameter>; <value>
Set this role's session default for the specified configuration parameter to the given value. If <value> is DEFAULT or, equivalently, RESET is used, the role-specific variable setting is removed, so the role will inherit the system-wide default setting in new sessions. Use RESET ALL to clear all role-specific settings. SET FROM CURRENT saves the session's current value of the parameter as the role-specific value. If IN DATABASE is specified, the configuration parameter is set or removed for the given role and database only.

Role-specific variable settings take effect only at login; SET ROLE and SET SESSION AUTHORIZATION do not process role-specific variable settings.

See ??? and ??? for more information about allowed parameter names and values.

Notes

Use CREATE ROLE to add new roles, and DROP ROLE to remove a role.

ALTER ROLE cannot change a role's memberships. Use GRANT and REVOKE to do that.

Caution must be exercised when specifying an unencrypted password with this command. The password will be transmitted to the server in cleartext, and it might also be logged in the client's command history or the server log. ??? contains a command \password that can be used to change a role's password without exposing the cleartext password.

It is also possible to tie a session default to a specific database rather than to a role; see ???. If there is a conflict, database-role-specific settings override role-specific ones, which in turn override database-specific ones.

Examples

Change a role's password:

ALTER ROLE davide WITH PASSWORD 'hu8jmn3';

Remove a role's password:

ALTER ROLE davide WITH PASSWORD NULL;

Change a password expiration date, specifying that the password should expire at midday on 4th May 2015 using the time zone which is one hour ahead of UTC:

ALTER ROLE chris VALID UNTIL 'May 4 12:00:00 2015 +1';

Make a password valid forever:

ALTER ROLE fred VALID UNTIL 'infinity';

Give a role the ability to manage other roles and create new databases:

ALTER ROLE miriam CREATEROLE CREATEDB;

Give a role a non-default setting of the ??? parameter:

ALTER ROLE worker_bee SET maintenance_work_mem = 100000;

Give a role a non-default, database-specific setting of the ??? parameter:

ALTER ROLE fred IN DATABASE devel SET client_min_messages = DEBUG;

Compatibility

The ALTER ROLE statement is a PostgreSQL extension.

See Also

alter_routine

ALTER ROUTINE

ALTER ROUTINE

7

SQL - Language Statements

ALTER ROUTINE

change the definition of a routine

ALTER ROUTINE

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ]

action

[ ... ] [ RESTRICT ] ALTER ROUTINE

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] RENAME TO

new_name

ALTER ROUTINE

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER ROUTINE

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] SET SCHEMA

new_schema

ALTER ROUTINE

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] [ NO ] DEPENDS ON EXTENSION

extension_name

where

action

is one of:

IMMUTABLE | STABLE | VOLATILE [ NOT ] LEAKPROOF [ EXTERNAL ] SECURITY INVOKER | [ EXTERNAL ] SECURITY DEFINER PARALLEL { UNSAFE | RESTRICTED | SAFE } COST

execution_cost

ROWS

result_rows

SET

configuration_parameter

{ TO | = } {

value

| DEFAULT } SET

configuration_parameter

FROM CURRENT RESET

configuration_parameter

RESET ALL

Description

ALTER ROUTINE changes the definition of a routine, which can be an aggregate function, a normal function, or a procedure. See under ???, ???, and ??? for the description of the parameters, more examples, and further details.

Examples

To rename the routine foo for type integer to foobar:

ALTER ROUTINE foo(integer) RENAME TO foobar;

This command will work independent of whether foo is an aggregate, function, or procedure.

Compatibility

This statement is partially compatible with the ALTER ROUTINE statement in the SQL standard. See under ??? and ??? for more details. Allowing routine names to refer to aggregate functions is a PostgreSQL extension.

See Also

Note that there is no CREATE ROUTINE command.

alter_rule

ALTER RULE

ALTER RULE

7

SQL - Language Statements

ALTER RULE

change the definition of a rule

ALTER RULE

name

ON

table_name

RENAME TO

new_name

Description

ALTER RULE changes properties of an existing rule. Currently, the only available action is to change the rule's name.

To use ALTER RULE, you must own the table or view that the rule applies to.

Parameters

<name>
The name of an existing rule to alter.

<table_name>
The name (optionally schema-qualified) of the table or view that the rule applies to.

<new_name>
The new name for the rule.

Examples

To rename an existing rule:

ALTER RULE notify_all ON emp RENAME TO notify_me;

Compatibility

ALTER RULE is a PostgreSQL language extension, as is the entire query rewrite system.

See Also

alter_schema

ALTER SCHEMA

ALTER SCHEMA

7

SQL - Language Statements

ALTER SCHEMA

change the definition of a schema

ALTER SCHEMA

name

RENAME TO

new_name

ALTER SCHEMA

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER }

Description

ALTER SCHEMA changes the definition of a schema.

You must own the schema to use ALTER SCHEMA. To rename a schema you must also have the CREATE privilege for the database. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have the CREATE privilege for the database. (Note that superusers have all these privileges automatically.)

Parameters

<name>
The name of an existing schema.

<new_name>
The new name of the schema. The new name cannot begin with pg_, as such names are reserved for system schemas.

<new_owner>
The new owner of the schema.

Compatibility

There is no ALTER SCHEMA statement in the SQL standard.

See Also

alter_sequence

ALTER SEQUENCE

ALTER SEQUENCE

7

SQL - Language Statements

ALTER SEQUENCE

change the definition of a sequence generator

ALTER SEQUENCE [ IF EXISTS ]

name

[ AS

data_type

] [ INCREMENT [ BY ]

increment

] [ MINVALUE

minvalue

| NO MINVALUE ] [ MAXVALUE

maxvalue

| NO MAXVALUE ] [ [ NO ] CYCLE ] [ START [ WITH ]

start

] [ RESTART [ [ WITH ]

restart

] ] [ CACHE

cache

] [ OWNED BY {

table_name

.

column_name

| NONE } ] ALTER SEQUENCE [ IF EXISTS ]

name

SET { LOGGED | UNLOGGED } ALTER SEQUENCE [ IF EXISTS ]

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER SEQUENCE [ IF EXISTS ]

name

RENAME TO

new_name

ALTER SEQUENCE [ IF EXISTS ]

name

SET SCHEMA

new_schema

Description

ALTER SEQUENCE changes the parameters of an existing sequence generator. Any parameters not specifically set in the ALTER SEQUENCE command retain their prior settings.

You must own the sequence to use ALTER SEQUENCE. To change a sequence's schema, you must also have CREATE privilege on the new schema. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the sequence's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the sequence. However, a superuser can alter ownership of any sequence anyway.)

Parameters

<name>
The name (optionally schema-qualified) of a sequence to be altered.

IF EXISTS
Do not throw an error if the sequence does not exist. A notice is issued in this case.

<data_type>
The optional clause AS data_type changes the data type of the sequence. Valid types are smallint, integer, and bigint.

Changing the data type automatically changes the minimum and maximum values of the sequence if and only if the previous minimum and maximum values were the minimum or maximum value of the old data type (in other words, if the sequence had been created using NO MINVALUE or NO MAXVALUE, implicitly or explicitly). Otherwise, the minimum and maximum values are preserved, unless new values are given as part of the same command. If the minimum and maximum values do not fit into the new data type, an error will be generated.

<increment>
The clause INCREMENT BY increment is optional. A positive value will make an ascending sequence, a negative one a descending sequence. If unspecified, the old increment value will be maintained.

<minvalue>; NO MINVALUE
The optional clause MINVALUE minvalue determines the minimum value a sequence can generate. If NO MINVALUE is specified, the defaults of 1 and the minimum value of the data type for ascending and descending sequences, respectively, will be used. If neither option is specified, the current minimum value will be maintained.

<maxvalue>; NO MAXVALUE
The optional clause MAXVALUE maxvalue determines the maximum value for the sequence. If NO MAXVALUE is specified, the defaults of the maximum value of the data type and -1 for ascending and descending sequences, respectively, will be used. If neither option is specified, the current maximum value will be maintained.

CYCLE
The optional CYCLE key word can be used to enable the sequence to wrap around when the <maxvalue> or <minvalue> has been reached by an ascending or descending sequence respectively. If the limit is reached, the next number generated will be the <minvalue> or <maxvalue>, respectively.

NO CYCLE
If the optional NO CYCLE key word is specified, any calls to nextval after the sequence has reached its maximum value will return an error. If neither CYCLE or NO CYCLE are specified, the old cycle behavior will be maintained.

<start>
The optional clause START WITH start changes the recorded start value of the sequence. This has no effect on the current sequence value; it simply sets the value that future ALTER SEQUENCE RESTART commands will use.

<restart>
The optional clause RESTART [ WITH restart ] changes the current value of the sequence. This is similar to calling the setval function with is_called = false: the specified value will be returned by the next call of nextval. Writing RESTART with no <restart> value is equivalent to supplying the start value that was recorded by CREATE SEQUENCE or last set by ALTER SEQUENCE START WITH.

In contrast to a setval call, a RESTART operation on a sequence is transactional and blocks concurrent transactions from obtaining numbers from the same sequence. If that's not the desired mode of operation, setval should be used.

<cache>
The clause CACHE cache enables sequence numbers to be preallocated and stored in memory for faster access. The minimum value is 1 (only one value can be generated at a time, i.e., no cache). If unspecified, the old cache value will be maintained.

SET { LOGGED | UNLOGGED }
This form changes the sequence from unlogged to logged or vice-versa (see ???). It cannot be applied to a temporary sequence.

OWNED BY <table_name>.<column_name>; OWNED BY NONE
The OWNED BY option causes the sequence to be associated with a specific table column, such that if that column (or its whole table) is dropped, the sequence will be automatically dropped as well. If specified, this association replaces any previously specified association for the sequence. The specified table must have the same owner and be in the same schema as the sequence. Specifying OWNED BY NONE removes any existing association, making the sequence “free-standing”.

<new_owner>
The user name of the new owner of the sequence.

<new_name>
The new name for the sequence.

<new_schema>
The new schema for the sequence.

Notes

ALTER SEQUENCE will not immediately affect nextval results in backends, other than the current one, that have preallocated (cached) sequence values. They will use up all cached values prior to noticing the changed sequence generation parameters. The current backend will be affected immediately.

ALTER SEQUENCE does not affect the currval status for the sequence. (Before PostgreSQL 8.3, it sometimes did.)

ALTER SEQUENCE blocks concurrent nextval, currval, lastval, and setval calls.

For historical reasons, ALTER TABLE can be used with sequences too; but the only variants of ALTER TABLE that are allowed with sequences are equivalent to the forms shown above.

Examples

Restart a sequence called serial, at 105:

ALTER SEQUENCE serial RESTART WITH 105;

Compatibility

ALTER SEQUENCE conforms to the SQL standard, except for the AS, START WITH, OWNED BY, OWNER TO, RENAME TO, and SET SCHEMA clauses, which are PostgreSQL extensions.

See Also

alter_server

ALTER SERVER

ALTER SERVER

7

SQL - Language Statements

ALTER SERVER

change the definition of a foreign server

ALTER SERVER

name

[ VERSION '

new_version

' ] [ OPTIONS ( [ ADD | SET | DROP ]

option

['

value

'] [, ... ] ) ] ALTER SERVER

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER SERVER

name

RENAME TO

new_name

Description

ALTER SERVER changes the definition of a foreign server. The first form changes the server version string or the generic options of the server (at least one clause is required). The second form changes the owner of the server.

To alter the server you must be the owner of the server. Additionally to alter the owner, you must be able to SET ROLE to the new owning role, and you must have USAGE privilege on the server's foreign-data wrapper. (Note that superusers satisfy all these criteria automatically.)

Parameters

<name>
The name of an existing server.

<new_version>
New server version.

OPTIONS ( [ ADD | SET | DROP ] option ['value'] [, ... ] )
Change options for the server. ADD, SET, and DROP specify the action to be performed. ADD is assumed if no operation is explicitly specified. Option names must be unique; names and values are also validated using the server's foreign-data wrapper library.

<new_owner>
The user name of the new owner of the foreign server.

<new_name>
The new name for the foreign server.

Examples

Alter server foo, add connection options:

ALTER SERVER foo OPTIONS (host 'foo', dbname 'foodb');

Alter server foo, change version, change host option:

ALTER SERVER foo VERSION '8.4' OPTIONS (SET host 'baz');

Compatibility

ALTER SERVER conforms to ISO/IEC 9075-9 (SQL/MED). The OWNER TO and RENAME forms are PostgreSQL extensions.

See Also

alter_statistics

ALTER STATISTICS

ALTER STATISTICS

7

SQL - Language Statements

ALTER STATISTICS

change the definition of an extended statistics object

ALTER STATISTICS

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER STATISTICS

name

RENAME TO

new_name

ALTER STATISTICS

name

SET SCHEMA

new_schema

ALTER STATISTICS

name

SET STATISTICS {

new_target

| DEFAULT }

Description

ALTER STATISTICS changes the parameters of an existing extended statistics object. Any parameters not specifically set in the ALTER STATISTICS command retain their prior settings.

You must own the statistics object to use ALTER STATISTICS. To change a statistics object's schema, you must also have CREATE privilege on the new schema. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the statistics object's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the statistics object. However, a superuser can alter ownership of any statistics object anyway.)

Parameters

<name>
The name (optionally schema-qualified) of the statistics object to be altered.

<new_owner>
The user name of the new owner of the statistics object.

<new_name>
The new name for the statistics object.

<new_schema>
The new schema for the statistics object.

<new_target>
The statistic-gathering target for this statistics object for subsequent ANALYZE operations. The target can be set in the range 0 to 10000. Set it to DEFAULT to revert to using the system default statistics target (???). (Setting to a value of -1 is an obsolete way spelling to get the same outcome.) For more information on the use of statistics by the PostgreSQL query planner, refer to ???.

Compatibility

There is no ALTER STATISTICS command in the SQL standard.

See Also

alter_subscription

ALTER SUBSCRIPTION

ALTER SUBSCRIPTION

7

SQL - Language Statements

ALTER SUBSCRIPTION

change the definition of a subscription

ALTER SUBSCRIPTION

name

SERVER

server_name

ALTER SUBSCRIPTION

name

CONNECTION '

conninfo

' ALTER SUBSCRIPTION

name

SET PUBLICATION

publication_name

[, ...] [ WITH (

publication_option

[=

value

] [, ... ] ) ] ALTER SUBSCRIPTION

name

ADD PUBLICATION

publication_name

[, ...] [ WITH (

publication_option

[=

value

] [, ... ] ) ] ALTER SUBSCRIPTION

name

DROP PUBLICATION

publication_name

[, ...] [ WITH (

publication_option

[=

value

] [, ... ] ) ] ALTER SUBSCRIPTION

name

REFRESH PUBLICATION [ WITH (

refresh_option

[=

value

] [, ... ] ) ] ALTER SUBSCRIPTION

name

REFRESH SEQUENCES ALTER SUBSCRIPTION

name

ENABLE ALTER SUBSCRIPTION

name

DISABLE ALTER SUBSCRIPTION

name

SET (

subscription_parameter

[=

value

] [, ... ] ) ALTER SUBSCRIPTION

name

SKIP (

skip_option

=

value

) ALTER SUBSCRIPTION

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER SUBSCRIPTION

name

RENAME TO

new_name

Description

ALTER SUBSCRIPTION can change most of the subscription properties that can be specified in ???.

You must own the subscription to use ALTER SUBSCRIPTION. To rename a subscription or alter the owner, you must have CREATE permission on the database. In addition, to alter the owner, you must be able to SET ROLE to the new owning role. If the subscription has password_required=false, only superusers can modify it.

When refreshing a publication we remove the relations that are no longer part of the publication and we also remove the table synchronization slots if there are any. It is necessary to remove these slots so that the resources allocated for the subscription on the remote host are released. If due to network breakdown or some other error, PostgreSQL is unable to remove the slots, an error will be reported. To proceed in this situation, the user either needs to retry the operation or disassociate the slot from the subscription and drop the subscription as explained in ???.

Commands ALTER SUBSCRIPTION ... REFRESH PUBLICATION, ALTER SUBSCRIPTION ... {SET|ADD|DROP} PUBLICATION ... with refresh option as true, ALTER SUBSCRIPTION ... SET (failover = true|false) and ALTER SUBSCRIPTION ... SET (two_phase = false) cannot be executed inside a transaction block.

Commands ALTER SUBSCRIPTION ... REFRESH PUBLICATION and ALTER SUBSCRIPTION ... {SET|ADD|DROP} PUBLICATION ... with refresh option as true also cannot be executed when the subscription has two_phase commit enabled, unless copy_data is false. See column subtwophasestate of pg_subscription to know the actual two-phase state.

Parameters

<name>
The name of a subscription whose properties are to be altered.

SERVER server_name
This clause replaces the foreign server or connection string originally set by ??? with the foreign server <server_name>.

CONNECTION 'conninfo'
This clause replaces the foreign server or connection string originally set by ??? with the connection string <conninfo>.

SET PUBLICATION publication_name; ADD PUBLICATION publication_name; DROP PUBLICATION publication_name
These forms change the list of subscribed publications. SET replaces the entire list of publications with a new list, ADD adds additional publications to the list of publications, and DROP removes the publications from the list of publications. We allow non-existent publications to be specified in ADD and SET variants so that users can add those later. See ??? for more information. By default, this command will also act like REFRESH PUBLICATION.

<publication_option> specifies additional options for this operation. The supported options are:

refresh (boolean)
When false, the command will not try to refresh table and sequence information. REFRESH PUBLICATION should then be executed separately. The default is true.

Additionally, the options described under REFRESH PUBLICATION may be specified, to control the implicit refresh operation.

REFRESH PUBLICATION
Fetch missing table and sequence information from the publisher. This will start replication of tables that were added to the subscribed-to publications since CREATE SUBSCRIPTION or the last invocation of REFRESH PUBLICATION.

The system catalog pg_subscription_rel is updated to record all tables and sequences known to the subscription, that are still part of the publication.

<refresh_option> specifies additional options for the refresh operation. The supported options are:

copy_data (boolean)
Specifies whether to copy pre-existing data for tables and synchronize sequences in the publications that are being subscribed to when the replication starts. The default is true.

Previously subscribed tables are not copied, even if a table's row filter WHERE clause has since been modified.

Previously subscribed sequences are not re-synchronized. To do that, use ALTER SUBSCRIPTION ... REFRESH SEQUENCES.

See ??? for recommendations on how to handle any warnings about sequence definition differences between the publisher and the subscriber, which might occur when copy_data = true.

See ??? for details of how copy_data = true can interact with the origin parameter.

See the binary parameter of CREATE SUBSCRIPTION for details about copying pre-existing data in binary format.

REFRESH SEQUENCES
Re-synchronize sequence data with the publisher. Unlike ALTER SUBSCRIPTION ... REFRESH PUBLICATION, which synchronizes the subscription's set of sequences with the publication (adding new and removing dropped sequences), REFRESH SEQUENCES will re-synchronize the sequence data for all currently subscribed sequences without changing which sequences are subscribed. Run REFRESH PUBLICATION first if the publication's set of sequences has changed.

[!NOTE] Sequence replication requires the publisher to be running PostgreSQL 19 or later.

See ??? for recommendations on how to handle any warnings about sequence definition differences between the publisher and the subscriber.

See ??? for recommendations on how to identify and handle out-of-sync sequences.

ENABLE
Enables a previously disabled subscription, starting the logical replication worker at the end of the transaction.

DISABLE
Disables a running subscription, stopping the logical replication worker at the end of the transaction.

SET ( subscription_parameter [= value] [, ... ] )
This clause alters parameters originally set by ???. See there for more information. The parameters that can be altered are slot_name, synchronous_commit, binary, streaming, disable_on_error, password_required, run_as_owner, origin, failover, two_phase, retain_dead_tuples, max_retention_duration, wal_receiver_timeout, and conflict_log_destination. Only a superuser can set password_required = false.

When altering the slot_name, the failover and two_phase property values of the named slot may differ from the counterpart failover and two_phase parameters specified in the subscription. When creating the slot, ensure the slot properties failover and two_phase match their counterpart parameters of the subscription. Otherwise, the slot on the publisher may behave differently from what these subscription options say: for example, the slot on the publisher could either be synced to the standbys even when the subscription's failover option is disabled or could be disabled for sync even when the subscription's failover option is enabled.

The failover, two_phase, and retain_dead_tuples parameters can only be altered when the subscription is disabled.

When altering two_phase from true to false, the backend process reports an error if any prepared transactions done by the logical replication worker (from when two_phase parameter was still true) are found. You can resolve prepared transactions on the publisher node, or manually roll them back on the subscriber, and then try again. The transactions prepared by logical replication worker corresponding to a particular subscription have the following pattern: “pg_gid_%u_%u” (parameters: subscription oid, remote transaction id xid). To resolve such transactions manually, you need to roll back all the prepared transactions with corresponding subscription IDs in their names. Applications can check pg_prepared_xacts to find the required prepared transactions. After the two_phase option is changed from true to false, the publisher will replicate the transactions again when they are committed.

If the retain_dead_tuples option is altered to false and no other subscription has this option enabled, the replication slot named “pg_conflict_detection”, created to retain dead tuples for conflict detection, will be dropped.

When the conflict_log_destination parameter is set to table or all, the system automatically creates the conflict log table. Conversely, if the destination is changed to log, the conflict log table is automatically dropped.

SKIP ( skip_option = value )
Skips applying all changes of the remote transaction. If incoming data violates any constraints, logical replication will stop until it is resolved. By using the ALTER SUBSCRIPTION ... SKIP command, the logical replication worker skips all data modification changes within the transaction. This option has no effect on the transactions that are already prepared by enabling two_phase on the subscriber. After the logical replication worker successfully skips the transaction or finishes a transaction, the LSN (stored in pg_subscription.subskiplsn) is cleared. See ??? for the details of logical replication conflicts.

<skip_option> specifies options for this operation. The supported option is:

lsn (pg_lsn)
Specifies the finish LSN of the remote transaction whose changes are to be skipped by the logical replication worker. The finish LSN is the LSN at which the transaction is either committed or prepared. Skipping individual subtransactions is not supported. Setting NONE resets the LSN.

<new_owner>
The user name of the new owner of the subscription.

<new_name>
The new name for the subscription.

When specifying a parameter of type boolean, the = <value> part can be omitted, which is equivalent to specifying TRUE.

Examples

Change the publication subscribed by a subscription to insert_only:

ALTER SUBSCRIPTION mysub SET PUBLICATION insert_only;

Disable (stop) the subscription:

ALTER SUBSCRIPTION mysub DISABLE;

Compatibility

ALTER SUBSCRIPTION is a PostgreSQL extension.

See Also

alter_system

ALTER SYSTEM

ALTER SYSTEM

7

SQL - Language Statements

ALTER SYSTEM

change a server configuration parameter

ALTER SYSTEM SET

configuration_parameter

{ TO | = } {

value

[, ...] | DEFAULT } ALTER SYSTEM RESET

configuration_parameter

ALTER SYSTEM RESET ALL

Description

ALTER SYSTEM is used for changing server configuration parameters across the entire database cluster. It can be more convenient than the traditional method of manually editing the postgresql.conf file. ALTER SYSTEM writes the given parameter setting to the postgresql.auto.conf file, which is read in addition to postgresql.conf. Setting a parameter to DEFAULT, or using the RESET variant, removes that configuration entry from the postgresql.auto.conf file. Use RESET ALL to remove all such configuration entries.

Values set with ALTER SYSTEM will be effective after the next server configuration reload, or after the next server restart in the case of parameters that can only be changed at server start. A server configuration reload can be commanded by calling the SQL function pg_reload_conf(), running pg_ctl reload, or sending a SIGHUP signal to the main server process.

Only superusers and users granted ALTER SYSTEM privilege on a parameter can change it using ALTER SYSTEM. Also, since this command acts directly on the file system and cannot be rolled back, it is not allowed inside a transaction block or function.

Parameters

<configuration_parameter>
Name of a settable configuration parameter. Available parameters are documented in ???.

<value>
New value of the parameter. Values can be specified as string constants, identifiers, numbers, or comma-separated lists of these, as appropriate for the particular parameter. Values that are neither numbers nor valid identifiers must be quoted. If the parameter accepts a list of values, NULL can be written to specify an empty list. DEFAULT can be written to specify removing the parameter and its value from postgresql.auto.conf.

For some list-accepting parameters, quoted values will produce double-quoted output to preserve whitespace and commas; for others, double-quotes must be used inside single-quoted strings to get this effect.

Notes

This command can't be used to set ???, ???, nor parameters that are not allowed in postgresql.conf (e.g., preset options).

See ??? for other ways to set the parameters.

ALTER SYSTEM can be disabled by setting ??? to off, but this is not a security mechanism (as explained in detail in the documentation for this parameter).

Examples

Set the wal_level:

ALTER SYSTEM SET wal_level = replica;

Undo that, restoring whatever setting was effective in postgresql.conf:

ALTER SYSTEM RESET wal_level;

Set the list of preloaded extension modules to be empty:

ALTER SYSTEM SET shared_preload_libraries TO NULL;

Compatibility

The ALTER SYSTEM statement is a PostgreSQL extension.

See Also

alter_table

ALTER TABLE

ALTER TABLE

7

SQL - Language Statements

ALTER TABLE

change the definition of a table

ALTER TABLE [ IF EXISTS ] [ ONLY ]

name

[ * ]

action

[, ... ] ALTER TABLE [ IF EXISTS ] [ ONLY ]

name

[ * ] RENAME [ COLUMN ]

column_name

TO

new_column_name

ALTER TABLE [ IF EXISTS ] [ ONLY ]

name

[ * ] RENAME CONSTRAINT

constraint_name

TO

new_constraint_name

ALTER TABLE [ IF EXISTS ]

name

RENAME TO

new_name

ALTER TABLE [ IF EXISTS ]

name

SET SCHEMA

new_schema

ALTER TABLE ALL IN TABLESPACE

name

[ OWNED BY

role_name

[, ... ] ] SET TABLESPACE

new_tablespace

[ NOWAIT ] ALTER TABLE [ IF EXISTS ]

name

ATTACH PARTITION

partition_name

{ FOR VALUES

partition_bound_spec

| DEFAULT } ALTER TABLE [ IF EXISTS ]

name

DETACH PARTITION

partition_name

[ CONCURRENTLY | FINALIZE ]

where

action

is one of:

ADD [ COLUMN ] [ IF NOT EXISTS ]

column_name

data_type

[ COLLATE

collation

] [

column_constraint

[ ... ] ] DROP [ COLUMN ] [ IF EXISTS ]

column_name

[ RESTRICT | CASCADE ] ALTER [ COLUMN ]

column_name

[ SET DATA ] TYPE

data_type

[ COLLATE

collation

] [ USING

expression

] ALTER [ COLUMN ]

column_name

SET DEFAULT

expression

ALTER [ COLUMN ]

column_name

DROP DEFAULT ALTER [ COLUMN ]

column_name

{ SET | DROP } NOT NULL ALTER [ COLUMN ]

column_name

SET EXPRESSION AS (

expression

) ALTER [ COLUMN ]

column_name

DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ]

column_name

ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ (

sequence_options

) ] ALTER [ COLUMN ]

column_name

{ SET GENERATED { ALWAYS | BY DEFAULT } | SET

sequence_option

| RESTART [ [ WITH ]

restart

] } [...] ALTER [ COLUMN ]

column_name

DROP IDENTITY [ IF EXISTS ] ALTER [ COLUMN ]

column_name

SET STATISTICS {

integer

| DEFAULT } ALTER [ COLUMN ]

column_name

SET (

attribute_option

=

value

[, ... ] ) ALTER [ COLUMN ]

column_name

RESET (

attribute_option

[, ... ] ) ALTER [ COLUMN ]

column_name

SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN | DEFAULT } ALTER [ COLUMN ]

column_name

SET COMPRESSION

compression_method

ADD

table_constraint

[ NOT VALID ] ADD

table_constraint_using_index

ALTER CONSTRAINT

constraint_name

[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ] [ ENFORCED | NOT ENFORCED ] ALTER CONSTRAINT

constraint_name

[ INHERIT | NO INHERIT ] VALIDATE CONSTRAINT

constraint_name

DROP CONSTRAINT [ IF EXISTS ]

constraint_name

[ RESTRICT | CASCADE ] DISABLE TRIGGER [

trigger_name

| ALL | USER ] ENABLE TRIGGER [

trigger_name

| ALL | USER ] ENABLE REPLICA TRIGGER

trigger_name

ENABLE ALWAYS TRIGGER

trigger_name

DISABLE RULE

rewrite_rule_name

ENABLE RULE

rewrite_rule_name

ENABLE REPLICA RULE

rewrite_rule_name

ENABLE ALWAYS RULE

rewrite_rule_name

DISABLE ROW LEVEL SECURITY ENABLE ROW LEVEL SECURITY FORCE ROW LEVEL SECURITY NO FORCE ROW LEVEL SECURITY CLUSTER ON

index_name

SET WITHOUT CLUSTER SET WITHOUT OIDS SET ACCESS METHOD {

new_access_method

| DEFAULT } SET TABLESPACE

new_tablespace

SET { LOGGED | UNLOGGED } SET (

storage_parameter

[=

value

] [, ... ] ) RESET (

storage_parameter

[, ... ] ) INHERIT

parent_table

NO INHERIT

parent_table

OF

type_name

NOT OF OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } REPLICA IDENTITY { DEFAULT | USING INDEX

index_name

| FULL | NOTHING }

and

partition_bound_spec

is:

IN (

partition_bound_expr

[, ...] ) | FROM ( {

partition_bound_expr

| MINVALUE | MAXVALUE } [, ...] ) TO ( {

partition_bound_expr

| MINVALUE | MAXVALUE } [, ...] ) | WITH ( MODULUS

numeric_literal

, REMAINDER

numeric_literal

)

and

column_constraint

is:

[ CONSTRAINT

constraint_name

] { NOT NULL [ NO INHERIT ] | NULL | CHECK (

expression

) [ NO INHERIT ] | DEFAULT

default_expr

| GENERATED ALWAYS AS (

generation_expr

) [ STORED | VIRTUAL ] | GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ (

sequence_options

) ] | UNIQUE [ NULLS [ NOT ] DISTINCT ]

index_parameters

| PRIMARY KEY

index_parameters

| REFERENCES

reftable

[ (

refcolumn

) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ] [ ON DELETE

referential_action

] [ ON UPDATE

referential_action

] } [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ] [ ENFORCED | NOT ENFORCED ]

and

table_constraint

is:

[ CONSTRAINT

constraint_name

] { CHECK (

expression

) [ NO INHERIT ] | NOT NULL

column_name

[ NO INHERIT ] | UNIQUE [ NULLS [ NOT ] DISTINCT ] (

column_name

[, ... ] [,

column_name

WITHOUT OVERLAPS ] )

index_parameters

| PRIMARY KEY (

column_name

[, ... ] [,

column_name

WITHOUT OVERLAPS ] )

index_parameters

| EXCLUDE [ USING

index_method

] (

exclude_element

WITH

operator

[, ... ] )

index_parameters

[ WHERE (

predicate

) ] | FOREIGN KEY (

column_name

[, ... ] [, PERIOD

column_name

] ) REFERENCES

reftable

[ (

refcolumn

[, ... ] [, PERIOD

refcolumn

] ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ] [ ON DELETE

referential_action

] [ ON UPDATE

referential_action

] } [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ] [ ENFORCED | NOT ENFORCED ]

and

table_constraint_using_index

is:

[ CONSTRAINT

constraint_name

] { UNIQUE | PRIMARY KEY } USING INDEX

index_name

[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]

index_parameters

in

UNIQUE

,

PRIMARY KEY

, and

EXCLUDE

constraints are:

[ INCLUDE (

column_name

[, ... ] ) ] [ WITH (

storage_parameter

[=

value

] [, ... ] ) ] [ USING INDEX TABLESPACE

tablespace_name

]

exclude_element

in an

EXCLUDE

constraint is:

{

column_name

| (

expression

) } [ COLLATE

collation

] [

opclass

[ (

opclass_parameter

=

value

[, ... ] ) ] ] [ ASC | DESC ] [ NULLS { FIRST | LAST } ]

referential_action

in a

FOREIGN KEY

/

REFERENCES

constraint is:

{ NO ACTION | RESTRICT | CASCADE | SET NULL [ (

column_name

[, ... ] ) ] | SET DEFAULT [ (

column_name

[, ... ] ) ] }

Description

ALTER TABLE changes the definition of an existing table. There are several subforms described below. Note that the lock level required may differ for each subform. An ACCESS EXCLUSIVE lock is acquired unless explicitly noted. When multiple subcommands are given, the lock acquired will be the strictest one required by any subcommand.

ADD [ COLUMN ] [ IF NOT EXISTS ]
This form adds a new column to the table, using the same syntax as CREATE TABLE. If IF NOT EXISTS is specified and a column already exists with this name, no error is thrown.

DROP [ COLUMN ] [ IF EXISTS ]
This form drops a column from a table. Indexes and table constraints involving the column will be automatically dropped as well. Multivariate statistics referencing the dropped column will also be removed if the removal of the column would cause the statistics to contain data for only a single column. You will need to say CASCADE if anything outside the table depends on the column, for example, foreign key references or views. If IF EXISTS is specified and the column does not exist, no error is thrown. In this case a notice is issued instead.

SET DATA TYPE
This form changes the type of a column of a table. Indexes and simple table constraints involving the column will be automatically converted to use the new column type by reparsing the originally supplied expression. The optional COLLATE clause specifies a collation for the new column; if omitted, the collation is the default for the new column type. The optional USING clause specifies how to compute the new column value from the old; if omitted, the default conversion is the same as an assignment cast from old data type to new. A USING clause must be provided if there is no implicit or assignment cast from old to new type.

When this form is used, the column's statistics are removed, so running ANALYZE on the table afterwards is recommended. For a virtual generated column, ANALYZE is not necessary unless extended statistics are defined on it, since such columns do not have statistics by default.

SET/DROP DEFAULT
These forms set or remove the default value for a column (where removal is equivalent to setting the default value to NULL). The new default value will only apply in subsequent INSERT or UPDATE commands; it does not cause rows already in the table to change.

SET/DROP NOT NULL
These forms change whether a column is marked to allow null values or to reject null values.

SET NOT NULL may only be applied to a column provided none of the records in the table contain a NULL value for the column. Ordinarily this is checked during the ALTER TABLE by scanning the entire table, unless NOT VALID is specified; however, if a valid CHECK constraint exists (and is not dropped in the same command) which proves no NULL can exist, then the table scan is skipped. If a column has an invalid not-null constraint, SET NOT NULL validates it.

If this table is a partition, one cannot perform DROP NOT NULL on a column if it is marked NOT NULL in the parent table. To drop the NOT NULL constraint from all the partitions, perform DROP NOT NULL on the parent table. Even if there is no NOT NULL constraint on the parent, such a constraint can still be added to individual partitions, if desired; that is, the children can disallow nulls even if the parent allows them, but not the other way around. It is also possible to drop the NOT NULL constraint from ONLY the parent table, which does not remove it from the children.

SET EXPRESSION AS
This form replaces the expression of a generated column. Existing data in a stored generated column is rewritten and all the future changes will apply the new generation expression. Replacing the expression of a virtual generated column does not require a table rewrite, but if the column is used in a constraint, the table will be scanned to check that existing rows meet the constraint.

When this form is used on a stored generated column, its statistics are removed, so running ANALYZE on the table afterwards is recommended. For a virtual generated column, ANALYZE is not necessary unless extended statistics are defined on it, since such columns do not have statistics by default.

DROP EXPRESSION [ IF EXISTS ]
This form turns a stored generated column into a normal base column. Existing data in the columns is retained, but future changes will no longer apply the generation expression.

This form is currently only supported for stored generated columns (not virtual ones).

If DROP EXPRESSION IF EXISTS is specified and the column is not a generated column, no error is thrown. In this case a notice is issued instead.

ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY; SET GENERATED { ALWAYS | BY DEFAULT }; DROP IDENTITY [ IF EXISTS ]
These forms change whether a column is an identity column or change the generation attribute of an existing identity column. See CREATE TABLE for details. Like SET DEFAULT, these forms only affect the behavior of subsequent INSERT and UPDATE commands; they do not cause rows already in the table to change.

If DROP IDENTITY IF EXISTS is specified and the column is not an identity column, no error is thrown. In this case a notice is issued instead.

SET sequence_option; RESTART
These forms alter the sequence that underlies an existing identity column. <sequence_option> is an option supported by ALTER SEQUENCE such as INCREMENT BY.

SET STATISTICS
This form sets the per-column statistics-gathering target for subsequent ANALYZE operations. The target can be set in the range 0 to 10000. Set it to DEFAULT to revert to using the system default statistics target (???). (Setting to a value of -1 is an obsolete way spelling to get the same outcome.) For more information on the use of statistics by the PostgreSQL query planner, refer to ???.

This form is not supported on virtual generated columns, since such columns do not have statistics by default. If extended statistics are defined on such columns, the statistics-gathering target may be set on the extended statistics object using ???.

SET STATISTICS acquires a SHARE UPDATE EXCLUSIVE lock.

SET ( attribute_option = value [, ... ] ); RESET ( attribute_option [, ... ] )
This form sets or resets per-attribute options. Currently, the only defined per-attribute options are n_distinct and n_distinct_inherited, which override the number-of-distinct-values estimates made by subsequent ANALYZE operations. n_distinct affects the statistics for the table itself, while n_distinct_inherited affects the statistics gathered for the table plus its inheritance children, and for the statistics gathered for partitioned tables. When the value specified is a positive value, the query planner will assume that the column contains exactly the specified number of distinct nonnull values. Fractional values may also be specified by using values below 0 and above or equal to -1. This instructs the query planner to estimate the number of distinct values by multiplying the absolute value of the specified number by the estimated number of rows in the table. For example, a value of -1 implies that all values in the column are distinct, while a value of -0.5 implies that each value appears twice on average. This can be useful when the size of the table changes over time. For more information on the use of statistics by the PostgreSQL query planner, refer to ???.

Changing per-attribute options acquires a SHARE UPDATE EXCLUSIVE lock.

SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN | DEFAULT }
This form sets the storage mode for a column. This controls whether this column is held inline or in a secondary TOAST table, and whether the data should be compressed or not. PLAIN must be used for fixed-length values such as integer and is inline, uncompressed. MAIN is for inline, compressible data. EXTERNAL is for external, uncompressed data, and EXTENDED is for external, compressed data. Writing DEFAULT sets the storage mode to the default mode for the column's data type. EXTENDED is the default for most data types that support non-PLAIN storage. Use of EXTERNAL will make substring operations on very large text and bytea values run faster, at the penalty of increased storage space. Note that ALTER TABLE ... SET STORAGE doesn't itself change anything in the table; it just sets the strategy to be pursued during future table updates. See ??? for more information.

SET COMPRESSION compression_method
This form sets the compression method for a column, determining how values inserted in future will be compressed (if the storage mode permits compression at all). This does not cause the table to be rewritten, so existing data may still be compressed with other compression methods. If the table is restored with pg_restore, then all values are rewritten with the configured compression method. However, when data is inserted from another relation (for example, by INSERT ... SELECT), values from the source table are not necessarily detoasted, so any previously compressed data may retain its existing compression method, rather than being recompressed with the compression method of the target column. The supported compression methods are pglz and lz4. (lz4 is available only if --with-lz4 was used when building PostgreSQL.) In addition, <compression_method> can be default, which selects the default behavior of consulting the ??? setting at the time of data insertion to determine the method to use.

ADD table_constraint [ NOT VALID ]
This form adds a new constraint to a table using the same constraint syntax as CREATE TABLE, plus the option NOT VALID, which is currently only allowed for foreign-key, CHECK, and not-null constraints.

Normally, this form will cause a scan of the table to verify that all existing rows in the table satisfy the new constraint. But if the NOT VALID option is used, this potentially-lengthy scan is skipped. The constraint will still be applied against subsequent inserts or updates (that is, they'll fail unless there is a matching row in the referenced table, in the case of foreign keys, or they'll fail unless the new row matches the specified check condition). But the database will not assume that the constraint holds for all rows in the table, until it is validated by using the VALIDATE CONSTRAINT option. See Notes below for more information about using the NOT VALID option.

Although most forms of ADD table_constraint require an ACCESS EXCLUSIVE lock, ADD FOREIGN KEY requires only a SHARE ROW EXCLUSIVE lock. Note that ADD FOREIGN KEY also acquires a SHARE ROW EXCLUSIVE lock on the referenced table, in addition to the lock on the table on which the constraint is declared.

Additional restrictions apply when unique or primary key constraints are added to partitioned tables; see CREATE TABLE.

ADD table_constraint_using_index
This form adds a new PRIMARY KEY or UNIQUE constraint to a table based on an existing unique index. All the columns of the index will be included in the constraint.

The index cannot have expression columns nor be a partial index. Also, it must be a b-tree index with default sort ordering. These restrictions ensure that the index is equivalent to one that would be built by a regular ADD PRIMARY KEY or ADD UNIQUE command.

If PRIMARY KEY is specified, and the index's columns are not already marked NOT NULL, then this command will attempt to do ALTER COLUMN SET NOT NULL against each such column. That requires a full table scan to verify the column(s) contain no nulls. In all other cases, this is a fast operation.

If a constraint name is provided then the index will be renamed to match the constraint name. Otherwise the constraint will be named the same as the index.

After this command is executed, the index is “owned” by the constraint, in the same way as if the index had been built by a regular ADD PRIMARY KEY or ADD UNIQUE command. In particular, dropping the constraint will make the index disappear too.

This form is not currently supported on partitioned tables.

[!NOTE] Adding a constraint using an existing index can be helpful in situations where a new constraint needs to be added without blocking table updates for a long time. To do that, create the index using CREATE UNIQUE INDEX CONCURRENTLY, and then convert it to a constraint using this syntax. See the example below.

ALTER CONSTRAINT
This form alters the attributes of a constraint that was previously created. Currently, the deferrability attributes can be altered only for FOREIGN KEY constraints. The enforceability attribute can be altered for FOREIGN KEY and CHECK constraints.

Setting a constraint to NOT ENFORCED causes the database system to stop checking it for new or updated rows. Setting a constraint to ENFORCED causes the database system to verify that existing rows satisfy the constraint and to check it for new or updated rows.

ALTER CONSTRAINT ... INHERIT; ALTER CONSTRAINT ... NO INHERIT
These forms modify a inheritable constraint so that it becomes not inheritable, or vice-versa. Only not-null constraints may be altered in this fashion at present. In addition to changing the inheritability status of the constraint, in the case where a non-inheritable constraint is being marked inheritable, if the table has children, an equivalent constraint will be added to them. If marking an inheritable constraint as non-inheritable on a table with children, then the corresponding constraint on children will be marked as no longer inherited, but not removed.

VALIDATE CONSTRAINT
This form validates a foreign key, check, or not-null constraint that was previously created as NOT VALID, by scanning the table to ensure there are no rows for which the constraint is not satisfied. If the constraint was set to NOT ENFORCED, an error is thrown. Nothing happens if the constraint is already marked valid. (See Notes below for an explanation of the usefulness of this command.)

This command acquires a SHARE UPDATE EXCLUSIVE lock.

DROP CONSTRAINT [ IF EXISTS ]
This form drops the specified constraint on a table, along with any index underlying the constraint. If IF EXISTS is specified and the constraint does not exist, no error is thrown. In this case a notice is issued instead.

DISABLE/ENABLE [ REPLICA | ALWAYS ] TRIGGER
These forms configure the firing of trigger(s) belonging to the table. A disabled trigger is still known to the system, but is not executed when its triggering event occurs. (For a deferred trigger, the enable status is checked when the event occurs, not when the trigger function is actually executed.) One can disable or enable a single trigger specified by name, or all triggers on the table, or only user triggers (this option excludes internally generated constraint triggers, such as those that are used to implement foreign key constraints or deferrable uniqueness and exclusion constraints). Disabling or enabling internally generated constraint triggers requires superuser privileges; it should be done with caution since of course the integrity of the constraint cannot be guaranteed if the triggers are not executed.

The trigger firing mechanism is also affected by the configuration variable ???. Simply enabled triggers (the default) will fire when the replication role is “origin” (the default) or “local”. Triggers configured as ENABLE REPLICA will only fire if the session is in “replica” mode, and triggers configured as ENABLE ALWAYS will fire regardless of the current replication role.

The effect of this mechanism is that in the default configuration, triggers do not fire on replicas. This is useful because if a trigger is used on the origin to propagate data between tables, then the replication system will also replicate the propagated data; so the trigger should not fire a second time on the replica, because that would lead to duplication. However, if a trigger is used for another purpose such as creating external alerts, then it might be appropriate to set it to ENABLE ALWAYS so that it is also fired on replicas.

When this command is applied to a partitioned table, the states of corresponding clone triggers in the partitions are updated too, unless ONLY is specified.

This command acquires a SHARE ROW EXCLUSIVE lock.

DISABLE/ENABLE [ REPLICA | ALWAYS ] RULE
These forms configure the firing of rewrite rules belonging to the table. A disabled rule is still known to the system, but is not applied during query rewriting. The semantics are as for disabled/enabled triggers. This configuration is ignored for ON SELECT rules, which are always applied in order to keep views working even if the current session is in a non-default replication role.

The rule firing mechanism is also affected by the configuration variable ???, analogous to triggers as described above.

DISABLE/ENABLE ROW LEVEL SECURITY
These forms control the application of row security policies belonging to the table. If enabled and no policies exist for the table, then a default-deny policy is applied. Note that policies can exist for a table even if row-level security is disabled. In this case, the policies will not be applied and the policies will be ignored. See also CREATE POLICY.

NO FORCE/FORCE ROW LEVEL SECURITY
These forms control the application of row security policies belonging to the table when the user is the table owner. If enabled, row-level security policies will be applied when the user is the table owner. If disabled (the default) then row-level security will not be applied when the user is the table owner. See also CREATE POLICY.

CLUSTER ON
This form selects the default index for future CLUSTER operations. It does not actually re-cluster the table.

Changing cluster options acquires a SHARE UPDATE EXCLUSIVE lock.

SET WITHOUT CLUSTER
This form removes the most recently used CLUSTER index specification from the table. This affects future cluster operations that don't specify an index.

Changing cluster options acquires a SHARE UPDATE EXCLUSIVE lock.

SET WITHOUT OIDS
Backward-compatible syntax for removing the oid system column. As oid system columns cannot be added anymore, this never has an effect.

SET ACCESS METHOD
This form changes the access method of the table by rewriting it using the indicated access method; specifying DEFAULT selects the access method set as the ??? configuration parameter. See ??? for more information.

When applied to a partitioned table, there is no data to rewrite, but partitions created afterwards will default to the given access method unless overridden by a USING clause. Specifying DEFAULT removes a previous value, causing future partitions to default to default_table_access_method.

SET TABLESPACE
This form changes the table's tablespace to the specified tablespace and moves the data file(s) associated with the table to the new tablespace. Indexes on the table, if any, are not moved; but they can be moved separately with additional SET TABLESPACE commands. When applied to a partitioned table, nothing is moved, but any partitions created afterwards with CREATE TABLE PARTITION OF will use that tablespace, unless overridden by a TABLESPACE clause.

All tables in the current database in a tablespace can be moved by using the ALL IN TABLESPACE form, which will lock all tables to be moved first and then move each one. This form also supports OWNED BY, which will only move tables owned by the roles specified. If the NOWAIT option is specified then the command will fail if it is unable to acquire all of the locks required immediately. Note that system catalogs are not moved by this command; use ALTER DATABASE or explicit ALTER TABLE invocations instead if desired. The information_schema relations are not considered part of the system catalogs and will be moved. See also CREATE TABLESPACE.

SET { LOGGED | UNLOGGED }
This form changes the table from unlogged to logged or vice-versa (see ???). It cannot be applied to a temporary table.

This also changes the persistence of any sequences linked to the table (for identity or serial columns). However, it is also possible to change the persistence of such sequences separately.

This form is not supported for partitioned tables.

SET ( storage_parameter [= value] [, ... ] )
This form changes one or more storage parameters for the table. See ??? in the CREATE TABLE documentation for details on the available parameters. Note that the table contents will not be modified immediately by this command; depending on the parameter you might need to rewrite the table to get the desired effects. That can be done with VACUUM FULL, CLUSTER or one of the forms of ALTER TABLE that forces a table rewrite. For planner related parameters, changes will take effect from the next time the table is locked so currently executing queries will not be affected.

SHARE UPDATE EXCLUSIVE lock will be taken for fillfactor, TOAST and autovacuum storage parameters, as well as the planner parameter parallel_workers.

RESET ( storage_parameter [, ... ] )
This form resets one or more storage parameters to their defaults. As with SET, a table rewrite might be needed to update the table entirely.

INHERIT parent_table
This form adds the target table as a new child of the specified parent table. Subsequently, queries against the parent will include records of the target table. To be added as a child, the target table must already contain all the same columns as the parent (it could have additional columns, too). The columns must have matching data types.

In addition, all CHECK and NOT NULL constraints on the parent must also exist on the child, except those marked non-inheritable (that is, created with ALTER TABLE ... ADD CONSTRAINT ... NO INHERIT), which are ignored. All child-table constraints matched must not be marked non-inheritable. Currently UNIQUE, PRIMARY KEY, and FOREIGN KEY constraints are not considered, but this might change in the future.

NO INHERIT parent_table
This form removes the target table from the list of children of the specified parent table. Queries against the parent table will no longer include records drawn from the target table.

OF type_name
This form links the table to a composite type as though CREATE TABLE OF had formed it. The table's list of column names and types must precisely match that of the composite type. The table must not inherit from any other table. These restrictions ensure that CREATE TABLE OF would permit an equivalent table definition.

NOT OF
This form dissociates a typed table from its type.

OWNER TO
This form changes the owner of the table, sequence, view, materialized view, or foreign table to the specified user.

REPLICA IDENTITY
This form changes the information which is written to the write-ahead log to identify rows which are updated or deleted. In most cases, the old value of each column is only logged if it differs from the new value; however, if the old value is stored externally, it is always logged regardless of whether it changed. This option has no effect except when logical replication is in use.

DEFAULT
Records the old values of the columns of the primary key. This is the default for non-system tables. When there is no primary key, the behavior is the same as NOTHING.

USING INDEX index_name
Records the old values of the columns covered by the named index, that must be unique, not partial, not deferrable, and include only columns marked NOT NULL. If this index is dropped, the behavior is the same as NOTHING.

FULL
Records the old values of all columns in the row.

NOTHING
Records no information about the old row. This is the default for system tables.

RENAME
The RENAME forms change the name of a table (or an index, sequence, view, materialized view, or foreign table), the name of an individual column in a table, or the name of a constraint of the table. When renaming a constraint that has an underlying index, the index is renamed as well. There is no effect on the stored data.

SET SCHEMA
This form moves the table into another schema. Associated indexes, constraints, and sequences owned by table columns are moved as well.

ATTACH PARTITION partition_name { FOR VALUES partition_bound_spec | DEFAULT }
This form attaches an existing table (which might itself be partitioned) as a partition of the target table. The table can be attached as a partition for specific values using FOR VALUES or as a default partition by using DEFAULT. For each index in the target table, if a valid equivalent index already exists in the partition, it will be attached to the target table's index, as if ALTER INDEX ATTACH PARTITION had been executed; otherwise, a new corresponding index will be created. Invalid indexes on the partition are skipped. Note that if the existing table is a foreign table, it is currently not allowed to attach the table as a partition of the target table if there are UNIQUE indexes on the target table. (See also ???.) For each user-defined row-level trigger that exists in the target table, a corresponding one is created in the attached table.

A partition using FOR VALUES uses same syntax for <partition_bound_spec> as CREATE TABLE. The partition bound specification must correspond to the partitioning strategy and partition key of the target table. The table to be attached must have all the same columns as the target table and no more; moreover, the column types must also match. Also, it must have all the NOT NULL and CHECK constraints of the target table, not marked NO INHERIT. Currently FOREIGN KEY constraints are not considered. UNIQUE and PRIMARY KEY constraints from the parent table will be created in the partition, if they don't already exist.

If the new partition is a regular table, a full table scan is performed to check that existing rows in the table do not violate the partition constraint. It is possible to avoid this scan by adding a valid CHECK constraint to the table that allows only rows satisfying the desired partition constraint before running this command. The CHECK constraint will be used to determine that the table need not be scanned to validate the partition constraint. This does not work, however, if any of the partition keys is an expression and the partition does not accept NULL values. If attaching a list partition that will not accept NULL values, also add a NOT NULL constraint to the partition key column, unless it's an expression.

If the new partition is a foreign table, nothing is done to verify that all the rows in the foreign table obey the partition constraint. (See the discussion in ??? about constraints on the foreign table.)

When a table has a default partition, defining a new partition changes the partition constraint for the default partition. The default partition can't contain any rows that would need to be moved to the new partition, and will be scanned to verify that none are present. This scan, like the scan of the new partition, can be avoided if an appropriate CHECK constraint is present. Also like the scan of the new partition, it is always skipped when the default partition is a foreign table.

Attaching a partition acquires a SHARE UPDATE EXCLUSIVE lock on the parent table, in addition to the ACCESS EXCLUSIVE locks on the table being attached and on the default partition (if any).

Further locks must also be held on all sub-partitions if the table being attached is itself a partitioned table. Likewise if the default partition is itself a partitioned table. The locking of the sub-partitions can be avoided by adding a CHECK constraint as described in ???.

DETACH PARTITION partition_name [ CONCURRENTLY | FINALIZE ]
This form detaches the specified partition of the target table. The detached partition continues to exist as a standalone table, but no longer has any ties to the table from which it was detached. Any indexes that were attached to the target table's indexes are detached. Any triggers that were created as clones of those in the target table are removed. SHARE lock is obtained on any tables that reference this partitioned table in foreign key constraints.

If CONCURRENTLY is specified, it runs using a reduced lock level to avoid blocking other sessions that might be accessing the partitioned table. In this mode, two transactions are used internally. During the first transaction, a SHARE UPDATE EXCLUSIVE lock is taken on both parent table and partition, and the partition is marked as undergoing detach; at that point, the transaction is committed and all other transactions using the partitioned table are waited for. Once all those transactions have completed, the second transaction acquires SHARE UPDATE EXCLUSIVE on the partitioned table and ACCESS EXCLUSIVE on the partition, and the detach process completes. A CHECK constraint that duplicates the partition constraint is added to the partition. CONCURRENTLY cannot be run in a transaction block and is not allowed if the partitioned table contains a default partition.

If FINALIZE is specified, a previous DETACH CONCURRENTLY invocation that was canceled or interrupted is completed. At most one partition in a partitioned table can be pending detach at a time.

All the forms of ALTER TABLE that act on a single table, except RENAME, SET SCHEMA, ATTACH PARTITION, and DETACH PARTITION can be combined into a list of multiple alterations to be applied together. For example, it is possible to add several columns and/or alter the type of several columns in a single command. This is particularly useful with large tables, since only one pass over the table need be made.

You must own the table to use ALTER TABLE. To change the schema or tablespace of a table, you must also have CREATE privilege on the new schema or tablespace. To add the table as a new child of a parent table, you must own the parent table as well. Also, to attach a table as a new partition of the table, you must own the table being attached. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the table's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the table. However, a superuser can alter ownership of any table anyway.) To add a column or alter a column type or use the OF clause, you must also have USAGE privilege on the data type.

Parameters

IF EXISTS
Do not throw an error if the table does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of an existing table to alter. If ONLY is specified before the table name, only that table is altered. If ONLY is not specified, the table and all its descendant tables (if any) are altered. Optionally, * can be specified after the table name to explicitly indicate that descendant tables are included.

<column_name>
Name of a new or existing column.

<new_column_name>
New name for an existing column.

<new_name>
New name for the table.

<data_type>
Data type of the new column, or new data type for an existing column.

<table_constraint>
New table constraint for the table.

<constraint_name>
Name of a new or existing constraint.

CASCADE
Automatically drop objects that depend on the dropped column or constraint (for example, views referencing the column), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the column or constraint if there are any dependent objects. This is the default behavior.

<trigger_name>
Name of a single trigger to disable or enable.

ALL
Disable or enable all triggers belonging to the table. (This requires superuser privilege if any of the triggers are internally generated constraint triggers, such as those that are used to implement foreign key constraints or deferrable uniqueness and exclusion constraints.)

USER
Disable or enable all triggers belonging to the table except for internally generated constraint triggers, such as those that are used to implement foreign key constraints or deferrable uniqueness and exclusion constraints.

<index_name>
The name of an existing index.

<storage_parameter>
The name of a table storage parameter.

<value>
The new value for a table storage parameter. This might be a number or a word depending on the parameter.

<parent_table>
A parent table to associate or de-associate with this table.

<new_owner>
The user name of the new owner of the table.

<new_access_method>
The name of the access method to which the table will be converted.

<new_tablespace>
The name of the tablespace to which the table will be moved.

<new_schema>
The name of the schema to which the table will be moved.

<partition_name>
The name of the table to attach as a new partition or to detach from this table.

<partition_bound_spec>
The partition bound specification for a new partition. Refer to ??? for more details on the syntax of the same.

Notes

The key word COLUMN is noise and can be omitted.

When a column is added with ADD COLUMN and a non-volatile DEFAULT is specified, the default value is evaluated at the time of the statement and the result stored in the table's metadata, where it will be returned when any existing rows are accessed. The value will be only applied when the table is rewritten, making the ALTER TABLE very fast even on large tables. If no column constraints are specified, NULL is used as the DEFAULT. In neither case is a rewrite of the table required.

Adding a column with a volatile DEFAULT (e.g., clock_timestamp()), a stored generated column, an identity column, or a column with a domain data type that has constraints will cause the entire table and its indexes to be rewritten. Adding a virtual generated column never requires a rewrite.

Changing the type of an existing column will normally cause the entire table and its indexes to be rewritten. As an exception, when changing the type of an existing column, if the USING clause does not change the column contents and the old type is either binary coercible to the new type or an unconstrained domain over the new type, a table rewrite is not needed. However, indexes will still be rebuilt unless the system can verify that the new index would be logically equivalent to the existing one. For example, if the collation for a column has been changed, an index rebuild is required because the new sort order might be different. However, in the absence of a collation change, a column can be changed from text to varchar (or vice versa) without rebuilding the indexes because these data types sort identically.

Table and/or index rebuilds may take a significant amount of time for a large table, and will temporarily require as much as double the disk space.

Adding a CHECK or NOT NULL constraint requires scanning the table to verify that existing rows meet the constraint, but does not require a table rewrite. If a CHECK constraint is added as NOT ENFORCED, no verification will be performed.

Similarly, when attaching a new partition it may be scanned to verify that existing rows meet the partition constraint.

The main reason for providing the option to specify multiple changes in a single ALTER TABLE is that multiple table scans or rewrites can thereby be combined into a single pass over the table.

Scanning a large table to verify new foreign-key, check, or not-null constraints can take a long time, and other updates to the table are locked out until the ALTER TABLE ADD CONSTRAINT command is committed. The main purpose of the NOT VALID constraint option is to reduce the impact of adding a constraint on concurrent updates. With NOT VALID, the ADD CONSTRAINT command does not scan the table and can be committed immediately. After that, a VALIDATE CONSTRAINT command can be issued to verify that existing rows satisfy the constraint. The validation step does not need to lock out concurrent updates, since it knows that other transactions will be enforcing the constraint for rows that they insert or update; only pre-existing rows need to be checked. Hence, validation acquires only a SHARE UPDATE EXCLUSIVE lock on the table being altered. (If the constraint is a foreign key then a ROW SHARE lock is also required on the table referenced by the constraint.) In addition to improving concurrency, it can be useful to use NOT VALID and VALIDATE CONSTRAINT in cases where the table is known to contain pre-existing violations. Once the constraint is in place, no new violations can be inserted, and the existing problems can be corrected at leisure until VALIDATE CONSTRAINT finally succeeds.

The DROP COLUMN form does not physically remove the column, but simply makes it invisible to SQL operations. Subsequent insert and update operations in the table will store a null value for the column. Thus, dropping a column is quick but it will not immediately reduce the on-disk size of your table, as the space occupied by the dropped column is not reclaimed. The space will be reclaimed over time as existing rows are updated.

To force immediate reclamation of space occupied by a dropped column, you can execute one of the forms of ALTER TABLE that performs a rewrite of the whole table. This results in reconstructing each row with the dropped column replaced by a null value.

The rewriting forms of ALTER TABLE are not MVCC-safe. After a table rewrite, the table will appear empty to concurrent transactions, if they are using a snapshot taken before the rewrite occurred. See ??? for more details.

The USING option of SET DATA TYPE can actually specify any expression involving the old values of the row; that is, it can refer to other columns as well as the one being converted. This allows very general conversions to be done with the SET DATA TYPE syntax. Because of this flexibility, the USING expression is not applied to the column's default value (if any); the result might not be a constant expression as required for a default. This means that when there is no implicit or assignment cast from old to new type, SET DATA TYPE might fail to convert the default even though a USING clause is supplied. In such cases, drop the default with DROP DEFAULT, perform the ALTER TYPE, and then use SET DEFAULT to add a suitable new default. Similar considerations apply to indexes and constraints involving the column.

If a table has any descendant tables, it is not permitted to add, rename, or change the type of a column in the parent table without doing the same to the descendants. This ensures that the descendants always have columns matching the parent. Similarly, a CHECK constraint cannot be renamed in the parent without also renaming it in all descendants, so that CHECK constraints also match between the parent and its descendants. (That restriction does not apply to index-based constraints, however.) Also, because selecting from the parent also selects from its descendants, a constraint on the parent cannot be marked valid unless it is also marked valid for those descendants. In all of these cases, ALTER TABLE ONLY will be rejected.

A recursive DROP COLUMN operation will remove a descendant table's column only if the descendant does not inherit that column from any other parents and never had an independent definition of the column. A nonrecursive DROP COLUMN (i.e., ALTER TABLE ONLY ... DROP COLUMN) never removes any descendant columns, but instead marks them as independently defined rather than inherited. A nonrecursive DROP COLUMN command will fail for a partitioned table, because all partitions of a table must have the same columns as the partitioning root.

The actions for identity columns (ADD GENERATED, SET etc., DROP IDENTITY), as well as the actions CLUSTER, OWNER, and TABLESPACE never recurse to descendant tables; that is, they always act as though ONLY were specified. Actions affecting trigger states recurse to partitions of partitioned tables (unless ONLY is specified), but never to traditional-inheritance descendants. Adding a constraint recurses only for CHECK constraints that are not marked NO INHERIT.

Changing any part of a system catalog table is not permitted.

Refer to ??? for a further description of valid parameters. ??? has further information on inheritance.

Examples

To add a column of type varchar to a table:

ALTER TABLE distributors ADD COLUMN address varchar(30);

That will cause all existing rows in the table to be filled with null values for the new column.

To add a column with a non-null default:

ALTER TABLE measurements
  ADD COLUMN mtime timestamp with time zone DEFAULT now();

Existing rows will be filled with the current time as the value of the new column, and then new rows will receive the time of their insertion.

To add a column and fill it with a value different from the default to be used later:

ALTER TABLE transactions
  ADD COLUMN status varchar(30) DEFAULT 'old',
  ALTER COLUMN status SET DEFAULT 'current';

Existing rows will be filled with old, but then the default for subsequent commands will be current. The effects are the same as if the two sub-commands had been issued in separate ALTER TABLE commands.

To drop a column from a table:

ALTER TABLE distributors DROP COLUMN address RESTRICT;

To change the types of two existing columns in one operation:

ALTER TABLE distributors
    ALTER COLUMN address TYPE varchar(80),
    ALTER COLUMN name TYPE varchar(100);

To change an integer column containing Unix timestamps to timestamp with time zone via a USING clause:

ALTER TABLE foo
    ALTER COLUMN foo_timestamp SET DATA TYPE timestamp with time zone
    USING
        timestamp with time zone 'epoch' + foo_timestamp * interval '1 second';

The same, when the column has a default expression that won't automatically cast to the new data type:

ALTER TABLE foo
    ALTER COLUMN foo_timestamp DROP DEFAULT,
    ALTER COLUMN foo_timestamp TYPE timestamp with time zone
    USING
        timestamp with time zone 'epoch' + foo_timestamp * interval '1 second',
    ALTER COLUMN foo_timestamp SET DEFAULT now();

To rename an existing column:

ALTER TABLE distributors RENAME COLUMN address TO city;

To rename an existing table:

ALTER TABLE distributors RENAME TO suppliers;

To rename an existing constraint:

ALTER TABLE distributors RENAME CONSTRAINT zipchk TO zip_check;

To add a not-null constraint to a column:

ALTER TABLE distributors ALTER COLUMN street SET NOT NULL;

To remove a not-null constraint from a column:

ALTER TABLE distributors ALTER COLUMN street DROP NOT NULL;

To add a check constraint to a table and all its children:

ALTER TABLE distributors ADD CONSTRAINT zipchk CHECK (char_length(zipcode) = 5);

To add a check constraint only to a table and not to its children:

ALTER TABLE distributors ADD CONSTRAINT zipchk CHECK (char_length(zipcode) = 5) NO INHERIT;

(The check constraint will not be inherited by future children, either.)

To remove a check constraint from a table and all its children:

ALTER TABLE distributors DROP CONSTRAINT zipchk;

To remove a check constraint from one table only:

ALTER TABLE ONLY distributors DROP CONSTRAINT zipchk;

(The check constraint remains in place for any child tables.)

To add a foreign key constraint to a table:

ALTER TABLE distributors ADD CONSTRAINT distfk FOREIGN KEY (address) REFERENCES addresses (address);

To add a foreign key constraint to a table with the least impact on other work:

ALTER TABLE distributors ADD CONSTRAINT distfk FOREIGN KEY (address) REFERENCES addresses (address) NOT VALID;
ALTER TABLE distributors VALIDATE CONSTRAINT distfk;

To add a (multicolumn) unique constraint to a table:

ALTER TABLE distributors ADD CONSTRAINT dist_id_zipcode_key UNIQUE (dist_id, zipcode);

To add an automatically named primary key constraint to a table, noting that a table can only ever have one primary key:

ALTER TABLE distributors ADD PRIMARY KEY (dist_id);

To move a table to a different tablespace:

ALTER TABLE distributors SET TABLESPACE fasttablespace;

To move a table to a different schema:

ALTER TABLE myschema.distributors SET SCHEMA yourschema;

To recreate a primary key constraint, without blocking updates while the index is rebuilt:

CREATE UNIQUE INDEX CONCURRENTLY dist_id_temp_idx ON distributors (dist_id);
ALTER TABLE distributors DROP CONSTRAINT distributors_pkey,
    ADD CONSTRAINT distributors_pkey PRIMARY KEY USING INDEX dist_id_temp_idx;

To attach a partition to a range-partitioned table:

ALTER TABLE measurement
    ATTACH PARTITION measurement_y2016m07 FOR VALUES FROM ('2016-07-01') TO ('2016-08-01');

To attach a partition to a list-partitioned table:

ALTER TABLE cities
    ATTACH PARTITION cities_ab FOR VALUES IN ('a', 'b');

To attach a partition to a hash-partitioned table:

ALTER TABLE orders
    ATTACH PARTITION orders_p4 FOR VALUES WITH (MODULUS 4, REMAINDER 3);

To attach a default partition to a partitioned table:

ALTER TABLE cities
    ATTACH PARTITION cities_partdef DEFAULT;

To detach a partition from a partitioned table:

ALTER TABLE measurement
    DETACH PARTITION measurement_y2015m12;

Compatibility

The forms ADD [COLUMN], DROP [COLUMN], DROP IDENTITY, RESTART, SET DEFAULT, SET DATA TYPE (without USING), SET GENERATED, and SET sequence_option conform with the SQL standard. The form ADD table_constraint conforms with the SQL standard when the USING INDEX and NOT VALID clauses are omitted and the constraint type is one of CHECK, UNIQUE, PRIMARY KEY, or REFERENCES. The other forms are PostgreSQL extensions of the SQL standard. Also, the ability to specify more than one manipulation in a single ALTER TABLE command is an extension.

ALTER TABLE DROP COLUMN can be used to drop the only column of a table, leaving a zero-column table. This is an extension of SQL, which disallows zero-column tables.

See Also

alter_tablespace

ALTER TABLESPACE

ALTER TABLESPACE

7

SQL - Language Statements

ALTER TABLESPACE

change the definition of a tablespace

ALTER TABLESPACE

name

RENAME TO

new_name

ALTER TABLESPACE

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER TABLESPACE

name

SET (

tablespace_option

=

value

[, ... ] ) ALTER TABLESPACE

name

RESET (

tablespace_option

[, ... ] )

Description

ALTER TABLESPACE can be used to change the definition of a tablespace.

You must own the tablespace to change the definition of a tablespace. To alter the owner, you must also be able to SET ROLE to the new owning role. (Note that superusers have these privileges automatically.)

Parameters

<name>
The name of an existing tablespace.

<new_name>
The new name of the tablespace. The new name cannot begin with pg_, as such names are reserved for system tablespaces.

<new_owner>
The new owner of the tablespace.

<tablespace_option>
A tablespace parameter to be set or reset. Currently, the only available parameters are seq_page_cost, random_page_cost, effective_io_concurrency and maintenance_io_concurrency. Setting these values for a particular tablespace will override the planner's usual estimate of the cost of reading pages from tables in that tablespace, and how many concurrent I/Os are issued, as established by the configuration parameters of the same name (see ???, ???, ???, ???). This may be useful if one tablespace is located on a disk which is faster or slower than the remainder of the I/O subsystem.

Examples

Rename tablespace index_space to fast_raid:

ALTER TABLESPACE index_space RENAME TO fast_raid;

Change the owner of tablespace index_space:

ALTER TABLESPACE index_space OWNER TO mary;

Compatibility

There is no ALTER TABLESPACE statement in the SQL standard.

See Also

alter_trigger

ALTER TRIGGER

ALTER TRIGGER

7

SQL - Language Statements

ALTER TRIGGER

change the definition of a trigger

ALTER TRIGGER

name

ON

table_name

RENAME TO

new_name

ALTER TRIGGER

name

ON

table_name

[ NO ] DEPENDS ON EXTENSION

extension_name

Description

ALTER TRIGGER changes properties of an existing trigger.

The RENAME clause changes the name of the given trigger without otherwise changing the trigger definition. If the table that the trigger is on is a partitioned table, then corresponding clone triggers in the partitions are renamed too.

The DEPENDS ON EXTENSION clause marks the trigger as dependent on an extension, such that if the extension is dropped, the trigger will automatically be dropped as well.

You must own the table on which the trigger acts to be allowed to change its properties.

Parameters

<name>
The name of an existing trigger to alter.

<table_name>
The name of the table on which this trigger acts.

<new_name>
The new name for the trigger.

<extension_name>
The name of the extension that the trigger is to depend on (or no longer dependent on, if NO is specified). A trigger that's marked as dependent on an extension is automatically dropped when the extension is dropped.

Notes

The ability to temporarily enable or disable a trigger is provided by ALTER TABLE, not by ALTER TRIGGER, because ALTER TRIGGER has no convenient way to express the option of enabling or disabling all of a table's triggers at once.

Examples

To rename an existing trigger:

ALTER TRIGGER emp_stamp ON emp RENAME TO emp_track_chgs;

To mark a trigger as being dependent on an extension:

ALTER TRIGGER emp_stamp ON emp DEPENDS ON EXTENSION emplib;

Compatibility

ALTER TRIGGER is a PostgreSQL extension of the SQL standard.

See Also

alter_tsconfig

ALTER TEXT SEARCH CONFIGURATION

ALTER TEXT SEARCH CONFIGURATION

7

SQL - Language Statements

ALTER TEXT SEARCH CONFIGURATION

change the definition of a text search configuration

ALTER TEXT SEARCH CONFIGURATION

name

ADD MAPPING FOR

token_type

[, ... ] WITH

dictionary_name

[, ... ] ALTER TEXT SEARCH CONFIGURATION

name

ALTER MAPPING FOR

token_type

[, ... ] WITH

dictionary_name

[, ... ] ALTER TEXT SEARCH CONFIGURATION

name

ALTER MAPPING REPLACE

old_dictionary

WITH

new_dictionary

ALTER TEXT SEARCH CONFIGURATION

name

ALTER MAPPING FOR

token_type

[, ... ] REPLACE

old_dictionary

WITH

new_dictionary

ALTER TEXT SEARCH CONFIGURATION

name

DROP MAPPING [ IF EXISTS ] FOR

token_type

[, ... ] ALTER TEXT SEARCH CONFIGURATION

name

RENAME TO

new_name

ALTER TEXT SEARCH CONFIGURATION

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER TEXT SEARCH CONFIGURATION

name

SET SCHEMA

new_schema

Description

ALTER TEXT SEARCH CONFIGURATION changes the definition of a text search configuration. You can modify its mappings from token types to dictionaries, or change the configuration's name or owner.

You must be the owner of the configuration to use ALTER TEXT SEARCH CONFIGURATION.

Parameters

<name>
The name (optionally schema-qualified) of an existing text search configuration.

<token_type>
The name of a token type that is emitted by the configuration's parser.

<dictionary_name>
The name of a text search dictionary to be consulted for the specified token type(s). If multiple dictionaries are listed, they are consulted in the specified order.

<old_dictionary>
The name of a text search dictionary to be replaced in the mapping.

<new_dictionary>
The name of a text search dictionary to be substituted for <old_dictionary>.

<new_name>
The new name of the text search configuration.

<new_owner>
The new owner of the text search configuration.

<new_schema>
The new schema for the text search configuration.

The ADD MAPPING FOR form installs a list of dictionaries to be consulted for the specified token type(s); it is an error if there is already a mapping for any of the token types. The ALTER MAPPING FOR form does the same, but first removing any existing mapping for those token types. The ALTER MAPPING REPLACE forms substitute <new_dictionary> for <old_dictionary> anywhere the latter appears. This is done for only the specified token types when FOR appears, or for all mappings of the configuration when it doesn't. The DROP MAPPING form removes all dictionaries for the specified token type(s), causing tokens of those types to be ignored by the text search configuration. It is an error if there is no mapping for the token types, unless IF EXISTS appears.

Examples

The following example replaces the english dictionary with the swedish dictionary anywhere that english is used within my_config.

ALTER TEXT SEARCH CONFIGURATION my_config
  ALTER MAPPING REPLACE english WITH swedish;

Compatibility

There is no ALTER TEXT SEARCH CONFIGURATION statement in the SQL standard.

See Also

alter_tsdictionary

ALTER TEXT SEARCH DICTIONARY

ALTER TEXT SEARCH DICTIONARY

7

SQL - Language Statements

ALTER TEXT SEARCH DICTIONARY

change the definition of a text search dictionary

ALTER TEXT SEARCH DICTIONARY

name

(

option

[ =

value

] [, ... ] ) ALTER TEXT SEARCH DICTIONARY

name

RENAME TO

new_name

ALTER TEXT SEARCH DICTIONARY

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER TEXT SEARCH DICTIONARY

name

SET SCHEMA

new_schema

Description

ALTER TEXT SEARCH DICTIONARY changes the definition of a text search dictionary. You can change the dictionary's template-specific options, or change the dictionary's name or owner.

You must be the owner of the dictionary to use ALTER TEXT SEARCH DICTIONARY.

Parameters

<name>
The name (optionally schema-qualified) of an existing text search dictionary.

<option>
The name of a template-specific option to be set for this dictionary.

<value>
The new value to use for a template-specific option. If the equal sign and value are omitted, then any previous setting for the option is removed from the dictionary, allowing the default to be used.

<new_name>
The new name of the text search dictionary.

<new_owner>
The new owner of the text search dictionary.

<new_schema>
The new schema for the text search dictionary.

Template-specific options can appear in any order.

Examples

The following example command changes the stopword list for a Snowball-based dictionary. Other parameters remain unchanged.

ALTER TEXT SEARCH DICTIONARY my_dict ( StopWords = newrussian );

The following example command changes the language option to dutch, and removes the stopword option entirely.

ALTER TEXT SEARCH DICTIONARY my_dict ( language = dutch, StopWords );

The following example command “updates” the dictionary's definition without actually changing anything.

ALTER TEXT SEARCH DICTIONARY my_dict ( dummy );

(The reason this works is that the option removal code doesn't complain if there is no such option.) This trick is useful when changing configuration files for the dictionary: the ALTER will force existing database sessions to re-read the configuration files, which otherwise they would never do if they had read them earlier.

Compatibility

There is no ALTER TEXT SEARCH DICTIONARY statement in the SQL standard.

See Also

alter_tsparser

ALTER TEXT SEARCH PARSER

ALTER TEXT SEARCH PARSER

7

SQL - Language Statements

ALTER TEXT SEARCH PARSER

change the definition of a text search parser

ALTER TEXT SEARCH PARSER

name

RENAME TO

new_name

ALTER TEXT SEARCH PARSER

name

SET SCHEMA

new_schema

Description

ALTER TEXT SEARCH PARSER changes the definition of a text search parser. Currently, the only supported functionality is to change the parser's name.

You must be a superuser to use ALTER TEXT SEARCH PARSER.

Parameters

<name>
The name (optionally schema-qualified) of an existing text search parser.

<new_name>
The new name of the text search parser.

<new_schema>
The new schema for the text search parser.

Compatibility

There is no ALTER TEXT SEARCH PARSER statement in the SQL standard.

See Also

alter_tstemplate

ALTER TEXT SEARCH TEMPLATE

ALTER TEXT SEARCH TEMPLATE

7

SQL - Language Statements

ALTER TEXT SEARCH TEMPLATE

change the definition of a text search template

ALTER TEXT SEARCH TEMPLATE

name

RENAME TO

new_name

ALTER TEXT SEARCH TEMPLATE

name

SET SCHEMA

new_schema

Description

ALTER TEXT SEARCH TEMPLATE changes the definition of a text search template. Currently, the only supported functionality is to change the template's name.

You must be a superuser to use ALTER TEXT SEARCH TEMPLATE.

Parameters

<name>
The name (optionally schema-qualified) of an existing text search template.

<new_name>
The new name of the text search template.

<new_schema>
The new schema for the text search template.

Compatibility

There is no ALTER TEXT SEARCH TEMPLATE statement in the SQL standard.

See Also

alter_type

ALTER TYPE

ALTER TYPE

7

SQL - Language Statements

ALTER TYPE

change the definition of a type

ALTER TYPE

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER TYPE

name

RENAME TO

new_name

ALTER TYPE

name

SET SCHEMA

new_schema

ALTER TYPE

name

RENAME ATTRIBUTE

attribute_name

TO

new_attribute_name

[ CASCADE | RESTRICT ] ALTER TYPE

name

action

[, ... ] ALTER TYPE

name

ADD VALUE [ IF NOT EXISTS ]

new_enum_value

[ { BEFORE | AFTER }

neighbor_enum_value

] ALTER TYPE

name

RENAME VALUE

existing_enum_value

TO

new_enum_value

ALTER TYPE

name

SET (

property

=

value

[, ... ] )

where

action

is one of:

ADD ATTRIBUTE

attribute_name

data_type

[ COLLATE

collation

] [ CASCADE | RESTRICT ] DROP ATTRIBUTE [ IF EXISTS ]

attribute_name

[ CASCADE | RESTRICT ] ALTER ATTRIBUTE

attribute_name

[ SET DATA ] TYPE

data_type

[ COLLATE

collation

] [ CASCADE | RESTRICT ]

Description

ALTER TYPE changes the definition of an existing type. There are several subforms:

OWNER
This form changes the owner of the type.

RENAME
This form changes the name of the type.

SET SCHEMA
This form moves the type into another schema.

RENAME ATTRIBUTE
This form is only usable with composite types. It changes the name of an individual attribute of the type.

ADD ATTRIBUTE
This form adds a new attribute to a composite type, using the same syntax as CREATE TYPE.

DROP ATTRIBUTE [ IF EXISTS ]
This form drops an attribute from a composite type. If IF EXISTS is specified and the attribute does not exist, no error is thrown. In this case a notice is issued instead.

ALTER ATTRIBUTE ... SET DATA TYPE
This form changes the type of an attribute of a composite type.

ADD VALUE [ IF NOT EXISTS ] [ BEFORE | AFTER ]
This form adds a new value to an enum type. The new value's place in the enum's ordering can be specified as being BEFORE or AFTER one of the existing values. Otherwise, the new item is added at the end of the list of values.

If IF NOT EXISTS is specified, it is not an error if the type already contains the new value: a notice is issued but no other action is taken. Otherwise, an error will occur if the new value is already present.

RENAME VALUE
This form renames a value of an enum type. The value's place in the enum's ordering is not affected. An error will occur if the specified value is not present or the new name is already present.

SET ( property = value [, ... ] )
This form is only applicable to base types. It allows adjustment of a subset of the base-type properties that can be set in CREATE TYPE. Specifically, these properties can be changed:

See ??? for more details about these type properties. Note that where appropriate, a change in these properties for a base type will be propagated automatically to domains based on that type.

The ADD ATTRIBUTE, DROP ATTRIBUTE, and ALTER ATTRIBUTE actions can be combined into a list of multiple alterations to apply in parallel. For example, it is possible to add several attributes and/or alter the type of several attributes in a single command.

You must own the type to use ALTER TYPE. To change the schema of a type, you must also have CREATE privilege on the new schema. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the type's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the type. However, a superuser can alter ownership of any type anyway.) To add an attribute or alter an attribute type, you must also have USAGE privilege on the attribute's data type.

Parameters

<name>
The name (possibly schema-qualified) of an existing type to alter.

<new_name>
The new name for the type.

<new_owner>
The user name of the new owner of the type.

<new_schema>
The new schema for the type.

<attribute_name>
The name of the attribute to add, alter, or drop.

<new_attribute_name>
The new name of the attribute to be renamed.

<data_type>
The data type of the attribute to add, or the new type of the attribute to alter.

<new_enum_value>
The new value to be added to an enum type's list of values, or the new name to be given to an existing value. Like all enum literals, it needs to be quoted.

<neighbor_enum_value>
The existing enum value that the new value should be added immediately before or after in the enum type's sort ordering. Like all enum literals, it needs to be quoted.

<existing_enum_value>
The existing enum value that should be renamed. Like all enum literals, it needs to be quoted.

<property>
The name of a base-type property to be modified; see above for possible values.

CASCADE
Automatically propagate the operation to typed tables of the type being altered, and their descendants.

RESTRICT
Refuse the operation if the type being altered is the type of a typed table. This is the default.

Notes

If ALTER TYPE ... ADD VALUE (the form that adds a new value to an enum type) is executed inside a transaction block, the new value cannot be used until after the transaction has been committed.

Comparisons involving an added enum value will sometimes be slower than comparisons involving only original members of the enum type. This will usually only occur if BEFORE or AFTER is used to set the new value's sort position somewhere other than at the end of the list. However, sometimes it will happen even though the new value is added at the end (this occurs if the OID counter “wrapped around” since the original creation of the enum type). The slowdown is usually insignificant; but if it matters, optimal performance can be regained by dropping and recreating the enum type, or by dumping and restoring the database.

Examples

To rename a data type:

ALTER TYPE electronic_mail RENAME TO email;

To change the owner of the type email to joe:

ALTER TYPE email OWNER TO joe;

To change the schema of the type email to customers:

ALTER TYPE email SET SCHEMA customers;

To add a new attribute to a composite type:

ALTER TYPE compfoo ADD ATTRIBUTE f3 int;

To add a new value to an enum type in a particular sort position:

ALTER TYPE colors ADD VALUE 'orange' AFTER 'red';

To rename an enum value:

ALTER TYPE colors RENAME VALUE 'purple' TO 'mauve';

To create binary I/O functions for an existing base type:

CREATE FUNCTION mytypesend(mytype) RETURNS bytea ...;
CREATE FUNCTION mytyperecv(internal, oid, integer) RETURNS mytype ...;
ALTER TYPE mytype SET (
    SEND = mytypesend,
    RECEIVE = mytyperecv
);

Compatibility

The variants to add and drop attributes are part of the SQL standard; the other variants are PostgreSQL extensions.

See Also

alter_user

ALTER USER

ALTER USER

7

SQL - Language Statements

ALTER USER

change a database role

ALTER USER

role_specification

[ WITH ]

option

[ ... ]

where

option

can be:

SUPERUSER | NOSUPERUSER | CREATEDB | NOCREATEDB | CREATEROLE | NOCREATEROLE | INHERIT | NOINHERIT | LOGIN | NOLOGIN | REPLICATION | NOREPLICATION | BYPASSRLS | NOBYPASSRLS | CONNECTION LIMIT

connlimit

| [ ENCRYPTED ] PASSWORD '

password

' | PASSWORD NULL | VALID UNTIL '

timestamp

' ALTER USER

name

RENAME TO

new_name

ALTER USER {

role_specification

| ALL } [ IN DATABASE

database_name

] SET

configuration_parameter

{ TO | = } {

value

| DEFAULT } ALTER USER {

role_specification

| ALL } [ IN DATABASE

database_name

] SET

configuration_parameter

FROM CURRENT ALTER USER {

role_specification

| ALL } [ IN DATABASE

database_name

] RESET

configuration_parameter

ALTER USER {

role_specification

| ALL } [ IN DATABASE

database_name

] RESET ALL

where

role_specification

can be:

role_name

| CURRENT_ROLE | CURRENT_USER | SESSION_USER

Description

ALTER USER is now an alias for ALTER ROLE.

Compatibility

The ALTER USER statement is a PostgreSQL extension. The SQL standard leaves the definition of users to the implementation.

See Also

alter_user_mapping

ALTER USER MAPPING

ALTER USER MAPPING

7

SQL - Language Statements

ALTER USER MAPPING

change the definition of a user mapping

ALTER USER MAPPING FOR {

user_name

| USER | CURRENT_ROLE | CURRENT_USER | SESSION_USER | PUBLIC } SERVER

server_name

OPTIONS ( [ ADD | SET | DROP ]

option

['

value

'] [, ... ] )

Description

ALTER USER MAPPING changes the definition of a user mapping.

The owner of a foreign server can alter user mappings for that server for any user. Also, a user can alter a user mapping for their own user name if USAGE privilege on the server has been granted to the user.

Parameters

<user_name>
User name of the mapping. CURRENT_ROLE, CURRENT_USER, and USER match the name of the current user. PUBLIC is used to match all present and future user names in the system.

<server_name>
Server name of the user mapping.

OPTIONS ( [ ADD | SET | DROP ] option ['value'] [, ... ] )
Change options for the user mapping. The new options override any previously specified options. ADD, SET, and DROP specify the action to be performed. ADD is assumed if no operation is explicitly specified. Option names must be unique; options are also validated by the server's foreign-data wrapper.

Examples

Change the password for user mapping bob, server foo:

ALTER USER MAPPING FOR bob SERVER foo OPTIONS (SET password 'public');

Compatibility

ALTER USER MAPPING conforms to ISO/IEC 9075-9 (SQL/MED). There is a subtle syntax issue: The standard omits the FOR key word. Since both CREATE USER MAPPING and DROP USER MAPPING use FOR in analogous positions, and IBM DB2 (being the other major SQL/MED implementation) also requires it for ALTER USER MAPPING, PostgreSQL diverges from the standard here in the interest of consistency and interoperability.

See Also

alter_view

ALTER VIEW

ALTER VIEW

7

SQL - Language Statements

ALTER VIEW

change the definition of a view

ALTER VIEW [ IF EXISTS ]

name

ALTER [ COLUMN ]

column_name

SET DEFAULT

expression

ALTER VIEW [ IF EXISTS ]

name

ALTER [ COLUMN ]

column_name

DROP DEFAULT ALTER VIEW [ IF EXISTS ]

name

OWNER TO {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ALTER VIEW [ IF EXISTS ]

name

RENAME [ COLUMN ]

column_name

TO

new_column_name

ALTER VIEW [ IF EXISTS ]

name

RENAME TO

new_name

ALTER VIEW [ IF EXISTS ]

name

SET SCHEMA

new_schema

ALTER VIEW [ IF EXISTS ]

name

SET (

view_option_name

[=

view_option_value

] [, ... ] ) ALTER VIEW [ IF EXISTS ]

name

RESET (

view_option_name

[, ... ] )

Description

ALTER VIEW changes various auxiliary properties of a view. (If you want to modify the view's defining query, use CREATE OR REPLACE VIEW.)

You must own the view to use ALTER VIEW. To change a view's schema, you must also have CREATE privilege on the new schema. To alter the owner, you must be able to SET ROLE to the new owning role, and that role must have CREATE privilege on the view's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the view. However, a superuser can alter ownership of any view anyway.)

Parameters

<name>
The name (optionally schema-qualified) of an existing view.

<column_name>
Name of an existing column.

<new_column_name>
New name for an existing column.

IF EXISTS
Do not throw an error if the view does not exist. A notice is issued in this case.

SET/DROP DEFAULT
These forms set or remove the default value for a column. A view column's default value is substituted into any INSERT or UPDATE command whose target is the view, before applying any rules or triggers for the view. The view's default will therefore take precedence over any default values from underlying relations.

<new_owner>
The user name of the new owner of the view.

<new_name>
The new name for the view.

<new_schema>
The new schema for the view.

SET ( view_option_name [= view_option_value] [, ... ] ); RESET ( view_option_name [, ... ] )
Sets or resets a view option. Currently supported options are:

check_option (enum)
Changes the check option of the view. The value must be local or cascaded.

security_barrier (boolean)
Changes the security-barrier property of the view. The value must be a Boolean value, such as true or false.

security_invoker (boolean)
Changes the security-invoker property of the view. The value must be a Boolean value, such as true or false.

Notes

For historical reasons, ALTER TABLE can be used with views too; but the only variants of ALTER TABLE that are allowed with views are equivalent to the ones shown above.

Examples

To rename the view foo to bar:

ALTER VIEW foo RENAME TO bar;

To attach a default column value to an updatable view:

CREATE TABLE base_table (id int, ts timestamptz);
CREATE VIEW a_view AS SELECT * FROM base_table;
ALTER VIEW a_view ALTER COLUMN ts SET DEFAULT now();
INSERT INTO base_table(id) VALUES(1);  -- ts will receive a NULL
INSERT INTO a_view(id) VALUES(2);  -- ts will receive the current time

Compatibility

ALTER VIEW is a PostgreSQL extension of the SQL standard.

See Also

analyze

ANALYZE

ANALYZE

7

SQL - Language Statements

ANALYZE

collect statistics about a database

ANALYZE [ (

option

[, ...] ) ] [

table_and_columns

[, ...] ]

where

option

can be one of:

VERBOSE [

boolean

] SKIP_LOCKED [

boolean

] BUFFER_USAGE_LIMIT

size

and

table_and_columns

is:

[ ONLY ]

table_name

[ * ] [ (

column_name

[, ...] ) ]

Description

ANALYZE collects statistics about the contents of tables in the database, and stores the results in the pg_statistic system catalog. Subsequently, the query planner uses these statistics to help determine the most efficient execution plans for queries.

Without a <table_and_columns> list, ANALYZE processes every table and materialized view in the current database that the current user has permission to analyze. With a list, ANALYZE processes only those table(s). It is further possible to give a list of column names for a table, in which case only the statistics for those columns are collected.

Parameters

VERBOSE
Prints detailed information about analyzing each table at INFO level.

SKIP_LOCKED
Specifies that ANALYZE should not wait for any conflicting locks to be released when beginning work on a relation: if a relation cannot be locked immediately without waiting, the relation is skipped. Note that even with this option, ANALYZE may still block when opening the relation's indexes or when acquiring sample rows from partitions, table inheritance children, and some types of foreign tables. Also, while ANALYZE ordinarily processes all partitions of specified partitioned tables, this option will cause ANALYZE to skip all partitions if there is a conflicting lock on the partitioned table.

BUFFER_USAGE_LIMIT
Specifies the Buffer Access Strategy ring buffer size for ANALYZE. This size is used to calculate the number of shared buffers which will be reused as part of this strategy. 0 disables use of a Buffer Access Strategy. When this option is not specified, ANALYZE uses the value from ???. Higher settings can allow ANALYZE to run more quickly, but having too large a setting may cause too many other useful pages to be evicted from shared buffers. The minimum value is 128 kB and the maximum value is 16 GB.

<boolean>
Specifies whether the selected option should be turned on or off. You can write TRUE, ON, or 1 to enable the option, and FALSE, OFF, or 0 to disable it. The <boolean> value can also be omitted, in which case TRUE is assumed.

<size>
Specifies an amount of memory in kilobytes. Sizes may also be specified as a string containing the numerical size followed by any one of the following memory units: B (bytes), kB (kilobytes), MB (megabytes), GB (gigabytes), or TB (terabytes).

<table_name>
The name (possibly schema-qualified) of a specific table to analyze. If omitted, all regular tables, partitioned tables, and materialized views in the current database are analyzed (but not foreign tables). If ONLY is specified before the table name, only that table is analyzed. If ONLY is not specified, the table and all its inheritance child tables or partitions (if any) are analyzed. Optionally, * can be specified after the table name to explicitly indicate that inheritance child tables (or partitions) are to be analyzed.

<column_name>
The name of a specific column to analyze. Defaults to all columns.

Outputs

When VERBOSE is specified, ANALYZE prints detailed information about analyzing each table at INFO level, including which table is currently being processed and various statistics about the tables.

Notes

To analyze a table, one must ordinarily have the MAINTAIN privilege on the table. However, database owners are allowed to analyze all tables in their databases, except shared catalogs. ANALYZE will skip over any tables that the calling user does not have permission to analyze.

Foreign tables are analyzed only when explicitly selected. Not all foreign data wrappers support ANALYZE. If the table's wrapper does not support ANALYZE, the command prints a warning and does nothing.

In the default PostgreSQL configuration, the autovacuum daemon (see ???) takes care of automatic analyzing of tables when they are first loaded with data, and as they change throughout regular operation. When autovacuum is disabled, it is a good idea to run ANALYZE periodically, or just after making major changes in the contents of a table. Accurate statistics will help the planner to choose the most appropriate query plan, and thereby improve the speed of query processing. A common strategy for read-mostly databases is to run VACUUM and ANALYZE once a day during a low-usage time of day. (This will not be sufficient if there is heavy update activity.)

While ANALYZE is running, the ??? is temporarily changed to pg_catalog, pg_temp.

ANALYZE requires only a read lock on the target table, so it can run in parallel with other non-DDL activity on the table.

The statistics collected by ANALYZE usually include a list of some of the most common values in each column and a histogram showing the approximate data distribution in each column. One or both of these can be omitted if ANALYZE deems them uninteresting (for example, in a unique-key column, there are no common values) or if the column data type does not support the appropriate operators. There is more information about the statistics in ???.

For large tables, ANALYZE takes a random sample of the table contents, rather than examining every row. This allows even very large tables to be analyzed in a small amount of time. Note, however, that the statistics are only approximate, and will change slightly each time ANALYZE is run, even if the actual table contents did not change. This might result in small changes in the planner's estimated costs shown by EXPLAIN. In rare situations, this non-determinism will cause the planner's choices of query plans to change after ANALYZE is run. To avoid this, raise the amount of statistics collected by ANALYZE, as described below.

The extent of analysis can be controlled by adjusting the ??? configuration variable, or on a column-by-column basis by setting the per-column statistics target with ALTER TABLE ... ALTER COLUMN ... SET STATISTICS. The target value sets the maximum number of entries in the most-common-value list and the maximum number of bins in the histogram. The default target value is 100, but this can be adjusted up or down to trade off accuracy of planner estimates against the time taken for ANALYZE and the amount of space occupied in pg_statistic. In particular, setting the statistics target to zero disables collection of statistics for that column. It might be useful to do that for columns that are never used as part of the WHERE, GROUP BY, or ORDER BY clauses of queries, since the planner will have no use for statistics on such columns.

The largest statistics target among the columns being analyzed determines the number of table rows sampled to prepare the statistics. Increasing the target causes a proportional increase in the time and space needed to do ANALYZE.

One of the values estimated by ANALYZE is the number of distinct values that appear in each column. Because only a subset of the rows are examined, this estimate can sometimes be quite inaccurate, even with the largest possible statistics target. If this inaccuracy leads to bad query plans, a more accurate value can be determined manually and then installed with ALTER TABLE ... ALTER COLUMN ... SET (n_distinct = ...).

If the table being analyzed has inheritance children, ANALYZE gathers two sets of statistics: one on the rows of the parent table only, and a second including rows of both the parent table and all of its children. This second set of statistics is needed when planning queries that process the inheritance tree as a whole. The autovacuum daemon, however, will only consider inserts or updates on the parent table itself when deciding whether to trigger an automatic analyze for that table. If that table is rarely inserted into or updated, the inheritance statistics will not be up to date unless you run ANALYZE manually. By default, ANALYZE will also recursively collect and update the statistics for each inheritance child table. The ONLY keyword may be used to disable this.

For partitioned tables, ANALYZE gathers statistics by sampling rows from all partitions. By default, ANALYZE will also recursively collect and update the statistics for each partition. The ONLY keyword may be used to disable this.

The autovacuum daemon does not process partitioned tables, nor does it process inheritance parents if only the children are ever modified. It is usually necessary to periodically run a manual ANALYZE to keep the statistics of the table hierarchy up to date.

If any child tables or partitions are foreign tables whose foreign data wrappers do not support ANALYZE, those tables are ignored while gathering inheritance statistics.

If the table being analyzed is completely empty, ANALYZE will not record new statistics for that table. Any existing statistics will be retained.

Each backend running ANALYZE will report its progress in the pg_stat_progress_analyze view. See ??? for details.

Compatibility

There is no ANALYZE statement in the SQL standard.

The following syntax was used before PostgreSQL version 11 and is still supported: ANALYZE [ VERBOSE ] [ <table_and_columns> [, ...] ]

See Also

begin

BEGIN

BEGIN

7

SQL - Language Statements

BEGIN

start a transaction block

BEGIN [ WORK | TRANSACTION ] [

transaction_mode

[, ...] ]

where

transaction_mode

is one of:

ISOLATION LEVEL { SERIALIZABLE | REPEATABLE READ | READ COMMITTED | READ UNCOMMITTED } READ WRITE | READ ONLY [ NOT ] DEFERRABLE

Description

BEGIN initiates a transaction block, that is, all statements after a BEGIN command will be executed in a single transaction until an explicit COMMIT or ROLLBACK is given. By default (without BEGIN), PostgreSQL executes transactions in “autocommit” mode, that is, each statement is executed in its own transaction and a commit is implicitly performed at the end of the statement (if execution was successful, otherwise a rollback is done).

Statements are executed more quickly in a transaction block, because transaction start/commit requires significant CPU and disk activity. Execution of multiple statements inside a transaction is also useful to ensure consistency when making several related changes: other sessions will be unable to see the intermediate states wherein not all the related updates have been done.

If the isolation level, read/write mode, or deferrable mode is specified, the new transaction has those characteristics, as if SET TRANSACTION was executed.

Parameters

WORK; TRANSACTION
Optional key words. They have no effect.

Refer to ??? for information on the meaning of the other parameters to this statement.

Notes

START TRANSACTION has the same functionality as BEGIN.

Use COMMIT or ROLLBACK to terminate a transaction block.

Issuing BEGIN when already inside a transaction block will provoke a warning message. The state of the transaction is not affected. To nest transactions within a transaction block, use savepoints (see ???).

For reasons of backwards compatibility, the commas between successive <transaction_modes> can be omitted.

Examples

To begin a transaction block:

BEGIN;

Compatibility

BEGIN is a PostgreSQL language extension. It is equivalent to the SQL-standard command START TRANSACTION, whose reference page contains additional compatibility information.

The DEFERRABLE <transaction_mode> is a PostgreSQL language extension.

Incidentally, the BEGIN key word is used for a different purpose in embedded SQL. You are advised to be careful about the transaction semantics when porting database applications.

See Also

call

CALL

CALL

7

SQL - Language Statements

CALL

invoke a procedure

CALL

name

( [

argument

] [, ...] )

Description

CALL executes a procedure.

If the procedure has any output parameters, then a result row will be returned, containing the values of those parameters.

Parameters

<name>
The name (optionally schema-qualified) of the procedure.

<argument>
An argument expression for the procedure call.

Arguments can include parameter names, using the syntax name => value. This works the same as in ordinary function calls; see ??? for details.

Arguments must be supplied for all procedure parameters that lack defaults, including OUT parameters. However, arguments matching OUT parameters are not evaluated, so it's customary to just write NULL for them. (Writing something else for an OUT parameter might cause compatibility problems with future PostgreSQL versions.)

Notes

The user must have EXECUTE privilege on the procedure in order to be allowed to invoke it.

To call a function (not a procedure), use SELECT instead.

If CALL is executed in a transaction block, then the called procedure cannot execute transaction control statements. Transaction control statements are only allowed if CALL is executed in its own transaction.

PL/pgSQL handles output parameters in CALL commands differently; see ???.

Examples

CALL do_db_maintenance();

Compatibility

CALL conforms to the SQL standard, except for the handling of output parameters. The standard says that users should write variables to receive the values of output parameters.

See Also

checkpoint

CHECKPOINT

CHECKPOINT

7

SQL - Language Statements

CHECKPOINT

force a write-ahead log checkpoint

CHECKPOINT [ (

option

[, ...] ) ]

where

option

can be one of:

FLUSH_UNLOGGED [

boolean

] MODE { FAST | SPREAD }

Description

A checkpoint is a point in the write-ahead log sequence at which all data files for permanent relations have been updated to reflect the information in the log. All data for permanent relation files will be flushed to disk. Dirty buffers of unlogged relations are not flushed unless FLUSH_UNLOGGED is specified. Refer to ??? for more details about what happens during a checkpoint.

By default, the CHECKPOINT command forces a fast checkpoint when the command is issued, without waiting for a regular checkpoint scheduled by the system (controlled by the settings in ???). To request the checkpoint be spread over a longer interval, set the MODE option to SPREAD. CHECKPOINT is not intended for use during normal operation.

The server may consolidate concurrently requested checkpoints. Such consolidated requests will contain a combined set of options. For example, if one session requests a fast checkpoint and another requests a spread checkpoint, the server may combine those requests and perform one fast checkpoint.

If executed during recovery, the CHECKPOINT command will force a restartpoint (see ???) rather than writing a new checkpoint.

Only superusers or users with the privileges of the ??? role can call CHECKPOINT.

Parameters

FLUSH_UNLOGGED
Normally, CHECKPOINT does not flush dirty buffers of unlogged relations. This option, which is disabled by default, enables flushing unlogged relations to disk.

MODE
When set to FAST, which is the default, the requested checkpoint will be completed as fast as possible, which may result in a significantly higher rate of I/O during the checkpoint.

MODE can also be set to SPREAD to request the checkpoint be spread over a longer interval (controlled via the settings in ???), like a regular checkpoint scheduled by the system. This can reduce the rate of I/O during the checkpoint.

<boolean>
Specifies whether the selected option should be turned on or off. You can write TRUE, ON, or 1 to enable the option, and FALSE, OFF, or 0 to disable it. The <boolean> value can also be omitted, in which case TRUE is assumed.

Compatibility

The CHECKPOINT command is a PostgreSQL language extension.

close

CLOSE

cursor

CLOSE

CLOSE

7

SQL - Language Statements

CLOSE

close a cursor

CLOSE {

name

| ALL }

Description

CLOSE frees the resources associated with an open cursor. After the cursor is closed, no subsequent operations are allowed on it. A cursor should be closed when it is no longer needed.

Every non-holdable open cursor is implicitly closed when a transaction is terminated by COMMIT or ROLLBACK. A holdable cursor is implicitly closed if the transaction that created it aborts via ROLLBACK. If the creating transaction successfully commits, the holdable cursor remains open until an explicit CLOSE is executed, or the client disconnects.

Parameters

<name>
The name of an open cursor to close.

ALL
Close all open cursors.

Notes

PostgreSQL does not have an explicit OPEN cursor statement; a cursor is considered open when it is declared. Use the DECLARE statement to declare a cursor.

You can see all available cursors by querying the pg_cursors system view.

If a cursor is closed after a savepoint which is later rolled back, the CLOSE is not rolled back; that is, the cursor remains closed.

Examples

Close the cursor liahona:

CLOSE liahona;

Compatibility

CLOSE is fully conforming with the SQL standard. CLOSE ALL is a PostgreSQL extension.

See Also

cluster

CLUSTER

CLUSTER

7

SQL - Language Statements

CLUSTER

cluster a table according to an index

CLUSTER [ (

option

[, ...] ) ] [

table_name

[ USING

index_name

] ]

where

option

can be one of:

VERBOSE [

boolean

]

Description

The CLUSTER command is equivalent to ??? with a USING INDEX clause. See there for more details.

Parameters

<table_name>
The name (possibly schema-qualified) of a table.

<index_name>
The name of an index.

VERBOSE
Prints a progress report as each table is clustered at INFO level.

<boolean>
Specifies whether the selected option should be turned on or off. You can write TRUE, ON, or 1 to enable the option, and FALSE, OFF, or 0 to disable it. The <boolean> value can also be omitted, in which case TRUE is assumed.

Notes

To cluster a table, one must have the MAINTAIN privilege on the table.

While CLUSTER is running, the ??? is temporarily changed to pg_catalog, pg_temp.

Because CLUSTER remembers which indexes are clustered, one can cluster the tables one wants clustered manually the first time, then set up a periodic maintenance script that executes CLUSTER without any parameters, so that the desired tables are periodically reclustered.

Each backend running CLUSTER will report its progress in the pg_stat_progress_cluster view. See ??? for details.

Clustering a partitioned table clusters each of its partitions using the partition of the specified partitioned index. When clustering a partitioned table, the index may not be omitted. Partitions that are foreign tables are skipped. CLUSTER on a partitioned table cannot be executed inside a transaction block.

Examples

Cluster the table employees on the basis of its index employees_ind:

CLUSTER employees USING employees_ind;

Cluster the employees table using the same index that was used before:

CLUSTER employees;

Cluster all tables in the database that have previously been clustered:

CLUSTER;

Compatibility

There is no CLUSTER statement in the SQL standard.

The following syntax was used before PostgreSQL 17 and is still supported: CLUSTER [ VERBOSE ] [ <table_name> [ USING <index_name> ] ]

The following syntax was used before PostgreSQL 8.3 and is still supported: CLUSTER <index_name> ON <table_name>

See Also

clusterdb

clusterdb

clusterdb

1

Application

clusterdb

cluster a

PostgreSQL

database

clusterdb

connection-option

option

--table

-t

table

dbname

-a

--all

Description

clusterdb is a utility for reclustering tables in a PostgreSQL database. It finds tables that have previously been clustered, and clusters them again on the same index that was last used. Tables that have never been clustered are not affected.

clusterdb is a wrapper around the SQL command ???. There is no effective difference between clustering databases via this utility and via other methods for accessing the server.

Options

clusterdb accepts the following command-line arguments:

-a; --all
Cluster all databases.

-d dbname; --dbname=dbname
Specifies the name of the database to be clustered, when -a/--all is not used. If this is not specified, the database name is read from the environment variable PGDATABASE. If that is not set, the user name specified for the connection is used. The <dbname> can be a connection string. If so, connection string parameters will override any conflicting command line options.

-e; --echo
Echo the commands that clusterdb generates and sends to the server.

-q; --quiet
Do not display progress messages.

-t table; --table=table
Cluster <table> only. Multiple tables can be clustered by writing multiple -t switches.

-v; --verbose
Print detailed information during processing.

-V; --version
Print the clusterdb version and exit.

-?; --help
Show help about clusterdb command line arguments, and exit.

clusterdb also accepts the following command-line arguments for connection parameters:

-h host; --host=host
Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for the Unix domain socket.

-p port; --port=port
Specifies the TCP port or local Unix domain socket file extension on which the server is listening for connections.

-U username; --username=username
User name to connect as.

-w; --no-password
Never issue a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password.

-W; --password
Force clusterdb to prompt for a password before connecting to a database.

This option is never essential, since clusterdb will automatically prompt for a password if the server demands password authentication. However, clusterdb will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt.

--maintenance-db=dbname
When the -a/--all is used, connect to this database to gather the list of databases to cluster. If not specified, the postgres database will be used, or if that does not exist, template1 will be used. This can be a connection string. If so, connection string parameters will override any conflicting command line options. Also, connection string parameters other than the database name itself will be re-used when connecting to other databases.

Environment

PGDATABASE; PGHOST; PGPORT; PGUSER
Default connection parameters

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

This utility, like most other PostgreSQL utilities, also uses the environment variables supported by libpq (see ???).

Diagnostics

In case of difficulty, see ??? and ??? for discussions of potential problems and error messages. The database server must be running at the targeted host. Also, any default connection settings and environment variables used by the libpq front-end library will apply.

Examples

To cluster the database test:

$ clusterdb test

To cluster a single table foo in a database named xyzzy:

$ clusterdb --table=foo xyzzy

See Also

comment

COMMENT

COMMENT

7

SQL - Language Statements

COMMENT

define or change the comment of an object

COMMENT ON { ACCESS METHOD

object_name

| AGGREGATE

aggregate_name

(

aggregate_signature

) | CAST (

source_type

AS

target_type

) | COLLATION

object_name

| COLUMN

relation_name

.

column_name

| CONSTRAINT

constraint_name

ON

table_name

| CONSTRAINT

constraint_name

ON DOMAIN

domain_name

| CONVERSION

object_name

| DATABASE

object_name

| DOMAIN

object_name

| EXTENSION

object_name

| EVENT TRIGGER

object_name

| FOREIGN DATA WRAPPER

object_name

| FOREIGN TABLE

object_name

| FUNCTION

function_name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] | INDEX

object_name

| LARGE OBJECT

large_object_oid

| MATERIALIZED VIEW

object_name

| OPERATOR

operator_name

(

left_type

,

right_type

) | OPERATOR CLASS

object_name

USING

index_method

| OPERATOR FAMILY

object_name

USING

index_method

| POLICY

policy_name

ON

table_name

| [ PROCEDURAL ] LANGUAGE

object_name

| PROCEDURE

procedure_name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] | PUBLICATION

object_name

| ROLE

object_name

| ROUTINE

routine_name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] | RULE

rule_name

ON

table_name

| SCHEMA

object_name

| SEQUENCE

object_name

| SERVER

object_name

| STATISTICS

object_name

| SUBSCRIPTION

object_name

| TABLE

object_name

| TABLESPACE

object_name

| TEXT SEARCH CONFIGURATION

object_name

| TEXT SEARCH DICTIONARY

object_name

| TEXT SEARCH PARSER

object_name

| TEXT SEARCH TEMPLATE

object_name

| TRANSFORM FOR

type_name

LANGUAGE

lang_name

| TRIGGER

trigger_name

ON

table_name

| TYPE

object_name

| VIEW

object_name

} IS {

string_literal

| NULL }

where

aggregate_signature

is:

* | [

argmode

] [

argname

]

argtype

[ , ... ] | [ [

argmode

] [

argname

]

argtype

[ , ... ] ] ORDER BY [

argmode

] [

argname

]

argtype

[ , ... ]

Description

COMMENT stores, replaces, or removes the comment on a database object.

Only one comment string is stored for each object. Issuing a new COMMENT command for the same object replaces the existing comment. Specifying NULL or an empty string ('') removes the comment. Comments are automatically dropped when their object is dropped.

A SHARE UPDATE EXCLUSIVE lock is acquired on the object to be commented.

For most kinds of object, only the object's owner can set the comment. Roles don't have owners, so the rule for COMMENT ON ROLE is that you must be superuser to comment on a superuser role, or have the CREATEROLE privilege and have been granted ADMIN OPTION on the target role. Likewise, access methods don't have owners either; you must be superuser to comment on an access method. Of course, a superuser can comment on anything.

Comments can be viewed using psql's \d family of commands. Other user interfaces to retrieve comments can be built atop the same built-in functions that psql uses, namely obj_description, col_description, and shobj_description (see ???).

Parameters

<object_name>; <relation_name>.<column_name>; <aggregate_name>; <constraint_name>; <function_name>; <operator_name>; <policy_name>; <procedure_name>; <routine_name>; <rule_name>; <trigger_name>
The name of the object to be commented. Names of objects that reside in schemas (tables, functions, etc.) can be schema-qualified. When commenting on a column, <relation_name> must refer to a table, view, composite type, or foreign table.

<table_name>; <domain_name>
When creating a comment on a constraint, a trigger, a rule or a policy these parameters specify the name of the table or domain on which that object is defined.

<source_type>
The name of the source data type of the cast.

<target_type>
The name of the target data type of the cast.

<argmode>
The mode of a function, procedure, or aggregate argument: IN, OUT, INOUT, or VARIADIC. If omitted, the default is IN. Note that COMMENT does not actually pay any attention to OUT arguments, since only the input arguments are needed to determine the function's identity. So it is sufficient to list the IN, INOUT, and VARIADIC arguments.

<argname>
The name of a function, procedure, or aggregate argument. Note that COMMENT does not actually pay any attention to argument names, since only the argument data types are needed to determine the function's identity.

<argtype>
The data type of a function, procedure, or aggregate argument.

<large_object_oid>
The OID of the large object.

<left_type>; <right_type>
The data type(s) of the operator's arguments (optionally schema-qualified). Write NONE for the missing argument of a prefix operator.

PROCEDURAL
This is a noise word.

<type_name>
The name of the data type of the transform.

<lang_name>
The name of the language of the transform.

<string_literal>
The new comment contents, written as a string literal. An empty string ('') removes the comment.

NULL
Write NULL to remove the comment.

Notes

There is presently no security mechanism for viewing comments: any user connected to a database can see all the comments for objects in that database. For shared objects such as databases, roles, and tablespaces, comments are stored globally so any user connected to any database in the cluster can see all the comments for shared objects. Therefore, don't put security-critical information in comments.

Examples

Attach a comment to the table mytable:

COMMENT ON TABLE mytable IS 'This is my table.';

Remove it again:

COMMENT ON TABLE mytable IS NULL;

Some more examples:

COMMENT ON ACCESS METHOD gin IS 'GIN index access method';
COMMENT ON AGGREGATE my_aggregate (double precision) IS 'Computes sample variance';
COMMENT ON CAST (text AS int4) IS 'Allow casts from text to int4';
COMMENT ON COLLATION "fr_CA" IS 'Canadian French';
COMMENT ON COLUMN my_table.my_column IS 'Employee ID number';
COMMENT ON CONVERSION my_conv IS 'Conversion to UTF8';
COMMENT ON CONSTRAINT bar_col_cons ON bar IS 'Constrains column col';
COMMENT ON CONSTRAINT dom_col_constr ON DOMAIN dom IS 'Constrains col of domain';
COMMENT ON DATABASE my_database IS 'Development Database';
COMMENT ON DOMAIN my_domain IS 'Email Address Domain';
COMMENT ON EVENT TRIGGER abort_ddl IS 'Aborts all DDL commands';
COMMENT ON EXTENSION hstore IS 'implements the hstore data type';
COMMENT ON FOREIGN DATA WRAPPER mywrapper IS 'my foreign data wrapper';
COMMENT ON FOREIGN TABLE my_foreign_table IS 'Employee Information in other database';
COMMENT ON FUNCTION my_function (timestamp) IS 'Returns Roman Numeral';
COMMENT ON INDEX my_index IS 'Enforces uniqueness on employee ID';
COMMENT ON LANGUAGE plpython IS 'Python support for stored procedures';
COMMENT ON LARGE OBJECT 346344 IS 'Planning document';
COMMENT ON MATERIALIZED VIEW my_matview IS 'Summary of order history';
COMMENT ON OPERATOR ^ (text, text) IS 'Performs intersection of two texts';
COMMENT ON OPERATOR - (NONE, integer) IS 'Unary minus';
COMMENT ON OPERATOR CLASS int4ops USING btree IS '4 byte integer operators for btrees';
COMMENT ON OPERATOR FAMILY integer_ops USING btree IS 'all integer operators for btrees';
COMMENT ON POLICY my_policy ON mytable IS 'Filter rows by users';
COMMENT ON PROCEDURE my_proc (integer, integer) IS 'Runs a report';
COMMENT ON PUBLICATION alltables IS 'Publishes all operations on all tables';
COMMENT ON ROLE my_role IS 'Administration group for finance tables';
COMMENT ON ROUTINE my_routine (integer, integer) IS 'Runs a routine (which is a function or procedure)';
COMMENT ON RULE my_rule ON my_table IS 'Logs updates of employee records';
COMMENT ON SCHEMA my_schema IS 'Departmental data';
COMMENT ON SEQUENCE my_sequence IS 'Used to generate primary keys';
COMMENT ON SERVER myserver IS 'my foreign server';
COMMENT ON STATISTICS my_statistics IS 'Improves planner row estimations';
COMMENT ON SUBSCRIPTION alltables IS 'Subscription for all operations on all tables';
COMMENT ON TABLE my_schema.my_table IS 'Employee Information';
COMMENT ON TABLESPACE my_tablespace IS 'Tablespace for indexes';
COMMENT ON TEXT SEARCH CONFIGURATION my_config IS 'Special word filtering';
COMMENT ON TEXT SEARCH DICTIONARY swedish IS 'Snowball stemmer for Swedish language';
COMMENT ON TEXT SEARCH PARSER my_parser IS 'Splits text into words';
COMMENT ON TEXT SEARCH TEMPLATE snowball IS 'Snowball stemmer';
COMMENT ON TRANSFORM FOR hstore LANGUAGE plpython3u IS 'Transform between hstore and Python dict';
COMMENT ON TRIGGER my_trigger ON my_table IS 'Used for RI';
COMMENT ON TYPE complex IS 'Complex number data type';
COMMENT ON VIEW my_view IS 'View of departmental costs';
COMMENT ON VIEW my_view IS NULL;

Compatibility

There is no COMMENT command in the SQL standard.

commit

COMMIT

COMMIT

7

SQL - Language Statements

COMMIT

commit the current transaction

COMMIT [ WORK | TRANSACTION ] [ AND [ NO ] CHAIN ]

Description

COMMIT commits the current transaction. All changes made by the transaction become visible to others and are guaranteed to be durable if a crash occurs.

If the transaction is in an aborted state then no changes will be made and the effect of the COMMIT will be identical to that of ROLLBACK, including the command tag output.

In either case, if the AND CHAIN parameter is specified then a new, identically configured, transaction is started.

For more information regarding transactions see ???.

Parameters

chained transactions

WORK; TRANSACTION
Optional key words. They have no effect.

AND CHAIN
If AND CHAIN is specified, a new transaction is immediately started with the same transaction characteristics (see ???) as the just finished one. Otherwise, no new transaction is started.

Outputs

On successful completion of a non-aborted transaction, a COMMIT command returns a command tag of the form

COMMIT

However, in an aborted transaction, a COMMIT command returns a command tag of the form

ROLLBACK

Notes

Use ??? to abort a transaction.

Issuing COMMIT when not inside a transaction does no harm, but it will provoke a warning message. COMMIT AND CHAIN when not inside a transaction is an error.

Examples

To commit the current transaction and make all changes permanent:

COMMIT;

Compatibility

The command COMMIT conforms to the SQL standard, except that no exception condition is raised in the case where the transaction was already aborted.

The form COMMIT TRANSACTION is a PostgreSQL extension.

See Also

commit_prepared

COMMIT PREPARED

COMMIT PREPARED

7

SQL - Language Statements

COMMIT PREPARED

commit a transaction that was earlier prepared for two-phase commit

COMMIT PREPARED

transaction_id

Description

COMMIT PREPARED commits a transaction that is in prepared state.

Parameters

<transaction_id>
The transaction identifier of the transaction that is to be committed.

Notes

To commit a prepared transaction, you must be either the same user that executed the transaction originally, or a superuser. But you do not have to be in the same session that executed the transaction.

This command cannot be executed inside a transaction block. The prepared transaction is committed immediately.

All currently available prepared transactions are listed in the pg_prepared_xacts system view.

Examples

Commit the transaction identified by the transaction identifier foobar:

COMMIT PREPARED 'foobar';

Compatibility

COMMIT PREPARED is a PostgreSQL extension. It is intended for use by external transaction management systems, some of which are covered by standards (such as X/Open XA), but the SQL side of those systems is not standardized.

See Also

copy

COPY

COPY

7

SQL - Language Statements

COPY

copy data between a file and a table

COPY

table_name

[ (

column_name

[, ...] ) ] FROM { '

filename

' | PROGRAM '

command

' | STDIN } [ [ WITH ] (

option

[, ...] ) ] [ WHERE

condition

] COPY {

table_name

[ (

column_name

[, ...] ) ] | (

query

) } TO { '

filename

' | PROGRAM '

command

' | STDOUT } [ [ WITH ] (

option

[, ...] ) ]

where

option

can be one of:

FORMAT

format_name

FREEZE [

boolean

] DELIMITER '

delimiter_character

' NULL '

null_string

' DEFAULT '

default_string

' HEADER [

boolean

|

integer

| MATCH ] QUOTE '

quote_character

' ESCAPE '

escape_character

' FORCE_ARRAY [

boolean

] FORCE_QUOTE { (

column_name

[, ...] ) | * } FORCE_NOT_NULL { (

column_name

[, ...] ) | * } FORCE_NULL { (

column_name

[, ...] ) | * } ON_ERROR

error_action

REJECT_LIMIT

maxerror

ENCODING '

encoding_name

' LOG_VERBOSITY

verbosity

Description

COPY moves data between PostgreSQL tables and standard file-system files. COPY TO copies the contents of a table to a file, while COPY FROM copies data from a file to a table (appending the data to whatever is in the table already). COPY TO can also copy the results of a SELECT query.

If a column list is specified, COPY TO copies only the data in the specified columns to the file. For COPY FROM, each field in the file is inserted, in order, into the specified column. Table columns not specified in the COPY FROM column list will receive their default values.

COPY with a file name instructs the PostgreSQL server to directly read from or write to a file. The file must be accessible by the PostgreSQL user (the user ID the server runs as) and the name must be specified from the viewpoint of the server. When PROGRAM is specified, the server executes the given command and reads from the standard output of the program, or writes to the standard input of the program. The command must be specified from the viewpoint of the server, and be executable by the PostgreSQL user. When STDIN or STDOUT is specified, data is transmitted via the connection between the client and the server.

Each backend running COPY will report its progress in the pg_stat_progress_copy view. See ??? for details.

By default, COPY will fail if it encounters an error during processing. For use cases where a best-effort attempt at loading the entire file is desired, the ON_ERROR clause can be used to specify some other behavior.

Parameters

<table_name>
The name (optionally schema-qualified) of an existing table.

<column_name>
An optional list of columns to be copied. If no column list is specified, all columns of the table except generated columns will be copied.

<query>
A SELECT, VALUES, INSERT, UPDATE, DELETE, or MERGE command whose results are to be copied. Note that parentheses are required around the query.

For INSERT, UPDATE, DELETE, and MERGE queries a RETURNING clause must be provided, and the target relation must not have a conditional rule, nor an ALSO rule, nor an INSTEAD rule that expands to multiple statements.

<filename>
The path name of the input or output file. An input file name can be an absolute or relative path, but an output file name must be an absolute path. Windows users might need to use an E'' string and double any backslashes used in the path name.

PROGRAM
A command to execute. In COPY FROM, the input is read from standard output of the command, and in COPY TO, the output is written to the standard input of the command.

Note that the command is invoked by the shell, so if you need to pass any arguments that come from an untrusted source, you must be careful to strip or escape any special characters that might have a special meaning for the shell. For security reasons, it is best to use a fixed command string, or at least avoid including any user input in it.

STDIN
Specifies that input comes from the client application.

STDOUT
Specifies that output goes to the client application.

<boolean>
Specifies whether the selected option should be turned on or off. You can write TRUE, ON, or 1 to enable the option, and FALSE, OFF, or 0 to disable it. The <boolean> value can also be omitted, in which case TRUE is assumed.

<integer>
Specifies a non-negative integer value passed to the selected option.

FORMAT
Selects the data format to be read or written: text, csv (Comma Separated Values), json (JavaScript Object Notation), or binary. The default is text. See File Formats below for details.

The json option is allowed only in COPY TO.

[!NOTE] In JSON format, SQL NULL values are output as JSON null. However, a JSON or JSONB column whose value is the JSON literal null is also output as null, making the two cases indistinguishable in the COPY output. For example:

COPY (SELECT j FROM (VALUES ('null'::json), (NULL::json)) v(j))
     TO stdout (FORMAT JSON);
{"j":null}
{"j":null}

FREEZE
Requests copying the data with rows already frozen, just as they would be after running the VACUUM FREEZE command. This is intended as a performance option for initial data loading. Rows will be frozen only if the table being loaded has been created or truncated in the current subtransaction, there are no cursors open and there are no older snapshots held by this transaction. It is currently not possible to perform a COPY FREEZE on a partitioned table or foreign table. This option is only allowed in COPY FROM.

Note that all other sessions will immediately be able to see the data once it has been successfully loaded. This violates the normal rules of MVCC visibility and users should be aware of the potential problems this might cause.

DELIMITER
Specifies the character that separates columns within each row (line) of the file. The default is a tab character in text format, a comma in CSV format. This must be a single one-byte character. This option is not allowed when using binary or json format.

NULL
Specifies the string that represents a null value. The default is \N (backslash-N) in text format, and an unquoted empty string in CSV format. You might prefer an empty string even in text format for cases where you don't want to distinguish nulls from empty strings. This option is not allowed when using binary or json format.

[!NOTE] When using COPY FROM, any data item that matches this string will be stored as a null value, so you should make sure that you use the same string as you used with COPY TO.

DEFAULT
Specifies the string that represents a default value. Each time the string is found in the input file, the default value of the corresponding column will be used. This option is allowed only in COPY FROM, and only when not using binary or json format.

HEADER
On output, if this option is set to true (or an equivalent Boolean value), the first line of the output will contain the column names from the table. Integer values 0 and 1 are accepted as Boolean values, but other integers are not allowed for COPY TO commands.

On input, if this option is set to true (or an equivalent Boolean value), the first line of the input is discarded. If set to a non-negative integer, that number of lines are discarded. If set to MATCH, the first line is discarded, and it must contain column names that exactly match the table's columns, in both number and order; otherwise, an error is raised. The MATCH value is only valid for COPY FROM commands.

This option is not allowed when using binary or json format.

QUOTE
Specifies the quoting character to be used when a data value is quoted. The default is double-quote. This must be a single one-byte character. This option is allowed only when using CSV format.

ESCAPE
Specifies the character that should appear before a data character that matches the QUOTE value. The default is the same as the QUOTE value (so that the quoting character is doubled if it appears in the data). This must be a single one-byte character. This option is allowed only when using CSV format.

FORCE_ARRAY
Force output of square brackets as array decorations at the beginning and end of output, and commas between the rows. It is allowed only in COPY TO, and only when using json format. The default is false.

FORCE_QUOTE
Forces quoting to be used for all non-NULL values in each specified column. NULL output is never quoted. If * is specified, non-NULL values will be quoted in all columns. This option is allowed only in COPY TO, and only when using CSV format.

FORCE_NOT_NULL
Do not match the specified columns' values against the null string. In the default case where the null string is empty, this means that empty values will be read as zero-length strings rather than nulls, even when they are not quoted. If * is specified, the option will be applied to all columns. This option is allowed only in COPY FROM, and only when using CSV format.

FORCE_NULL
Match the specified columns' values against the null string, even if it has been quoted, and if a match is found set the value to NULL. In the default case where the null string is empty, this converts a quoted empty string into NULL. If * is specified, the option will be applied to all columns. This option is allowed only in COPY FROM, and only when using CSV format.

ON_ERROR
Specifies how to behave when encountering an error converting a column's input value into its data type. An <error_action> value of stop means fail the command, while ignore means discard the input row and continue with the next one, and set_null means replace the field containing the invalid input value with a null value and continue to the next field. The default is stop.

The ignore and set_null options are applicable only for COPY FROM when the FORMAT is text or csv.

If ON_ERROR is set to ignore or set_null, a NOTICE message is emitted at the end of the COPY FROM command containing the count of rows that were ignored or changed, if at least one row was affected.

When LOG_VERBOSITY option is set to verbose, for ignore option, a NOTICE message containing the line of the input file and the column name whose input conversion has failed is emitted for each discarded row; for set_null option, a NOTICE message containing the line of the input file and the column name where value was replaced with NULL for each input conversion failure. When it is set to silent, no message is emitted regarding input conversion failed rows.

REJECT_LIMIT
Specifies the maximum number of errors tolerated while converting a column's input value to its data type, when ON_ERROR is set to ignore. If the input causes more errors than the specified value, the COPY command fails, even with ON_ERROR set to ignore. This clause must be used with ON_ERROR=ignore and <maxerror> must be positive bigint. If not specified, ON_ERROR=ignore allows an unlimited number of errors, meaning COPY will skip all erroneous data.

ENCODING
Specifies that the file is encoded in the <encoding_name>. If this option is omitted, the current client encoding is used. See the Notes below for more details.

LOG_VERBOSITY
Specifies the amount of messages emitted by a COPY command: default, verbose, or silent. If verbose is specified, additional messages are emitted during processing. silent suppresses both verbose and default messages.

This is currently used in COPY FROM command when ON_ERROR option is set to ignore or set_null.

WHERE
The optional WHERE clause has the general form WHERE <condition> where <condition> is any expression that evaluates to a result of type boolean. Any row that does not satisfy this condition will not be inserted to the table. A row satisfies the condition if it returns true when the actual row values are substituted for any variable references.

Currently, subqueries, system columns, and generated columns are not allowed in WHERE expressions, and the evaluation does not see any changes made by the COPY itself (this matters when the expression contains calls to VOLATILE functions).

Outputs

On successful completion, a COPY command returns a command tag of the form

COPY count

The <count> is the number of rows copied.

[!NOTE] psql will print this command tag only if the command was not COPY ... TO STDOUT, or the equivalent psql meta-command \copy ... to stdout. This is to prevent confusing the command tag with the data that was just printed.

Notes

COPY TO can be used with plain tables, populated materialized views, and partitioned tables. For non-partitioned tables, COPY <table> copies the same rows as SELECT * FROM ONLY table. For partitioned tables, it copies the same rows as SELECT * FROM table. However it doesn't directly support other relation types, such as inheritance child tables, or views. To copy all rows from such relations, use COPY (SELECT * FROM table) TO.

COPY FROM can be used with plain, foreign, or partitioned tables or with views that have INSTEAD OF INSERT triggers.

You must have select privilege on the table whose values are read by COPY TO, and insert privilege on the table into which values are inserted by COPY FROM. It is sufficient to have column privileges on the column(s) listed in the command.

If row-level security is enabled for the table, the relevant SELECT policies will apply to COPY table TO statements. Currently, COPY FROM is not supported for tables with row-level security. Use equivalent INSERT statements instead.

Files named in a COPY command are read or written directly by the server, not by the client application. Therefore, they must reside on or be accessible to the database server machine, not the client. They must be accessible to and readable or writable by the PostgreSQL user (the user ID the server runs as), not the client. Similarly, the command specified with PROGRAM is executed directly by the server, not by the client application, must be executable by the PostgreSQL user. COPY naming a file or command is only allowed to database superusers or users who are granted one of the roles pg_read_server_files, pg_write_server_files, or pg_execute_server_program, since it allows reading or writing any file or running a program that the server has privileges to access.

Do not confuse COPY with the psql instruction \copy. \copy invokes COPY FROM STDIN or COPY TO STDOUT, and then fetches/stores the data in a file accessible to the psql client. Thus, file accessibility and access rights depend on the client rather than the server when \copy is used.

Executing a command with PROGRAM might be restricted by the operating system's access control mechanisms, such as SELinux.

COPY FROM will invoke any triggers and check constraints on the destination table. However, it will not invoke rules.

For identity columns, the COPY FROM command will always write the column values provided in the input data, like the INSERT option OVERRIDING SYSTEM VALUE.

COPY input and output is affected by DateStyle. To ensure portability to other PostgreSQL installations that might use non-default DateStyle settings, DateStyle should be set to ISO before using COPY TO. It is also a good idea to avoid dumping data with IntervalStyle set to sql_standard, because negative interval values might be misinterpreted by a server that has a different setting for IntervalStyle.

Input data is interpreted according to ENCODING option or the current client encoding, and output data is encoded in ENCODING or the current client encoding, even if the data does not pass through the client but is read from or written to a file directly by the server.

The COPY FROM command physically inserts input rows into the table as it progresses. If the command fails, these rows are left in a deleted state; these rows will not be visible, but still occupy disk space. This might amount to considerable wasted disk space if the failure happened well into a large copy operation. VACUUM should be used to recover the wasted space.

FORCE_NULL and FORCE_NOT_NULL can be used simultaneously on the same column. This results in converting quoted null strings to null values and unquoted null strings to empty strings.

File Formats

Text Format

When the text format is used, the data read or written is a text file with one line per table row. Columns in a row are separated by the delimiter character. The column values themselves are strings generated by the output function, or acceptable to the input function, of each attribute's data type. The specified null string is used in place of columns that are null. COPY FROM will raise an error if any line of the input file contains more or fewer columns than are expected.

End of data can be represented by a line containing just backslash-period (\.). An end-of-data marker is not necessary when reading from a file, since the end of file serves perfectly well; in that context this provision exists only for backward compatibility. However, psql uses \. to terminate a COPY FROM STDIN operation (that is, reading in-line COPY data in an SQL script). In that context the rule is needed to be able to end the operation before the end of the script.

Backslash characters (\) can be used in the COPY data to quote data characters that might otherwise be taken as row or column delimiters. In particular, the following characters must be preceded by a backslash if they appear as part of a column value: backslash itself, newline, carriage return, and the current delimiter character.

The specified null string is sent by COPY TO without adding any backslashes; conversely, COPY FROM matches the input against the null string before removing backslashes. Therefore, a null string such as \N cannot be confused with the actual data value \N (which would be represented as \\N).

The following special backslash sequences are recognized by COPY FROM:

Sequence Represents
\b Backspace (ASCII 8)
\f Form feed (ASCII 12)
\n Newline (ASCII 10)
\r Carriage return (ASCII 13)
\t Tab (ASCII 9)
\v Vertical tab (ASCII 11)
\<digits> Backslash followed by one to three octal digits specifies the byte with that numeric code
\x<digits> Backslash x followed by one or two hex digits specifies the byte with that numeric code

Presently, COPY TO will never emit an octal or hex-digits backslash sequence, but it does use the other sequences listed above for those control characters.

Any other backslashed character that is not mentioned in the above table will be taken to represent itself. However, beware of adding backslashes unnecessarily, since that might accidentally produce a string matching the end-of-data marker (\.) or the null string (\N by default). These strings will be recognized before any other backslash processing is done.

It is strongly recommended that applications generating COPY data convert data newlines and carriage returns to the \n and \r sequences respectively. At present it is possible to represent a data carriage return by a backslash and carriage return, and to represent a data newline by a backslash and newline. However, these representations might not be accepted in future releases. They are also highly vulnerable to corruption if the COPY file is transferred across different machines (for example, from Unix to Windows or vice versa).

All backslash sequences are interpreted after encoding conversion. The bytes specified with the octal and hex-digit backslash sequences must form valid characters in the database encoding.

COPY TO will terminate each row with a Unix-style newline (“\n”). Servers running on Microsoft Windows instead output carriage return/newline (“\r\n”), but only for COPY to a server file; for consistency across platforms, COPY TO STDOUT always sends “\n” regardless of server platform. COPY FROM can handle lines ending with newlines, carriage returns, or carriage return/newlines. To reduce the risk of error due to un-backslashed newlines or carriage returns that were meant as data, COPY FROM will complain if the line endings in the input are not all alike.

CSV Format

This format option is used for importing and exporting the Comma- Separated Value (CSV) file format used by many other programs, such as spreadsheets. Instead of the escaping rules used by PostgreSQL's standard text format, it produces and recognizes the common CSV escaping mechanism.

The values in each record are separated by the DELIMITER character. If the value contains the delimiter character, the QUOTE character, the NULL string, a carriage return, or line feed character, then the whole value is prefixed and suffixed by the QUOTE character, and any occurrence within the value of a QUOTE character or the ESCAPE character is preceded by the escape character. You can also use FORCE_QUOTE to force quotes when outputting non-NULL values in specific columns.

The CSV format has no standard way to distinguish a NULL value from an empty string. PostgreSQL's COPY handles this by quoting. A NULL is output as the NULL parameter string and is not quoted, while a non-NULL value matching the NULL parameter string is quoted. For example, with the default settings, a NULL is written as an unquoted empty string, while an empty string data value is written with double quotes (""). Reading values follows similar rules. You can use FORCE_NOT_NULL to prevent NULL input comparisons for specific columns. You can also use FORCE_NULL to convert quoted null string data values to NULL.

Because backslash is not a special character in the CSV format, the end-of-data marker used in text mode (\.) is not normally treated as special when reading CSV data. An exception is that psql will terminate a COPY FROM STDIN operation (that is, reading in-line COPY data in an SQL script) at a line containing only \., whether it is text or CSV mode.

[!NOTE] PostgreSQL versions before v18 always recognized unquoted \. as an end-of-data marker, even when reading from a separate file. For compatibility with older versions, COPY TO will quote \. when it's alone on a line, even though this is no longer necessary.

[!NOTE] In CSV format, all characters are significant. A quoted value surrounded by white space, or any characters other than DELIMITER, will include those characters. This can cause errors if you import data from a system that pads CSV lines with white space out to some fixed width. If such a situation arises you might need to preprocess the CSV file to remove the trailing white space, before importing the data into PostgreSQL.

[!NOTE] CSV format will both recognize and produce CSV files with quoted values containing embedded carriage returns and line feeds. Thus the files are not strictly one line per table row like text-format files.

[!NOTE] Many programs produce strange and occasionally perverse CSV files, so the file format is more a convention than a standard. Thus you might encounter some files that cannot be imported using this mechanism, and COPY might produce files that other programs cannot process.

Binary Format

The binary format option causes all data to be stored/read as binary format rather than as text. It is somewhat faster than the text and CSV formats, but a binary-format file is less portable across machine architectures and PostgreSQL versions. Also, the binary format is very data type specific; for example it will not work to output binary data from a smallint column and read it into an integer column, even though that would work fine in text format.

The binary file format consists of a file header, zero or more tuples containing the row data, and a file trailer. Headers and data are in network byte order.

[!NOTE] PostgreSQL releases before 7.4 used a different binary file format.

File Header

The file header consists of 15 bytes of fixed fields, followed by a variable-length header extension area. The fixed fields are:

Signature
11-byte sequence PGCOPY\n\377\r\n\0 — note that the zero byte is a required part of the signature. (The signature is designed to allow easy identification of files that have been munged by a non-8-bit-clean transfer. This signature will be changed by end-of-line-translation filters, dropped zero bytes, dropped high bits, or parity changes.)

Flags field
32-bit integer bit mask to denote important aspects of the file format. Bits are numbered from 0 (LSB) to 31 (MSB). Note that this field is stored in network byte order (most significant byte first), as are all the integer fields used in the file format. Bits 16–31 are reserved to denote critical file format issues; a reader should abort if it finds an unexpected bit set in this range. Bits 0–15 are reserved to signal backwards-compatible format issues; a reader should simply ignore any unexpected bits set in this range. Currently only one flag bit is defined, and the rest must be zero:

Bit 16
If 1, OIDs are included in the data; if 0, not. Oid system columns are not supported in PostgreSQL anymore, but the format still contains the indicator.

Header extension area length
32-bit integer, length in bytes of remainder of header, not including self. Currently, this is zero, and the first tuple follows immediately. Future changes to the format might allow additional data to be present in the header. A reader should silently skip over any header extension data it does not know what to do with.

The header extension area is envisioned to contain a sequence of self-identifying chunks. The flags field is not intended to tell readers what is in the extension area. Specific design of header extension contents is left for a later release.

This design allows for both backwards-compatible header additions (add header extension chunks, or set low-order flag bits) and non-backwards-compatible changes (set high-order flag bits to signal such changes, and add supporting data to the extension area if needed).

Tuples

Each tuple begins with a 16-bit integer count of the number of fields in the tuple. (Presently, all tuples in a table will have the same count, but that might not always be true.) Then, repeated for each field in the tuple, there is a 32-bit length word followed by that many bytes of field data. (The length word does not include itself, and can be zero.) As a special case, -1 indicates a NULL field value. No value bytes follow in the NULL case.

There is no alignment padding or any other extra data between fields.

Presently, all data values in a binary-format file are assumed to be in binary format (format code one). It is anticipated that a future extension might add a header field that allows per-column format codes to be specified.

To determine the appropriate binary format for the actual tuple data you should consult the PostgreSQL source, in particular the *send and *recv functions for each column's data type (typically these functions are found in the src/backend/utils/adt/ directory of the source distribution).

If OIDs are included in the file, the OID field immediately follows the field-count word. It is a normal field except that it's not included in the field-count. Note that oid system columns are not supported in current versions of PostgreSQL.

File Trailer

The file trailer consists of a 16-bit integer word containing -1. This is easily distinguished from a tuple's field-count word.

A reader should report an error if a field-count word is neither -1 nor the expected number of columns. This provides an extra check against somehow getting out of sync with the data.

Examples

The following example copies a table to the client using the vertical bar (|) as the field delimiter:

COPY country TO STDOUT (DELIMITER '|');

When the FORCE_ARRAY option is enabled, the entire output is wrapped in a single JSON array with rows separated by commas:

COPY (SELECT * FROM (VALUES(1),(2)) val(id)) TO STDOUT  (FORMAT JSON, FORCE_ARRAY);

The output is as follows:

[
 {"id":1}
,{"id":2}
]

To copy data from a file into the country table:

COPY country FROM '/usr1/proj/bray/sql/country_data';

To copy into a file just the countries whose names start with 'A':

COPY (SELECT * FROM country WHERE country_name LIKE 'A%') TO '/usr1/proj/bray/sql/a_list_countries.copy';

To copy into a compressed file, you can pipe the output through an external compression program:

COPY country TO PROGRAM 'gzip > /usr1/proj/bray/sql/country_data.gz';

Here is a sample of data suitable for copying into a table from STDIN:

AF      AFGHANISTAN
AL      ALBANIA
DZ      ALGERIA
ZM      ZAMBIA
ZW      ZIMBABWE

Note that the white space on each line is actually a tab character.

The following is the same data, output in binary format. The data is shown after filtering through the Unix utility od -c. The table has three columns; the first has type char(2), the second has type text, and the third has type integer. All the rows have a null value in the third column.

0000000   P   G   C   O   P   Y  \n 377  \r  \n  \0  \0  \0  \0  \0  \0
0000020  \0  \0  \0  \0 003  \0  \0  \0 002   A   F  \0  \0  \0 013   A
0000040   F   G   H   A   N   I   S   T   A   N 377 377 377 377  \0 003
0000060  \0  \0  \0 002   A   L  \0  \0  \0 007   A   L   B   A   N   I
0000100   A 377 377 377 377  \0 003  \0  \0  \0 002   D   Z  \0  \0  \0
0000120 007   A   L   G   E   R   I   A 377 377 377 377  \0 003  \0  \0
0000140  \0 002   Z   M  \0  \0  \0 006   Z   A   M   B   I   A 377 377
0000160 377 377  \0 003  \0  \0  \0 002   Z   W  \0  \0  \0  \b   Z   I
0000200   M   B   A   B   W   E 377 377 377 377 377 377

Compatibility

There is no COPY statement in the SQL standard.

The following syntax was used before PostgreSQL version 9.0 and is still supported: COPY <table_name> [ ( <column_name> [, ...] ) ] FROM { '<filename>' | STDIN } [ [ WITH ] [ BINARY ] [ DELIMITER [ AS ] '<delimiter_character>' ] [ NULL [ AS ] '<null_string>' ] [ CSV [ HEADER ] [ QUOTE [ AS ] '<quote_character>' ] [ ESCAPE [ AS ] '<escape_character>' ] [ FORCE NOT NULL <column_name> [, ...] ] ] ] COPY { <table_name> [ ( <column_name> [, ...] ) ] | ( <query> ) } TO { '<filename>' | STDOUT } [ [ WITH ] [ BINARY ] [ DELIMITER [ AS ] '<delimiter_character>' ] [ NULL [ AS ] '<null_string>' ] [ CSV [ HEADER ] [ QUOTE [ AS ] '<quote_character>' ] [ ESCAPE [ AS ] '<escape_character>' ] [ FORCE QUOTE { <column_name> [, ...] | * } ] ] ] Note that in this syntax, BINARY and CSV are treated as independent keywords, not as arguments of a FORMAT option.

The following syntax was used before PostgreSQL version 7.3 and is still supported: COPY [ BINARY ] <table_name> FROM { '<filename>' | STDIN } [ [USING] DELIMITERS '<delimiter_character>' ] [ WITH NULL AS '<null_string>' ] COPY [ BINARY ] <table_name> TO { '<filename>' | STDOUT } [ [USING] DELIMITERS '<delimiter_character>' ] [ WITH NULL AS '<null_string>' ]

See Also

create_access_method

CREATE ACCESS METHOD

CREATE ACCESS METHOD

7

SQL - Language Statements

CREATE ACCESS METHOD

define a new access method

CREATE ACCESS METHOD

name

TYPE

access_method_type

HANDLER

handler_function

Description

CREATE ACCESS METHOD creates a new access method.

The access method name must be unique within the database.

Only superusers can define new access methods.

Parameters

<name>
The name of the access method to be created.

<access_method_type>
This clause specifies the type of access method to define. Only TABLE and INDEX are supported at present.

<handler_function>
<handler_function> is the name (possibly schema-qualified) of a previously registered function that represents the access method. The handler function must be declared to take a single argument of type internal, and its return type depends on the type of access method; for TABLE access methods, it must be table_am_handler and for INDEX access methods, it must be index_am_handler. The C-level API that the handler function must implement varies depending on the type of access method. The table access method API is described in ??? and the index access method API is described in ???.

Examples

Create an index access method heptree with handler function heptree_handler:

CREATE ACCESS METHOD heptree TYPE INDEX HANDLER heptree_handler;

Compatibility

CREATE ACCESS METHOD is a PostgreSQL extension.

See Also

create_aggregate

CREATE AGGREGATE

CREATE AGGREGATE

7

SQL - Language Statements

CREATE AGGREGATE

define a new aggregate function

CREATE [ OR REPLACE ] AGGREGATE

name

( [

argmode

] [

argname

]

arg_data_type

[ , ... ] ) ( SFUNC =

sfunc

, STYPE =

state_data_type

[ , SSPACE =

state_data_size

] [ , FINALFUNC =

ffunc

] [ , FINALFUNC_EXTRA ] [ , FINALFUNC_MODIFY = { READ_ONLY | SHAREABLE | READ_WRITE } ] [ , COMBINEFUNC =

combinefunc

] [ , SERIALFUNC =

serialfunc

] [ , DESERIALFUNC =

deserialfunc

] [ , INITCOND =

initial_condition

] [ , MSFUNC =

msfunc

] [ , MINVFUNC =

minvfunc

] [ , MSTYPE =

mstate_data_type

] [ , MSSPACE =

mstate_data_size

] [ , MFINALFUNC =

mffunc

] [ , MFINALFUNC_EXTRA ] [ , MFINALFUNC_MODIFY = { READ_ONLY | SHAREABLE | READ_WRITE } ] [ , MINITCOND =

minitial_condition

] [ , SORTOP =

sort_operator

] [ , SUPPORT =

support_function

] [ , PARALLEL = { SAFE | RESTRICTED | UNSAFE } ] ) CREATE [ OR REPLACE ] AGGREGATE

name

( [ [

argmode

] [

argname

]

arg_data_type

[ , ... ] ] ORDER BY [

argmode

] [

argname

]

arg_data_type

[ , ... ] ) ( SFUNC =

sfunc

, STYPE =

state_data_type

[ , SSPACE =

state_data_size

] [ , FINALFUNC =

ffunc

] [ , FINALFUNC_EXTRA ] [ , FINALFUNC_MODIFY = { READ_ONLY | SHAREABLE | READ_WRITE } ] [ , INITCOND =

initial_condition

] [ , SUPPORT =

support_function

] [ , PARALLEL = { SAFE | RESTRICTED | UNSAFE } ] [ , HYPOTHETICAL ] )

or the old syntax

CREATE [ OR REPLACE ] AGGREGATE

name

( BASETYPE =

base_type

, SFUNC =

sfunc

, STYPE =

state_data_type

[ , SSPACE =

state_data_size

] [ , FINALFUNC =

ffunc

] [ , FINALFUNC_EXTRA ] [ , FINALFUNC_MODIFY = { READ_ONLY | SHAREABLE | READ_WRITE } ] [ , COMBINEFUNC =

combinefunc

] [ , SERIALFUNC =

serialfunc

] [ , DESERIALFUNC =

deserialfunc

] [ , INITCOND =

initial_condition

] [ , MSFUNC =

msfunc

] [ , MINVFUNC =

minvfunc

] [ , MSTYPE =

mstate_data_type

] [ , MSSPACE =

mstate_data_size

] [ , MFINALFUNC =

mffunc

] [ , MFINALFUNC_EXTRA ] [ , MFINALFUNC_MODIFY = { READ_ONLY | SHAREABLE | READ_WRITE } ] [ , MINITCOND =

minitial_condition

] [ , SORTOP =

sort_operator

] [ , SUPPORT =

support_function

] )

Description

CREATE AGGREGATE defines a new aggregate function. CREATE OR REPLACE AGGREGATE will either define a new aggregate function or replace an existing definition. Some basic and commonly-used aggregate functions are included with the distribution; they are documented in ???. If one defines new types or needs an aggregate function not already provided, then CREATE AGGREGATE can be used to provide the desired features.

When replacing an existing definition, the argument types, result type, and number of direct arguments may not be changed. Also, the new definition must be of the same kind (ordinary aggregate, ordered-set aggregate, or hypothetical-set aggregate) as the old one.

If a schema name is given (for example, CREATE AGGREGATE myschema.myagg ...) then the aggregate function is created in the specified schema. Otherwise it is created in the current schema.

An aggregate function is identified by its name and input data type(s). Two aggregates in the same schema can have the same name if they operate on different input types. The name and input data type(s) of an aggregate must also be distinct from the name and input data type(s) of every ordinary function in the same schema. This behavior is identical to overloading of ordinary function names (see ???).

A simple aggregate function is made from one or two ordinary functions: a state transition function <sfunc>, and an optional final calculation function <ffunc>. These are used as follows:

sfunc( internal-state, next-data-values ) ---> next-internal-state
ffunc( internal-state ) ---> aggregate-value

PostgreSQL creates a temporary variable of data type <stype> to hold the current internal state of the aggregate. At each input row, the aggregate argument value(s) are calculated and the state transition function is invoked with the current state value and the new argument value(s) to calculate a new internal state value. After all the rows have been processed, the final function is invoked once to calculate the aggregate's return value. If there is no final function then the ending state value is returned as-is.

An aggregate function can provide an initial condition, that is, an initial value for the internal state value. This is specified and stored in the database as a value of type text, but it must be a valid external representation of a constant of the state value data type. If it is not supplied then the state value starts out null.

If the state transition function is declared “strict”, then it cannot be called with null inputs. With such a transition function, aggregate execution behaves as follows. Rows with any null input values are ignored (the function is not called and the previous state value is retained). If the initial state value is null, then at the first row with all-nonnull input values, the first argument value replaces the state value, and the transition function is invoked at each subsequent row with all-nonnull input values. This is handy for implementing aggregates like max. Note that this behavior is only available when <state_data_type> is the same as the first <arg_data_type>. When these types are different, you must supply a nonnull initial condition or use a nonstrict transition function.

If the state transition function is not strict, then it will be called unconditionally at each input row, and must deal with null inputs and null state values for itself. This allows the aggregate author to have full control over the aggregate's handling of null values.

If the final function is declared “strict”, then it will not be called when the ending state value is null; instead a null result will be returned automatically. (Of course this is just the normal behavior of strict functions.) In any case the final function has the option of returning a null value. For example, the final function for avg returns null when it sees there were zero input rows.

Sometimes it is useful to declare the final function as taking not just the state value, but extra parameters corresponding to the aggregate's input values. The main reason for doing this is if the final function is polymorphic and the state value's data type would be inadequate to pin down the result type. These extra parameters are always passed as NULL (and so the final function must not be strict when the FINALFUNC_EXTRA option is used), but nonetheless they are valid parameters. The final function could for example make use of get_fn_expr_argtype to identify the actual argument type in the current call.

An aggregate can optionally support moving-aggregate mode, as described in ???. This requires specifying the MSFUNC, MINVFUNC, and MSTYPE parameters, and optionally the MSSPACE, MFINALFUNC, MFINALFUNC_EXTRA, MFINALFUNC_MODIFY, and MINITCOND parameters. Except for MINVFUNC, these parameters work like the corresponding simple-aggregate parameters without M; they define a separate implementation of the aggregate that includes an inverse transition function.

The syntax with ORDER BY in the parameter list creates a special type of aggregate called an ordered-set aggregate; or if HYPOTHETICAL is specified, then a hypothetical-set aggregate is created. These aggregates operate over groups of sorted values in order-dependent ways, so that specification of an input sort order is an essential part of a call. Also, they can have direct arguments, which are arguments that are evaluated only once per aggregation rather than once per input row. Hypothetical-set aggregates are a subclass of ordered-set aggregates in which some of the direct arguments are required to match, in number and data types, the aggregated argument columns. This allows the values of those direct arguments to be added to the collection of aggregate-input rows as an additional “hypothetical” row.

An aggregate can optionally support partial aggregation, as described in ???. This requires specifying the COMBINEFUNC parameter. If the <state_data_type> is internal, it's usually also appropriate to provide the SERIALFUNC and DESERIALFUNC parameters so that parallel aggregation is possible. Note that the aggregate must also be marked PARALLEL SAFE to enable parallel aggregation.

Aggregates that behave like MIN or MAX can sometimes be optimized by looking into an index instead of scanning every input row. If this aggregate can be so optimized, indicate it by specifying a sort operator. The basic requirement is that the aggregate must yield the first element in the sort ordering induced by the operator; in other words:

SELECT agg(col) FROM tab;

must be equivalent to:

SELECT col FROM tab ORDER BY col USING sortop LIMIT 1;

Further assumptions are that the aggregate ignores null inputs, and that it delivers a null result if and only if there were no non-null inputs. Ordinarily, a data type's < operator is the proper sort operator for MIN, and > is the proper sort operator for MAX. Note that the optimization will never actually take effect unless the specified operator is the “less than” or “greater than” strategy member of a B-tree index operator class.

To be able to create an aggregate function, you must have USAGE privilege on the argument types, the state type(s), and the return type, as well as EXECUTE privilege on the supporting functions.

Parameters

<name>
The name (optionally schema-qualified) of the aggregate function to create.

<argmode>
The mode of an argument: IN or VARIADIC. (Aggregate functions do not support OUT arguments.) If omitted, the default is IN. Only the last argument can be marked VARIADIC.

<argname>
The name of an argument. This is currently only useful for documentation purposes. If omitted, the argument has no name.

<arg_data_type>
An input data type on which this aggregate function operates. To create a zero-argument aggregate function, write * in place of the list of argument specifications. (An example of such an aggregate is count(*).)

<base_type>
In the old syntax for CREATE AGGREGATE, the input data type is specified by a basetype parameter rather than being written next to the aggregate name. Note that this syntax allows only one input parameter. To define a zero-argument aggregate function with this syntax, specify the basetype as "ANY" (not *). Ordered-set aggregates cannot be defined with the old syntax.

<sfunc>
The name of the state transition function to be called for each input row. For a normal <N>-argument aggregate function, the <sfunc> must take <N>+1 arguments, the first being of type <state_data_type> and the rest matching the declared input data type(s) of the aggregate. The function must return a value of type <state_data_type>. This function takes the current state value and the current input data value(s), and returns the next state value.

For ordered-set (including hypothetical-set) aggregates, the state transition function receives only the current state value and the aggregated arguments, not the direct arguments. Otherwise it is the same.

<state_data_type>
The data type for the aggregate's state value.

<state_data_size>
The approximate average size (in bytes) of the aggregate's state value. If this parameter is omitted or is zero, a default estimate is used based on the <state_data_type>. If set to a negative value, it indicates the state data can grow unboundedly in size, such as when the aggregate accumulates input rows (e.g., array_agg, string_agg). The planner uses this value to estimate the memory required for a grouped aggregate query and to avoid optimizations that may cause excessive memory usage.

<ffunc>
The name of the final function called to compute the aggregate's result after all input rows have been traversed. For a normal aggregate, this function must take a single argument of type <state_data_type>. The return data type of the aggregate is defined as the return type of this function. If <ffunc> is not specified, then the ending state value is used as the aggregate's result, and the return type is <state_data_type>.

For ordered-set (including hypothetical-set) aggregates, the final function receives not only the final state value, but also the values of all the direct arguments.

If FINALFUNC_EXTRA is specified, then in addition to the final state value and any direct arguments, the final function receives extra NULL values corresponding to the aggregate's regular (aggregated) arguments. This is mainly useful to allow correct resolution of the aggregate result type when a polymorphic aggregate is being defined.

FINALFUNC_MODIFY = { READ_ONLY | SHAREABLE | READ_WRITE }
This option specifies whether the final function is a pure function that does not modify its arguments. READ_ONLY indicates it does not; the other two values indicate that it may change the transition state value. See Notes below for more detail. The default is READ_ONLY, except for ordered-set aggregates, for which the default is READ_WRITE.

<combinefunc>
The <combinefunc> function may optionally be specified to allow the aggregate function to support partial aggregation. If provided, the <combinefunc> must combine two <state_data_type> values, each containing the result of aggregation over some subset of the input values, to produce a new <state_data_type> that represents the result of aggregating over both sets of inputs. This function can be thought of as an <sfunc>, where instead of acting upon an individual input row and adding it to the running aggregate state, it adds another aggregate state to the running state.

The <combinefunc> must be declared as taking two arguments of the <state_data_type> and returning a value of the <state_data_type>. Optionally this function may be “strict”. In this case the function will not be called when either of the input states are null; the other state will be taken as the correct result.

For aggregate functions whose <state_data_type> is internal, the <combinefunc> must not be strict. In this case the <combinefunc> must ensure that null states are handled correctly and that the state being returned is properly stored in the aggregate memory context.

<serialfunc>
An aggregate function whose <state_data_type> is internal can participate in parallel aggregation only if it has a <serialfunc> function, which must serialize the aggregate state into a bytea value for transmission to another process. This function must take a single argument of type internal and return type bytea. A corresponding <deserialfunc> is also required.

<deserialfunc>
Deserialize a previously serialized aggregate state back into <state_data_type>. This function must take two arguments of types bytea and internal, and produce a result of type internal. (Note: the second, internal argument is unused, but is required for type safety reasons.)

<initial_condition>
The initial setting for the state value. This must be a string constant in the form accepted for the data type <state_data_type>. If not specified, the state value starts out null.

<msfunc>
The name of the forward state transition function to be called for each input row in moving-aggregate mode. This is exactly like the regular transition function, except that its first argument and result are of type <mstate_data_type>, which might be different from <state_data_type>.

<minvfunc>
The name of the inverse state transition function to be used in moving-aggregate mode. This function has the same argument and result types as <msfunc>, but it is used to remove a value from the current aggregate state, rather than add a value to it. The inverse transition function must have the same strictness attribute as the forward state transition function.

<mstate_data_type>
The data type for the aggregate's state value, when using moving-aggregate mode.

<mstate_data_size>
The approximate average size (in bytes) of the aggregate's state value, when using moving-aggregate mode. This works the same as <state_data_size>, except that negative values are not used to indicate unbounded state size.

<mffunc>
The name of the final function called to compute the aggregate's result after all input rows have been traversed, when using moving-aggregate mode. This works the same as <ffunc>, except that its first argument's type is <mstate_data_type> and extra dummy arguments are specified by writing MFINALFUNC_EXTRA. The aggregate result type determined by <mffunc> or <mstate_data_type> must match that determined by the aggregate's regular implementation.

MFINALFUNC_MODIFY = { READ_ONLY | SHAREABLE | READ_WRITE }
This option is like FINALFUNC_MODIFY, but it describes the behavior of the moving-aggregate final function.

<minitial_condition>
The initial setting for the state value, when using moving-aggregate mode. This works the same as <initial_condition>.

<sort_operator>
The associated sort operator for a MIN- or MAX-like aggregate. This is just an operator name (possibly schema-qualified). The operator is assumed to have the same input data types as the aggregate (which must be a single-argument normal aggregate).

<support_function>
The name (optionally schema-qualified) of a planner support function to use for this aggregate. See ??? for details. You must be superuser to use this option.

PARALLEL = { SAFE | RESTRICTED | UNSAFE }
The meanings of PARALLEL SAFE, PARALLEL RESTRICTED, and PARALLEL UNSAFE are the same as in CREATE FUNCTION. An aggregate will not be considered for parallelization if it is marked PARALLEL UNSAFE (which is the default!) or PARALLEL RESTRICTED. Note that the parallel-safety markings of the aggregate's support functions are not consulted by the planner, only the marking of the aggregate itself.

HYPOTHETICAL
For ordered-set aggregates only, this flag specifies that the aggregate arguments are to be processed according to the requirements for hypothetical-set aggregates: that is, the last few direct arguments must match the data types of the aggregated (WITHIN GROUP) arguments. The HYPOTHETICAL flag has no effect on run-time behavior, only on parse-time resolution of the data types and collations of the aggregate's arguments.

The parameters of CREATE AGGREGATE can be written in any order, not just the order illustrated above.

Notes

In parameters that specify support function names, you can write a schema name if needed, for example SFUNC = public.sum. Do not write argument types there, however — the argument types of the support functions are determined from other parameters.

Ordinarily, PostgreSQL functions are expected to be true functions that do not modify their input values. However, an aggregate transition function, when used in the context of an aggregate, is allowed to cheat and modify its transition-state argument in place. This can provide substantial performance benefits compared to making a fresh copy of the transition state each time.

Likewise, while an aggregate final function is normally expected not to modify its input values, sometimes it is impractical to avoid modifying the transition-state argument. Such behavior must be declared using the FINALFUNC_MODIFY parameter. The READ_WRITE value indicates that the final function modifies the transition state in unspecified ways. This value prevents use of the aggregate as a window function, and it also prevents merging of transition states for aggregate calls that share the same input values and transition functions. The SHAREABLE value indicates that the transition function cannot be applied after the final function, but multiple final-function calls can be performed on the ending transition state value. This value prevents use of the aggregate as a window function, but it allows merging of transition states. (That is, the optimization of interest here is not applying the same final function repeatedly, but applying different final functions to the same ending transition state value. This is allowed as long as none of the final functions are marked READ_WRITE.)

If an aggregate supports moving-aggregate mode, it will improve calculation efficiency when the aggregate is used as a window function for a window with moving frame start (that is, a frame start mode other than UNBOUNDED PRECEDING). Conceptually, the forward transition function adds input values to the aggregate's state when they enter the window frame from the bottom, and the inverse transition function removes them again when they leave the frame at the top. So, when values are removed, they are always removed in the same order they were added. Whenever the inverse transition function is invoked, it will thus receive the earliest added but not yet removed argument value(s). The inverse transition function can assume that at least one row will remain in the current state after it removes the oldest row. (When this would not be the case, the window function mechanism simply starts a fresh aggregation, rather than using the inverse transition function.)

The forward transition function for moving-aggregate mode is not allowed to return NULL as the new state value. If the inverse transition function returns NULL, this is taken as an indication that the inverse function cannot reverse the state calculation for this particular input, and so the aggregate calculation will be redone from scratch for the current frame starting position. This convention allows moving-aggregate mode to be used in situations where there are some infrequent cases that are impractical to reverse out of the running state value.

If no moving-aggregate implementation is supplied, the aggregate can still be used with moving frames, but PostgreSQL will recompute the whole aggregation whenever the start of the frame moves. Note that whether or not the aggregate supports moving-aggregate mode, PostgreSQL can handle a moving frame end without recalculation; this is done by continuing to add new values to the aggregate's state. This is why use of an aggregate as a window function requires that the final function be read-only: it must not damage the aggregate's state value, so that the aggregation can be continued even after an aggregate result value has been obtained for one set of frame boundaries.

The syntax for ordered-set aggregates allows VARIADIC to be specified for both the last direct parameter and the last aggregated (WITHIN GROUP) parameter. However, the current implementation restricts use of VARIADIC in two ways. First, ordered-set aggregates can only use VARIADIC "any", not other variadic array types. Second, if the last direct parameter is VARIADIC "any", then there can be only one aggregated parameter and it must also be VARIADIC "any". (In the representation used in the system catalogs, these two parameters are merged into a single VARIADIC "any" item, since pg_proc cannot represent functions with more than one VARIADIC parameter.) If the aggregate is a hypothetical-set aggregate, the direct arguments that match the VARIADIC "any" parameter are the hypothetical ones; any preceding parameters represent additional direct arguments that are not constrained to match the aggregated arguments.

Currently, ordered-set aggregates do not need to support moving-aggregate mode, since they cannot be used as window functions.

Partial (including parallel) aggregation is currently not supported for ordered-set aggregates. Also, it will never be used for aggregate calls that include DISTINCT or ORDER BY clauses, since those semantics cannot be supported during partial aggregation.

Examples

See ???.

Compatibility

CREATE AGGREGATE is a PostgreSQL language extension. The SQL standard does not provide for user-defined aggregate functions.

See Also

create_cast

CREATE CAST

CREATE CAST

7

SQL - Language Statements

CREATE CAST

define a new cast

CREATE CAST (

source_type

AS

target_type

) WITH FUNCTION

function_name

[ (

argument_type

[, ...]) ] [ AS ASSIGNMENT | AS IMPLICIT ] CREATE CAST (

source_type

AS

target_type

) WITHOUT FUNCTION [ AS ASSIGNMENT | AS IMPLICIT ] CREATE CAST (

source_type

AS

target_type

) WITH INOUT [ AS ASSIGNMENT | AS IMPLICIT ]

Description

CREATE CAST defines a new cast. A cast specifies how to perform a conversion between two data types. For example,

SELECT CAST(42 AS float8);

converts the integer constant 42 to type float8 by invoking a previously specified function, in this case float8(int4). (If no suitable cast has been defined, the conversion fails.)

Two types can be binary coercible, which means that the conversion can be performed “for free” without invoking any function. This requires that corresponding values use the same internal representation. For instance, the types text and varchar are binary coercible both ways. Binary coercibility is not necessarily a symmetric relationship. For example, the cast from xml to text can be performed for free in the present implementation, but the reverse direction requires a function that performs at least a syntax check. (Two types that are binary coercible both ways are also referred to as binary compatible.)

You can define a cast as an I/O conversion cast by using the WITH INOUT syntax. An I/O conversion cast is performed by invoking the output function of the source data type, and passing the resulting string to the input function of the target data type. In many common cases, this feature avoids the need to write a separate cast function for conversion. An I/O conversion cast acts the same as a regular function-based cast; only the implementation is different.

By default, a cast can be invoked only by an explicit cast request, that is an explicit CAST(x AS typename) or <x>::<typename> construct.

If the cast is marked AS ASSIGNMENT then it can be invoked implicitly when assigning a value to a column of the target data type. For example, supposing that foo.f1 is a column of type text, then:

INSERT INTO foo (f1) VALUES (42);

will be allowed if the cast from type integer to type text is marked AS ASSIGNMENT, otherwise not. (We generally use the term assignment cast to describe this kind of cast.)

If the cast is marked AS IMPLICIT then it can be invoked implicitly in any context, whether assignment or internally in an expression. (We generally use the term implicit cast to describe this kind of cast.) For example, consider this query:

SELECT 2 + 4.0;

The parser initially marks the constants as being of type integer and numeric respectively. There is no integer + numeric operator in the system catalogs, but there is a numeric + numeric operator. The query will therefore succeed if a cast from integer to numeric is available and is marked AS IMPLICIT — which in fact it is. The parser will apply the implicit cast and resolve the query as if it had been written

SELECT CAST ( 2 AS numeric ) + 4.0;

Now, the catalogs also provide a cast from numeric to integer. If that cast were marked AS IMPLICIT — which it is not — then the parser would be faced with choosing between the above interpretation and the alternative of casting the numeric constant to integer and applying the integer + integer operator. Lacking any knowledge of which choice to prefer, it would give up and declare the query ambiguous. The fact that only one of the two casts is implicit is the way in which we teach the parser to prefer resolution of a mixed numeric-and-integer expression as numeric; there is no built-in knowledge about that.

It is wise to be conservative about marking casts as implicit. An overabundance of implicit casting paths can cause PostgreSQL to choose surprising interpretations of commands, or to be unable to resolve commands at all because there are multiple possible interpretations. A good rule of thumb is to make a cast implicitly invokable only for information-preserving transformations between types in the same general type category. For example, the cast from int2 to int4 can reasonably be implicit, but the cast from float8 to int4 should probably be assignment-only. Cross-type-category casts, such as text to int4, are best made explicit-only.

[!NOTE] Sometimes it is necessary for usability or standards-compliance reasons to provide multiple implicit casts among a set of types, resulting in ambiguity that cannot be avoided as above. The parser has a fallback heuristic based on type categories and preferred types that can help to provide desired behavior in such cases. See ??? for more information.

To be able to create a cast, you must own the source or the target data type and have USAGE privilege on the other type. To create a binary-coercible cast, you must be superuser. (This restriction is made because an erroneous binary-coercible cast conversion can easily crash the server.)

Parameters

<source_type>
The name of the source data type of the cast.

<target_type>
The name of the target data type of the cast.

function_name[(argument_type [, ...])]
The function used to perform the cast. The function name can be schema-qualified. If it is not, the function will be looked up in the schema search path. The function's result data type must match the target type of the cast. Its arguments are discussed below. If no argument list is specified, the function name must be unique in its schema.

WITHOUT FUNCTION
Indicates that the source type is binary-coercible to the target type, so no function is required to perform the cast.

WITH INOUT
Indicates that the cast is an I/O conversion cast, performed by invoking the output function of the source data type, and passing the resulting string to the input function of the target data type.

AS ASSIGNMENT
Indicates that the cast can be invoked implicitly in assignment contexts.

AS IMPLICIT
Indicates that the cast can be invoked implicitly in any context.

Cast implementation functions can have one to three arguments. The first argument type must be identical to or binary-coercible from the cast's source type. The second argument, if present, must be type integer; it receives the type modifier associated with the destination type, or -1 if there is none. The third argument, if present, must be type boolean; it receives true if the cast is an explicit cast, false otherwise. (Bizarrely, the SQL standard demands different behaviors for explicit and implicit casts in some cases. This argument is supplied for functions that must implement such casts. It is not recommended that you design your own data types so that this matters.)

The return type of a cast function must be identical to or binary-coercible to the cast's target type.

Ordinarily a cast must have different source and target data types. However, it is allowed to declare a cast with identical source and target types if it has a cast implementation function with more than one argument. This is used to represent type-specific length coercion functions in the system catalogs. The named function is used to coerce a value of the type to the type modifier value given by its second argument.

When a cast has different source and target types and a function that takes more than one argument, it supports converting from one type to another and applying a length coercion in a single step. When no such entry is available, coercion to a type that uses a type modifier involves two cast steps, one to convert between data types and a second to apply the modifier.

A cast to or from a domain type currently has no effect. Casting to or from a domain uses the casts associated with its underlying type.

Notes

Use DROP CAST to remove user-defined casts.

Remember that if you want to be able to convert types both ways you need to declare casts both ways explicitly.

cast

I/O conversion

It is normally not necessary to create casts between user-defined types and the standard string types (text, varchar, and char(n), as well as user-defined types that are defined to be in the string category). PostgreSQL provides automatic I/O conversion casts for that. The automatic casts to string types are treated as assignment casts, while the automatic casts from string types are explicit-only. You can override this behavior by declaring your own cast to replace an automatic cast, but usually the only reason to do so is if you want the conversion to be more easily invokable than the standard assignment-only or explicit-only setting. Another possible reason is that you want the conversion to behave differently from the type's I/O function; but that is sufficiently surprising that you should think twice about whether it's a good idea. (A small number of the built-in types do indeed have different behaviors for conversions, mostly because of requirements of the SQL standard.)

While not required, it is recommended that you continue to follow this old convention of naming cast implementation functions after the target data type. Many users are used to being able to cast data types using a function-style notation, that is <typename>(<x>). This notation is in fact nothing more nor less than a call of the cast implementation function; it is not specially treated as a cast. If your conversion functions are not named to support this convention then you will have surprised users. Since PostgreSQL allows overloading of the same function name with different argument types, there is no difficulty in having multiple conversion functions from different types that all use the target type's name.

[!NOTE] Actually the preceding paragraph is an oversimplification: there are two cases in which a function-call construct will be treated as a cast request without having matched it to an actual function. If a function call <name>(<x>) does not exactly match any existing function, but <name> is the name of a data type and pg_cast provides a binary-coercible cast to this type from the type of <x>, then the call will be construed as a binary-coercible cast. This exception is made so that binary-coercible casts can be invoked using functional syntax, even though they lack any function. Likewise, if there is no pg_cast entry but the cast would be to or from a string type, the call will be construed as an I/O conversion cast. This exception allows I/O conversion casts to be invoked using functional syntax.

[!NOTE] There is also an exception to the exception: I/O conversion casts from composite types to string types cannot be invoked using functional syntax, but must be written in explicit cast syntax (either CAST or :: notation). This exception was added because after the introduction of automatically-provided I/O conversion casts, it was found too easy to accidentally invoke such a cast when a function or column reference was intended.

Examples

To create an assignment cast from type bigint to type int4 using the function int4(bigint):

CREATE CAST (bigint AS int4) WITH FUNCTION int4(bigint) AS ASSIGNMENT;

(This cast is already predefined in the system.)

Compatibility

The CREATE CAST command conforms to the SQL standard, except that SQL does not make provisions for binary-coercible types or extra arguments to implementation functions. AS IMPLICIT is a PostgreSQL extension, too.

See Also

???, ???, ???

create_collation

CREATE COLLATION

CREATE COLLATION

7

SQL - Language Statements

CREATE COLLATION

define a new collation

CREATE COLLATION [ IF NOT EXISTS ]

name

( [ LOCALE =

locale

, ] [ LC_COLLATE =

lc_collate

, ] [ LC_CTYPE =

lc_ctype

, ] [ PROVIDER =

provider

, ] [ DETERMINISTIC =

boolean

, ] [ RULES =

rules

, ] [ VERSION =

version

] ) CREATE COLLATION [ IF NOT EXISTS ]

name

FROM

existing_collation

Description

CREATE COLLATION defines a new collation using the specified operating system locale settings, or by copying an existing collation.

To be able to create a collation, you must have CREATE privilege on the destination schema.

Parameters

IF NOT EXISTS
Do not throw an error if a collation with the same name already exists. A notice is issued in this case. Note that there is no guarantee that the existing collation is anything like the one that would have been created.

<name>
The name of the collation. The collation name can be schema-qualified. If it is not, the collation is defined in the current schema. The collation name must be unique within that schema. (The system catalogs can contain collations with the same name for other encodings, but these are ignored if the database encoding does not match.)

<locale>
The locale name for this collation. See ??? and ??? for details.

If <provider> is libc, this is a shortcut for setting LC_COLLATE and LC_CTYPE at once. If you specify <locale>, you cannot specify either of those parameters.

If <provider> is builtin, then <locale> must be specified and set to either C, C.UTF-8 or PG_UNICODE_FAST.

<lc_collate>
If <provider> is libc, use the specified operating system locale for the LC_COLLATE locale category.

<lc_ctype>
If <provider> is libc, use the specified operating system locale for the LC_CTYPE locale category.

<provider>
Specifies the provider to use for locale services associated with this collation. Possible values are builtin, icu (if the server was built with ICU support) or libc. libc is the default. See ??? for details.

DETERMINISTIC
Specifies whether the collation should use deterministic comparisons. The default is true. A deterministic comparison considers strings that are not byte-wise equal to be unequal even if they are considered logically equal by the comparison. PostgreSQL breaks ties using a byte-wise comparison. Comparison that is not deterministic can make the collation be, say, case- or accent-insensitive. For that, you need to choose an appropriate LOCALE setting and set the collation to not deterministic here.

Nondeterministic collations are only supported with the ICU provider.

<rules>
Specifies additional collation rules to customize the behavior of the collation. This is supported for ICU only. See ??? for details.

<version>
Specifies the version string to store with the collation. Normally, this should be omitted, which will cause the version to be computed from the actual version of the collation as provided by the operating system. This option is intended to be used by pg_upgrade for copying the version from an existing installation.

See also ??? for how to handle collation version mismatches.

<existing_collation>
The name of an existing collation to copy. The new collation will have the same properties as the existing one, but it will be an independent object.

Notes

CREATE COLLATION takes a SHARE ROW EXCLUSIVE lock, which is self-conflicting, on the pg_collation system catalog, so only one CREATE COLLATION command can run at a time.

Use DROP COLLATION to remove user-defined collations.

See ??? for more information on how to create collations.

When using the libc collation provider, the locale must be applicable to the current database encoding. See ??? for the precise rules.

Examples

To create a collation from the operating system locale fr_FR.utf8 (assuming the current database encoding is UTF8):

CREATE COLLATION french (locale = 'fr_FR.utf8');

To create a collation using the ICU provider using German phone book sort order:

CREATE COLLATION german_phonebook (provider = icu, locale = 'de-u-co-phonebk');

To create a collation using the ICU provider, based on the root ICU locale, with custom rules:

CREATE COLLATION custom (provider = icu, locale = 'und', rules = '&V << w <<< W');

See ??? for further details and examples on the rules syntax.

To create a collation from an existing collation:

CREATE COLLATION german FROM "de_DE";

This can be convenient to be able to use operating-system-independent collation names in applications.

Compatibility

There is a CREATE COLLATION statement in the SQL standard, but it is limited to copying an existing collation. The syntax to create a new collation is a PostgreSQL extension.

See Also

create_conversion

CREATE CONVERSION

CREATE CONVERSION

7

SQL - Language Statements

CREATE CONVERSION

define a new encoding conversion

CREATE [ DEFAULT ] CONVERSION

name

FOR

source_encoding

TO

dest_encoding

FROM

function_name

Description

CREATE CONVERSION defines a new conversion between two character set encodings.

Conversions that are marked DEFAULT can be used for automatic encoding conversion between client and server. To support that usage, two conversions, from encoding A to B and from encoding B to A, must be defined.

To be able to create a conversion, you must have EXECUTE privilege on the function and CREATE privilege on the destination schema.

Parameters

DEFAULT
The DEFAULT clause indicates that this conversion is the default for this particular source to destination encoding. There should be only one default encoding in a schema for the encoding pair.

<name>
The name of the conversion. The conversion name can be schema-qualified. If it is not, the conversion is defined in the current schema. The conversion name must be unique within a schema.

<source_encoding>
The source encoding name.

<dest_encoding>
The destination encoding name.

<function_name>
The function used to perform the conversion. The function name can be schema-qualified. If it is not, the function will be looked up in the path.

The function must have the following signature:

conv_proc(
    integer,  -- source encoding ID
    integer,  -- destination encoding ID
    cstring,  -- source string (null terminated C string)
    internal, -- destination (fill with a null terminated C string)
    integer,  -- source string length
    boolean   -- if true, don't throw an error if conversion fails
) RETURNS integer;

The return value is the number of source bytes that were successfully converted. If the last argument is false, the function must throw an error on invalid input, and the return value is always equal to the source string length.

Notes

Neither the source nor the destination encoding can be SQL_ASCII, as the server's behavior for cases involving the SQL_ASCII “encoding” is hard-wired.

Use DROP CONVERSION to remove user-defined conversions.

The privileges required to create a conversion might be changed in a future release.

Examples

To create a conversion from encoding UTF8 to LATIN1 using myfunc:

CREATE CONVERSION myconv FOR 'UTF8' TO 'LATIN1' FROM myfunc;

Compatibility

CREATE CONVERSION is a PostgreSQL extension. There is no CREATE CONVERSION statement in the SQL standard, but a CREATE TRANSLATION statement that is very similar in purpose and syntax.

See Also

create_database

CREATE DATABASE

CREATE DATABASE

7

SQL - Language Statements

CREATE DATABASE

create a new database

CREATE DATABASE

name

[ WITH ] [ OWNER [=]

user_name

] [ TEMPLATE [=]

template

] [ ENCODING [=]

encoding

] [ STRATEGY [=]

strategy

] [ LOCALE [=]

locale

] [ LC_COLLATE [=]

lc_collate

] [ LC_CTYPE [=]

lc_ctype

] [ BUILTIN_LOCALE [=]

builtin_locale

] [ ICU_LOCALE [=]

icu_locale

] [ ICU_RULES [=]

icu_rules

] [ LOCALE_PROVIDER [=]

locale_provider

] [ COLLATION_VERSION =

collation_version

] [ TABLESPACE [=]

tablespace_name

] [ ALLOW_CONNECTIONS [=]

allowconn

] [ CONNECTION LIMIT [=]

connlimit

] [ IS_TEMPLATE [=]

istemplate

] [ OID [=]

oid

]

Description

CREATE DATABASE creates a new PostgreSQL database.

To create a database, you must be a superuser or have the special CREATEDB privilege. See ???.

By default, the new database will be created by cloning the standard system database template1. A different template can be specified by writing TEMPLATE name. In particular, by writing TEMPLATE template0, you can create a pristine database (one where no user-defined objects exist and where the system objects have not been altered) containing only the standard objects predefined by your version of PostgreSQL. This is useful if you wish to avoid copying any installation-local objects that might have been added to template1.

Parameters

<name>
The name of a database to create.

<user_name>
The role name of the user who will own the new database, or DEFAULT to use the default (namely, the user executing the command). To create a database owned by another role, you must be able to SET ROLE to that role.

<template>
The name of the template from which to create the new database, or DEFAULT to use the default template (template1).

<encoding>
Character set encoding to use in the new database. Specify a string constant (e.g., 'SQL_ASCII'), or an integer encoding number, or DEFAULT to use the default encoding (namely, the encoding of the template database). The character sets supported by the PostgreSQL server are described in ???. See below for additional restrictions.

<strategy>
Strategy to be used in creating the new database. If the WAL_LOG strategy is used, the database will be copied block by block and each block will be separately written to the write-ahead log. This is the most efficient strategy in cases where the template database is small, and therefore it is the default. The older FILE_COPY strategy is also available. This strategy writes a small record to the write-ahead log for each tablespace used by the target database. Each such record represents copying an entire directory to a new location at the filesystem level. While this does reduce the write-ahead log volume substantially, especially if the template database is large, it also forces the system to perform a checkpoint both before and after the creation of the new database. In some situations, this may have a noticeable negative impact on overall system performance. The FILE_COPY strategy is affected by the ??? setting.

<locale>
Sets the default collation order and character classification in the new database. Collation affects the sort order applied to strings, e.g., in queries with ORDER BY, as well as the order used in indexes on text columns. Character classification affects the categorization of characters, e.g., lower, upper, and digit. Also sets the associated aspects of the operating system environment, LC_COLLATE and LC_CTYPE. The default is the same setting as the template database. See ??? and ??? for details.

Can be overridden by setting varlistentry_title, varlistentry_title, varlistentry_title, or varlistentry_title individually.

If varlistentry_title is builtin, then <locale> or <builtin_locale> must be specified and set to either C, C.UTF-8, or PG_UNICODE_FAST.

[!TIP] The other locale settings ???, ???, ???, and ??? are not fixed per database and are not set by this command. If you want to make them the default for a specific database, you can use ALTER DATABASE ... SET.

<lc_collate>
Sets LC_COLLATE in the database server's operating system environment. The default is the setting of varlistentry_title if specified, otherwise the same setting as the template database. See below for additional restrictions.

If varlistentry_title is libc, also sets the default collation order to use in the new database, overriding the setting varlistentry_title.

<lc_ctype>
Sets LC_CTYPE in the database server's operating system environment. The default is the setting of varlistentry_title if specified, otherwise the same setting as the template database. See below for additional restrictions.

If varlistentry_title is libc, also sets the default character classification to use in the new database, overriding the setting varlistentry_title.

<builtin_locale>
Specifies the builtin provider locale for the database default collation order and character classification, overriding the setting varlistentry_title. The locale provider must be builtin. The default is the setting of varlistentry_title if specified; otherwise the same setting as the template database.

The locales available for the builtin provider are C, C.UTF-8 and PG_UNICODE_FAST.

<icu_locale>
Specifies the ICU locale (see ???) for the database default collation order and character classification, overriding the setting varlistentry_title. The locale provider must be ICU. The default is the setting of varlistentry_title if specified; otherwise the same setting as the template database.

<icu_rules>
Specifies additional collation rules to customize the behavior of the default collation of this database. This is supported for ICU only. See ??? for details.

<locale_provider>
Specifies the provider to use for the default collation in this database. Possible values are builtin, icu (if the server was built with ICU support) or libc. By default, the provider is the same as that of the varlistentry_title. See ??? for details.

<collation_version>
Specifies the collation version string to store with the database. Normally, this should be omitted, which will cause the version to be computed from the actual version of the database collation as provided by the operating system. This option is intended to be used by pg_upgrade for copying the version from an existing installation.

See also ??? for how to handle database collation version mismatches.

<tablespace_name>
The name of the tablespace that will be associated with the new database, or DEFAULT to use the template database's tablespace. This tablespace will be the default tablespace used for objects created in this database. See ??? for more information.

<allowconn>
If false then no one can connect to this database. The default is true, allowing connections (except as restricted by other mechanisms, such as GRANT/REVOKE CONNECT).

<connlimit>
How many concurrent connections can be made to this database. -1 (the default) means no limit.

<istemplate>
If true, then this database can be cloned by any user with CREATEDB privileges; if false (the default), then only superusers or the owner of the database can clone it.

<oid>
The object identifier to be used for the new database. If this parameter is not specified, PostgreSQL will choose a suitable OID automatically. This parameter is primarily intended for internal use by pg_upgrade, and only pg_upgrade can specify a value less than 16384.

Optional parameters can be written in any order, not only the order illustrated above.

Notes

CREATE DATABASE cannot be executed inside a transaction block.

Errors along the line of “could not initialize database directory” are most likely related to insufficient permissions on the data directory, a full disk, or other file system problems.

Use DROP DATABASE to remove a database.

The program ??? is a wrapper program around this command, provided for convenience.

Database-level configuration parameters (set via ALTER DATABASE) and database-level permissions (set via GRANT) are not copied from the template database.

Although it is possible to copy a database other than template1 by specifying its name as the template, this is not (yet) intended as a general-purpose “COPY DATABASE” facility. The principal limitation is that no other sessions can be connected to the template database while it is being copied. CREATE DATABASE will fail if any other connection exists when it starts; otherwise, new connections to the template database are locked out until CREATE DATABASE completes. See ??? for more information.

The character set encoding specified for the new database must be compatible with the chosen locale settings (LC_COLLATE and LC_CTYPE). If the locale is C (or equivalently POSIX), then all encodings are allowed, but for other locale settings there is only one encoding that will work properly. (On Windows, however, UTF-8 encoding can be used with any locale.) CREATE DATABASE will allow superusers to specify SQL_ASCII encoding regardless of the locale settings, but this choice is deprecated and may result in misbehavior of character-string functions if data that is not encoding-compatible with the locale is stored in the database.

The encoding and locale settings must match those of the template database, except when template0 is used as template. This is because other databases might contain data that does not match the specified encoding, or might contain indexes whose sort ordering is affected by LC_COLLATE and LC_CTYPE. Copying such data would result in a database that is corrupt according to the new settings. template0, however, is known to not contain any data or indexes that would be affected.

There is currently no option to use a database locale with nondeterministic comparisons (see CREATE COLLATION for an explanation). If this is needed, then per-column collations would need to be used.

The CONNECTION LIMIT option is only enforced approximately; if two new sessions start at about the same time when just one connection “slot” remains for the database, it is possible that both will fail. Also, the limit is not enforced against superusers or background worker processes.

Examples

To create a new database:

CREATE DATABASE lusiadas;

To create a database sales owned by user salesapp with a default tablespace of salesspace:

CREATE DATABASE sales OWNER salesapp TABLESPACE salesspace;

To create a database music with a different locale:

CREATE DATABASE music
    LOCALE 'sv_SE.utf8'
    TEMPLATE template0;

In this example, the TEMPLATE template0 clause is required if the specified locale is different from the one in template1. (If it is not, then specifying the locale explicitly is redundant.)

To create a database music2 with a different locale and a different character set encoding:

CREATE DATABASE music2
    LOCALE 'sv_SE.iso885915'
    ENCODING LATIN9
    TEMPLATE template0;

The specified locale and encoding settings must match, or an error will be reported.

Note that locale names are specific to the operating system, so that the above commands might not work in the same way everywhere.

Compatibility

There is no CREATE DATABASE statement in the SQL standard. Databases are equivalent to catalogs, whose creation is implementation-defined.

See Also

create_domain

CREATE DOMAIN

CREATE DOMAIN

7

SQL - Language Statements

CREATE DOMAIN

define a new domain

CREATE DOMAIN

name

[ AS ]

data_type

[ COLLATE

collation

] [ DEFAULT

expression

] [

domain_constraint

[ ... ] ]

where

domain_constraint

is:

[ CONSTRAINT

constraint_name

] { NOT NULL | NULL | CHECK (

expression

) }

Description

CREATE DOMAIN creates a new domain. A domain is essentially a data type with optional constraints (restrictions on the allowed set of values). The user who defines a domain becomes its owner.

If a schema name is given (for example, CREATE DOMAIN myschema.mydomain ...) then the domain is created in the specified schema. Otherwise it is created in the current schema. The domain name must be unique among the types and domains existing in its schema.

Domains are useful for abstracting common constraints on fields into a single location for maintenance. For example, several tables might contain email address columns, all requiring the same CHECK constraint to verify the address syntax. Define a domain rather than setting up each table's constraint individually.

To be able to create a domain, you must have USAGE privilege on the underlying type.

Parameters

<name>
The name (optionally schema-qualified) of a domain to be created.

<data_type>
The underlying data type of the domain. This can include array specifiers.

<collation>
An optional collation for the domain. If no collation is specified, the domain has the same collation behavior as its underlying data type. The underlying type must be collatable if COLLATE is specified.

DEFAULT expression
The DEFAULT clause specifies a default value for columns of the domain data type. The value is any variable-free expression (but subqueries are not allowed). The data type of the default expression must match the data type of the domain. If no default value is specified, then the default value is the null value.

The default expression will be used in any insert operation that does not specify a value for the column. If a default value is defined for a particular column, it overrides any default associated with the domain. In turn, the domain default overrides any default value associated with the underlying data type.

CONSTRAINT constraint_name
An optional name for a constraint. If not specified, the system generates a name.

NOT NULL
Values of this domain are prevented from being null (but see notes below).

NULL
Values of this domain are allowed to be null. This is the default.

This clause is only intended for compatibility with nonstandard SQL databases. Its use is discouraged in new applications.

CHECK (expression)
CHECK clauses specify integrity constraints or tests which values of the domain must satisfy. Each constraint must be an expression producing a Boolean result. It should use the key word VALUE to refer to the value being tested. Expressions evaluating to TRUE or UNKNOWN succeed. If the expression produces a FALSE result, an error is reported and the value is not allowed to be converted to the domain type.

Currently, CHECK expressions cannot contain subqueries nor refer to variables other than VALUE.

When a domain has multiple CHECK constraints, they will be tested in alphabetical order by name. (PostgreSQL versions before 9.5 did not honor any particular firing order for CHECK constraints.)

Notes

Domain constraints, particularly NOT NULL, are checked when converting a value to the domain type. It is possible for a column that is nominally of the domain type to read as null despite there being such a constraint. For example, this can happen in an outer-join query, if the domain column is on the nullable side of the outer join. A more subtle example is

INSERT INTO tab (domcol) VALUES ((SELECT domcol FROM tab WHERE false));

The empty scalar sub-SELECT will produce a null value that is considered to be of the domain type, so no further constraint checking is applied to it, and the insertion will succeed.

It is very difficult to avoid such problems, because of SQL's general assumption that a null value is a valid value of every data type. Best practice therefore is to design a domain's constraints so that a null value is allowed, and then to apply column NOT NULL constraints to columns of the domain type as needed, rather than directly to the domain type.

PostgreSQL assumes that CHECK constraints' conditions are immutable, that is, they will always give the same result for the same input value. This assumption is what justifies examining CHECK constraints only when a value is first converted to be of a domain type, and not at other times. (This is essentially the same as the treatment of table CHECK constraints, as described in ???.)

An example of a common way to break this assumption is to reference a user-defined function in a CHECK expression, and then change the behavior of that function. PostgreSQL does not disallow that, but it will not notice if there are stored values of the domain type that now violate the CHECK constraint. That would cause a subsequent database dump and restore to fail. The recommended way to handle such a change is to drop the constraint (using ALTER DOMAIN), adjust the function definition, and re-add the constraint, thereby rechecking it against stored data.

It's also good practice to ensure that domain CHECK expressions will not throw errors.

Examples

This example creates the us_postal_code data type and then uses the type in a table definition. A regular expression test is used to verify that the value looks like a valid US postal code:

CREATE DOMAIN us_postal_code AS TEXT
CHECK(
   VALUE ~ '^\d{5}$'
OR VALUE ~ '^\d{5}-\d{4}$'
);

CREATE TABLE us_snail_addy (
  address_id SERIAL PRIMARY KEY,
  street1 TEXT NOT NULL,
  street2 TEXT,
  street3 TEXT,
  city TEXT NOT NULL,
  postal us_postal_code NOT NULL
);

Compatibility

The command CREATE DOMAIN conforms to the SQL standard.

The syntax NOT NULL in this command is a PostgreSQL extension. (A standard-conforming way to write the same for non-composite data types would be CHECK (VALUE IS NOT NULL). However, per refsect1_title, such constraints are best avoided in practice anyway.) The NULL “constraint” is a PostgreSQL extension (see also ???).

See Also

create_event_trigger

CREATE EVENT TRIGGER

CREATE EVENT TRIGGER

7

SQL - Language Statements

CREATE EVENT TRIGGER

define a new event trigger

CREATE EVENT TRIGGER

name

ON

event

[ WHEN

filter_variable

IN (

filter_value

[, ... ]) [ AND ... ] ] EXECUTE { FUNCTION | PROCEDURE }

function_name

()

Description

CREATE EVENT TRIGGER creates a new event trigger. Whenever the designated event occurs and the WHEN condition associated with the trigger, if any, is satisfied, the trigger function will be executed. For a general introduction to event triggers, see ???. The user who creates an event trigger becomes its owner.

Parameters

<name>
The name to give the new trigger. This name must be unique within the database.

<event>
The name of the event that triggers a call to the given function. See ??? for more information on event names.

<filter_variable>
The name of a variable used to filter events. This makes it possible to restrict the firing of the trigger to a subset of the cases in which it is supported. Currently the only supported <filter_variable> is TAG.

<filter_value>
A list of values for the associated <filter_variable> for which the trigger should fire. For TAG, this means a list of command tags (e.g., 'DROP FUNCTION').

<function_name>
A user-supplied function that is declared as taking no argument and returning type event_trigger.

In the syntax of CREATE EVENT TRIGGER, the keywords FUNCTION and PROCEDURE are equivalent, but the referenced function must in any case be a function, not a procedure. The use of the keyword PROCEDURE here is historical and deprecated.

Notes

Only superusers can create event triggers.

Event triggers are disabled in single-user mode (see ???) as well as when ??? is set to false. If an erroneous event trigger disables the database so much that you can't even drop the trigger, restart with ??? set to false to temporarily disable event triggers, or in single-user mode, and you'll be able to do that.

Examples

Forbid the execution of any DDL command:

CREATE OR REPLACE FUNCTION abort_any_command()
  RETURNS event_trigger
 LANGUAGE plpgsql
  AS $$
BEGIN
  RAISE EXCEPTION 'command % is disabled', tg_tag;
END;
$$;

CREATE EVENT TRIGGER abort_ddl ON ddl_command_start
   EXECUTE FUNCTION abort_any_command();

Compatibility

There is no CREATE EVENT TRIGGER statement in the SQL standard.

See Also

create_extension

CREATE EXTENSION

CREATE EXTENSION

7

SQL - Language Statements

CREATE EXTENSION

install an extension

CREATE EXTENSION [ IF NOT EXISTS ]

extension_name

[ WITH ] [ SCHEMA

schema_name

] [ VERSION

version

] [ CASCADE ]

Description

CREATE EXTENSION loads a new extension into the current database. There must not be an extension of the same name already loaded.

Loading an extension essentially amounts to running the extension's script file. The script will typically create new SQL objects such as functions, data types, operators and index support methods. CREATE EXTENSION additionally records the identities of all the created objects, so that they can be dropped again if DROP EXTENSION is issued.

The user who runs CREATE EXTENSION becomes the owner of the extension for purposes of later privilege checks, and normally also becomes the owner of any objects created by the extension's script.

Loading an extension ordinarily requires the same privileges that would be required to create its component objects. For many extensions this means superuser privileges are needed. However, if the extension is marked trusted in its control file, then it can be installed by any user who has CREATE privilege on the current database. In this case the extension object itself will be owned by the calling user, but the contained objects will be owned by the bootstrap superuser (unless the extension's script explicitly assigns them to the calling user). This configuration gives the calling user the right to drop the extension, but not to modify individual objects within it.

Parameters

IF NOT EXISTS
Do not throw an error if an extension with the same name already exists. A notice is issued in this case. Note that there is no guarantee that the existing extension is anything like the one that would have been created from the currently-available script file.

<extension_name>
The name of the extension to be installed. PostgreSQL will create the extension using details from the file extension_name.control, found via the server's extension control path (set by ???.)

<schema_name>
The name of the schema in which to install the extension's objects, given that the extension allows its contents to be relocated. The named schema must already exist. If not specified, and the extension's control file does not specify a schema either, the current default object creation schema is used.

If the extension specifies a schema parameter in its control file, then that schema cannot be overridden with a SCHEMA clause. Normally, an error will be raised if a SCHEMA clause is given and it conflicts with the extension's schema parameter. However, if the CASCADE clause is also given, then <schema_name> is ignored when it conflicts. The given <schema_name> will be used for installation of any needed extensions that do not specify schema in their control files.

Remember that the extension itself is not considered to be within any schema: extensions have unqualified names that must be unique database-wide. But objects belonging to the extension can be within schemas.

<version>
The version of the extension to install. This can be written as either an identifier or a string literal. The default version is whatever is specified in the extension's control file.

CASCADE
Automatically install any extensions that this extension depends on that are not already installed. Their dependencies are likewise automatically installed, recursively. The SCHEMA clause, if given, applies to all extensions that get installed this way. Other options of the statement are not applied to automatically-installed extensions; in particular, their default versions are always selected.

Notes

Before you can use CREATE EXTENSION to load an extension into a database, the extension's supporting files must be installed. Information about installing the extensions supplied with PostgreSQL can be found in Additional Supplied Modules.

The extensions currently available for loading can be identified from the pg_available_extensions or pg_available_extension_versions system views.

[!CAUTION] Installing an extension as superuser requires trusting that the extension's author wrote the extension installation script in a secure fashion. It is not terribly difficult for a malicious user to create trojan-horse objects that will compromise later execution of a carelessly-written extension script, allowing that user to acquire superuser privileges. However, trojan-horse objects are only hazardous if they are in the search_path during script execution, meaning that they are in the extension's installation target schema or in the schema of some extension it depends on. Therefore, a good rule of thumb when dealing with extensions whose scripts have not been carefully vetted is to install them only into schemas for which CREATE privilege has not been and will not be granted to any untrusted users. Likewise for any extensions they depend on.

The extensions supplied with PostgreSQL are believed to be secure against installation-time attacks of this sort, except for a few that depend on other extensions. As stated in the documentation for those extensions, they should be installed into secure schemas, or installed into the same schemas as the extensions they depend on, or both.

For information about writing new extensions, see ???.

Examples

Install the hstore extension into the current database, placing its objects in schema addons:

CREATE EXTENSION hstore SCHEMA addons;

Another way to accomplish the same thing:

SET search_path = addons;
CREATE EXTENSION hstore;

Compatibility

CREATE EXTENSION is a PostgreSQL extension.

See Also

create_foreign_data_wrapper

CREATE FOREIGN DATA WRAPPER

CREATE FOREIGN DATA WRAPPER

7

SQL - Language Statements

CREATE FOREIGN DATA WRAPPER

define a new foreign-data wrapper

CREATE FOREIGN DATA WRAPPER

name

[ HANDLER

handler_function

| NO HANDLER ] [ VALIDATOR

validator_function

| NO VALIDATOR ] [ CONNECTION

connection_function

| NO CONNECTION ] [ OPTIONS (

option

'

value

' [, ... ] ) ]

Description

CREATE FOREIGN DATA WRAPPER creates a new foreign-data wrapper. The user who defines a foreign-data wrapper becomes its owner.

The foreign-data wrapper name must be unique within the database.

Only superusers can create foreign-data wrappers.

Parameters

<name>
The name of the foreign-data wrapper to be created.

HANDLER handler_function
<handler_function> is the name of a previously registered function that will be called to retrieve the execution functions for foreign tables. The handler function must take no arguments, and its return type must be fdw_handler.

It is possible to create a foreign-data wrapper with no handler function, but foreign tables using such a wrapper can only be declared, not accessed.

VALIDATOR validator_function
<validator_function> is the name of a previously registered function that will be called to check the generic options given to the foreign-data wrapper, as well as options for foreign servers, user mappings and foreign tables using the foreign-data wrapper. If no validator function or NO VALIDATOR is specified, then options will not be checked at creation time. (Foreign-data wrappers will possibly ignore or reject invalid option specifications at run time, depending on the implementation.) The validator function must take two arguments: one of type text[], which will contain the array of options as stored in the system catalogs, and one of type oid, which will be the OID of the system catalog containing the options. The return type is ignored; the function should report invalid options using the ereport(ERROR) function.

CONNECTION connection_function
<connection_function> is the name of a previously registered function that will be called to generate the postgres connection string when a foreign server is used as part of ???. If no connection function or NO CONNECTION is specified, then servers using this foreign data wrapper cannot be used for CREATE SUBSCRIPTION. The connection function must take three arguments: one of type oid for the user, one of type oid for the server, and an unused third argument of type internal (which prevents calling the function in other contexts).

OPTIONS ( option 'value' [, ... ] )
This clause specifies options for the new foreign-data wrapper. The allowed option names and values are specific to each foreign data wrapper and are validated using the foreign-data wrapper's validator function. Option names must be unique.

Notes

PostgreSQL's foreign-data functionality is still under active development. Optimization of queries is primitive (and mostly left to the wrapper, too). Thus, there is considerable room for future performance improvements.

Examples

Create a useless foreign-data wrapper dummy:

CREATE FOREIGN DATA WRAPPER dummy;

Create a foreign-data wrapper file with handler function file_fdw_handler:

CREATE FOREIGN DATA WRAPPER file HANDLER file_fdw_handler;

Create a foreign-data wrapper mywrapper with some options:

CREATE FOREIGN DATA WRAPPER mywrapper
    OPTIONS (debug 'true');

Compatibility

CREATE FOREIGN DATA WRAPPER conforms to ISO/IEC 9075-9 (SQL/MED), with the exception that the HANDLER and VALIDATOR clauses are extensions and the standard clauses LIBRARY and LANGUAGE are not implemented in PostgreSQL.

Note, however, that the SQL/MED functionality as a whole is not yet conforming.

See Also

create_foreign_table

CREATE FOREIGN TABLE

CREATE FOREIGN TABLE

7

SQL - Language Statements

CREATE FOREIGN TABLE

define a new foreign table

CREATE FOREIGN TABLE [ IF NOT EXISTS ]

table_name

( [ {

column_name

data_type

[ OPTIONS (

option

'

value

' [, ... ] ) ] [ COLLATE

collation

] [

column_constraint

[ ... ] ] |

table_constraint

| LIKE

source_table

[

like_option

... ] } [, ... ] ] ) [ INHERITS (

parent_table

[, ... ] ) ] SERVER

server_name

[ OPTIONS (

option

'

value

' [, ... ] ) ] CREATE FOREIGN TABLE [ IF NOT EXISTS ]

table_name

PARTITION OF

parent_table

[ ( {

column_name

[ WITH OPTIONS ] [

column_constraint

[ ... ] ] |

table_constraint

} [, ... ] ) ] { FOR VALUES

partition_bound_spec

| DEFAULT } SERVER

server_name

[ OPTIONS (

option

'

value

' [, ... ] ) ]

where

column_constraint

is:

[ CONSTRAINT

constraint_name

] { NOT NULL [ NO INHERIT ] | NULL | CHECK (

expression

) [ NO INHERIT ] | DEFAULT

default_expr

| GENERATED ALWAYS AS (

generation_expr

) [ STORED | VIRTUAL ] } [ ENFORCED | NOT ENFORCED ]

and

table_constraint

is:

[ CONSTRAINT

constraint_name

] { NOT NULL

column_name

[ NO INHERIT ] | CHECK (

expression

) [ NO INHERIT ] } [ ENFORCED | NOT ENFORCED ]

and

like_option

is:

{ INCLUDING | EXCLUDING } { COMMENTS | CONSTRAINTS | DEFAULTS | GENERATED | STATISTICS | ALL }

and

partition_bound_spec

is:

IN (

partition_bound_expr

[, ...] ) | FROM ( {

partition_bound_expr

| MINVALUE | MAXVALUE } [, ...] ) TO ( {

partition_bound_expr

| MINVALUE | MAXVALUE } [, ...] ) | WITH ( MODULUS

numeric_literal

, REMAINDER

numeric_literal

)

Description

CREATE FOREIGN TABLE creates a new foreign table in the current database. The table will be owned by the user issuing the command.

If a schema name is given (for example, CREATE FOREIGN TABLE myschema.mytable ...) then the table is created in the specified schema. Otherwise it is created in the current schema. The name of the foreign table must be distinct from the name of any other relation (table, sequence, index, view, materialized view, or foreign table) in the same schema.

CREATE FOREIGN TABLE also automatically creates a data type that represents the composite type corresponding to one row of the foreign table. Therefore, foreign tables cannot have the same name as any existing data type in the same schema.

If PARTITION OF clause is specified then the table is created as a partition of parent_table with specified bounds.

To be able to create a foreign table, you must have USAGE privilege on the foreign server, as well as USAGE privilege on all column types used in the table.

Parameters

IF NOT EXISTS
Do not throw an error if a relation with the same name already exists. A notice is issued in this case. Note that there is no guarantee that the existing relation is anything like the one that would have been created.

<table_name>
The name (optionally schema-qualified) of the table to be created.

<column_name>
The name of a column to be created in the new table.

<data_type>
The data type of the column. This can include array specifiers. For more information on the data types supported by PostgreSQL, refer to ???.

COLLATE collation
The COLLATE clause assigns a collation to the column (which must be of a collatable data type). If not specified, the column data type's default collation is used.

INHERITS ( parent_table [, ... ] )
The optional INHERITS clause specifies a list of tables from which the new foreign table automatically inherits all columns. Parent tables can be plain tables or foreign tables. See the similar form of CREATE TABLE for more details.

PARTITION OF parent_table { FOR VALUES partition_bound_spec | DEFAULT }
This form can be used to create the foreign table as partition of the given parent table with specified partition bound values. See the similar form of CREATE TABLE for more details. Note that it is currently not allowed to create the foreign table as a partition of the parent table if there are UNIQUE indexes on the parent table. (See also ALTER TABLE ATTACH PARTITION.)

LIKE source_table [ like_option ... ]
The LIKE clause specifies a table from which the new table automatically copies all column names, their data types, and their not-null constraints.

Unlike INHERITS, the new table and original table are completely decoupled after creation is complete. Changes to the original table will not be applied to the new table, and it is not possible to include data of the new table in scans of the original table.

Also unlike INHERITS, columns and constraints copied by LIKE are not merged with similarly named columns and constraints. If the same name is specified explicitly or in another LIKE clause, an error is signaled.

The optional <like_option> clauses specify which additional properties of the original table to copy. Specifying INCLUDING copies the property, specifying EXCLUDING omits the property. EXCLUDING is the default. If multiple specifications are made for the same kind of object, the last one is used. The available options are:

INCLUDING COMMENTS
Comments for the copied columns, constraints, and extended statistics will be copied. The default behavior is to exclude comments, resulting in the corresponding objects in the new table having no comments.

INCLUDING CONSTRAINTS
CHECK constraints will be copied. No distinction is made between column constraints and table constraints. Not-null constraints are always copied to the new table.

INCLUDING DEFAULTS
Default expressions for the copied column definitions will be copied. Otherwise, default expressions are not copied, resulting in the copied columns in the new table having null defaults. Note that copying defaults that call database-modification functions, such as nextval, may create a functional linkage between the original and new tables.

INCLUDING GENERATED
Any generation expressions of copied column definitions will be copied. By default, new columns will be regular base columns.

INCLUDING STATISTICS
Extended statistics are copied to the new table.

INCLUDING ALL
INCLUDING ALL is an abbreviated form selecting all the available individual options. (It could be useful to write individual EXCLUDING clauses after INCLUDING ALL to select all but some specific options.)

CONSTRAINT constraint_name
An optional name for a column or table constraint. If the constraint is violated, the constraint name is present in error messages, so constraint names like col must be positive can be used to communicate helpful constraint information to client applications. (Double-quotes are needed to specify constraint names that contain spaces.) If a constraint name is not specified, the system generates a name.

NOT NULL [ NO INHERIT ]
The column is not allowed to contain null values.

A constraint marked with NO INHERIT will not propagate to child tables.

NULL
The column is allowed to contain null values. This is the default.

This clause is only provided for compatibility with non-standard SQL databases. Its use is discouraged in new applications.

CHECK ( expression ) [ NO INHERIT ]
The CHECK clause specifies an expression producing a Boolean result which each row in the foreign table is expected to satisfy; that is, the expression should produce TRUE or UNKNOWN, never FALSE, for all rows in the foreign table. A check constraint specified as a column constraint should reference that column's value only, while an expression appearing in a table constraint can reference multiple columns.

Currently, CHECK expressions cannot contain subqueries nor refer to variables other than columns of the current row. The system column tableoid may be referenced, but not any other system column.

A constraint marked with NO INHERIT will not propagate to child tables.

DEFAULT default_expr
The DEFAULT clause assigns a default data value for the column whose column definition it appears within. The value is any variable-free expression (subqueries and cross-references to other columns in the current table are not allowed). The data type of the default expression must match the data type of the column.

The default expression will be used in any insert operation that does not specify a value for the column. If there is no default for a column, then the default is null.

GENERATED ALWAYS AS ( generation_expr ) [ STORED | VIRTUAL ]
This clause creates the column as a generated column. The column cannot be written to, and when read the result of the specified expression will be returned.

When VIRTUAL is specified, the column will be computed when it is read. (The foreign-data wrapper will see it as a null value in new rows and may choose to store it as a null value or ignore it altogether.) When STORED is specified, the column will be computed on write. (The computed value will be presented to the foreign-data wrapper for storage and must be returned on reading.) VIRTUAL is the default.

The generation expression can refer to other columns in the table, but not other generated columns. Any functions and operators used must be immutable. References to other tables are not allowed.

<server_name>
The name of an existing foreign server to use for the foreign table. For details on defining a server, see ???.

OPTIONS ( option 'value' [, ...] )
Options to be associated with the new foreign table or one of its columns. The allowed option names and values are specific to each foreign data wrapper and are validated using the foreign-data wrapper's validator function. Duplicate option names are not allowed (although it's OK for a table option and a column option to have the same name).

Notes

Constraints on foreign tables (such as CHECK or NOT NULL clauses) are not enforced by the core PostgreSQL system, and most foreign data wrappers do not attempt to enforce them either; that is, the constraint is simply assumed to hold true. There would be little point in such enforcement since it would only apply to rows inserted or updated via the foreign table, and not to rows modified by other means, such as directly on the remote server. Instead, a constraint attached to a foreign table should represent a constraint that is being enforced by the remote server.

Some special-purpose foreign data wrappers might be the only access mechanism for the data they access, and in that case it might be appropriate for the foreign data wrapper itself to perform constraint enforcement. But you should not assume that a wrapper does that unless its documentation says so.

Although PostgreSQL does not attempt to enforce constraints on foreign tables, it does assume that they are correct for purposes of query optimization. If there are rows visible in the foreign table that do not satisfy a declared constraint, queries on the table might produce errors or incorrect answers. It is the user's responsibility to ensure that the constraint definition matches reality.

[!CAUTION] When a foreign table is used as a partition of a partitioned table, there is an implicit constraint that its contents must satisfy the partitioning rule. Again, it is the user's responsibility to ensure that that is true, which is best done by installing a matching constraint on the remote server.

Within a partitioned table containing foreign-table partitions, an UPDATE that changes the partition key value can cause a row to be moved from a local partition to a foreign-table partition, provided the foreign data wrapper supports tuple routing. However, it is not currently possible to move a row from a foreign-table partition to another partition. An UPDATE that would require doing that will fail due to the partitioning constraint, assuming that that is properly enforced by the remote server.

Similar considerations apply to generated columns. Stored generated columns are computed on insert or update on the local PostgreSQL server and handed to the foreign-data wrapper for writing out to the foreign data store, but it is not enforced that a query of the foreign table returns values for stored generated columns that are consistent with the generation expression. Again, this might result in incorrect query results.

Examples

Create foreign table films, which will be accessed through the server film_server:

CREATE FOREIGN TABLE films (
    code        char(5) NOT NULL,
    title       varchar(40) NOT NULL,
    did         integer NOT NULL,
    date_prod   date,
    kind        varchar(10),
    len         interval hour to minute
)
SERVER film_server;

Create foreign table measurement_y2016m07, which will be accessed through the server server_07, as a partition of the range partitioned table measurement:

CREATE FOREIGN TABLE measurement_y2016m07
    PARTITION OF measurement FOR VALUES FROM ('2016-07-01') TO ('2016-08-01')
    SERVER server_07;

Compatibility

The CREATE FOREIGN TABLE command largely conforms to the SQL standard; however, much as with CREATE TABLE, NULL constraints and zero-column foreign tables are permitted. The ability to specify column default values is also a PostgreSQL extension. Table inheritance, in the form defined by PostgreSQL, is nonstandard. The LIKE clause, as supported in this command, is nonstandard.

See Also

create_function

CREATE FUNCTION

CREATE FUNCTION

7

SQL - Language Statements

CREATE FUNCTION

define a new function

CREATE [ OR REPLACE ] FUNCTION

name

( [ [

argmode

] [

argname

]

argtype

[ { DEFAULT | = }

default_expr

] [, ...] ] ) [ RETURNS

rettype

| RETURNS TABLE (

column_name

column_type

[, ...] ) ] { LANGUAGE

lang_name

| TRANSFORM { FOR TYPE

type_name

} [, ... ] | WINDOW | { IMMUTABLE | STABLE | VOLATILE } | [ NOT ] LEAKPROOF | { CALLED ON NULL INPUT | RETURNS NULL ON NULL INPUT | STRICT } | { [ EXTERNAL ] SECURITY INVOKER | [ EXTERNAL ] SECURITY DEFINER } | PARALLEL { UNSAFE | RESTRICTED | SAFE } | COST

execution_cost

| ROWS

result_rows

| SUPPORT

support_function

| SET

configuration_parameter

{ TO

value

| =

value

| FROM CURRENT } | AS '

definition

' | AS '

obj_file

', '

' |

sql_body

} ...

Description

CREATE FUNCTION defines a new function. CREATE OR REPLACE FUNCTION will either create a new function, or replace an existing definition. To be able to define a function, the user must have the USAGE privilege on the language.

If a schema name is included, then the function is created in the specified schema. Otherwise it is created in the current schema. The name of the new function must not match any existing function or procedure with the same input argument types in the same schema. However, functions and procedures of different argument types can share a name (this is called overloading).

To replace the current definition of an existing function, use CREATE OR REPLACE FUNCTION. It is not possible to change the name or argument types of a function this way (if you tried, you would actually be creating a new, distinct function). Also, CREATE OR REPLACE FUNCTION will not let you change the return type of an existing function. To do that, you must drop and recreate the function. (When using OUT parameters, that means you cannot change the types of any OUT parameters except by dropping the function.)

When CREATE OR REPLACE FUNCTION is used to replace an existing function, the ownership and permissions of the function do not change. All other function properties are assigned the values specified or implied in the command. You must own the function to replace it (this includes being a member of the owning role).

If you drop and then recreate a function, the new function is not the same entity as the old; you will have to drop existing rules, views, triggers, etc. that refer to the old function. Use CREATE OR REPLACE FUNCTION to change a function definition without breaking objects that refer to the function. Also, ALTER FUNCTION can be used to change most of the auxiliary properties of an existing function.

The user that creates the function becomes the owner of the function.

To be able to create a function, you must have USAGE privilege on the argument types and the return type.

Refer to ??? for further information on writing functions.

Parameters

<name>
The name (optionally schema-qualified) of the function to create.

<argmode>
The mode of an argument: IN, OUT, INOUT, or VARIADIC. If omitted, the default is IN. Only OUT arguments can follow a VARIADIC one. Also, OUT and INOUT arguments cannot be used together with the RETURNS TABLE notation.

<argname>
The name of an argument. Some languages (including SQL and PL/pgSQL) let you use the name in the function body. For other languages the name of an input argument is just extra documentation, so far as the function itself is concerned; but you can use input argument names when calling a function to improve readability (see ???). In any case, the name of an output argument is significant, because it defines the column name in the result row type. (If you omit the name for an output argument, the system will choose a default column name.)

<argtype>
The data type(s) of the function's arguments (optionally schema-qualified), if any. The argument types can be base, composite, or domain types, or can reference the type of a table column.

Depending on the implementation language it might also be allowed to specify “pseudo-types” such as cstring. Pseudo-types indicate that the actual argument type is either incompletely specified, or outside the set of ordinary SQL data types.

The type of a column is referenced by writing table_name.column_name%TYPE. Using this feature can sometimes help make a function independent of changes to the definition of a table.

<default_expr>
An expression to be used as default value if the parameter is not specified. The expression has to be coercible to the argument type of the parameter. Only input (including INOUT) parameters can have a default value. All input parameters following a parameter with a default value must have default values as well.

<rettype>
The return data type (optionally schema-qualified). The return type can be a base, composite, or domain type, or can reference the type of a table column. Depending on the implementation language it might also be allowed to specify “pseudo-types” such as cstring. If the function is not supposed to return a value, specify void as the return type.

When there are OUT or INOUT parameters, the RETURNS clause can be omitted. If present, it must agree with the result type implied by the output parameters: RECORD if there are multiple output parameters, or the same type as the single output parameter.

The SETOF modifier indicates that the function will return a set of items, rather than a single item.

The type of a column is referenced by writing table_name.column_name%TYPE.

<column_name>
The name of an output column in the RETURNS TABLE syntax. This is effectively another way of declaring a named OUT parameter, except that RETURNS TABLE also implies RETURNS SETOF.

<column_type>
The data type of an output column in the RETURNS TABLE syntax.

<lang_name>
The name of the language that the function is implemented in. It can be sql, c, internal, or the name of a user-defined procedural language, e.g., plpgsql. The default is sql if <sql_body> is specified. Enclosing the name in single quotes is deprecated and requires matching case.

TRANSFORM { FOR TYPE type_name } [, ... ]
Lists which transforms a call to the function should apply. Transforms convert between SQL types and language-specific data types; see ???. Procedural language implementations usually have hardcoded knowledge of the built-in types, so those don't need to be listed here. If a procedural language implementation does not know how to handle a type and no transform is supplied, it will fall back to a default behavior for converting data types, but this depends on the implementation.

WINDOW
WINDOW indicates that the function is a window function rather than a plain function. This is currently only useful for functions written in C. The WINDOW attribute cannot be changed when replacing an existing function definition.

IMMUTABLE; STABLE; VOLATILE
These attributes inform the query optimizer about the behavior of the function. At most one choice can be specified. If none of these appear, VOLATILE is the default assumption.

IMMUTABLE indicates that the function cannot modify the database and always returns the same result when given the same argument values; that is, it does not do database lookups or otherwise use information not directly present in its argument list. If this option is given, any call of the function with all-constant arguments can be immediately replaced with the function value.

STABLE indicates that the function cannot modify the database, and that within a single table scan it will consistently return the same result for the same argument values, but that its result could change across SQL statements. This is the appropriate selection for functions whose results depend on database lookups, parameter variables (such as the current time zone), etc. (It is inappropriate for AFTER triggers that wish to query rows modified by the current command.) Also note that the current_timestamp family of functions qualify as stable, since their values do not change within a transaction.

VOLATILE indicates that the function value can change even within a single table scan, so no optimizations can be made. Relatively few database functions are volatile in this sense; some examples are random(), currval(), timeofday(). But note that any function that has side-effects must be classified volatile, even if its result is quite predictable, to prevent calls from being optimized away; an example is setval().

For additional details see ???.

LEAKPROOF
LEAKPROOF indicates that the function has no side effects. It reveals no information about its arguments other than by its return value. For example, a function which throws an error message for some argument values but not others, or which includes the argument values in any error message, is not leakproof. This affects how the system executes queries against views created with the security_barrier option or tables with row level security enabled. The system will enforce conditions from security policies and security barrier views before any user-supplied conditions from the query itself that contain non-leakproof functions, in order to prevent the inadvertent exposure of data. Functions and operators marked as leakproof are assumed to be trustworthy, and may be executed before conditions from security policies and security barrier views. In addition, functions which do not take arguments or which are not passed any arguments from the security barrier view or table do not have to be marked as leakproof to be executed before security conditions. See ??? and ???. This option can only be set by the superuser.

CALLED ON NULL INPUT; RETURNS NULL ON NULL INPUT; STRICT
CALLED ON NULL INPUT (the default) indicates that the function will be called normally when some of its arguments are null. It is then the function author's responsibility to check for null values if necessary and respond appropriately.

RETURNS NULL ON NULL INPUT or STRICT indicates that the function always returns null whenever any of its arguments are null. If this parameter is specified, the function is not executed when there are null arguments; instead a null result is assumed automatically.

EXTERNAL SECURITY INVOKER; EXTERNAL SECURITY DEFINER
SECURITY INVOKER indicates that the function is to be executed with the privileges of the user that calls it. That is the default. SECURITY DEFINER specifies that the function is to be executed with the privileges of the user that owns it. For information on how to write SECURITY DEFINER functions safely, see below.

The key word EXTERNAL is allowed for SQL conformance, but it is optional since, unlike in SQL, this feature applies to all functions not only external ones.

PARALLEL
PARALLEL UNSAFE indicates that the function can't be executed in parallel mode; the presence of such a function in an SQL statement forces a serial execution plan. This is the default. PARALLEL RESTRICTED indicates that the function can be executed in parallel mode, but only in the parallel group leader process. PARALLEL SAFE indicates that the function is safe to run in parallel mode without restriction, including in parallel worker processes.

Functions should be labeled parallel unsafe if they modify any database state, change the transaction state (other than by using a subtransaction for error recovery), access sequences (e.g., by calling currval) or make persistent changes to settings. They should be labeled parallel restricted if they access temporary tables, client connection state, cursors, prepared statements, or miscellaneous backend-local state which the system cannot synchronize in parallel mode (e.g., setseed cannot be executed other than by the group leader because a change made by another process would not be reflected in the leader). In general, if a function is labeled as being safe when it is restricted or unsafe, or if it is labeled as being restricted when it is in fact unsafe, it may throw errors or produce wrong answers when used in a parallel query. C-language functions could in theory exhibit totally undefined behavior if mislabeled, since there is no way for the system to protect itself against arbitrary C code, but in most likely cases the result will be no worse than for any other function. If in doubt, functions should be labeled as UNSAFE, which is the default.

COST <execution_cost>
A positive number giving the estimated execution cost for the function, in units of ???. If the function returns a set, this is the cost per returned row. If the cost is not specified, 1 unit is assumed for C-language and internal functions, and 100 units for functions in all other languages. Larger values cause the planner to try to avoid evaluating the function more often than necessary.

ROWS <result_rows>
A positive number giving the estimated number of rows that the planner should expect the function to return. This is only allowed when the function is declared to return a set. The default assumption is 1000 rows.

SUPPORT <support_function>
The name (optionally schema-qualified) of a planner support function to use for this function. See ??? for details. You must be superuser to use this option.

<configuration_parameter>; <value>
The SET clause causes the specified configuration parameter to be set to the specified value when the function is entered, and then restored to its prior value when the function exits. SET FROM CURRENT saves the value of the parameter that is current when CREATE FUNCTION is executed as the value to be applied when the function is entered.

If a SET clause is attached to a function, then the effects of a SET LOCAL command executed inside the function for the same variable are restricted to the function: the configuration parameter's prior value is still restored at function exit. However, an ordinary SET command (without LOCAL) overrides the SET clause, much as it would do for a previous SET LOCAL command: the effects of such a command will persist after function exit, unless the current transaction is rolled back.

See ??? and ??? for more information about allowed parameter names and values.

<definition>
A string constant defining the function; the meaning depends on the language. It can be an internal function name, the path to an object file, an SQL command, or text in a procedural language.

It is often helpful to use dollar quoting (see ???) to write the function definition string, rather than the normal single quote syntax. Without dollar quoting, any single quotes or backslashes in the function definition must be escaped by doubling them.

When repeated CREATE FUNCTION calls refer to the same object file, the file is only loaded once per session. To unload and reload the file (perhaps during development), start a new session.

<sql_body>
The body of a LANGUAGE SQL function. This can either be a single statement

RETURN expression

or a block

BEGIN ATOMIC
  statement;
  statement;
  ...
  statement;
END

This is similar to writing the text of the function body as a string constant (see <definition> above), but there are some differences: This form only works for LANGUAGE SQL, the string constant form works for all languages. This form is parsed at function definition time, the string constant form is parsed at execution time; therefore this form cannot support polymorphic argument types and other constructs that are not resolvable at function definition time. This form tracks dependencies between the function and objects used in the function body, so DROP ... CASCADE will work correctly, whereas the form using string literals may leave dangling functions. Finally, this form is more compatible with the SQL standard and other SQL implementations.

Overloading

PostgreSQL allows function overloading; that is, the same name can be used for several different functions so long as they have distinct input argument types. Whether or not you use it, this capability entails security precautions when calling functions in databases where some users mistrust other users; see ???.

Two functions are considered the same if they have the same names and input argument types, ignoring any OUT parameters. Thus for example these declarations conflict:

CREATE FUNCTION foo(int) ...
CREATE FUNCTION foo(int, OUT text) ...

Functions that have different argument type lists will not be considered to conflict at creation time, but if defaults are provided they might conflict in use. For example, consider

CREATE FUNCTION foo(int) ...
CREATE FUNCTION foo(int, int default 42) ...

A call foo(10) will fail due to the ambiguity about which function should be called.

Notes

The full SQL type syntax is allowed for declaring a function's arguments and return value. However, parenthesized type modifiers (e.g., the precision field for type numeric) are discarded by CREATE FUNCTION. Thus for example CREATE FUNCTION foo (varchar(10)) ... is exactly the same as CREATE FUNCTION foo (varchar) ....

When replacing an existing function with CREATE OR REPLACE FUNCTION, there are restrictions on changing parameter names. You cannot change the name already assigned to any input parameter (although you can add names to parameters that had none before). If there is more than one output parameter, you cannot change the names of the output parameters, because that would change the column names of the anonymous composite type that describes the function's result. These restrictions are made to ensure that existing calls of the function do not stop working when it is replaced.

If a function is declared STRICT with a VARIADIC argument, the strictness check tests that the variadic array as a whole is non-null. The function will still be called if the array has null elements.

Examples

Add two integers using an SQL function:

CREATE FUNCTION add(integer, integer) RETURNS integer
    AS 'SELECT $1 + $2;'
    LANGUAGE SQL
    IMMUTABLE
    RETURNS NULL ON NULL INPUT;

The same function written in a more SQL-conforming style, using argument names and an unquoted body:

CREATE FUNCTION add(a integer, b integer) RETURNS integer
    LANGUAGE SQL
    IMMUTABLE
    RETURNS NULL ON NULL INPUT
    RETURN a + b;

Increment an integer, making use of an argument name, in PL/pgSQL:

CREATE OR REPLACE FUNCTION increment(i integer) RETURNS integer AS $$
        BEGIN
                RETURN i + 1;
        END;
$$ LANGUAGE plpgsql;

Return a record containing multiple output parameters:

CREATE FUNCTION dup(IN int, OUT f1 int, OUT f2 text)
    AS $$ SELECT $1, CAST($1 AS text) || ' is text' $$
    LANGUAGE SQL;

SELECT * FROM dup(42);

You can do the same thing more verbosely with an explicitly named composite type:

CREATE TYPE dup_result AS (f1 int, f2 text);

CREATE FUNCTION dup(int) RETURNS dup_result
    AS $$ SELECT $1, CAST($1 AS text) || ' is text' $$
    LANGUAGE SQL;

SELECT * FROM dup(42);

Another way to return multiple columns is to use a TABLE function:

CREATE FUNCTION dup(int) RETURNS TABLE(f1 int, f2 text)
    AS $$ SELECT $1, CAST($1 AS text) || ' is text' $$
    LANGUAGE SQL;

SELECT * FROM dup(42);

However, a TABLE function is different from the preceding examples, because it actually returns a set of records, not just one record.

Writing SECURITY DEFINER Functions Safely

search_path

configuration parameter

use in securing functions

createrole_self_grant

configuration parameter

use in securing functions

Because a SECURITY DEFINER function is executed with the privileges of the user that owns it, care is needed to ensure that the function cannot be misused. For security, ??? should be set to exclude any schemas writable by untrusted users. This prevents malicious users from creating objects (e.g., tables, functions, and operators) that mask objects intended to be used by the function. Particularly important in this regard is the temporary-table schema, which is searched first by default, and is normally writable by anyone. A secure arrangement can be obtained by forcing the temporary schema to be searched last. To do this, write pg_temp as the last entry in search_path. This function illustrates safe usage:

CREATE FUNCTION check_password(uname TEXT, pass TEXT)
RETURNS BOOLEAN AS $$
DECLARE passed BOOLEAN;
BEGIN
        SELECT  (pwd = $2) INTO passed
        FROM    pwds
        WHERE   username = $1;

        RETURN passed;
END;
$$  LANGUAGE plpgsql
    SECURITY DEFINER
    -- Set a secure search_path: trusted schema(s), then 'pg_temp'.
    SET search_path = admin, pg_temp;

This function's intention is to access a table admin.pwds. But without the SET clause, or with a SET clause mentioning only admin, the function could be subverted by creating a temporary table named pwds.

If the security definer function intends to create roles, and if it is running as a non-superuser, createrole_self_grant should also be set to a known value using the SET clause.

Another point to keep in mind is that by default, execute privilege is granted to PUBLIC for newly created functions (see ??? for more information). Frequently you will wish to restrict use of a security definer function to only some users. To do that, you must revoke the default PUBLIC privileges and then grant execute privilege selectively. To avoid having a window where the new function is accessible to all, create it and set the privileges within a single transaction. For example:

BEGIN;
CREATE FUNCTION check_password(uname TEXT, pass TEXT) ... SECURITY DEFINER;
REVOKE ALL ON FUNCTION check_password(uname TEXT, pass TEXT) FROM PUBLIC;
GRANT EXECUTE ON FUNCTION check_password(uname TEXT, pass TEXT) TO admins;
COMMIT;

Compatibility

A CREATE FUNCTION command is defined in the SQL standard. The PostgreSQL implementation can be used in a compatible way but has many extensions. Conversely, the SQL standard specifies a number of optional features that are not implemented in PostgreSQL.

The following are important compatibility issues:

Simple LANGUAGE SQL functions can be written in a way that is both standard-conforming and portable to other implementations. More complex functions using advanced features, optimization attributes, or other languages will necessarily be specific to PostgreSQL in a significant way.

See Also

create_group

CREATE GROUP

CREATE GROUP

7

SQL - Language Statements

CREATE GROUP

define a new database role

CREATE GROUP

name

[ [ WITH ]

option

[ ... ] ]

where

option

can be:

SUPERUSER | NOSUPERUSER | CREATEDB | NOCREATEDB | CREATEROLE | NOCREATEROLE | INHERIT | NOINHERIT | LOGIN | NOLOGIN | REPLICATION | NOREPLICATION | BYPASSRLS | NOBYPASSRLS | CONNECTION LIMIT

connlimit

| [ ENCRYPTED ] PASSWORD '

password

' | PASSWORD NULL | VALID UNTIL '

timestamp

' | IN ROLE

role_name

[, ...] | ROLE

role_name

[, ...] | ADMIN

role_name

[, ...] | SYSID

uid

Description

CREATE GROUP is now an alias for ???.

Compatibility

There is no CREATE GROUP statement in the SQL standard.

See Also

create_index

CREATE INDEX

CREATE INDEX

7

SQL - Language Statements

CREATE INDEX

define a new index

CREATE [ UNIQUE ] INDEX [ CONCURRENTLY ] [ [ IF NOT EXISTS ]

name

] ON [ ONLY ]

table_name

[ USING

method

] ( {

column_name

| (

expression

) } [ COLLATE

collation

] [

opclass

[ (

opclass_parameter

=

value

[, ... ] ) ] ] [ ASC | DESC ] [ NULLS { FIRST | LAST } ] [, ...] ) [ INCLUDE (

column_name

[, ...] ) ] [ NULLS [ NOT ] DISTINCT ] [ WITH (

storage_parameter

[=

value

] [, ... ] ) ] [ TABLESPACE

tablespace_name

] [ WHERE

predicate

]

Description

CREATE INDEX constructs an index on the specified column(s) of the specified relation, which can be a table or a materialized view. Indexes are primarily used to enhance database performance (though inappropriate use can result in slower performance).

The key field(s) for the index are specified as column names, or alternatively as expressions written in parentheses. Multiple fields can be specified if the index method supports multicolumn indexes.

An index field can be an expression computed from the values of one or more columns of the table row. This feature can be used to obtain fast access to data based on some transformation of the basic data. For example, an index computed on upper(col) would allow the clause WHERE upper(col) = 'JIM' to use an index.

PostgreSQL provides the index methods B-tree, hash, GiST, SP-GiST, GIN, and BRIN. Users can also define their own index methods, but that is fairly complicated.

When the WHERE clause is present, a partial index is created. A partial index is an index that contains entries for only a portion of a table, usually a portion that is more useful for indexing than the rest of the table. For example, if you have a table that contains both billed and unbilled orders where the unbilled orders take up a small fraction of the total table and yet that is an often used section, you can improve performance by creating an index on just that portion. Another possible application is to use WHERE with UNIQUE to enforce uniqueness over a subset of a table. See ??? for more discussion.

The expression used in the WHERE clause can refer only to columns of the underlying table, but it can use all columns, not just the ones being indexed. Presently, subqueries and aggregate expressions are also forbidden in WHERE. The same restrictions apply to index fields that are expressions.

All functions and operators used in an index definition must be “immutable”, that is, their results must depend only on their arguments and never on any outside influence (such as the contents of another table or the current time). This restriction ensures that the behavior of the index is well-defined. To use a user-defined function in an index expression or WHERE clause, remember to mark the function immutable when you create it.

Parameters

UNIQUE
Causes the system to check for duplicate values in the table when the index is created (if data already exist) and each time data is added. Attempts to insert or update data which would result in duplicate entries will generate an error.

Additional restrictions apply when unique indexes are applied to partitioned tables; see ???.

CONCURRENTLY
When this option is used, PostgreSQL will build the index without taking any locks that prevent concurrent inserts, updates, or deletes on the table; whereas a standard index build locks out writes (but not reads) on the table until it's done. There are several caveats to be aware of when using this option — see Building Indexes Concurrently below.

For temporary tables, CREATE INDEX is always non-concurrent, as no other session can access them, and non-concurrent index creation is cheaper.

IF NOT EXISTS
Do not throw an error if a relation with the same name already exists. A notice is issued in this case. Note that there is no guarantee that the existing index is anything like the one that would have been created. Index name is required when IF NOT EXISTS is specified.

INCLUDE
The optional INCLUDE clause specifies a list of columns which will be included in the index as non-key columns. A non-key column cannot be used in an index scan search qualification, and it is disregarded for purposes of any uniqueness or exclusion constraint enforced by the index. However, an index-only scan can return the contents of non-key columns without having to visit the index's table, since they are available directly from the index entry. Thus, addition of non-key columns allows index-only scans to be used for queries that otherwise could not use them.

It's wise to be conservative about adding non-key columns to an index, especially wide columns. If an index tuple exceeds the maximum size allowed for the index type, data insertion will fail. In any case, non-key columns duplicate data from the index's table and bloat the size of the index, thus potentially slowing searches. Furthermore, B-tree deduplication is never used with indexes that have a non-key column.

Columns listed in the INCLUDE clause don't need appropriate operator classes; the clause can include columns whose data types don't have operator classes defined for a given access method.

Expressions are not supported as included columns since they cannot be used in index-only scans.

Currently, the B-tree, GiST and SP-GiST index access methods support this feature. In these indexes, the values of columns listed in the INCLUDE clause are included in leaf tuples which correspond to heap tuples, but are not included in upper-level index entries used for tree navigation.

<name>
The name of the index to be created. No schema name can be included here; the index is always created in the same schema as its parent table. The name of the index must be distinct from the name of any other relation (table, sequence, index, view, materialized view, or foreign table) in that schema. If the name is omitted, PostgreSQL chooses a suitable name based on the parent table's name and the indexed column name(s).

ONLY
Indicates not to recurse creating indexes on partitions, if the table is partitioned. The default is to recurse.

<table_name>
The name (possibly schema-qualified) of the table to be indexed.

<method>
The name of the index method to be used. Choices are btree, hash, gist, spgist, gin, brin, or user-installed access methods like bloom. The default method is btree.

<column_name>
The name of a column of the table.

<expression>
An expression based on one or more columns of the table. The expression usually must be written with surrounding parentheses, as shown in the syntax. However, the parentheses can be omitted if the expression has the form of a function call.

<collation>
The name of the collation to use for the index. By default, the index uses the collation declared for the column to be indexed or the result collation of the expression to be indexed. Indexes with non-default collations can be useful for queries that involve expressions using non-default collations.

<opclass>
The name of an operator class. See below for details.

<opclass_parameter>
The name of an operator class parameter. See below for details.

ASC
Specifies ascending sort order (which is the default).

DESC
Specifies descending sort order.

NULLS FIRST
Specifies that nulls sort before non-nulls. This is the default when DESC is specified.

NULLS LAST
Specifies that nulls sort after non-nulls. This is the default when DESC is not specified.

NULLS DISTINCT; NULLS NOT DISTINCT
Specifies whether for a unique index, null values should be considered distinct (not equal). The default is that they are distinct, so that a unique index could contain multiple null values in a column.

<storage_parameter>
The name of an index-method-specific storage parameter. See Index Storage Parameters below for details.

<tablespace_name>
The tablespace in which to create the index. If not specified, ??? is consulted, or ??? for indexes on temporary tables.

<predicate>
The constraint expression for a partial index.

Index Storage Parameters

The optional WITH clause specifies storage parameters for the index. Each index method has its own set of allowed storage parameters.

The B-tree, hash, GiST and SP-GiST index methods all accept this parameter:

fillfactor (integer)
Controls how full the index method will try to pack index pages. For B-trees, leaf pages are filled to this percentage during initial index builds, and also when extending the index at the right (adding new largest key values). If pages subsequently become completely full, they will be split, leading to fragmentation of the on-disk index structure. B-trees use a default fillfactor of 90, but any integer value from 10 to 100 can be selected.

B-tree indexes on tables where many inserts and/or updates are anticipated can benefit from lower fillfactor settings at CREATE INDEX time (following bulk loading into the table). Values in the range of 50 - 90 can usefully “smooth out” the rate of page splits during the early life of the B-tree index (lowering fillfactor like this may even lower the absolute number of page splits, though this effect is highly workload dependent). The B-tree bottom-up index deletion technique described in ??? is dependent on having some “extra” space on pages to store “extra” tuple versions, and so can be affected by fillfactor (though the effect is usually not significant).

In other specific cases it might be useful to increase fillfactor to 100 at CREATE INDEX time as a way of maximizing space utilization. You should only consider this when you are completely sure that the table is static (i.e. that it will never be affected by either inserts or updates). A fillfactor setting of 100 otherwise risks harming performance: even a few updates or inserts will cause a sudden flood of page splits.

The other index methods use fillfactor in different but roughly analogous ways; the default fillfactor varies between methods.

B-tree indexes additionally accept this parameter:

deduplicate_items (boolean)
Controls usage of the B-tree deduplication technique described in ???. Set to ON or OFF to enable or disable the optimization. (Alternative spellings of ON and OFF are allowed as described in ???.) The default is ON.

[!NOTE] Turning deduplicate_items off via ALTER INDEX prevents future insertions from triggering deduplication, but does not in itself make existing posting list tuples use the standard tuple representation.

GiST indexes additionally accept this parameter:

buffering (enum)
Controls whether the buffered build technique described in ??? is used to build the index. With OFF buffering is disabled, with ON it is enabled, and with AUTO it is initially disabled, but is turned on on-the-fly once the index size reaches ???. The default is AUTO. Note that if sorted build is possible, it will be used instead of buffered build unless buffering=ON is specified.

GIN indexes accept these parameters:

fastupdate (boolean)
Controls usage of the fast update technique described in ???. ON enables fast update, OFF disables it. The default is ON.

[!NOTE] Turning fastupdate off via ALTER INDEX prevents future insertions from going into the list of pending index entries, but does not in itself flush existing entries. You might want to VACUUM the table or call the gin_clean_pending_list function afterward to ensure the pending list is emptied.

gin_pending_list_limit (integer)
Overrides the global setting of ??? for this index. This value is specified in kilobytes.

BRIN indexes accept these parameters:

pages_per_range (integer)
Defines the number of table blocks that make up one block range for each entry of a BRIN index (see ??? for more details). The default is 128.

autosummarize (boolean)
Defines whether a summarization run is queued for the previous page range whenever an insertion is detected on the next one (see ??? for more details). The default is off.

Building Indexes Concurrently

index

building concurrently

Creating an index can interfere with regular operation of a database. Normally PostgreSQL locks the table to be indexed against writes and performs the entire index build with a single scan of the table. Other transactions can still read the table, but if they try to insert, update, or delete rows in the table they will block until the index build is finished. This could have a severe effect if the system is a live production database. Very large tables can take many hours to be indexed, and even for smaller tables, an index build can lock out writers for periods that are unacceptably long for a production system.

PostgreSQL supports building indexes without locking out writes. This method is invoked by specifying the CONCURRENTLY option of CREATE INDEX. When this option is used, PostgreSQL must perform two scans of the table, and in addition it must wait for all existing transactions that could potentially modify or use the index to terminate. Thus this method requires more total work than a standard index build and takes significantly longer to complete. However, since it allows normal operations to continue while the index is built, this method is useful for adding new indexes in a production environment. Of course, the extra CPU and I/O load imposed by the index creation might slow other operations.

In a concurrent index build, the index is actually entered as an “invalid” index into the system catalogs in one transaction, then two table scans occur in two more transactions. Before each table scan, the index build must wait for existing transactions that have modified the table to terminate. After the second scan, the index build must wait for any transactions that have a snapshot (see ???) predating the second scan to terminate, including transactions used by any phase of concurrent index builds on other tables, if the indexes involved are partial or have columns that are not simple column references. Then finally the index can be marked “valid” and ready for use, and the CREATE INDEX command terminates. Even then, however, the index may not be immediately usable for queries: in the worst case, it cannot be used as long as transactions exist that predate the start of the index build.

If a problem arises while scanning the table, such as a deadlock or a uniqueness violation in a unique index, the CREATE INDEX command will fail but leave behind an “invalid” index. This index will be ignored for querying purposes because it might be incomplete; however it will still consume update overhead. The psql \d command will report such an index as INVALID:

postgres=# \d tab
       Table "public.tab"
 Column |  Type   | Collation | Nullable | Default
--------+---------+-----------+----------+---------
 col    | integer |           |          |
Indexes:
    "idx" btree (col) INVALID

Another caveat when building a unique index concurrently is that the uniqueness constraint is already being enforced against other transactions when the second table scan begins. This means that constraint violations could be reported in other queries prior to the index becoming available for use, or even in cases where the index build eventually fails. Also, if a failure does occur in the second scan, the “invalid” index continues to enforce its uniqueness constraint afterwards.

Concurrent builds of expression indexes and partial indexes are supported. Errors occurring in the evaluation of these expressions could cause behavior similar to that described above for unique constraint violations.

Regular index builds permit other regular index builds on the same table to occur simultaneously, but only one concurrent index build can occur on a table at a time. In either case, schema modification of the table is not allowed while the index is being built. Another difference is that a regular CREATE INDEX command can be performed within a transaction block, but CREATE INDEX CONCURRENTLY cannot.

Concurrent builds for indexes on partitioned tables are currently not supported. However, you may concurrently build the index on each partition individually and then finally create the partitioned index non-concurrently in order to reduce the time where writes to the partitioned table will be locked out. In this case, building the partitioned index is a metadata only operation.

Notes

See ??? for information about when indexes can be used, when they are not used, and in which particular situations they can be useful.

Currently, only the B-tree, GiST, GIN, and BRIN index methods support multiple-key-column indexes. Whether there can be multiple key columns is independent of whether INCLUDE columns can be added to the index. Indexes can have up to 32 columns, including INCLUDE columns. (This limit can be altered when building PostgreSQL.) Only B-tree currently supports unique indexes.

An operator class with optional parameters can be specified for each column of an index. The operator class identifies the operators to be used by the index for that column. For example, a B-tree index on four-byte integers would use the int4_ops class; this operator class includes comparison functions for four-byte integers. In practice the default operator class for the column's data type is usually sufficient. The main point of having operator classes is that for some data types, there could be more than one meaningful ordering. For example, we might want to sort a complex-number data type either by absolute value or by real part. We could do this by defining two operator classes for the data type and then selecting the proper class when creating an index. More information about operator classes is in ??? and in ???.

When CREATE INDEX is invoked on a partitioned table, the default behavior is to recurse to all partitions to ensure they all have matching indexes. Each partition is first checked to determine whether an equivalent index already exists, and if so, that index will become attached as a partition index to the index being created, which will become its parent index. If no matching index exists, a new index will be created and automatically attached; the name of the new index in each partition will be determined as if no index name had been specified in the command. If the ONLY option is specified, no recursion is done, and the index is marked invalid. (ALTER INDEX ... ATTACH PARTITION marks the index valid, once all partitions acquire matching indexes.) Note, however, that any partition that is created in the future using CREATE TABLE ... PARTITION OF will automatically have a matching index, regardless of whether ONLY is specified.

For index methods that support ordered scans (currently, only B-tree), the optional clauses ASC, DESC, NULLS FIRST, and/or NULLS LAST can be specified to modify the sort ordering of the index. Since an ordered index can be scanned either forward or backward, it is not normally useful to create a single-column DESC index — that sort ordering is already available with a regular index. The value of these options is that multicolumn indexes can be created that match the sort ordering requested by a mixed-ordering query, such as SELECT ... ORDER BY x ASC, y DESC. The NULLS options are useful if you need to support “nulls sort low” behavior, rather than the default “nulls sort high”, in queries that depend on indexes to avoid sorting steps.

The system regularly collects statistics on all of a table's columns. Newly-created non-expression indexes can immediately use these statistics to determine an index's usefulness. For new expression indexes, it is necessary to run ANALYZE or wait for the autovacuum daemon to analyze the table to generate statistics for these indexes.

While CREATE INDEX is running, the ??? is temporarily changed to pg_catalog, pg_temp.

For most index methods, the speed of creating an index is dependent on the setting of ???. Larger values will reduce the time needed for index creation, so long as you don't make it larger than the amount of memory really available, which would drive the machine into swapping.

PostgreSQL can build indexes while leveraging multiple CPUs in order to process the table rows faster. This feature is known as parallel index build. For index methods that support building indexes in parallel (currently, B-tree, GIN, and BRIN), maintenance_work_mem specifies the maximum amount of memory that can be used by each index build operation as a whole, regardless of how many worker processes were started. Generally, a cost model automatically determines how many worker processes should be requested, if any.

Parallel index builds may benefit from increasing maintenance_work_mem where an equivalent serial index build will see little or no benefit. Note that maintenance_work_mem may influence the number of worker processes requested, since parallel workers must have at least a 32MB share of the total maintenance_work_mem budget. There must also be a remaining 32MB share for the leader process. Increasing ??? may allow more workers to be used, which will reduce the time needed for index creation, so long as the index build is not already I/O bound. Of course, there should also be sufficient CPU capacity that would otherwise lie idle.

Setting a value for parallel_workers via ALTER TABLE directly controls how many parallel worker processes will be requested by a CREATE INDEX against the table. This bypasses the cost model completely, and prevents maintenance_work_mem from affecting how many parallel workers are requested. Setting parallel_workers to 0 via ALTER TABLE will disable parallel index builds on the table in all cases.

[!TIP] You might want to reset parallel_workers after setting it as part of tuning an index build. This avoids inadvertent changes to query plans, since parallel_workers affects all parallel table scans.

While CREATE INDEX with the CONCURRENTLY option supports parallel builds without special restrictions, only the first table scan is actually performed in parallel.

Use DROP INDEX to remove an index.

Like any long-running transaction, CREATE INDEX on a table can affect which tuples can be removed by concurrent VACUUM on any other table.

Prior releases of PostgreSQL also had an R-tree index method. This method has been removed because it had no significant advantages over the GiST method. If USING rtree is specified, CREATE INDEX will interpret it as USING gist, to simplify conversion of old databases to GiST.

Each backend running CREATE INDEX will report its progress in the pg_stat_progress_create_index view. See ??? for details.

Examples

To create a unique B-tree index on the column title in the table films:

CREATE UNIQUE INDEX title_idx ON films (title);

To create a unique B-tree index on the column title with included columns director and rating in the table films:

CREATE UNIQUE INDEX title_idx ON films (title) INCLUDE (director, rating);

To create a B-Tree index with deduplication disabled:

CREATE INDEX title_idx ON films (title) WITH (deduplicate_items = off);

To create an index on the expression lower(title), allowing efficient case-insensitive searches:

CREATE INDEX ON films ((lower(title)));

(In this example we have chosen to omit the index name, so the system will choose a name, typically films_lower_idx.)

To create an index with non-default collation:

CREATE INDEX title_idx_german ON films (title COLLATE "de_DE");

To create an index with non-default sort ordering of nulls:

CREATE INDEX title_idx_nulls_low ON films (title NULLS FIRST);

To create an index with non-default fill factor:

CREATE UNIQUE INDEX title_idx ON films (title) WITH (fillfactor = 70);

To create a GIN index with fast updates disabled:

CREATE INDEX gin_idx ON documents_table USING GIN (locations) WITH (fastupdate = off);

To create an index on the column code in the table films and have the index reside in the tablespace indexspace:

CREATE INDEX code_idx ON films (code) TABLESPACE indexspace;

To create a GiST index on a point attribute so that we can efficiently use box operators on the result of the conversion function:

CREATE INDEX pointloc
    ON points USING gist (box(location,location));
SELECT * FROM points
    WHERE box(location,location) && '(0,0),(1,1)'::box;

To create an index without locking out writes to the table:

CREATE INDEX CONCURRENTLY sales_quantity_index ON sales_table (quantity);

Compatibility

CREATE INDEX is a PostgreSQL language extension. There are no provisions for indexes in the SQL standard.

See Also

create_language

CREATE LANGUAGE

CREATE LANGUAGE

7

SQL - Language Statements

CREATE LANGUAGE

define a new procedural language

CREATE [ OR REPLACE ] [ TRUSTED ] [ PROCEDURAL ] LANGUAGE

name

HANDLER

call_handler

[ INLINE

inline_handler

] [ VALIDATOR

valfunction

] CREATE [ OR REPLACE ] [ TRUSTED ] [ PROCEDURAL ] LANGUAGE

name

Description

CREATE LANGUAGE registers a new procedural language with a PostgreSQL database. Subsequently, functions and procedures can be defined in this new language.

CREATE LANGUAGE effectively associates the language name with handler function(s) that are responsible for executing functions written in the language. Refer to ??? for more information about language handlers.

CREATE OR REPLACE LANGUAGE will either create a new language, or replace an existing definition. If the language already exists, its parameters are updated according to the command, but the language's ownership and permissions settings do not change, and any existing functions written in the language are assumed to still be valid.

One must have the PostgreSQL superuser privilege to register a new language or change an existing language's parameters. However, once the language is created it is valid to assign ownership of it to a non-superuser, who may then drop it, change its permissions, rename it, or assign it to a new owner. (Do not, however, assign ownership of the underlying C functions to a non-superuser; that would create a privilege escalation path for that user.)

The form of CREATE LANGUAGE that does not supply any handler function is obsolete. For backwards compatibility with old dump files, it is interpreted as CREATE EXTENSION. That will work if the language has been packaged into an extension of the same name, which is the conventional way to set up procedural languages.

Parameters

TRUSTED
TRUSTED specifies that the language does not grant access to data that the user would not otherwise have. If this key word is omitted when registering the language, only users with the PostgreSQL superuser privilege can use this language to create new functions.

PROCEDURAL
This is a noise word.

<name>
The name of the new procedural language. The name must be unique among the languages in the database.

HANDLER <call_handler>
<call_handler> is the name of a previously registered function that will be called to execute the procedural language's functions. The call handler for a procedural language must be written in a compiled language such as C with version 1 call convention and registered with PostgreSQL as a function taking no arguments and returning the language_handler type, a placeholder type that is simply used to identify the function as a call handler.

INLINE <inline_handler>
<inline_handler> is the name of a previously registered function that will be called to execute an anonymous code block (DO command) in this language. If no <inline_handler> function is specified, the language does not support anonymous code blocks. The handler function must take one argument of type internal, which will be the DO command's internal representation, and it will typically return void. The return value of the handler is ignored.

VALIDATOR <valfunction>
<valfunction> is the name of a previously registered function that will be called when a new function in the language is created, to validate the new function. If no validator function is specified, then a new function will not be checked when it is created. The validator function must take one argument of type oid, which will be the OID of the to-be-created function, and will typically return void.

A validator function would typically inspect the function body for syntactical correctness, but it can also look at other properties of the function, for example if the language cannot handle certain argument types. To signal an error, the validator function should use the ereport() function. The return value of the function is ignored.

Notes

Use DROP LANGUAGE to drop procedural languages.

The system catalog pg_language (see ???) records information about the currently installed languages. Also, the psql command \dL lists the installed languages.

To create functions in a procedural language, a user must have the USAGE privilege for the language. By default, USAGE is granted to PUBLIC (i.e., everyone) for trusted languages. This can be revoked if desired.

Procedural languages are local to individual databases. However, a language can be installed into the template1 database, which will cause it to be available automatically in all subsequently-created databases.

Examples

A minimal sequence for creating a new procedural language is:

CREATE FUNCTION plsample_call_handler() RETURNS language_handler
    AS '$libdir/plsample'
    LANGUAGE C;
CREATE LANGUAGE plsample
    HANDLER plsample_call_handler;

Typically that would be written in an extension's creation script, and users would do this to install the extension:

CREATE EXTENSION plsample;

Compatibility

CREATE LANGUAGE is a PostgreSQL extension.

See Also

create_materialized_view

CREATE MATERIALIZED VIEW

CREATE MATERIALIZED VIEW

7

SQL - Language Statements

CREATE MATERIALIZED VIEW

define a new materialized view

CREATE MATERIALIZED VIEW [ IF NOT EXISTS ]

table_name

[ (

column_name

[, ...] ) ] [ USING

method

] [ WITH (

storage_parameter

[=

value

] [, ... ] ) ] [ TABLESPACE

tablespace_name

] AS

query

[ WITH [ NO ] DATA ]

Description

CREATE MATERIALIZED VIEW defines a materialized view of a query. The query is executed and used to populate the view at the time the command is issued (unless WITH NO DATA is used) and may be refreshed later using REFRESH MATERIALIZED VIEW.

CREATE MATERIALIZED VIEW is similar to CREATE TABLE AS, except that it also remembers the query used to initialize the view, so that it can be refreshed later upon demand. A materialized view has many of the same properties as a table, but there is no support for temporary materialized views.

CREATE MATERIALIZED VIEW requires CREATE privilege on the schema used for the materialized view.

Parameters

IF NOT EXISTS
Do not throw an error if a materialized view with the same name already exists. A notice is issued in this case. Note that there is no guarantee that the existing materialized view is anything like the one that would have been created.

<table_name>
The name (optionally schema-qualified) of the materialized view to be created. The name must be distinct from the name of any other relation (table, sequence, index, view, materialized view, or foreign table) in the same schema.

<column_name>
The name of a column in the new materialized view. If column names are not provided, they are taken from the output column names of the query.

USING method
This optional clause specifies the table access method to use to store the contents for the new materialized view; the method needs be an access method of type TABLE. See ??? for more information. If this option is not specified, the default table access method is chosen for the new materialized view. See ??? for more information.

WITH ( storage_parameter [= value] [, ... ] )
This clause specifies optional storage parameters for the new materialized view; see ??? in the ??? documentation for more information. All parameters supported for CREATE TABLE are also supported for CREATE MATERIALIZED VIEW. See ??? for more information.

TABLESPACE tablespace_name
The <tablespace_name> is the name of the tablespace in which the new materialized view is to be created. If not specified, ??? is consulted.

<query>
A SELECT, TABLE, or VALUES command. This query will run within a security-restricted operation; in particular, calls to functions that themselves create temporary tables will fail. Also, while the query is running, the ??? is temporarily changed to pg_catalog, pg_temp.

WITH [ NO ] DATA
This clause specifies whether or not the materialized view should be populated at creation time. If not, the materialized view will be flagged as unscannable and cannot be queried until REFRESH MATERIALIZED VIEW is used.

Compatibility

CREATE MATERIALIZED VIEW is a PostgreSQL extension.

See Also

create_opclass

CREATE OPERATOR CLASS

CREATE OPERATOR CLASS

7

SQL - Language Statements

CREATE OPERATOR CLASS

define a new operator class

CREATE OPERATOR CLASS

name

[ DEFAULT ] FOR TYPE

data_type

USING

index_method

[ FAMILY

family_name

] AS { OPERATOR

strategy_number

operator_name

[ (

op_type

,

op_type

) ] [ FOR SEARCH | FOR ORDER BY

sort_family_name

] | FUNCTION

support_number

[ (

op_type

[ ,

op_type

] ) ]

function_name

(

argument_type

[, ...] ) | STORAGE

storage_type

} [, ... ]

Description

CREATE OPERATOR CLASS creates a new operator class. An operator class defines how a particular data type can be used with an index. The operator class specifies that certain operators will fill particular roles or “strategies” for this data type and this index method. The operator class also specifies the support functions to be used by the index method when the operator class is selected for an index column. All the operators and functions used by an operator class must be defined before the operator class can be created.

If a schema name is given then the operator class is created in the specified schema. Otherwise it is created in the current schema. Two operator classes in the same schema can have the same name only if they are for different index methods.

The user who defines an operator class becomes its owner. Presently, the creating user must be a superuser. (This restriction is made because an erroneous operator class definition could confuse or even crash the server.)

CREATE OPERATOR CLASS does not presently check whether the operator class definition includes all the operators and functions required by the index method, nor whether the operators and functions form a self-consistent set. It is the user's responsibility to define a valid operator class.

Refer to ??? for further information.

Parameters

<name>
The name of the operator class to be created. The name can be schema-qualified.

DEFAULT
If present, the operator class will become the default operator class for its data type. At most one operator class can be the default for a specific data type and index method.

<data_type>
The column data type that this operator class is for.

<index_method>
The name of the index method this operator class is for.

<family_name>
The name of the existing operator family to add this operator class to. If not specified, a family named the same as the operator class is used (creating it, if it doesn't already exist).

<strategy_number>
The index method's strategy number for an operator associated with the operator class.

<operator_name>
The name (optionally schema-qualified) of an operator associated with the operator class.

<op_type>
In an OPERATOR clause, the operand data type(s) of the operator, or NONE to signify a prefix operator. The operand data types can be omitted in the normal case where they are the same as the operator class's data type.

In a FUNCTION clause, the operand data type(s) the function is intended to support, if different from the input data type(s) of the function (for B-tree comparison functions and hash functions) or the class's data type (for B-tree sort support functions, B-tree equal image functions, and all functions in GiST, SP-GiST, GIN and BRIN operator classes). These defaults are correct, and so <op_type> need not be specified in FUNCTION clauses, except for the case of a B-tree sort support function that is meant to support cross-data-type comparisons.

<sort_family_name>
The name (optionally schema-qualified) of an existing btree operator family that describes the sort ordering associated with an ordering operator.

If neither FOR SEARCH nor FOR ORDER BY is specified, FOR SEARCH is the default.

<support_number>
The index method's support function number for a function associated with the operator class.

<function_name>
The name (optionally schema-qualified) of a function that is an index method support function for the operator class.

<argument_type>
The parameter data type(s) of the function.

<storage_type>
The data type actually stored in the index. Normally this is the same as the column data type, but some index methods (currently GiST, GIN, SP-GiST and BRIN) allow it to be different. The STORAGE clause must be omitted unless the index method allows a different type to be used. If the column <data_type> is specified as anyarray, the <storage_type> can be declared as anyelement to indicate that the index entries are members of the element type belonging to the actual array type that each particular index is created for.

The OPERATOR, FUNCTION, and STORAGE clauses can appear in any order.

Notes

Because the index machinery does not check access permissions on functions before using them, including a function or operator in an operator class is tantamount to granting public execute permission on it. This is usually not an issue for the sorts of functions that are useful in an operator class.

The operators should not be defined by SQL functions. An SQL function is likely to be inlined into the calling query, which will prevent the optimizer from recognizing that the query matches an index.

Examples

The following example command defines a GiST index operator class for the data type _int4 (array of int4). See the ??? module for the complete example.

CREATE OPERATOR CLASS gist__int_ops
    DEFAULT FOR TYPE _int4 USING gist AS
        OPERATOR        3       &&,
        OPERATOR        6       = (anyarray, anyarray),
        OPERATOR        7       @>,
        OPERATOR        8       <@,
        OPERATOR        20      @@ (_int4, query_int),
        FUNCTION        1       g_int_consistent (internal, _int4, smallint, oid, internal),
        FUNCTION        2       g_int_union (internal, internal),
        FUNCTION        3       g_int_compress (internal),
        FUNCTION        4       g_int_decompress (internal),
        FUNCTION        5       g_int_penalty (internal, internal, internal),
        FUNCTION        6       g_int_picksplit (internal, internal),
        FUNCTION        7       g_int_same (_int4, _int4, internal);

Compatibility

CREATE OPERATOR CLASS is a PostgreSQL extension. There is no CREATE OPERATOR CLASS statement in the SQL standard.

See Also

create_operator

CREATE OPERATOR

CREATE OPERATOR

7

SQL - Language Statements

CREATE OPERATOR

define a new operator

CREATE OPERATOR

name

( {FUNCTION|PROCEDURE} =

function_name

[, LEFTARG =

left_type

] , RIGHTARG =

right_type

[, COMMUTATOR =

com_op

] [, NEGATOR =

neg_op

] [, RESTRICT =

res_proc

] [, JOIN =

join_proc

] [, HASHES ] [, MERGES ] )

Description

CREATE OPERATOR defines a new operator, <name>. The user who defines an operator becomes its owner. If a schema name is given then the operator is created in the specified schema. Otherwise it is created in the current schema.

The operator name is a sequence of up to NAMEDATALEN-1 (63 by default) characters from the following list:


+ - * / < > = ~ ! @ # % ^ & | ` ?\

There are a few restrictions on your choice of name:

The operator != is mapped to <> on input, so these two names are always equivalent.

For binary operators, both LEFTARG and RIGHTARG must be defined. For prefix operators, only RIGHTARG must be defined. The <function_name> function must have been previously defined using CREATE FUNCTION and must be defined to accept the correct number of arguments (either one or two) of the indicated types.

In the syntax of CREATE OPERATOR, the keywords FUNCTION and PROCEDURE are equivalent, but the referenced function must in any case be a function, not a procedure. The use of the keyword PROCEDURE here is historical and deprecated.

The other clauses specify optional operator optimization attributes. Their meaning is detailed in ???.

To be able to create an operator, you must have USAGE privilege on the argument types and the return type, as well as EXECUTE privilege on the underlying function. If a commutator or negator operator is specified, you must own those operators.

Parameters

<name>
The name of the operator to be defined. See above for allowable characters. The name can be schema-qualified, for example CREATE OPERATOR myschema.+ (...). If not, then the operator is created in the current schema. Two operators in the same schema can have the same name if they operate on different data types. This is called overloading.

<function_name>
The function used to implement this operator.

<left_type>
The data type of the operator's left operand, if any. This option would be omitted for a prefix operator.

<right_type>
The data type of the operator's right operand.

<com_op>
The commutator of this operator.

<neg_op>
The negator of this operator.

<res_proc>
The restriction selectivity estimator function for this operator.

<join_proc>
The join selectivity estimator function for this operator.

HASHES
Indicates this operator can support a hash join.

MERGES
Indicates this operator can support a merge join.

To give a schema-qualified operator name in <com_op> or the other optional arguments, use the OPERATOR() syntax, for example:

COMMUTATOR = OPERATOR(myschema.===) ,

Notes

Refer to ??? and ??? for further information.

When you are defining a self-commutative operator, you just do it. When you are defining a pair of commutative operators, things are a little trickier: how can the first one to be defined refer to the other one, which you haven't defined yet? There are three solutions to this problem:

In all three cases, you must own both operators in order to mark them as commutators.

Pairs of negator operators can be defined using the same methods as for commutator pairs.

It is not possible to specify an operator's lexical precedence in CREATE OPERATOR, because the parser's precedence behavior is hard-wired. See ??? for precedence details.

The obsolete options SORT1, SORT2, LTCMP, and GTCMP were formerly used to specify the names of sort operators associated with a merge-joinable operator. This is no longer necessary, since information about associated operators is found by looking at B-tree operator families instead. If one of these options is given, it is ignored except for implicitly setting MERGES true.

Use DROP OPERATOR to delete user-defined operators from a database. Use ALTER OPERATOR to modify operators in a database.

Examples

The following command defines a new operator, area-equality, for the data type box:

CREATE OPERATOR === (
    LEFTARG = box,
    RIGHTARG = box,
    FUNCTION = area_equal_function,
    COMMUTATOR = ===,
    NEGATOR = !==,
    RESTRICT = area_restriction_function,
    JOIN = area_join_function,
    HASHES, MERGES
);

Compatibility

CREATE OPERATOR is a PostgreSQL extension. There are no provisions for user-defined operators in the SQL standard.

See Also

create_opfamily

CREATE OPERATOR FAMILY

CREATE OPERATOR FAMILY

7

SQL - Language Statements

CREATE OPERATOR FAMILY

define a new operator family

CREATE OPERATOR FAMILY

name

USING

index_method

Description

CREATE OPERATOR FAMILY creates a new operator family. An operator family defines a collection of related operator classes, and perhaps some additional operators and support functions that are compatible with these operator classes but not essential for the functioning of any individual index. (Operators and functions that are essential to indexes should be grouped within the relevant operator class, rather than being “loose” in the operator family. Typically, single-data-type operators are bound to operator classes, while cross-data-type operators can be loose in an operator family containing operator classes for both data types.)

The new operator family is initially empty. It should be populated by issuing subsequent CREATE OPERATOR CLASS commands to add contained operator classes, and optionally ALTER OPERATOR FAMILY commands to add “loose” operators and their corresponding support functions.

If a schema name is given then the operator family is created in the specified schema. Otherwise it is created in the current schema. Two operator families in the same schema can have the same name only if they are for different index methods.

The user who defines an operator family becomes its owner. Presently, the creating user must be a superuser. (This restriction is made because an erroneous operator family definition could confuse or even crash the server.)

Refer to ??? for further information.

Parameters

<name>
The name of the operator family to be created. The name can be schema-qualified.

<index_method>
The name of the index method this operator family is for.

Compatibility

CREATE OPERATOR FAMILY is a PostgreSQL extension. There is no CREATE OPERATOR FAMILY statement in the SQL standard.

See Also

create_policy

CREATE POLICY

CREATE POLICY

7

SQL - Language Statements

CREATE POLICY

define a new row-level security policy for a table

CREATE POLICY

name

ON

table_name

[ AS { PERMISSIVE | RESTRICTIVE } ] [ FOR { ALL | SELECT | INSERT | UPDATE | DELETE } ] [ TO {

role_name

| PUBLIC | CURRENT_ROLE | CURRENT_USER | SESSION_USER } [, ...] ] [ USING (

using_expression

) ] [ WITH CHECK (

check_expression

) ]

Description

The CREATE POLICY command defines a new row-level security policy for a table. Note that row-level security must be enabled on the table (using ALTER TABLE ... ENABLE ROW LEVEL SECURITY) in order for created policies to be applied.

A policy grants the permission to select, insert, update, or delete rows that match the relevant policy expression. Existing table rows are checked against the expression specified in USING, while new rows that would be created via INSERT or UPDATE are checked against the expression specified in WITH CHECK. When a USING expression returns true for a given row then that row is visible to the user, while if false or null is returned then the row is not visible. Typically, no error occurs when a row is not visible, but see Policies Applied by Command Type for exceptions. When a WITH CHECK expression returns true for a row then that row is inserted or updated, while if false or null is returned then an error occurs.

For INSERT, UPDATE, and MERGE statements, WITH CHECK expressions are enforced after BEFORE triggers are fired, and before any actual data modifications are made. Thus a BEFORE ROW trigger may modify the data to be inserted, affecting the result of the security policy check. WITH CHECK expressions are enforced before any other constraints.

Policy names are per-table. Therefore, one policy name can be used for many different tables and have a definition for each table which is appropriate to that table.

Policies can be applied for specific commands or for specific roles. The default for newly created policies is that they apply for all commands and roles, unless otherwise specified. Multiple policies may apply to a single command; see below for more details. Policies Applied by Command Type summarizes how the different types of policy apply to specific commands.

For policies that can have both USING and WITH CHECK expressions (ALL and UPDATE), if no WITH CHECK expression is defined, then the USING expression will be used both to determine which rows are visible (normal USING case) and which new rows will be allowed to be added (WITH CHECK case).

If row-level security is enabled for a table, but no applicable policies exist, a “default deny” policy is assumed, so that no rows will be visible or updatable.

Parameters

<name>
The name of the policy to be created. This must be distinct from the name of any other policy for the table.

<table_name>
The name (optionally schema-qualified) of the table the policy applies to.

PERMISSIVE
Specify that the policy is to be created as a permissive policy. All permissive policies which are applicable to a given query will be combined together using the Boolean “OR” operator. By creating permissive policies, administrators can add to the set of records which can be accessed. Policies are permissive by default.

RESTRICTIVE
Specify that the policy is to be created as a restrictive policy. All restrictive policies which are applicable to a given query will be combined together using the Boolean “AND” operator. By creating restrictive policies, administrators can reduce the set of records which can be accessed as all restrictive policies must be passed for each record.

Note that there needs to be at least one permissive policy to grant access to records before restrictive policies can be usefully used to reduce that access. If only restrictive policies exist, then no records will be accessible. When a mix of permissive and restrictive policies are present, a record is only accessible if at least one of the permissive policies passes, in addition to all the restrictive policies.

<command>
The command to which the policy applies. Valid options are ALL, SELECT, INSERT, UPDATE, and DELETE. ALL is the default. See below for specifics regarding how these are applied.

<role_name>
The role(s) to which the policy is to be applied. The default is PUBLIC, which will apply the policy to all roles.

<using_expression>
Any SQL conditional expression (returning boolean). The conditional expression cannot contain any aggregate or window functions. This expression will be added to queries that refer to the table if row-level security is enabled. Rows for which the expression returns true will be visible. Any rows for which the expression returns false or null will not be visible to the user (in a SELECT), and will not be available for modification (in an UPDATE or DELETE). Typically, such rows are silently suppressed; no error is reported (but see Policies Applied by Command Type for exceptions).

<check_expression>
Any SQL conditional expression (returning boolean). The conditional expression cannot contain any aggregate or window functions. This expression will be used in INSERT and UPDATE queries against the table if row-level security is enabled. Only rows for which the expression evaluates to true will be allowed. An error will be thrown if the expression evaluates to false or null for any of the records inserted or any of the records that result from the update. Note that the <check_expression> is evaluated against the proposed new contents of the row, not the original contents.

Per-Command Policies

ALL
Using ALL for a policy means that it will apply to all commands, regardless of the type of command. If an ALL policy exists and more specific policies exist, then both the ALL policy and the more specific policy (or policies) will be applied. Additionally, ALL policies will be applied to both the selection side of a query and the modification side, using the USING expression for both cases if only a USING expression has been defined.

As an example, if an UPDATE is issued, then the ALL policy will be applicable both to what the UPDATE will be able to select as rows to be updated (applying the USING expression), and to the resulting updated rows, to check if they are permitted to be added to the table (applying the WITH CHECK expression, if defined, and the USING expression otherwise). If an INSERT or UPDATE command attempts to add rows to the table that do not pass the ALL policy's WITH CHECK expression (or its USING expression, if it does not have a WITH CHECK expression), the entire command will be aborted.

SELECT
Using SELECT for a policy means that it will apply to SELECT queries and whenever SELECT permissions are required on the relation the policy is defined for. The result is that only those records from the relation that pass the SELECT policy will be returned during a SELECT query, and that queries that require SELECT permissions, such as UPDATE, DELETE, and MERGE, will also only see those records that are allowed by the SELECT policy. A SELECT policy cannot have a WITH CHECK expression, as it only applies in cases where records are being retrieved from the relation, except as described below.

If a data-modifying query has a RETURNING clause, SELECT permissions are required on the relation, and any newly inserted or updated rows from the relation must satisfy the relation's SELECT policies in order to be available to the RETURNING clause. If a newly inserted or updated row does not satisfy the relation's SELECT policies, an error will be thrown (inserted or updated rows to be returned are never silently ignored).

If an INSERT has an ON CONFLICT DO SELECT/UPDATE clause, or an ON CONFLICT DO NOTHING clause with an arbiter index or constraint specification, then SELECT permissions are required on the relation, and the rows proposed for insertion are checked using the relation's SELECT policies. If a row proposed for insertion does not satisfy the relation's SELECT policies, an error is thrown (the INSERT is never silently avoided). In addition, if the UPDATE path is taken, the row to be updated and the new updated row are checked against the relation's SELECT policies, and an error is thrown if they are not satisfied (an auxiliary UPDATE is never silently avoided).

A MERGE command requires SELECT permissions on both the source and target relations, and so each relation's SELECT policies are applied before they are joined, and the MERGE actions will only see those records that are allowed by those policies. In addition, if an UPDATE action is executed, the target relation's SELECT policies are applied to the updated row, as for a standalone UPDATE, except that an error is thrown if they are not satisfied.

INSERT
Using INSERT for a policy means that it will apply to INSERT commands and MERGE commands that contain INSERT actions. Rows being inserted that do not pass this policy will result in a policy violation error, and the entire INSERT command will be aborted. An INSERT policy cannot have a USING expression, as it only applies in cases where records are being added to the relation.

Note that an INSERT with an ON CONFLICT clause will check the INSERT policies' WITH CHECK expressions for all rows proposed for insertion, regardless of whether or not they end up being inserted.

UPDATE
Using UPDATE for a policy means that it will apply to UPDATE and SELECT FOR UPDATE/SHARE commands, as well as auxiliary ON CONFLICT DO UPDATE and ON CONFLICT DO SELECT FOR UPDATE/SHARE clauses of INSERT commands, and MERGE commands containing UPDATE actions. Since an UPDATE command involves pulling an existing record and replacing it with a new modified record, UPDATE policies accept both a USING expression and a WITH CHECK expression. The USING expression determines which records the UPDATE command will see to operate against, while the WITH CHECK expression defines which modified rows are allowed to be stored back into the relation.

Any rows whose updated values do not pass the WITH CHECK expression will cause an error, and the entire command will be aborted. If only a USING clause is specified, then that clause will be used for both USING and WITH CHECK cases.

Typically an UPDATE command also needs to read data from columns in the relation being updated (e.g., in a WHERE clause or a RETURNING clause, or in an expression on the right hand side of the SET clause). In this case, SELECT rights are also required on the relation being updated, and the appropriate SELECT or ALL policies will be applied in addition to the UPDATE policies. Thus the user must have access to the row(s) being updated through a SELECT or ALL policy in addition to being granted permission to update the row(s) via an UPDATE or ALL policy.

When an INSERT command has an auxiliary ON CONFLICT DO UPDATE clause, if the UPDATE path is taken, the row to be updated is first checked against the USING expressions of any UPDATE policies, and then the new updated row is checked against the WITH CHECK expressions. Note, however, that unlike a standalone UPDATE command, if the existing row does not pass the USING expressions, an error will be thrown (the UPDATE path will never be silently avoided). The same applies to an UPDATE action of a MERGE command.

DELETE
Using DELETE for a policy means that it will apply to DELETE commands and MERGE commands containing DELETE actions. For a DELETE command, only rows that pass this policy will be seen by the DELETE command. There can be rows that are visible through a SELECT policy that are not available for deletion, if they do not pass the USING expression for the DELETE policy. Note, however, that a DELETE action in a MERGE command will see rows that are visible through SELECT policies, and if the DELETE policy does not pass for such a row, an error will be thrown.

In most cases a DELETE command also needs to read data from columns in the relation that it is deleting from (e.g., in a WHERE clause or a RETURNING clause). In this case, SELECT rights are also required on the relation, and the appropriate SELECT or ALL policies will be applied in addition to the DELETE policies. Thus the user must have access to the row(s) being deleted through a SELECT or ALL policy in addition to being granted permission to delete the row(s) via a DELETE or ALL policy.

A DELETE policy cannot have a WITH CHECK expression, as it only applies in cases where records are being deleted from the relation, so that there is no new row to check.

Policies Applied by Command Type summarizes how the different types of policy apply to specific commands. In the table, “check” means that the policy expression is checked and an error is thrown if it returns false or null, whereas “filter” means that the row is silently ignored if the policy expression returns false or null.

Policies Applied by Command Type
Command SELECT/ALL policy
USING expression
SELECT / COPY ... TO Filter existing row
SELECT FOR UPDATE/SHARE Filter existing row
INSERT Check new row 1
UPDATE Filter existing row  & check new row 
DELETE Filter existing row 
INSERT ... ON CONFLICT Check new row 23
ON CONFLICT DO UPDATE Check existing & new rows 4
ON CONFLICT DO SELECT Check existing row
ON CONFLICT DO SELECT FOR UPDATE/SHARE Check existing row
MERGE Filter source & target rows
MERGE ... THEN INSERT Check new row 
MERGE ... THEN UPDATE Check new row
MERGE ... THEN DELETE —

  1. If read access is required to either the existing or new row (for example, a WHERE or RETURNING clause that refers to columns from the relation).↩︎

  2. If an arbiter index or constraint is specified.↩︎

  3. Row proposed for insertion is checked regardless of whether or not a conflict occurs.↩︎

  4. New row of the auxiliary UPDATE command, which might be different from the new row of the original INSERT command.↩︎

Application of Multiple Policies

When multiple policies of different command types apply to the same command (for example, SELECT and UPDATE policies applied to an UPDATE command), then the user must have both types of permissions (for example, permission to select rows from the relation as well as permission to update them). Thus the expressions for one type of policy are combined with the expressions for the other type of policy using the AND operator.

When multiple policies of the same command type apply to the same command, then there must be at least one PERMISSIVE policy granting access to the relation, and all of the RESTRICTIVE policies must pass. Thus all the PERMISSIVE policy expressions are combined using OR, all the RESTRICTIVE policy expressions are combined using AND, and the results are combined using AND. If there are no PERMISSIVE policies, then access is denied.

Note that, for the purposes of combining multiple policies, ALL policies are treated as having the same type as whichever other type of policy is being applied.

For example, in an UPDATE command requiring both SELECT and UPDATE permissions, if there are multiple applicable policies of each type, they will be combined as follows:

expression from RESTRICTIVE SELECT/ALL policy 1
AND
expression from RESTRICTIVE SELECT/ALL policy 2
AND
...
AND
(
  expression from PERMISSIVE SELECT/ALL policy 1
  OR
  expression from PERMISSIVE SELECT/ALL policy 2
  OR
  ...
)
AND
expression from RESTRICTIVE UPDATE/ALL policy 1
AND
expression from RESTRICTIVE UPDATE/ALL policy 2
AND
...
AND
(
  expression from PERMISSIVE UPDATE/ALL policy 1
  OR
  expression from PERMISSIVE UPDATE/ALL policy 2
  OR
  ...
)

Notes

You must be the owner of a table to create or change policies for it.

While policies will be applied for explicit queries against tables in the database, they are not applied when the system is performing internal referential integrity checks or validating constraints. This means there are indirect ways to determine that a given value exists. An example of this is attempting to insert a duplicate value into a column that is a primary key or has a unique constraint. If the insert fails then the user can infer that the value already exists. (This example assumes that the user is permitted by policy to insert records which they are not allowed to see.) Another example is where a user is allowed to insert into a table which references another, otherwise hidden table. Existence can be determined by the user inserting values into the referencing table, where success would indicate that the value exists in the referenced table. These issues can be addressed by carefully crafting policies to prevent users from being able to insert, delete, or update records at all which might possibly indicate a value they are not otherwise able to see, or by using generated values (e.g., surrogate keys) instead of keys with external meanings.

Generally, the system will enforce filter conditions imposed using security policies prior to qualifications that appear in user queries, in order to prevent inadvertent exposure of the protected data to user-defined functions which might not be trustworthy. However, functions and operators marked by the system (or the system administrator) as LEAKPROOF may be evaluated before policy expressions, as they are assumed to be trustworthy.

Since policy expressions are added to the user's query directly, they will be run with the rights of the user running the overall query. Therefore, users who are using a given policy must be able to access any tables or functions referenced in the expression or they will simply receive a permission denied error when attempting to query the table that has row-level security enabled. This does not change how views work, however. As with normal queries and views, permission checks and policies for the tables which are referenced by a view will use the view owner's rights and any policies which apply to the view owner, except if the view is defined using the security_invoker option (see CREATE VIEW).

No separate policy exists for MERGE. Instead, the policies defined for SELECT, INSERT, UPDATE, and DELETE are applied while executing MERGE, depending on the actions that are performed.

Additional discussion and practical examples can be found in ???.

Compatibility

CREATE POLICY is a PostgreSQL extension.

See Also

create_procedure

CREATE PROCEDURE

CREATE PROCEDURE

7

SQL - Language Statements

CREATE PROCEDURE

define a new procedure

CREATE [ OR REPLACE ] PROCEDURE

name

( [ [

argmode

] [

argname

]

argtype

[ { DEFAULT | = }

default_expr

] [, ...] ] ) { LANGUAGE

lang_name

| TRANSFORM { FOR TYPE

type_name

} [, ... ] | [ EXTERNAL ] SECURITY INVOKER | [ EXTERNAL ] SECURITY DEFINER | SET

configuration_parameter

{ TO

value

| =

value

| FROM CURRENT } | AS '

definition

' | AS '

obj_file

', '

' |

sql_body

} ...

Description

CREATE PROCEDURE defines a new procedure. CREATE OR REPLACE PROCEDURE will either create a new procedure, or replace an existing definition. To be able to define a procedure, the user must have the USAGE privilege on the language.

If a schema name is included, then the procedure is created in the specified schema. Otherwise it is created in the current schema. The name of the new procedure must not match any existing procedure or function with the same input argument types in the same schema. However, procedures and functions of different argument types can share a name (this is called overloading).

To replace the current definition of an existing procedure, use CREATE OR REPLACE PROCEDURE. It is not possible to change the name or argument types of a procedure this way (if you tried, you would actually be creating a new, distinct procedure).

When CREATE OR REPLACE PROCEDURE is used to replace an existing procedure, the ownership and permissions of the procedure do not change. All other procedure properties are assigned the values specified or implied in the command. You must own the procedure to replace it (this includes being a member of the owning role).

The user that creates the procedure becomes the owner of the procedure.

To be able to create a procedure, you must have USAGE privilege on the argument types.

Refer to ??? for further information on writing procedures.

Parameters

<name>
The name (optionally schema-qualified) of the procedure to create.

<argmode>
The mode of an argument: IN, OUT, INOUT, or VARIADIC. If omitted, the default is IN.

<argname>
The name of an argument.

<argtype>
The data type(s) of the procedure's arguments (optionally schema-qualified), if any. The argument types can be base, composite, or domain types, or can reference the type of a table column.

Depending on the implementation language it might also be allowed to specify “pseudo-types” such as cstring. Pseudo-types indicate that the actual argument type is either incompletely specified, or outside the set of ordinary SQL data types.

The type of a column is referenced by writing table_name.column_name%TYPE. Using this feature can sometimes help make a procedure independent of changes to the definition of a table.

<default_expr>
An expression to be used as default value if the parameter is not specified. The expression has to be coercible to the argument type of the parameter. All input parameters following a parameter with a default value must have default values as well.

<lang_name>
The name of the language that the procedure is implemented in. It can be sql, c, internal, or the name of a user-defined procedural language, e.g., plpgsql. The default is sql if <sql_body> is specified. Enclosing the name in single quotes is deprecated and requires matching case.

TRANSFORM { FOR TYPE type_name } [, ... ]
Lists which transforms a call to the procedure should apply. Transforms convert between SQL types and language-specific data types; see ???. Procedural language implementations usually have hardcoded knowledge of the built-in types, so those don't need to be listed here. If a procedural language implementation does not know how to handle a type and no transform is supplied, it will fall back to a default behavior for converting data types, but this depends on the implementation.

EXTERNAL SECURITY INVOKER; EXTERNAL SECURITY DEFINER
SECURITY INVOKER indicates that the procedure is to be executed with the privileges of the user that calls it. That is the default. SECURITY DEFINER specifies that the procedure is to be executed with the privileges of the user that owns it.

The key word EXTERNAL is allowed for SQL conformance, but it is optional since, unlike in SQL, this feature applies to all procedures not only external ones.

A SECURITY DEFINER procedure cannot execute transaction control statements (for example, COMMIT and ROLLBACK, depending on the language).

<configuration_parameter>; <value>
The SET clause causes the specified configuration parameter to be set to the specified value when the procedure is entered, and then restored to its prior value when the procedure exits. SET FROM CURRENT saves the value of the parameter that is current when CREATE PROCEDURE is executed as the value to be applied when the procedure is entered.

If a SET clause is attached to a procedure, then the effects of a SET LOCAL command executed inside the procedure for the same variable are restricted to the procedure: the configuration parameter's prior value is still restored at procedure exit. However, an ordinary SET command (without LOCAL) overrides the SET clause, much as it would do for a previous SET LOCAL command: the effects of such a command will persist after procedure exit, unless the current transaction is rolled back.

If a SET clause is attached to a procedure, then that procedure cannot execute transaction control statements (for example, COMMIT and ROLLBACK, depending on the language).

See ??? and ??? for more information about allowed parameter names and values.

<definition>
A string constant defining the procedure; the meaning depends on the language. It can be an internal procedure name, the path to an object file, an SQL command, or text in a procedural language.

It is often helpful to use dollar quoting (see ???) to write the procedure definition string, rather than the normal single quote syntax. Without dollar quoting, any single quotes or backslashes in the procedure definition must be escaped by doubling them.

When repeated CREATE PROCEDURE calls refer to the same object file, the file is only loaded once per session. To unload and reload the file (perhaps during development), start a new session.

<sql_body>
The body of a LANGUAGE SQL procedure. This should be a block

BEGIN ATOMIC
  statement;
  statement;
  ...
  statement;
END

This is similar to writing the text of the procedure body as a string constant (see <definition> above), but there are some differences: This form only works for LANGUAGE SQL, the string constant form works for all languages. This form is parsed at procedure definition time, the string constant form is parsed at execution time; therefore this form cannot support polymorphic argument types and other constructs that are not resolvable at procedure definition time. This form tracks dependencies between the procedure and objects used in the procedure body, so DROP ... CASCADE will work correctly, whereas the form using string literals may leave dangling procedures. Finally, this form is more compatible with the SQL standard and other SQL implementations.

Notes

See ??? for more details on function creation that also apply to procedures.

Use ??? to execute a procedure.

Examples

CREATE PROCEDURE insert_data(a integer, b integer)
LANGUAGE SQL
AS $$
INSERT INTO tbl VALUES (a);
INSERT INTO tbl VALUES (b);
$$;

or

CREATE PROCEDURE insert_data(a integer, b integer)
LANGUAGE SQL
BEGIN ATOMIC
  INSERT INTO tbl VALUES (a);
  INSERT INTO tbl VALUES (b);
END;

and call like this:

CALL insert_data(1, 2);

Compatibility

A CREATE PROCEDURE command is defined in the SQL standard. The PostgreSQL implementation can be used in a compatible way but has many extensions. For details see also ???.

See Also

create_publication

CREATE PUBLICATION

CREATE PUBLICATION

7

SQL - Language Statements

CREATE PUBLICATION

define a new publication

CREATE PUBLICATION

name

[ FOR {

publication_object

[, ... ] |

publication_all_object

[, ... ] } ] [ WITH (

publication_parameter

[=

value

] [, ... ] ) ]

where

publication_object

is one of:

TABLE

table_and_columns

[, ... ] TABLES IN SCHEMA {

schema_name

| CURRENT_SCHEMA } [, ... ]

and

publication_all_object

is one of:

ALL TABLES [ EXCEPT (

except_table_object

[, ... ] ) ] ALL SEQUENCES

and

table_and_columns

is:

table_object

[ (

column_name

[, ... ] ) ] [ WHERE (

expression

) ]

and

except_table_object

is:

TABLE

table_object

[, ... ]

and

table_object

is:

[ ONLY ]

table_name

[ * ]

Description

CREATE PUBLICATION adds a new publication into the current database. The publication name must be distinct from the name of any existing publication in the current database.

A publication is essentially a group of tables whose data changes are intended to be replicated through logical replication. See ??? for details about how publications fit into the logical replication setup.

Parameters

<name>
The name of the new publication.

<table_name>
Name of an existing table.

<column_name>
Name of an existing column of <table_name>.

<schema_name>
Name of an existing schema.

FOR TABLE
Specifies a list of tables to add to the publication. If ONLY is specified before the <table_name>, only that table is added to the publication. If ONLY is not specified, the table and all its descendant tables (if any) are added. Optionally, * can be specified after the <table_name> to explicitly indicate that descendant tables are included. This does not apply to a partitioned table, however. The partitions of a partitioned table are always implicitly considered part of the publication, so they are never explicitly added to the publication.

If the optional WHERE clause is specified, it defines a row filter expression. Rows for which the <expression> evaluates to false or null will not be published. Note that parentheses are required around the expression. It has no effect on TRUNCATE commands.

When a column list is specified, only the named columns are replicated. The column list can contain stored generated columns as well. If the column list is omitted, the publication will replicate all non-generated columns (including any added in the future) by default. Stored generated columns can also be replicated if publish_generated_columns is set to stored. Specifying a column list has no effect on TRUNCATE commands. See ??? for details about column lists.

Only persistent base tables and partitioned tables can be part of a publication. Temporary tables, unlogged tables, foreign tables, materialized views, and regular views cannot be part of a publication.

Specifying a column list when the publication also publishes FOR TABLES IN SCHEMA is not supported.

When a partitioned table is added to a publication, all of its existing and future partitions are implicitly considered to be part of the publication. So, even operations that are performed directly on a partition are also published via publications that its ancestors are part of.

FOR TABLES IN SCHEMA
Marks the publication as one that replicates changes for all tables in the specified list of schemas, including tables created in the future.

Specifying a schema when the publication also publishes a table with a column list is not supported.

Only persistent base tables and partitioned tables present in the schema will be included as part of the publication. Temporary tables, unlogged tables, foreign tables, materialized views, and regular views from the schema will not be part of the publication.

When a partitioned table is published via a schema-level publication, all of its existing and future partitions are implicitly considered to be part of the publication, regardless of whether they are from the publication schema or not. So, even operations that are performed directly on a partition are also published via publications that its ancestors are part of.

FOR ALL TABLES
Marks the publication as one that replicates changes for all tables in the database, including tables created in the future. Tables listed in EXCEPT clause are excluded from the publication.

FOR ALL SEQUENCES
Marks the publication as one that synchronizes changes for all sequences in the database, including sequences created in the future.

Only persistent sequences are included in the publication. Temporary sequences and unlogged sequences are excluded from the publication.

EXCEPT
This clause specifies a list of tables to be excluded from the publication.

Once a table is excluded, the exclusion applies to that table regardless of its name or schema. Renaming the table or moving it to another schema using ALTER TABLE ... SET SCHEMA does not remove the exclusion.

For inherited tables, if ONLY is specified before the <table_name>, only that table is excluded from the publication. If ONLY is not specified, the table and all its descendant tables (if any) are excluded. Optionally, * can be specified after the <table_name> to explicitly indicate that descendant tables are excluded.

For partitioned tables, only the root partitioned table may be specified in EXCEPT. Doing so excludes the root table and all of its partitions from replication. The optional ONLY and * has no effect for partitioned tables.

There can be a case where a subscription includes multiple publications. In such a case, a table or partition that is included in one publication but excluded (explicitly or implicitly) by the EXCEPT clause of another is considered included for replication.

WITH ( publication_parameter [= value] [, ... ] )
This clause specifies optional parameters for a publication when publishing tables. This clause is not applicable to sequences. The following parameters are supported:

publish (string)
This parameter determines which DML operations will be published by the new publication to the subscribers. The value is a comma-separated list of operations. The allowed operations are insert, update, delete, and truncate. The default is to publish all actions, and so the default value for this option is 'insert, update, delete, truncate'.

This parameter only affects DML operations. In particular, the initial data synchronization (see ???) for logical replication does not take this parameter into account when copying existing table data.

publish_generated_columns (enum)
Specifies whether the generated columns present in the tables associated with the publication should be replicated. Possible values are none and stored.

The default is none meaning the generated columns present in the tables associated with publication will not be replicated.

If set to stored, the stored generated columns present in the tables associated with publication will be replicated.

[!NOTE] If the subscriber is from a release prior to 18, then initial table synchronization won't copy generated columns even if the parameter publish_generated_columns is stored in the publisher.

See ??? for more details about logical replication of generated columns.

publish_via_partition_root (boolean)
This parameter controls how changes to a partitioned table (or any of its partitions) are published. When set to true, changes are published using the identity and schema of the root partitioned table. When set to false (the default), changes are published using the identity and schema of the individual partitions where the changes actually occurred. Enabling this option allows the changes to be replicated into a non-partitioned table or into a partitioned table whose partition structure differs from that of the publisher.

There can be a case where a subscription combines multiple publications. If a partitioned table is published by any subscribed publications which set publish_via_partition_root = true, changes on this partitioned table (or on its partitions) will be published using the identity and schema of this partitioned table rather than that of the individual partitions.

This parameter also affects how row filters and column lists are chosen for partitions; see below for details.

If this is enabled, TRUNCATE operations performed directly on partitions are not replicated.

When specifying a parameter of type boolean, the = <value> part can be omitted, which is equivalent to specifying TRUE.

Notes

If FOR TABLE, FOR TABLES IN SCHEMA, FOR ALL TABLES or FOR ALL SEQUENCES are not specified, then the publication starts out with an empty set of tables. That is useful if tables or schemas are to be added later.

The creation of a publication does not start replication. It only defines a grouping and filtering logic for future subscribers.

To create a publication, the invoking user must have the CREATE privilege for the current database. (Of course, superusers bypass this check.)

To add a table to a publication, the invoking user must have ownership rights on the table. The FOR TABLES IN SCHEMA, FOR ALL TABLES and FOR ALL SEQUENCES clauses require the invoking user to be a superuser.

The tables added to a publication that publishes UPDATE and/or DELETE operations must have REPLICA IDENTITY defined. Otherwise those operations will be disallowed on those tables.

Any column list must include the REPLICA IDENTITY columns in order for UPDATE or DELETE operations to be published. There are no column list restrictions if the publication publishes only INSERT operations.

A row filter expression (i.e., the WHERE clause) must contain only columns that are covered by the REPLICA IDENTITY, in order for UPDATE and DELETE operations to be published. For publication of INSERT operations, any column may be used in the WHERE expression. The row filter allows simple expressions that don't have user-defined functions, user-defined operators, user-defined types, user-defined collations, non-immutable built-in functions, or references to system columns.

The generated columns that are part of REPLICA IDENTITY must be published explicitly either by listing them in the column list or by enabling the publish_generated_columns option, in order for UPDATE and DELETE operations to be published.

The row filter on a table becomes redundant if FOR TABLES IN SCHEMA is specified and the table belongs to the referred schema.

For published partitioned tables, the row filter for each partition is taken from the published partitioned table if the publication parameter publish_via_partition_root is true, or from the partition itself if it is false (the default). See ??? for details about row filters. Similarly, for published partitioned tables, the column list for each partition is taken from the published partitioned table if the publication parameter publish_via_partition_root is true, or from the partition itself if it is false.

For an INSERT ... ON CONFLICT command, the publication will publish the operation that results from the command. Depending on the outcome, it may be published as either INSERT or UPDATE, or it may not be published at all.

For a MERGE command, the publication will publish an INSERT, UPDATE, or DELETE for each row inserted, updated, or deleted.

ATTACHing a table into a partition tree whose root is published using a publication with publish_via_partition_root set to true does not result in the table's existing contents being replicated.

COPY ... FROM commands are published as INSERT operations.

DDL operations are not published.

The WHERE clause expression is executed with the role used for the replication connection.

Examples

Create a publication that publishes all changes in two tables:

CREATE PUBLICATION mypublication FOR TABLE users, departments;

Create a publication that publishes all changes from active departments:

CREATE PUBLICATION active_departments FOR TABLE departments WHERE (active IS TRUE);

Create a publication that publishes all changes in all tables:

CREATE PUBLICATION alltables FOR ALL TABLES;

Create a publication that only publishes INSERT operations in one table:

CREATE PUBLICATION insert_only FOR TABLE mydata
    WITH (publish = 'insert');

Create a publication that publishes all changes for tables users, departments and all changes for all the tables present in the schema production:

CREATE PUBLICATION production_publication FOR TABLE users, departments, TABLES IN SCHEMA production;

Create a publication that publishes all changes for all the tables present in the schemas marketing and sales:

CREATE PUBLICATION sales_publication FOR TABLES IN SCHEMA marketing, sales;

Create a publication that publishes all changes for table users, but replicates only columns user_id and firstname:

CREATE PUBLICATION users_filtered FOR TABLE users (user_id, firstname);

Create a publication that publishes all sequences for synchronization:

CREATE PUBLICATION all_sequences FOR ALL SEQUENCES;

Create a publication that publishes all changes in all tables, and all sequences for synchronization:

CREATE PUBLICATION all_tables_sequences FOR ALL TABLES, ALL SEQUENCES;

Create a publication that publishes all changes in all tables except users and departments:

CREATE PUBLICATION all_tables_except FOR ALL TABLES EXCEPT (TABLE users, departments);

Create a publication that publishes all sequences for synchronization, and all changes in all tables except users and departments:

CREATE PUBLICATION all_sequences_tables_except FOR ALL SEQUENCES, ALL TABLES EXCEPT (TABLE users, departments);

Compatibility

CREATE PUBLICATION is a PostgreSQL extension.

See Also

create_role

CREATE ROLE

CREATE ROLE

7

SQL - Language Statements

CREATE ROLE

define a new database role

CREATE ROLE

name

[ [ WITH ]

option

[ ... ] ]

where

option

can be:

SUPERUSER | NOSUPERUSER | CREATEDB | NOCREATEDB | CREATEROLE | NOCREATEROLE | INHERIT | NOINHERIT | LOGIN | NOLOGIN | REPLICATION | NOREPLICATION | BYPASSRLS | NOBYPASSRLS | CONNECTION LIMIT

connlimit

| [ ENCRYPTED ] PASSWORD '

password

' | PASSWORD NULL | VALID UNTIL '

timestamp

' | IN ROLE

role_name

[, ...] | ROLE

role_name

[, ...] | ADMIN

role_name

[, ...] | SYSID

uid

Description

CREATE ROLE adds a new role to a PostgreSQL database cluster. A role is an entity that can own database objects and have database privileges; a role can be considered a “user”, a “group”, or both depending on how it is used. Refer to ??? and ??? for information about managing users and authentication. You must have CREATEROLE privilege or be a database superuser to use this command.

Note that roles are defined at the database cluster level, and so are valid in all databases in the cluster.

During role creation it is possible to immediately assign the newly created role to be a member of an existing role, and also assign existing roles to be members of the newly created role. The rules for which initial role membership options are enabled are described below in the IN ROLE, ROLE, and ADMIN clauses. The ??? command has fine-grained option control during membership creation, and the ability to modify these options after the new role is created.

Parameters

<name>
The name of the new role.

SUPERUSER; NOSUPERUSER
These clauses determine whether the new role is a “superuser”, who can override all access restrictions within the database. Superuser status is dangerous and should be used only when really needed. You must yourself be a superuser to create a new superuser. If not specified, NOSUPERUSER is the default.

CREATEDB; NOCREATEDB
These clauses define a role's ability to create databases. If CREATEDB is specified, the role being defined will be allowed to create new databases. Specifying NOCREATEDB will deny a role the ability to create databases. If not specified, NOCREATEDB is the default. Only superuser roles or roles with CREATEDB can specify CREATEDB.

CREATEROLE; NOCREATEROLE
These clauses determine whether a role will be permitted to create, alter, drop, comment on, and change the security label for other roles. See ??? for more details about what capabilities are conferred by this privilege. If not specified, NOCREATEROLE is the default.

INHERIT; NOINHERIT
This affects the membership inheritance status when this role is added as a member of another role, both in this and future commands. Specifically, it controls the inheritance status of memberships added with this command using the IN ROLE clause, and in later commands using the ROLE clause. It is also used as the default inheritance status when adding this role as a member using the GRANT command. If not specified, INHERIT is the default.

In PostgreSQL versions before 16, inheritance was a role-level attribute that controlled all runtime membership checks for that role.

LOGIN; NOLOGIN
These clauses determine whether a role is allowed to log in; that is, whether the role can be given as the initial session authorization name during client connection. A role having the LOGIN attribute can be thought of as a user. Roles without this attribute are useful for managing database privileges, but are not users in the usual sense of the word. If not specified, NOLOGIN is the default, except when CREATE ROLE is invoked through its alternative spelling CREATE USER.

REPLICATION; NOREPLICATION
These clauses determine whether a role is a replication role. A role must have this attribute (or be a superuser) in order to be able to connect to the server in replication mode (physical or logical replication) and in order to be able to create or drop replication slots. A role having the REPLICATION attribute is a very highly privileged role, and should only be used on roles actually used for replication. If not specified, NOREPLICATION is the default. Only superuser roles or roles with REPLICATION can specify REPLICATION.

BYPASSRLS; NOBYPASSRLS
These clauses determine whether a role bypasses every row-level security (RLS) policy. NOBYPASSRLS is the default. Only superuser roles or roles with BYPASSRLS can specify BYPASSRLS.

Note that pg_dump will set row_security to OFF by default, to ensure all contents of a table are dumped out. If the user running pg_dump does not have appropriate permissions, an error will be returned. However, superusers and the owner of the table being dumped always bypass RLS.

CONNECTION LIMIT <connlimit>
If role can log in, this specifies how many concurrent connections the role can make. -1 (the default) means no limit. Note that only normal connections are counted towards this limit. Neither prepared transactions nor background worker connections are counted towards this limit.

[ ENCRYPTED ] PASSWORD '<password>'; PASSWORD NULL
Sets the role's password. (A password is only of use for roles having the LOGIN attribute, but you can nonetheless define one for roles without it.) If you do not plan to use password authentication you can omit this option. If no password is specified, the password will be set to null and password authentication will always fail for that user. A null password can optionally be written explicitly as PASSWORD NULL.

[!NOTE] Specifying an empty string will also set the password to null, but that was not the case before PostgreSQL version 10. In earlier versions, an empty string could be used, or not, depending on the authentication method and the exact version, and libpq would refuse to use it in any case. To avoid the ambiguity, specifying an empty string should be avoided.

The password is always stored encrypted in the system catalogs. The ENCRYPTED keyword has no effect, but is accepted for backwards compatibility. The method of encryption is determined by the configuration parameter ???. If the presented password string is already in MD5-encrypted or SCRAM-encrypted format, then it is stored as-is regardless of password_encryption (since the system cannot decrypt the specified encrypted password string, to encrypt it in a different format). This allows reloading of encrypted passwords during dump/restore.

[!WARNING] Support for MD5-encrypted passwords is deprecated and will be removed in a future release of PostgreSQL. Refer to ??? for details about migrating to another password type.

VALID UNTIL '<timestamp>'
The VALID UNTIL clause sets a date and time after which the role's password is no longer valid. If this clause is omitted the password will be valid for all time.

IN ROLE <role_name>
The IN ROLE clause causes the new role to be automatically added as a member of the specified existing roles. The new membership will have the SET option enabled and the ADMIN option disabled. The INHERIT option will be enabled unless the NOINHERIT option is specified.

ROLE <role_name>
The ROLE clause causes one or more specified existing roles to be automatically added as members, with the SET option enabled. This in effect makes the new role a “group”. Roles named in this clause with the role-level INHERIT attribute will have the INHERIT option enabled in the new membership. New memberships will have the ADMIN option disabled.

ADMIN <role_name>
The ADMIN clause has the same effect as ROLE, but the named roles are added as members of the new role with ADMIN enabled, giving them the right to grant membership in the new role to others.

SYSID <uid>
The SYSID clause is ignored, but is accepted for backwards compatibility.

Notes

Use ALTER ROLE to change the attributes of a role, and DROP ROLE to remove a role. All the attributes specified by CREATE ROLE can be modified by later ALTER ROLE commands.

The preferred way to add and remove members of roles that are being used as groups is to use GRANT and REVOKE.

The VALID UNTIL clause defines an expiration time for a password only, not for the role per se. In particular, the expiration time is not enforced when logging in using a non-password-based authentication method.

The role attributes defined here are non-inheritable, i.e., being a member of a role with, e.g., CREATEDB will not allow the member to create new databases even if the membership grant has the INHERIT option. Of course, if the membership grant has the SET option the member role would be able to SET ROLE to the createdb role and then create a new database.

The membership grants created by the IN ROLE, ROLE, and ADMIN clauses have the role executing this command as the grantor.

The INHERIT attribute is the default for reasons of backwards compatibility: in prior releases of PostgreSQL, users always had access to all privileges of groups they were members of. However, NOINHERIT provides a closer match to the semantics specified in the SQL standard.

PostgreSQL includes a program ??? that has the same functionality as CREATE ROLE (in fact, it calls this command) but can be run from the command shell.

The CONNECTION LIMIT option is only enforced approximately; if two new sessions start at about the same time when just one connection “slot” remains for the role, it is possible that both will fail. Also, the limit is never enforced for superusers.

Caution must be exercised when specifying an unencrypted password with this command. The password will be transmitted to the server in cleartext, and it might also be logged in the client's command history or the server log. The command ???, however, transmits the password encrypted. Also, ??? contains a command \password that can be used to safely change the password later.

Examples

Create a role that can log in, but don't give it a password:

CREATE ROLE jonathan LOGIN;

Create a role with a password:

CREATE USER davide WITH PASSWORD 'jw8s0F4';

(CREATE USER is the same as CREATE ROLE except that it implies LOGIN.)

Create a role with a password that is valid until the end of 2004. After one second has ticked in 2005, the password is no longer valid.

CREATE ROLE miriam WITH LOGIN PASSWORD 'jw8s0F4' VALID UNTIL '2005-01-01';

Create a role that can create databases and manage roles:

CREATE ROLE admin WITH CREATEDB CREATEROLE;

Compatibility

The CREATE ROLE statement is in the SQL standard, but the standard only requires the syntax CREATE ROLE <name> [ WITH ADMIN <role_name> ] Multiple initial administrators, and all the other options of CREATE ROLE, are PostgreSQL extensions.

The SQL standard defines the concepts of users and roles, but it regards them as distinct concepts and leaves all commands defining users to be specified by each database implementation. In PostgreSQL we have chosen to unify users and roles into a single kind of entity. Roles therefore have many more optional attributes than they do in the standard.

The behavior specified by the SQL standard is most closely approximated creating SQL-standard users as PostgreSQL roles with the NOINHERIT option, and SQL-standard roles as PostgreSQL roles with the INHERIT option.

The USER clause has the same behavior as ROLE but has been deprecated: USER <role_name> [, ...]

The IN GROUP clause has the same behavior as IN ROLE but has been deprecated: IN GROUP <role_name> [, ...]

See Also

create_rule

CREATE RULE

CREATE RULE

7

SQL - Language Statements

CREATE RULE

define a new rewrite rule

CREATE [ OR REPLACE ] RULE

name

AS ON

event

TO

table_name

[ WHERE

condition

] DO [ ALSO | INSTEAD ] { NOTHING |

command

| (

command

;

command

... ) }

where

event

can be one of:

SELECT | INSERT | UPDATE | DELETE

Description

CREATE RULE defines a new rule applying to a specified table or view. CREATE OR REPLACE RULE will either create a new rule, or replace an existing rule of the same name for the same table.

The PostgreSQL rule system allows one to define an alternative action to be performed on insertions, updates, or deletions in database tables. Roughly speaking, a rule causes additional commands to be executed when a given command on a given table is executed. Alternatively, an INSTEAD rule can replace a given command by another, or cause a command not to be executed at all. Rules are used to implement SQL views as well. It is important to realize that a rule is really a command transformation mechanism, or command macro. The transformation happens before the execution of the command starts. If you actually want an operation that fires independently for each physical row, you probably want to use a trigger, not a rule. More information about the rules system is in ???.

Presently, ON SELECT rules can only be attached to views. Such a rule must be named "_RETURN", must be an unconditional INSTEAD rule, and must have an action that consists of a single SELECT command. This command defines the visible contents of the view. (The view itself is basically a dummy table with no storage.) It's best to regard such a rule as an implementation detail. While a view can be redefined via CREATE OR REPLACE RULE "_RETURN" AS ..., it's better style to use CREATE OR REPLACE VIEW.

You can create the illusion of an updatable view by defining ON INSERT, ON UPDATE, and ON DELETE rules (or any subset of those that's sufficient for your purposes) to replace update actions on the view with appropriate updates on other tables. If you want to support INSERT RETURNING and so on, then be sure to put a suitable RETURNING clause into each of these rules.

There is a catch if you try to use conditional rules for complex view updates: there must be an unconditional INSTEAD rule for each action you wish to allow on the view. If the rule is conditional, or is not INSTEAD, then the system will still reject attempts to perform the update action, because it thinks it might end up trying to perform the action on the dummy table of the view in some cases. If you want to handle all the useful cases in conditional rules, add an unconditional DO INSTEAD NOTHING rule to ensure that the system understands it will never be called on to update the dummy table. Then make the conditional rules non-INSTEAD; in the cases where they are applied, they add to the default INSTEAD NOTHING action. (This method does not currently work to support RETURNING queries, however.)

[!NOTE] A view that is simple enough to be automatically updatable (see ???) does not require a user-created rule in order to be updatable. While you can create an explicit rule anyway, the automatic update transformation will generally outperform an explicit rule.

Another alternative worth considering is to use INSTEAD OF triggers (see ???) in place of rules.

Parameters

<name>
The name of a rule to create. This must be distinct from the name of any other rule for the same table. Multiple rules on the same table and same event type are applied in alphabetical name order.

<event>
The event is one of SELECT, INSERT, UPDATE, or DELETE. Note that an INSERT containing an ON CONFLICT clause cannot be used on tables that have either INSERT or UPDATE rules. Consider using an updatable view instead.

<table_name>
The name (optionally schema-qualified) of the table or view the rule applies to.

<condition>
Any SQL conditional expression (returning boolean). The condition expression cannot refer to any tables except NEW and OLD, and cannot contain aggregate functions.

INSTEAD
INSTEAD indicates that the commands should be executed instead of the original command.

ALSO
ALSO indicates that the commands should be executed in addition to the original command.

If neither ALSO nor INSTEAD is specified, ALSO is the default.

<command>
The command or commands that make up the rule action. Valid commands are SELECT, INSERT, UPDATE, DELETE, or NOTIFY.

Within <condition> and <command>, the special table names NEW and OLD can be used to refer to values in the referenced table. NEW is valid in ON INSERT and ON UPDATE rules to refer to the new row being inserted or updated. OLD is valid in ON UPDATE and ON DELETE rules to refer to the existing row being updated or deleted.

Notes

You must be the owner of a table to create or change rules for it.

In a rule for INSERT, UPDATE, or DELETE on a view, you can add a RETURNING clause that emits the view's columns. This clause will be used to compute the outputs if the rule is triggered by an INSERT RETURNING, UPDATE RETURNING, or DELETE RETURNING command respectively. When the rule is triggered by a command without RETURNING, the rule's RETURNING clause will be ignored. The current implementation allows only unconditional INSTEAD rules to contain RETURNING; furthermore there can be at most one RETURNING clause among all the rules for the same event. (This ensures that there is only one candidate RETURNING clause to be used to compute the results.) RETURNING queries on the view will be rejected if there is no RETURNING clause in any available rule.

It is very important to take care to avoid circular rules. For example, though each of the following two rule definitions are accepted by PostgreSQL, the SELECT command would cause PostgreSQL to report an error because of recursive expansion of a rule:

CREATE RULE "_RETURN" AS
    ON SELECT TO t1
    DO INSTEAD
        SELECT * FROM t2;

CREATE RULE "_RETURN" AS
    ON SELECT TO t2
    DO INSTEAD
        SELECT * FROM t1;

SELECT * FROM t1;

Presently, if a rule action contains a NOTIFY command, the NOTIFY command will be executed unconditionally, that is, the NOTIFY will be issued even if there are not any rows that the rule should apply to. For example, in:

CREATE RULE notify_me AS ON UPDATE TO mytable DO ALSO NOTIFY mytable;

UPDATE mytable SET name = 'foo' WHERE id = 42;

one NOTIFY event will be sent during the UPDATE, whether or not there are any rows that match the condition id = 42. This is an implementation restriction that might be fixed in future releases.

Compatibility

CREATE RULE is a PostgreSQL language extension, as is the entire query rewrite system.

See Also

create_schema

CREATE SCHEMA

CREATE SCHEMA

7

SQL - Language Statements

CREATE SCHEMA

define a new schema

CREATE SCHEMA

schema_name

[ AUTHORIZATION

role_specification

] [

schema_element

[ ... ] ] CREATE SCHEMA AUTHORIZATION

role_specification

[

schema_element

[ ... ] ] CREATE SCHEMA IF NOT EXISTS

schema_name

[ AUTHORIZATION

role_specification

] CREATE SCHEMA IF NOT EXISTS AUTHORIZATION

role_specification

where

role_specification

can be:

user_name

| CURRENT_ROLE | CURRENT_USER | SESSION_USER

Description

CREATE SCHEMA enters a new schema into the current database. The schema name must be distinct from the name of any existing schema in the current database.

A schema is essentially a namespace: it contains named objects (tables, data types, functions, and operators) whose names can duplicate those of other objects existing in other schemas. Named objects are accessed either by “qualifying” their names with the schema name as a prefix, or by setting a search path that includes the desired schema(s). A CREATE command specifying an unqualified object name creates the object in the current schema (the one at the front of the search path, which can be determined with the function current_schema).

Optionally, CREATE SCHEMA can include subcommands to create objects within the new schema. The subcommands are treated essentially the same as separate commands issued after creating the schema, except that if the AUTHORIZATION clause is used, all the created objects will be owned by that user.

Parameters

<schema_name>
The name of a schema to be created. If this is omitted, the <user_name> is used as the schema name. The name cannot begin with pg_, as such names are reserved for system schemas.

<user_name>
The role name of the user who will own the new schema. If omitted, defaults to the user executing the command. To create a schema owned by another role, you must be able to SET ROLE to that role.

<schema_element>
An SQL statement defining an object to be created within the schema. Currently, only CREATE TABLE, CREATE VIEW, CREATE INDEX, CREATE SEQUENCE, CREATE TRIGGER and GRANT are accepted as clauses within CREATE SCHEMA. Other kinds of objects may be created in separate commands after the schema is created.

IF NOT EXISTS
Do nothing (except issuing a notice) if a schema with the same name already exists. <schema_element> subcommands cannot be included when this option is used.

Notes

To create a schema, the invoking user must have the CREATE privilege for the current database. (Of course, superusers bypass this check.)

Examples

Create a schema:

CREATE SCHEMA myschema;

Create a schema for user joe; the schema will also be named joe:

CREATE SCHEMA AUTHORIZATION joe;

Create a schema named test that will be owned by user joe, unless there already is a schema named test. (It does not matter whether joe owns the pre-existing schema.)

CREATE SCHEMA IF NOT EXISTS test AUTHORIZATION joe;

Create a schema and create a table and view within it:

CREATE SCHEMA hollywood
    CREATE TABLE films (title text, release date, awards text[])
    CREATE VIEW winners AS
        SELECT title, release FROM films WHERE awards IS NOT NULL;

Notice that the individual subcommands do not end with semicolons.

The following is an equivalent way of accomplishing the same result:

CREATE SCHEMA hollywood;
CREATE TABLE hollywood.films (title text, release date, awards text[]);
CREATE VIEW hollywood.winners AS
    SELECT title, release FROM hollywood.films WHERE awards IS NOT NULL;

Compatibility

The SQL standard allows a DEFAULT CHARACTER SET clause in CREATE SCHEMA, as well as more subcommand types than are presently accepted by PostgreSQL.

The SQL standard suggests, without saying so in so many words, that the subcommands in CREATE SCHEMA can appear in any order; but the only actual requirement it gives is that foreign keys referencing table definitions that appear in later CREATE SCHEMA subcommands should be accepted. The present PostgreSQL implementation handles that case, but does not handle all other cases of forward references in subcommands; it might sometimes be necessary to reorder the subcommands in order to avoid forward references.

According to the SQL standard, the owner of a schema always owns all objects within it. PostgreSQL allows schemas to contain objects owned by users other than the schema owner. This can happen only if the schema owner grants the CREATE privilege on their schema to someone else, or a superuser chooses to create objects in it.

The IF NOT EXISTS option is a PostgreSQL extension.

See Also

create_sequence

CREATE SEQUENCE

CREATE SEQUENCE

7

SQL - Language Statements

CREATE SEQUENCE

define a new sequence generator

CREATE [ { TEMPORARY | TEMP } | UNLOGGED ] SEQUENCE [ IF NOT EXISTS ]

name

[ AS

data_type

] [ INCREMENT [ BY ]

increment

] [ MINVALUE

minvalue

| NO MINVALUE ] [ MAXVALUE

maxvalue

| NO MAXVALUE ] [ [ NO ] CYCLE ] [ START [ WITH ]

start

] [ CACHE

cache

] [ OWNED BY {

table_name

.

column_name

| NONE } ]

Description

CREATE SEQUENCE creates a new sequence number generator. This involves creating and initializing a new special single-row table with the name <name>. The generator will be owned by the user issuing the command.

If a schema name is given then the sequence is created in the specified schema. Otherwise it is created in the current schema. Temporary sequences exist in a special schema, so a schema name cannot be given when creating a temporary sequence. The sequence name must be distinct from the name of any other relation (table, sequence, index, view, materialized view, or foreign table) in the same schema.

After a sequence is created, you use the functions nextval, currval, and setval to operate on the sequence. These functions are documented in ???.

Although you cannot update a sequence directly, you can use a query like:

SELECT * FROM name;

to examine the parameters and current state of a sequence. In particular, the last_value field of the sequence shows the last value allocated by any session. (Of course, this value might be obsolete by the time it's printed, if other sessions are actively doing nextval calls.)

Parameters

TEMPORARY or TEMP
If specified, the sequence object is created only for this session, and is automatically dropped on session exit. Existing permanent sequences with the same name are not visible (in this session) while the temporary sequence exists, unless they are referenced with schema-qualified names.

UNLOGGED
If specified, the sequence is created as an unlogged sequence. Changes to unlogged sequences are not written to the write-ahead log. They are not crash-safe: an unlogged sequence is automatically reset to its initial state after a crash or unclean shutdown. Unlogged sequences are also not replicated to standby servers.

Unlike unlogged tables, unlogged sequences do not offer a significant performance advantage. This option is mainly intended for sequences associated with unlogged tables via identity columns or serial columns. In those cases, it usually wouldn't make sense to have the sequence WAL-logged and replicated but not its associated table.

IF NOT EXISTS
Do not throw an error if a relation with the same name already exists. A notice is issued in this case. Note that there is no guarantee that the existing relation is anything like the sequence that would have been created — it might not even be a sequence.

<name>
The name (optionally schema-qualified) of the sequence to be created.

<data_type>
The optional clause AS data_type specifies the data type of the sequence. Valid types are smallint, integer, and bigint. bigint is the default. The data type determines the default minimum and maximum values of the sequence.

<increment>
The optional clause INCREMENT BY increment specifies which value is added to the current sequence value to create a new value. A positive value will make an ascending sequence, a negative one a descending sequence. The default value is 1.

<minvalue>; NO MINVALUE
The optional clause MINVALUE minvalue determines the minimum value a sequence can generate. If this clause is not supplied or NO MINVALUE is specified, then defaults will be used. The default for an ascending sequence is 1. The default for a descending sequence is the minimum value of the data type.

<maxvalue>; NO MAXVALUE
The optional clause MAXVALUE maxvalue determines the maximum value for the sequence. If this clause is not supplied or NO MAXVALUE is specified, then default values will be used. The default for an ascending sequence is the maximum value of the data type. The default for a descending sequence is -1.

CYCLE; NO CYCLE
The CYCLE option allows the sequence to wrap around when the <maxvalue> or <minvalue> has been reached by an ascending or descending sequence respectively. If the limit is reached, the next number generated will be the <minvalue> or <maxvalue>, respectively.

If NO CYCLE is specified, any calls to nextval after the sequence has reached its maximum value will return an error. If neither CYCLE or NO CYCLE are specified, NO CYCLE is the default.

<start>
The optional clause START WITH start allows the sequence to begin anywhere. The default starting value is <minvalue> for ascending sequences and <maxvalue> for descending ones.

<cache>
The optional clause CACHE cache specifies how many sequence numbers are to be preallocated and stored in memory for faster access. The minimum value is 1 (only one value can be generated at a time, i.e., no cache), and this is also the default.

OWNED BY <table_name>.<column_name>; OWNED BY NONE
The OWNED BY option causes the sequence to be associated with a specific table column, such that if that column (or its whole table) is dropped, the sequence will be automatically dropped as well. The specified table must have the same owner and be in the same schema as the sequence. OWNED BY NONE, the default, specifies that there is no such association.

Notes

Use DROP SEQUENCE to remove a sequence.

Sequences are based on bigint arithmetic, so the range cannot exceed the range of an eight-byte integer (-9223372036854775808 to 9223372036854775807).

Because nextval and setval calls are never rolled back, sequence objects cannot be used if “gapless” assignment of sequence numbers is needed. It is possible to build gapless assignment by using exclusive locking of a table containing a counter; but this solution is much more expensive than sequence objects, especially if many transactions need sequence numbers concurrently.

Unexpected results might be obtained if a <cache> setting greater than one is used for a sequence object that will be used concurrently by multiple sessions. Each session will allocate and cache successive sequence values during one access to the sequence object and increase the sequence object's last_value accordingly. Then, the next <cache>-1 uses of nextval within that session simply return the preallocated values without touching the sequence object. So, any numbers allocated but not used within a session will be lost when that session ends, resulting in “holes” in the sequence.

Furthermore, although multiple sessions are guaranteed to allocate distinct sequence values, the values might be generated out of sequence when all the sessions are considered. For example, with a <cache> setting of 10, session A might reserve values 1..10 and return nextval=1, then session B might reserve values 11..20 and return nextval=11 before session A has generated nextval=2. Thus, with a <cache> setting of one it is safe to assume that nextval values are generated sequentially; with a <cache> setting greater than one you should only assume that the nextval values are all distinct, not that they are generated purely sequentially. Also, last_value will reflect the latest value reserved by any session, whether or not it has yet been returned by nextval.

Another consideration is that a setval executed on such a sequence will not be noticed by other sessions until they have used up any preallocated values they have cached.

Examples

Create an ascending sequence called serial, starting at 101:

CREATE SEQUENCE serial START 101;

Select the next number from this sequence:

SELECT nextval('serial');

 nextval
---------
     101

Select the next number from this sequence:

SELECT nextval('serial');

 nextval
---------
     102

Use this sequence in an INSERT command:

INSERT INTO distributors VALUES (nextval('serial'), 'nothing');

Update the sequence value after a COPY FROM:

BEGIN;
COPY distributors FROM 'input_file';
SELECT setval('serial', max(id)) FROM distributors;
END;

Compatibility

CREATE SEQUENCE conforms to the SQL standard, with the following exceptions:

See Also

create_server

CREATE SERVER

CREATE SERVER

7

SQL - Language Statements

CREATE SERVER

define a new foreign server

CREATE SERVER [ IF NOT EXISTS ]

server_name

[ TYPE '

server_type

' ] [ VERSION '

server_version

' ] FOREIGN DATA WRAPPER

fdw_name

[ OPTIONS (

option

'

value

' [, ... ] ) ]

Description

CREATE SERVER defines a new foreign server. The user who defines the server becomes its owner.

A foreign server typically encapsulates connection information that a foreign-data wrapper uses to access an external data resource. Additional user-specific connection information may be specified by means of user mappings.

If the foreign data wrapper <fdw_name> is specified with a CONNECTION clause, then ??? may use this foreign server for connection information.

The server name must be unique within the database.

Creating a server requires USAGE privilege on the foreign-data wrapper being used.

Parameters

IF NOT EXISTS
Do not throw an error if a server with the same name already exists. A notice is issued in this case. Note that there is no guarantee that the existing server is anything like the one that would have been created.

<server_name>
The name of the foreign server to be created.

<server_type>
Optional server type, potentially useful to foreign-data wrappers.

<server_version>
Optional server version, potentially useful to foreign-data wrappers.

<fdw_name>
The name of the foreign-data wrapper that manages the server.

OPTIONS ( option 'value' [, ... ] )
This clause specifies the options for the server. The options typically define the connection details of the server, but the actual names and values are dependent on the server's foreign-data wrapper.

Notes

When using the ??? module, a foreign server's name can be used as an argument of the ??? function to indicate the connection parameters. It is necessary to have the USAGE privilege on the foreign server to be able to use it in this way.

If the foreign server supports sort pushdown, it is necessary for it to have the same sort ordering as the local server.

Examples

Create a server myserver that uses the foreign-data wrapper postgres_fdw:

CREATE SERVER myserver FOREIGN DATA WRAPPER postgres_fdw OPTIONS (host 'foo', dbname 'foodb', port '5432');

See ??? for more details.

Compatibility

CREATE SERVER conforms to ISO/IEC 9075-9 (SQL/MED).

See Also

create_statistics

CREATE STATISTICS

CREATE STATISTICS

7

SQL - Language Statements

CREATE STATISTICS

define extended statistics

CREATE STATISTICS [ [ IF NOT EXISTS ]

statistics_name

] ON {

column_name

| (

expression

) } FROM

table_name

CREATE STATISTICS [ [ IF NOT EXISTS ]

statistics_name

] [ (

statistics_kind

[, ... ] ) ] ON {

column_name

| (

expression

) }, {

column_name

| (

expression

) } [, ...] FROM

table_name

Description

CREATE STATISTICS will create a new extended statistics object tracking data about the specified table, foreign table or materialized view. The statistics object will be created in the current database and will be owned by the user issuing the command.

The CREATE STATISTICS command has two basic forms. The first form allows univariate statistics for a single expression or virtual generated column to be collected, providing benefits similar to an expression index without the overhead of index maintenance. This form does not allow the statistics kind to be specified, since the various statistics kinds refer only to multivariate statistics. The second form of the command allows multivariate statistics on multiple columns and/or expressions to be collected, optionally specifying which statistics kinds to include. This form will also automatically cause univariate statistics to be collected on any expressions and virtual generated columns included in the list.

If a schema name is given (for example, CREATE STATISTICS myschema.mystat ...) then the statistics object is created in the specified schema. Otherwise it is created in the current schema. If given, the name of the statistics object must be distinct from the name of any other statistics object in the same schema.

Parameters

IF NOT EXISTS
Do not throw an error if a statistics object with the same name already exists. A notice is issued in this case. Note that only the name of the statistics object is considered here, not the details of its definition. Statistics name is required when IF NOT EXISTS is specified.

<statistics_name>
The name (optionally schema-qualified) of the statistics object to be created. If the name is omitted, PostgreSQL chooses a suitable name based on the parent table's name and the defined column name(s) and/or expression(s).

<statistics_kind>
A multivariate statistics kind to be computed in this statistics object. Currently supported kinds are ndistinct, which enables n-distinct statistics, dependencies, which enables functional dependency statistics, and mcv which enables most-common values lists. If this clause is omitted, all supported statistics kinds are included in the statistics object. Univariate expression statistics are built automatically if the statistics definition includes any complex expressions or references to virtual generated columns rather than just simple column references. For more information, see ??? and ???.

<column_name>
The name of a table column to be covered by the computed statistics. This may be used to build univariate statistics on a single virtual generated column, or as part of a list of multiple columns (virtual or non-virtual) and/or expressions to build multivariate statistics. In the latter case, separate univariate statistics are built automatically for each expression and virtual generated column in the list.

Defining extended statistics on a single non-virtual column is not supported or necessary, because statistics are built automatically on such columns without defining extended statistics.

<expression>
An expression to be covered by the computed statistics. This may be used to build univariate statistics on a single expression, or as part of a list of multiple column names and/or expressions to build multivariate statistics. In the latter case, separate univariate statistics are built automatically for each expression and virtual generated column in the list.

<table_name>
The name (optionally schema-qualified) of the table containing the column(s) the statistics are computed on; see ??? for an explanation of the handling of inheritance and partitions.

Notes

You must be the owner of a table to create a statistics object reading it. Once created, however, the ownership of the statistics object is independent of the underlying table(s).

Expression statistics are per-expression and are similar to creating an index on the expression, except that they avoid the overhead of index maintenance. Expression statistics are built automatically for each expression in the statistics object definition. Extended statistics on a virtual generated column behave the same as expression statistics on the column's generation expression.

Extended statistics are not currently used by the planner for selectivity estimations made for table joins. This limitation will likely be removed in a future version of PostgreSQL.

Examples

Create table t1 with two functionally dependent columns, i.e., knowledge of a value in the first column is sufficient for determining the value in the other column. Then functional dependency statistics are built on those columns:

CREATE TABLE t1 (
    a   int,
    b   int
);

INSERT INTO t1 SELECT i/100, i/500
                 FROM generate_series(1,1000000) s(i);

ANALYZE t1;

-- the number of matching rows will be drastically underestimated:
EXPLAIN ANALYZE SELECT * FROM t1 WHERE (a = 1) AND (b = 0);

CREATE STATISTICS s1 (dependencies) ON a, b FROM t1;

ANALYZE t1;

-- now the row count estimate is more accurate:
EXPLAIN ANALYZE SELECT * FROM t1 WHERE (a = 1) AND (b = 0);

Without functional-dependency statistics, the planner would assume that the two WHERE conditions are independent, and would multiply their selectivities together to arrive at a much-too-small row count estimate. With such statistics, the planner recognizes that the WHERE conditions are redundant and does not underestimate the row count.

Create table t2 with two perfectly correlated columns (containing identical data), and an MCV list on those columns:

CREATE TABLE t2 (
    a   int,
    b   int
);

INSERT INTO t2 SELECT mod(i,100), mod(i,100)
                 FROM generate_series(1,1000000) s(i);

CREATE STATISTICS s2 (mcv) ON a, b FROM t2;

ANALYZE t2;

-- valid combination (found in MCV)
EXPLAIN ANALYZE SELECT * FROM t2 WHERE (a = 1) AND (b = 1);

-- invalid combination (not found in MCV)
EXPLAIN ANALYZE SELECT * FROM t2 WHERE (a = 1) AND (b = 2);

The MCV list gives the planner more detailed information about the specific values that commonly appear in the table, as well as an upper bound on the selectivities of combinations of values that do not appear in the table, allowing it to generate better estimates in both cases.

Create table t3 with a single timestamp column, and run queries using expressions on that column. Without extended statistics, the planner has no information about the data distribution for the expressions, and uses default estimates. The planner also does not realize that the value of the date truncated to the month is fully determined by the value of the date truncated to the day. Then expression and ndistinct statistics are built on those two expressions:

CREATE TABLE t3 (
    a   timestamp
);

INSERT INTO t3 SELECT i FROM generate_series('2020-01-01'::timestamp,
                                             '2020-12-31'::timestamp,
                                             '1 minute'::interval) s(i);

ANALYZE t3;

-- the number of matching rows will be drastically underestimated:
EXPLAIN ANALYZE SELECT * FROM t3
  WHERE date_trunc('month', a) = '2020-01-01'::timestamp;

EXPLAIN ANALYZE SELECT * FROM t3
  WHERE date_trunc('day', a) BETWEEN '2020-01-01'::timestamp
                                 AND '2020-06-30'::timestamp;

EXPLAIN ANALYZE SELECT date_trunc('month', a), date_trunc('day', a)
   FROM t3 GROUP BY 1, 2;

-- build ndistinct statistics on the pair of expressions (per-expression
-- statistics are built automatically)
CREATE STATISTICS s3 (ndistinct) ON date_trunc('month', a), date_trunc('day', a) FROM t3;

ANALYZE t3;

-- now the row count estimates are more accurate:
EXPLAIN ANALYZE SELECT * FROM t3
  WHERE date_trunc('month', a) = '2020-01-01'::timestamp;

EXPLAIN ANALYZE SELECT * FROM t3
  WHERE date_trunc('day', a) BETWEEN '2020-01-01'::timestamp
                                 AND '2020-06-30'::timestamp;

EXPLAIN ANALYZE SELECT date_trunc('month', a), date_trunc('day', a)
   FROM t3 GROUP BY 1, 2;

Without expression and ndistinct statistics, the planner has no information about the number of distinct values for the expressions, and has to rely on default estimates. The equality and range conditions are assumed to have 0.5% selectivity, and the number of distinct values in the expression is assumed to be the same as for the column (i.e. unique). This results in a significant underestimate of the row count in the first two queries. Moreover, the planner has no information about the relationship between the expressions, so it assumes the two WHERE and GROUP BY conditions are independent, and multiplies their selectivities together to arrive at a severe overestimate of the group count in the aggregate query. This is further exacerbated by the lack of accurate statistics for the expressions, forcing the planner to use a default ndistinct estimate for the expression derived from ndistinct for the column. With such statistics, the planner recognizes that the conditions are correlated, and arrives at much more accurate estimates.

Compatibility

There is no CREATE STATISTICS command in the SQL standard.

See Also

create_subscription

CREATE SUBSCRIPTION

CREATE SUBSCRIPTION

7

SQL - Language Statements

CREATE SUBSCRIPTION

define a new subscription

CREATE SUBSCRIPTION

subscription_name

{ SERVER

server_name

| CONNECTION '

conninfo

' } PUBLICATION

publication_name

[, ...] [ WITH (

subscription_parameter

[=

value

] [, ... ] ) ]

Description

CREATE SUBSCRIPTION adds a new logical-replication subscription. The user that creates a subscription becomes the owner of the subscription. The subscription name must be distinct from the name of any existing subscription in the current database.

A subscription represents a replication connection to the publisher. Hence, in addition to adding definitions in the local catalogs, this command normally creates a replication slot on the publisher.

A logical replication worker will be started to replicate data for the new subscription at the commit of the transaction where this command is run, unless the subscription is initially disabled.

To be able to create a subscription, you must have the privileges of the pg_create_subscription role, as well as CREATE privileges on the current database.

Additional information about subscriptions and logical replication as a whole is available at ??? and ???.

Parameters

<subscription_name>
The name of the new subscription.

SERVER server_name
A foreign server to use for the connection. The server's foreign data wrapper must have a <connection_function> registered, and a user mapping for the subscription owner on the server must exist. Additionally, the subscription owner must have USAGE privileges on <server_name>.

CONNECTION 'conninfo'
The libpq connection string defining how to connect to the publisher database. For details see ???.

PUBLICATION publication_name [, ...]
Names of the publications on the publisher to subscribe to.

WITH ( subscription_parameter [= value] [, ... ] )
This clause specifies optional parameters for a subscription.

The following parameters control what happens during subscription creation:

connect (boolean)
Specifies whether the CREATE SUBSCRIPTION command should connect to the publisher at all. The default is true. Setting this to false will force the values of create_slot, enabled and copy_data to false. (You cannot combine setting connect to false with setting create_slot, enabled, or copy_data to true.)

Since no connection is made when this option is false, no tables and sequences are subscribed. To initiate replication, you must manually create the replication slot, enable the failover if required, enable the subscription, and refresh the subscription. See ??? for examples.

create_slot (boolean)
Specifies whether the command should create the replication slot on the publisher. The default is true.

If set to false, you are responsible for creating the publisher's slot in some other way. See ??? for examples.

enabled (boolean)
Specifies whether the subscription should be actively replicating or whether it should just be set up but not started yet. The default is true.

slot_name (string)
Name of the publisher's replication slot to use. The default is to use the name of the subscription for the slot name. The name cannot be pg_conflict_detection as it is reserved for the conflict detection.

Setting slot_name to NONE means there will be no replication slot associated with the subscription. Such subscriptions must also have both enabled and create_slot set to false. Use this when you will be creating the replication slot later manually. See ??? for examples.

When setting slot_name to a valid name and create_slot to false, the failover property value of the named slot may differ from the counterpart failover parameter specified in the subscription. Always ensure the slot property failover matches the counterpart parameter of the subscription and vice versa. Otherwise, the slot on the publisher may behave differently from what these subscription options say: for example, the slot on the publisher could either be synced to the standbys even when the subscription's failover option is disabled or could be disabled for sync even when the subscription's failover option is enabled.

The following parameters control the subscription's replication behavior after it has been created:

binary (boolean)
Specifies whether the subscription will request the publisher to send the data in binary format (as opposed to text). The default is false. Any initial table synchronization copy (see copy_data) also uses the same format. Binary format can be faster than the text format, but it is less portable across machine architectures and PostgreSQL versions. Binary format is very data type specific; for example, it will not allow copying from a smallint column to an integer column, even though that would work fine in text format. Even when this option is enabled, only data types having binary send and receive functions will be transferred in binary. Note that the initial synchronization requires all data types to have binary send and receive functions, otherwise the synchronization will fail (see ??? for more about send/receive functions). This parameter has no effect for sequences.

When doing cross-version replication, it could be that the publisher has a binary send function for some data type, but the subscriber lacks a binary receive function for that type. In such a case, data transfer will fail, and the binary option cannot be used.

If the publisher is a PostgreSQL version before 16, then any initial table synchronization will use text format even if binary = true.

conflict_log_destination (enum)
Specifies the destination for recording logical replication conflicts. The available values are:

log
Conflict details are recorded in the server log. This is the default behavior.

table
The system automatically creates a structured table named pg_conflict_log_<subid> in the pg_conflict schema. This allows for easy querying and analysis of conflicts.

[!CAUTION] The conflict log table is strictly tied to the lifecycle of the subscription or the conflict_log_destination setting. If the subscription is dropped, or if the destination is changed to log, the table is dropped.

all
This is equivalent to configuring both destinations log and table together.

copy_data (boolean)
Specifies whether to copy pre-existing data in the publications that are being subscribed to when the replication starts. The default is true.

If the publications contain WHERE clauses, it will affect what data is copied. Refer to the Notes for details.

See Notes for details of how copy_data = true can interact with the origin parameter.

See ??? for recommendations on how to handle any warnings about sequence definition differences between the publisher and the subscriber, which might occur when copy_data = true.

streaming (enum)
Specifies whether to enable streaming of in-progress transactions for this subscription. The default value is parallel, meaning incoming changes are directly applied via one of the parallel apply workers, if available. If no parallel apply worker is free to handle streaming transactions then the changes are written to temporary files and applied after the transaction is committed. Note that if an error happens in a parallel apply worker, the finish LSN of the remote transaction might not be reported in the server log. This parameter has no effect for sequences.

[!CAUTION] There is a risk of deadlock when the schemas of the publisher and subscriber differ, although such cases are rare. The apply worker is equipped to retry these transactions automatically.

If set to on, the incoming changes are written to temporary files and then applied only after the transaction is committed on the publisher and received by the subscriber.

If set to off, all transactions are fully decoded on the publisher and only then sent to the subscriber as a whole.

synchronous_commit (enum)
The value of this parameter overrides the ??? setting within this subscription's apply worker processes. The default value is off. This parameter has no effect for sequences.

It is safe to use off for logical replication: If the subscriber loses transactions because of missing synchronization, the data will be sent again from the publisher.

A different setting might be appropriate when doing synchronous logical replication. The logical replication workers report the positions of writes and flushes to the publisher, and when using synchronous replication, the publisher will wait for the actual flush. This means that setting synchronous_commit for the subscriber to off when the subscription is used for synchronous replication might increase the latency for COMMIT on the publisher. In this scenario, it can be advantageous to set synchronous_commit to local or higher.

two_phase (boolean)
Specifies whether two-phase commit is enabled for this subscription. The default is false. This parameter has no effect for sequences.

When two-phase commit is enabled, prepared transactions are sent to the subscriber at the time of PREPARE TRANSACTION, and are processed as two-phase transactions on the subscriber too. Otherwise, prepared transactions are sent to the subscriber only when committed, and are then processed immediately by the subscriber.

The implementation of two-phase commit requires that replication has successfully finished the initial table synchronization phase. So even when two_phase is enabled for a subscription, the internal two-phase state remains temporarily “pending” until the initialization phase completes. See column subtwophasestate of pg_subscription to know the actual two-phase state.

disable_on_error (boolean)
Specifies whether the subscription should be automatically disabled if any errors are detected by subscription workers during data replication from the publisher. The default is false.

password_required (boolean)
If set to true, connections to the publisher made as a result of this subscription must use password authentication and the password must be specified as a part of the connection string. This setting is ignored when the subscription is owned by a superuser. The default is true. Only superusers can set this value to false.

run_as_owner (boolean)
If true, all replication actions are performed as the subscription owner. If false, replication workers will perform actions on each table or sequence as the owner of that relation. The latter configuration is generally much more secure; for details, see ???. The default is false.

origin (string)
Specifies whether the subscription will request the publisher to only send changes that don't have an origin or send changes regardless of origin. Setting origin to none means that the subscription will request the publisher to only send changes that don't have an origin. Setting origin to any means that the publisher sends changes regardless of their origin. The default is any. This parameter has no effect for sequences.

See Notes for details of how copy_data = true can interact with the origin parameter.

failover (boolean)
Specifies whether the replication slots associated with the subscription are enabled to be synced to the standbys so that logical replication can be resumed from the new primary after failover. The default is false.

retain_dead_tuples (boolean)
Specifies whether the information (e.g., dead tuples, commit timestamps, and origins) required for conflict detection on the subscriber is retained. The default is false. If set to true, the detection of ??? is enabled, and a physical replication slot named “pg_conflict_detection” is created on the subscriber to prevent the information for detecting conflicts from being removed. This parameter has no effect for sequences.

Note that the information useful for conflict detection is retained only after the creation of the slot. You can verify the existence of this slot by querying pg_replication_slots. And even if multiple subscriptions on one node enable this option, only one replication slot will be created. Also, wal_level must be set to replica or higher to allow the replication slot to be used.

[!CAUTION] Note that the information for conflict detection cannot be purged if the subscription is disabled; thus, the information will accumulate until the subscription is enabled. To prevent excessive accumulation, it is recommended to disable retain_dead_tuples if the subscription will be inactive for an extended period. The same applies while the subscription is enabled but its apply worker is not running, for example because the worker repeatedly fails to apply a change. The oldest information being retained can be seen as the xmin of the “pg_conflict_detection” slot in pg_replication_slots.

Additionally when enabling retain_dead_tuples for conflict detection in logical replication, it is important to design the replication topology to balance data retention requirements with overall system performance. This option provides minimal performance overhead when applied appropriately. The following scenarios illustrate effective usage patterns when enabling this option.

a. Large Tables with Bidirectional Writes: For large tables subject to concurrent writes on both publisher and subscriber nodes, publishers can define row filters when creating publications to segment data. This allows multiple subscriptions to replicate exclusive subsets of the table in parallel, optimizing the throughput.

b. Write-Enabled Subscribers: If a subscriber node is expected to perform write operations, replication can be structured using multiple publications and subscriptions. By distributing tables across these publications, the workload is spread among several apply workers, improving concurrency and reducing contention.

c. Read-Only Subscribers: In configurations involving single or multiple publisher nodes performing concurrent write operations, read-only subscriber nodes may replicate changes without seeing a performance impact if it does index scan. However, if the subscriber is impacted due to replication lag or scan performance (say due to sequential scans), it needs to follow one of the two previous strategies to distribute the workload on the subscriber.

This option cannot be enabled if the publisher is a physical standby.

Enabling this option ensures retention of information useful for conflict detection solely for changes occurring locally on the publisher. For the changes originating from different origins, reliable conflict detection cannot be guaranteed.

Detection of ??? also requires track_commit_timestamp to be enabled on the subscriber, and a warning is issued if retain_dead_tuples is enabled while it is not. If track_commit_timestamp is disabled afterwards, the information for conflict detection continues to be retained, but the conflict is reported as ??? instead, so the retention no longer serves its purpose.

max_retention_duration (integer)
Maximum duration in milliseconds for which this subscription's apply worker is allowed to retain the information useful for conflict detection when retain_dead_tuples is enabled. The default value is 0, indicating that the information is retained until it is no longer needed for detection purposes.

The information useful for conflict detection is no longer retained if all apply workers associated with the subscriptions, where retain_dead_tuples is enabled, confirm that the retention duration has exceeded the max_retention_duration set within the corresponding subscription. The retention will automatically resume when at least one apply worker confirms that the retention duration is within the specified limit, or when a new subscription is created with retain_dead_tuples = true. Alternatively, retention can be manually resumed by re-enabling retain_dead_tuples.

Note that overall retention will not stop if other subscriptions that have a value greater than 0 for this parameter have not exceeded it, or if they set this option to 0.

This option is effective only when retain_dead_tuples is enabled and the apply worker associated with the subscription is active.

[!CAUTION] Note that the retention duration is not evaluated while the subscription is disabled or its apply worker is not running. Thus, the information retained for conflict detection will continue to accumulate regardless of this setting until the subscription is enabled or its apply worker resumes. To prevent excessive accumulation, consider disabling retain_dead_tuples if the subscription will be inactive for an extended period.

[!WARNING] Note that setting a non-zero value for this option could lead to information for conflict detection being removed prematurely, potentially resulting in incorrect conflict detection.

wal_receiver_timeout (text)
The value of this parameter overrides the ??? setting within this subscription's apply worker processes. The default value is -1, which means it does not override the global setting, i.e., the value from the server configuration, command line, role or database settings will be used instead.

When specifying a parameter of type boolean, the = <value> part can be omitted, which is equivalent to specifying TRUE.

Notes

See ??? for details on how to configure access control between the subscription and the publication instance.

When creating a replication slot (the default behavior), CREATE SUBSCRIPTION cannot be executed inside a transaction block.

Creating a subscription that connects to the same database cluster (for example, to replicate between databases in the same cluster or to replicate within the same database) will only succeed if the replication slot is not created as part of the same command. Otherwise, the CREATE SUBSCRIPTION call will hang. To make this work, create the replication slot separately (using the function pg_create_logical_replication_slot with the plugin name pgoutput) and create the subscription using the parameter create_slot = false. See ??? for examples. This is an implementation restriction that might be lifted in a future release.

If any table in the publication has a WHERE clause, rows for which the <expression> evaluates to false or NULL will not be published. If the subscription has several publications in which the same table has been published with different WHERE clauses, a row will be published if any of the expressions (referring to that publish operation) are satisfied. In the case of different WHERE clauses, if one of the publications has no WHERE clause (referring to that publish operation) or the publication is declared as FOR ALL TABLES or FOR TABLES IN SCHEMA, rows are always published regardless of the definition of the other expressions. If the subscriber is a PostgreSQL version before 15, then any row filtering is ignored during the initial data synchronization phase. For this case, the user might want to consider deleting any initially copied data that would be incompatible with subsequent filtering. Because initial data synchronization does not take into account the publication publish parameter when copying existing table data, some rows may be copied that would not be replicated using DML. See ??? for examples.

Subscriptions having several publications in which the same table has been published with different column lists are not supported.

We allow non-existent publications to be specified so that users can add those later. This means pg_subscription can have non-existent publications.

When using a subscription parameter combination of copy_data = true and origin = NONE, the initial sync table data is copied directly from the publisher, meaning that knowledge of the true origin of that data is not possible. If the publisher also has subscriptions then the copied table data might have originated from further upstream. This scenario is detected and a WARNING is logged to the user, but the warning is only an indication of a potential problem; it is the user's responsibility to make the necessary checks to ensure the copied data origins are really as wanted or not.

To find which tables might potentially include non-local origins (due to other subscriptions created on the publisher) try this SQL query:

# substitute <pub-names> below with your publication name(s) to be queried
SELECT DISTINCT PT.schemaname, PT.tablename
FROM pg_publication_tables PT
     JOIN pg_class C ON (C.relname = PT.tablename)
     JOIN pg_namespace N ON (N.nspname = PT.schemaname),
     pg_subscription_rel PS
WHERE C.relnamespace = N.oid AND
      (PS.srrelid = C.oid OR
      C.oid IN (SELECT relid FROM pg_partition_ancestors(PS.srrelid) UNION
                SELECT relid FROM pg_partition_tree(PS.srrelid))) AND
      PT.pubname IN (<pub-names>);

Examples

Create a subscription to a remote server that replicates tables in the publications mypublication and insert_only and starts replicating immediately on commit:

CREATE SUBSCRIPTION mysub
         CONNECTION 'host=192.168.1.50 port=5432 user=foo dbname=foodb'
        PUBLICATION mypublication, insert_only;

Create a subscription to a remote server that replicates tables in the insert_only publication and does not start replicating until enabled at a later time.

CREATE SUBSCRIPTION mysub
         CONNECTION 'host=192.168.1.50 port=5432 user=foo dbname=foodb'
        PUBLICATION insert_only
               WITH (enabled = false);

Compatibility

CREATE SUBSCRIPTION is a PostgreSQL extension.

See Also

create_table

CREATE TABLE

CREATE TABLE

7

SQL - Language Statements

CREATE TABLE

define a new table

CREATE [

persistence_mode

] TABLE [ IF NOT EXISTS ]

table_name

( [ {

column_name

data_type

[

column_storage

] [

column_compression

] [ COLLATE

collation

] [

column_constraint

[ ... ] ] |

table_constraint

| LIKE

source_table

[

like_option

... ] } [, ... ] ] ) [ INHERITS (

parent_table

[, ... ] ) ] [ PARTITION BY { RANGE | LIST | HASH } ( {

column_name

| (

expression

) } [ COLLATE

collation

] [

opclass

] [, ... ] ) ] [ USING

method

] [ WITH (

storage_parameter

[=

value

] [, ... ] ) | WITHOUT OIDS ] [ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ] [ TABLESPACE

tablespace_name

] CREATE [

persistence_mode

] TABLE [ IF NOT EXISTS ]

table_name

OF

type_name

[ ( {

column_name

[ WITH OPTIONS ] [

column_constraint

[ ... ] ] |

table_constraint

} [, ... ] ) ] [ PARTITION BY { RANGE | LIST | HASH } ( {

column_name

| (

expression

) } [ COLLATE

collation

] [

opclass

] [, ... ] ) ] [ USING

method

] [ WITH (

storage_parameter

[=

value

] [, ... ] ) | WITHOUT OIDS ] [ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ] [ TABLESPACE

tablespace_name

] CREATE [

persistence_mode

] TABLE [ IF NOT EXISTS ]

table_name

PARTITION OF

parent_table

[ ( {

column_name

[ WITH OPTIONS ] [

column_constraint

[ ... ] ] |

table_constraint

} [, ... ] ) ] { FOR VALUES

partition_bound_spec

| DEFAULT } [ PARTITION BY { RANGE | LIST | HASH } ( {

column_name

| (

expression

) } [ COLLATE

collation

] [

opclass

] [, ... ] ) ] [ USING

method

] [ WITH (

storage_parameter

[=

value

] [, ... ] ) | WITHOUT OIDS ] [ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ] [ TABLESPACE

tablespace_name

]

where

persistence_mode

is:

{ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } } | UNLOGGED

and

column_storage

is:

STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN | DEFAULT }

and

column_compression

is:

COMPRESSION

compression_method

and

column_constraint

is:

[ CONSTRAINT

constraint_name

] { NOT NULL [ NO INHERIT ] | NULL | CHECK (

expression

) [ NO INHERIT ] | DEFAULT

default_expr

| GENERATED ALWAYS AS (

generation_expr

) [ STORED | VIRTUAL ] | GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ (

sequence_options

) ] | UNIQUE [ NULLS [ NOT ] DISTINCT ]

index_parameters

| PRIMARY KEY

index_parameters

| REFERENCES

reftable

[ (

refcolumn

) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ] [ ON DELETE

referential_action

] [ ON UPDATE

referential_action

] } [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ] [ ENFORCED | NOT ENFORCED ]

and

table_constraint

is:

[ CONSTRAINT

constraint_name

] { CHECK (

expression

) [ NO INHERIT ] | NOT NULL

column_name

[ NO INHERIT ] | UNIQUE [ NULLS [ NOT ] DISTINCT ] (

column_name

[, ... ] [,

column_name

WITHOUT OVERLAPS ] )

index_parameters

| PRIMARY KEY (

column_name

[, ... ] [,

column_name

WITHOUT OVERLAPS ] )

index_parameters

| EXCLUDE [ USING

index_method

] (

exclude_element

WITH

operator

[, ... ] )

index_parameters

[ WHERE (

predicate

) ] | FOREIGN KEY (

column_name

[, ... ] [, PERIOD

column_name

] ) REFERENCES

reftable

[ (

refcolumn

[, ... ] [, PERIOD

refcolumn

] ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ] [ ON DELETE

referential_action

] [ ON UPDATE

referential_action

] } [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ] [ ENFORCED | NOT ENFORCED ]

and

like_option

is:

{ INCLUDING | EXCLUDING } { COMMENTS | COMPRESSION | CONSTRAINTS | DEFAULTS | GENERATED | IDENTITY | INDEXES | STATISTICS | STORAGE | ALL }

and

partition_bound_spec

is:

IN (

partition_bound_expr

[, ...] ) | FROM ( {

partition_bound_expr

| MINVALUE | MAXVALUE } [, ...] ) TO ( {

partition_bound_expr

| MINVALUE | MAXVALUE } [, ...] ) | WITH ( MODULUS

numeric_literal

, REMAINDER

numeric_literal

)

and

index_parameters

in

UNIQUE

,

PRIMARY KEY

, and

EXCLUDE

constraints are:

[ INCLUDE (

column_name

[, ... ] ) ] [ WITH (

storage_parameter

[=

value

] [, ... ] ) ] [ USING INDEX TABLESPACE

tablespace_name

]

and

exclude_element

in an

EXCLUDE

constraint is:

{

column_name

| (

expression

) } [ COLLATE

collation

] [

opclass

[ (

opclass_parameter

=

value

[, ... ] ) ] ] [ ASC | DESC ] [ NULLS { FIRST | LAST } ]

and

referential_action

in a

FOREIGN KEY

/

REFERENCES

constraint is:

{ NO ACTION | RESTRICT | CASCADE | SET NULL [ (

column_name

[, ... ] ) ] | SET DEFAULT [ (

column_name

[, ... ] ) ] }

Description

CREATE TABLE will create a new, initially empty table in the current database. The table will be owned by the user issuing the command.

The durability characteristics of a table are governed by its persistence mode. By default, the data will be persistent and crash-safe. For less stringent requirements and better performance, a table can be specified as temporary or unlogged.

If a schema name is given (for example, CREATE TABLE myschema.mytable ...) then the table is created in the specified schema. Otherwise it is created in the current schema. Temporary tables exist in a special schema, so a schema name cannot be given when creating a temporary table. The name of the table must be distinct from the name of any other relation (table, sequence, index, view, materialized view, or foreign table) in the same schema.

CREATE TABLE also automatically creates a data type that represents the composite type corresponding to one row of the table. Therefore, tables cannot have the same name as any existing data type in the same schema.

The optional constraint clauses specify constraints (tests) that new or updated rows must satisfy for an insert or update operation to succeed. A constraint is an SQL object that helps define the set of valid values in the table in various ways.

There are two ways to define constraints: table constraints and column constraints. A column constraint is defined as part of a column definition. A table constraint definition is not tied to a particular column, and it can encompass more than one column. Every column constraint can also be written as a table constraint; a column constraint is only a notational convenience for use when the constraint only affects one column.

To be able to create a table, you must have USAGE privilege on all column types or the type in the OF clause, respectively.

Parameters

TEMPORARY or TEMP
If specified, the table is created as a temporary table. Temporary tables are automatically dropped at the end of a session, or optionally at the end of the current transaction (see ON COMMIT below). The default search_path includes the temporary schema first and so identically named existing permanent tables are not chosen for new plans while the temporary table exists, unless they are referenced with schema-qualified names. Any indexes created on a temporary table are automatically temporary as well.

The autovacuum daemon cannot access and therefore cannot vacuum or analyze temporary tables. For this reason, appropriate vacuum and analyze operations should be performed via session SQL commands. For example, if a temporary table is going to be used in complex queries, it is wise to run ANALYZE on the temporary table after it is populated.

Optionally, GLOBAL or LOCAL can be written before TEMPORARY or TEMP. This presently makes no difference in PostgreSQL and is deprecated; see Compatibility below.

UNLOGGED
If specified, the table is created as an unlogged table. Data written to unlogged tables is not written to the write-ahead log (see ???), which makes them considerably faster than ordinary tables. However, they are not crash-safe: an unlogged table is automatically truncated after a crash or unclean shutdown. The contents of an unlogged table are also not replicated to standby servers. Any indexes created on an unlogged table are automatically unlogged as well.

If this is specified, any sequences created together with the unlogged table (for identity or serial columns) are also created as unlogged.

This form is not supported for partitioned tables.

IF NOT EXISTS
Do not throw an error if a relation with the same name already exists. A notice is issued in this case. Note that there is no guarantee that the existing relation is anything like the one that would have been created.

<table_name>
The name (optionally schema-qualified) of the table to be created.

OF type_name
Creates a typed table, which takes its structure from the specified stand-alone composite type (that is, one created using ???) though it still produces a new composite type as well. The table will have a dependency on the referenced type, meaning that cascaded alter and drop actions on that type will propagate to the table.

A typed table always has the same column names and data types as the type it is derived from, so you cannot specify additional columns. But the CREATE TABLE command can add defaults and constraints to the table, as well as specify storage parameters.

<column_name>
The name of a column to be created in the new table.

<data_type>
The data type of the column. This can include array specifiers. For more information on the data types supported by PostgreSQL, refer to ???.

COLLATE collation
The COLLATE clause assigns a collation to the column (which must be of a collatable data type). If not specified, the column data type's default collation is used.

STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN | DEFAULT }
This clause sets the storage mode for the column. This controls whether this column is held inline or in a secondary TOAST table, and whether the data should be compressed or not. PLAIN must be used for fixed-length values such as integer and is inline, uncompressed. MAIN is for inline, compressible data. EXTERNAL is for external, uncompressed data, and EXTENDED is for external, compressed data. Writing DEFAULT sets the storage mode to the default mode for the column's data type. EXTENDED is the default for most data types that support non-PLAIN storage. Use of EXTERNAL will make substring operations on very large text and bytea values run faster, at the penalty of increased storage space. See ??? for more information.

COMPRESSION compression_method
The COMPRESSION clause sets the compression method for the column. Compression is supported only for variable-width data types, and is used only when the column's storage mode is main or extended. (See ??? for information on column storage modes.) Setting this property for a partitioned table has no direct effect, because such tables have no storage of their own, but the configured value will be inherited by newly-created partitions. The supported compression methods are pglz and lz4. (lz4 is available only if --with-lz4 was used when building PostgreSQL.) In addition, <compression_method> can be default to explicitly specify the default behavior, which is to consult the ??? setting at the time of data insertion to determine the method to use.

INHERITS ( parent_table [, ... ] )
The optional INHERITS clause specifies a list of tables from which the new table automatically inherits all columns. Parent tables can be plain tables or foreign tables.

Use of INHERITS creates a persistent relationship between the new child table and its parent table(s). Schema modifications to the parent(s) normally propagate to children as well, and by default the data of the child table is included in scans of the parent(s).

If the same column name exists in more than one parent table, an error is reported unless the data types of the columns match in each of the parent tables. If there is no conflict, then the duplicate columns are merged to form a single column in the new table. If the column name list of the new table contains a column name that is also inherited, the data type must likewise match the inherited column(s), and the column definitions are merged into one. If the new table explicitly specifies a default value for the column, this default overrides any defaults from inherited declarations of the column. Otherwise, any parents that specify default values for the column must all specify the same default, or an error will be reported.

CHECK constraints are merged in essentially the same way as columns: if multiple parent tables and/or the new table definition contain identically-named CHECK constraints, these constraints must all have the same check expression, or an error will be reported. Constraints having the same name and expression will be merged into one copy. A constraint marked NO INHERIT in a parent will not be considered. Notice that an unnamed CHECK constraint in the new table will never be merged, since a unique name will always be chosen for it.

Column STORAGE settings are also copied from parent tables.

If a column in the parent table is an identity column, that property is not inherited. A column in the child table can be declared identity column if desired.

PARTITION BY { RANGE | LIST | HASH } ( { column_name | ( expression ) } [ opclass ] [, ...] )
The optional PARTITION BY clause specifies a strategy of partitioning the table. The table thus created is called a partitioned table. The parenthesized list of columns or expressions forms the partition key for the table. When using range or hash partitioning, the partition key can include multiple columns or expressions (up to 32, but this limit can be altered when building PostgreSQL), but for list partitioning, the partition key must consist of a single column or expression.

Range and list partitioning require a btree operator class, while hash partitioning requires a hash operator class. If no operator class is specified explicitly, the default operator class of the appropriate type will be used; if no default operator class exists, an error will be raised. When hash partitioning is used, the operator class used must implement support function 2 (see ??? for details).

A partitioned table is divided into sub-tables (called partitions), which are created using separate CREATE TABLE commands. The partitioned table is itself empty. A data row inserted into the table is routed to a partition based on the value of columns or expressions in the partition key. If no existing partition matches the values in the new row, an error will be reported.

See ??? for more discussion on table partitioning.

PARTITION OF parent_table { FOR VALUES partition_bound_spec | DEFAULT }
Creates the table as a partition of the specified parent table. The table can be created either as a partition for specific values using FOR VALUES or as a default partition using DEFAULT. Any indexes, constraints and user-defined row-level triggers that exist in the parent table are cloned on the new partition.

The <partition_bound_spec> must correspond to the partitioning method and partition key of the parent table, and must not overlap with any existing partition of that parent. The form with IN is used for list partitioning, the form with FROM and TO is used for range partitioning, and the form with WITH is used for hash partitioning.

<partition_bound_expr> is any variable-free expression (subqueries, window functions, aggregate functions, and set-returning functions are not allowed). Its data type must match the data type of the corresponding partition key column. The expression is evaluated once at table creation time, so it can even contain volatile expressions such as CURRENT_TIMESTAMP.

When creating a list partition, NULL can be specified to signify that the partition allows the partition key column to be null. However, there cannot be more than one such list partition for a given parent table. NULL cannot be specified for range partitions.

When creating a range partition, the lower bound specified with FROM is an inclusive bound, whereas the upper bound specified with TO is an exclusive bound. That is, the values specified in the FROM list are valid values of the corresponding partition key columns for this partition, whereas those in the TO list are not. Note that this statement must be understood according to the rules of row-wise comparison (???). For example, given PARTITION BY RANGE (x,y), a partition bound FROM (1, 2) TO (3, 4) allows x=1 with any y>=2, x=2 with any non-null y, and x=3 with any y<4.

The special values MINVALUE and MAXVALUE may be used when creating a range partition to indicate that there is no lower or upper bound on the column's value. For example, a partition defined using FROM (MINVALUE) TO (10) allows any values less than 10, and a partition defined using FROM (10) TO (MAXVALUE) allows any values greater than or equal to 10.

When creating a range partition involving more than one column, it can also make sense to use MAXVALUE as part of the lower bound, and MINVALUE as part of the upper bound. For example, a partition defined using FROM (0, MAXVALUE) TO (10, MAXVALUE) allows any rows where the first partition key column is greater than 0 and less than or equal to 10. Similarly, a partition defined using FROM ('a', MINVALUE) TO ('b', MINVALUE) allows any rows where the first partition key column starts with "a".

Note that if MINVALUE or MAXVALUE is used for one column of a partitioning bound, the same value must be used for all subsequent columns. For example, (10, MINVALUE, 0) is not a valid bound; you should write (10, MINVALUE, MINVALUE).

Also note that some element types, such as timestamp, have a notion of "infinity", which is just another value that can be stored. This is different from MINVALUE and MAXVALUE, which are not real values that can be stored, but rather they are ways of saying that the value is unbounded. MAXVALUE can be thought of as being greater than any other value, including "infinity" and MINVALUE as being less than any other value, including "minus infinity". Thus the range FROM ('infinity') TO (MAXVALUE) is not an empty range; it allows precisely one value to be stored — "infinity".

If DEFAULT is specified, the table will be created as the default partition of the parent table. This option is not available for hash-partitioned tables. A partition key value not fitting into any other partition of the given parent will be routed to the default partition.

When a table has an existing DEFAULT partition and a new partition is added to it, the default partition must be scanned to verify that it does not contain any rows which properly belong in the new partition. If the default partition contains a large number of rows, this may be slow. The scan will be skipped if the default partition is a foreign table or if it has a constraint which proves that it cannot contain rows which should be placed in the new partition.

When creating a hash partition, a modulus and remainder must be specified. The modulus must be a positive integer, and the remainder must be a non-negative integer less than the modulus. Typically, when initially setting up a hash-partitioned table, you should choose a modulus equal to the number of partitions and assign every table the same modulus and a different remainder (see examples, below). However, it is not required that every partition have the same modulus, only that every modulus which occurs among the partitions of a hash-partitioned table is a factor of the next larger modulus. This allows the number of partitions to be increased incrementally without needing to move all the data at once. For example, suppose you have a hash-partitioned table with 8 partitions, each of which has modulus 8, but find it necessary to increase the number of partitions to 16. You can detach one of the modulus-8 partitions, create two new modulus-16 partitions covering the same portion of the key space (one with a remainder equal to the remainder of the detached partition, and the other with a remainder equal to that value plus 8), and repopulate them with data. You can then repeat this -- perhaps at a later time -- for each modulus-8 partition until none remain. While this may still involve a large amount of data movement at each step, it is still better than having to create a whole new table and move all the data at once.

A partition must have the same column names and types as the partitioned table to which it belongs. Modifications to the column names or types of a partitioned table will automatically propagate to all partitions. CHECK constraints will be inherited automatically by every partition, but an individual partition may specify additional CHECK constraints; additional constraints with the same name and condition as in the parent will be merged with the parent constraint. Defaults may be specified separately for each partition. But note that a partition's default value is not applied when inserting a tuple through a partitioned table.

Rows inserted into a partitioned table will be automatically routed to the correct partition. If no suitable partition exists, an error will occur.

Operations such as TRUNCATE which normally affect a table and all of its inheritance children will cascade to all partitions, but may also be performed on an individual partition.

Note that creating a partition using PARTITION OF requires taking an ACCESS EXCLUSIVE lock on the parent partitioned table. Likewise, dropping a partition with DROP TABLE requires taking an ACCESS EXCLUSIVE lock on the parent table. It is possible to use ALTER TABLE ATTACH/DETACH PARTITION to perform these operations with a weaker lock, thus reducing interference with concurrent operations on the partitioned table.

LIKE source_table [ like_option ... ]
The LIKE clause specifies a table from which the new table automatically copies all column names, their data types, and their not-null constraints.

Unlike INHERITS, the new table and original table are completely decoupled after creation is complete. Changes to the original table will not be applied to the new table, and it is not possible to include data of the new table in scans of the original table.

Also unlike INHERITS, columns and constraints copied by LIKE are not merged with similarly named columns and constraints. If the same name is specified explicitly or in another LIKE clause, an error is signaled.

The optional <like_option> clauses specify which additional properties of the original table to copy. Specifying INCLUDING copies the property, specifying EXCLUDING omits the property. EXCLUDING is the default. If multiple specifications are made for the same kind of object, the last one is used. The available options are:

INCLUDING COMMENTS
Comments for the copied columns, check constraints, not-null constraints, indexes, and extended statistics will be copied. The default behavior is to exclude comments, resulting in the corresponding objects in the new table having no comments.

INCLUDING COMPRESSION
Compression method of the columns will be copied. The default behavior is to exclude compression methods, resulting in columns having the default compression method.

INCLUDING CONSTRAINTS
CHECK constraints will be copied. No distinction is made between column constraints and table constraints. Not-null constraints are always copied to the new table.

INCLUDING DEFAULTS
Default expressions for the copied column definitions will be copied. Otherwise, default expressions are not copied, resulting in the copied columns in the new table having null defaults. Note that copying defaults that call database-modification functions, such as nextval, may create a functional linkage between the original and new tables.

INCLUDING GENERATED
Any generation expressions as well as the stored/virtual choice of copied column definitions will be copied. By default, new columns will be regular base columns.

INCLUDING IDENTITY
Any identity specifications of copied column definitions will be copied. A new sequence is created for each identity column of the new table, separate from the sequences associated with the old table.

INCLUDING INDEXES
Indexes, PRIMARY KEY, UNIQUE, and EXCLUDE constraints on the original table will be created on the new table. Names for the new indexes and constraints are chosen according to the default rules, regardless of how the originals were named. (This behavior avoids possible duplicate-name failures for the new indexes.)

INCLUDING STATISTICS
Extended statistics are copied to the new table.

INCLUDING STORAGE
STORAGE settings for the copied column definitions will be copied. The default behavior is to exclude STORAGE settings, resulting in the copied columns in the new table having type-specific default settings. For more on STORAGE settings, see ???.

INCLUDING ALL
INCLUDING ALL is an abbreviated form selecting all the available individual options. (It could be useful to write individual EXCLUDING clauses after INCLUDING ALL to select all but some specific options.)

The LIKE clause can also be used to copy column definitions from views, foreign tables, or composite types. Inapplicable options (e.g., INCLUDING INDEXES from a view) are ignored.

CONSTRAINT constraint_name
An optional name for a column or table constraint. If the constraint is violated, the constraint name is present in error messages, so constraint names like col must be positive can be used to communicate helpful constraint information to client applications. (Double-quotes are needed to specify constraint names that contain spaces.) If a constraint name is not specified, the system generates a name.

NOT NULL [ NO INHERIT ]
The column is not allowed to contain null values.

A constraint marked with NO INHERIT will not propagate to child tables.

NULL
The column is allowed to contain null values. This is the default.

This clause is only provided for compatibility with non-standard SQL databases. Its use is discouraged in new applications.

CHECK ( expression ) [ NO INHERIT ]
The CHECK clause specifies an expression producing a Boolean result which new or updated rows must satisfy for an insert or update operation to succeed. Expressions evaluating to TRUE or UNKNOWN succeed. Should any row of an insert or update operation produce a FALSE result, an error exception is raised and the insert or update does not alter the database. A check constraint specified as a column constraint should reference that column's value only, while an expression appearing in a table constraint can reference multiple columns.

Currently, CHECK expressions cannot contain subqueries nor refer to variables other than columns of the current row (see ???). The system column tableoid may be referenced, but not any other system column.

A constraint marked with NO INHERIT will not propagate to child tables.

When a table has multiple CHECK constraints, they will be tested for each row in alphabetical order by name, after checking NOT NULL constraints. (PostgreSQL versions before 9.5 did not honor any particular firing order for CHECK constraints.)

DEFAULT default_expr
The DEFAULT clause assigns a default data value for the column whose column definition it appears within. The value is any variable-free expression (in particular, cross-references to other columns in the current table are not allowed). Subqueries are not allowed either. The data type of the default expression must match the data type of the column.

The default expression will be used in any insert operation that does not specify a value for the column. If there is no default for a column, then the default is null.

GENERATED ALWAYS AS ( generation_expr ) [ STORED | VIRTUAL ]
This clause creates the column as a generated column. The column cannot be written to, and when read the result of the specified expression will be returned.

When VIRTUAL is specified, the column will be computed when it is read, and it will not occupy any storage. When STORED is specified, the column will be computed on write and will be stored on disk. VIRTUAL is the default.

The generation expression can refer to other columns in the table, but not other generated columns. Any functions and operators used must be immutable. References to other tables are not allowed.

A virtual generated column cannot have a user-defined type, and the generation expression of a virtual generated column must not reference user-defined functions or types, that is, it can only use built-in functions or types. This applies also indirectly, such as for functions or types that underlie operators or casts. (This restriction does not exist for stored generated columns.)

GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( sequence_options ) ]
This clause creates the column as an identity column. It will have an implicit sequence attached to it and in newly-inserted rows the column will automatically have values from the sequence assigned to it. Such a column is implicitly NOT NULL.

The clauses ALWAYS and BY DEFAULT determine how explicitly user-specified values are handled in INSERT and UPDATE commands.

In an INSERT command, if ALWAYS is selected, a user-specified value is only accepted if the INSERT statement specifies OVERRIDING SYSTEM VALUE. If BY DEFAULT is selected, then the user-specified value takes precedence. See ??? for details. (In the COPY command, user-specified values are always used regardless of this setting.)

In an UPDATE command, if ALWAYS is selected, any update of the column to any value other than DEFAULT will be rejected. If BY DEFAULT is selected, the column can be updated normally. (There is no OVERRIDING clause for the UPDATE command.)

The optional <sequence_options> clause can be used to override the parameters of the sequence. The available options include those shown for ???, plus SEQUENCE NAME name, LOGGED, and UNLOGGED, which allow selection of the name and persistence level of the sequence. Without SEQUENCE NAME, the system chooses an unused name for the sequence. Without LOGGED or UNLOGGED, the sequence will have the same persistence level as the table.

UNIQUE [ NULLS [ NOT ] DISTINCT ] (column constraint); UNIQUE [ NULLS [ NOT ] DISTINCT ] ( column_name [, ... ] [, column_name WITHOUT OVERLAPS ] ) [INCLUDE ( column_name [, ...])] (table constraint)
The UNIQUE constraint specifies that a group of one or more columns of a table can contain only unique values. The behavior of a unique table constraint is the same as that of a unique column constraint, with the additional capability to span multiple columns. The constraint therefore enforces that any two rows must differ in at least one of these columns.

If the WITHOUT OVERLAPS option is specified for the last column, then that column is checked for overlaps instead of equality. In that case, the other columns of the constraint will allow duplicates so long as the duplicates don't overlap in the WITHOUT OVERLAPS column. (This is sometimes called a temporal key, if the column is a range of dates or timestamps, but PostgreSQL allows ranges over any base type.) In effect, such a constraint is enforced with an EXCLUDE constraint rather than a UNIQUE constraint. So for example UNIQUE (id, valid_at WITHOUT OVERLAPS) behaves like EXCLUDE USING GIST (id WITH =, valid_at WITH &&). The WITHOUT OVERLAPS column must have a range or multirange type. Empty ranges/multiranges are not permitted. The non-WITHOUT OVERLAPS columns of the constraint can be any type that can be compared for equality in a GiST index. By default, only range types are supported, but you can use other types by adding the ??? extension (which is the expected way to use this feature).

For the purpose of a unique constraint, null values are not considered equal, unless NULLS NOT DISTINCT is specified.

Each unique constraint should name a set of columns that is different from the set of columns named by any other unique or primary key constraint defined for the table. (Otherwise, redundant unique constraints will be discarded.)

When establishing a unique constraint for a multi-level partition hierarchy, all the columns in the partition key of the target partitioned table, as well as those of all its descendant partitioned tables, must be included in the constraint definition.

Adding a unique constraint will automatically create a unique btree index on the column or group of columns used in the constraint. But if the constraint includes a WITHOUT OVERLAPS clause, it will use a GiST index. The created index has the same name as the unique constraint.

The optional INCLUDE clause adds to that index one or more columns that are simply “payload”: uniqueness is not enforced on them, and the index cannot be searched on the basis of those columns. However they can be retrieved by an index-only scan. Note that although the constraint is not enforced on included columns, it still depends on them. Consequently, some operations on such columns (e.g., DROP COLUMN) can cause cascaded constraint and index deletion.

PRIMARY KEY (column constraint); PRIMARY KEY ( column_name [, ... ] [, column_name WITHOUT OVERLAPS ] ) [INCLUDE ( column_name [, ...])] (table constraint)
The PRIMARY KEY constraint specifies that a column or columns of a table can contain only unique (non-duplicate), nonnull values. Only one primary key can be specified for a table, whether as a column constraint or a table constraint.

The primary key constraint should name a set of columns that is different from the set of columns named by any unique constraint defined for the same table. (Otherwise, the unique constraint is redundant and will be discarded.)

PRIMARY KEY enforces the same data constraints as a combination of UNIQUE and NOT NULL. However, identifying a set of columns as the primary key also provides metadata about the design of the schema, since a primary key implies that other tables can rely on this set of columns as a unique identifier for rows.

When placed on a partitioned table, PRIMARY KEY constraints share the restrictions previously described for UNIQUE constraints.

Adding a PRIMARY KEY constraint will automatically create a unique btree index on the column or group of columns used in the constraint, or GiST if WITHOUT OVERLAPS was specified.

The optional INCLUDE clause adds to that index one or more columns that are simply “payload”: uniqueness is not enforced on them, and the index cannot be searched on the basis of those columns. However they can be retrieved by an index-only scan. Note that although the constraint is not enforced on included columns, it still depends on them. Consequently, some operations on such columns (e.g., DROP COLUMN) can cause cascaded constraint and index deletion.

EXCLUDE [ USING index_method ] ( exclude_element WITH operator [, ... ] ) index_parameters [ WHERE ( predicate ) ]
The EXCLUDE clause defines an exclusion constraint, which guarantees that if any two rows are compared on the specified column(s) or expression(s) using the specified operator(s), not all of these comparisons will return TRUE. If all of the specified operators test for equality, this is equivalent to a UNIQUE constraint, although an ordinary unique constraint will be faster. However, exclusion constraints can specify constraints that are more general than simple equality. For example, you can specify a constraint that no two rows in the table contain overlapping circles (see ???) by using the && operator. The operator(s) are required to be commutative.

Exclusion constraints are implemented using an index that has the same name as the constraint, so each specified operator must be associated with an appropriate operator class (see ???) for the index access method <index_method>. Each <exclude_element> defines a column of the index, so it can optionally specify a collation, an operator class, operator class parameters, and/or ordering options; these are described fully under ???.

The access method must support amgettuple (see ???); at present this means GIN cannot be used. Although it's allowed, there is little point in using B-tree or hash indexes with an exclusion constraint, because this does nothing that an ordinary unique constraint doesn't do better. So in practice the access method will always be GiST or SP-GiST.

The <predicate> allows you to specify an exclusion constraint on a subset of the table; internally this creates a partial index. Note that parentheses are required around the predicate.

When establishing an exclusion constraint for a multi-level partition hierarchy, all the columns in the partition key of the target partitioned table, as well as those of all its descendant partitioned tables, must be included in the constraint definition. Additionally, those columns must be compared using the equality operator. These restrictions ensure that potentially-conflicting rows will exist in the same partition. The constraint may also refer to other columns which are not a part of any partition key, which can be compared using any appropriate operator.

REFERENCES reftable [ ( refcolumn ) ] [ MATCH matchtype ] [ ON DELETE referential_action ] [ ON UPDATE referential_action ] (column constraint); FOREIGN KEY ( column_name [, ... ] [, PERIOD column_name ] ) REFERENCES reftable [ ( refcolumn [, ... ] [, PERIOD refcolumn ] ) ] [ MATCH matchtype ] [ ON DELETE referential_action ] [ ON UPDATE referential_action ] (table constraint)
These clauses specify a foreign key constraint, which requires that a group of one or more columns of the new table must only contain values that match values in the referenced column(s) of some row of the referenced table. If the <refcolumn> list is omitted, the primary key of the <reftable> is used. Otherwise, the <refcolumn> list must refer to the columns of a non-deferrable unique or primary key constraint or be the columns of a non-partial unique index.

If the last column is marked with PERIOD, it is treated in a special way. While the non-PERIOD columns are compared for equality (and there must be at least one of them), the PERIOD column is not. Instead, the constraint is considered satisfied if the referenced table has matching records (based on the non-PERIOD parts of the key) whose combined PERIOD values completely cover the referencing record's. In other words, the reference must have a referent for its entire duration. This column must be a range or multirange type. In addition, the referenced table must have a primary key or unique constraint declared with WITHOUT OVERLAPS. Finally, if the foreign key has a PERIOD <column_name> specification the corresponding <refcolumn>, if present, must also be marked PERIOD. If the <refcolumn> clause is omitted, and thus the reftable's primary key constraint chosen, the primary key must have its final column marked WITHOUT OVERLAPS.

For each pair of referencing and referenced column, if they are of a collatable data type, then the collations must either be both deterministic or else both the same. This ensures that both columns have a consistent notion of equality.

The user must have REFERENCES permission on the referenced table (either the whole table, or the specific referenced columns). The addition of a foreign key constraint requires a SHARE ROW EXCLUSIVE lock on the referenced table. Note that foreign key constraints cannot be defined between temporary tables and permanent tables.

A value inserted into the referencing column(s) is matched against the values of the referenced table and referenced columns using the given match type. There are three match types: MATCH FULL, MATCH PARTIAL, and MATCH SIMPLE (which is the default). MATCH FULL will not allow one column of a multicolumn foreign key to be null unless all foreign key columns are null; if they are all null, the row is not required to have a match in the referenced table. MATCH SIMPLE allows any of the foreign key columns to be null; if any of them are null, the row is not required to have a match in the referenced table. MATCH PARTIAL is not yet implemented. (Of course, NOT NULL constraints can be applied to the referencing column(s) to prevent these cases from arising.)

In addition, when the data in the referenced columns is changed, certain actions are performed on the data in this table's columns. The ON DELETE clause specifies the action to perform when a referenced row in the referenced table is being deleted. Likewise, the ON UPDATE clause specifies the action to perform when a referenced column in the referenced table is being updated to a new value. If the row is updated, but the referenced column is not actually changed, no action is done. Referential actions are executed as part of the data changing command, even if the constraint is deferred. There are the following possible actions for each clause:

NO ACTION
Produce an error if the deletion or update would create a foreign key constraint violation. If the constraint is deferred, this error will be produced at constraint check time if there still exist any referencing rows. This is the default action.

RESTRICT
Produce an error if a row to be deleted or updated matches a row in the referencing table. This prevents the action even if the state after the action would not violate the foreign key constraint. In particular, it prevents updates of referenced rows to values that are distinct but compare as equal. (But it does not prevent “no-op” updates that update a column to the same value.)

In a temporal foreign key, this option is not supported.

CASCADE
Delete any rows referencing the deleted row, or update the values of the referencing column(s) to the new values of the referenced columns, respectively.

In a temporal foreign key, this option is not supported.

SET NULL [ ( column_name [, ... ] ) ]
Set all of the referencing columns, or a specified subset of the referencing columns, to null. A subset of columns can only be specified for ON DELETE actions.

In a temporal foreign key, this option is not supported.

SET DEFAULT [ ( column_name [, ... ] ) ]
Set all of the referencing columns, or a specified subset of the referencing columns, to their default values. A subset of columns can only be specified for ON DELETE actions. (There must be a row in the referenced table matching the default values, if they are not null, or the operation will fail.)

In a temporal foreign key, this option is not supported.

If the referenced column(s) are changed frequently, it might be wise to add an index to the referencing column(s) so that referential actions associated with the foreign key constraint can be performed more efficiently.

DEFERRABLE; NOT DEFERRABLE
This controls whether the constraint can be deferred. A constraint that is not deferrable will be checked immediately after every command. Checking of constraints that are deferrable can be postponed until the end of the transaction (using the SET CONSTRAINTS command). NOT DEFERRABLE is the default. Currently, only UNIQUE, PRIMARY KEY, EXCLUDE, and REFERENCES (foreign key) constraints accept this clause. NOT NULL and CHECK constraints are not deferrable. Note that deferrable constraints cannot be used as conflict arbiters in an INSERT statement that includes an ON CONFLICT clause.

INITIALLY IMMEDIATE; INITIALLY DEFERRED
If a constraint is deferrable, this clause specifies the default time to check the constraint. If the constraint is INITIALLY IMMEDIATE, it is checked after each statement. This is the default. If the constraint is INITIALLY DEFERRED, it is checked only at the end of the transaction. The constraint check time can be altered with the SET CONSTRAINTS command.

ENFORCED; NOT ENFORCED
When the constraint is ENFORCED, then the database system will ensure that the constraint is satisfied, by checking the constraint at appropriate times (after each statement or at the end of the transaction, as appropriate). That is the default. If the constraint is NOT ENFORCED, the database system will not check the constraint. It is then up to the application code to ensure that the constraints are satisfied. The database system might still assume that the data actually satisfies the constraint for optimization decisions where this does not affect the correctness of the result.

NOT ENFORCED constraints can be useful as documentation if the actual checking of the constraint at run time is too expensive.

This is currently only supported for foreign key and CHECK constraints.

USING method
This optional clause specifies the table access method to use to store the contents for the new table; the method needs be an access method of type TABLE. See ??? for more information. If this option is not specified, the default table access method is chosen for the new table. See ??? for more information.

When creating a partition, the table access method is the access method of its partitioned table, if set.

WITH ( storage_parameter [= value] [, ... ] )
This clause specifies optional storage parameters for a table or index; see Storage Parameters below for more information. For backward-compatibility the WITH clause for a table can also include OIDS=FALSE to specify that rows of the new table should not contain OIDs (object identifiers), OIDS=TRUE is not supported anymore.

WITHOUT OIDS
This is backward-compatible syntax for declaring a table WITHOUT OIDS, creating a table WITH OIDS is not supported anymore.

ON COMMIT
The behavior of temporary tables at the end of a transaction block can be controlled using ON COMMIT. The three options are:

PRESERVE ROWS
No special action is taken at the ends of transactions. This is the default behavior.

DELETE ROWS
All rows in the temporary table will be deleted at the end of each transaction block. Essentially, an automatic TRUNCATE is done at each commit. When used on a partitioned table, this is not cascaded to its partitions.

DROP
The temporary table will be dropped at the end of the current transaction block. When used on a partitioned table, this action drops its partitions and when used on tables with inheritance children, it drops the dependent children.

TABLESPACE tablespace_name
The <tablespace_name> is the name of the tablespace in which the new table is to be created. If not specified, ??? is consulted, or ??? if the table is temporary. For partitioned tables, since no storage is required for the table itself, the tablespace specified overrides default_tablespace as the default tablespace to use for any newly created partitions when no other tablespace is explicitly specified.

USING INDEX TABLESPACE tablespace_name
This clause allows selection of the tablespace in which the index associated with a UNIQUE, PRIMARY KEY, or EXCLUDE constraint will be created. If not specified, ??? is consulted, or ??? if the table is temporary.

Storage Parameters

storage parameters

The WITH clause can specify storage parameters for tables, and for indexes associated with a UNIQUE, PRIMARY KEY, or EXCLUDE constraint. Storage parameters for indexes are documented in ???. The storage parameters currently available for tables are listed below. For many of these parameters, as shown, there is an additional parameter with the same name prefixed with toast., which controls the behavior of the table's secondary TOAST table, if any (see ??? for more information about TOAST). If a table parameter value is set and the equivalent toast. parameter is not, the TOAST table will use the table's parameter value. Specifying these parameters for partitioned tables is not supported, but you may specify them for individual leaf partitions.

fillfactor (integer)
The fillfactor for a table is a percentage between 10 and 100. 100 (complete packing) is the default. When a smaller fillfactor is specified, INSERT operations pack table pages only to the indicated percentage; the remaining space on each page is reserved for updating rows on that page. This gives UPDATE a chance to place the updated copy of a row on the same page as the original, which is more efficient than placing it on a different page, and makes heap-only tuple updates more likely. For a table whose entries are never updated, complete packing is the best choice, but in heavily updated tables smaller fillfactors are appropriate. This parameter cannot be set for TOAST tables.

toast_tuple_target (integer)
The toast_tuple_target specifies the minimum tuple length required before we try to compress and/or move long column values into TOAST tables, and is also the target length we try to reduce the length below once toasting begins. This affects columns marked as External (for move), Main (for compression), or Extended (for both) and applies only to new tuples. There is no effect on existing rows. By default this parameter is set to allow at least 4 tuples per block, which with the default block size will be 2040 bytes. Valid values are between 128 bytes and the (block size - header), by default 8160 bytes. Changing this value may not be useful for very short or very long rows. Note that the default setting is often close to optimal, and it is possible that setting this parameter could have negative effects in some cases. This parameter cannot be set for TOAST tables.

toast_value_type (enum)
Specifies the attribute type of chunk_id to use when creating a TOAST relation for this table. The default is oid; oid8 assigns oid8 instead, which allows a larger number of distinct TOASTed values at the price of four extra bytes per out-of-line pointer. This parameter cannot be set for TOAST tables.

The value is only consulted when the TOAST relation is created. Changing it afterwards has no effect on an existing TOAST relation, and a table rewrite such as CLUSTER, REPACK or VACUUM FULL preserves the type already in use. A dump and restore, on the other hand, recreates the TOAST relation and therefore applies the current value of this parameter.

parallel_workers (integer)
This sets the number of workers that should be used to assist a parallel scan of this table. If not set, the system will determine a value based on the relation size. The actual number of workers chosen by the planner or by utility statements that use parallel scans may be less, for example due to the setting of ???.

autovacuum_enabled, toast.autovacuum_enabled (boolean)
Enables or disables the autovacuum daemon for a particular table. If true, the autovacuum daemon will perform automatic VACUUM and/or ANALYZE operations on this table following the rules discussed in ???. If false, this table will not be autovacuumed, except to prevent transaction ID wraparound. See ??? for more about wraparound prevention. Note that the autovacuum daemon does not run at all (except to prevent transaction ID wraparound) if the ??? parameter is false; setting individual tables' storage parameters does not override that. Therefore there is seldom much point in explicitly setting this storage parameter to true, only to false.

vacuum_index_cleanup, toast.vacuum_index_cleanup (enum)
Forces or disables index cleanup when VACUUM is run on this table. The default value is AUTO. With OFF, index cleanup is disabled, with ON it is enabled, and with AUTO a decision is made dynamically, each time VACUUM runs. The dynamic behavior allows VACUUM to avoid needlessly scanning indexes to remove very few dead tuples. Forcibly disabling all index cleanup can speed up VACUUM very significantly, but may also lead to severely bloated indexes if table modifications are frequent. The INDEX_CLEANUP parameter of VACUUM, if specified, overrides the value of this option.

vacuum_truncate, toast.vacuum_truncate (boolean)
Per-table value for ??? parameter. The TRUNCATE parameter of VACUUM, if specified, overrides the value of this option.

autovacuum_parallel_workers (integer)
Per-table value for ??? parameter. If -1 is specified, autovacuum_max_parallel_workers value will be used. If set to 0, parallel vacuum is disabled for this table. The default value is -1.

autovacuum_vacuum_threshold, toast.autovacuum_vacuum_threshold (integer)
Per-table value for ??? parameter.

autovacuum_vacuum_max_threshold, toast.autovacuum_vacuum_max_threshold (integer)
Per-table value for ??? parameter.

autovacuum_vacuum_scale_factor, toast.autovacuum_vacuum_scale_factor (floating point)
Per-table value for ??? parameter.

autovacuum_vacuum_insert_threshold, toast.autovacuum_vacuum_insert_threshold (integer)
Per-table value for ??? parameter. The special value of -1 may be used to disable insert vacuums on the table.

autovacuum_vacuum_insert_scale_factor, toast.autovacuum_vacuum_insert_scale_factor (floating point)
Per-table value for ??? parameter.

autovacuum_analyze_threshold (integer)
Per-table value for ??? parameter.

autovacuum_analyze_scale_factor (floating point)
Per-table value for ??? parameter.

autovacuum_vacuum_cost_delay, toast.autovacuum_vacuum_cost_delay (floating point)
Per-table value for ??? parameter.

autovacuum_vacuum_cost_limit, toast.autovacuum_vacuum_cost_limit (integer)
Per-table value for ??? parameter.

autovacuum_freeze_min_age, toast.autovacuum_freeze_min_age (integer)
Per-table value for ??? parameter. Note that autovacuum will ignore per-table autovacuum_freeze_min_age parameters that are larger than half the system-wide ??? setting.

autovacuum_freeze_max_age, toast.autovacuum_freeze_max_age (integer)
Per-table value for ??? parameter. Note that autovacuum will ignore per-table autovacuum_freeze_max_age parameters that are larger than the system-wide setting (it can only be set smaller).

autovacuum_freeze_table_age, toast.autovacuum_freeze_table_age (integer)
Per-table value for ??? parameter.

autovacuum_multixact_freeze_min_age, toast.autovacuum_multixact_freeze_min_age (integer)
Per-table value for ??? parameter. Note that autovacuum will ignore per-table autovacuum_multixact_freeze_min_age parameters that are larger than half the system-wide ??? setting.

autovacuum_multixact_freeze_max_age, toast.autovacuum_multixact_freeze_max_age (integer)
Per-table value for ??? parameter. Note that autovacuum will ignore per-table autovacuum_multixact_freeze_max_age parameters that are larger than the system-wide setting (it can only be set smaller).

autovacuum_multixact_freeze_table_age, toast.autovacuum_multixact_freeze_table_age (integer)
Per-table value for ??? parameter.

log_autovacuum_min_duration, toast.log_autovacuum_min_duration (integer)
Per-table value for ??? parameter.

log_autoanalyze_min_duration (integer)
Per-table value for ??? parameter.

vacuum_max_eager_freeze_failure_rate, toast.vacuum_max_eager_freeze_failure_rate (floating point)
Per-table value for ??? parameter.

user_catalog_table (boolean)
Declare the table as an additional catalog table for purposes of logical replication. See ??? for details. This parameter cannot be set for TOAST tables.

Notes

PostgreSQL automatically creates an index for each unique constraint and primary key constraint to enforce uniqueness. Thus, it is not necessary to create an index explicitly for primary key columns. (See ??? for more information.)

Unique constraints and primary keys are not inherited in the current implementation. This makes the combination of inheritance and unique constraints rather dysfunctional.

A table cannot have more than 1600 columns. (In practice, the effective limit is usually lower because of tuple-length constraints.)

Examples

Create table films and table distributors:

CREATE TABLE films (
    code        char(5) CONSTRAINT firstkey PRIMARY KEY,
    title       varchar(40) NOT NULL,
    did         integer NOT NULL,
    date_prod   date,
    kind        varchar(10),
    len         interval hour to minute
);

CREATE TABLE distributors (
     did    integer PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY,
     name   varchar(40) NOT NULL CHECK (name <> '')
);

Create a table with a 2-dimensional array:

CREATE TABLE array_int (
    vector  int[][]
);

Define a unique table constraint for the table films. Unique table constraints can be defined on one or more columns of the table:

CREATE TABLE films (
    code        char(5),
    title       varchar(40),
    did         integer,
    date_prod   date,
    kind        varchar(10),
    len         interval hour to minute,
    CONSTRAINT production UNIQUE(date_prod)
);

Define a check column constraint:

CREATE TABLE distributors (
    did     integer CHECK (did > 100),
    name    varchar(40)
);

Define a check table constraint:

CREATE TABLE distributors (
    did     integer,
    name    varchar(40),
    CONSTRAINT con1 CHECK (did > 100 AND name <> '')
);

Define a primary key table constraint for the table films:

CREATE TABLE films (
    code        char(5),
    title       varchar(40),
    did         integer,
    date_prod   date,
    kind        varchar(10),
    len         interval hour to minute,
    CONSTRAINT code_title PRIMARY KEY(code,title)
);

Define a primary key constraint for table distributors. The following two examples are equivalent, the first using the table constraint syntax, the second the column constraint syntax:

CREATE TABLE distributors (
    did     integer,
    name    varchar(40),
    PRIMARY KEY(did)
);

CREATE TABLE distributors (
    did     integer PRIMARY KEY,
    name    varchar(40)
);

Assign a literal constant default value for the column name, arrange for the default value of column did to be generated by selecting the next value of a sequence object, and make the default value of modtime be the time at which the row is inserted:

CREATE TABLE distributors (
    name      varchar(40) DEFAULT 'Luso Films',
    did       integer DEFAULT nextval('distributors_serial'),
    modtime   timestamp DEFAULT current_timestamp
);

Define two NOT NULL column constraints on the table distributors, one of which is explicitly given a name:

CREATE TABLE distributors (
    did     integer CONSTRAINT no_null NOT NULL,
    name    varchar(40) NOT NULL
);

Define a unique constraint for the name column:

CREATE TABLE distributors (
    did     integer,
    name    varchar(40) UNIQUE
);

The same, specified as a table constraint:

CREATE TABLE distributors (
    did     integer,
    name    varchar(40),
    UNIQUE(name)
);

Create the same table, specifying 70% fill factor for both the table and its unique index:

CREATE TABLE distributors (
    did     integer,
    name    varchar(40),
    UNIQUE(name) WITH (fillfactor=70)
)
WITH (fillfactor=70);

Create table circles with an exclusion constraint that prevents any two circles from overlapping:

CREATE TABLE circles (
    c circle,
    EXCLUDE USING gist (c WITH &&)
);

Create table cinemas in tablespace diskvol1:

CREATE TABLE cinemas (
        id serial,
        name text,
        location text
) TABLESPACE diskvol1;

Create a composite type and a typed table:

CREATE TYPE employee_type AS (name text, salary numeric);

CREATE TABLE employees OF employee_type (
    PRIMARY KEY (name),
    salary WITH OPTIONS DEFAULT 1000
);

Create a range partitioned table:

CREATE TABLE measurement (
    logdate         date NOT NULL,
    peaktemp        int,
    unitsales       int
) PARTITION BY RANGE (logdate);

Create a range partitioned table with multiple columns in the partition key:

CREATE TABLE measurement_year_month (
    logdate         date NOT NULL,
    peaktemp        int,
    unitsales       int
) PARTITION BY RANGE (EXTRACT(YEAR FROM logdate), EXTRACT(MONTH FROM logdate));

Create a list partitioned table:

CREATE TABLE cities (
    city_id      bigserial NOT NULL,
    name         text NOT NULL,
    population   bigint
) PARTITION BY LIST (left(lower(name), 1));

Create a hash partitioned table:

CREATE TABLE orders (
    order_id     bigint NOT NULL,
    cust_id      bigint NOT NULL,
    status       text
) PARTITION BY HASH (order_id);

Create partition of a range partitioned table:

CREATE TABLE measurement_y2016m07
    PARTITION OF measurement (
    unitsales DEFAULT 0
) FOR VALUES FROM ('2016-07-01') TO ('2016-08-01');

Create a few partitions of a range partitioned table with multiple columns in the partition key:

CREATE TABLE measurement_ym_older
    PARTITION OF measurement_year_month
    FOR VALUES FROM (MINVALUE, MINVALUE) TO (2016, 11);

CREATE TABLE measurement_ym_y2016m11
    PARTITION OF measurement_year_month
    FOR VALUES FROM (2016, 11) TO (2016, 12);

CREATE TABLE measurement_ym_y2016m12
    PARTITION OF measurement_year_month
    FOR VALUES FROM (2016, 12) TO (2017, 01);

CREATE TABLE measurement_ym_y2017m01
    PARTITION OF measurement_year_month
    FOR VALUES FROM (2017, 01) TO (2017, 02);

Create partition of a list partitioned table:

CREATE TABLE cities_ab
    PARTITION OF cities (
    CONSTRAINT city_id_nonzero CHECK (city_id != 0)
) FOR VALUES IN ('a', 'b');

Create partition of a list partitioned table that is itself further partitioned and then add a partition to it:

CREATE TABLE cities_ab
    PARTITION OF cities (
    CONSTRAINT city_id_nonzero CHECK (city_id != 0)
) FOR VALUES IN ('a', 'b') PARTITION BY RANGE (population);

CREATE TABLE cities_ab_10000_to_100000
    PARTITION OF cities_ab FOR VALUES FROM (10000) TO (100000);

Create partitions of a hash partitioned table:

CREATE TABLE orders_p1 PARTITION OF orders
    FOR VALUES WITH (MODULUS 4, REMAINDER 0);
CREATE TABLE orders_p2 PARTITION OF orders
    FOR VALUES WITH (MODULUS 4, REMAINDER 1);
CREATE TABLE orders_p3 PARTITION OF orders
    FOR VALUES WITH (MODULUS 4, REMAINDER 2);
CREATE TABLE orders_p4 PARTITION OF orders
    FOR VALUES WITH (MODULUS 4, REMAINDER 3);

Create a default partition:

CREATE TABLE cities_partdef
    PARTITION OF cities DEFAULT;

Compatibility

The CREATE TABLE command conforms to the SQL standard, with exceptions listed below.

Temporary Tables

Although the syntax of CREATE TEMPORARY TABLE resembles that of the SQL standard, the effect is not the same. In the standard, temporary tables are defined just once and automatically exist (starting with empty contents) in every session that needs them. PostgreSQL instead requires each session to issue its own CREATE TEMPORARY TABLE command for each temporary table to be used. This allows different sessions to use the same temporary table name for different purposes, whereas the standard's approach constrains all instances of a given temporary table name to have the same table structure.

The standard's definition of the behavior of temporary tables is widely ignored. PostgreSQL's behavior on this point is similar to that of several other SQL databases.

The SQL standard also distinguishes between global and local temporary tables, where a local temporary table has a separate set of contents for each SQL module within each session, though its definition is still shared across sessions. Since PostgreSQL does not support SQL modules, this distinction is not relevant in PostgreSQL.

For compatibility's sake, PostgreSQL will accept the GLOBAL and LOCAL keywords in a temporary table declaration, but they currently have no effect. Use of these keywords is discouraged, since future versions of PostgreSQL might adopt a more standard-compliant interpretation of their meaning.

The ON COMMIT clause for temporary tables also resembles the SQL standard, but has some differences. If the ON COMMIT clause is omitted, SQL specifies that the default behavior is ON COMMIT DELETE ROWS. However, the default behavior in PostgreSQL is ON COMMIT PRESERVE ROWS. The ON COMMIT DROP option does not exist in SQL.

Non-Deferred Uniqueness Constraints

When a UNIQUE or PRIMARY KEY constraint is not deferrable, PostgreSQL checks for uniqueness immediately whenever a row is inserted or modified. The SQL standard says that uniqueness should be enforced only at the end of the statement; this makes a difference when, for example, a single command updates multiple key values. To obtain standard-compliant behavior, declare the constraint as DEFERRABLE but not deferred (i.e., INITIALLY IMMEDIATE). Be aware that this can be significantly slower than immediate uniqueness checking.

Column Check Constraints

The SQL standard says that CHECK column constraints can only refer to the column they apply to; only CHECK table constraints can refer to multiple columns. PostgreSQL does not enforce this restriction; it treats column and table check constraints alike.

EXCLUDE Constraint

The EXCLUDE constraint type is a PostgreSQL extension.

Foreign Key Constraints

The ability to specify column lists in the foreign key actions SET DEFAULT and SET NULL is a PostgreSQL extension.

It is a PostgreSQL extension that a foreign key constraint may reference columns of a unique index instead of columns of a primary key or unique constraint.

NULL “Constraint”

The NULL “constraint” (actually a non-constraint) is a PostgreSQL extension to the SQL standard that is included for compatibility with some other database systems (and for symmetry with the NOT NULL constraint). Since it is the default for any column, its presence is simply noise.

Constraint Naming

The SQL standard says that table and domain constraints must have names that are unique across the schema containing the table or domain. PostgreSQL is laxer: it only requires constraint names to be unique across the constraints attached to a particular table or domain. However, this extra freedom does not exist for index-based constraints (UNIQUE, PRIMARY KEY, and EXCLUDE constraints), because the associated index is named the same as the constraint, and index names must be unique across all relations within the same schema.

Inheritance

Multiple inheritance via the INHERITS clause is a PostgreSQL language extension. SQL:1999 and later define single inheritance using a different syntax and different semantics. SQL:1999-style inheritance is not yet supported by PostgreSQL.

Zero-Column Tables

PostgreSQL allows a table of no columns to be created (for example, CREATE TABLE foo();). This is an extension from the SQL standard, which does not allow zero-column tables. Zero-column tables are not in themselves very useful, but disallowing them creates odd special cases for ALTER TABLE DROP COLUMN, so it seems cleaner to ignore this spec restriction.

Multiple Identity Columns

PostgreSQL allows a table to have more than one identity column. The standard specifies that a table can have at most one identity column. This is relaxed mainly to give more flexibility for doing schema changes or migrations. Note that the INSERT command supports only one override clause that applies to the entire statement, so having multiple identity columns with different behaviors is not well supported.

Generated Columns

The options STORED and VIRTUAL are not standard but are also used by other SQL implementations. The SQL standard does not specify the storage of generated columns.

LIKE Clause

While a LIKE clause exists in the SQL standard, many of the options that PostgreSQL accepts for it are not in the standard, and some of the standard's options are not implemented by PostgreSQL.

WITH Clause

The WITH clause is a PostgreSQL extension; storage parameters are not in the standard.

Tablespaces

The PostgreSQL concept of tablespaces is not part of the standard. Hence, the clauses TABLESPACE and USING INDEX TABLESPACE are extensions.

Typed Tables

Typed tables implement a subset of the SQL standard. According to the standard, a typed table has columns corresponding to the underlying composite type as well as one other column that is the “self-referencing column”. PostgreSQL does not support self-referencing columns explicitly.

PARTITION BY Clause

The PARTITION BY clause is a PostgreSQL extension.

PARTITION OF Clause

The PARTITION OF clause is a PostgreSQL extension.

See Also

create_table_as

CREATE TABLE AS

CREATE TABLE AS

7

SQL - Language Statements

CREATE TABLE AS

define a new table from the results of a query

CREATE [

persistence_mode

] TABLE [ IF NOT EXISTS ]

table_name

[ (

column_name

[, ...] ) ] [ USING

method

] [ WITH (

storage_parameter

[=

value

] [, ... ] ) | WITHOUT OIDS ] [ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ] [ TABLESPACE

tablespace_name

] AS

query

[ WITH [ NO ] DATA ]

where

persistence_mode

is:

{ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } } | UNLOGGED

Description

CREATE TABLE AS creates a table and fills it with data computed by a SELECT command. The table columns have the names and data types associated with the output columns of the SELECT (except that you can override the column names by giving an explicit list of new column names).

CREATE TABLE AS bears some resemblance to creating a view, but it is really quite different: it creates a new table and evaluates the query just once to fill the new table initially. The new table will not track subsequent changes to the source tables of the query. In contrast, a view re-evaluates its defining SELECT statement whenever it is queried.

CREATE TABLE AS requires CREATE privilege on the schema used for the table.

Parameters

GLOBAL or LOCAL
Ignored for compatibility. Use of these keywords is deprecated; refer to ??? for details.

TEMPORARY or TEMP
If specified, the table is created as a temporary table. Refer to ??? for details.

UNLOGGED
If specified, the table is created as an unlogged table. Refer to ??? for details.

IF NOT EXISTS
Do not throw an error if a relation with the same name already exists; simply issue a notice and leave the table unmodified.

<table_name>
The name (optionally schema-qualified) of the table to be created.

<column_name>
The name of a column in the new table. If column names are not provided, they are taken from the output column names of the query.

USING method
This optional clause specifies the table access method to use to store the contents for the new table; the method needs be an access method of type TABLE. See ??? for more information. If this option is not specified, the default table access method is chosen for the new table. See ??? for more information.

WITH ( storage_parameter [= value] [, ... ] )
This clause specifies optional storage parameters for the new table; see ??? in the ??? documentation for more information. For backward-compatibility the WITH clause for a table can also include OIDS=FALSE to specify that rows of the new table should contain no OIDs (object identifiers), OIDS=TRUE is not supported anymore.

WITHOUT OIDS
This is backward-compatible syntax for declaring a table WITHOUT OIDS, creating a table WITH OIDS is not supported anymore.

ON COMMIT
The behavior of temporary tables at the end of a transaction block can be controlled using ON COMMIT. The three options are:

PRESERVE ROWS
No special action is taken at the ends of transactions. This is the default behavior.

DELETE ROWS
All rows in the temporary table will be deleted at the end of each transaction block. Essentially, an automatic TRUNCATE is done at each commit.

DROP
The temporary table will be dropped at the end of the current transaction block.

TABLESPACE tablespace_name
The <tablespace_name> is the name of the tablespace in which the new table is to be created. If not specified, ??? is consulted, or ??? if the table is temporary.

<query>
A SELECT, TABLE, or VALUES command, or an EXECUTE command that runs a prepared SELECT, TABLE, or VALUES query.

WITH [ NO ] DATA
This clause specifies whether or not the data produced by the query should be copied into the new table. If not, only the table structure is copied. The default is to copy the data.

Notes

This command is functionally similar to ???, but it is preferred since it is less likely to be confused with other uses of the SELECT INTO syntax. Furthermore, CREATE TABLE AS offers a superset of the functionality offered by SELECT INTO.

Examples

Create a new table films_recent consisting of only recent entries from the table films:

CREATE TABLE films_recent AS
  SELECT * FROM films WHERE date_prod >= '2002-01-01';

To copy a table completely, the short form using the TABLE command can also be used:

CREATE TABLE films2 AS
  TABLE films;

Create a new temporary table films_recent, consisting of only recent entries from the table films, using a prepared statement. The new table will be dropped at commit:

PREPARE recentfilms(date) AS
  SELECT * FROM films WHERE date_prod > $1;
CREATE TEMP TABLE films_recent ON COMMIT DROP AS
  EXECUTE recentfilms('2002-01-01');

Compatibility

CREATE TABLE AS conforms to the SQL standard. The following are nonstandard extensions:

See Also

create_tablespace

CREATE TABLESPACE

CREATE TABLESPACE

7

SQL - Language Statements

CREATE TABLESPACE

define a new tablespace

CREATE TABLESPACE

tablespace_name

[ OWNER {

new_owner

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } ] LOCATION '

directory

' [ WITH (

tablespace_option

=

value

[, ... ] ) ]

Description

CREATE TABLESPACE registers a new cluster-wide tablespace. The tablespace name must be distinct from the name of any existing tablespace in the database cluster.

A tablespace allows superusers to define an alternative location on the file system where the data files containing database objects (such as tables and indexes) can reside.

A user with appropriate privileges can pass <tablespace_name> to CREATE DATABASE, CREATE TABLE, CREATE INDEX or ADD CONSTRAINT to have the data files for these objects stored within the specified tablespace.

[!WARNING] A tablespace cannot be used independently of the cluster in which it is defined; see ???.

Parameters

<tablespace_name>
The name of a tablespace to be created. The name cannot begin with pg_, as such names are reserved for system tablespaces.

<user_name>
The name of the user who will own the tablespace. If omitted, defaults to the user executing the command. Only superusers can create tablespaces, but they can assign ownership of tablespaces to non-superusers.

<directory>
The directory that will be used for the tablespace. The directory must exist (CREATE TABLESPACE will not create it), should be empty, and must be owned by the PostgreSQL system user. The directory must be specified by an absolute path name.

<tablespace_option>
A tablespace parameter to be set or reset. Currently, the only available parameters are seq_page_cost, random_page_cost, effective_io_concurrency and maintenance_io_concurrency. Setting these values for a particular tablespace will override the planner's usual estimate of the cost of reading pages from tables in that tablespace, and how many concurrent I/Os are issued, as established by the configuration parameters of the same name (see ???, ???, ???, ???). This may be useful if one tablespace is located on a disk which is faster or slower than the remainder of the I/O subsystem.

Notes

CREATE TABLESPACE cannot be executed inside a transaction block.

Examples

To create a tablespace dbspace at file system location /data/dbs, first create the directory using operating system facilities and set the correct ownership:

mkdir /data/dbs
chown postgres:postgres /data/dbs

Then issue the tablespace creation command inside PostgreSQL:

CREATE TABLESPACE dbspace LOCATION '/data/dbs';

To create a tablespace owned by a different database user, use a command like this:

CREATE TABLESPACE indexspace OWNER genevieve LOCATION '/data/indexes';

Compatibility

CREATE TABLESPACE is a PostgreSQL extension.

See Also

create_transform

CREATE TRANSFORM

CREATE TRANSFORM

7

SQL - Language Statements

CREATE TRANSFORM

define a new transform

CREATE [ OR REPLACE ] TRANSFORM FOR

type_name

LANGUAGE

lang_name

( FROM SQL WITH FUNCTION

from_sql_function_name

[ (

argument_type

[, ...]) ], TO SQL WITH FUNCTION

to_sql_function_name

[ (

argument_type

[, ...]) ] );

Description

CREATE TRANSFORM defines a new transform. CREATE OR REPLACE TRANSFORM will either create a new transform, or replace an existing definition.

A transform specifies how to adapt a data type to a procedural language. For example, when writing a function in PL/Python using the hstore type, PL/Python has no prior knowledge how to present hstore values in the Python environment. Language implementations usually default to using the text representation, but that is inconvenient when, for example, an associative array or a list would be more appropriate.

A transform specifies two functions:

It is not necessary to provide both of these functions. If one is not specified, the language-specific default behavior will be used if necessary. (To prevent a transformation in a certain direction from happening at all, you could also write a transform function that always errors out.)

To be able to create a transform, you must own and have USAGE privilege on the type, have USAGE privilege on the language, and own and have EXECUTE privilege on the from-SQL and to-SQL functions, if specified.

Parameters

<type_name>
The name of the data type of the transform.

<lang_name>
The name of the language of the transform.

from_sql_function_name[(argument_type [, ...])]
The name of the function for converting the type from the SQL environment to the language. It must take one argument of type internal and return type internal. The actual argument will be of the type for the transform, and the function should be coded as if it were. (But it is not allowed to declare an SQL-level function returning internal without at least one argument of type internal.) The actual return value will be something specific to the language implementation. If no argument list is specified, the function name must be unique in its schema.

to_sql_function_name[(argument_type [, ...])]
The name of the function for converting the type from the language to the SQL environment. It must take one argument of type internal and return the type that is the type for the transform. The actual argument value will be something specific to the language implementation. If no argument list is specified, the function name must be unique in its schema.

Notes

Use DROP TRANSFORM to remove transforms.

Examples

To create a transform for type hstore and language plpython3u, first set up the type and the language:

CREATE TYPE hstore ...;

CREATE EXTENSION plpython3u;

Then create the necessary functions:

CREATE FUNCTION hstore_to_plpython(val internal) RETURNS internal
LANGUAGE C STRICT IMMUTABLE
AS ...;

CREATE FUNCTION plpython_to_hstore(val internal) RETURNS hstore
LANGUAGE C STRICT IMMUTABLE
AS ...;

And finally create the transform to connect them all together:

CREATE TRANSFORM FOR hstore LANGUAGE plpython3u (
    FROM SQL WITH FUNCTION hstore_to_plpython(internal),
    TO SQL WITH FUNCTION plpython_to_hstore(internal)
);

In practice, these commands would be wrapped up in an extension.

The contrib section contains a number of extensions that provide transforms, which can serve as real-world examples.

Compatibility

This form of CREATE TRANSFORM is a PostgreSQL extension. There is a CREATE TRANSFORM command in the SQL standard, but it is for adapting data types to client languages. That usage is not supported by PostgreSQL.

See Also

???, ???, ???, ???

create_trigger

CREATE TRIGGER

transition tables

ephemeral named relation

CREATE TRIGGER

7

SQL - Language Statements

CREATE TRIGGER

define a new trigger

CREATE [ OR REPLACE ] [ CONSTRAINT ] TRIGGER

name

{ BEFORE | AFTER | INSTEAD OF } {

event

[ OR ... ] } ON

table_name

[ FROM

referenced_table_name

] [ NOT DEFERRABLE | [ DEFERRABLE ] [ INITIALLY IMMEDIATE | INITIALLY DEFERRED ] ] [ ENFORCED ] [ REFERENCING { { OLD | NEW } TABLE [ AS ]

transition_relation_name

} [ ... ] ] [ FOR [ EACH ] { ROW | STATEMENT } ] [ WHEN (

condition

) ] EXECUTE { FUNCTION | PROCEDURE }

function_name

(

arguments

)

where

event

can be one of:

INSERT UPDATE [ OF

column_name

[, ... ] ] DELETE TRUNCATE

Description

CREATE TRIGGER creates a new trigger. CREATE OR REPLACE TRIGGER will either create a new trigger, or replace an existing trigger. The trigger will be associated with the specified table, view, or foreign table and will execute the specified function <function_name> when certain operations are performed on that table.

To replace the current definition of an existing trigger, use CREATE OR REPLACE TRIGGER, specifying the existing trigger's name and parent table. All other properties are replaced.

The trigger can be specified to fire before the operation is attempted on a row (before constraints are checked and the INSERT, UPDATE, or DELETE is attempted); or after the operation has completed (after constraints are checked and the INSERT, UPDATE, or DELETE has completed); or instead of the operation (in the case of inserts, updates or deletes on a view). If the trigger fires before or instead of the event, the trigger can skip the operation for the current row, or change the row being inserted (for INSERT and UPDATE operations only). If the trigger fires after the event, all changes, including the effects of other triggers, are “visible” to the trigger.

A trigger that is marked FOR EACH ROW is called once for every row that the operation modifies. For example, a DELETE that affects 10 rows will cause any ON DELETE triggers on the target relation to be called 10 separate times, once for each deleted row. In contrast, a trigger that is marked FOR EACH STATEMENT only executes once for any given operation, regardless of how many rows it modifies (in particular, an operation that modifies zero rows will still result in the execution of any applicable FOR EACH STATEMENT triggers).

Triggers that are specified to fire INSTEAD OF the trigger event must be marked FOR EACH ROW, and can only be defined on views. BEFORE and AFTER triggers on a view must be marked as FOR EACH STATEMENT.

In addition, triggers may be defined to fire for TRUNCATE, though only FOR EACH STATEMENT.

The following table summarizes which types of triggers may be used on tables, views, and foreign tables:

When Event Row-level Statement-level
BEFORE INSERT/UPDATE/DELETE Tables and foreign tables Tables, views, and foreign tables
TRUNCATE — Tables and foreign tables
AFTER INSERT/UPDATE/DELETE Tables and foreign tables Tables, views, and foreign tables
TRUNCATE — Tables and foreign tables
INSTEAD OF INSERT/UPDATE/DELETE Views —
TRUNCATE — —

Also, a trigger definition can specify a Boolean WHEN condition, which will be tested to see whether the trigger should be fired. In row-level triggers the WHEN condition can examine the old and/or new values of columns of the row. Statement-level triggers can also have WHEN conditions, although the feature is not so useful for them since the condition cannot refer to any values in the table.

If multiple triggers of the same kind are defined for the same event, they will be fired in alphabetical order by name.

When the CONSTRAINT option is specified, this command creates a constraint trigger. This is the same as a regular trigger except that the timing of the trigger firing can be adjusted using SET CONSTRAINTS. Constraint triggers must be AFTER ROW triggers on plain tables (not foreign tables). They can be fired either at the end of the statement causing the triggering event, or at the end of the containing transaction; in the latter case they are said to be deferred. A pending deferred-trigger firing can also be forced to happen immediately by using SET CONSTRAINTS. Constraint triggers are expected to raise an exception when the constraints they implement are violated.

The REFERENCING option enables collection of transition relations, which are row sets that include all of the rows inserted, deleted, or modified by the current SQL statement. This feature lets the trigger see a global view of what the statement did, not just one row at a time. This option is only allowed for an AFTER trigger on a plain table (not a foreign table). The trigger should not be a constraint trigger. Also, if the trigger is an UPDATE trigger, it must not specify a <column_name> list when using this option. OLD TABLE may only be specified once, and only for a trigger that can fire on UPDATE or DELETE; it creates a transition relation containing the before-images of all rows updated or deleted by the statement. Similarly, NEW TABLE may only be specified once, and only for a trigger that can fire on UPDATE or INSERT; it creates a transition relation containing the after-images of all rows updated or inserted by the statement.

SELECT does not modify any rows so you cannot create SELECT triggers. Rules and views may provide workable solutions to problems that seem to need SELECT triggers.

Refer to ??? for more information about triggers.

Parameters

<name>
The name to give the new trigger. This must be distinct from the name of any other trigger for the same table. The name cannot be schema-qualified — the trigger inherits the schema of its table. For a constraint trigger, this is also the name to use when modifying the trigger's behavior using SET CONSTRAINTS.

BEFORE; AFTER; INSTEAD OF
Determines whether the function is called before, after, or instead of the event. A constraint trigger can only be specified as AFTER.

<event>
One of INSERT, UPDATE, DELETE, or TRUNCATE; this specifies the event that will fire the trigger. Multiple events can be specified using OR, except when transition relations are requested.

For UPDATE events, it is possible to specify a list of columns using this syntax: UPDATE OF <column_name1> [, <column_name2> ... ] The trigger will only fire if at least one of the listed columns is mentioned as a target of the UPDATE command or if one of the listed columns is a generated column that depends on a column that is the target of the UPDATE.

INSTEAD OF UPDATE events do not allow a list of columns. A column list cannot be specified when requesting transition relations, either.

<table_name>
The name (optionally schema-qualified) of the table, view, or foreign table the trigger is for.

<referenced_table_name>
The (possibly schema-qualified) name of another table referenced by the constraint. This option is used for foreign-key constraints and is not recommended for general use. This can only be specified for constraint triggers.

DEFERRABLE; NOT DEFERRABLE; INITIALLY IMMEDIATE; INITIALLY DEFERRED
The default timing of the trigger. See the ??? documentation for details of these constraint options. This can only be specified for constraint triggers.

ENFORCED
This is a noise word. Constraint triggers are always enforced.

REFERENCING
This keyword immediately precedes the declaration of one or two relation names that provide access to the transition relations of the triggering statement.

OLD TABLE; NEW TABLE
This clause indicates whether the following relation name is for the before-image transition relation or the after-image transition relation.

<transition_relation_name>
The (unqualified) name to be used within the trigger for this transition relation.

FOR EACH ROW; FOR EACH STATEMENT
This specifies whether the trigger function should be fired once for every row affected by the trigger event, or just once per SQL statement. If neither is specified, FOR EACH STATEMENT is the default. Constraint triggers can only be specified FOR EACH ROW.

<condition>
A Boolean expression that determines whether the trigger function will actually be executed. If WHEN is specified, the function will only be called if the <condition> returns true. In FOR EACH ROW triggers, the WHEN condition can refer to columns of the old and/or new row values by writing OLD.column_name or NEW.column_name respectively. Of course, INSERT triggers cannot refer to OLD and DELETE triggers cannot refer to NEW.

INSTEAD OF triggers do not support WHEN conditions.

Currently, WHEN expressions cannot contain subqueries.

Note that for constraint triggers, evaluation of the WHEN condition is not deferred, but occurs immediately after the row update operation is performed. If the condition does not evaluate to true then the trigger is not queued for deferred execution.

<function_name>
A user-supplied function that is declared as taking no arguments and returning type trigger, which is executed when the trigger fires.

In the syntax of CREATE TRIGGER, the keywords FUNCTION and PROCEDURE are equivalent, but the referenced function must in any case be a function, not a procedure. The use of the keyword PROCEDURE here is historical and deprecated.

<arguments>
An optional comma-separated list of arguments to be provided to the function when the trigger is executed. The arguments are literal string constants. Simple names and numeric constants can be written here, too, but they will all be converted to strings. Please check the description of the implementation language of the trigger function to find out how these arguments can be accessed within the function; it might be different from normal function arguments.

Notes

To create or replace a trigger on a table, the user must have the TRIGGER privilege on the table. The user must also have EXECUTE privilege on the trigger function.

Use DROP TRIGGER to remove a trigger.

Creating a row-level trigger on a partitioned table will cause an identical “clone” trigger to be created on each of its existing partitions; and any partitions created or attached later will have an identical trigger, too. If there is a conflictingly-named trigger on a child partition already, an error occurs unless CREATE OR REPLACE TRIGGER is used, in which case that trigger is replaced with a clone trigger. When a partition is detached from its parent, its clone triggers are removed.

A column-specific trigger (one defined using the UPDATE OF column_name syntax) will fire when any of its columns are listed as targets in the UPDATE command's SET list. It is possible for a column's value to change even when the trigger is not fired, because changes made to the row's contents by BEFORE UPDATE triggers are not considered. Conversely, a command such as UPDATE ... SET x = x ... will fire a trigger on column x, even though the column's value did not change.

In a BEFORE trigger, the WHEN condition is evaluated just before the function is or would be executed, so using WHEN is not materially different from testing the same condition at the beginning of the trigger function. Note in particular that the NEW row seen by the condition is the current value, as possibly modified by earlier triggers. Also, a BEFORE trigger's WHEN condition is not allowed to examine the system columns of the NEW row (such as ctid), because those won't have been set yet.

In an AFTER trigger, the WHEN condition is evaluated just after the row update occurs, and it determines whether an event is queued to fire the trigger at the end of statement. So when an AFTER trigger's WHEN condition does not return true, it is not necessary to queue an event nor to re-fetch the row at end of statement. This can result in significant speedups in statements that modify many rows, if the trigger only needs to be fired for a few of the rows.

In some cases it is possible for a single SQL command to fire more than one kind of trigger. For instance an INSERT with an ON CONFLICT DO UPDATE clause may cause both insert and update operations, so it will fire both kinds of triggers as needed. The transition relations supplied to triggers are specific to their event type; thus an INSERT trigger will see only the inserted rows, while an UPDATE trigger will see only the updated rows.

Row updates or deletions caused by foreign-key enforcement actions, such as ON UPDATE CASCADE or ON DELETE SET NULL, are treated as part of the SQL command that caused them (note that such actions are never deferred). Relevant triggers on the affected table will be fired, so that this provides another way in which an SQL command might fire triggers not directly matching its type. In simple cases, triggers that request transition relations will see all changes caused in their table by a single original SQL command as a single transition relation. However, there are cases in which the presence of an AFTER ROW trigger that requests transition relations will cause the foreign-key enforcement actions triggered by a single SQL command to be split into multiple steps, each with its own transition relation(s). In such cases, any statement-level triggers that are present will be fired once per creation of a transition relation set, ensuring that the triggers see each affected row in a transition relation once and only once.

Statement-level triggers on a view are fired only if the action on the view is handled by a row-level INSTEAD OF trigger. If the action is handled by an INSTEAD rule, then whatever statements are emitted by the rule are executed in place of the original statement naming the view, so that the triggers that will be fired are those on tables named in the replacement statements. Similarly, if the view is automatically updatable, then the action is handled by automatically rewriting the statement into an action on the view's base table, so that the base table's statement-level triggers are the ones that are fired.

Modifying a partitioned table or a table with inheritance children fires statement-level triggers attached to the explicitly named table, but not statement-level triggers for its partitions or child tables. In contrast, row-level triggers are fired on the rows in affected partitions or child tables, even if they are not explicitly named in the query. If a statement-level trigger has been defined with transition relations named by a REFERENCING clause, then before and after images of rows are visible from all affected partitions or child tables. In the case of inheritance children, the row images include only columns that are present in the table that the trigger is attached to.

Currently, row-level triggers with transition relations cannot be defined on partitions or inheritance child tables. Also, triggers on partitioned tables may not be INSTEAD OF.

Currently, the OR REPLACE option is not supported for constraint triggers.

Replacing an existing trigger within a transaction that has already performed updating actions on the trigger's table is not recommended. Trigger firing decisions, or portions of firing decisions, that have already been made will not be reconsidered, so the effects could be surprising.

There are a few built-in trigger functions that can be used to solve common problems without having to write your own trigger code; see ???.

Examples

Execute the function check_account_update whenever a row of the table accounts is about to be updated:

CREATE TRIGGER check_update
    BEFORE UPDATE ON accounts
    FOR EACH ROW
    EXECUTE FUNCTION check_account_update();

Modify that trigger definition to only execute the function if column balance is specified as a target in the UPDATE command:

CREATE OR REPLACE TRIGGER check_update
    BEFORE UPDATE OF balance ON accounts
    FOR EACH ROW
    EXECUTE FUNCTION check_account_update();

This form only executes the function if column balance has in fact changed value:

CREATE TRIGGER check_update
    BEFORE UPDATE ON accounts
    FOR EACH ROW
    WHEN (OLD.balance IS DISTINCT FROM NEW.balance)
    EXECUTE FUNCTION check_account_update();

Call a function to log updates of accounts, but only if something changed:

CREATE TRIGGER log_update
    AFTER UPDATE ON accounts
    FOR EACH ROW
    WHEN (OLD.* IS DISTINCT FROM NEW.*)
    EXECUTE FUNCTION log_account_update();

Execute the function view_insert_row for each row to insert rows into the tables underlying a view:

CREATE TRIGGER view_insert
    INSTEAD OF INSERT ON my_view
    FOR EACH ROW
    EXECUTE FUNCTION view_insert_row();

Execute the function check_transfer_balances_to_zero for each statement to confirm that the transfer rows offset to a net of zero:

CREATE TRIGGER transfer_insert
    AFTER INSERT ON transfer
    REFERENCING NEW TABLE AS inserted
    FOR EACH STATEMENT
    EXECUTE FUNCTION check_transfer_balances_to_zero();

Execute the function check_matching_pairs for each row to confirm that changes are made to matching pairs at the same time (by the same statement):

CREATE TRIGGER paired_items_update
    AFTER UPDATE ON paired_items
    REFERENCING NEW TABLE AS newtab OLD TABLE AS oldtab
    FOR EACH ROW
    EXECUTE FUNCTION check_matching_pairs();

??? contains a complete example of a trigger function written in C.

Compatibility

The CREATE TRIGGER statement in PostgreSQL implements a subset of the SQL standard. The following functionalities are currently missing:

SQL specifies that multiple triggers should be fired in time-of-creation order. PostgreSQL uses name order, which was judged to be more convenient.

SQL specifies that BEFORE DELETE triggers on cascaded deletes fire after the cascaded DELETE completes. The PostgreSQL behavior is for BEFORE DELETE to always fire before the delete action, even a cascading one. This is considered more consistent. There is also nonstandard behavior if BEFORE triggers modify rows or prevent updates during an update that is caused by a referential action. This can lead to constraint violations or stored data that does not honor the referential constraint.

The ability to specify multiple actions for a single trigger using OR is a PostgreSQL extension of the SQL standard.

The ability to fire triggers for TRUNCATE is a PostgreSQL extension of the SQL standard, as is the ability to define statement-level triggers on views.

CREATE CONSTRAINT TRIGGER is a PostgreSQL extension of the SQL standard. So is the OR REPLACE option.

See Also

create_tsconfig

CREATE TEXT SEARCH CONFIGURATION

CREATE TEXT SEARCH CONFIGURATION

7

SQL - Language Statements

CREATE TEXT SEARCH CONFIGURATION

define a new text search configuration

CREATE TEXT SEARCH CONFIGURATION

name

( PARSER =

parser_name

| COPY =

source_config

)

Description

CREATE TEXT SEARCH CONFIGURATION creates a new text search configuration. A text search configuration specifies a text search parser that can divide a string into tokens, plus dictionaries that can be used to determine which tokens are of interest for searching.

If only the parser is specified, then the new text search configuration initially has no mappings from token types to dictionaries, and therefore will ignore all words. Subsequent ALTER TEXT SEARCH CONFIGURATION commands must be used to create mappings to make the configuration useful. Alternatively, an existing text search configuration can be copied.

If a schema name is given then the text search configuration is created in the specified schema. Otherwise it is created in the current schema.

The user who defines a text search configuration becomes its owner.

Refer to ??? for further information.

Parameters

<name>
The name of the text search configuration to be created. The name can be schema-qualified.

<parser_name>
The name of the text search parser to use for this configuration.

<source_config>
The name of an existing text search configuration to copy.

Notes

The PARSER and COPY options are mutually exclusive, because when an existing configuration is copied, its parser selection is copied too.

Compatibility

There is no CREATE TEXT SEARCH CONFIGURATION statement in the SQL standard.

See Also

create_tsdictionary

CREATE TEXT SEARCH DICTIONARY

CREATE TEXT SEARCH DICTIONARY

7

SQL - Language Statements

CREATE TEXT SEARCH DICTIONARY

define a new text search dictionary

CREATE TEXT SEARCH DICTIONARY

name

( TEMPLATE =

template

[,

option

=

value

[, ... ]] )

Description

CREATE TEXT SEARCH DICTIONARY creates a new text search dictionary. A text search dictionary specifies a way of recognizing interesting or uninteresting words for searching. A dictionary depends on a text search template, which specifies the functions that actually perform the work. Typically the dictionary provides some options that control the detailed behavior of the template's functions.

If a schema name is given then the text search dictionary is created in the specified schema. Otherwise it is created in the current schema.

The user who defines a text search dictionary becomes its owner.

Refer to ??? for further information.

Parameters

<name>
The name of the text search dictionary to be created. The name can be schema-qualified.

<template>
The name of the text search template that will define the basic behavior of this dictionary.

<option>
The name of a template-specific option to be set for this dictionary.

<value>
The value to use for a template-specific option. If the value is not a simple identifier or number, it must be quoted (but you can always quote it, if you wish).

The options can appear in any order.

Examples

The following example command creates a Snowball-based dictionary with a nonstandard list of stop words.

CREATE TEXT SEARCH DICTIONARY my_russian (
    template = snowball,
    language = russian,
    stopwords = myrussian
);

Compatibility

There is no CREATE TEXT SEARCH DICTIONARY statement in the SQL standard.

See Also

create_tsparser

CREATE TEXT SEARCH PARSER

CREATE TEXT SEARCH PARSER

7

SQL - Language Statements

CREATE TEXT SEARCH PARSER

define a new text search parser

CREATE TEXT SEARCH PARSER

name

( START =

start_function

, GETTOKEN =

gettoken_function

, END =

end_function

, LEXTYPES =

lextypes_function

[, HEADLINE =

headline_function

] )

Description

CREATE TEXT SEARCH PARSER creates a new text search parser. A text search parser defines a method for splitting a text string into tokens and assigning types (categories) to the tokens. A parser is not particularly useful by itself, but must be bound into a text search configuration along with some text search dictionaries to be used for searching.

If a schema name is given then the text search parser is created in the specified schema. Otherwise it is created in the current schema.

You must be a superuser to use CREATE TEXT SEARCH PARSER. (This restriction is made because an erroneous text search parser definition could confuse or even crash the server.)

Refer to ??? for further information.

Parameters

<name>
The name of the text search parser to be created. The name can be schema-qualified.

<start_function>
The name of the start function for the parser.

<gettoken_function>
The name of the get-next-token function for the parser.

<end_function>
The name of the end function for the parser.

<lextypes_function>
The name of the lextypes function for the parser (a function that returns information about the set of token types it produces).

<headline_function>
The name of the headline function for the parser (a function that summarizes a set of tokens).

The function names can be schema-qualified if necessary. Argument types are not given, since the argument list for each type of function is predetermined. All except the headline function are required.

The arguments can appear in any order, not only the one shown above.

Compatibility

There is no CREATE TEXT SEARCH PARSER statement in the SQL standard.

See Also

create_tstemplate

CREATE TEXT SEARCH TEMPLATE

CREATE TEXT SEARCH TEMPLATE

7

SQL - Language Statements

CREATE TEXT SEARCH TEMPLATE

define a new text search template

CREATE TEXT SEARCH TEMPLATE

name

( [ INIT =

init_function

, ] LEXIZE =

lexize_function

)

Description

CREATE TEXT SEARCH TEMPLATE creates a new text search template. Text search templates define the functions that implement text search dictionaries. A template is not useful by itself, but must be instantiated as a dictionary to be used. The dictionary typically specifies parameters to be given to the template functions.

If a schema name is given then the text search template is created in the specified schema. Otherwise it is created in the current schema.

You must be a superuser to use CREATE TEXT SEARCH TEMPLATE. This restriction is made because an erroneous text search template definition could confuse or even crash the server. The reason for separating templates from dictionaries is that a template encapsulates the “unsafe” aspects of defining a dictionary. The parameters that can be set when defining a dictionary are safe for unprivileged users to set, and so creating a dictionary need not be a privileged operation.

Refer to ??? for further information.

Parameters

<name>
The name of the text search template to be created. The name can be schema-qualified.

<init_function>
The name of the init function for the template.

<lexize_function>
The name of the lexize function for the template.

The function names can be schema-qualified if necessary. Argument types are not given, since the argument list for each type of function is predetermined. The lexize function is required, but the init function is optional.

The arguments can appear in any order, not only the one shown above.

Compatibility

There is no CREATE TEXT SEARCH TEMPLATE statement in the SQL standard.

See Also

create_type

CREATE TYPE

CREATE TYPE

7

SQL - Language Statements

CREATE TYPE

define a new data type

CREATE TYPE

name

AS ( [

attribute_name

data_type

[ COLLATE

collation

] [, ... ] ] ) CREATE TYPE

name

AS ENUM ( [ '

label

' [, ... ] ] ) CREATE TYPE

name

AS RANGE ( SUBTYPE =

subtype

[ , SUBTYPE_OPCLASS =

subtype_operator_class

] [ , COLLATION =

collation

] [ , CANONICAL =

canonical_function

] [ , SUBTYPE_DIFF =

subtype_diff_function

] [ , MULTIRANGE_TYPE_NAME =

multirange_type_name

] ) CREATE TYPE

name

( INPUT =

input_function

, OUTPUT =

output_function

[ , RECEIVE =

receive_function

] [ , SEND =

send_function

] [ , TYPMOD_IN =

type_modifier_input_function

] [ , TYPMOD_OUT =

type_modifier_output_function

] [ , ANALYZE =

analyze_function

] [ , SUBSCRIPT =

subscript_function

] [ , INTERNALLENGTH = {

internallength

| VARIABLE } ] [ , PASSEDBYVALUE ] [ , ALIGNMENT =

alignment

] [ , STORAGE =

storage

] [ , LIKE =

like_type

] [ , CATEGORY =

category

] [ , PREFERRED =

preferred

] [ , DEFAULT =

default

] [ , ELEMENT =

element

] [ , DELIMITER =

delimiter

] [ , COLLATABLE =

collatable

] ) CREATE TYPE

name

Description

CREATE TYPE registers a new data type for use in the current database. The user who defines a type becomes its owner.

If a schema name is given then the type is created in the specified schema. Otherwise it is created in the current schema. The type name must be distinct from the name of any existing type or domain in the same schema. (Because tables have associated data types, the type name must also be distinct from the name of any existing table in the same schema.)

There are five forms of CREATE TYPE, as shown in the syntax synopsis above. They respectively create a composite type, an enum type, a range type, a base type, or a shell type. The first four of these are discussed in turn below. A shell type is simply a placeholder for a type to be defined later; it is created by issuing CREATE TYPE with no parameters except for the type name. Shell types are needed as forward references when creating range types and base types, as discussed in those sections.

Composite Types

The first form of CREATE TYPE creates a composite type. The composite type is specified by a list of attribute names and data types. An attribute's collation can be specified too, if its data type is collatable. A composite type is essentially the same as the row type of a table, but using CREATE TYPE avoids the need to create an actual table when all that is wanted is to define a type. A stand-alone composite type is useful, for example, as the argument or return type of a function.

To be able to create a composite type, you must have USAGE privilege on all attribute types.

Enumerated Types

The second form of CREATE TYPE creates an enumerated (enum) type, as described in ???. Enum types take a list of quoted labels, each of which must be less than NAMEDATALEN bytes long (64 bytes in a standard PostgreSQL build). (It is possible to create an enumerated type with zero labels, but such a type cannot be used to hold values before at least one label is added using ALTER TYPE.)

Range Types

The third form of CREATE TYPE creates a new range type, as described in ???.

The range type's <subtype> can be any type with an associated b-tree operator class (to determine the ordering of values for the range type). Normally the subtype's default b-tree operator class is used to determine ordering; to use a non-default operator class, specify its name with <subtype_opclass>. If the subtype is collatable, and you want to use a non-default collation in the range's ordering, specify the desired collation with the <collation> option.

The optional <canonical> function must take one argument of the range type being defined, and return a value of the same type. This is used to convert range values to a canonical form, when applicable. See ??? for more information. Creating a <canonical> function is a bit tricky, since it must be defined before the range type can be declared. To do this, you must first create a shell type, which is a placeholder type that has no properties except a name and an owner. This is done by issuing the command CREATE TYPE name, with no additional parameters. Then the function can be declared using the shell type as argument and result, and finally the range type can be declared using the same name. This automatically replaces the shell type entry with a valid range type.

The optional <subtype_diff> function must take two values of the <subtype> type as argument, and return a double precision value representing the difference between the two given values. While this is optional, providing it allows much greater efficiency of GiST indexes on columns of the range type. See ??? for more information.

The optional <multirange_type_name> parameter specifies the name of the corresponding multirange type. If not specified, this name is chosen automatically as follows. If the range type name contains the substring range, then the multirange type name is formed by replacement of the range substring with multirange in the range type name. Otherwise, the multirange type name is formed by appending a _multirange suffix to the range type name.

To be able to create a range type, you must have USAGE privilege on the subtype.

Base Types

The fourth form of CREATE TYPE creates a new base type (scalar type). To create a new base type, you must be a superuser. (This restriction is made because an erroneous type definition could confuse or even crash the server.)

The parameters can appear in any order, not only that illustrated above, and most are optional. You must register two or more functions (using CREATE FUNCTION) before defining the type. The support functions <input_function> and <output_function> are required, while the functions <receive_function>, <send_function>, <type_modifier_input_function>, <type_modifier_output_function>, <analyze_function>, and <subscript_function> are optional. Generally these functions have to be coded in C or another low-level language.

The <input_function> converts the type's external textual representation to the internal representation used by the operators and functions defined for the type. <output_function> performs the reverse transformation. The input function can be declared as taking one argument of type cstring, or as taking three arguments of types cstring, oid, integer. The first argument is the input text as a C string, the second argument is the type's own OID (except for array types, which instead receive their element type's OID), and the third is the typmod of the destination column, if known (-1 will be passed if not). The input function must return a value of the data type itself. Usually, an input function should be declared STRICT; if it is not, it will be called with a NULL first parameter when reading a NULL input value. The function must still return NULL in this case, unless it raises an error. (This case is mainly meant to support domain input functions, which might need to reject NULL inputs.) The output function must be declared as taking one argument of the new data type. The output function must return type cstring. Output functions are not invoked for NULL values.

The optional <receive_function> converts the type's external binary representation to the internal representation. If this function is not supplied, the type cannot participate in binary input. The binary representation should be chosen to be cheap to convert to internal form, while being reasonably portable. (For example, the standard integer data types use network byte order as the external binary representation, while the internal representation is in the machine's native byte order.) The receive function should perform adequate checking to ensure that the value is valid. The receive function can be declared as taking one argument of type internal, or as taking three arguments of types internal, oid, integer. The first argument is a pointer to a StringInfo buffer holding the received byte string; the optional arguments are the same as for the text input function. The receive function must return a value of the data type itself. Usually, a receive function should be declared STRICT; if it is not, it will be called with a NULL first parameter when reading a NULL input value. The function must still return NULL in this case, unless it raises an error. (This case is mainly meant to support domain receive functions, which might need to reject NULL inputs.) Similarly, the optional <send_function> converts from the internal representation to the external binary representation. If this function is not supplied, the type cannot participate in binary output. The send function must be declared as taking one argument of the new data type. The send function must return type bytea. Send functions are not invoked for NULL values.

You should at this point be wondering how the input and output functions can be declared to have results or arguments of the new type, when they have to be created before the new type can be created. The answer is that the type should first be defined as a shell type, which is a placeholder type that has no properties except a name and an owner. This is done by issuing the command CREATE TYPE name, with no additional parameters. Then the C I/O functions can be defined referencing the shell type. Finally, CREATE TYPE with a full definition replaces the shell entry with a complete, valid type definition, after which the new type can be used normally.

The optional <type_modifier_input_function> and <type_modifier_output_function> are needed if the type supports modifiers, that is optional constraints attached to a type declaration, such as char(5) or numeric(30,2). PostgreSQL allows user-defined types to take one or more simple constants or identifiers as modifiers. However, this information must be capable of being packed into a single non-negative integer value for storage in the system catalogs. The <type_modifier_input_function> is passed the declared modifier(s) in the form of a cstring array. It must check the values for validity (throwing an error if they are wrong), and if they are correct, return a single non-negative integer value that will be stored as the column “typmod”. Type modifiers will be rejected if the type does not have a <type_modifier_input_function>. The <type_modifier_output_function> converts the internal integer typmod value back to the correct form for user display. It must return a cstring value that is the exact string to append to the type name; for example numeric's function might return (30,2). It is allowed to omit the <type_modifier_output_function>, in which case the default display format is just the stored typmod integer value enclosed in parentheses.

The optional <analyze_function> performs type-specific statistics collection for columns of the data type. By default, ANALYZE will attempt to gather statistics using the type's “equals” and “less-than” operators, if there is a default b-tree operator class for the type. For non-scalar types this behavior is likely to be unsuitable, so it can be overridden by specifying a custom analysis function. The analysis function must be declared to take a single argument of type internal, and return a boolean result. The detailed API for analysis functions appears in src/include/commands/vacuum.h.

The optional <subscript_function> allows the data type to be subscripted in SQL commands. Specifying this function does not cause the type to be considered a “true” array type; for example, it will not be a candidate for the result type of ARRAY[] constructs. But if subscripting a value of the type is a natural notation for extracting data from it, then a <subscript_function> can be written to define what that means. The subscript function must be declared to take a single argument of type internal, and return an internal result, which is a pointer to a struct of methods (functions) that implement subscripting. The detailed API for subscript functions appears in src/include/nodes/subscripting.h. It may also be useful to read the array implementation in src/backend/utils/adt/arraysubs.c, or the simpler code in contrib/hstore/hstore_subs.c. Additional information appears in Array Types below.

While the details of the new type's internal representation are only known to the I/O functions and other functions you create to work with the type, there are several properties of the internal representation that must be declared to PostgreSQL. Foremost of these is <internallength>. Base data types can be fixed-length, in which case <internallength> is a positive integer, or variable-length, indicated by setting <internallength> to VARIABLE. (Internally, this is represented by setting typlen to -1.) The internal representation of all variable-length types must start with a 4-byte integer giving the total length of this value of the type. (Note that the length field is often encoded, as described in ???; it's unwise to access it directly.)

The optional flag PASSEDBYVALUE indicates that values of this data type are passed by value, rather than by reference. Types passed by value must be fixed-length, and their internal representation cannot be larger than the size of the Datum type (4 bytes on some machines, 8 bytes on others).

The <alignment> parameter specifies the storage alignment required for the data type. The allowed values equate to alignment on 1, 2, 4, or 8 byte boundaries. Note that variable-length types must have an alignment of at least 4, since they necessarily contain an int4 as their first component.

The <storage> parameter allows selection of storage strategies for variable-length data types. (Only plain is allowed for fixed-length types.) plain specifies that data of the type will always be stored in-line and not compressed. extended specifies that the system will first try to compress a long data value, and will move the value out of the main table row if it's still too long. external allows the value to be moved out of the main table, but the system will not try to compress it. main allows compression, but discourages moving the value out of the main table. (Data items with this storage strategy might still be moved out of the main table if there is no other way to make a row fit, but they will be kept in the main table preferentially over extended and external items.)

All <storage> values other than plain imply that the functions of the data type can handle values that have been toasted, as described in ??? and ???. The specific other value given merely determines the default TOAST storage strategy for columns of a toastable data type; users can pick other strategies for individual columns using ALTER TABLE SET STORAGE.

The <like_type> parameter provides an alternative method for specifying the basic representation properties of a data type: copy them from some existing type. The values of <internallength>, <passedbyvalue>, <alignment>, and <storage> are copied from the named type. (It is possible, though usually undesirable, to override some of these values by specifying them along with the LIKE clause.) Specifying representation this way is especially useful when the low-level implementation of the new type “piggybacks” on an existing type in some fashion.

The <category> and <preferred> parameters can be used to help control which implicit cast will be applied in ambiguous situations. Each data type belongs to a category named by a single ASCII character, and each type is either “preferred” or not within its category. The parser will prefer casting to preferred types (but only from other types within the same category) when this rule is helpful in resolving overloaded functions or operators. For more details see ???. For types that have no implicit casts to or from any other types, it is sufficient to leave these settings at the defaults. However, for a group of related types that have implicit casts, it is often helpful to mark them all as belonging to a category and select one or two of the “most general” types as being preferred within the category. The <category> parameter is especially useful when adding a user-defined type to an existing built-in category, such as the numeric or string types. However, it is also possible to create new entirely-user-defined type categories. Select any ASCII character other than an upper-case letter to name such a category.

A default value can be specified, in case a user wants columns of the data type to default to something other than the null value. Specify the default with the DEFAULT key word. (Such a default can be overridden by an explicit DEFAULT clause attached to a particular column.)

To indicate that a type is a fixed-length array type, specify the type of the array elements using the ELEMENT key word. For example, to define an array of 4-byte integers (int4), specify ELEMENT = int4. For more details, see Array Types below.

To indicate the delimiter to be used between values in the external representation of arrays of this type, <delimiter> can be set to a specific character. The default delimiter is the comma (,). Note that the delimiter is associated with the array element type, not the array type itself.

If the optional Boolean parameter <collatable> is true, column definitions and expressions of the type may carry collation information through use of the COLLATE clause. It is up to the implementations of the functions operating on the type to actually make use of the collation information; this does not happen automatically merely by marking the type collatable.

Array Types

Whenever a user-defined type is created, PostgreSQL automatically creates an associated array type, whose name consists of the element type's name prepended with an underscore, and truncated if necessary to keep it less than NAMEDATALEN bytes long. (If the name so generated collides with an existing type name, the process is repeated until a non-colliding name is found.) This implicitly-created array type is variable length and uses the built-in input and output functions array_in and array_out. Furthermore, this type is what the system uses for constructs such as ARRAY[] over the user-defined type. The array type tracks any changes in its element type's owner or schema, and is dropped if the element type is.

You might reasonably ask why there is an ELEMENT option, if the system makes the correct array type automatically. The main case where it's useful to use ELEMENT is when you are making a fixed-length type that happens to be internally an array of a number of identical things, and you want to allow these things to be accessed directly by subscripting, in addition to whatever operations you plan to provide for the type as a whole. For example, type point is represented as just two floating-point numbers, which can be accessed using point[0] and point[1]. Note that this facility only works for fixed-length types whose internal form is exactly a sequence of identical fixed-length fields. For historical reasons (i.e., this is clearly wrong but it's far too late to change it), subscripting of fixed-length array types starts from zero, rather than from one as for variable-length arrays.

Specifying the SUBSCRIPT option allows a data type to be subscripted, even though the system does not otherwise regard it as an array type. The behavior just described for fixed-length arrays is actually implemented by the SUBSCRIPT handler function raw_array_subscript_handler, which is used automatically if you specify ELEMENT for a fixed-length type without also writing SUBSCRIPT.

When specifying a custom SUBSCRIPT function, it is not necessary to specify ELEMENT unless the SUBSCRIPT handler function needs to consult typelem to find out what to return. Be aware that specifying ELEMENT causes the system to assume that the new type contains, or is somehow physically dependent on, the element type; thus for example changing properties of the element type won't be allowed if there are any columns of the dependent type.

Parameters

<name>
The name (optionally schema-qualified) of a type to be created.

<attribute_name>
The name of an attribute (column) for the composite type.

<data_type>
The name of an existing data type to become a column of the composite type.

<collation>
The name of an existing collation to be associated with a column of a composite type, or with a range type.

<label>
A string literal representing the textual label associated with one value of an enum type.

<subtype>
The name of the element type that the range type will represent ranges of.

<subtype_operator_class>
The name of a b-tree operator class for the subtype.

<canonical_function>
The name of the canonicalization function for the range type.

<subtype_diff_function>
The name of a difference function for the subtype.

<multirange_type_name>
The name of the corresponding multirange type.

<input_function>
The name of a function that converts data from the type's external textual form to its internal form.

<output_function>
The name of a function that converts data from the type's internal form to its external textual form.

<receive_function>
The name of a function that converts data from the type's external binary form to its internal form.

<send_function>
The name of a function that converts data from the type's internal form to its external binary form.

<type_modifier_input_function>
The name of a function that converts an array of modifier(s) for the type into internal form.

<type_modifier_output_function>
The name of a function that converts the internal form of the type's modifier(s) to external textual form.

<analyze_function>
The name of a function that performs statistical analysis for the data type.

<subscript_function>
The name of a function that defines what subscripting a value of the data type does.

<internallength>
A numeric constant that specifies the length in bytes of the new type's internal representation. The default assumption is that it is variable-length.

<alignment>
The storage alignment requirement of the data type. If specified, it must be char, int2, int4, or double; the default is int4.

<storage>
The storage strategy for the data type. If specified, must be plain, external, extended, or main; the default is plain.

<like_type>
The name of an existing data type that the new type will have the same representation as. The values of <internallength>, <passedbyvalue>, <alignment>, and <storage> are copied from that type, unless overridden by explicit specification elsewhere in this CREATE TYPE command.

<category>
The category code (a single ASCII character) for this type. The default is 'U' for “user-defined type”. Other standard category codes can be found in ???. You may also choose other ASCII characters in order to create custom categories.

<preferred>
True if this type is a preferred type within its type category, else false. The default is false. Be very careful about creating a new preferred type within an existing type category, as this could cause surprising changes in behavior.

<default>
The default value for the data type. If this is omitted, the default is null.

<element>
The type being created is an array; this specifies the type of the array elements.

<delimiter>
The delimiter character to be used between values in arrays made of this type.

<collatable>
True if this type's operations can use collation information. The default is false.

Notes

Because there are no restrictions on use of a data type once it's been created, creating a base type or range type is tantamount to granting public execute permission on the functions mentioned in the type definition. This is usually not an issue for the sorts of functions that are useful in a type definition. But you might want to think twice before designing a type in a way that would require “secret” information to be used while converting it to or from external form.

Before PostgreSQL version 8.3, the name of a generated array type was always exactly the element type's name with one underscore character (_) prepended. (Type names were therefore restricted in length to one fewer character than other names.) While this is still usually the case, the array type name may vary from this in case of maximum-length names or collisions with user type names that begin with underscore. Writing code that depends on this convention is therefore deprecated. Instead, use pg_type.typarray to locate the array type associated with a given type.

It may be advisable to avoid using type and table names that begin with underscore. While the server will change generated array type names to avoid collisions with user-given names, there is still risk of confusion, particularly with old client software that may assume that type names beginning with underscores always represent arrays.

Before PostgreSQL version 8.2, the shell-type creation syntax CREATE TYPE name did not exist. The way to create a new base type was to create its input function first. In this approach, PostgreSQL will first see the name of the new data type as the return type of the input function. The shell type is implicitly created in this situation, and then it can be referenced in the definitions of the remaining I/O functions. This approach still works, but is deprecated and might be disallowed in some future release. Also, to avoid accidentally cluttering the catalogs with shell types as a result of simple typos in function definitions, a shell type will only be made this way when the input function is written in C.

In PostgreSQL version 16 and later, it is desirable for base types' input functions to return “soft” errors using the new errsave()/ereturn() mechanism, rather than throwing ereport() exceptions as in previous versions. See src/backend/utils/fmgr/README for more information.

Examples

This example creates a composite type and uses it in a function definition:

CREATE TYPE compfoo AS (f1 int, f2 text);

CREATE FUNCTION getfoo() RETURNS SETOF compfoo AS $$
    SELECT fooid, fooname FROM foo
$$ LANGUAGE SQL;

This example creates an enumerated type and uses it in a table definition:

CREATE TYPE bug_status AS ENUM ('new', 'open', 'closed');

CREATE TABLE bug (
    id serial,
    description text,
    status bug_status
);

This example creates a range type:

CREATE TYPE float8_range AS RANGE (subtype = float8, subtype_diff = float8mi);

This example creates the base data type box and then uses the type in a table definition:

CREATE TYPE box;

CREATE FUNCTION my_box_in_function(cstring) RETURNS box AS ... ;
CREATE FUNCTION my_box_out_function(box) RETURNS cstring AS ... ;

CREATE TYPE box (
    INTERNALLENGTH = 16,
    INPUT = my_box_in_function,
    OUTPUT = my_box_out_function
);

CREATE TABLE myboxes (
    id integer,
    description box
);

If the internal structure of box were an array of four float4 elements, we might instead use:

CREATE TYPE box (
    INTERNALLENGTH = 16,
    INPUT = my_box_in_function,
    OUTPUT = my_box_out_function,
    ELEMENT = float4
);

which would allow a box value's component numbers to be accessed by subscripting. Otherwise the type behaves the same as before.

This example creates a large object type and uses it in a table definition:

CREATE TYPE bigobj (
    INPUT = lo_filein, OUTPUT = lo_fileout,
    INTERNALLENGTH = VARIABLE
);
CREATE TABLE big_objs (
    id integer,
    obj bigobj
);

More examples, including suitable input and output functions, are in ???.

Compatibility

The first form of the CREATE TYPE command, which creates a composite type, conforms to the SQL standard. The other forms are PostgreSQL extensions. The CREATE TYPE statement in the SQL standard also defines other forms that are not implemented in PostgreSQL.

The ability to create a composite type with zero attributes is a PostgreSQL-specific deviation from the standard (analogous to the same case in CREATE TABLE).

See Also

create_user

CREATE USER

CREATE USER

7

SQL - Language Statements

CREATE USER

define a new database role

CREATE USER

name

[ [ WITH ]

option

[ ... ] ]

where

option

can be:

SUPERUSER | NOSUPERUSER | CREATEDB | NOCREATEDB | CREATEROLE | NOCREATEROLE | INHERIT | NOINHERIT | LOGIN | NOLOGIN | REPLICATION | NOREPLICATION | BYPASSRLS | NOBYPASSRLS | CONNECTION LIMIT

connlimit

| [ ENCRYPTED ] PASSWORD '

password

' | PASSWORD NULL | VALID UNTIL '

timestamp

' | IN ROLE

role_name

[, ...] | ROLE

role_name

[, ...] | ADMIN

role_name

[, ...] | SYSID

uid

Description

CREATE USER is now an alias for CREATE ROLE. The only difference is that when the command is spelled CREATE USER, LOGIN is assumed by default, whereas NOLOGIN is assumed when the command is spelled CREATE ROLE.

Compatibility

The CREATE USER statement is a PostgreSQL extension. The SQL standard leaves the definition of users to the implementation.

See Also

create_user_mapping

CREATE USER MAPPING

CREATE USER MAPPING

7

SQL - Language Statements

CREATE USER MAPPING

define a new mapping of a user to a foreign server

CREATE USER MAPPING [ IF NOT EXISTS ] FOR {

user_name

| USER | CURRENT_ROLE | CURRENT_USER | PUBLIC } SERVER

server_name

[ OPTIONS (

option

'

value

' [ , ... ] ) ]

Description

CREATE USER MAPPING defines a mapping of a user to a foreign server. A user mapping typically encapsulates connection information that a foreign-data wrapper uses together with the information encapsulated by a foreign server to access an external data resource.

The owner of a foreign server can create user mappings for that server for any user. Also, a user can create a user mapping for their own user name if USAGE privilege on the server has been granted to the user.

Parameters

IF NOT EXISTS
Do not throw an error if a mapping of the given user to the given foreign server already exists. A notice is issued in this case. Note that there is no guarantee that the existing user mapping is anything like the one that would have been created.

<user_name>
The name of an existing user that is mapped to foreign server. CURRENT_ROLE, CURRENT_USER, and USER match the name of the current user. When PUBLIC is specified, a so-called public mapping is created that is used when no user-specific mapping is applicable.

<server_name>
The name of an existing server for which the user mapping is to be created.

OPTIONS ( option 'value' [, ... ] )
This clause specifies the options of the user mapping. The options typically define the actual user name and password of the mapping. Option names must be unique. The allowed option names and values are specific to the server's foreign-data wrapper.

Examples

Create a user mapping for user bob, server foo:

CREATE USER MAPPING FOR bob SERVER foo OPTIONS (user 'bob', password 'secret');

Compatibility

CREATE USER MAPPING conforms to ISO/IEC 9075-9 (SQL/MED).

See Also

create_view

CREATE VIEW

CREATE VIEW

7

SQL - Language Statements

CREATE VIEW

define a new view

CREATE [ OR REPLACE ] [ TEMP | TEMPORARY ] [ RECURSIVE ] VIEW

name

[ (

column_name

[, ...] ) ] [ WITH (

view_option_name

[=

view_option_value

] [, ... ] ) ] AS

query

[ WITH [ CASCADED | LOCAL ] CHECK OPTION ]

Description

CREATE VIEW defines a view of a query. The view is not physically materialized. Instead, the query is run every time the view is referenced in a query.

CREATE OR REPLACE VIEW is similar, but if a view of the same name already exists, it is replaced. The new query must generate the same columns that were generated by the existing view query (that is, the same column names in the same order and with the same data types), but it may add additional columns to the end of the list. The calculations giving rise to the output columns may be completely different.

If a schema name is given (for example, CREATE VIEW myschema.myview ...) then the view is created in the specified schema. Otherwise it is created in the current schema. Temporary views exist in a special schema, so a schema name cannot be given when creating a temporary view. The name of the view must be distinct from the name of any other relation (table, sequence, index, view, materialized view, or foreign table) in the same schema.

Parameters

TEMPORARY or TEMP
If specified, the view is created as a temporary view. Temporary views are automatically dropped at the end of the current session. Existing permanent relations with the same name are not visible to the current session while the temporary view exists, unless they are referenced with schema-qualified names.

If any of the tables referenced by the view are temporary, the view is created as a temporary view (whether TEMPORARY is specified or not).

RECURSIVE
Creates a recursive view. The syntax CREATE RECURSIVE VIEW [ <schema> . ] <view_name> (<column_names>) AS SELECT <...>; is equivalent to CREATE VIEW [ <schema> . ] <view_name> AS WITH RECURSIVE <view_name> (<column_names>) AS (SELECT <...>) SELECT <column_names> FROM <view_name>; A view column name list must be specified for a recursive view.

<name>
The name (optionally schema-qualified) of a view to be created.

<column_name>
An optional list of names to be used for columns of the view. If not given, the column names are deduced from the query.

WITH ( view_option_name [= view_option_value] [, ... ] )
This clause specifies optional parameters for a view; the following parameters are supported:

check_option (enum)
This parameter may be either local or cascaded, and is equivalent to specifying WITH [ CASCADED | LOCAL ] CHECK OPTION (see below).

security_barrier (boolean)
This should be used if the view is intended to provide row-level security. See ??? for full details.

security_invoker (boolean)
This option causes the underlying base relations to be checked against the privileges of the user of the view rather than the view owner. See the notes below for full details.

All of the above options can be changed on existing views using ALTER VIEW.

<query>
A SELECT or VALUES command which will provide the columns and rows of the view.

WITH [ CASCADED | LOCAL ] CHECK OPTION
This option controls the behavior of automatically updatable views. When this option is specified, INSERT, UPDATE, and MERGE commands on the view will be checked to ensure that new rows satisfy the view-defining condition (that is, the new rows are checked to ensure that they are visible through the view). If they are not, the update will be rejected. If the CHECK OPTION is not specified, INSERT, UPDATE, and MERGE commands on the view are allowed to create rows that are not visible through the view. The following check options are supported:

LOCAL
New rows are only checked against the conditions defined directly in the view itself. Any conditions defined on underlying base views are not checked (unless they also specify the CHECK OPTION).

CASCADED
New rows are checked against the conditions of the view and all underlying base views. If the CHECK OPTION is specified, and neither LOCAL nor CASCADED is specified, then CASCADED is assumed.

The CHECK OPTION may not be used with RECURSIVE views.

Note that the CHECK OPTION is only supported on views that are automatically updatable, and do not have INSTEAD OF triggers or INSTEAD rules. If an automatically updatable view is defined on top of a base view that has INSTEAD OF triggers, then the LOCAL CHECK OPTION may be used to check the conditions on the automatically updatable view, but the conditions on the base view with INSTEAD OF triggers will not be checked (a cascaded check option will not cascade down to a trigger-updatable view, and any check options defined directly on a trigger-updatable view will be ignored). If the view or any of its base relations has an INSTEAD rule that causes the INSERT or UPDATE command to be rewritten, then all check options will be ignored in the rewritten query, including any checks from automatically updatable views defined on top of the relation with the INSTEAD rule. MERGE is not supported if the view or any of its base relations have rules.

Notes

Use the DROP VIEW statement to drop views.

Be careful that the names and types of the view's columns will be assigned the way you want. For example:

CREATE VIEW vista AS SELECT 'Hello World';

is bad form because the column name defaults to ?column?; also, the column data type defaults to text, which might not be what you wanted. Better style for a string literal in a view's result is something like:

CREATE VIEW vista AS SELECT text 'Hello World' AS hello;

By default, access to the underlying base relations referenced in the view is determined by the permissions of the view owner. In some cases, this can be used to provide secure but restricted access to the underlying tables. However, not all views are secure against tampering; see ??? for details.

If the view has the security_invoker property set to true, access to the underlying base relations is determined by the permissions of the user executing the query, rather than the view owner. Thus, the user of a security invoker view must have the relevant permissions on the view and its underlying base relations.

If any of the underlying base relations is a security invoker view, it will be treated as if it had been accessed directly from the original query. Thus, a security invoker view will always check its underlying base relations using the permissions of the current user, even if it is accessed from a view without the security_invoker property.

If any of the underlying base relations has row-level security enabled, then by default, the row-level security policies of the view owner are applied, and access to any additional relations referred to by those policies is determined by the permissions of the view owner. However, if the view has security_invoker set to true, then the policies and permissions of the invoking user are used instead, as if the base relations had been referenced directly from the query using the view.

Functions called in the view are treated the same as if they had been called directly from the query using the view. Therefore, the user of a view must have permissions to call all functions used by the view. Functions in the view are executed with the privileges of the user executing the query or the function owner, depending on whether the functions are defined as SECURITY INVOKER or SECURITY DEFINER. Thus, for example, calling CURRENT_USER directly in a view will always return the invoking user, not the view owner. This is not affected by the view's security_invoker setting, and so a view with security_invoker set to false is not equivalent to a SECURITY DEFINER function and those concepts should not be confused.

The user creating or replacing a view must have USAGE privileges on any schemas referred to in the view query, in order to look up the referenced objects in those schemas. Note, however, that this lookup only happens when the view is created or replaced. Therefore, the user of the view only requires the USAGE privilege on the schema containing the view, not on the schemas referred to in the view query, even for a security invoker view.

When CREATE OR REPLACE VIEW is used on an existing view, only the view's defining SELECT rule, plus any WITH ( ... ) parameters and its CHECK OPTION are changed. Other view properties, including ownership, permissions, and non-SELECT rules, remain unchanged. You must own the view to replace it (this includes being a member of the owning role).

Updatable Views

updatable views

Simple views are automatically updatable: the system will allow INSERT, UPDATE, DELETE, and MERGE statements to be used on the view in the same way as on a regular table. A view is automatically updatable if it satisfies all of the following conditions:

An automatically updatable view may contain a mix of updatable and non-updatable columns. A column is updatable if it is a simple reference to an updatable column of the underlying base relation; otherwise the column is read-only, and an error will be raised if an INSERT, UPDATE, or MERGE statement attempts to assign a value to it.

If the view is automatically updatable the system will convert any INSERT, UPDATE, DELETE, or MERGE statement on the view into the corresponding statement on the underlying base relation. INSERT statements that have an ON CONFLICT clause are fully supported.

If an automatically updatable view contains a WHERE condition, the condition restricts which rows of the base relation are available to be modified by UPDATE, DELETE, and MERGE statements on the view. However, an UPDATE or MERGE is allowed to change a row so that it no longer satisfies the WHERE condition, and thus is no longer visible through the view. Similarly, an INSERT or MERGE command can potentially insert base-relation rows that do not satisfy the WHERE condition and thus are not visible through the view (ON CONFLICT DO SELECT/UPDATE may similarly affect an existing row not visible through the view). The CHECK OPTION may be used to prevent INSERT, UPDATE, and MERGE commands from creating such rows that are not visible through the view.

If an automatically updatable view is marked with the security_barrier property then all the view's WHERE conditions (and any conditions using operators which are marked as LEAKPROOF) will always be evaluated before any conditions that a user of the view has added. See ??? for full details. Note that, due to this, rows which are not ultimately returned (because they do not pass the user's WHERE conditions) may still end up being locked. EXPLAIN can be used to see which conditions are applied at the relation level (and therefore do not lock rows) and which are not.

A more complex view that does not satisfy all these conditions is read-only by default: the system will not allow an INSERT, UPDATE, DELETE, or MERGE on the view. You can get the effect of an updatable view by creating INSTEAD OF triggers on the view, which must convert attempted inserts, etc. on the view into appropriate actions on other tables. For more information see ???. Another possibility is to create rules (see ???), but in practice triggers are easier to understand and use correctly. Also note that MERGE is not supported on relations with rules.

Note that the user performing the insert, update or delete on the view must have the corresponding insert, update or delete privilege on the view. In addition, by default, the view's owner must have the relevant privileges on the underlying base relations, whereas the user performing the update does not need any permissions on the underlying base relations (see ???). However, if the view has security_invoker set to true, the user performing the update, rather than the view owner, must have the relevant privileges on the underlying base relations.

Examples

Create a view consisting of all comedy films:

CREATE VIEW comedies AS
    SELECT *
    FROM films
    WHERE kind = 'Comedy';

This will create a view containing the columns that are in the film table at the time of view creation. Though * was used to create the view, columns added later to the table will not be part of the view.

Create a view with LOCAL CHECK OPTION:

CREATE VIEW universal_comedies AS
    SELECT *
    FROM comedies
    WHERE classification = 'U'
    WITH LOCAL CHECK OPTION;

This will create a view based on the comedies view, showing only films with kind = 'Comedy' and classification = 'U'. Any attempt to INSERT or UPDATE a row in the view will be rejected if the new row doesn't have classification = 'U', but the film kind will not be checked.

Create a view with CASCADED CHECK OPTION:

CREATE VIEW pg_comedies AS
    SELECT *
    FROM comedies
    WHERE classification = 'PG'
    WITH CASCADED CHECK OPTION;

This will create a view that checks both the kind and classification of new rows.

Create a view with a mix of updatable and non-updatable columns:

CREATE VIEW comedies AS
    SELECT f.*,
           country_code_to_name(f.country_code) AS country,
           (SELECT avg(r.rating)
            FROM user_ratings r
            WHERE r.film_id = f.id) AS avg_rating
    FROM films f
    WHERE f.kind = 'Comedy';

This view will support INSERT, UPDATE and DELETE. All the columns from the films table will be updatable, whereas the computed columns country and avg_rating will be read-only.

Create a recursive view consisting of the numbers from 1 to 100:

CREATE RECURSIVE VIEW public.nums_1_100 (n) AS
    VALUES (1)
UNION ALL
    SELECT n+1 FROM nums_1_100 WHERE n < 100;

Notice that although the recursive view's name is schema-qualified in this CREATE, its internal self-reference is not schema-qualified. This is because the implicitly-created CTE's name cannot be schema-qualified.

Compatibility

CREATE OR REPLACE VIEW is a PostgreSQL language extension. So is the concept of a temporary view. The WITH ( ... ) clause is an extension as well, as are security barrier views and security invoker views.

See Also

createdb

createdb

createdb

1

Application

createdb

create a new

PostgreSQL

database

createdb

connection-option

option

dbname

description

Description

createdb creates a new PostgreSQL database.

Normally, the database user who executes this command becomes the owner of the new database. However, a different owner can be specified via the -O option, if the executing user has appropriate privileges.

createdb is a wrapper around the SQL command CREATE DATABASE. There is no effective difference between creating databases via this utility and via other methods for accessing the server.

Options

createdb accepts the following command-line arguments:

<dbname>
Specifies the name of the database to be created. The name must be unique among all PostgreSQL databases in this cluster. The default is to create a database with the same name as the current system user.

<description>
Specifies a comment to be associated with the newly created database.

-D tablespace; --tablespace=tablespace
Specifies the default tablespace for the database. (This name is processed as a double-quoted identifier.)

-e; --echo
Echo the commands that createdb generates and sends to the server.

-E encoding; --encoding=encoding
Specifies the character encoding scheme to be used in this database. The character sets supported by the PostgreSQL server are described in ???.

-l locale; --locale=locale
Specifies the locale to be used in this database. This is equivalent to specifying --lc-collate, --lc-ctype, and --icu-locale to the same value. Some locales are only valid for ICU and must be set with --icu-locale.

--lc-collate=locale
Specifies the LC_COLLATE setting to be used in this database.

--lc-ctype=locale
Specifies the LC_CTYPE setting to be used in this database.

--builtin-locale=locale
Specifies the locale name when the builtin provider is used. Locale support is described in ???.

--icu-locale=locale
Specifies the ICU locale ID to be used in this database, if the ICU locale provider is selected.

--icu-rules=rules
Specifies additional collation rules to customize the behavior of the default collation of this database. This is supported for ICU only.

--locale-provider={builtin|libc|icu}
Specifies the locale provider for the database's default collation.

-O owner; --owner=owner
Specifies the database user who will own the new database. (This name is processed as a double-quoted identifier.)

-S strategy; --strategy=strategy
Specifies the database creation strategy. See ??? for more details.

-T template; --template=template
Specifies the template database from which to build this database. (This name is processed as a double-quoted identifier.)

-V; --version
Print the createdb version and exit.

-?; --help
Show help about createdb command line arguments, and exit.

The options -D, -l, -E, -O, and -T correspond to options of the underlying SQL command CREATE DATABASE; see there for more information about them.

createdb also accepts the following command-line arguments for connection parameters:

-h host; --host=host
Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for the Unix domain socket.

-p port; --port=port
Specifies the TCP port or the local Unix domain socket file extension on which the server is listening for connections.

-U username; --username=username
User name to connect as.

-w; --no-password
Never issue a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password.

-W; --password
Force createdb to prompt for a password before connecting to a database.

This option is never essential, since createdb will automatically prompt for a password if the server demands password authentication. However, createdb will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt.

--maintenance-db=dbname
Specifies the name of the database to connect to when creating the new database. If not specified, the postgres database will be used; if that does not exist (or if it is the name of the new database being created), template1 will be used. This can be a connection string. If so, connection string parameters will override any conflicting command line options.

Environment

PGDATABASE
If set, the name of the database to create, unless overridden on the command line.

PGHOST; PGPORT; PGUSER
Default connection parameters. PGUSER also determines the name of the database to create, if it is not specified on the command line or by PGDATABASE.

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

This utility, like most other PostgreSQL utilities, also uses the environment variables supported by libpq (see ???).

Diagnostics

In case of difficulty, see ??? and ??? for discussions of potential problems and error messages. The database server must be running at the targeted host. Also, any default connection settings and environment variables used by the libpq front-end library will apply.

Examples

To create the database demo using the default database server:

$ createdb demo

To create the database demo using the server on host eden, port 5000, using the template0 template database, here is the command-line command and the underlying SQL command:

$ createdb -p 5000 -h eden -T template0 -e demo
CREATE DATABASE demo TEMPLATE template0;

See Also

createuser

createuser

createuser

1

Application

createuser

define a new

PostgreSQL

user account

createuser

connection-option

option

username

Description

createuser creates a new PostgreSQL user (or more precisely, a role). Only superusers and users with CREATEROLE privilege can create new users, so createuser must be invoked by someone who can connect as a superuser or a user with CREATEROLE privilege.

If you wish to create a role with the SUPERUSER, REPLICATION, or BYPASSRLS privilege, you must connect as a superuser, not merely with CREATEROLE privilege. Being a superuser implies the ability to bypass all access permission checks within the database, so superuser access should not be granted lightly. CREATEROLE also conveys very extensive privileges.

createuser is a wrapper around the SQL command CREATE ROLE. There is no effective difference between creating users via this utility and via other methods for accessing the server.

Options

createuser accepts the following command-line arguments:

<username>
Specifies the name of the PostgreSQL user to be created. This name must be different from all existing roles in this PostgreSQL installation.

-a role; --with-admin=role
Specifies an existing role that will be automatically added as a member of the new role with admin option, giving it the right to grant membership in the new role to others. Multiple existing roles can be specified by writing multiple -a switches.

-c number; --connection-limit=number
Set a maximum number of connections for the new user. The default is to set no limit.

-d; --createdb
The new user will be allowed to create databases.

-D; --no-createdb
The new user will not be allowed to create databases. This is the default.

-e; --echo
Echo the commands that createuser generates and sends to the server.

-E; --encrypted
This option is obsolete but still accepted for backward compatibility.

-g role; --member-of=role; --role=role (deprecated)
Specifies the new role should be automatically added as a member of the specified existing role. Multiple existing roles can be specified by writing multiple -g switches.

-i; --inherit
The new role will automatically inherit privileges of roles it is a member of. This is the default.

-I; --no-inherit
The new role will not automatically inherit privileges of roles it is a member of.

--interactive
Prompt for the user name if none is specified on the command line, and also prompt for whichever of the options -d/-D, -r/-R, -s/-S is not specified on the command line. (This was the default behavior up to PostgreSQL 9.1.)

-l; --login
The new user will be allowed to log in (that is, the user name can be used as the initial session user identifier). This is the default.

-L; --no-login
The new user will not be allowed to log in. (A role without login privilege is still useful as a means of managing database permissions.)

-m role; --with-member=role
Specifies an existing role that will be automatically added as a member of the new role. Multiple existing roles can be specified by writing multiple -m switches.

-P; --pwprompt
If given, createuser will issue a prompt for the password of the new user. This is not necessary if you do not plan on using password authentication.

-r; --createrole
The new user will be allowed to create, alter, drop, comment on, change the security label for other roles; that is, this user will have CREATEROLE privilege. See ??? for more details about what capabilities are conferred by this privilege.

-R; --no-createrole
The new user will not be allowed to create new roles. This is the default.

-s; --superuser
The new user will be a superuser.

-S; --no-superuser
The new user will not be a superuser. This is the default.

-v timestamp; --valid-until=timestamp
Set a date and time after which the role's password is no longer valid. The default is to set no password expiry date.

-V; --version
Print the createuser version and exit.

--bypassrls
The new user will bypass every row-level security (RLS) policy.

--no-bypassrls
The new user will not bypass row-level security (RLS) policies. This is the default.

--replication
The new user will have the REPLICATION privilege, which is described more fully in the documentation for ???.

--no-replication
The new user will not have the REPLICATION privilege, which is described more fully in the documentation for ???. This is the default.

-?; --help
Show help about createuser command line arguments, and exit.

createuser also accepts the following command-line arguments for connection parameters:

-h host; --host=host
Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for the Unix domain socket.

-p port; --port=port
Specifies the TCP port or local Unix domain socket file extension on which the server is listening for connections.

-U username; --username=username
User name to connect as (not the user name to create).

-w; --no-password
Never issue a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password.

-W; --password
Force createuser to prompt for a password (for connecting to the server, not for the password of the new user).

This option is never essential, since createuser will automatically prompt for a password if the server demands password authentication. However, createuser will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt.

Environment

PGHOST; PGPORT; PGUSER
Default connection parameters

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

This utility, like most other PostgreSQL utilities, also uses the environment variables supported by libpq (see ???).

Diagnostics

In case of difficulty, see ??? and ??? for discussions of potential problems and error messages. The database server must be running at the targeted host. Also, any default connection settings and environment variables used by the libpq front-end library will apply.

Examples

To create a user joe on the default database server:

$ createuser joe

To create a user joe on the default database server with prompting for some additional attributes:

$ createuser --interactive joe
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) n
Shall the new role be allowed to create more new roles? (y/n) n

To create the same user joe using the server on host eden, port 5000, with attributes explicitly specified, taking a look at the underlying command:

$ createuser -h eden -p 5000 -S -D -R -e joe
CREATE ROLE joe NOSUPERUSER NOCREATEDB NOCREATEROLE INHERIT LOGIN;

To create the user joe as a superuser, and assign a password immediately:

$ createuser -P -s -e joe
Enter password for new role: xyzzy
Enter it again: xyzzy
CREATE ROLE joe PASSWORD 'SCRAM-SHA-256$4096:44560wPMLfjqiAzyPDZ/eQ==$4CA054rZlSFEq8Z3FEhToBTa2X6KnWFxFkPwIbKoDe0=:L/nbSZRCjp6RhOhKK56GoR1zibCCSePKshCJ9lnl3yw=' SUPERUSER CREATEDB CREATEROLE INHERIT LOGIN NOREPLICATION NOBYPASSRLS;

In the above example, the new password isn't actually echoed when typed, but we show what was typed for clarity. As you see, the password is encrypted before it is sent to the client.

See Also

deallocate

DEALLOCATE

prepared statements

removing

DEALLOCATE

7

SQL - Language Statements

DEALLOCATE

deallocate a prepared statement

DEALLOCATE [ PREPARE ] {

name

| ALL }

Description

DEALLOCATE is used to deallocate a previously prepared SQL statement. If you do not explicitly deallocate a prepared statement, it is deallocated when the session ends.

For more information on prepared statements, see ???.

Parameters

PREPARE
This key word is ignored.

<name>
The name of the prepared statement to deallocate.

ALL
Deallocate all prepared statements.

Compatibility

The SQL standard includes a DEALLOCATE statement, but it is only for use in embedded SQL.

See Also

declare

DECLARE

cursor

DECLARE

portal

DECLARE

DECLARE

7

SQL - Language Statements

DECLARE

define a cursor

DECLARE

name

[ BINARY ] [ ASENSITIVE | INSENSITIVE ] [ [ NO ] SCROLL ] CURSOR [ { WITH | WITHOUT } HOLD ] FOR

query

Description

DECLARE allows a user to create cursors, which can be used to retrieve a small number of rows at a time out of a larger query. After the cursor is created, rows are fetched from it using FETCH.

[!NOTE] This page describes usage of cursors at the SQL command level. If you are trying to use cursors inside a PL/pgSQL function, the rules are different — see ???.

Parameters

<name>
The name of the cursor to be created. This must be different from any other active cursor name in the session.

BINARY
Causes the cursor to return data in binary rather than in text format.

ASENSITIVE; INSENSITIVE
Cursor sensitivity determines whether changes to the data underlying the cursor, done in the same transaction, after the cursor has been declared, are visible in the cursor. INSENSITIVE means they are not visible, ASENSITIVE means the behavior is implementation-dependent. A third behavior, SENSITIVE, meaning that such changes are visible in the cursor, is not available in PostgreSQL. In PostgreSQL, all cursors are insensitive; so these key words have no effect and are only accepted for compatibility with the SQL standard.

Specifying INSENSITIVE together with FOR UPDATE or FOR SHARE is an error.

SCROLL; NO SCROLL
SCROLL specifies that the cursor can be used to retrieve rows in a nonsequential fashion (e.g., backward). Depending upon the complexity of the query's execution plan, specifying SCROLL might impose a performance penalty on the query's execution time. NO SCROLL specifies that the cursor cannot be used to retrieve rows in a nonsequential fashion. The default is to allow scrolling in some cases; this is not the same as specifying SCROLL. See Notes below for details.

WITH HOLD; WITHOUT HOLD
WITH HOLD specifies that the cursor can continue to be used after the transaction that created it successfully commits. WITHOUT HOLD specifies that the cursor cannot be used outside of the transaction that created it. If neither WITHOUT HOLD nor WITH HOLD is specified, WITHOUT HOLD is the default.

<query>
A SELECT or VALUES command which will provide the rows to be returned by the cursor.

The key words ASENSITIVE, BINARY, INSENSITIVE, and SCROLL can appear in any order.

Notes

Normal cursors return data in text format, the same as a SELECT would produce. The BINARY option specifies that the cursor should return data in binary format. This reduces conversion effort for both the server and client, at the cost of more programmer effort to deal with platform-dependent binary data formats. As an example, if a query returns a value of one from an integer column, you would get a string of 1 with a default cursor, whereas with a binary cursor you would get a 4-byte field containing the internal representation of the value (in big-endian byte order).

Binary cursors should be used carefully. Many applications, including psql, are not prepared to handle binary cursors and expect data to come back in the text format.

[!NOTE] When the client application uses the “extended query” protocol to issue a FETCH command, the Bind protocol message specifies whether data is to be retrieved in text or binary format. This choice overrides the way that the cursor is defined. The concept of a binary cursor as such is thus obsolete when using extended query protocol — any cursor can be treated as either text or binary.

Unless WITH HOLD is specified, the cursor created by this command can only be used within the current transaction. Thus, DECLARE without WITH HOLD is useless outside a transaction block: the cursor would survive only to the completion of the statement. Therefore PostgreSQL reports an error if such a command is used outside a transaction block. Use BEGIN and COMMIT (or ROLLBACK) to define a transaction block.

If WITH HOLD is specified and the transaction that created the cursor successfully commits, the cursor can continue to be accessed by subsequent transactions in the same session. (But if the creating transaction is aborted, the cursor is removed.) A cursor created with WITH HOLD is closed when an explicit CLOSE command is issued on it, or the session ends. In the current implementation, the rows represented by a held cursor are copied into a temporary file or memory area so that they remain available for subsequent transactions.

WITH HOLD may not be specified when the query includes FOR UPDATE or FOR SHARE.

The SCROLL option should be specified when defining a cursor that will be used to fetch backwards. This is required by the SQL standard. However, for compatibility with earlier versions, PostgreSQL will allow backward fetches without SCROLL, if the cursor's query plan is simple enough that no extra overhead is needed to support it. However, application developers are advised not to rely on using backward fetches from a cursor that has not been created with SCROLL. If NO SCROLL is specified, then backward fetches are disallowed in any case.

Backward fetches are also disallowed when the query includes FOR UPDATE or FOR SHARE; therefore SCROLL may not be specified in this case.

[!CAUTION] Scrollable cursors may give unexpected results if they invoke any volatile functions (see ???). When a previously fetched row is re-fetched, the functions might be re-executed, perhaps leading to results different from the first time. It's best to specify NO SCROLL for a query involving volatile functions. If that is not practical, one workaround is to declare the cursor SCROLL WITH HOLD and commit the transaction before reading any rows from it. This will force the entire output of the cursor to be materialized in temporary storage, so that volatile functions are executed exactly once for each row.

If the cursor's query includes FOR UPDATE or FOR SHARE, then returned rows are locked at the time they are first fetched, in the same way as for a regular SELECT command with these options. In addition, the returned rows will be the most up-to-date versions.

[!CAUTION] It is generally recommended to use FOR UPDATE if the cursor is intended to be used with UPDATE ... WHERE CURRENT OF or DELETE ... WHERE CURRENT OF. Using FOR UPDATE prevents other sessions from changing the rows between the time they are fetched and the time they are updated. Without FOR UPDATE, a subsequent WHERE CURRENT OF command will have no effect if the row was changed since the cursor was created.

Another reason to use FOR UPDATE is that without it, a subsequent WHERE CURRENT OF might fail if the cursor query does not meet the SQL standard's rules for being “simply updatable” (in particular, the cursor must reference just one table and not use grouping or ORDER BY). Cursors that are not simply updatable might work, or might not, depending on plan choice details; so in the worst case, an application might work in testing and then fail in production. If FOR UPDATE is specified, the cursor is guaranteed to be updatable.

The main reason not to use FOR UPDATE with WHERE CURRENT OF is if you need the cursor to be scrollable, or to be isolated from concurrent updates (that is, continue to show the old data). If this is a requirement, pay close heed to the caveats shown above.

The SQL standard only makes provisions for cursors in embedded SQL. The PostgreSQL server does not implement an OPEN statement for cursors; a cursor is considered to be open when it is declared. However, ECPG, the embedded SQL preprocessor for PostgreSQL, supports the standard SQL cursor conventions, including those involving DECLARE and OPEN statements.

The server data structure underlying an open cursor is called a portal. Portal names are exposed in the client protocol: a client can fetch rows directly from an open portal, if it knows the portal name. When creating a cursor with DECLARE, the portal name is the same as the cursor name.

You can see all available cursors by querying the pg_cursors system view.

Examples

To declare a cursor:

DECLARE liahona CURSOR FOR SELECT * FROM films;

See ??? for more examples of cursor usage.

Compatibility

The SQL standard allows cursors only in embedded SQL and in modules. PostgreSQL permits cursors to be used interactively.

According to the SQL standard, changes made to insensitive cursors by UPDATE ... WHERE CURRENT OF and DELETE ... WHERE CURRENT OF statements are visible in that same cursor. PostgreSQL treats these statements like all other data changing statements in that they are not visible in insensitive cursors.

Binary cursors are a PostgreSQL extension.

See Also

delete

DELETE

DELETE

7

SQL - Language Statements

DELETE

delete rows of a table

[ WITH [ RECURSIVE ]

with_query

[, ...] ] DELETE FROM [ ONLY ]

table_name

[ * ] [ [ AS ]

alias

] [ USING

from_item

[, ...] ] [ WHERE

condition

| WHERE CURRENT OF

cursor_name

] [ RETURNING [ WITH ( { OLD | NEW } AS

output_alias

[, ...] ) ] { * |

output_expression

[ [ AS ]

output_name

] } [, ...] ]

Description

DELETE deletes rows that satisfy the WHERE clause from the specified table. If the WHERE clause is absent, the effect is to delete all rows in the table. The result is a valid, but empty table.

[!TIP] TRUNCATE provides a faster mechanism to remove all rows from a table.

There are two ways to delete rows in a table using information contained in other tables in the database: using sub-selects, or specifying additional tables in the USING clause. Which technique is more appropriate depends on the specific circumstances.

The optional RETURNING clause causes DELETE to compute and return value(s) based on each row actually deleted. Any expression using the table's columns, and/or columns of other tables mentioned in USING, can be computed. The syntax of the RETURNING list is identical to that of the output list of SELECT.

You must have the DELETE privilege on the table to delete from it, as well as the SELECT privilege for any table in the USING clause or whose values are read in the <condition>.

Parameters

<with_query>
The WITH clause allows you to specify one or more subqueries that can be referenced by name in the DELETE query. See ??? and ??? for details.

<table_name>
The name (optionally schema-qualified) of the table to delete rows from. If ONLY is specified before the table name, matching rows are deleted from the named table only. If ONLY is not specified, matching rows are also deleted from any tables inheriting from the named table. Optionally, * can be specified after the table name to explicitly indicate that descendant tables are included.

<alias>
A substitute name for the target table. When an alias is provided, it completely hides the actual name of the table. For example, given DELETE FROM foo AS f, the remainder of the DELETE statement must refer to this table as f not foo.

<from_item>
A table expression allowing columns from other tables to appear in the WHERE condition. This uses the same syntax as the FROM clause of a SELECT statement; for example, an alias for the table name can be specified. Do not repeat the target table as a <from_item> unless you wish to set up a self-join (in which case it must appear with an alias in the <from_item>).

<condition>
An expression that returns a value of type boolean. Only rows for which this expression returns true will be deleted.

<cursor_name>
The name of the cursor to use in a WHERE CURRENT OF condition. The row to be deleted is the one most recently fetched from this cursor. The cursor must be a non-grouping query on the DELETE's target table. Note that WHERE CURRENT OF cannot be specified together with a Boolean condition. See ??? for more information about using cursors with WHERE CURRENT OF.

<output_alias>
An optional substitute name for OLD or NEW rows in the RETURNING list.

By default, old values from the target table can be returned by writing OLD.column_name or OLD.*, and new values can be returned by writing NEW.column_name or NEW.*. When an alias is provided, these names are hidden and the old or new rows must be referred to using the alias. For example RETURNING WITH (OLD AS o, NEW AS n) o.*, n.*.

<output_expression>
An expression to be computed and returned by the DELETE command after each row is deleted. The expression can use any column names of the table named by <table_name> or table(s) listed in USING. Write * to return all columns.

A column name or * may be qualified using OLD or NEW, or the corresponding <output_alias> for OLD or NEW, to cause old or new values to be returned. An unqualified column name, or *, or a column name or * qualified using the target table name or alias will return old values.

For a simple DELETE, all new values will be NULL. However, if an ON DELETE rule causes an INSERT or UPDATE to be executed instead, the new values may be non-NULL.

<output_name>
A name to use for a returned column.

Outputs

On successful completion, a DELETE command returns a command tag of the form

DELETE count

The <count> is the number of rows deleted. Note that the number may be less than the number of rows that matched the <condition> when deletes were suppressed by a BEFORE DELETE trigger. If <count> is 0, no rows were deleted by the query (this is not considered an error).

If the DELETE command contains a RETURNING clause, the result will be similar to that of a SELECT statement containing the columns and values defined in the RETURNING list, computed over the row(s) deleted by the command.

Notes

PostgreSQL lets you reference columns of other tables in the WHERE condition by specifying the other tables in the USING clause. For example, to delete all films produced by a given producer, one can do:

DELETE FROM films USING producers
  WHERE producer_id = producers.id AND producers.name = 'foo';

What is essentially happening here is a join between films and producers, with all successfully joined films rows being marked for deletion. This syntax is not standard. A more standard way to do it is:

DELETE FROM films
  WHERE producer_id IN (SELECT id FROM producers WHERE name = 'foo');

In some cases the join style is easier to write or faster to execute than the sub-select style.

Examples

Delete all films but musicals:

DELETE FROM films WHERE kind <> 'Musical';

Clear the table films:

DELETE FROM films;

Delete completed tasks, returning full details of the deleted rows:

DELETE FROM tasks WHERE status = 'DONE' RETURNING *;

Delete the row of tasks on which the cursor c_tasks is currently positioned:

DELETE FROM tasks WHERE CURRENT OF c_tasks;

While there is no LIMIT clause for DELETE, it is possible to get a similar effect using the same method described in the documentation of UPDATE:

WITH delete_batch AS (
  SELECT l.ctid FROM user_logs AS l
    WHERE l.status = 'archived'
    ORDER BY l.creation_date
    FOR UPDATE
    LIMIT 10000
)
DELETE FROM user_logs AS dl
  USING delete_batch AS del
  WHERE dl.ctid = del.ctid;

This use of ctid is only safe because the query is repeatedly run, avoiding the problem of changed ctids.

Compatibility

This command conforms to the SQL standard, except that the USING and RETURNING clauses are PostgreSQL extensions, as is the ability to use WITH with DELETE.

See Also

discard

DISCARD

DISCARD

7

SQL - Language Statements

DISCARD

discard session state

DISCARD { ALL | PLANS | SEQUENCES | TEMPORARY | TEMP }

Description

DISCARD releases internal resources associated with a database session. This command is useful for partially or fully resetting the session's state. There are several subcommands to release different types of resources; the DISCARD ALL variant subsumes all the others, and also resets additional state.

Parameters

PLANS
Releases all cached query plans, forcing re-planning to occur the next time the associated prepared statement is used.

SEQUENCES
Discards all cached sequence-related state, including currval()/lastval() information and any preallocated sequence values that have not yet been returned by nextval(). (See ??? for a description of preallocated sequence values.)

TEMPORARY or TEMP
Drops all temporary tables created in the current session.

ALL
Releases all temporary resources associated with the current session and resets the session to its initial state. Currently, this has the same effect as executing the following sequence of statements:

CLOSE ALL;
SET SESSION AUTHORIZATION DEFAULT;
RESET ALL;
DEALLOCATE ALL;
UNLISTEN *;
SELECT pg_advisory_unlock_all();
DISCARD PLANS;
DISCARD TEMP;
DISCARD SEQUENCES;

Notes

DISCARD ALL cannot be executed inside a transaction block.

Compatibility

DISCARD is a PostgreSQL extension.

do

DO

anonymous code blocks

DO

7

SQL - Language Statements

DO

execute an anonymous code block

DO [ LANGUAGE

lang_name

]

code

Description

DO executes an anonymous code block, or in other words a transient anonymous function in a procedural language.

The code block is treated as though it were the body of a function with no parameters, returning void. It is parsed and executed a single time.

The optional LANGUAGE clause can be written either before or after the code block.

Parameters

<code>
The procedural language code to be executed. This must be specified as a string literal, just as in CREATE FUNCTION. Use of a dollar-quoted literal is recommended.

<lang_name>
The name of the procedural language the code is written in. If omitted, the default is plpgsql.

Notes

The procedural language to be used must already have been installed into the current database by means of CREATE EXTENSION. plpgsql is installed by default, but other languages are not.

The user must have USAGE privilege for the procedural language, or must be a superuser if the language is untrusted. This is the same privilege requirement as for creating a function in the language.

If DO is executed in a transaction block, then the procedure code cannot execute transaction control statements. Transaction control statements are only allowed if DO is executed in its own transaction.

Examples

Grant all privileges on all views in schema public to role webuser:

DO $$
DECLARE r record;
BEGIN
    FOR r IN SELECT table_schema, table_name FROM information_schema.tables
             WHERE table_type = 'VIEW' AND table_schema = 'public'
    LOOP
        EXECUTE 'GRANT ALL ON ' || quote_ident(r.table_schema) || '.' || quote_ident(r.table_name) || ' TO webuser';
    END LOOP;
END$$;

Compatibility

There is no DO statement in the SQL standard.

See Also

drop_access_method

DROP ACCESS METHOD

DROP ACCESS METHOD

7

SQL - Language Statements

DROP ACCESS METHOD

remove an access method

DROP ACCESS METHOD [ IF EXISTS ]

name

[ CASCADE | RESTRICT ]

Description

DROP ACCESS METHOD removes an existing access method. Only superusers can drop access methods.

Parameters

IF EXISTS
Do not throw an error if the access method does not exist. A notice is issued in this case.

<name>
The name of an existing access method.

CASCADE
Automatically drop objects that depend on the access method (such as operator classes, operator families, and indexes), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the access method if any objects depend on it. This is the default.

Examples

Drop the access method heptree:

DROP ACCESS METHOD heptree;

Compatibility

DROP ACCESS METHOD is a PostgreSQL extension.

See Also

drop_aggregate

DROP AGGREGATE

DROP AGGREGATE

7

SQL - Language Statements

DROP AGGREGATE

remove an aggregate function

DROP AGGREGATE [ IF EXISTS ]

name

(

aggregate_signature

) [, ...] [ CASCADE | RESTRICT ]

where

aggregate_signature

is:

* | [

argmode

] [

argname

]

argtype

[ , ... ] | [ [

argmode

] [

argname

]

argtype

[ , ... ] ] ORDER BY [

argmode

] [

argname

]

argtype

[ , ... ]

Description

DROP AGGREGATE removes an existing aggregate function. To execute this command the current user must be the owner of the aggregate function.

Parameters

IF EXISTS
Do not throw an error if the aggregate does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of an existing aggregate function.

<argmode>
The mode of an argument: IN or VARIADIC. If omitted, the default is IN.

<argname>
The name of an argument. Note that DROP AGGREGATE does not actually pay any attention to argument names, since only the argument data types are needed to determine the aggregate function's identity.

<argtype>
An input data type on which the aggregate function operates. To reference a zero-argument aggregate function, write * in place of the list of argument specifications. To reference an ordered-set aggregate function, write ORDER BY between the direct and aggregated argument specifications.

CASCADE
Automatically drop objects that depend on the aggregate function (such as views using it), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the aggregate function if any objects depend on it. This is the default.

Notes

Alternative syntaxes for referencing ordered-set aggregates are described under ???.

Examples

To remove the aggregate function myavg for type integer:

DROP AGGREGATE myavg(integer);

To remove the hypothetical-set aggregate function myrank, which takes an arbitrary list of ordering columns and a matching list of direct arguments:

DROP AGGREGATE myrank(VARIADIC "any" ORDER BY VARIADIC "any");

To remove multiple aggregate functions in one command:

DROP AGGREGATE myavg(integer), myavg(bigint);

Compatibility

There is no DROP AGGREGATE statement in the SQL standard.

See Also

drop_cast

DROP CAST

DROP CAST

7

SQL - Language Statements

DROP CAST

remove a cast

DROP CAST [ IF EXISTS ] (

source_type

AS

target_type

) [ CASCADE | RESTRICT ]

Description

DROP CAST removes a previously defined cast.

To be able to drop a cast, you must own the source or the target data type. These are the same privileges that are required to create a cast.

Parameters

IF EXISTS
Do not throw an error if the cast does not exist. A notice is issued in this case.

<source_type>
The name of the source data type of the cast.

<target_type>
The name of the target data type of the cast.

CASCADE; RESTRICT
These key words do not have any effect, since there are no dependencies on casts.

Examples

To drop the cast from type text to type int:

DROP CAST (text AS int);

Compatibility

The DROP CAST command conforms to the SQL standard.

See Also

drop_collation

DROP COLLATION

DROP COLLATION

7

SQL - Language Statements

DROP COLLATION

remove a collation

DROP COLLATION [ IF EXISTS ]

name

[ CASCADE | RESTRICT ]

Description

DROP COLLATION removes a previously defined collation. To be able to drop a collation, you must own the collation.

Parameters

IF EXISTS
Do not throw an error if the collation does not exist. A notice is issued in this case.

<name>
The name of the collation. The collation name can be schema-qualified.

CASCADE
Automatically drop objects that depend on the collation, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the collation if any objects depend on it. This is the default.

Examples

To drop the collation named german:

DROP COLLATION german;

Compatibility

The DROP COLLATION command conforms to the SQL standard, apart from the IF EXISTS option, which is a PostgreSQL extension.

See Also

drop_conversion

DROP CONVERSION

DROP CONVERSION

7

SQL - Language Statements

DROP CONVERSION

remove a conversion

DROP CONVERSION [ IF EXISTS ]

name

[ CASCADE | RESTRICT ]

Description

DROP CONVERSION removes a previously defined conversion. To be able to drop a conversion, you must own the conversion.

Parameters

IF EXISTS
Do not throw an error if the conversion does not exist. A notice is issued in this case.

<name>
The name of the conversion. The conversion name can be schema-qualified.

CASCADE; RESTRICT
These key words do not have any effect, since there are no dependencies on conversions.

Examples

To drop the conversion named myname:

DROP CONVERSION myname;

Compatibility

There is no DROP CONVERSION statement in the SQL standard, but a DROP TRANSLATION statement that goes along with the CREATE TRANSLATION statement that is similar to the CREATE CONVERSION statement in PostgreSQL.

See Also

drop_database

DROP DATABASE

DROP DATABASE

7

SQL - Language Statements

DROP DATABASE

remove a database

DROP DATABASE [ IF EXISTS ]

name

[ [ WITH ] (

option

[, ...] ) ]

where

option

can be:

FORCE

Description

DROP DATABASE drops a database. It removes the catalog entries for the database and deletes the directory containing the data. It can only be executed by the database owner. It cannot be executed while you are connected to the target database. (Connect to postgres or any other database to issue this command.) Also, if anyone else is connected to the target database, this command will fail unless you use the FORCE option described below.

DROP DATABASE cannot be undone. Use it with care!

Parameters

IF EXISTS
Do not throw an error if the database does not exist. A notice is issued in this case.

<name>
The name of the database to remove.

FORCE
Attempt to terminate all existing connections to the target database. It doesn't terminate if prepared transactions, active logical replication slots or subscriptions are present in the target database.

This terminates background worker connections and connections that the current user has permission to terminate with pg_terminate_backend, described in ???. If connections would remain, this command will fail.

Notes

DROP DATABASE cannot be executed inside a transaction block.

This command cannot be executed while connected to the target database. Thus, it might be more convenient to use the program ??? instead, which is a wrapper around this command.

Compatibility

There is no DROP DATABASE statement in the SQL standard.

See Also

drop_domain

DROP DOMAIN

DROP DOMAIN

7

SQL - Language Statements

DROP DOMAIN

remove a domain

DROP DOMAIN [ IF EXISTS ]

name

[, ...] [ CASCADE | RESTRICT ]

Description

DROP DOMAIN removes a domain. Only the owner of a domain can remove it.

Parameters

IF EXISTS
Do not throw an error if the domain does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of an existing domain.

CASCADE
Automatically drop objects that depend on the domain (such as table columns), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the domain if any objects depend on it. This is the default.

Examples

To remove the domain box:

DROP DOMAIN box;

Compatibility

This command conforms to the SQL standard, except for the IF EXISTS option, which is a PostgreSQL extension.

See Also

drop_event_trigger

DROP EVENT TRIGGER

DROP EVENT TRIGGER

7

SQL - Language Statements

DROP EVENT TRIGGER

remove an event trigger

DROP EVENT TRIGGER [ IF EXISTS ]

name

[ CASCADE | RESTRICT ]

Description

DROP EVENT TRIGGER removes an existing event trigger. To execute this command, the current user must be the owner of the event trigger.

Parameters

IF EXISTS
Do not throw an error if the event trigger does not exist. A notice is issued in this case.

<name>
The name of the event trigger to remove.

CASCADE
Automatically drop objects that depend on the trigger, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the trigger if any objects depend on it. This is the default.

Examples

Destroy the trigger snitch:

DROP EVENT TRIGGER snitch;

Compatibility

There is no DROP EVENT TRIGGER statement in the SQL standard.

See Also

drop_extension

DROP EXTENSION

DROP EXTENSION

7

SQL - Language Statements

DROP EXTENSION

remove an extension

DROP EXTENSION [ IF EXISTS ]

name

[, ...] [ CASCADE | RESTRICT ]

Description

DROP EXTENSION removes extensions from the database. Dropping an extension causes its member objects, and other explicitly dependent routines (see ???, the DEPENDS ON EXTENSION extension_name action), to be dropped as well.

You must own the extension to use DROP EXTENSION.

Parameters

IF EXISTS
Do not throw an error if the extension does not exist. A notice is issued in this case.

<name>
The name of an installed extension.

CASCADE
Automatically drop objects that depend on the extension, and in turn all objects that depend on those objects (see ???).

RESTRICT
This option prevents the specified extensions from being dropped if other objects, besides these extensions, their members, and their explicitly dependent routines, depend on them. This is the default.

Examples

To remove the extension hstore from the current database:

DROP EXTENSION hstore;

This command will fail if any of hstore's objects are in use in the database, for example if any tables have columns of the hstore type. Add the CASCADE option to forcibly remove those dependent objects as well.

Compatibility

DROP EXTENSION is a PostgreSQL extension.

See Also

drop_foreign_data_wrapper

DROP FOREIGN DATA WRAPPER

DROP FOREIGN DATA WRAPPER

7

SQL - Language Statements

DROP FOREIGN DATA WRAPPER

remove a foreign-data wrapper

DROP FOREIGN DATA WRAPPER [ IF EXISTS ]

name

[, ...] [ CASCADE | RESTRICT ]

Description

DROP FOREIGN DATA WRAPPER removes an existing foreign-data wrapper. To execute this command, the current user must be the owner of the foreign-data wrapper.

Parameters

IF EXISTS
Do not throw an error if the foreign-data wrapper does not exist. A notice is issued in this case.

<name>
The name of an existing foreign-data wrapper.

CASCADE
Automatically drop objects that depend on the foreign-data wrapper (such as foreign tables and servers), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the foreign-data wrapper if any objects depend on it. This is the default.

Examples

Drop the foreign-data wrapper dbi:

DROP FOREIGN DATA WRAPPER dbi;

Compatibility

DROP FOREIGN DATA WRAPPER conforms to ISO/IEC 9075-9 (SQL/MED). The IF EXISTS clause is a PostgreSQL extension.

See Also

drop_foreign_table

DROP FOREIGN TABLE

DROP FOREIGN TABLE

7

SQL - Language Statements

DROP FOREIGN TABLE

remove a foreign table

DROP FOREIGN TABLE [ IF EXISTS ]

name

[, ...] [ CASCADE | RESTRICT ]

Description

DROP FOREIGN TABLE removes a foreign table. Only the owner of a foreign table can remove it.

Parameters

IF EXISTS
Do not throw an error if the foreign table does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of the foreign table to drop.

CASCADE
Automatically drop objects that depend on the foreign table (such as views), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the foreign table if any objects depend on it. This is the default.

Examples

To destroy two foreign tables, films and distributors:

DROP FOREIGN TABLE films, distributors;

Compatibility

This command conforms to ISO/IEC 9075-9 (SQL/MED), except that the standard only allows one foreign table to be dropped per command, and apart from the IF EXISTS option, which is a PostgreSQL extension.

See Also

drop_function

DROP FUNCTION

DROP FUNCTION

7

SQL - Language Statements

DROP FUNCTION

remove a function

DROP FUNCTION [ IF EXISTS ]

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] [, ...] [ CASCADE | RESTRICT ]

Description

DROP FUNCTION removes the definition of an existing function. To execute this command the user must be the owner of the function. The argument types to the function must be specified, since several different functions can exist with the same name and different argument lists.

Parameters

IF EXISTS
Do not throw an error if the function does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of an existing function. If no argument list is specified, the name must be unique in its schema.

<argmode>
The mode of an argument: IN, OUT, INOUT, or VARIADIC. If omitted, the default is IN. Note that DROP FUNCTION does not actually pay any attention to OUT arguments, since only the input arguments are needed to determine the function's identity. So it is sufficient to list the IN, INOUT, and VARIADIC arguments.

<argname>
The name of an argument. Note that DROP FUNCTION does not actually pay any attention to argument names, since only the argument data types are needed to determine the function's identity.

<argtype>
The data type(s) of the function's arguments (optionally schema-qualified), if any.

CASCADE
Automatically drop objects that depend on the function (such as operators or triggers), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the function if any objects depend on it. This is the default.

Examples

This command removes the square root function:

DROP FUNCTION sqrt(integer);

Drop multiple functions in one command:

DROP FUNCTION sqrt(integer), sqrt(bigint);

If the function name is unique in its schema, it can be referred to without an argument list:

DROP FUNCTION update_employee_salaries;

Note that this is different from

DROP FUNCTION update_employee_salaries();

which refers to a function with zero arguments, whereas the first variant can refer to a function with any number of arguments, including zero, as long as the name is unique.

Compatibility

This command conforms to the SQL standard, with these PostgreSQL extensions:

See Also

drop_group

DROP GROUP

DROP GROUP

7

SQL - Language Statements

DROP GROUP

remove a database role

DROP GROUP [ IF EXISTS ]

name

[, ...]

Description

DROP GROUP is now an alias for DROP ROLE.

Compatibility

There is no DROP GROUP statement in the SQL standard.

See Also

drop_index

DROP INDEX

DROP INDEX

7

SQL - Language Statements

DROP INDEX

remove an index

DROP INDEX [ CONCURRENTLY ] [ IF EXISTS ]

name

[, ...] [ CASCADE | RESTRICT ]

Description

DROP INDEX drops an existing index from the database system. To execute this command you must be the owner of the index.

Parameters

CONCURRENTLY
Drop the index without locking out concurrent selects, inserts, updates, and deletes on the index's table. A normal DROP INDEX acquires an ACCESS EXCLUSIVE lock on the table, blocking other accesses until the index drop can be completed. With this option, the command instead waits until conflicting transactions have completed.

There are several caveats to be aware of when using this option. Only one index name can be specified, and the CASCADE option is not supported. (Thus, an index that supports a UNIQUE or PRIMARY KEY constraint cannot be dropped this way.) Also, regular DROP INDEX commands can be performed within a transaction block, but DROP INDEX CONCURRENTLY cannot. Lastly, indexes on partitioned tables cannot be dropped using this option.

For temporary tables, DROP INDEX is always non-concurrent, as no other session can access them, and non-concurrent index drop is cheaper.

IF EXISTS
Do not throw an error if the index does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of an index to remove.

CASCADE
Automatically drop objects that depend on the index, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the index if any objects depend on it. This is the default.

Examples

This command will remove the index title_idx:

DROP INDEX title_idx;

Compatibility

DROP INDEX is a PostgreSQL language extension. There are no provisions for indexes in the SQL standard.

See Also

drop_language

DROP LANGUAGE

DROP LANGUAGE

7

SQL - Language Statements

DROP LANGUAGE

remove a procedural language

DROP [ PROCEDURAL ] LANGUAGE [ IF EXISTS ]

name

[ CASCADE | RESTRICT ]

Description

DROP LANGUAGE removes the definition of a previously registered procedural language. You must be a superuser or the owner of the language to use DROP LANGUAGE.

[!NOTE] As of PostgreSQL 9.1, most procedural languages have been made into “extensions”, and should therefore be removed with DROP EXTENSION not DROP LANGUAGE.

Parameters

IF EXISTS
Do not throw an error if the language does not exist. A notice is issued in this case.

<name>
The name of an existing procedural language.

CASCADE
Automatically drop objects that depend on the language (such as functions in the language), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the language if any objects depend on it. This is the default.

Examples

This command removes the procedural language plsample:

DROP LANGUAGE plsample;

Compatibility

There is no DROP LANGUAGE statement in the SQL standard.

See Also

drop_materialized_view

DROP MATERIALIZED VIEW

DROP MATERIALIZED VIEW

7

SQL - Language Statements

DROP MATERIALIZED VIEW

remove a materialized view

DROP MATERIALIZED VIEW [ IF EXISTS ]

name

[, ...] [ CASCADE | RESTRICT ]

Description

DROP MATERIALIZED VIEW drops an existing materialized view. To execute this command you must be the owner of the materialized view.

Parameters

IF EXISTS
Do not throw an error if the materialized view does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of the materialized view to remove.

CASCADE
Automatically drop objects that depend on the materialized view (such as other materialized views, or regular views), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the materialized view if any objects depend on it. This is the default.

Examples

This command will remove the materialized view called order_summary:

DROP MATERIALIZED VIEW order_summary;

Compatibility

DROP MATERIALIZED VIEW is a PostgreSQL extension.

See Also

drop_opclass

DROP OPERATOR CLASS

DROP OPERATOR CLASS

7

SQL - Language Statements

DROP OPERATOR CLASS

remove an operator class

DROP OPERATOR CLASS [ IF EXISTS ]

name

USING

index_method

[ CASCADE | RESTRICT ]

Description

DROP OPERATOR CLASS drops an existing operator class. To execute this command you must be the owner of the operator class.

DROP OPERATOR CLASS does not drop any of the operators or functions referenced by the class. If there are any indexes depending on the operator class, you will need to specify CASCADE for the drop to complete.

Parameters

IF EXISTS
Do not throw an error if the operator class does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of an existing operator class.

<index_method>
The name of the index access method the operator class is for.

CASCADE
Automatically drop objects that depend on the operator class (such as indexes), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the operator class if any objects depend on it. This is the default.

Notes

DROP OPERATOR CLASS will not drop the operator family containing the class, even if there is nothing else left in the family (in particular, in the case where the family was implicitly created by CREATE OPERATOR CLASS). An empty operator family is harmless, but for the sake of tidiness you might wish to remove the family with DROP OPERATOR FAMILY; or perhaps better, use DROP OPERATOR FAMILY in the first place.

Examples

Remove the B-tree operator class widget_ops:

DROP OPERATOR CLASS widget_ops USING btree;

This command will not succeed if there are any existing indexes that use the operator class. Add CASCADE to drop such indexes along with the operator class.

Compatibility

There is no DROP OPERATOR CLASS statement in the SQL standard.

See Also

drop_operator

DROP OPERATOR

DROP OPERATOR

7

SQL - Language Statements

DROP OPERATOR

remove an operator

DROP OPERATOR [ IF EXISTS ]

name

( {

left_type

| NONE } ,

right_type

) [, ...] [ CASCADE | RESTRICT ]

Description

DROP OPERATOR drops an existing operator from the database system. To execute this command you must be the owner of the operator.

Parameters

IF EXISTS
Do not throw an error if the operator does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of an existing operator.

<left_type>
The data type of the operator's left operand; write NONE if the operator has no left operand.

<right_type>
The data type of the operator's right operand.

CASCADE
Automatically drop objects that depend on the operator (such as views using it), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the operator if any objects depend on it. This is the default.

Examples

Remove the power operator a^b for type integer:

DROP OPERATOR ^ (integer, integer);

Remove the bitwise-complement prefix operator ~b for type bit:

DROP OPERATOR ~ (none, bit);

Remove multiple operators in one command:

DROP OPERATOR ~ (none, bit), ^ (integer, integer);

Compatibility

There is no DROP OPERATOR statement in the SQL standard.

See Also

drop_opfamily

DROP OPERATOR FAMILY

DROP OPERATOR FAMILY

7

SQL - Language Statements

DROP OPERATOR FAMILY

remove an operator family

DROP OPERATOR FAMILY [ IF EXISTS ]

name

USING

index_method

[ CASCADE | RESTRICT ]

Description

DROP OPERATOR FAMILY drops an existing operator family. To execute this command you must be the owner of the operator family.

DROP OPERATOR FAMILY includes dropping any operator classes contained in the family, but it does not drop any of the operators or functions referenced by the family. If there are any indexes depending on operator classes within the family, you will need to specify CASCADE for the drop to complete.

Parameters

IF EXISTS
Do not throw an error if the operator family does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of an existing operator family.

<index_method>
The name of the index access method the operator family is for.

CASCADE
Automatically drop objects that depend on the operator family, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the operator family if any objects depend on it. This is the default.

Examples

Remove the B-tree operator family float_ops:

DROP OPERATOR FAMILY float_ops USING btree;

This command will not succeed if there are any existing indexes that use operator classes within the family. Add CASCADE to drop such indexes along with the operator family.

Compatibility

There is no DROP OPERATOR FAMILY statement in the SQL standard.

See Also

drop_owned

DROP OWNED

DROP OWNED

7

SQL - Language Statements

DROP OWNED

remove database objects owned by a database role

DROP OWNED BY {

name

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } [, ...] [ CASCADE | RESTRICT ]

Description

DROP OWNED drops all the objects within the current database that are owned by one of the specified roles. Any privileges granted to the given roles on objects in the current database or on shared objects (databases, tablespaces, configuration parameters) will also be revoked.

Parameters

<name>
The name of a role whose objects will be dropped, and whose privileges will be revoked.

CASCADE
Automatically drop objects that depend on the affected objects, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the objects owned by a role if any other database objects depend on one of the affected objects. This is the default.

Notes

DROP OWNED is often used to prepare for the removal of one or more roles. Because DROP OWNED only affects the objects in the current database, it is usually necessary to execute this command in each database that contains objects owned by a role that is to be removed.

Using the CASCADE option might make the command recurse to objects owned by other users.

The REASSIGN OWNED command is an alternative that reassigns the ownership of all the database objects owned by one or more roles. However, REASSIGN OWNED does not deal with privileges for other objects.

Databases and tablespaces owned by the role(s) will not be removed.

See ??? for more discussion.

Compatibility

The DROP OWNED command is a PostgreSQL extension.

See Also

drop_policy

DROP POLICY

DROP POLICY

7

SQL - Language Statements

DROP POLICY

remove a row-level security policy from a table

DROP POLICY [ IF EXISTS ]

name

ON

table_name

[ CASCADE | RESTRICT ]

Description

DROP POLICY removes the specified policy from the table. Note that if the last policy is removed for a table and the table still has row-level security enabled via ALTER TABLE, then the default-deny policy will be used. ALTER TABLE ... DISABLE ROW LEVEL SECURITY can be used to disable row-level security for a table, whether policies for the table exist or not.

Parameters

IF EXISTS
Do not throw an error if the policy does not exist. A notice is issued in this case.

<name>
The name of the policy to drop.

<table_name>
The name (optionally schema-qualified) of the table that the policy is on.

CASCADE; RESTRICT
These key words do not have any effect, since there are no dependencies on policies.

Examples

To drop the policy called p1 on the table named my_table:

DROP POLICY p1 ON my_table;

Compatibility

DROP POLICY is a PostgreSQL extension.

See Also

drop_procedure

DROP PROCEDURE

DROP PROCEDURE

7

SQL - Language Statements

DROP PROCEDURE

remove a procedure

DROP PROCEDURE [ IF EXISTS ]

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] [, ...] [ CASCADE | RESTRICT ]

Description

DROP PROCEDURE removes the definition of one or more existing procedures. To execute this command the user must be the owner of the procedure(s). The argument types to the procedure(s) usually must be specified, since several different procedures can exist with the same name and different argument lists.

Parameters

IF EXISTS
Do not throw an error if the procedure does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of an existing procedure.

<argmode>
The mode of an argument: IN, OUT, INOUT, or VARIADIC. If omitted, the default is IN (but see below).

<argname>
The name of an argument. Note that DROP PROCEDURE does not actually pay any attention to argument names, since only the argument data types are used to determine the procedure's identity.

<argtype>
The data type(s) of the procedure's arguments (optionally schema-qualified), if any. See below for details.

CASCADE
Automatically drop objects that depend on the procedure, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the procedure if any objects depend on it. This is the default.

Notes

If there is only one procedure of the given name, the argument list can be omitted. Omit the parentheses too in this case.

In PostgreSQL, it's sufficient to list the input (including INOUT) arguments, because no two routines of the same name are allowed to share the same input-argument list. Moreover, the DROP command will not actually check that you wrote the types of OUT arguments correctly; so any arguments that are explicitly marked OUT are just noise. But writing them is recommendable for consistency with the corresponding CREATE command.

For compatibility with the SQL standard, it is also allowed to write all the argument data types (including those of OUT arguments) without any <argmode> markers. When this is done, the types of the procedure's OUT argument(s) will be verified against the command. This provision creates an ambiguity, in that when the argument list contains no <argmode> markers, it's unclear which rule is intended. The DROP command will attempt the lookup both ways, and will throw an error if two different procedures are found. To avoid the risk of such ambiguity, it's recommendable to write IN markers explicitly rather than letting them be defaulted, thus forcing the traditional PostgreSQL interpretation to be used.

The lookup rules just explained are also used by other commands that act on existing procedures, such as ALTER PROCEDURE and COMMENT ON PROCEDURE.

Examples

If there is only one procedure do_db_maintenance, this command is sufficient to drop it:

DROP PROCEDURE do_db_maintenance;

Given this procedure definition:

CREATE PROCEDURE do_db_maintenance(IN target_schema text, OUT results text) ...

any one of these commands would work to drop it:

DROP PROCEDURE do_db_maintenance(IN target_schema text, OUT results text);
DROP PROCEDURE do_db_maintenance(IN text, OUT text);
DROP PROCEDURE do_db_maintenance(IN text);
DROP PROCEDURE do_db_maintenance(text);
DROP PROCEDURE do_db_maintenance(text, text);  -- potentially ambiguous

However, the last example would be ambiguous if there is also, say,

CREATE PROCEDURE do_db_maintenance(IN target_schema text, IN options text) ...

Compatibility

This command conforms to the SQL standard, with these PostgreSQL extensions:

See Also

drop_publication

DROP PUBLICATION

DROP PUBLICATION

7

SQL - Language Statements

DROP PUBLICATION

remove a publication

DROP PUBLICATION [ IF EXISTS ]

name

[, ...] [ CASCADE | RESTRICT ]

Description

DROP PUBLICATION removes an existing publication from the database.

A publication can only be dropped by its owner or a superuser.

Parameters

IF EXISTS
Do not throw an error if the publication does not exist. A notice is issued in this case.

<name>
The name of an existing publication.

CASCADE; RESTRICT
These key words do not have any effect, since there are no dependencies on publications.

Examples

Drop a publication:

DROP PUBLICATION mypublication;

Compatibility

DROP PUBLICATION is a PostgreSQL extension.

See Also

drop_role

DROP ROLE

DROP ROLE

7

SQL - Language Statements

DROP ROLE

remove a database role

DROP ROLE [ IF EXISTS ]

name

[, ...]

Description

DROP ROLE removes the specified role(s). To drop a superuser role, you must be a superuser yourself; to drop non-superuser roles, you must have CREATEROLE privilege and have been granted ADMIN OPTION on the role.

A role cannot be removed if it is still referenced in any database of the cluster; an error will be raised if so. Before dropping the role, you must drop all the objects it owns (or reassign their ownership) and revoke any privileges the role has been granted on other objects. The REASSIGN OWNED and DROP OWNED commands can be useful for this purpose; see ??? for more discussion.

However, it is not necessary to remove role memberships involving the role; DROP ROLE automatically revokes any memberships of the target role in other roles, and of other roles in the target role. The other roles are not dropped nor otherwise affected.

Parameters

IF EXISTS
Do not throw an error if the role does not exist. A notice is issued in this case.

<name>
The name of the role to remove.

Notes

PostgreSQL includes a program ??? that has the same functionality as this command (in fact, it calls this command) but can be run from the command shell.

Examples

To drop a role:

DROP ROLE jonathan;

Compatibility

The SQL standard defines DROP ROLE, but it allows only one role to be dropped at a time, and it specifies different privilege requirements than PostgreSQL uses.

See Also

drop_routine

DROP ROUTINE

DROP ROUTINE

7

SQL - Language Statements

DROP ROUTINE

remove a routine

DROP ROUTINE [ IF EXISTS ]

name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] [, ...] [ CASCADE | RESTRICT ]

Description

DROP ROUTINE removes the definition of one or more existing routines. The term “routine” includes aggregate functions, normal functions, and procedures. See under ???, ???, and ??? for the description of the parameters, more examples, and further details.

Notes

The lookup rules used by DROP ROUTINE are fundamentally the same as for DROP PROCEDURE; in particular, DROP ROUTINE shares that command's behavior of considering an argument list that has no <argmode> markers to be possibly using the SQL standard's definition that OUT arguments are included in the list. (DROP AGGREGATE and DROP FUNCTION do not do that.)

In some cases where the same name is shared by routines of different kinds, it is possible for DROP ROUTINE to fail with an ambiguity error when a more specific command (DROP FUNCTION, etc.) would work. Specifying the argument type list more carefully will also resolve such problems.

These lookup rules are also used by other commands that act on existing routines, such as ALTER ROUTINE and COMMENT ON ROUTINE.

Examples

To drop the routine foo for type integer:

DROP ROUTINE foo(integer);

This command will work independent of whether foo is an aggregate, function, or procedure.

Compatibility

This command conforms to the SQL standard, with these PostgreSQL extensions:

See Also

Note that there is no CREATE ROUTINE command.

drop_rule

DROP RULE

DROP RULE

7

SQL - Language Statements

DROP RULE

remove a rewrite rule

DROP RULE [ IF EXISTS ]

name

ON

table_name

[ CASCADE | RESTRICT ]

Description

DROP RULE drops a rewrite rule.

Parameters

IF EXISTS
Do not throw an error if the rule does not exist. A notice is issued in this case.

<name>
The name of the rule to drop.

<table_name>
The name (optionally schema-qualified) of the table or view that the rule applies to.

CASCADE
Automatically drop objects that depend on the rule, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the rule if any objects depend on it. This is the default.

Examples

To drop the rewrite rule newrule:

DROP RULE newrule ON mytable;

Compatibility

DROP RULE is a PostgreSQL language extension, as is the entire query rewrite system.

See Also

drop_schema

DROP SCHEMA

DROP SCHEMA

7

SQL - Language Statements

DROP SCHEMA

remove a schema

DROP SCHEMA [ IF EXISTS ]

name

[, ...] [ CASCADE | RESTRICT ]

Description

DROP SCHEMA removes schemas from the database.

A schema can only be dropped by its owner or a superuser. Note that the owner can drop the schema (and thereby all contained objects) even if they do not own some of the objects within the schema.

Parameters

IF EXISTS
Do not throw an error if the schema does not exist. A notice is issued in this case.

<name>
The name of a schema.

CASCADE
Automatically drop objects (tables, functions, etc.) that are contained in the schema, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the schema if it contains any objects. This is the default.

Notes

Using the CASCADE option might make the command remove objects in other schemas besides the one(s) named.

Examples

To remove schema mystuff from the database, along with everything it contains:

DROP SCHEMA mystuff CASCADE;

Compatibility

DROP SCHEMA is fully conforming with the SQL standard, except that the standard only allows one schema to be dropped per command, and apart from the IF EXISTS option, which is a PostgreSQL extension.

See Also

drop_sequence

DROP SEQUENCE

DROP SEQUENCE

7

SQL - Language Statements

DROP SEQUENCE

remove a sequence

DROP SEQUENCE [ IF EXISTS ]

name

[, ...] [ CASCADE | RESTRICT ]

Description

DROP SEQUENCE removes sequence number generators. A sequence can only be dropped by its owner or a superuser.

Parameters

IF EXISTS
Do not throw an error if the sequence does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of a sequence.

CASCADE
Automatically drop objects that depend on the sequence, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the sequence if any objects depend on it. This is the default.

Examples

To remove the sequence serial:

DROP SEQUENCE serial;

Compatibility

DROP SEQUENCE conforms to the SQL standard, except that the standard only allows one sequence to be dropped per command, and apart from the IF EXISTS option, which is a PostgreSQL extension.

See Also

drop_server

DROP SERVER

DROP SERVER

7

SQL - Language Statements

DROP SERVER

remove a foreign server descriptor

DROP SERVER [ IF EXISTS ]

name

[, ...] [ CASCADE | RESTRICT ]

Description

DROP SERVER removes an existing foreign server descriptor. To execute this command, the current user must be the owner of the server.

Parameters

IF EXISTS
Do not throw an error if the server does not exist. A notice is issued in this case.

<name>
The name of an existing server.

CASCADE
Automatically drop objects that depend on the server (such as user mappings), and in turn all objects that depend on those objects (see ???). However, a subscription that uses the server is never dropped automatically; it must be dropped with DROP SUBSCRIPTION before the server can be dropped.

RESTRICT
Refuse to drop the server if any objects depend on it. This is the default.

Examples

Drop a server foo if it exists:

DROP SERVER IF EXISTS foo;

Compatibility

DROP SERVER conforms to ISO/IEC 9075-9 (SQL/MED). The IF EXISTS clause is a PostgreSQL extension.

See Also

drop_statistics

DROP STATISTICS

DROP STATISTICS

7

SQL - Language Statements

DROP STATISTICS

remove extended statistics

DROP STATISTICS [ IF EXISTS ]

name

[, ...] [ CASCADE | RESTRICT ]

Description

DROP STATISTICS removes statistics object(s) from the database. Only the statistics object's owner, the schema owner, or a superuser can drop a statistics object.

Parameters

IF EXISTS
Do not throw an error if the statistics object does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of the statistics object to drop.

CASCADE; RESTRICT
These key words do not have any effect, since there are no dependencies on statistics.

Examples

To destroy two statistics objects in different schemas, without failing if they don't exist:

DROP STATISTICS IF EXISTS
    accounting.users_uid_creation,
    public.grants_user_role;

Compatibility

There is no DROP STATISTICS command in the SQL standard.

See Also

drop_subscription

DROP SUBSCRIPTION

DROP SUBSCRIPTION

7

SQL - Language Statements

DROP SUBSCRIPTION

remove a subscription

DROP SUBSCRIPTION [ IF EXISTS ]

name

[ CASCADE | RESTRICT ]

Description

DROP SUBSCRIPTION removes a subscription from the database cluster.

To execute this command the user must be the owner of the subscription.

DROP SUBSCRIPTION cannot be executed inside a transaction block if the subscription is associated with a replication slot. (You can use ALTER SUBSCRIPTION to unset the slot.)

Parameters

IF EXISTS
Do not throw an error if the subscription does not exist. A notice is issued in this case.

<name>
The name of a subscription to be dropped.

CASCADE; RESTRICT
These key words do not have any effect, since there are no dependencies on subscriptions.

Notes

When dropping a subscription that is associated with a replication slot on the remote host (the normal state), DROP SUBSCRIPTION will connect to the remote host and try to drop the replication slot (and any remaining table synchronization slots) as part of its operation. This is necessary so that the resources allocated for the subscription on the remote host are released. If this fails, either because the remote host is not reachable or because the remote replication slot cannot be dropped or does not exist or never existed, the DROP SUBSCRIPTION command will fail. To proceed in this situation, first disable the subscription by executing ALTER SUBSCRIPTION ... DISABLE, and then disassociate it from the replication slot by executing ALTER SUBSCRIPTION ... SET (slot_name = NONE). After that, DROP SUBSCRIPTION will not attempt to drop the subscription's own replication slot. It may still connect to the publisher to drop internally-created table synchronization slots if some table synchronization is left unfinished; if the publisher is unreachable, those slots (and the main slot, if it still exists) must be dropped manually. Otherwise it/they will continue to reserve WAL and might eventually cause the disk to fill up. See also ???.

If a subscription is associated with a replication slot, then DROP SUBSCRIPTION cannot be executed inside a transaction block.

If a conflict log table exists for the subscription (that is, when conflict_log_destination is set to table or all), DROP SUBSCRIPTION automatically drops the associated conflict log table.

Examples

Drop a subscription:

DROP SUBSCRIPTION mysub;

Compatibility

DROP SUBSCRIPTION is a PostgreSQL extension.

See Also

drop_table

DROP TABLE

DROP TABLE

7

SQL - Language Statements

DROP TABLE

remove a table

DROP TABLE [ IF EXISTS ]

name

[, ...] [ CASCADE | RESTRICT ]

Description

DROP TABLE removes tables from the database. Only the table owner, the schema owner, and superuser can drop a table. To empty a table of rows without destroying the table, use DELETE or TRUNCATE.

DROP TABLE always removes any indexes, rules, triggers, and constraints that exist for the target table. However, to drop a table that is referenced by a view or a foreign-key constraint of another table, CASCADE must be specified. (CASCADE will remove a dependent view entirely, but in the foreign-key case it will only remove the foreign-key constraint, not the other table entirely.)

Parameters

IF EXISTS
Do not throw an error if the table does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of the table to drop.

CASCADE
Automatically drop objects that depend on the table (such as views), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the table if any objects depend on it. This is the default.

Examples

To destroy two tables, films and distributors:

DROP TABLE films, distributors;

Compatibility

This command conforms to the SQL standard, except that the standard only allows one table to be dropped per command, and apart from the IF EXISTS option, which is a PostgreSQL extension.

See Also

drop_tablespace

DROP TABLESPACE

DROP TABLESPACE

7

SQL - Language Statements

DROP TABLESPACE

remove a tablespace

DROP TABLESPACE [ IF EXISTS ]

name

Description

DROP TABLESPACE removes a tablespace from the system.

A tablespace can only be dropped by its owner or a superuser. The tablespace must be empty of all database objects before it can be dropped. It is possible that objects in other databases might still reside in the tablespace even if no objects in the current database are using the tablespace. Also, if the tablespace is listed in the ??? setting of any active session, the DROP might fail due to temporary files residing in the tablespace.

Parameters

IF EXISTS
Do not throw an error if the tablespace does not exist. A notice is issued in this case.

<name>
The name of a tablespace.

Notes

DROP TABLESPACE cannot be executed inside a transaction block.

Examples

To remove tablespace mystuff from the system:

DROP TABLESPACE mystuff;

Compatibility

DROP TABLESPACE is a PostgreSQL extension.

See Also

drop_transform

DROP TRANSFORM

DROP TRANSFORM

7

SQL - Language Statements

DROP TRANSFORM

remove a transform

DROP TRANSFORM [ IF EXISTS ] FOR

type_name

LANGUAGE

lang_name

[ CASCADE | RESTRICT ]

Description

DROP TRANSFORM removes a previously defined transform.

To be able to drop a transform, you must own the type and the language. These are the same privileges that are required to create a transform.

Parameters

IF EXISTS
Do not throw an error if the transform does not exist. A notice is issued in this case.

<type_name>
The name of the data type of the transform.

<lang_name>
The name of the language of the transform.

CASCADE
Automatically drop objects that depend on the transform, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the transform if any objects depend on it. This is the default.

Examples

To drop the transform for type hstore and language plpython3u:

DROP TRANSFORM FOR hstore LANGUAGE plpython3u;

Compatibility

This form of DROP TRANSFORM is a PostgreSQL extension. See ??? for details.

See Also

drop_trigger

DROP TRIGGER

DROP TRIGGER

7

SQL - Language Statements

DROP TRIGGER

remove a trigger

DROP TRIGGER [ IF EXISTS ]

name

ON

table_name

[ CASCADE | RESTRICT ]

Description

DROP TRIGGER removes an existing trigger definition. To execute this command, the current user must be the owner of the table for which the trigger is defined.

Parameters

IF EXISTS
Do not throw an error if the trigger does not exist. A notice is issued in this case.

<name>
The name of the trigger to remove.

<table_name>
The name (optionally schema-qualified) of the table for which the trigger is defined.

CASCADE
Automatically drop objects that depend on the trigger, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the trigger if any objects depend on it. This is the default.

Examples

Destroy the trigger if_dist_exists on the table films:

DROP TRIGGER if_dist_exists ON films;

Compatibility

The DROP TRIGGER statement in PostgreSQL is incompatible with the SQL standard. In the SQL standard, trigger names are not local to tables, so the command is simply DROP TRIGGER name.

See Also

drop_tsconfig

DROP TEXT SEARCH CONFIGURATION

DROP TEXT SEARCH CONFIGURATION

7

SQL - Language Statements

DROP TEXT SEARCH CONFIGURATION

remove a text search configuration

DROP TEXT SEARCH CONFIGURATION [ IF EXISTS ]

name

[ CASCADE | RESTRICT ]

Description

DROP TEXT SEARCH CONFIGURATION drops an existing text search configuration. To execute this command you must be the owner of the configuration.

Parameters

IF EXISTS
Do not throw an error if the text search configuration does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of an existing text search configuration.

CASCADE
Automatically drop objects that depend on the text search configuration, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the text search configuration if any objects depend on it. This is the default.

Examples

Remove the text search configuration my_english:

DROP TEXT SEARCH CONFIGURATION my_english;

This command will not succeed if there are any existing indexes that reference the configuration in to_tsvector calls. Add CASCADE to drop such indexes along with the text search configuration.

Compatibility

There is no DROP TEXT SEARCH CONFIGURATION statement in the SQL standard.

See Also

drop_tsdictionary

DROP TEXT SEARCH DICTIONARY

DROP TEXT SEARCH DICTIONARY

7

SQL - Language Statements

DROP TEXT SEARCH DICTIONARY

remove a text search dictionary

DROP TEXT SEARCH DICTIONARY [ IF EXISTS ]

name

[ CASCADE | RESTRICT ]

Description

DROP TEXT SEARCH DICTIONARY drops an existing text search dictionary. To execute this command you must be the owner of the dictionary.

Parameters

IF EXISTS
Do not throw an error if the text search dictionary does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of an existing text search dictionary.

CASCADE
Automatically drop objects that depend on the text search dictionary, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the text search dictionary if any objects depend on it. This is the default.

Examples

Remove the text search dictionary english:

DROP TEXT SEARCH DICTIONARY english;

This command will not succeed if there are any existing text search configurations that use the dictionary. Add CASCADE to drop such configurations along with the dictionary.

Compatibility

There is no DROP TEXT SEARCH DICTIONARY statement in the SQL standard.

See Also

drop_tsparser

DROP TEXT SEARCH PARSER

DROP TEXT SEARCH PARSER

7

SQL - Language Statements

DROP TEXT SEARCH PARSER

remove a text search parser

DROP TEXT SEARCH PARSER [ IF EXISTS ]

name

[ CASCADE | RESTRICT ]

Description

DROP TEXT SEARCH PARSER drops an existing text search parser. You must be a superuser to use this command.

Parameters

IF EXISTS
Do not throw an error if the text search parser does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of an existing text search parser.

CASCADE
Automatically drop objects that depend on the text search parser, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the text search parser if any objects depend on it. This is the default.

Examples

Remove the text search parser my_parser:

DROP TEXT SEARCH PARSER my_parser;

This command will not succeed if there are any existing text search configurations that use the parser. Add CASCADE to drop such configurations along with the parser.

Compatibility

There is no DROP TEXT SEARCH PARSER statement in the SQL standard.

See Also

drop_tstemplate

DROP TEXT SEARCH TEMPLATE

DROP TEXT SEARCH TEMPLATE

7

SQL - Language Statements

DROP TEXT SEARCH TEMPLATE

remove a text search template

DROP TEXT SEARCH TEMPLATE [ IF EXISTS ]

name

[ CASCADE | RESTRICT ]

Description

DROP TEXT SEARCH TEMPLATE drops an existing text search template. You must be a superuser to use this command.

Parameters

IF EXISTS
Do not throw an error if the text search template does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of an existing text search template.

CASCADE
Automatically drop objects that depend on the text search template, and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the text search template if any objects depend on it. This is the default.

Examples

Remove the text search template thesaurus:

DROP TEXT SEARCH TEMPLATE thesaurus;

This command will not succeed if there are any existing text search dictionaries that use the template. Add CASCADE to drop such dictionaries along with the template.

Compatibility

There is no DROP TEXT SEARCH TEMPLATE statement in the SQL standard.

See Also

drop_type

DROP TYPE

DROP TYPE

7

SQL - Language Statements

DROP TYPE

remove a data type

DROP TYPE [ IF EXISTS ]

name

[, ...] [ CASCADE | RESTRICT ]

Description

DROP TYPE removes a user-defined data type. Only the owner of a type can remove it.

Parameters

IF EXISTS
Do not throw an error if the type does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of the data type to remove.

CASCADE
Automatically drop objects that depend on the type (such as table columns, functions, and operators), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the type if any objects depend on it. This is the default.

Examples

To remove the data type box:

DROP TYPE box;

Compatibility

This command is similar to the corresponding command in the SQL standard, apart from the IF EXISTS option, which is a PostgreSQL extension. But note that much of the CREATE TYPE command and the data type extension mechanisms in PostgreSQL differ from the SQL standard.

See Also

drop_user

DROP USER

DROP USER

7

SQL - Language Statements

DROP USER

remove a database role

DROP USER [ IF EXISTS ]

name

[, ...]

Description

DROP USER is simply an alternate spelling of DROP ROLE.

Compatibility

The DROP USER statement is a PostgreSQL extension. The SQL standard leaves the definition of users to the implementation.

See Also

drop_user_mapping

DROP USER MAPPING

DROP USER MAPPING

7

SQL - Language Statements

DROP USER MAPPING

remove a user mapping for a foreign server

DROP USER MAPPING [ IF EXISTS ] FOR {

user_name

| USER | CURRENT_ROLE | CURRENT_USER | PUBLIC } SERVER

server_name

Description

DROP USER MAPPING removes an existing user mapping from foreign server.

The owner of a foreign server can drop user mappings for that server for any user. Also, a user can drop a user mapping for their own user name if USAGE privilege on the server has been granted to the user.

Parameters

IF EXISTS
Do not throw an error if the user mapping does not exist. A notice is issued in this case.

<user_name>
User name of the mapping. CURRENT_ROLE, CURRENT_USER, and USER match the name of the current user. PUBLIC is used to match all present and future user names in the system.

<server_name>
Server name of the user mapping.

Examples

Drop a user mapping bob, server foo if it exists:

DROP USER MAPPING IF EXISTS FOR bob SERVER foo;

Compatibility

DROP USER MAPPING conforms to ISO/IEC 9075-9 (SQL/MED). The IF EXISTS clause is a PostgreSQL extension.

See Also

drop_view

DROP VIEW

DROP VIEW

7

SQL - Language Statements

DROP VIEW

remove a view

DROP VIEW [ IF EXISTS ]

name

[, ...] [ CASCADE | RESTRICT ]

Description

DROP VIEW drops an existing view. To execute this command you must be the owner of the view.

Parameters

IF EXISTS
Do not throw an error if the view does not exist. A notice is issued in this case.

<name>
The name (optionally schema-qualified) of the view to remove.

CASCADE
Automatically drop objects that depend on the view (such as other views), and in turn all objects that depend on those objects (see ???).

RESTRICT
Refuse to drop the view if any objects depend on it. This is the default.

Examples

This command will remove the view called kinds:

DROP VIEW kinds;

Compatibility

This command conforms to the SQL standard, except that the standard only allows one view to be dropped per command, and apart from the IF EXISTS option, which is a PostgreSQL extension.

See Also

dropdb

dropdb

dropdb

1

Application

dropdb

remove a

PostgreSQL

database

dropdb

connection-option

option

dbname

Description

dropdb destroys an existing PostgreSQL database. The user who executes this command must be a database superuser or the owner of the database.

dropdb is a wrapper around the SQL command DROP DATABASE. There is no effective difference between dropping databases via this utility and via other methods for accessing the server.

Options

dropdb accepts the following command-line arguments:

<dbname>
Specifies the name of the database to be removed.

-e; --echo
Echo the commands that dropdb generates and sends to the server.

-f; --force
Attempt to terminate all existing connections to the target database before dropping it. See ??? for more information on this option.

-i; --interactive
Issues a verification prompt before doing anything destructive.

-V; --version
Print the dropdb version and exit.

--if-exists
Do not throw an error if the database does not exist. A notice is issued in this case.

-?; --help
Show help about dropdb command line arguments, and exit.

dropdb also accepts the following command-line arguments for connection parameters:

-h host; --host=host
Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for the Unix domain socket.

-p port; --port=port
Specifies the TCP port or local Unix domain socket file extension on which the server is listening for connections.

-U username; --username=username
User name to connect as.

-w; --no-password
Never issue a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password.

-W; --password
Force dropdb to prompt for a password before connecting to a database.

This option is never essential, since dropdb will automatically prompt for a password if the server demands password authentication. However, dropdb will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt.

--maintenance-db=dbname
Specifies the name of the database to connect to in order to drop the target database. If not specified, the postgres database will be used; if that does not exist (or is the database being dropped), template1 will be used. This can be a connection string. If so, connection string parameters will override any conflicting command line options.

Environment

PGHOST; PGPORT; PGUSER
Default connection parameters

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

This utility, like most other PostgreSQL utilities, also uses the environment variables supported by libpq (see ???).

Diagnostics

In case of difficulty, see ??? and ??? for discussions of potential problems and error messages. The database server must be running at the targeted host. Also, any default connection settings and environment variables used by the libpq front-end library will apply.

Examples

To destroy the database demo on the default database server:

$ dropdb demo

To destroy the database demo using the server on host eden, port 5000, with verification and a peek at the underlying command:

$ dropdb -p 5000 -h eden -i -e demo
Database "demo" will be permanently deleted.
Are you sure? (y/n) y
DROP DATABASE demo;

See Also

dropuser

dropuser

dropuser

1

Application

dropuser

remove a

PostgreSQL

user account

dropuser

connection-option

option

username

Description

dropuser removes an existing PostgreSQL user. Superusers can use this command to remove any role; otherwise, only non-superuser roles can be removed, and only by a user who possesses the CREATEROLE privilege and has been granted ADMIN OPTION on the target role.

dropuser is a wrapper around the SQL command DROP ROLE. There is no effective difference between dropping users via this utility and via other methods for accessing the server.

Options

dropuser accepts the following command-line arguments:

<username>
Specifies the name of the PostgreSQL user to be removed. You will be prompted for a name if none is specified on the command line and the -i/--interactive option is used.

-e; --echo
Echo the commands that dropuser generates and sends to the server.

-i; --interactive
Prompt for confirmation before actually removing the user, and prompt for the user name if none is specified on the command line.

-V; --version
Print the dropuser version and exit.

--if-exists
Do not throw an error if the user does not exist. A notice is issued in this case.

-?; --help
Show help about dropuser command line arguments, and exit.

dropuser also accepts the following command-line arguments for connection parameters:

-h host; --host=host
Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for the Unix domain socket.

-p port; --port=port
Specifies the TCP port or local Unix domain socket file extension on which the server is listening for connections.

-U username; --username=username
User name to connect as (not the user name to drop).

-w; --no-password
Never issue a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password.

-W; --password
Force dropuser to prompt for a password before connecting to a database.

This option is never essential, since dropuser will automatically prompt for a password if the server demands password authentication. However, dropuser will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt.

Environment

PGHOST; PGPORT; PGUSER
Default connection parameters

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

This utility, like most other PostgreSQL utilities, also uses the environment variables supported by libpq (see ???).

Diagnostics

In case of difficulty, see ??? and ??? for discussions of potential problems and error messages. The database server must be running at the targeted host. Also, any default connection settings and environment variables used by the libpq front-end library will apply.

Examples

To remove user joe from the default database server:

$ dropuser joe

To remove user joe using the server on host eden, port 5000, with verification and a peek at the underlying command:

$ dropuser -p 5000 -h eden -i -e joe
Role "joe" will be permanently removed.
Are you sure? (y/n) y
DROP ROLE joe;

See Also

ecpg-ref

ecpg

ecpg

1

Application

ecpg

embedded SQL C preprocessor

ecpg

option

file

Description

ecpg is the embedded SQL preprocessor for C programs. It converts C programs with embedded SQL statements to normal C code by replacing the SQL invocations with special function calls. The output files can then be processed with any C compiler tool chain.

ecpg will convert each input file given on the command line to the corresponding C output file. If an input file name does not have any extension, .pgc is assumed. The file's extension will be replaced by .c to construct the output file name. But the output file name can be overridden using the -o option.

If an input file name is just -, ecpg reads the program from standard input (and writes to standard output, unless that is overridden with -o).

This reference page does not describe the embedded SQL language. See ??? for more information on that topic.

Options

ecpg accepts the following command-line arguments:

-c
Automatically generate certain C code from SQL code. Currently, this works for EXEC SQL TYPE.

-C mode
Set a compatibility mode. <mode> can be INFORMIX, INFORMIX_SE, or ORACLE.

-D symbol[=value]
Define a preprocessor symbol, equivalently to the EXEC SQL DEFINE directive. If no <value> is specified, the symbol is defined with the value 1.

-h
Process header files. When this option is specified, the output file extension becomes .h not .c, and the default input file extension is .pgh not .pgc. Also, the -c option is forced on.

-i
Parse system include files as well.

-I directory
Specify an additional include path, used to find files included via EXEC SQL INCLUDE. Defaults are . (current directory), /usr/local/include, the PostgreSQL include directory which is defined at compile time (default: /usr/local/pgsql/include), and /usr/include, in that order.

-o filename
Specifies that ecpg should write all its output to the given <filename>. Write -o - to send all output to standard output.

-r option
Selects run-time behavior. <Option> can be one of the following:

no_indicator
Do not use indicators but instead use special values to represent null values. Historically there have been databases using this approach.

prepare
Prepare all statements before using them. Libecpg will keep a cache of prepared statements and reuse a statement if it gets executed again. If the cache runs full, libecpg will free the least used statement.

questionmarks
Allow question mark as placeholder for compatibility reasons. This used to be the default long ago.

-t
Turn on autocommit of transactions. In this mode, each SQL command is automatically committed unless it is inside an explicit transaction block. In the default mode, commands are committed only when EXEC SQL COMMIT is issued.

-v
Print additional information including the version and the "include" path.

--version
Print the ecpg version and exit.

-?; --help
Show help about ecpg command line arguments, and exit.

Notes

When compiling the preprocessed C code files, the compiler needs to be able to find the ECPG header files in the PostgreSQL include directory. Therefore, you might have to use the -I option when invoking the compiler (e.g., -I/usr/local/pgsql/include).

Programs using C code with embedded SQL have to be linked against the libecpg library, for example using the linker options -L/usr/local/pgsql/lib -lecpg.

The value of either of these directories that is appropriate for the installation can be found out using ???.

Examples

If you have an embedded SQL C source file named prog1.pgc, you can create an executable program using the following sequence of commands:

ecpg prog1.pgc
cc -I/usr/local/pgsql/include -c prog1.c
cc -o prog1 prog1.o -L/usr/local/pgsql/lib -lecpg

end

END

END

7

SQL - Language Statements

END

commit the current transaction

END [ WORK | TRANSACTION ] [ AND [ NO ] CHAIN ]

Description

END commits the current transaction. All changes made by the transaction become visible to others and are guaranteed to be durable if a crash occurs. This command is a PostgreSQL extension that is equivalent to COMMIT.

Parameters

WORK; TRANSACTION
Optional key words. They have no effect.

AND CHAIN
If AND CHAIN is specified, a new transaction is immediately started with the same transaction characteristics (see ???) as the just finished one. Otherwise, no new transaction is started.

Notes

Use ROLLBACK to abort a transaction.

Issuing END when not inside a transaction does no harm, but it will provoke a warning message.

Examples

To commit the current transaction and make all changes permanent:

END;

Compatibility

END is a PostgreSQL extension that provides functionality equivalent to COMMIT, which is specified in the SQL standard.

See Also

execute

EXECUTE

prepared statements

executing

EXECUTE

7

SQL - Language Statements

EXECUTE

execute a prepared statement

EXECUTE

name

[ (

parameter

[, ...] ) ]

Description

EXECUTE is used to execute a previously prepared statement. Since prepared statements only exist for the duration of a session, the prepared statement must have been created by a PREPARE statement executed earlier in the current session.

If the PREPARE statement that created the statement specified some parameters, a compatible set of parameters must be passed to the EXECUTE statement, or else an error is raised. Note that (unlike functions) prepared statements are not overloaded based on the type or number of their parameters; the name of a prepared statement must be unique within a database session.

For more information on the creation and usage of prepared statements, see ???.

Parameters

<name>
The name of the prepared statement to execute.

<parameter>
The actual value of a parameter to the prepared statement. This must be an expression yielding a value that is compatible with the data type of this parameter, as was determined when the prepared statement was created.

Outputs

The command tag returned by EXECUTE is that of the prepared statement, and not EXECUTE.

Examples

Examples are given in ??? in the ??? documentation.

Compatibility

The SQL standard includes an EXECUTE statement, but it is only for use in embedded SQL. This version of the EXECUTE statement also uses a somewhat different syntax.

See Also

fetch

FETCH

cursor

FETCH

FETCH

7

SQL - Language Statements

FETCH

retrieve rows from a query using a cursor

FETCH [

direction

] [ FROM | IN ]

cursor_name

where

direction

can be one of:

NEXT PRIOR FIRST LAST ABSOLUTE

count

RELATIVE

count

count

ALL FORWARD FORWARD

count

FORWARD ALL BACKWARD BACKWARD

count

BACKWARD ALL

Description

FETCH retrieves rows using a previously-created cursor.

A cursor has an associated position, which is used by FETCH. The cursor position can be before the first row of the query result, on any particular row of the result, or after the last row of the result. When created, a cursor is positioned before the first row. After fetching some rows, the cursor is positioned on the row most recently retrieved. If FETCH runs off the end of the available rows then the cursor is left positioned after the last row, or before the first row if fetching backward. FETCH ALL or FETCH BACKWARD ALL will always leave the cursor positioned after the last row or before the first row.

The forms NEXT, PRIOR, FIRST, LAST, ABSOLUTE, RELATIVE fetch a single row after moving the cursor appropriately. If there is no such row, an empty result is returned, and the cursor is left positioned before the first row or after the last row as appropriate.

The forms using FORWARD and BACKWARD retrieve the indicated number of rows moving in the forward or backward direction, leaving the cursor positioned on the last-returned row (or after/before all rows, if the <count> exceeds the number of rows available).

RELATIVE 0, FORWARD 0, and BACKWARD 0 all request fetching the current row without moving the cursor, that is, re-fetching the most recently fetched row. This will succeed unless the cursor is positioned before the first row or after the last row; in which case, no row is returned.

[!NOTE] This page describes usage of cursors at the SQL command level. If you are trying to use cursors inside a PL/pgSQL function, the rules are different — see ???.

Parameters

<direction>
<direction> defines the fetch direction and number of rows to fetch. It can be one of the following:

NEXT
Fetch the next row. This is the default if <direction> is omitted.

PRIOR
Fetch the prior row.

FIRST
Fetch the first row of the query (same as ABSOLUTE 1).

LAST
Fetch the last row of the query (same as ABSOLUTE -1).

ABSOLUTE count
Fetch the <count>'th row of the query, or the abs(count)'th row from the end if <count> is negative. Position before first row or after last row if <count> is out of range; in particular, ABSOLUTE 0 positions before the first row.

RELATIVE count
Fetch the <count>'th succeeding row, or the abs(count)'th prior row if <count> is negative. RELATIVE 0 re-fetches the current row, if any.

<count>
Fetch the next <count> rows (same as FORWARD count).

ALL
Fetch all remaining rows (same as FORWARD ALL).

FORWARD
Fetch the next row (same as NEXT).

FORWARD count
Fetch the next <count> rows. FORWARD 0 re-fetches the current row.

FORWARD ALL
Fetch all remaining rows.

BACKWARD
Fetch the prior row (same as PRIOR).

BACKWARD count
Fetch the prior <count> rows (scanning backwards). BACKWARD 0 re-fetches the current row.

BACKWARD ALL
Fetch all prior rows (scanning backwards).

<count>
<count> is a possibly-signed integer constant, determining the location or number of rows to fetch. For FORWARD and BACKWARD cases, specifying a negative <count> is equivalent to changing the sense of FORWARD and BACKWARD.

<cursor_name>
An open cursor's name.

Outputs

On successful completion, a FETCH command returns a command tag of the form

FETCH count

The <count> is the number of rows fetched (possibly zero). Note that in psql, the command tag will not actually be displayed, since psql displays the fetched rows instead.

Notes

The cursor should be declared with the SCROLL option if one intends to use any variants of FETCH other than FETCH NEXT or FETCH FORWARD with a positive count. For simple queries PostgreSQL will allow backwards fetch from cursors not declared with SCROLL, but this behavior is best not relied on. If the cursor is declared with NO SCROLL, no backward fetches are allowed.

ABSOLUTE fetches are not any faster than navigating to the desired row with a relative move: the underlying implementation must traverse all the intermediate rows anyway. Negative absolute fetches are even worse: the query must be read to the end to find the last row, and then traversed backward from there. However, rewinding to the start of the query (as with FETCH ABSOLUTE 0) is fast.

DECLARE is used to define a cursor. Use MOVE to change cursor position without retrieving data.

Examples

The following example traverses a table using a cursor:

BEGIN WORK;

-- Set up a cursor:
DECLARE liahona SCROLL CURSOR FOR SELECT * FROM films;

-- Fetch the first 5 rows in the cursor liahona:
FETCH FORWARD 5 FROM liahona;

 code  |          title          | did | date_prod  |   kind   |  len
-------+-------------------------+-----+------------+----------+-------
 BL101 | The Third Man           | 101 | 1949-12-23 | Drama    | 01:44
 BL102 | The African Queen       | 101 | 1951-08-11 | Romantic | 01:43
 JL201 | Une Femme est une Femme | 102 | 1961-03-12 | Romantic | 01:25
 P_301 | Vertigo                 | 103 | 1958-11-14 | Action   | 02:08
 P_302 | Becket                  | 103 | 1964-02-03 | Drama    | 02:28

-- Fetch the previous row:
FETCH PRIOR FROM liahona;

 code  |  title  | did | date_prod  |  kind  |  len
-------+---------+-----+------------+--------+-------
 P_301 | Vertigo | 103 | 1958-11-14 | Action | 02:08

-- Close the cursor and end the transaction:
CLOSE liahona;
COMMIT WORK;

Compatibility

The SQL standard defines FETCH for use in embedded SQL only. The variant of FETCH described here returns the data as if it were a SELECT result rather than placing it in host variables. Other than this point, FETCH is fully upward-compatible with the SQL standard.

The FETCH forms involving FORWARD and BACKWARD, as well as the forms FETCH count and FETCH ALL, in which FORWARD is implicit, are PostgreSQL extensions.

The SQL standard allows only FROM preceding the cursor name; the option to use IN, or to leave them out altogether, is an extension.

See Also

grant

GRANT

GRANT

7

SQL - Language Statements

GRANT

define access privileges

GRANT { { SELECT | INSERT | UPDATE | DELETE | TRUNCATE | REFERENCES | TRIGGER | MAINTAIN } [, ...] | ALL [ PRIVILEGES ] } ON { [ TABLE ]

table_name

[, ...] | ALL TABLES IN SCHEMA

schema_name

[, ...] } TO

role_specification

[, ...] [ WITH GRANT OPTION ] [ GRANTED BY

role_specification

] GRANT { { SELECT | INSERT | UPDATE | REFERENCES } (

column_name

[, ...] ) [, ...] | ALL [ PRIVILEGES ] (

column_name

[, ...] ) } ON [ TABLE ]

table_name

[, ...] TO

role_specification

[, ...] [ WITH GRANT OPTION ] [ GRANTED BY

role_specification

] GRANT { { USAGE | SELECT | UPDATE } [, ...] | ALL [ PRIVILEGES ] } ON { SEQUENCE

sequence_name

[, ...] | ALL SEQUENCES IN SCHEMA

schema_name

[, ...] } TO

role_specification

[, ...] [ WITH GRANT OPTION ] [ GRANTED BY

role_specification

] GRANT { { CREATE | CONNECT | TEMPORARY | TEMP } [, ...] | ALL [ PRIVILEGES ] } ON DATABASE

database_name

[, ...] TO

role_specification

[, ...] [ WITH GRANT OPTION ] [ GRANTED BY

role_specification

] GRANT { USAGE | ALL [ PRIVILEGES ] } ON DOMAIN

domain_name

[, ...] TO

role_specification

[, ...] [ WITH GRANT OPTION ] [ GRANTED BY

role_specification

] GRANT { USAGE | ALL [ PRIVILEGES ] } ON FOREIGN DATA WRAPPER

fdw_name

[, ...] TO

role_specification

[, ...] [ WITH GRANT OPTION ] [ GRANTED BY

role_specification

] GRANT { USAGE | ALL [ PRIVILEGES ] } ON FOREIGN SERVER

server_name

[, ...] TO

role_specification

[, ...] [ WITH GRANT OPTION ] [ GRANTED BY

role_specification

] GRANT { EXECUTE | ALL [ PRIVILEGES ] } ON { { FUNCTION | PROCEDURE | ROUTINE }

routine_name

[ ( [ [

argmode

] [

arg_name

]

arg_type

[, ...] ] ) ] [, ...] | ALL { FUNCTIONS | PROCEDURES | ROUTINES } IN SCHEMA

schema_name

[, ...] } TO

role_specification

[, ...] [ WITH GRANT OPTION ] [ GRANTED BY

role_specification

] GRANT { USAGE | ALL [ PRIVILEGES ] } ON LANGUAGE

lang_name

[, ...] TO

role_specification

[, ...] [ WITH GRANT OPTION ] [ GRANTED BY

role_specification

] GRANT { { SELECT | UPDATE } [, ...] | ALL [ PRIVILEGES ] } ON LARGE OBJECT

loid

[, ...] TO

role_specification

[, ...] [ WITH GRANT OPTION ] [ GRANTED BY

role_specification

] GRANT { { SET | ALTER SYSTEM } [, ... ] | ALL [ PRIVILEGES ] } ON PARAMETER

configuration_parameter

[, ...] TO

role_specification

[, ...] [ WITH GRANT OPTION ] [ GRANTED BY

role_specification

] GRANT { { CREATE | USAGE } [, ...] | ALL [ PRIVILEGES ] } ON SCHEMA

schema_name

[, ...] TO

role_specification

[, ...] [ WITH GRANT OPTION ] [ GRANTED BY

role_specification

] GRANT { CREATE | ALL [ PRIVILEGES ] } ON TABLESPACE

tablespace_name

[, ...] TO

role_specification

[, ...] [ WITH GRANT OPTION ] [ GRANTED BY

role_specification

] GRANT { USAGE | ALL [ PRIVILEGES ] } ON TYPE

type_name

[, ...] TO

role_specification

[, ...] [ WITH GRANT OPTION ] [ GRANTED BY

role_specification

] GRANT

role_name

[, ...] TO

role_specification

[, ...] [ WITH { ADMIN | INHERIT | SET } { OPTION | TRUE | FALSE } ] [ GRANTED BY

role_specification

]

where

role_specification

can be:

[ GROUP ]

role_name

| PUBLIC | CURRENT_ROLE | CURRENT_USER | SESSION_USER

Description

The GRANT command has two basic variants: one that grants privileges on a database object (table, column, view, foreign table, sequence, database, foreign-data wrapper, foreign server, function, procedure, procedural language, large object, configuration parameter, schema, tablespace, or type), and one that grants membership in a role. These variants are similar in many ways, but they are different enough to be described separately.

GRANT on Database Objects

This variant of the GRANT command gives specific privileges on a database object to one or more roles. These privileges are added to those already granted, if any.

The key word PUBLIC indicates that the privileges are to be granted to all roles, including those that might be created later. PUBLIC can be thought of as an implicitly defined group that always includes all roles. Any particular role will have the sum of privileges granted directly to it, privileges granted to any role it is presently a member of, and privileges granted to PUBLIC.

If WITH GRANT OPTION is specified, the recipient of the privilege can in turn grant it to others. Without a grant option, the recipient cannot do that. Grant options cannot be granted to PUBLIC.

If GRANTED BY is specified, the grant is recorded as having been done by the specified role. A role can only attribute a grant to another role if it inherits the privileges of that role.

There is no need to grant privileges to the owner of an object (usually the user that created it), as the owner has all privileges by default. (The owner could, however, choose to revoke some of their own privileges for safety.)

The right to drop an object, or to alter its definition in any way, is not treated as a grantable privilege; it is inherent in the owner, and cannot be granted or revoked. (However, a similar effect can be obtained by granting or revoking membership in the role that owns the object; see below.) The owner implicitly has all grant options for the object, too.

The possible privileges are:

SELECT; INSERT; UPDATE; DELETE; TRUNCATE; REFERENCES; TRIGGER; CREATE; CONNECT; TEMPORARY; EXECUTE; USAGE; SET; ALTER SYSTEM; MAINTAIN
Specific types of privileges, as defined in ???.

TEMP
Alternative spelling for TEMPORARY.

ALL PRIVILEGES
Grant all of the privileges available for the object's type. The PRIVILEGES key word is optional in PostgreSQL, though it is required by strict SQL.

The FUNCTION syntax works for plain functions, aggregate functions, and window functions, but not for procedures; use PROCEDURE for those. Alternatively, use ROUTINE to refer to a function, aggregate function, window function, or procedure regardless of its precise type.

There is also an option to grant privileges on all objects of the same type within one or more schemas. This functionality is currently supported only for tables, sequences, functions, and procedures. ALL TABLES also affects views and foreign tables, just like the specific-object GRANT command. ALL FUNCTIONS also affects aggregate and window functions, but not procedures, again just like the specific-object GRANT command. Use ALL ROUTINES to include procedures.

GRANT on Roles

This variant of the GRANT command grants membership in a role to one or more other roles, and the modification of membership options SET, INHERIT, and ADMIN; see ??? for details. Membership in a role is significant because it potentially allows access to the privileges granted to a role to each of its members, and potentially also the ability to make changes to the role itself. However, the actual permissions conferred depend on the options associated with the grant. To modify that options of an existing membership, simply specify the membership with updated option values.

Each of the options described below can be set to either TRUE or FALSE. The keyword OPTION is accepted as a synonym for TRUE, so that WITH ADMIN OPTION is a synonym for WITH ADMIN TRUE. When altering an existing membership the omission of an option results in the current value being retained.

The ADMIN option allows the member to in turn grant membership in the role to others, and revoke membership in the role as well. Without the admin option, ordinary users cannot do that. A role is not considered to hold WITH ADMIN OPTION on itself. Database superusers can grant or revoke membership in any role to anyone. This option defaults to FALSE.

The INHERIT option controls the inheritance status of the new membership; see ??? for details on inheritance. If it is set to TRUE, it causes the new member to inherit from the granted role. If set to FALSE, the new member does not inherit. If unspecified when creating a new role membership, this defaults to the inheritance attribute of the new member.

The SET option, if it is set to TRUE, allows the member to change to the granted role using the SET ROLE command. If a role is an indirect member of another role, it can use SET ROLE to change to that role only if there is a chain of grants each of which has SET TRUE. This option defaults to TRUE.

To create an object owned by another role or give ownership of an existing object to another role, you must have the ability to SET ROLE to that role; otherwise, commands such as ALTER ... OWNER TO or CREATE DATABASE ... OWNER will fail. However, a user who inherits the privileges of a role but does not have the ability to SET ROLE to that role may be able to obtain full access to the role by manipulating existing objects owned by that role (e.g. they could redefine an existing function to act as a Trojan horse). Therefore, if a role's privileges are to be inherited but should not be accessible via SET ROLE, it should not own any SQL objects.

If GRANTED BY is specified, the grant is recorded as having been done by the specified role. A user can only attribute a grant to another role if it inherits the privileges of that role. The role recorded as the grantor must have ADMIN OPTION on the target role, unless it is the bootstrap superuser. When a grant is recorded as having a grantor other than the bootstrap superuser, it depends on the grantor continuing to possess ADMIN OPTION on the role; so, if ADMIN OPTION is revoked, dependent grants must be revoked as well.

Unlike the case with privileges, membership in a role cannot be granted to PUBLIC. Note also that this form of the command does not allow the noise word GROUP in <role_specification>.

Notes

The REVOKE command is used to revoke access privileges.

Since PostgreSQL 8.1, the concepts of users and groups have been unified into a single kind of entity called a role. It is therefore no longer necessary to use the keyword GROUP to identify whether a grantee is a user or a group. GROUP is still allowed in the command, but it is a noise word.

A user may perform SELECT, INSERT, etc. on a column if they hold that privilege for either the specific column or its whole table. Granting the privilege at the table level and then revoking it for one column will not do what one might wish: the table-level grant is unaffected by a column-level operation.

When a non-owner of an object attempts to GRANT privileges on the object, the command will fail outright if the user has no privileges whatsoever on the object. As long as some privilege is available, the command will proceed, but it will grant only those privileges for which the user has grant options. The GRANT ALL PRIVILEGES forms will issue a warning message if no grant options are held, while the other forms will issue a warning if grant options for any of the privileges specifically named in the command are not held. (In principle these statements apply to the object owner as well, but since the owner is always treated as holding all grant options, the cases can never occur.)

It should be noted that database superusers can access all objects regardless of object privilege settings. This is comparable to the rights of root in a Unix system. As with root, it's unwise to operate as a superuser except when absolutely necessary.

If a superuser chooses to issue a GRANT or REVOKE command, the command is performed as though it were issued by the owner of the affected object. In particular, privileges granted via such a command will appear to have been granted by the object owner. (For role membership, the membership appears to have been granted by the bootstrap superuser.)

GRANT and REVOKE can also be done by a role that is not the owner of the affected object, but is a member of the role that owns the object, or is a member of a role that holds privileges WITH GRANT OPTION on the object. In this case the privileges will be recorded as having been granted by the role that actually owns the object or holds the privileges WITH GRANT OPTION. For example, if table t1 is owned by role g1, of which role u1 is a member, then u1 can grant privileges on t1 to u2, but those privileges will appear to have been granted directly by g1. Any other member of role g1 could revoke them later.

If the role executing GRANT holds the required privileges indirectly via more than one role membership path, it is unspecified which containing role will be recorded as having done the grant. In such cases it is best practice to use SET ROLE to become the specific role you want to do the GRANT as.

Granting permission on a table does not automatically extend permissions to any sequences used by the table, including sequences tied to SERIAL columns. Permissions on sequences must be set separately.

See ??? for more information about specific privilege types, as well as how to inspect objects' privileges.

Examples

Grant insert privilege to all users on table films:

GRANT INSERT ON films TO PUBLIC;

Grant all available privileges to user manuel on view kinds:

GRANT ALL PRIVILEGES ON kinds TO manuel;

Note that while the above will indeed grant all privileges if executed by a superuser or the owner of kinds, when executed by someone else it will only grant those permissions for which the someone else has grant options.

Grant membership in role admins to user joe:

GRANT admins TO joe;

Compatibility

According to the SQL standard, the PRIVILEGES key word in ALL PRIVILEGES is required. The SQL standard does not support setting the privileges on more than one object per command.

PostgreSQL allows an object owner to revoke their own ordinary privileges: for example, a table owner can make the table read-only to themselves by revoking their own INSERT, UPDATE, DELETE, and TRUNCATE privileges. This is not possible according to the SQL standard. The reason is that PostgreSQL treats the owner's privileges as having been granted by the owner to themselves; therefore they can revoke them too. In the SQL standard, the owner's privileges are granted by an assumed entity “_SYSTEM”. Not being “_SYSTEM”, the owner cannot revoke these rights.

According to the SQL standard, grant options can be granted to PUBLIC; PostgreSQL only supports granting grant options to roles.

The SQL standard allows the GRANTED BY option to specify only CURRENT_USER or CURRENT_ROLE. The other variants are PostgreSQL extensions.

The SQL standard provides for a USAGE privilege on other kinds of objects: character sets, collations, translations.

In the SQL standard, sequences only have a USAGE privilege, which controls the use of the NEXT VALUE FOR expression, which is equivalent to the function nextval in PostgreSQL. The sequence privileges SELECT and UPDATE are PostgreSQL extensions. The application of the sequence USAGE privilege to the currval function is also a PostgreSQL extension (as is the function itself).

Privileges on databases, tablespaces, schemas, languages, and configuration parameters are PostgreSQL extensions.

See Also

import_foreign_schema

IMPORT FOREIGN SCHEMA

IMPORT FOREIGN SCHEMA

7

SQL - Language Statements

IMPORT FOREIGN SCHEMA

import table definitions from a foreign server

IMPORT FOREIGN SCHEMA

remote_schema

[ { LIMIT TO | EXCEPT } (

table_name

[, ...] ) ] FROM SERVER

server_name

INTO

local_schema

[ OPTIONS (

option

'

value

' [, ... ] ) ]

Description

IMPORT FOREIGN SCHEMA creates foreign tables that represent tables existing on a foreign server. The new foreign tables will be owned by the user issuing the command and are created with the correct column definitions and options to match the remote tables.

By default, all tables and views existing in a particular schema on the foreign server are imported. Optionally, the list of tables can be limited to a specified subset, or specific tables can be excluded. The new foreign tables are all created in the target schema, which must already exist.

To use IMPORT FOREIGN SCHEMA, the user must have USAGE privilege on the foreign server, as well as CREATE privilege on the target schema.

Parameters

<remote_schema>
The remote schema to import from. The specific meaning of a remote schema depends on the foreign data wrapper in use.

LIMIT TO ( table_name [, ...] )
Import only foreign tables matching one of the given table names. Other tables existing in the foreign schema will be ignored.

EXCEPT ( table_name [, ...] )
Exclude specified foreign tables from the import. All tables existing in the foreign schema will be imported except the ones listed here.

<server_name>
The foreign server to import from.

<local_schema>
The schema in which the imported foreign tables will be created.

OPTIONS ( option 'value' [, ...] )
Options to be used during the import. The allowed option names and values are specific to each foreign data wrapper.

Examples

Import table definitions from a remote schema foreign_films on server film_server, creating the foreign tables in local schema films:

IMPORT FOREIGN SCHEMA foreign_films
    FROM SERVER film_server INTO films;

As above, but import only the two tables actors and directors (if they exist):

IMPORT FOREIGN SCHEMA foreign_films LIMIT TO (actors, directors)
    FROM SERVER film_server INTO films;

Compatibility

The IMPORT FOREIGN SCHEMA command conforms to the SQL standard, except that the OPTIONS clause is a PostgreSQL extension.

See Also

initdb

initdb

initdb

1

Application

initdb

create a new

PostgreSQL

database cluster

initdb

option

--pgdata

-D

datadir

Description

initdb creates a new PostgreSQL database cluster.

Creating a database cluster consists of creating the directories in which the cluster data will live, generating the shared catalog tables (tables that belong to the whole cluster rather than to any particular database), and creating the postgres, template1, and template0 databases. The postgres database is a default database meant for use by users, utilities and third party applications. template1 and template0 are meant as source databases to be copied by later CREATE DATABASE commands. template0 should never be modified, but you can add objects to template1, which by default will be copied into databases created later. See ??? for more details.

Although initdb will attempt to create the specified data directory, it might not have permission if the parent directory of the desired data directory is root-owned. To initialize in such a setup, create an empty data directory as root, then use chown to assign ownership of that directory to the database user account, then su to become the database user to run initdb.

initdb must be run as the user that will own the server process, because the server needs to have access to the files and directories that initdb creates. Since the server cannot be run as root, you must not run initdb as root either. (It will in fact refuse to do so.)

For security reasons the new cluster created by initdb will only be accessible by the cluster owner by default. The --allow-group-access option allows any user in the same group as the cluster owner to read files in the cluster. This is useful for performing backups as a non-privileged user.

initdb initializes the database cluster's default locale and character set encoding. These can also be set separately for each database when it is created. initdb determines those settings for the template databases, which will serve as the default for all other databases.

By default, initdb uses the locale provider libc (see ???). The libc locale provider takes the locale settings from the environment, and determines the encoding from the locale settings.

To choose a different locale for the cluster, use the option --locale. There are also individual options --lc-* and --icu-locale (see below) to set values for the individual locale categories. Note that inconsistent settings for different locale categories can give nonsensical results, so this should be used with care.

Alternatively, initdb can use the ICU library to provide locale services by specifying --locale-provider=icu. The server must be built with ICU support. To choose the specific ICU locale ID to apply, use the option --icu-locale. Note that for implementation reasons and to support legacy code, initdb will still select and initialize libc locale settings when the ICU locale provider is used.

When initdb runs, it will print out the locale settings it has chosen. If you have complex requirements or specified multiple options, it is advisable to check that the result matches what was intended.

More details about locale settings can be found in ???.

To alter the default encoding, use the --encoding. More details can be found in ???.

Options

-A authmethod; --auth=authmethod
This option specifies the default authentication method for local users used in pg_hba.conf (host and local lines). See ??? for an overview of valid values.

initdb will prepopulate pg_hba.conf entries using the specified authentication method for non-replication as well as replication connections.

Do not use trust unless you trust all local users on your system. trust is the default for ease of installation.

--auth-host=authmethod
This option specifies the authentication method for local users via TCP/IP connections used in pg_hba.conf (host lines).

--auth-local=authmethod
This option specifies the authentication method for local users via Unix-domain socket connections used in pg_hba.conf (local lines).

-D datadir; --pgdata=datadir
This option specifies the directory where the database cluster should be stored. This is the only information required by initdb, but you can avoid writing it by setting the PGDATA environment variable, which can be convenient since the database server (postgres) can find the data directory later by the same variable.

-E encoding; --encoding=encoding
Selects the encoding of the template databases. This will also be the default encoding of any database you create later, unless you override it then. The character sets supported by the PostgreSQL server are described in ???.

By default, the template database encoding is derived from the locale. If varlistentry_title is specified (or equivalently, if the locale is C or POSIX), then the default is UTF8 for the ICU provider and SQL_ASCII for the libc provider.

-g; --allow-group-access
Allows users in the same group as the cluster owner to read all cluster files created by initdb. This option is ignored on Windows as it does not support POSIX-style group permissions.

--icu-locale=locale
Specifies the ICU locale when the ICU provider is used. Locale support is described in ???.

--icu-rules=rules
Specifies additional collation rules to customize the behavior of the default collation. This is supported for ICU only.

-k; --data-checksums
Use checksums on data pages to help detect corruption by the I/O system that would otherwise be silent. This is enabled by default; use varlistentry_title to disable checksums.

Enabling checksums might incur a small performance penalty. If set, checksums are calculated for all objects, in all databases. All checksum failures will be reported in the pg_stat_database view. See ??? for details.

--locale=locale
Sets the default locale for the database cluster. If this option is not specified, the locale is inherited from the environment that initdb runs in. Locale support is described in ???.

If --locale-provider is builtin, --locale or --builtin-locale must be specified and set to C, C.UTF-8 or PG_UNICODE_FAST.

--lc-collate=locale; --lc-ctype=locale; --lc-messages=locale; --lc-monetary=locale; --lc-numeric=locale; --lc-time=locale
Like --locale, but only sets the locale in the specified category.

--no-locale
Equivalent to --locale=C.

--builtin-locale=locale
Specifies the locale name when the builtin provider is used. Locale support is described in ???.

--locale-provider={builtin|libc|icu}
This option sets the locale provider for databases created in the new cluster. It can be overridden in the CREATE DATABASE command when new databases are subsequently created. The default is libc (see ???).

--no-data-checksums
Do not enable data checksums.

--pwfile=filename
Makes initdb read the bootstrap superuser's password from a file. The first line of the file is taken as the password.

-T config; --text-search-config=config
Sets the default text search configuration. See ??? for further information.

-U username; --username=username
Sets the user name of the bootstrap superuser. This defaults to the name of the operating-system user running initdb.

-W; --pwprompt
Makes initdb prompt for a password to give the bootstrap superuser. If you don't plan on using password authentication, this is not important. Otherwise you won't be able to use password authentication until you have a password set up.

-X directory; --waldir=directory
This option specifies the directory where the write-ahead log should be stored.

--wal-segsize=size
Set the WAL segment size, in megabytes. This is the size of each individual file in the WAL log. The default size is 16 megabytes. The value must be a power of 2 between 1 and 1024 (megabytes). This option can only be set during initialization, and cannot be changed later.

It may be useful to adjust this size to control the granularity of WAL log shipping or archiving. Also, in databases with a high volume of WAL, the sheer number of WAL files per directory can become a performance and management problem. Increasing the WAL file size will reduce the number of WAL files.

Other, less commonly used, options are also available:

-c name=value; --set name=value
Forcibly set the server parameter <name> to <value> during initdb, and also install that setting in the generated postgresql.conf file, so that it will apply during future server runs. This option can be given more than once to set several parameters. It is primarily useful when the environment is such that the server will not start at all using the default parameters.

-d; --debug
Print debugging output from the bootstrap backend and a few other messages of lesser interest for the general public. The bootstrap backend is the program initdb uses to create the catalog tables. This option generates a tremendous amount of extremely boring output.

--discard-caches
Run the bootstrap backend with the debug_discard_caches=1 option. This takes a very long time and is only of use for deep debugging.

-L directory
Specifies where initdb should find its input files to initialize the database cluster. This is normally not necessary. You will be told if you need to specify their location explicitly.

-n; --no-clean
By default, when initdb determines that an error prevented it from completely creating the database cluster, it removes any files it might have created before discovering that it cannot finish the job. This option inhibits tidying-up and is thus useful for debugging.

-N; --no-sync
By default, initdb will wait for all files to be written safely to disk. This option causes initdb to return without waiting, which is faster, but means that a subsequent operating system crash can leave the data directory corrupt. Generally, this option is useful for testing, but should not be used when creating a production installation.

--no-sync-data-files
By default, initdb safely writes all database files to disk. This option instructs initdb to skip synchronizing all files in the individual database directories, the database directories themselves, and the tablespace directories, i.e., everything in the base subdirectory and any other tablespace directories. Other files, such as those in pg_wal and pg_xact, will still be synchronized unless the --no-sync option is also specified.

Note that if --no-sync-data-files is used in conjunction with --sync-method=syncfs, some or all of the aforementioned files and directories will be synchronized because syncfs processes entire file systems.

This option is primarily intended for internal use by tools that separately ensure the skipped files are synchronized to disk.

--no-instructions
By default, initdb will write instructions for how to start the cluster at the end of its output. This option causes those instructions to be left out. This is primarily intended for use by tools that wrap initdb in platform-specific behavior, where those instructions are likely to be incorrect.

-s; --show
Show internal settings and exit, without doing anything else. This can be used to debug the initdb installation.

--sync-method=method
When set to fsync, which is the default, initdb will recursively open and synchronize all files in the data directory. The search for files will follow symbolic links for the WAL directory and each configured tablespace.

On Linux, syncfs may be used instead to ask the operating system to synchronize the whole file systems that contain the data directory, the WAL files, and each tablespace. See ??? for information about the caveats to be aware of when using syncfs.

This option has no effect when --no-sync is used.

-S; --sync-only
Safely write all database files to disk and exit. This does not perform any of the normal initdb operations. Generally, this option is useful for ensuring reliable recovery after changing ??? from off to on.

Other options:

-V; --version
Print the initdb version and exit.

-?; --help
Show help about initdb command line arguments, and exit.

Environment

PGDATA
Specifies the directory where the database cluster is to be stored; can be overridden using the -D option.

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

TZ
Specifies the default time zone of the created database cluster. The value should be a full time zone name (see ???).

Notes

initdb can also be invoked via pg_ctl initdb.

See Also

insert

INSERT

INSERT

7

SQL - Language Statements

INSERT

create new rows in a table

[ WITH [ RECURSIVE ]

with_query

[, ...] ] INSERT INTO

table_name

[ AS

alias

] [ (

column_name

[, ...] ) ] [ OVERRIDING { SYSTEM | USER } VALUE ] { DEFAULT VALUES | VALUES ( {

expression

| DEFAULT } [, ...] ) [, ...] |

query

} [ ON CONFLICT [

conflict_target

]

conflict_action

] [ RETURNING [ WITH ( { OLD | NEW } AS

output_alias

[, ...] ) ] { * |

output_expression

[ [ AS ]

output_name

] } [, ...] ]

where

conflict_target

can be one of:

( {

index_column_name

| (

index_expression

) } [ COLLATE

collation

] [

opclass

] [, ...] ) [ WHERE

index_predicate

] ON CONSTRAINT

constraint_name

and

conflict_action

is one of:

DO NOTHING DO SELECT [ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } ] [ WHERE

condition

] DO UPDATE SET {

column_name

= {

expression

| DEFAULT } | (

column_name

[, ...] ) = [ ROW ] ( {

expression

| DEFAULT } [, ...] ) | (

column_name

[, ...] ) = (

sub-SELECT

) } [, ...] [ WHERE

condition

]

Description

INSERT inserts new rows into a table. One can insert one or more rows specified by value expressions, or zero or more rows resulting from a query.

The target column names can be listed in any order. If no list of column names is given at all, the default is all the columns of the table in their declared order; or the first <N> column names, if there are only <N> columns supplied by the VALUES clause or <query>. The values supplied by the VALUES clause or <query> are associated with the explicit or implicit column list left-to-right.

Each column not present in the explicit or implicit column list will be filled with a default value, either its declared default value or null if there is none.

If the expression for any column is not of the correct data type, automatic type conversion will be attempted.

INSERT into tables that lack unique indexes will not be blocked by concurrent activity. Tables with unique indexes might block if concurrent sessions perform actions that lock or modify rows matching the unique index values being inserted; the details are covered in ???. ON CONFLICT can be used to specify an alternative action to raising a unique constraint or exclusion constraint violation error. (See ON CONFLICT Clause below.)

The optional RETURNING clause causes INSERT to compute and return value(s) based on each row actually inserted (or selected or updated, if an ON CONFLICT DO SELECT/UPDATE clause was used). This is primarily useful for obtaining values that were supplied by defaults, such as a serial sequence number. However, any expression using the table's columns is allowed. The syntax of the RETURNING list is identical to that of the output list of SELECT. Only rows that were successfully inserted, updated, or selected will be returned. For example, if a row was locked but not updated or selected because an ON CONFLICT ... WHERE clause <condition> was not satisfied, the row will not be returned.

You must have INSERT privilege on a table in order to insert into it. If ON CONFLICT DO UPDATE is present, UPDATE privilege on the table is also required. If ON CONFLICT DO SELECT is present, SELECT privilege on the table is required. If ON CONFLICT DO SELECT FOR UPDATE/SHARE is used, UPDATE privilege is required on at least one column, in addition to SELECT privilege.

If a column list is specified, you only need INSERT privilege on the listed columns. Similarly, when ON CONFLICT DO UPDATE is specified, you only need UPDATE privilege on the column(s) that are listed to be updated. However, all forms of ON CONFLICT also require SELECT privilege on any column whose values are read. This includes any column mentioned in <conflict_target> (including columns referred to by the arbiter constraint), and any column mentioned in an ON CONFLICT DO UPDATE <expression>, or a WHERE clause <condition>.

Use of the RETURNING clause requires SELECT privilege on all columns mentioned in RETURNING. If you use the <query> clause to insert rows from a query, you of course need to have SELECT privilege on any table or column used in the query.

Parameters

Inserting

This section covers parameters that may be used when only inserting new rows. Parameters exclusively used with the ON CONFLICT clause are described separately.

<with_query>
The WITH clause allows you to specify one or more subqueries that can be referenced by name in the INSERT query. See ??? and ??? for details.

It is possible for the <query> (SELECT statement) to also contain a WITH clause. In such a case both sets of <with_query> can be referenced within the <query>, but the second one takes precedence since it is more closely nested.

<table_name>
The name (optionally schema-qualified) of an existing table.

<alias>
A substitute name for <table_name>. When an alias is provided, it completely hides the actual name of the table. This is particularly useful when ON CONFLICT DO UPDATE targets a table named excluded, since that will otherwise be taken as the name of the special table representing the row proposed for insertion.

<column_name>
The name of a column in the table named by <table_name>. The column name can be qualified with a subfield name or array subscript, if needed. (Inserting into only some fields of a composite column leaves the other fields null.) When referencing a column with ON CONFLICT DO UPDATE, do not include the table's name in the specification of a target column. For example, INSERT INTO table_name ... ON CONFLICT DO UPDATE SET table_name.col = 1 is invalid (this follows the general behavior for UPDATE).

OVERRIDING SYSTEM VALUE
If this clause is specified, then any values supplied for identity columns will override the default sequence-generated values.

For an identity column defined as GENERATED ALWAYS, it is an error to insert an explicit value (other than DEFAULT) without specifying either OVERRIDING SYSTEM VALUE or OVERRIDING USER VALUE. (For an identity column defined as GENERATED BY DEFAULT, OVERRIDING SYSTEM VALUE is the normal behavior and specifying it does nothing, but PostgreSQL allows it as an extension.)

OVERRIDING USER VALUE
If this clause is specified, then any values supplied for identity columns are ignored and the default sequence-generated values are applied.

This clause is useful for example when copying values between tables. Writing INSERT INTO tbl2 OVERRIDING USER VALUE SELECT * FROM tbl1 will copy from tbl1 all columns that are not identity columns in tbl2 while values for the identity columns in tbl2 will be generated by the sequences associated with tbl2.

DEFAULT VALUES
All columns will be filled with their default values, as if DEFAULT were explicitly specified for each column. (An OVERRIDING clause is not permitted in this form.)

<expression>
An expression or value to assign to the corresponding column.

DEFAULT
The corresponding column will be filled with its default value. An identity column will be filled with a new value generated by the associated sequence. For a generated column, specifying this is permitted but merely specifies the normal behavior of computing the column from its generation expression.

<query>
A query (SELECT statement) that supplies the rows to be inserted. Refer to the ??? statement for a description of the syntax.

<output_alias>
An optional substitute name for OLD or NEW rows in the RETURNING list.

By default, old values from the target table can be returned by writing OLD.column_name or OLD.*, and new values can be returned by writing NEW.column_name or NEW.*. When an alias is provided, these names are hidden and the old or new rows must be referred to using the alias. For example RETURNING WITH (OLD AS o, NEW AS n) o.*, n.*.

<output_expression>
An expression to be computed and returned by the INSERT command after each row is inserted or updated. The expression can use any column names of the table named by <table_name>. Write * to return all columns of the inserted or updated row(s).

A column name or * may be qualified using OLD or NEW, or the corresponding <output_alias> for OLD or NEW, to cause old or new values to be returned. An unqualified column name, or *, or a column name or * qualified using the target table name or alias will return new values.

For a simple INSERT, all old values will be NULL. However, for an INSERT with an ON CONFLICT DO SELECT/UPDATE clause, the old values may be non-NULL (when the row proposed for insertion conflicts with an existing row). If the SELECT path is taken, the new values will be identical to the old values, since no modification takes place.

<output_name>
A name to use for a returned column.

ON CONFLICT Clause

UPSERT

ON CONFLICT

The optional ON CONFLICT clause specifies an alternative action to raising a unique violation or exclusion constraint violation error. For each individual row proposed for insertion, either the insertion proceeds, or, if an arbiter constraint or index specified by conflict_target is violated, the alternative conflict_action is taken. ON CONFLICT DO NOTHING simply avoids inserting a row as its alternative action. ON CONFLICT DO UPDATE updates the existing row that conflicts with the row proposed for insertion as its alternative action. ON CONFLICT DO SELECT returns the existing row that conflicts with the row proposed for insertion, optionally with row-level locking.

conflict_target can perform unique index inference. When performing inference, it consists of one or more <index_column_name> columns and/or <index_expression> expressions, and an optional <index_predicate>. All <table_name> unique indexes that, without regard to order, contain exactly the conflict_target-specified columns/expressions are inferred (chosen) as arbiter indexes. If an <index_predicate> is specified, it must, as a further requirement for inference, satisfy arbiter indexes. Note that this means a non-partial unique index (a unique index without a predicate) will be inferred (and thus used by ON CONFLICT) if such an index satisfying every other criteria is available. If an attempt at inference is unsuccessful, an error is raised.

ON CONFLICT DO UPDATE guarantees an atomic INSERT or UPDATE outcome; provided there is no independent error, one of those two outcomes is guaranteed, even under high concurrency. This is also known as UPSERT — “UPDATE or INSERT”.

ON CONFLICT DO SELECT similarly allows an atomic INSERT or SELECT outcome. This is also known as idempotent insert or get or create. For ON CONFLICT DO SELECT, a RETURNING clause must be provided.

<conflict_target>
Specifies which conflicts ON CONFLICT takes the alternative action on by choosing arbiter indexes. Either performs unique index inference, or names a constraint explicitly. For ON CONFLICT DO NOTHING, it is optional to specify a conflict_target; when omitted, conflicts with all usable constraints (and unique indexes) are handled. For ON CONFLICT DO UPDATE and ON CONFLICT DO SELECT, a conflict_target must be provided.

<conflict_action>
conflict_action specifies an alternative ON CONFLICT action. It can be DO NOTHING, a DO SELECT clause that allows conflicting rows to be returned, or a DO UPDATE clause specifying the exact details of the UPDATE action to be performed in case of a conflict.

The SET clause in DO UPDATE and the WHERE clause in both DO SELECT and DO UPDATE have access to the existing row using the table's name (or an alias), and to the row proposed for insertion using the special excluded table. SELECT privilege is required on any column in the target table where corresponding excluded columns are read.

Note that the effects of all per-row BEFORE INSERT triggers are reflected in excluded values, since those effects may have contributed to the row being excluded from insertion.

<index_column_name>
The name of a <table_name> column. Used to infer arbiter indexes. Follows CREATE INDEX format. SELECT privilege on <index_column_name> is required.

<index_expression>
Similar to <index_column_name>, but used to infer expressions on <table_name> columns appearing within index definitions (not simple columns). Follows CREATE INDEX format. SELECT privilege on any column appearing within <index_expression> is required.

<collation>
When specified, mandates that corresponding <index_column_name> or <index_expression> use a particular collation in order to be matched during inference. Typically this is omitted, as collations usually do not affect whether or not a constraint violation occurs. Follows CREATE INDEX format.

<opclass>
When specified, mandates that corresponding <index_column_name> or <index_expression> use particular operator class in order to be matched during inference. Typically this is omitted, as the equality semantics are often equivalent across a type's operator classes anyway, or because it's sufficient to trust that the defined unique indexes have the pertinent definition of equality. Follows CREATE INDEX format.

<index_predicate>
Used to allow inference of partial unique indexes. Any indexes that satisfy the predicate (which need not actually be partial indexes) can be inferred. Follows CREATE INDEX format. SELECT privilege on any column appearing within <index_predicate> is required.

<constraint_name>
Explicitly specifies an arbiter constraint by name, rather than inferring a constraint or index.

FOR UPDATE; FOR NO KEY UPDATE; FOR SHARE; FOR KEY SHARE
When specified in an ON CONFLICT DO SELECT clause, conflicting table rows are locked against concurrent updates. See ??? in the ??? documentation.

<condition>
An expression that returns a value of type boolean. Only rows for which this expression returns true will be updated or selected for return, although all conflicting rows will be locked when ON CONFLICT DO UPDATE or ON CONFLICT DO SELECT FOR UPDATE/SHARE is specified. Note that <condition> is evaluated last, after a conflict has been identified as a candidate to update or select.

Note that exclusion constraints are not supported as arbiters with ON CONFLICT DO SELECT/UPDATE. In all cases, only NOT DEFERRABLE constraints and unique indexes are supported as arbiters.

INSERT with an ON CONFLICT DO UPDATE clause is a “deterministic” statement. This means that the command will not be allowed to affect any single existing row more than once; a cardinality violation error will be raised when this situation arises. Rows proposed for insertion should not duplicate each other in terms of attributes constrained by an arbiter index or constraint.

Note that it is currently not supported for the ON CONFLICT DO UPDATE clause of an INSERT applied to a partitioned table to update the partition key of a conflicting row such that it requires the row be moved to a new partition.

[!TIP] It is often preferable to use unique index inference rather than naming a constraint directly using ON CONFLICT ON CONSTRAINT <constraint_name>. Inference will continue to work correctly when the underlying index is replaced by another more or less equivalent index in an overlapping way, for example when using CREATE UNIQUE INDEX ... CONCURRENTLY before dropping the index being replaced.

Outputs

On successful completion, an INSERT command returns a command tag of the form

INSERT oid count

The <count> is the number of rows inserted, updated, or selected for return. <oid> is always 0 (it used to be the OID assigned to the inserted row if <count> was exactly one and the target table was declared WITH OIDS and 0 otherwise, but creating a table WITH OIDS is not supported anymore).

If the INSERT command contains a RETURNING clause, the result will be similar to that of a SELECT statement containing the columns and values defined in the RETURNING list, computed over the row(s) affected by the command.

Notes

If the specified table is a partitioned table, each row is routed to the appropriate partition and inserted into it. If the specified table is a partition, an error will occur if one of the input rows violates the partition constraint.

You may also wish to consider using MERGE, since that allows mixing INSERT, UPDATE, and DELETE within a single statement. See ???.

Examples

Insert a single row into table films:

INSERT INTO films VALUES
    ('UA502', 'Bananas', 105, '1971-07-13', 'Comedy', '82 minutes');

In this example, the len column is omitted and therefore it will have the default value:

INSERT INTO films (code, title, did, date_prod, kind)
    VALUES ('T_601', 'Yojimbo', 106, '1961-06-16', 'Drama');

This example uses the DEFAULT clause for the date columns rather than specifying a value:

INSERT INTO films VALUES
    ('UA502', 'Bananas', 105, DEFAULT, 'Comedy', '82 minutes');
INSERT INTO films (code, title, did, date_prod, kind)
    VALUES ('T_601', 'Yojimbo', 106, DEFAULT, 'Drama');

To insert a row consisting entirely of default values:

INSERT INTO films DEFAULT VALUES;

To insert multiple rows using the multirow VALUES syntax:

INSERT INTO films (code, title, did, date_prod, kind) VALUES
    ('B6717', 'Tampopo', 110, '1985-02-10', 'Comedy'),
    ('HG120', 'The Dinner Game', 140, DEFAULT, 'Comedy');

This example inserts some rows into table films from a table tmp_films with the same column layout as films:

INSERT INTO films SELECT * FROM tmp_films WHERE date_prod < '2004-05-07';

This example inserts into array columns:

-- Create an empty 3x3 gameboard for noughts-and-crosses
INSERT INTO tictactoe (game, board[1:3][1:3])
    VALUES (1, '{{" "," "," "},{" "," "," "},{" "," "," "}}');
-- The subscripts in the above example aren't really needed
INSERT INTO tictactoe (game, board)
    VALUES (2, '{{X," "," "},{" ",O," "},{" ",X," "}}');

Insert a single row into table distributors, returning the sequence number generated by the DEFAULT clause:

INSERT INTO distributors (did, dname) VALUES (DEFAULT, 'XYZ Widgets')
   RETURNING did;

Increment the sales count of the salesperson who manages the account for Acme Corporation, and record the whole updated row along with current time in a log table:

WITH upd AS (
  UPDATE employees SET sales_count = sales_count + 1 WHERE id =
    (SELECT sales_person FROM accounts WHERE name = 'Acme Corporation')
    RETURNING *
)
INSERT INTO employees_log SELECT *, current_timestamp FROM upd;

Insert or update new distributors as appropriate. Assumes a unique index has been defined that constrains values appearing in the did column. Note that the special excluded table is used to reference values originally proposed for insertion:

INSERT INTO distributors (did, dname)
    VALUES (5, 'Gizmo Transglobal'), (6, 'Associated Computing, Inc')
    ON CONFLICT (did) DO UPDATE SET dname = EXCLUDED.dname;

Insert or update new distributors as above, returning information about any existing values that were updated, together with the new data inserted. Note that the returned values for old_did and old_dname will be NULL for non-conflicting rows:

INSERT INTO distributors (did, dname)
    VALUES (5, 'Gizmo Transglobal'), (6, 'Associated Computing, Inc')
    ON CONFLICT (did) DO UPDATE SET dname = EXCLUDED.dname
    RETURNING old.did AS old_did, old.dname AS old_dname,
              new.did AS new_did, new.dname AS new_dname;

Insert a distributor, or do nothing for rows proposed for insertion when an existing, excluded row (a row with a matching constrained column or columns after before row insert triggers fire) exists. Example assumes a unique index has been defined that constrains values appearing in the did column:

INSERT INTO distributors (did, dname) VALUES (7, 'Redline GmbH')
    ON CONFLICT (did) DO NOTHING;

Insert or update new distributors as appropriate. Example assumes a unique index has been defined that constrains values appearing in the did column. WHERE clause is used to limit the rows actually updated (any existing row not updated will still be locked, though):

-- Don't update existing distributors based in a certain ZIP code
INSERT INTO distributors AS d (did, dname) VALUES (8, 'Anvil Distribution')
    ON CONFLICT (did) DO UPDATE
    SET dname = EXCLUDED.dname || ' (formerly ' || d.dname || ')'
    WHERE d.zipcode <> '21201';

-- Name a constraint directly in the statement (uses associated
-- index to arbitrate taking the DO NOTHING action)
INSERT INTO distributors (did, dname) VALUES (9, 'Antwerp Design')
    ON CONFLICT ON CONSTRAINT distributors_pkey DO NOTHING;

Insert new distributor if possible, otherwise return the existing distributor row. Example assumes a unique index has been defined that constrains values appearing in the did column. This is useful for get-or-create patterns:

INSERT INTO distributors (did, dname) VALUES (11, 'Global Electronics')
    ON CONFLICT (did) DO SELECT
    RETURNING *;

Insert a new distributor if the ID doesn't match, otherwise return the existing row, if its name doesn't match:

INSERT INTO distributors AS d (did, dname) VALUES (12, 'Micro Devices Inc')
    ON CONFLICT (did) DO SELECT WHERE d.dname != EXCLUDED.dname
    RETURNING *;

Insert a new distributor or return and lock the existing row for update. This is useful when you need to ensure exclusive access to the row:

INSERT INTO distributors (did, dname) VALUES (13, 'Advanced Systems')
    ON CONFLICT (did) DO SELECT FOR UPDATE
    RETURNING *;

Insert new distributor if possible; otherwise DO NOTHING. Example assumes a unique index has been defined that constrains values appearing in the did column on a subset of rows where the is_active Boolean column evaluates to true:

-- This statement could infer a partial unique index on "did"
-- with a predicate of "WHERE is_active", but it could also
-- just use a regular unique constraint on "did"
INSERT INTO distributors (did, dname) VALUES (10, 'Conrad International')
    ON CONFLICT (did) WHERE is_active DO NOTHING;

Compatibility

INSERT conforms to the SQL standard, except that the RETURNING clause is a PostgreSQL extension, as is the ability to use WITH with INSERT, and the ability to specify an alternative action with ON CONFLICT. Also, the case in which a column name list is omitted, but not all the columns are filled from the VALUES clause or <query>, is disallowed by the standard. If you prefer a more SQL standard conforming statement than ON CONFLICT, see ???.

The SQL standard specifies that OVERRIDING SYSTEM VALUE can only be specified if an identity column that is generated always exists. PostgreSQL allows the clause in any case and ignores it if it is not applicable.

Possible limitations of the <query> clause are documented under ???.

listen

LISTEN

LISTEN

7

SQL - Language Statements

LISTEN

listen for a notification

LISTEN

channel

Description

LISTEN registers the current session as a listener on the notification channel named <channel>. If the current session is already registered as a listener for this notification channel, nothing is done.

Whenever the command NOTIFY channel is invoked, either by this session or another one connected to the same database, all the sessions currently listening on that notification channel are notified, and each will in turn notify its connected client application.

A session can be unregistered for a given notification channel with the UNLISTEN command. A session's listen registrations are automatically cleared when the session ends.

The method a client application must use to detect notification events depends on which PostgreSQL application programming interface it uses. With the libpq library, the application issues LISTEN as an ordinary SQL command, and then must periodically call the function PQnotifies to find out whether any notification events have been received. Other interfaces such as libpgtcl provide higher-level methods for handling notify events; indeed, with libpgtcl the application programmer should not even issue LISTEN or UNLISTEN directly. See the documentation for the interface you are using for more details.

Parameters

<channel>
Name of a notification channel (any identifier).

Notes

LISTEN takes effect at transaction commit. If LISTEN or UNLISTEN is executed within a transaction that later rolls back, the set of notification channels being listened to is unchanged.

A transaction that has executed LISTEN cannot be prepared for two-phase commit.

There is a race condition when first setting up a listening session: if concurrently-committing transactions are sending notify events, exactly which of those will the newly listening session receive? The answer is that the session will receive all events committed after an instant during the transaction's commit step. But that is slightly later than any database state that the transaction could have observed in queries. This leads to the following rule for using LISTEN: first execute (and commit!) that command, then in a new transaction inspect the database state as needed by the application logic, then rely on notifications to find out about subsequent changes to the database state. The first few received notifications might refer to updates already observed in the initial database inspection, but this is usually harmless.

??? contains a more extensive discussion of the use of LISTEN and NOTIFY.

Examples

Configure and execute a listen/notify sequence from psql:

LISTEN virtual;
NOTIFY virtual;
Asynchronous notification "virtual" received from server process with PID 8448.

Compatibility

There is no LISTEN statement in the SQL standard.

See Also

load

LOAD

LOAD

7

SQL - Language Statements

LOAD

load a shared library file

LOAD '

filename

'

Description

This command loads a shared library file into the PostgreSQL server's address space. If the file has been loaded already, the command does nothing. Shared library files that contain C functions are automatically loaded whenever one of their functions is called. Therefore, an explicit LOAD is usually only needed to load a library that modifies the server's behavior through “hooks” rather than providing a set of functions.

The library file name is typically given as just a bare file name, which is sought in the server's library search path (set by ???). Alternatively it can be given as a full path name. In either case the platform's standard shared library file name extension may be omitted. See ??? for more information on this topic.

$libdir/plugins

Non-superusers can only apply LOAD to library files located in $libdir/plugins/ — the specified <filename> must begin with exactly that string. (It is the database administrator's responsibility to ensure that only “safe” libraries are installed there.)

Compatibility

LOAD is a PostgreSQL extension.

See Also

???

lock

LOCK

LOCK

7

SQL - Language Statements

LOCK

lock a table

LOCK [ TABLE ] [ ONLY ]

name

[ * ] [, ...] [ IN

lockmode

MODE ] [ NOWAIT ]

where

lockmode

is one of:

ACCESS SHARE | ROW SHARE | ROW EXCLUSIVE | SHARE UPDATE EXCLUSIVE | SHARE | SHARE ROW EXCLUSIVE | EXCLUSIVE | ACCESS EXCLUSIVE

Description

LOCK TABLE obtains a table-level lock, waiting if necessary for any conflicting locks to be released. If NOWAIT is specified, LOCK TABLE does not wait to acquire the desired lock: if it cannot be acquired immediately, the command is aborted and an error is emitted. Once obtained, the lock is held for the remainder of the current transaction. (There is no UNLOCK TABLE command; locks are always released at transaction end.)

When a view is locked, all relations appearing in the view definition query are also locked recursively with the same lock mode.

When acquiring locks automatically for commands that reference tables, PostgreSQL always uses the least restrictive lock mode possible. LOCK TABLE provides for cases when you might need more restrictive locking. For example, suppose an application runs a transaction at the READ COMMITTED isolation level and needs to ensure that data in a table remains stable for the duration of the transaction. To achieve this you could obtain SHARE lock mode over the table before querying. This will prevent concurrent data changes and ensure subsequent reads of the table see a stable view of committed data, because SHARE lock mode conflicts with the ROW EXCLUSIVE lock acquired by writers, and your LOCK TABLE name IN SHARE MODE statement will wait until any concurrent holders of ROW EXCLUSIVE mode locks commit or roll back. Thus, once you obtain the lock, there are no uncommitted writes outstanding; furthermore none can begin until you release the lock.

To achieve a similar effect when running a transaction at the REPEATABLE READ or SERIALIZABLE isolation level, you have to execute the LOCK TABLE statement before executing any SELECT or data modification statement. A REPEATABLE READ or SERIALIZABLE transaction's view of data will be frozen when its first SELECT or data modification statement begins. A LOCK TABLE later in the transaction will still prevent concurrent writes — but it won't ensure that what the transaction reads corresponds to the latest committed values.

If a transaction of this sort is going to change the data in the table, then it should use SHARE ROW EXCLUSIVE lock mode instead of SHARE mode. This ensures that only one transaction of this type runs at a time. Without this, a deadlock is possible: two transactions might both acquire SHARE mode, and then be unable to also acquire ROW EXCLUSIVE mode to actually perform their updates. (Note that a transaction's own locks never conflict, so a transaction can acquire ROW EXCLUSIVE mode when it holds SHARE mode — but not if anyone else holds SHARE mode.) To avoid deadlocks, make sure all transactions acquire locks on the same objects in the same order, and if multiple lock modes are involved for a single object, then transactions should always acquire the most restrictive mode first.

More information about the lock modes and locking strategies can be found in ???.

Parameters

<name>
The name (optionally schema-qualified) of an existing table to lock. If ONLY is specified before the table name, only that table is locked. If ONLY is not specified, the table and all its descendant tables (if any) are locked. Optionally, * can be specified after the table name to explicitly indicate that descendant tables are included.

The command LOCK TABLE a, b; is equivalent to LOCK TABLE a; LOCK TABLE b;. The tables are locked one-by-one in the order specified in the LOCK TABLE command.

<lockmode>
The lock mode specifies which locks this lock conflicts with. Lock modes are described in ???.

If no lock mode is specified, then ACCESS EXCLUSIVE, the most restrictive mode, is used.

NOWAIT
Specifies that LOCK TABLE should not wait for any conflicting locks to be released: if the specified lock(s) cannot be acquired immediately without waiting, the transaction is aborted.

Notes

To lock a table, the user must have the right privilege for the specified <lockmode>. If the user has MAINTAIN, UPDATE, DELETE, or TRUNCATE privileges on the table, any <lockmode> is permitted. If the user has INSERT privileges on the table, ROW EXCLUSIVE MODE (or a less-conflicting mode as described in ???) is permitted. If a user has SELECT privileges on the table, ACCESS SHARE MODE is permitted.

The user performing the lock on the view must have the corresponding privilege on the view. In addition, by default, the view's owner must have the relevant privileges on the underlying base relations, whereas the user performing the lock does not need any permissions on the underlying base relations. However, if the view has security_invoker set to true (see CREATE VIEW), the user performing the lock, rather than the view owner, must have the relevant privileges on the underlying base relations.

LOCK TABLE is useless outside a transaction block: the lock would remain held only to the completion of the statement. Therefore PostgreSQL reports an error if LOCK is used outside a transaction block. Use BEGIN and COMMIT (or ROLLBACK) to define a transaction block.

LOCK TABLE only deals with table-level locks, and so the mode names involving ROW are all misnomers. These mode names should generally be read as indicating the intention of the user to acquire row-level locks within the locked table. Also, ROW EXCLUSIVE mode is a shareable table lock. Keep in mind that all the lock modes have identical semantics so far as LOCK TABLE is concerned, differing only in the rules about which modes conflict with which. For information on how to acquire an actual row-level lock, see ??? and ??? in the ??? documentation.

Examples

Obtain a SHARE lock on a primary key table when going to perform inserts into a foreign key table:

BEGIN WORK;
LOCK TABLE films IN SHARE MODE;
SELECT id FROM films
    WHERE name = 'Star Wars: Episode I - The Phantom Menace';
-- Do ROLLBACK if record was not returned
INSERT INTO films_user_comments VALUES
    (_id_, 'GREAT! I was waiting for it for so long!');
COMMIT WORK;

Take a SHARE ROW EXCLUSIVE lock on a primary key table when going to perform a delete operation:

BEGIN WORK;
LOCK TABLE films IN SHARE ROW EXCLUSIVE MODE;
DELETE FROM films_user_comments WHERE id IN
    (SELECT id FROM films WHERE rating < 5);
DELETE FROM films WHERE rating < 5;
COMMIT WORK;

Compatibility

There is no LOCK TABLE in the SQL standard, which instead uses SET TRANSACTION to specify concurrency levels on transactions. PostgreSQL supports that too; see ??? for details.

Except for ACCESS SHARE, ACCESS EXCLUSIVE, and SHARE UPDATE EXCLUSIVE lock modes, the PostgreSQL lock modes and the LOCK TABLE syntax are compatible with those present in Oracle.

merge

MERGE

MERGE

7

SQL - Language Statements

MERGE

conditionally insert, update, or delete rows of a table

[ WITH

with_query

[, ...] ] MERGE INTO [ ONLY ]

target_table_name

[ * ] [ [ AS ]

target_alias

] USING

data_source

ON

join_condition

when_clause

[...] [ RETURNING [ WITH ( { OLD | NEW } AS

output_alias

[, ...] ) ] { * |

output_expression

[ [ AS ]

output_name

] } [, ...] ]

where

data_source

is:

{ [ ONLY ]

source_table_name

[ * ] | (

source_query

) } [ [ AS ]

source_alias

]

and

when_clause

is:

{ WHEN MATCHED [ AND

condition

] THEN {

merge_update

|

merge_delete

| DO NOTHING } | WHEN NOT MATCHED BY SOURCE [ AND

condition

] THEN {

merge_update

|

merge_delete

| DO NOTHING } | WHEN NOT MATCHED [ BY TARGET ] [ AND

condition

] THEN {

merge_insert

| DO NOTHING } }

and

merge_insert

is:

INSERT [(

column_name

[, ...] )] [ OVERRIDING { SYSTEM | USER } VALUE ] { VALUES ( {

expression

| DEFAULT } [, ...] ) | DEFAULT VALUES }

and

merge_update

is:

UPDATE SET {

column_name

= {

expression

| DEFAULT } | (

column_name

[, ...] ) = [ ROW ] ( {

expression

| DEFAULT } [, ...] ) | (

column_name

[, ...] ) = (

sub-SELECT

) } [, ...]

and

merge_delete

is:

DELETE

Description

MERGE performs actions that modify rows in the target table identified as <target_table_name>, using the <data_source>. MERGE provides a single SQL statement that can conditionally INSERT, UPDATE or DELETE rows, a task that would otherwise require multiple procedural language statements.

First, the MERGE command performs a join from <data_source> to the target table producing zero or more candidate change rows. For each candidate change row, the status of MATCHED, NOT MATCHED BY SOURCE, or NOT MATCHED [BY TARGET] is set just once, after which WHEN clauses are evaluated in the order specified. For each candidate change row, the first clause to evaluate as true is executed. No more than one WHEN clause is executed for any candidate change row.

MERGE actions have the same effect as regular UPDATE, INSERT, or DELETE commands of the same names. The syntax of those commands is different, notably that there is no WHERE clause and no table name is specified. All actions refer to the target table, though modifications to other tables may be made using triggers.

When DO NOTHING is specified, the source row is skipped. Since actions are evaluated in their specified order, DO NOTHING can be handy to skip non-interesting source rows before more fine-grained handling.

The optional RETURNING clause causes MERGE to compute and return value(s) based on each row inserted, updated, or deleted. Any expression using the source or target table's columns, or the merge_action() function can be computed. By default, when an INSERT or UPDATE action is performed, the new values of the target table's columns are used, and when a DELETE is performed, the old values of the target table's columns are used, but it is also possible to explicitly request old and new values. The syntax of the RETURNING list is identical to that of the output list of SELECT.

There is no separate MERGE privilege. If you specify an update action, you must have the UPDATE privilege on the column(s) of the target table that are referred to in the SET clause. If you specify an insert action, you must have the INSERT privilege on the target table. If you specify a delete action, you must have the DELETE privilege on the target table. If you specify a DO NOTHING action, you must have the SELECT privilege on at least one column of the target table. You will also need SELECT privilege on any column(s) of the <data_source> and of the target table referred to in any condition (including join_condition) or expression. Privileges are tested once at statement start and are checked whether or not particular WHEN clauses are executed.

MERGE is not supported if the target table is a materialized view, foreign table, or if it has any rules defined on it.

Parameters

<with_query>
The WITH clause allows you to specify one or more subqueries that can be referenced by name in the MERGE query. See ??? and ??? for details. Note that WITH RECURSIVE is not supported by MERGE.

<target_table_name>
The name (optionally schema-qualified) of the target table or view to merge into. If ONLY is specified before a table name, matching rows are updated or deleted in the named table only. If ONLY is not specified, matching rows are also updated or deleted in any tables inheriting from the named table. Optionally, * can be specified after the table name to explicitly indicate that descendant tables are included. The ONLY keyword and * option do not affect insert actions, which always insert into the named table only.

If <target_table_name> is a view, it must either be automatically updatable with no INSTEAD OF triggers, or it must have INSTEAD OF triggers for every type of action (INSERT, UPDATE, and DELETE) specified in the WHEN clauses. Views with rules are not supported.

<target_alias>
A substitute name for the target table. When an alias is provided, it completely hides the actual name of the table. For example, given MERGE INTO foo AS f, the remainder of the MERGE statement must refer to this table as f not foo.

<source_table_name>
The name (optionally schema-qualified) of the source table, view, or transition table. If ONLY is specified before the table name, matching rows are included from the named table only. If ONLY is not specified, matching rows are also included from any tables inheriting from the named table. Optionally, * can be specified after the table name to explicitly indicate that descendant tables are included.

<source_query>
A query (SELECT statement or VALUES statement) that supplies the rows to be merged into the target table. Refer to the ??? statement or ??? statement for a description of the syntax.

<source_alias>
A substitute name for the data source. When an alias is provided, it completely hides the actual name of the table or the fact that a query was issued.

<join_condition>
<join_condition> is an expression resulting in a value of type boolean (similar to a WHERE clause) that specifies which rows in the <data_source> match rows in the target table.

[!WARNING] Only columns from the target table that attempt to match <data_source> rows should appear in <join_condition>. <join_condition> subexpressions that only reference the target table's columns can affect which action is taken, often in surprising ways.

If both WHEN NOT MATCHED BY SOURCE and WHEN NOT MATCHED [BY TARGET] clauses are specified, the MERGE command will perform a FULL join between <data_source> and the target table. For this to work, at least one <join_condition> subexpression must use an operator that can support a hash join, or all of the subexpressions must use operators that can support a merge join.

<when_clause>
At least one WHEN clause is required.

The WHEN clause may specify WHEN MATCHED, WHEN NOT MATCHED BY SOURCE, or WHEN NOT MATCHED [BY TARGET]. Note that the SQL standard only defines WHEN MATCHED and WHEN NOT MATCHED (which is defined to mean no matching target row). WHEN NOT MATCHED BY SOURCE is an extension to the SQL standard, as is the option to append BY TARGET to WHEN NOT MATCHED, to make its meaning more explicit.

If the WHEN clause specifies WHEN MATCHED and the candidate change row matches a row in the <data_source> to a row in the target table, the WHEN clause is executed if the <condition> is absent or it evaluates to true.

If the WHEN clause specifies WHEN NOT MATCHED BY SOURCE and the candidate change row represents a row in the target table that does not match a row in the <data_source>, the WHEN clause is executed if the <condition> is absent or it evaluates to true.

If the WHEN clause specifies WHEN NOT MATCHED [BY TARGET] and the candidate change row represents a row in the <data_source> that does not match a row in the target table, the WHEN clause is executed if the <condition> is absent or it evaluates to true.

<condition>
An expression that returns a value of type boolean. If this expression for a WHEN clause returns true, then the action for that clause is executed for that row.

A condition on a WHEN MATCHED clause can refer to columns in both the source and the target relations. A condition on a WHEN NOT MATCHED BY SOURCE clause can only refer to columns from the target relation, since by definition there is no matching source row. A condition on a WHEN NOT MATCHED [BY TARGET] clause can only refer to columns from the source relation, since by definition there is no matching target row. Only the system attributes from the target table are accessible.

<merge_insert>
The specification of an INSERT action that inserts one row into the target table. The target column names can be listed in any order. If no list of column names is given at all, the default is all the columns of the table in their declared order.

Each column not present in the explicit or implicit column list will be filled with a default value, either its declared default value or null if there is none.

If the target table is a partitioned table, each row is routed to the appropriate partition and inserted into it. If the target table is a partition, an error will occur if any input row violates the partition constraint.

Column names may not be specified more than once. INSERT actions cannot contain sub-selects.

Only one VALUES clause can be specified. The VALUES clause can only refer to columns from the source relation, since by definition there is no matching target row.

<merge_update>
The specification of an UPDATE action that updates the current row of the target table. Column names may not be specified more than once.

Neither a table name nor a WHERE clause are allowed.

<merge_delete>
Specifies a DELETE action that deletes the current row of the target table. Do not include the table name or any other clauses, as you would normally do with a ??? command.

<column_name>
The name of a column in the target table. The column name can be qualified with a subfield name or array subscript, if needed. (Inserting into only some fields of a composite column leaves the other fields null.) Do not include the table's name in the specification of a target column.

OVERRIDING SYSTEM VALUE
Without this clause, it is an error to specify an explicit value (other than DEFAULT) for an identity column defined as GENERATED ALWAYS. This clause overrides that restriction.

OVERRIDING USER VALUE
If this clause is specified, then any values supplied for identity columns defined as GENERATED BY DEFAULT are ignored and the default sequence-generated values are applied.

DEFAULT VALUES
All columns will be filled with their default values. (An OVERRIDING clause is not permitted in this form.)

<expression>
An expression to assign to the column. If used in a WHEN MATCHED clause, the expression can use values from the original row in the target table, and values from the <data_source> row. If used in a WHEN NOT MATCHED BY SOURCE clause, the expression can only use values from the original row in the target table. If used in a WHEN NOT MATCHED [BY TARGET] clause, the expression can only use values from the <data_source> row.

DEFAULT
Set the column to its default value (which will be NULL if no specific default expression has been assigned to it).

<sub-SELECT>
A SELECT sub-query that produces as many output columns as are listed in the parenthesized column list preceding it. The sub-query must yield no more than one row when executed. If it yields one row, its column values are assigned to the target columns; if it yields no rows, NULL values are assigned to the target columns. If used in a WHEN MATCHED clause, the sub-query can refer to values from the original row in the target table, and values from the <data_source> row. If used in a WHEN NOT MATCHED BY SOURCE clause, the sub-query can only refer to values from the original row in the target table.

<output_alias>
An optional substitute name for OLD or NEW rows in the RETURNING list.

By default, old values from the target table can be returned by writing OLD.column_name or OLD.*, and new values can be returned by writing NEW.column_name or NEW.*. When an alias is provided, these names are hidden and the old or new rows must be referred to using the alias. For example RETURNING WITH (OLD AS o, NEW AS n) o.*, n.*.

<output_expression>
An expression to be computed and returned by the MERGE command after each row is changed (whether inserted, updated, or deleted). The expression can use any columns of the source or target tables, or the merge_action() function to return additional information about the action executed.

Writing * will return all columns from the source table, followed by all columns from the target table. Often this will lead to a lot of duplication, since it is common for the source and target tables to have a lot of the same columns. This can be avoided by qualifying the * with the name or alias of the source or target table.

A column name or * may also be qualified using OLD or NEW, or the corresponding <output_alias> for OLD or NEW, to cause old or new values from the target table to be returned. An unqualified column name from the target table, or a column name or * qualified using the target table name or alias will return new values for INSERT and UPDATE actions, and old values for DELETE actions.

<output_name>
A name to use for a returned column.

Outputs

On successful completion, a MERGE command returns a command tag of the form

MERGE total_count

The <total_count> is the total number of rows changed (whether inserted, updated, or deleted). If <total_count> is 0, no rows were changed in any way.

If the MERGE command contains a RETURNING clause, the result will be similar to that of a SELECT statement containing the columns and values defined in the RETURNING list, computed over the row(s) inserted, updated, or deleted by the command.

Notes

The following steps take place during the execution of MERGE.

  1. Perform any BEFORE STATEMENT triggers for all actions specified, whether or not their WHEN clauses match.

  2. Perform a join from source to target table. The resulting query will be optimized normally and will produce a set of candidate change rows. For each candidate change row,

    1. Evaluate whether each row is MATCHED, NOT MATCHED BY SOURCE, or NOT MATCHED [BY TARGET].

    2. Test each WHEN condition in the order specified until one returns true.

    3. When a condition returns true, perform the following actions:

      1. Perform any BEFORE ROW triggers that fire for the action's event type.

      2. Perform the specified action, invoking any check constraints on the target table.

      3. Perform any AFTER ROW triggers that fire for the action's event type.

      If the target relation is a view with INSTEAD OF ROW triggers for the action's event type, they are used to perform the action instead.

  3. Perform any AFTER STATEMENT triggers for actions specified, whether or not they actually occur. This is similar to the behavior of an UPDATE statement that modifies no rows.

In summary, statement triggers for an event type (say, INSERT) will be fired whenever we specify an action of that kind. In contrast, row-level triggers will fire only for the specific event type being executed. So a MERGE command might fire statement triggers for both UPDATE and INSERT, even though only UPDATE row triggers were fired.

You should ensure that the join produces at most one candidate change row for each target row. In other words, a target row shouldn't join to more than one data source row. If it does, then only one of the candidate change rows will be used to modify the target row; later attempts to modify the row will cause an error. This can also occur if row triggers make changes to the target table and the rows so modified are then subsequently also modified by MERGE. If the repeated action is an INSERT, this will cause a uniqueness violation, while a repeated UPDATE or DELETE will cause a cardinality violation; the latter behavior is required by the SQL standard. This differs from historical PostgreSQL behavior of joins in UPDATE and DELETE statements where second and subsequent attempts to modify the same row are simply ignored.

If a WHEN clause omits an AND sub-clause, it becomes the final reachable clause of that kind (MATCHED, NOT MATCHED BY SOURCE, or NOT MATCHED [BY TARGET]). If a later WHEN clause of that kind is specified it would be provably unreachable and an error is raised. If no final reachable clause is specified of either kind, it is possible that no action will be taken for a candidate change row.

The order in which rows are generated from the data source is indeterminate by default. A <source_query> can be used to specify a consistent ordering, if required, which might be needed to avoid deadlocks between concurrent transactions.

When MERGE is run concurrently with other commands that modify the target table, the usual transaction isolation rules apply; see ??? for an explanation on the behavior at each isolation level. You may also wish to consider using INSERT ... ON CONFLICT as an alternative statement which offers the ability to run an UPDATE or return the existing row (with DO SELECT) if a concurrent INSERT occurs. There are a variety of differences and restrictions between the two statement types and they are not interchangeable.

Examples

Perform maintenance on customer_accounts based upon new recent_transactions.

MERGE INTO customer_account ca
USING recent_transactions t
ON t.customer_id = ca.customer_id
WHEN MATCHED THEN
  UPDATE SET balance = balance + transaction_value
WHEN NOT MATCHED THEN
  INSERT (customer_id, balance)
  VALUES (t.customer_id, t.transaction_value);

Attempt to insert a new stock item along with the quantity of stock. If the item already exists, instead update the stock count of the existing item. Don't allow entries that have zero stock. Return details of all changes made.

MERGE INTO wines w
USING wine_stock_changes s
ON s.winename = w.winename
WHEN NOT MATCHED AND s.stock_delta > 0 THEN
  INSERT VALUES(s.winename, s.stock_delta)
WHEN MATCHED AND w.stock + s.stock_delta > 0 THEN
  UPDATE SET stock = w.stock + s.stock_delta
WHEN MATCHED THEN
  DELETE
RETURNING merge_action(), w.winename, old.stock AS old_stock, new.stock AS new_stock;

The wine_stock_changes table might be, for example, a temporary table recently loaded into the database.

Update wines based on a replacement wine list, inserting rows for any new stock, updating modified stock entries, and deleting any wines not present in the new list.

MERGE INTO wines w
USING new_wine_list s
ON s.winename = w.winename
WHEN NOT MATCHED BY TARGET THEN
  INSERT VALUES(s.winename, s.stock)
WHEN MATCHED AND w.stock != s.stock THEN
  UPDATE SET stock = s.stock
WHEN NOT MATCHED BY SOURCE THEN
  DELETE;

Compatibility

This command conforms to the SQL standard.

The WITH clause, BY SOURCE and BY TARGET qualifiers to WHEN NOT MATCHED, DO NOTHING action, and RETURNING clause are extensions to the SQL standard.

move

MOVE

cursor

MOVE

MOVE

7

SQL - Language Statements

MOVE

position a cursor

MOVE [

direction

] [ FROM | IN ]

cursor_name

where

direction

can be one of:

NEXT PRIOR FIRST LAST ABSOLUTE

count

RELATIVE

count

count

ALL FORWARD FORWARD

count

FORWARD ALL BACKWARD BACKWARD

count

BACKWARD ALL

Description

MOVE repositions a cursor without retrieving any data. MOVE works exactly like the FETCH command, except it only positions the cursor and does not return rows.

The parameters for the MOVE command are identical to those of the FETCH command; refer to ??? for details on syntax and usage.

Outputs

On successful completion, a MOVE command returns a command tag of the form

MOVE count

The <count> is the number of rows that a FETCH command with the same parameters would have returned (possibly zero).

Examples

BEGIN WORK;
DECLARE liahona CURSOR FOR SELECT * FROM films;

-- Skip the first 5 rows:
MOVE FORWARD 5 IN liahona;
MOVE 5

-- Fetch the 6th row from the cursor liahona:
FETCH 1 FROM liahona;
 code  | title  | did | date_prod  |  kind  |  len
-------+--------+-----+------------+--------+-------
 P_303 | 48 Hrs | 103 | 1982-10-22 | Action | 01:37
(1 row)

-- Close the cursor liahona and end the transaction:
CLOSE liahona;
COMMIT WORK;

Compatibility

There is no MOVE statement in the SQL standard.

See Also

notify

NOTIFY

NOTIFY

7

SQL - Language Statements

NOTIFY

generate a notification

NOTIFY

channel

[ ,

payload

]

Description

The NOTIFY command sends a notification event together with an optional “payload” string to each client application that has previously executed LISTEN channel for the specified channel name in the current database. Notifications are visible to all users.

NOTIFY provides a simple interprocess communication mechanism for a collection of processes accessing the same PostgreSQL database. A payload string can be sent along with the notification, and higher-level mechanisms for passing structured data can be built by using tables in the database to pass additional data from notifier to listener(s).

The information passed to the client for a notification event includes the notification channel name, the notifying session's server process PID, and the payload string, which is an empty string if it has not been specified.

It is up to the database designer to define the channel names that will be used in a given database and what each one means. Commonly, the channel name is the same as the name of some table in the database, and the notify event essentially means, “I changed this table, take a look at it to see what's new”. But no such association is enforced by the NOTIFY and LISTEN commands. For example, a database designer could use several different channel names to signal different sorts of changes to a single table. Alternatively, the payload string could be used to differentiate various cases.

When NOTIFY is used to signal the occurrence of changes to a particular table, a useful programming technique is to put the NOTIFY in a statement trigger that is triggered by table updates. In this way, notification happens automatically when the table is changed, and the application programmer cannot accidentally forget to do it.

NOTIFY interacts with SQL transactions in some important ways. Firstly, if a NOTIFY is executed inside a transaction, the notify events are not delivered until and unless the transaction is committed. This is appropriate, since if the transaction is aborted, all the commands within it have had no effect, including NOTIFY. But it can be disconcerting if one is expecting the notification events to be delivered immediately. Secondly, if a listening session receives a notification signal while it is within a transaction, the notification event will not be delivered to its connected client until just after the transaction is completed (either committed or aborted). Again, the reasoning is that if a notification were delivered within a transaction that was later aborted, one would want the notification to be undone somehow — but the server cannot “take back” a notification once it has sent it to the client. So notification events are only delivered between transactions. The upshot of this is that applications using NOTIFY for real-time signaling should try to keep their transactions short.

If the same channel name is signaled multiple times with identical payload strings within the same transaction, only one instance of the notification event is delivered to listeners. On the other hand, notifications with distinct payload strings will always be delivered as distinct notifications. Similarly, notifications from different transactions will never get folded into one notification. Except for dropping later instances of duplicate notifications, NOTIFY guarantees that notifications from the same transaction get delivered in the order they were sent. It is also guaranteed that messages from different transactions are delivered in the order in which the transactions committed.

It is common for a client that executes NOTIFY to be listening on the same notification channel itself. In that case it will get back a notification event, just like all the other listening sessions. Depending on the application logic, this could result in useless work, for example, reading a database table to find the same updates that that session just wrote out. It is possible to avoid such extra work by noticing whether the notifying session's server process PID (supplied in the notification event message) is the same as one's own session's PID (available from libpq). When they are the same, the notification event is one's own work bouncing back, and can be ignored.

Parameters

<channel>
Name of the notification channel to be signaled (any identifier).

<payload>
The “payload” string to be communicated along with the notification. This must be specified as a simple string literal. In the default configuration it must be shorter than 8000 bytes. (If binary data or large amounts of information need to be communicated, it's best to put it in a database table and send the key of the record.)

Notes

There is a queue that holds notifications that have been sent but not yet processed by all listening sessions. If this queue becomes full, transactions calling NOTIFY will fail at commit. The queue is quite large (8GB in a standard installation) and should be sufficiently sized for almost every use case. However, no cleanup can take place if a session executes LISTEN and then enters a transaction for a very long time. Once the queue is half full you will see warnings in the log file pointing you to the session that is preventing cleanup. In this case you should make sure that this session ends its current transaction so that cleanup can proceed.

The function pg_notification_queue_usage returns the fraction of the queue that is currently occupied by pending notifications. See ??? for more information.

A transaction that has executed NOTIFY cannot be prepared for two-phase commit.

pg_notify

pg_notify

To send a notification you can also use the function pg_notify(text, text). The function takes the channel name as the first argument and the payload as the second. The function is much easier to use than the NOTIFY command if you need to work with non-constant channel names and payloads.

Examples

Configure and execute a listen/notify sequence from psql:

LISTEN virtual;
NOTIFY virtual;
Asynchronous notification "virtual" received from server process with PID 8448.
NOTIFY virtual, 'This is the payload';
Asynchronous notification "virtual" with payload "This is the payload" received from server process with PID 8448.

LISTEN foo;
SELECT pg_notify('fo' || 'o', 'pay' || 'load');
Asynchronous notification "foo" with payload "payload" received from server process with PID 14728.

Compatibility

There is no NOTIFY statement in the SQL standard.

See Also

pg_amcheck

pg_amcheck

pg_amcheck

1

Application

pg_amcheck

checks for corruption in one or more

PostgreSQL

databases

pg_amcheck

option

dbname

Description

pg_amcheck supports running ???'s corruption checking functions against one or more databases, with options to select which schemas, tables and indexes to check, which kinds of checking to perform, and whether to perform the checks in parallel, and if so, the number of parallel connections to establish and use.

Only ordinary and TOAST table relations, materialized views, sequences, and btree indexes are currently supported. Other relation types are silently skipped.

If dbname is specified, it should be the name of a single database to check, and no other database selection options should be present. Otherwise, if any database selection options are present, all matching databases will be checked. If no such options are present, the default database will be checked. Database selection options include --all, --database and --exclude-database. They also include --relation, --exclude-relation, --table, --exclude-table, --index, and --exclude-index, but only when such options are used with a three-part pattern (e.g. mydb*.myschema*.myrel*). Finally, they include --schema and --exclude-schema when such options are used with a two-part pattern (e.g. mydb*.myschema*).

<dbname> can also be a connection string.

Options

The following command-line options control what is checked:

-a; --all
Check all databases, except for any excluded via --exclude-database.

-d pattern; --database=pattern
Check databases matching the specified <pattern>, except for any excluded by --exclude-database. This option can be specified more than once.

-D pattern; --exclude-database=pattern
Exclude databases matching the given <pattern>. This option can be specified more than once.

-i pattern; --index=pattern
Check indexes matching the specified <pattern>, unless they are otherwise excluded. This option can be specified more than once.

This is similar to the --relation option, except that it applies only to indexes, not to other relation types.

-I pattern; --exclude-index=pattern
Exclude indexes matching the specified <pattern>. This option can be specified more than once.

This is similar to the --exclude-relation option, except that it applies only to indexes, not other relation types.

-r pattern; --relation=pattern
Check relations matching the specified <pattern>, unless they are otherwise excluded. This option can be specified more than once.

Patterns may be unqualified, e.g. myrel*, or they may be schema-qualified, e.g. myschema*.myrel* or database-qualified and schema-qualified, e.g. mydb*.myschema*.myrel*. A database-qualified pattern will add matching databases to the list of databases to be checked.

-R pattern; --exclude-relation=pattern
Exclude relations matching the specified <pattern>. This option can be specified more than once.

As with --relation, the <pattern> may be unqualified, schema-qualified, or database- and schema-qualified.

-s pattern; --schema=pattern
Check tables and indexes in schemas matching the specified <pattern>, unless they are otherwise excluded. This option can be specified more than once.

To select only tables in schemas matching a particular pattern, consider using something like --table=SCHEMAPAT.* --no-dependent-indexes. To select only indexes, consider using something like --index=SCHEMAPAT.*.

A schema pattern may be database-qualified. For example, you may write --schema=mydb*.myschema* to select schemas matching myschema* in databases matching mydb*.

-S pattern; --exclude-schema=pattern
Exclude tables and indexes in schemas matching the specified <pattern>. This option can be specified more than once.

As with --schema, the pattern may be database-qualified.

-t pattern; --table=pattern
Check tables matching the specified <pattern>, unless they are otherwise excluded. This option can be specified more than once.

This is similar to the --relation option, except that it applies only to tables, materialized views, and sequences, not to indexes.

-T pattern; --exclude-table=pattern
Exclude tables matching the specified <pattern>. This option can be specified more than once.

This is similar to the --exclude-relation option, except that it applies only to tables, materialized views, and sequences, not to indexes.

--no-dependent-indexes
By default, if a table is checked, any btree indexes of that table will also be checked, even if they are not explicitly selected by an option such as --index or --relation. This option suppresses that behavior.

--no-dependent-toast
By default, if a table is checked, its TOAST table, if any, will also be checked, even if it is not explicitly selected by an option such as --table or --relation. This option suppresses that behavior.

--no-strict-names
By default, if an argument to --database, --table, --index, or --relation matches no objects, it is a fatal error. This option downgrades that error to a warning.

The following command-line options control checking of tables:

--exclude-toast-pointers
By default, whenever a TOAST pointer is encountered in a table, a lookup is performed to ensure that it references apparently-valid entries in the TOAST table. These checks can be quite slow, and this option can be used to skip them.

--on-error-stop
After reporting all corruptions on the first page of a table where corruption is found, stop processing that table relation and move on to the next table or index.

Note that index checking always stops after the first corrupt page. This option only has meaning relative to table relations.

--skip=option
If all-frozen is given, table corruption checks will skip over pages in all tables that are marked as all frozen.

If all-visible is given, table corruption checks will skip over pages in all tables that are marked as all visible.

By default, no pages are skipped. This can be specified as none, but since this is the default, it need not be mentioned.

--startblock=block
Start checking at the specified block number. An error will occur if the table relation being checked has fewer than this number of blocks. This option does not apply to indexes, and is probably only useful when checking a single table relation. See --endblock for further caveats.

--endblock=block
End checking at the specified block number. An error will occur if the table relation being checked has fewer than this number of blocks. This option does not apply to indexes, and is probably only useful when checking a single table relation. If both a regular table and a TOAST table are checked, this option will apply to both, but higher-numbered TOAST blocks may still be accessed while validating TOAST pointers, unless that is suppressed using --exclude-toast-pointers.

The following command-line options control checking of B-tree indexes:

--checkunique
For each index with unique constraint checked, verify that no more than one among duplicate entries is visible in the index using ???'s checkunique option.

--heapallindexed
For each index checked, verify the presence of all heap tuples as index tuples in the index using ???'s heapallindexed option.

--parent-check
For each btree index checked, use ???'s bt_index_parent_check function, which performs additional checks of parent/child relationships during index checking.

The default is to use amcheck's bt_index_check function, but note that use of the --rootdescend option implicitly selects bt_index_parent_check.

--rootdescend
For each index checked, re-find tuples on the leaf level by performing a new search from the root page for each tuple using ???'s rootdescend option.

Use of this option implicitly also selects the --parent-check option.

This form of verification was originally written to help in the development of btree index features. It may be of limited use or even of no use in helping detect the kinds of corruption that occur in practice. It may also cause corruption checking to take considerably longer and consume considerably more resources on the server.

[!WARNING] The extra checks performed against B-tree indexes when the --parent-check option or the --rootdescend option is specified require relatively strong relation-level locks. These checks are the only checks that will block concurrent data modification from INSERT, UPDATE, and DELETE commands.

The following command-line options control the connection to the server:

-h hostname; --host=hostname
Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for the Unix domain socket.

-p port; --port=port
Specifies the TCP port or local Unix domain socket file extension on which the server is listening for connections.

-U; --username=username
User name to connect as.

-w; --no-password
Never issue a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password.

-W; --password
Force pg_amcheck to prompt for a password before connecting to a database.

This option is never essential, since pg_amcheck will automatically prompt for a password if the server demands password authentication. However, pg_amcheck will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt.

--maintenance-db=dbname
Specifies a database or connection string to be used to discover the list of databases to be checked. If neither --all nor any option including a database pattern is used, no such connection is required and this option does nothing. Otherwise, any connection string parameters other than the database name which are included in the value for this option will also be used when connecting to the databases being checked. If this option is omitted, the default is postgres or, if that fails, template1.

Other options are also available:

-e; --echo
Echo to stdout all SQL sent to the server.

-j num; --jobs=num
Use <num> concurrent connections to the server, or one per object to be checked, whichever is less.

The default is to use a single connection.

-P; --progress
Show progress information. Progress information includes the number of relations for which checking has been completed, and the total size of those relations. It also includes the total number of relations that will eventually be checked, and the estimated size of those relations.

-v; --verbose
Print more messages. In particular, this will print a message for each relation being checked, and will increase the level of detail shown for server errors.

-V; --version
Print the pg_amcheck version and exit.

--install-missing; --install-missing=schema
Install any missing extensions that are required to check the database(s). If not yet installed, each extension's objects will be installed into the given <schema>, or if not specified into schema pg_catalog.

At present, the only required extension is ???.

-?; --help
Show help about pg_amcheck command line arguments, and exit.

Environment

pg_amcheck, like most other PostgreSQL utilities, also uses the environment variables supported by libpq (see ???).

The environment variable PG_COLOR specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

Notes

pg_amcheck is designed to work with PostgreSQL 14.0 and later.

See Also

pg_basebackup

pg_basebackup

pg_basebackup

1

Application

pg_basebackup

take a base backup of a

PostgreSQL

cluster

pg_basebackup

option

Description

pg_basebackup is used to take a base backup of a running PostgreSQL database cluster. The backup is taken without affecting other clients of the database, and can be used both for point-in-time recovery (see ???) and as the starting point for a log-shipping or streaming-replication standby server (see ???).

pg_basebackup can take a full or incremental base backup of the database. When used to take a full backup, it makes an exact copy of the database cluster's files. When used to take an incremental backup, some files that would have been part of a full backup may be replaced with incremental versions of the same files, containing only those blocks that have been modified since the reference backup. An incremental backup cannot be used directly; instead, ??? must first be used to combine it with the previous backups upon which it depends. See ??? for more information about incremental backups, and ??? for steps to recover from a backup.

In any mode, pg_basebackup makes sure the server is put into and out of backup mode automatically. Backups are always taken of the entire database cluster; it is not possible to back up individual databases or database objects. For selective backups, another tool such as ??? must be used.

The backup is made over a regular PostgreSQL connection that uses the replication protocol. The connection must be made with a user ID that has REPLICATION permissions (see ???) or is a superuser, and pg_hba.conf must permit the replication connection. The server must also be configured with ??? set high enough to provide at least one walsender for the backup plus one for WAL streaming (if used).

There can be multiple pg_basebackups running at the same time, but it is usually better from a performance point of view to take only one backup, and copy the result.

pg_basebackup can make a base backup from not only a primary server but also a standby. To take a backup from a standby, set up the standby so that it can accept replication connections (that is, set max_wal_senders and ???, and configure its pg_hba.conf appropriately). You will also need to enable ??? on the primary.

Note that there are some limitations in taking a backup from a standby:

Whenever pg_basebackup is taking a base backup, the server's pg_stat_progress_basebackup view will report the progress of the backup. See ??? for details.

Options

The following command-line options control the location and format of the output:

-D directory; --pgdata=directory
Sets the target directory to write the output to. pg_basebackup will create this directory (and any missing parent directories) if it does not exist. If it already exists, it must be empty.

When the backup is in tar format, the target directory may be specified as - (dash), causing the tar file to be written to stdout.

This option is required.

-F format; --format=format
Selects the format for the output. <format> can be one of the following:

p; plain
Write the output as plain files, with the same layout as the source server's data directory and tablespaces. When the cluster has no additional tablespaces, the whole database will be placed in the target directory. If the cluster contains additional tablespaces, the main data directory will be placed in the target directory, but all other tablespaces will be placed in the same absolute path as they have on the source server. (See --tablespace-mapping to change that.)

This is the default format.

t; tar
Write the output as tar files in the target directory. The main data directory's contents will be written to a file named base.tar, and each other tablespace will be written to a separate tar file named after that tablespace's OID.

If the target directory is specified as - (dash), the tar contents will be written to standard output, suitable for piping to (for example) gzip. This is only allowed if the cluster has no additional tablespaces and WAL streaming is not used.

-i old_manifest_file; --incremental=old_manifest_file
Performs an incremental backup. The backup manifest for the reference backup must be provided, and will be uploaded to the server, which will respond by sending the requested incremental backup.

-R; --write-recovery-conf
Creates a standby.signal file and appends connection settings to the postgresql.auto.conf file in the target directory (or within the base archive file when using tar format). This eases setting up a standby server using the results of the backup.

The postgresql.auto.conf file will record the connection settings and, if specified, the replication slot that pg_basebackup is using, so that streaming replication and logical replication slot synchronization will use the same settings later on. The dbname will be recorded only if the dbname was specified explicitly in the connection string or environment variable.

-t target; --target=target
Instructs the server where to place the base backup. The default target is client, which specifies that the backup should be sent to the machine where pg_basebackup is running. If the target is instead set to server:/some/path, the backup will be stored on the machine where the server is running in the /some/path directory. Storing a backup on the server requires superuser privileges or having privileges of the pg_write_server_files role. If the target is set to blackhole, the contents are discarded and not stored anywhere. This should only be used for testing purposes, as you will not end up with an actual backup.

Since WAL streaming is implemented by pg_basebackup rather than by the server, this option cannot be used together with -Xstream. Since that is the default, when this option is specified, you must also specify either -Xfetch or -Xnone.

-T olddir=newdir; --tablespace-mapping=olddir=newdir
Relocates the tablespace in directory <olddir> to <newdir> during the backup. To be effective, <olddir> must exactly match the path specification of the tablespace as it is defined on the source server. (But it is not an error if there is no tablespace in <olddir> on the source server.) Meanwhile <newdir> is a directory in the receiving host's filesystem. As with the main target directory, <newdir> need not exist already, but if it does exist it must be empty. Both <olddir> and <newdir> must be absolute paths. If either path needs to contain an equal sign (=), precede that with a backslash. This option can be specified multiple times for multiple tablespaces.

If a tablespace is relocated in this way, the symbolic links inside the main data directory are updated to point to the new location. So the new data directory is ready to be used for a new server instance with all tablespaces in the updated locations.

Currently, this option only works with plain output format; it is ignored if tar format is selected.

--waldir=waldir
Sets the directory to write WAL (write-ahead log) files to. By default WAL files will be placed in the pg_wal subdirectory of the target directory, but this option can be used to place them elsewhere. <waldir> must be an absolute path. As with the main target directory, <waldir> need not exist already, but if it does exist it must be empty. This option can only be specified when the backup is in plain format.

-X method; --wal-method=method
Includes the required WAL (write-ahead log) files in the backup. This will include all write-ahead logs generated during the backup. Unless the method none is specified, it is possible to start a postmaster in the target directory without the need to consult the WAL archive, thus making the output a completely standalone backup.

The following <method>s for collecting the write-ahead logs are supported:

n; none
Don't include write-ahead logs in the backup.

f; fetch
The write-ahead log files are collected at the end of the backup. Therefore, it is necessary for the source server's ??? parameter to be set high enough that the required log data is not removed before the end of the backup. If the required log data has been recycled before it's time to transfer it, the backup will fail and be unusable.

When tar format is used, the write-ahead log files will be included in the base.tar file.

s; stream
Stream write-ahead log data while the backup is being taken. This method will open a second connection to the server and start streaming the write-ahead log in parallel while running the backup. Therefore, it will require two replication connections not just one. As long as the client can keep up with the write-ahead log data, using this method requires no extra write-ahead logs to be saved on the source server.

When tar format is used, the write-ahead log files will be written to a separate file named pg_wal.tar (if the server is a version earlier than 10, the file will be named pg_xlog.tar).

This value is the default.

-z; --gzip
Enables gzip compression of tar file output, with the default compression level. Compression is only available when using the tar format, and the suffix .gz will automatically be added to all tar filenames.

-Z level; -Z [{client|server}-]method[:detail]; --compress=level; --compress=[{client|server}-]method[:detail]
Requests compression of the backup. If client or server is included, it specifies where the compression is to be performed. Compressing on the server will reduce transfer bandwidth but will increase server CPU consumption. The default is client except when --target is used. In that case, the backup is not being sent to the client, so only server compression is sensible. When -Xstream, which is the default, is used, server-side compression will not be applied to the WAL. To compress the WAL, use client-side compression, or specify -Xfetch.

The compression method can be set to gzip, lz4, zstd, none for no compression or an integer (no compression if 0, gzip if greater than 0). A compression detail string can optionally be specified. If the detail string is an integer, it specifies the compression level. Otherwise, it should be a comma-separated list of items, each of the form <keyword> or <keyword=value>. Currently, the supported keywords are level, long, and workers. The detail string cannot be used when the compression method is specified as a plain integer.

If no compression level is specified, the default compression level will be used. If only a level is specified without mentioning an algorithm, gzip compression will be used if the level is greater than 0, and no compression will be used if the level is 0.

When the tar format is used with gzip, lz4, or zstd, the suffix .gz, .lz4, or .zst, respectively, will be automatically added to all tar filenames. When the plain format is used, client-side compression may not be specified, but it is still possible to request server-side compression. If this is done, the server will compress the backup for transmission, and the client will decompress and extract it.

When this option is used in combination with -Xstream, pg_wal.tar will be compressed using gzip if client-side gzip compression is selected, but will not be compressed if any other compression algorithm is selected, or if server-side compression is selected.

The following command-line options control the generation of the backup and the invocation of the program:

-c {fast|spread}; --checkpoint={fast|spread}
Sets checkpoint mode to fast or spread (see ???). The default is spread.

-C; --create-slot
Specifies that the replication slot named by the --slot option should be created before starting the backup. An error is raised if the slot already exists.

-l label; --label=label
Sets the label for the backup. If none is specified, a default value of “pg_basebackup base backup” will be used.

-n; --no-clean
By default, when pg_basebackup aborts with an error, it removes any directories it might have created before discovering that it cannot finish the job (for example, the target directory and write-ahead log directory). This option inhibits tidying-up and is thus useful for debugging.

Note that tablespace directories are not cleaned up either way.

-N; --no-sync
By default, pg_basebackup will wait for all files to be written safely to disk. This option causes pg_basebackup to return without waiting, which is faster, but means that a subsequent operating system crash can leave the base backup corrupt. Generally, this option is useful for testing but should not be used when creating a production installation.

-P; --progress
Enables progress reporting. Turning this on will deliver an approximate progress report during the backup. Since the database may change during the backup, this is only an approximation and may not end at exactly 100%. In particular, when WAL log is included in the backup, the total amount of data cannot be estimated in advance, and in this case the estimated target size will increase once it passes the total estimate without WAL.

-r rate; --max-rate=rate
Sets the maximum transfer rate at which data is collected from the source server. This can be useful to limit the impact of pg_basebackup on the server. Values are in kilobytes per second. Use a suffix of M to indicate megabytes per second. A suffix of k is also accepted, and has no effect. Valid values are between 32 kilobytes per second and 1024 megabytes per second.

This option always affects transfer of the data directory. Transfer of WAL files is only affected if the collection method is fetch.

-S slotname; --slot=slotname
This option can only be used together with -X stream. It causes WAL streaming to use the specified replication slot. If the base backup is intended to be used as a streaming-replication standby using a replication slot, the standby should then use the same replication slot name as ???. This ensures that the primary server does not remove any necessary WAL data in the time between the end of the base backup and the start of streaming replication on the new standby.

The specified replication slot has to exist unless the option -C is also used.

If this option is not specified and the server supports temporary replication slots (version 10 and later), then a temporary replication slot is automatically used for WAL streaming.

--sync-method=method
When set to fsync, which is the default, pg_basebackup will recursively open and synchronize all files in the backup directory. When the plain format is used, the search for files will follow symbolic links for the WAL directory and each configured tablespace.

On Linux, syncfs may be used instead to ask the operating system to synchronize the whole file system that contains the backup directory. When the plain format is used, pg_basebackup will also synchronize the file systems that contain the WAL files and each tablespace. See ??? for information about the caveats to be aware of when using syncfs.

This option has no effect when --no-sync is used.

-v; --verbose
Enables verbose mode. Will output some extra steps during startup and shutdown, as well as show the exact file name that is currently being processed if progress reporting is also enabled.

--manifest-checksums=algorithm
Specifies the checksum algorithm that should be applied to each file included in the backup manifest. Currently, the available algorithms are NONE, CRC32C, SHA224, SHA256, SHA384, and SHA512. The default is CRC32C.

If NONE is selected, the backup manifest will not contain any checksums. Otherwise, it will contain a checksum of each file in the backup using the specified algorithm. In addition, the manifest will always contain a SHA256 checksum of its own contents. The SHA algorithms are significantly more CPU-intensive than CRC32C, so selecting one of them may increase the time required to complete the backup.

Using a SHA hash function provides a cryptographically secure digest of each file for users who wish to verify that the backup has not been tampered with, while the CRC-32C algorithm provides a checksum that is much faster to calculate; it is good at catching errors due to accidental changes but is not resistant to malicious modifications. Note that, to be useful against an adversary who has access to the backup, the backup manifest would need to be stored securely elsewhere or otherwise verified not to have been modified since the backup was taken.

??? can be used to check the integrity of a backup against the backup manifest.

--manifest-force-encode
Forces all filenames in the backup manifest to be hex-encoded. If this option is not specified, only non-UTF8 filenames are hex-encoded. This option is mostly intended to test that tools which read a backup manifest file properly handle this case.

--no-estimate-size
Prevents the server from estimating the total amount of backup data that will be streamed, resulting in the backup_total column in the pg_stat_progress_basebackup view always being NULL.

Without this option, the backup will start by estimating the total amount of data that will be streamed, and then go back and send the actual contents. This may make the backup take slightly longer, and in particular it will take longer before the first data is sent. This option is useful to avoid such estimation time if it's too long.

This option is not allowed when using --progress.

--no-manifest
Disables generation of a backup manifest. If this option is not specified, the server will generate and send a backup manifest which can be verified using ???. The manifest is a list of every file present in the backup with the exception of any WAL files that may be included. It also stores the size, last modification time, and an optional checksum for each file.

--no-slot
Prevents the creation of a temporary replication slot for the backup.

By default, if log streaming is selected but no slot name is given with the -S option, then a temporary replication slot is created (if supported by the source server).

The main purpose of this option is to allow taking a base backup when the server has no free replication slots. Using a replication slot is almost always preferred, because it prevents needed WAL from being removed by the server during the backup.

--no-verify-checksums
Disables verification of checksums, if they are enabled on the server the base backup is taken from.

By default, checksums are verified and checksum failures will result in a non-zero exit status. However, the base backup will not be removed in such a case, as if the --no-clean option had been used. Checksum verification failures will also be reported in the pg_stat_database view.

Checksums must be enabled on the server for the duration of the base backup in order to be verified. If checksums are in the process of being enabled when the base backup starts then checksum verification is disabled for the base backup.

The following command-line options control the connection to the source server:

-d connstr; --dbname=connstr
Specifies parameters used to connect to the server, as a connection string; these will override any conflicting command line options.

This option is called --dbname for consistency with other client applications, but because pg_basebackup doesn't connect to any particular database in the cluster, any database name included in the connection string will be ignored by the server. However, a database name supplied that way overrides the default database name (replication) for purposes of looking up the replication connection's password in ~/.pgpass. Similarly, middleware or proxies used in connecting to PostgreSQL might utilize the name for purposes such as connection routing. The database name can also be used by logical replication slot synchronization.

-h host; --host=host
Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for a Unix domain socket. The default is taken from the PGHOST environment variable, if set, else a Unix domain socket connection is attempted.

-p port; --port=port
Specifies the TCP port or local Unix domain socket file extension on which the server is listening for connections. Defaults to the PGPORT environment variable, if set, or a compiled-in default.

-s interval; --status-interval=interval
Specifies the number of seconds between status packets sent back to the source server. Smaller values allow more accurate monitoring of backup progress from the server. A value of zero disables periodic status updates completely, although an update will still be sent when requested by the server, to avoid timeout-based disconnects. The default value is 10 seconds.

-U username; --username=username
Specifies the user name to connect as.

-w; --no-password
Prevents issuing a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password.

-W; --password
Forces pg_basebackup to prompt for a password before connecting to the source server.

This option is never essential, since pg_basebackup will automatically prompt for a password if the server demands password authentication. However, pg_basebackup will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt.

Other options are also available:

-V; --version
Prints the pg_basebackup version and exits.

-?; --help
Shows help about pg_basebackup command line arguments, and exits.

Environment

This utility, like most other PostgreSQL utilities, uses the environment variables supported by libpq (see ???).

The environment variable PG_COLOR specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

Notes

At the beginning of the backup, a checkpoint needs to be performed on the source server. This can take some time (especially if the option --checkpoint=fast is not used), during which pg_basebackup will appear to be idle.

The backup will include all files in the data directory and tablespaces, including the configuration files and any additional files placed in the directory by third parties, except certain temporary files managed by PostgreSQL and operating system files. But only regular files and directories are copied, except that symbolic links used for tablespaces are preserved. Symbolic links pointing to certain directories known to PostgreSQL are copied as empty directories. Other symbolic links and special device files are skipped. See ??? for the precise details.

In plain format, tablespaces will be backed up to the same path they have on the source server, unless the option --tablespace-mapping is used. Without this option, running a plain format base backup on the same host as the server will not work if tablespaces are in use, because the backup would have to be written to the same directory locations as the original tablespaces.

When tar format is used, it is the user's responsibility to unpack each tar file before starting a PostgreSQL server that uses the data. If there are additional tablespaces, the tar files for them need to be unpacked in the correct locations. In this case the symbolic links for those tablespaces will be created by the server according to the contents of the tablespace_map file that is included in the base.tar file.

pg_basebackup works with servers of the same or older major version, down to 9.1. However, WAL streaming mode (-X stream) only works with server version 9.3 and later, the tar format (--format=tar) only works with server version 9.5 and later, and incremental backup (--incremental) only works with server version 17 and later.

pg_basebackup will preserve group permissions for data files if group permissions are enabled on the source cluster.

Examples

To create a base backup of the server at mydbserver and store it in the local directory /usr/local/pgsql/data:

$ pg_basebackup -h mydbserver -D /usr/local/pgsql/data

To create a backup of the local server with one compressed tar file for each tablespace, and store it in the directory backup, showing a progress report while running:

$ pg_basebackup -D backup -Ft -z -P

To create a backup of a single-tablespace local database and compress this with bzip2:

$ pg_basebackup -D - -Ft -X fetch | bzip2 > backup.tar.bz2

(This command will fail if there are multiple tablespaces in the database.)

To create a backup of a local database where the tablespace in /opt/ts is relocated to ./backup/ts:

$ pg_basebackup -D backup/data -T /opt/ts=$(pwd)/backup/ts

To create a backup of the local server with one tar file for each tablespace compressed with gzip at level 9, stored in the directory backup:

$ pg_basebackup -D backup -Ft --compress=gzip:9

See Also

pg_checksums

pg_checksums

pg_checksums

1

Application

pg_checksums

enable, disable or check data checksums in a

PostgreSQL

database cluster

pg_checksums

option

-D

--pgdata

datadir

Description

pg_checksums checks, enables or disables data checksums in a PostgreSQL cluster. The server must be shut down cleanly before running pg_checksums. Checksums can also be enabled while the cluster is running using ??? processing, see ??? for further details on the different approaches. When verifying checksums, the exit status is zero if there are no checksum errors, and nonzero if at least one checksum failure is detected. When enabling or disabling checksums, the exit status is nonzero if the operation failed.

If checksums were in the process of being enabled or disabled using ??? when the cluster was shut down, the control file still records that interrupted state, and pg_checksums refuses to run in any mode. Start the cluster and shut it down cleanly to reset the state, then retry; on a standby, let replication complete the transition first.

When verifying checksums, every file in the cluster is scanned. When enabling checksums, each relation file block with a changed checksum is rewritten in-place. Disabling checksums only updates the file pg_control.

Options

The following command-line options are available:

-D datadir; --pgdata=datadir
Specifies the directory where the database cluster is stored.

-c; --check
Checks checksums. This is the default mode if nothing else is specified.

-d; --disable
Disables checksums.

-e; --enable
Enables checksums.

-f filenode; --filenode=filenode
Only validate checksums in the relation with filenode <filenode>.

-N; --no-sync
By default, pg_checksums will wait for all files to be written safely to disk. This option causes pg_checksums to return without waiting, which is faster, but means that a subsequent operating system crash can leave the updated data directory corrupt. Generally, this option is useful for testing but should not be used on a production installation. This option has no effect when using --check.

-P; --progress
Enable progress reporting. Turning this on will deliver a progress report while checking or enabling checksums.

--sync-method=method
When set to fsync, which is the default, pg_checksums will recursively open and synchronize all files in the data directory. The search for files will follow symbolic links for the WAL directory and each configured tablespace.

On Linux, syncfs may be used instead to ask the operating system to synchronize the whole file systems that contain the data directory, the WAL files, and each tablespace. See ??? for information about the caveats to be aware of when using syncfs.

This option has no effect when --no-sync is used.

-v; --verbose
Enable verbose output. Lists all checked files.

-V; --version
Print the pg_checksums version and exit.

-?; --help
Show help about pg_checksums command line arguments, and exit.

Environment

PGDATA
Specifies the directory where the database cluster is stored; can be overridden using the -D option.

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

Notes

Enabling checksums in a large cluster can potentially take a long time. During this operation, the cluster or other programs that write to the data directory must not be started or else data loss may occur.

Enabling or disabling checksums with pg_checksums changes only the local data directory; the new state is not replicated to any other node. In a replication setup the same change must be applied to every node:

  1. All nodes participating in the replication must be stopped with a clean shutdown; pg_checksums refuses to run on a data directory left behind by an immediate shutdown. Before stopping a standby, make sure it has replayed all WAL of the primary: stop the primary first, read its “Latest checkpoint location” with ???, and check that pg_last_wal_replay_lsn() on the standby has advanced past it. Comparing the replay position with pg_last_wal_receive_lsn() is not enough, as it only shows that the WAL the standby received has been replayed.

  2. Run pg_checksums on the data directory of each node in the replication setup. Nodes can be processed in parallel while they are shut down. Processing must complete successfully on all nodes before continuing.

  3. Start the nodes normally, verify that ??? matches on all of them, and monitor the logs of the standbys for data checksum state mismatch warnings.

The replay requirement in step_title exists because an offline change is recorded only in the control file and has no defined ordering against WAL the node has not replayed yet; see ???. A node stopped before replaying an online checksum state change applies that change when it is restarted, overriding the offline change, and the states of the nodes silently diverge until a later checkpoint record triggers the warning described below. Because of this it is best not to mix the two mechanisms: change the state of a replication setup either with the offline procedure above or with an online transition, and make sure the previous change has reached every node before starting the next one.

If the change is applied inconsistently, each node keeps its own state, and a standby logs a warning when the state recorded in the replayed WAL differs from its own. The same warning can appear transiently while a standby catches up over WAL written before a consistent change; it stops once a checkpoint record carrying the new state has been replayed.

A standby whose data directory was never checksummed must not have checksums enabled by catching up this way. Converge the cluster by running pg_checksums on it while stopped as outlined above, by enabling checksums online, or by recreating it from a base backup. Note that an online enable only starts from a primary whose checksums are off, so if they are already enabled there, disable them online first.

If pg_checksums is aborted or killed while enabling or disabling checksums, the cluster's data checksum configuration remains unchanged, and pg_checksums can be re-run to perform the same operation.

Tools that copy relation file blocks directly between nodes, such as ???, require all nodes to be in the same data checksum state, else there is risk for data corruption.

The target cluster must have the same major version as pg_checksums.

pg_combinebackup

pg_combinebackup

pg_combinebackup

1

Application

pg_combinebackup

reconstruct a full backup from an incremental backup and dependent backups

pg_combinebackup

option

backup_directory

Description

pg_combinebackup is used to reconstruct a synthetic full backup from an incremental backup and the earlier backups upon which it depends.

Specify all of the required backups on the command line from oldest to newest. That is, the first backup directory should be the path to the full backup, and the last should be the path to the final incremental backup that you wish to restore. The reconstructed backup will be written to the output directory specified by the -o option.

pg_combinebackup will attempt to verify that the backups you specify form a legal backup chain from which a correct full backup can be reconstructed. However, it is not designed to help you keep track of which backups depend on which other backups. If you remove one or more of the previous backups upon which your incremental backup relies, you will not be able to restore it. Moreover, pg_combinebackup only attempts to verify that the backups have the correct relationship to each other, not that each individual backup is intact; for that, use ???.

Since the output of pg_combinebackup is a synthetic full backup, it can be used as an input to a future invocation of pg_combinebackup. The synthetic full backup would be specified on the command line in lieu of the chain of backups from which it was reconstructed.

Options

-d; --debug
Print lots of debug logging output on stderr.

-k; --link
Use hard links instead of copying files to the synthetic backup. Reconstruction of the synthetic backup might be faster (no file copying) and use less disk space, but care must be taken when using the output directory, because any modifications to that directory (for example, starting the server) can also affect the input directories. Likewise, changes to the input directories (for example, starting the server on the full backup) could affect the output directory. Thus, this option is best used when the input directories are only copies that will be removed after pg_combinebackup has completed.

Requires that the input backups and the output directory are in the same file system.

If a backup manifest is not available or does not contain checksum of the right type, hard links will still be created, but the file will be also read block-by-block for the checksum calculation.

-n; --dry-run
The -n/--dry-run option instructs pg_combinebackup to figure out what would be done without actually creating the target directory or any output files. It is particularly useful in combination with --debug.

-N; --no-sync
By default, pg_combinebackup will wait for all files to be written safely to disk. This option causes pg_combinebackup to return without waiting, which is faster, but means that a subsequent operating system crash can leave the output backup corrupt. Generally, this option is useful for testing but should not be used when creating a production installation.

-o outputdir; --output=outputdir
Specifies the output directory to which the synthetic full backup should be written. Currently, this argument is required.

-T olddir=newdir; --tablespace-mapping=olddir=newdir
Relocates the tablespace in directory <olddir> to <newdir> during the backup. <olddir> is the absolute path of the tablespace as it exists in the final backup specified on the command line, and <newdir> is the absolute path to use for the tablespace in the reconstructed backup. If either path needs to contain an equal sign (=), precede that with a backslash. This option can be specified multiple times for multiple tablespaces.

--clone
Use efficient file cloning (also known as “reflinks” on some systems) instead of copying files to the new data directory, which can result in near-instantaneous copying of the data files.

If a backup manifest is not available or does not contain checksum of the right type, file cloning will be used to copy the file, but the file will be also read block-by-block for the checksum calculation.

File cloning is only supported on some operating systems and file systems. If it is selected but not supported, the pg_combinebackup run will error. At present, it is supported on Linux (kernel 4.5 or later) with Btrfs and XFS (on file systems created with reflink support), and on macOS with APFS.

--copy
Perform regular file copy. This is the default. (See also --copy-file-range, --clone, and -k/--link.)

--copy-file-range
Use the copy_file_range system call for efficient copying. On some file systems this gives results similar to --clone, sharing physical disk blocks, while on others it may still copy blocks, but do so via an optimized path. At present, it is supported on Linux and FreeBSD.

If a backup manifest is not available or does not contain checksum of the right type, copy_file_range will be used to copy the file, but the file will be also read block-by-block for the checksum calculation.

--manifest-checksums=algorithm
Like ???, pg_combinebackup writes a backup manifest in the output directory. This option specifies the checksum algorithm that should be applied to each file included in the backup manifest. Currently, the available algorithms are NONE, CRC32C, SHA224, SHA256, SHA384, and SHA512. The default is CRC32C.

--no-manifest
Disables generation of a backup manifest. If this option is not specified, a backup manifest for the reconstructed backup will be written to the output directory.

--sync-method=method
When set to fsync, which is the default, pg_combinebackup will recursively open and synchronize all files in the backup directory. When the plain format is used, the search for files will follow symbolic links for the WAL directory and each configured tablespace.

On Linux, syncfs may be used instead to ask the operating system to synchronize the whole file system that contains the backup directory. When the plain format is used, pg_combinebackup will also synchronize the file systems that contain the WAL files and each tablespace. See ??? for information about the caveats to be aware of when using syncfs.

This option has no effect when --no-sync is used.

-V; --version
Prints the pg_combinebackup version and exits.

-?; --help
Shows help about pg_combinebackup command line arguments, and exits.

Limitations

pg_combinebackup does not recompute page checksums when writing the output directory. It therefore refuses a chain in which some of the backups used for reconstruction were taken with checksums disabled but the final backup was taken with checksums enabled: the resulting directory would contain pages with invalid checksums, and the cluster would fail checksum verification as soon as it reads them. Take a new full backup after enabling data checksums with ???.

The reverse case is accepted: when the final backup was taken with checksums disabled, stale checksums copied from an earlier backup are never verified.

Environment

This utility, like most other PostgreSQL utilities, uses the environment variables supported by libpq (see ???).

The environment variable PG_COLOR specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

See Also

pg_config-ref

pg_config

pg_config

1

Application

pg_config

retrieve information about the installed version of

PostgreSQL

pg_config

option

Description

The pg_config utility prints configuration parameters of the currently installed version of PostgreSQL. It is intended, for example, to be used by software packages that want to interface to PostgreSQL to facilitate finding the required header files and libraries.

Options

To use pg_config, supply one or more of the following options:

--bindir
Print the location of user executables. Use this, for example, to find the psql program. This is normally also the location where the pg_config program resides.

--docdir
Print the location of documentation files.

--htmldir
Print the location of HTML documentation files.

--includedir
Print the location of C header files of the client interfaces.

--pkgincludedir
Print the location of other C header files.

--includedir-server
Print the location of C header files for server programming.

--libdir
Print the location of object code libraries.

--pkglibdir
Print the location of dynamically loadable modules, or where the server would search for them. (Other architecture-dependent data files might also be installed in this directory.)

--localedir
Print the location of locale support files. (This will be an empty string if locale support was not configured when PostgreSQL was built.)

--mandir
Print the location of manual pages.

--sharedir
Print the location of architecture-independent support files.

--sysconfdir
Print the location of system-wide configuration files.

--pgxs
Print the location of extension makefiles.

--configure
Print the options that were given to the configure script when PostgreSQL was configured for building. This can be used to reproduce the identical configuration, or to find out with what options a binary package was built. (Note however that binary packages often contain vendor-specific custom patches.) See also the examples below.

--cc
Print the value of the CC variable that was used for building PostgreSQL. This shows the C compiler used.

--cppflags
Print the value of the CPPFLAGS variable that was used for building PostgreSQL. This shows C compiler switches needed at preprocessing time (typically, -I switches).

--cflags
Print the value of the CFLAGS variable that was used for building PostgreSQL. This shows C compiler switches.

--cflags_sl
Print the value of the CFLAGS_SL variable that was used for building PostgreSQL. This shows extra C compiler switches used for building shared libraries.

--ldflags
Print the value of the LDFLAGS variable that was used for building PostgreSQL. This shows linker switches.

--ldflags_ex
Print the value of the LDFLAGS_EX variable that was used for building PostgreSQL. This shows linker switches used for building executables only.

--ldflags_sl
Print the value of the LDFLAGS_SL variable that was used for building PostgreSQL. This shows linker switches used for building shared libraries only.

--libs
Print the value of the LIBS variable that was used for building PostgreSQL. This normally contains -l switches for external libraries linked into PostgreSQL.

--version
Print the version of PostgreSQL.

-?; --help
Show help about pg_config command line arguments, and exit.

If more than one option is given, the information is printed in that order, one item per line. If no options are given, all available information is printed, with labels.

Notes

The options --docdir, --pkgincludedir, --localedir, --mandir, --sharedir, --sysconfdir, --cc, --cppflags, --cflags, --cflags_sl, --ldflags, --ldflags_sl, and --libs were added in PostgreSQL 8.1. The option --htmldir was added in PostgreSQL 8.4. The option --ldflags_ex was added in PostgreSQL 9.0.

Example

To reproduce the build configuration of the current PostgreSQL installation, run the following command:

eval ./configure `pg_config --configure`

The output of pg_config --configure contains shell quotation marks so arguments with spaces are represented correctly. Therefore, using eval is required for proper results.

pg_controldata

pg_controldata

pg_controldata

1

Application

pg_controldata

display control information of a

PostgreSQL

database cluster

pg_controldata

option

-D

--pgdata

datadir

Description

pg_controldata prints information initialized during initdb, such as the catalog version. It also shows information about write-ahead logging and checkpoint processing. This information is cluster-wide, and not specific to any one database.

This utility can only be run by the user who initialized the cluster because it requires read access to the data directory. You can specify the data directory on the command line, or use the environment variable PGDATA.

Options

-D datadir; --pgdata=datadir
Specifies the directory where the database cluster is stored.

-V; --version
Print the pg_controldata version and exit.

-?; --help
Show help about pg_controldata command line arguments, and exit.

Environment

PGDATA
Default data directory location

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

pg_createsubscriber

pg_createsubscriber

pg_createsubscriber

1

Application

pg_createsubscriber

convert a physical replica into a new logical replica

pg_createsubscriber

option

-d

--database

dbname

-D

--pgdata

datadir

-P

--publisher-server

connstr

Description

pg_createsubscriber creates a new logical replica from a physical standby server. All tables in the specified database are included in the logical replication setup. A pair of publication and subscription objects are created for each database. It must be run at the target server.

After a successful run, the state of the target server is analogous to a fresh logical replication setup. The main difference between the logical replication setup and pg_createsubscriber is how the data synchronization is done. pg_createsubscriber does not copy the initial table data. It does only the synchronization phase, which ensures each table is brought up to a synchronized state.

pg_createsubscriber targets large database systems because in logical replication setup, most of the time is spent doing the initial data copy. Furthermore, a side effect of this long time spent synchronizing data is usually a large amount of changes to be applied (that were produced during the initial data copy), which increases even more the time when the logical replica will be available. For smaller databases, it is recommended to set up logical replication with initial data synchronization. For details, see the CREATE SUBSCRIPTION copy_data option.

Options

pg_createsubscriber accepts the following command-line arguments:

-a; --all
Create one subscription per database on the target server. Exceptions are template databases and databases that don't allow connections. To discover the list of all databases, connect to the source server using the database name specified in the --publisher-server connection string, or if not specified, the postgres database will be used, or if that does not exist, template1 will be used. Automatically generated names for subscriptions, publications, and replication slots are used when this option is specified. This option cannot be used along with --database, --publication, --replication-slot, or --subscription.

-d dbname; --database=dbname
The name of the database in which to create a subscription. Multiple databases can be selected by writing multiple -d switches. This option cannot be used together with -a. If -d option is not provided, the database name will be obtained from -P option. If the database name is not specified in either the -d option, or the -P option, and -a option is not specified, an error will be reported.

-D datadir; --pgdata=datadir
The target directory that contains a cluster directory from a physical replica.

-l directory; --logdir=directory
Specify the name of the log directory. A new directory is created with this name if it does not exist. A subdirectory with a timestamp indicating the time at which pg_createsubscriber was run will be created. The following two log files will be created in the subdirectory.

By default, the umask is set to 077 so that the log files are only readable by the user running the command. However, if the target data directory is configured to allow group-read access, pg_createsubscriber will adjust the log file permissions to match. This ensures that the log file security remains consistent with the database cluster itself.

-n; --dry-run
Do everything except actually modifying the target directory.

-p port; --subscriber-port=port
The port number on which the target server is listening for connections. Defaults to running the target server on port 50432 to avoid unintended client connections.

-P connstr; --publisher-server=connstr
The connection string to the publisher. For details see ???.

-s dir; --socketdir=dir
The directory to use for postmaster sockets on target server. The default is current directory.

-t seconds; --recovery-timeout=seconds
The maximum number of seconds to wait for recovery to end. Setting to 0 disables. The default is 0.

-T; --enable-two-phase
Enables two_phase commit for the subscription. When multiple databases are specified, this option applies uniformly to all subscriptions created on those databases. The default is false.

-U username; --subscriber-username=username
The user name to connect as on target server. Defaults to the current operating system user name.

-v; --verbose
Enables verbose mode. This will cause pg_createsubscriber to output progress messages and detailed information about each step to standard error. Repeating the option causes additional debug-level messages to appear on standard error.

--clean=objtype
Drop all objects of the specified type from specified databases on the target server.

The objects selected to be dropped are individually logged, including during a --dry-run. There is no opportunity to affect or stop the dropping of the selected objects, so consider taking a backup of them using pg_dump.

--config-file=filename
Use the specified main server configuration file for the target data directory. pg_createsubscriber internally uses the pg_ctl command to start and stop the target server. It allows you to specify the actual postgresql.conf configuration file if it is stored outside the data directory.

--publication=name
The publication name to set up the logical replication. Multiple publications can be specified by writing multiple --publication switches. The number of publication names must match the number of specified databases, otherwise an error is reported. The order of the multiple publication name switches must match the order of database switches. If this option is not specified, a generated name is assigned to the publication name. This option cannot be used together with --all.

If a specified publication already exists on the publisher, it is reused. It is useful to partially replicate the database if the specified publication includes a list of tables. If the publication does not exist, it is automatically created with FOR ALL TABLES. Use --dry-run option to preview which publications will be reused and which will be created.

--replication-slot=name
The replication slot name to set up the logical replication. Multiple replication slots can be specified by writing multiple --replication-slot switches. The number of replication slot names must match the number of specified databases, otherwise an error is reported. The order of the multiple replication slot name switches must match the order of database switches. If this option is not specified, the subscription name is assigned to the replication slot name. This option cannot be used together with --all.

--subscription=name
The subscription name to set up the logical replication. Multiple subscriptions can be specified by writing multiple --subscription switches. The number of subscription names must match the number of specified databases, otherwise an error is reported. The order of the multiple subscription name switches must match the order of database switches. If this option is not specified, a generated name is assigned to the subscription name. This option cannot be used together with --all.

-V; --version
Print the pg_createsubscriber version and exit.

-?; --help
Show help about pg_createsubscriber command line arguments, and exit.

Notes

Prerequisites

There are some prerequisites for pg_createsubscriber to convert the target server into a logical replica. If these are not met, an error will be reported. The source and target servers must have the same major version as the pg_createsubscriber. The given target data directory must have the same system identifier as the source data directory. The given database user for the target data directory must have privileges for creating subscriptions and using pg_replication_origin_advance().

The target server must be used as a physical standby. The target server must have ??? and ??? configured to a value greater than or equal to the number of specified databases. The target server must have ??? configured to a value greater than the number of specified databases. The target server must accept local connections. If you are planning to use the --enable-two-phase switch then you will also need to set the ??? appropriately.

The source server must accept connections from the target server. The source server must not be in recovery. The source server must have ??? as replica or logical. The source server must have ??? configured to a value greater than or equal to the number of specified databases plus existing replication slots. The source server must have ??? configured to a value greater than or equal to the number of specified databases and existing WAL sender processes.

Warnings

If pg_createsubscriber fails after the target server was promoted, then the data directory is likely not in a state that can be recovered. In such case, creating a new standby server is recommended.

pg_createsubscriber usually starts the target server with different connection settings during transformation. Hence, connections to the target server should fail.

Since DDL commands are not replicated by logical replication, avoid executing DDL commands that change the database schema while running pg_createsubscriber. If the target server has already been converted to logical replica, the DDL commands might not be replicated, which might cause an error.

If pg_createsubscriber fails while processing, objects (publications, replication slots) created on the source server are removed. The removal might fail if the target server cannot connect to the source server. In such a case, a warning message will inform the objects left. If the target server is running, it will be stopped.

If the replication is using ???, it will be removed from the source server after the logical replication setup.

If the target server is a synchronous replica, transaction commits on the primary might wait for replication while running pg_createsubscriber.

Unless the --enable-two-phase switch is specified, pg_createsubscriber sets up logical replication with two-phase commit disabled. This means that any prepared transactions will be replicated at the time of COMMIT PREPARED, without advance preparation. Once setup is complete, you can manually drop and re-create the subscription(s) with the two_phase option enabled.

pg_createsubscriber changes the system identifier using pg_resetwal. It would avoid situations in which the target server might use WAL files from the source server. If the target server has a standby, replication will break and a fresh standby should be created.

Replication failures can occur if required WAL files are missing. To prevent this, the source server must set ??? to -1 to ensure that required WAL files are not prematurely removed.

How It Works

The basic idea is to have a replication start point from the source server and set up a logical replication to start from this point:

  1. Start the target server with the specified command-line options. If the target server is already running, pg_createsubscriber will terminate with an error.

  2. Check if the target server can be converted. There are also a few checks on the source server. If any of the prerequisites are not met, pg_createsubscriber will terminate with an error.

  3. Create a publication and replication slot for each specified database on the source server. Each publication is created using FOR ALL TABLES. If the --publication option is not specified, the publication has the following name pattern: “pg_createsubscriber_%u_%x” (parameter: database oid, random int). If the --replication-slot option is not specified, the replication slot has the following name pattern: “pg_createsubscriber_%u_%x” (parameters: database oid, random int). These replication slots will be used by the subscriptions in a future step. The last replication slot LSN is used as a stopping point in the ??? parameter and by the subscriptions as a replication start point. It guarantees that no transaction will be lost.

  4. Write recovery parameters into the separate configuration file pg_createsubscriber.conf that is included from postgresql.auto.conf using include_if_exists in the target data directory, then restart the target server. It specifies an LSN (???) of the write-ahead log location up to which recovery will proceed. It also specifies promote as the action that the server should take once the recovery target is reached. Additional recovery parameters are added to avoid unexpected behavior during the recovery process such as end of the recovery as soon as a consistent state is reached (WAL should be applied until the replication start location) and multiple recovery targets that can cause a failure. This step finishes once the server ends standby mode and is accepting read-write transactions. If --recovery-timeout option is set, pg_createsubscriber terminates if recovery does not end until the given number of seconds. Upon completion, the included configuration file is renamed to pg_createsubscriber.conf.disabled so as it is no longer loaded on subsequent restarts.

  5. Create a subscription for each specified database on the target server. If the --subscription option is not specified, the subscription has the following name pattern: “pg_createsubscriber_%u_%x” (parameters: database oid, random int). It does not copy existing data from the source server. It does not create a replication slot. Instead, it uses the replication slot that was created in a previous step. The subscription is created but it is not enabled yet. The reason is the replication progress must be set to the replication start point before starting the replication.

  6. Drop publications on the target server that were replicated because they were created before the replication start location. It has no use on the subscriber.

  7. Set the replication progress to the replication start point for each subscription. When the target server starts the recovery process, it catches up to the replication start point. This is the exact LSN to be used as a initial replication location for each subscription. The replication origin name is obtained since the subscription was created. The replication origin name and the replication start point are used in pg_replication_origin_advance() to set up the initial replication location.

  8. Enable the subscription for each specified database on the target server. The subscription starts applying transactions from the replication start point.

  9. If the standby server was using ???, it has no use from now on so drop it.

  10. If the standby server contains failover replication slots, they cannot be synchronized anymore, so drop them.

  11. Update the system identifier on the target server. The ??? is run to modify the system identifier. The target server is stopped as a pg_resetwal requirement.

Examples

To create a logical replica for databases hr and finance from a physical replica at foo:

$ pg_createsubscriber -D /usr/local/pgsql/data -P "host=foo" -d hr -d finance

See Also

pg_ctl-ref

pg_ctl

pg_ctl

1

Application

pg_ctl

initialize, start, stop, or control a

PostgreSQL

server

pg_ctl

init[db]

-D

datadir

-s

-o

initdb-options

pg_ctl

start

-D

datadir

-l

filename

-W

-t

seconds

-s

-o

options

-p

path

-c

pg_ctl

stop

-D

datadir

-m

s[mart]

f[ast]

i[mmediate]

-W

-t

seconds

-s

pg_ctl

restart

-D

datadir

-m

s[mart]

f[ast]

i[mmediate]

-W

-t

seconds

-s

-o

options

-c

pg_ctl

reload

-D

datadir

-s

pg_ctl

status

-D

datadir

pg_ctl

promote

-D

datadir

-W

-t

seconds

-s

pg_ctl

logrotate

-D

datadir

-s

pg_ctl

kill

signal_name

process_id

On Microsoft Windows, also:

pg_ctl

register

-D

datadir

-N

servicename

-U

username

-P

password

-S

a[uto]

d[emand]

-e

source

-W

-t

seconds

-s

-o

options

pg_ctl

unregister

-N

servicename

Description

pg_ctl is a utility for initializing a PostgreSQL database cluster, starting, stopping, or restarting the PostgreSQL database server (???), or displaying the status of a running server. Although the server can be started manually, pg_ctl encapsulates tasks such as redirecting log output and properly detaching from the terminal and process group. It also provides convenient options for controlled shutdown.

The init or initdb mode creates a new PostgreSQL database cluster, that is, a collection of databases that will be managed by a single server instance. This mode invokes the initdb command. See ??? for details.

start mode launches a new server. The server is started in the background, and its standard input is attached to /dev/null (or nul on Windows). On Unix-like systems, by default, the server's standard output and standard error are sent to pg_ctl's standard output (not standard error). The standard output of pg_ctl should then be redirected to a file or piped to another process such as a log rotating program like rotatelogs; otherwise postgres will write its output to the controlling terminal (from the background) and will not leave the shell's process group. On Windows, by default the server's standard output and standard error are sent to the terminal. These default behaviors can be changed by using -l to append the server's output to a log file. Use of either -l or output redirection is recommended.

stop mode shuts down the server that is running in the specified data directory. Three different shutdown methods can be selected with the -m option. “Smart” mode disallows new connections, then waits for all existing clients to disconnect. If the server is in hot standby, recovery and streaming replication will be terminated once all clients have disconnected. “Fast” mode (the default) does not wait for clients to disconnect. All active transactions are rolled back and clients are forcibly disconnected, then the server is shut down. “Immediate” mode will abort all server processes immediately, without a clean shutdown. This choice will lead to a crash-recovery cycle during the next server start.

restart mode effectively executes a stop followed by a start. This allows changing the postgres command-line options, or changing configuration-file options that cannot be changed without restarting the server. If relative paths were used on the command line during server start, restart might fail unless pg_ctl is executed in the same current directory as it was during server start.

reload mode simply sends the postgres server process a SIGHUP signal, causing it to reread its configuration files (postgresql.conf, pg_hba.conf, etc.). This allows changing configuration-file options that do not require a full server restart to take effect.

status mode checks whether a server is running in the specified data directory. If it is, the server's PID and the command line options that were used to invoke it are displayed. If the server is not running, pg_ctl returns an exit status of 3. If an accessible data directory is not specified, pg_ctl returns an exit status of 4.

promote mode commands the standby server that is running in the specified data directory to end standby mode and begin read-write operations.

logrotate mode rotates the server log file. For details on how to use this mode with external log rotation tools, see ???.

kill mode sends a signal to a specified process. This is primarily valuable on Microsoft Windows which does not have a built-in kill command. Use --help to see a list of supported signal names.

register mode registers the PostgreSQL server as a system service on Microsoft Windows. The -S option allows selection of service start type, either “auto” (start service automatically on system startup) or “demand” (start service on demand).

unregister mode unregisters a system service on Microsoft Windows. This undoes the effects of the register command.

Options

-c; --core-files
Attempt to allow server crashes to produce core files, on platforms where this is possible, by lifting any soft resource limit placed on core files. This is useful in debugging or diagnosing problems by allowing a stack trace to be obtained from a failed server process.

-D datadir; --pgdata=datadir
Specifies the file system location of the database configuration files. If this option is omitted, the environment variable PGDATA is used.

-l filename; --log=filename
Append the server log output to <filename>. If the file does not exist, it is created. By default, only the cluster owner can access the log file. If group access is enabled in the cluster, users in the same group as the cluster owner can also read it.

-m mode; --mode=mode
Specifies the shutdown mode. <mode> can be smart, fast, or immediate, or the first letter of one of these three. If this option is omitted, fast is the default.

-o options; --options=options
Specifies options to be passed directly to the postgres command. -o can be specified multiple times, with all the given options being passed through.

The <options> should usually be surrounded by single or double quotes to ensure that they are passed through as a group.

-o initdb-options; --options=initdb-options
Specifies options to be passed directly to the initdb command. -o can be specified multiple times, with all the given options being passed through.

The <initdb-options> should usually be surrounded by single or double quotes to ensure that they are passed through as a group.

-p path
Specifies the location of the postgres executable. By default the postgres executable is taken from the same directory as pg_ctl, or failing that, the hard-wired installation directory. It is not necessary to use this option unless you are doing something unusual and get errors that the postgres executable was not found.

In init mode, this option analogously specifies the location of the initdb executable.

-s; --silent
Print only errors, no informational messages.

-t seconds; --timeout=seconds
Specifies the maximum number of seconds to wait when waiting for an operation to complete (see option -w). Defaults to the value of the PGCTLTIMEOUT environment variable or, if not set, to 60 seconds.

-V; --version
Print the pg_ctl version and exit.

-w; --wait
Wait for the operation to complete. This is supported for the modes start, stop, restart, promote, and register, and is the default for those modes.

When waiting, pg_ctl repeatedly checks the server's PID file, sleeping for a short amount of time between checks. Startup is considered complete when the PID file indicates that the server is ready to accept connections. Shutdown is considered complete when the server removes the PID file. pg_ctl returns an exit code based on the success of the startup or shutdown.

If the operation does not complete within the timeout (see option -t), then pg_ctl exits with a nonzero exit status. But note that the operation might continue in the background and eventually succeed.

-W; --no-wait
Do not wait for the operation to complete. This is the opposite of the option -w.

If waiting is disabled, the requested action is triggered, but there is no feedback about its success. In that case, the server log file or an external monitoring system would have to be used to check the progress and success of the operation.

In prior releases of PostgreSQL, this was the default except for the stop mode.

-?; --help
Show help about pg_ctl command line arguments, and exit.

If an option is specified that is valid, but not relevant to the selected operating mode, pg_ctl ignores it.

Options for Windows

-e source
Name of the event source for pg_ctl to use for logging to the event log when running as a Windows service. The default is PostgreSQL. Note that this only controls messages sent from pg_ctl itself; once started, the server will use the event source specified by its ??? parameter. Should the server fail very early in startup, before that parameter has been set, it might also log using the default event source name PostgreSQL.

-N servicename
Name of the system service to register. This name will be used as both the service name and the display name. The default is PostgreSQL.

-P password
Password for the user to run the service as.

-S start-type
Start type of the system service. <start-type> can be auto, or demand, or the first letter of one of these two. If this option is omitted, auto is the default.

-U username
User name for the user to run the service as. For domain users, use the format DOMAIN\username.

Environment

PGCTLTIMEOUT
Default limit on the number of seconds to wait when waiting for startup or shutdown to complete. If not set, the default is 60 seconds.

PGDATA
Default data directory location.

Most pg_ctl modes require knowing the data directory location; therefore, the -D option is required unless PGDATA is set.

For additional variables that affect the server, see ???.

Files

postmaster.pid
pg_ctl examines this file in the data directory to determine whether the server is currently running.

postmaster.opts
If this file exists in the data directory, pg_ctl (in restart mode) will pass the contents of the file as options to postgres, unless overridden by the -o option. The contents of this file are also displayed in status mode.

Examples

Starting the Server

To start the server, waiting until the server is accepting connections:

$ pg_ctl start

To start the server using port 5433, and running without fsync, use:

$ pg_ctl -o "-F -p 5433" start

Stopping the Server

To stop the server, use:

$ pg_ctl stop

The -m option allows control over how the server shuts down:

$ pg_ctl stop -m smart

Restarting the Server

Restarting the server is almost equivalent to stopping the server and starting it again, except that by default, pg_ctl saves and reuses the command line options that were passed to the previously-running instance. To restart the server using the same options as before, use:

$ pg_ctl restart

But if -o is specified, that replaces any previous options. To restart using port 5433, disabling fsync upon restart:

$ pg_ctl -o "-F -p 5433" restart

Showing the Server Status

Here is sample status output from pg_ctl:

$ pg_ctl status

pg_ctl: server is running (PID: 13718)
/usr/local/pgsql/bin/postgres "-D" "/usr/local/pgsql/data" "-p" "5433" "-B" "128"

The second line is the command that would be invoked in restart mode.

See Also

pg_dump

pg_dump

pg_dump

1

Application

pg_dump

export a

PostgreSQL

database as an SQL script or to other formats

pg_dump

connection-option

option

dbname

Description

pg_dump is a utility for exporting a PostgreSQL database. It makes consistent exports even if the database is being used concurrently. pg_dump does not block other users accessing the database (readers or writers). Note, however, that except in simple cases, pg_dump is generally not the right choice for taking regular backups of production databases. See ??? for further discussion.

pg_dump only dumps a single database. To export an entire cluster, or to export global objects that are common to all databases in a cluster (such as roles and tablespaces), use ???.

Dumps can be output in script or archive file formats. Script dumps are plain-text files containing the SQL commands required to reconstruct the database to the state it was in at the time it was saved. To restore from such a script, feed it to ???. Script files can be used to reconstruct the database even on other machines and other architectures; with some modifications, even on other SQL database products.

The alternative archive file formats must be used with ??? to rebuild the database. They allow pg_restore to be selective about what is restored, or even to reorder the items prior to being restored. The archive file formats are designed to be portable across architectures.

When used with one of the archive file formats and combined with pg_restore, pg_dump provides a flexible archival and transfer mechanism. pg_dump can be used to export an entire database, then pg_restore can be used to examine the archive and/or select which parts of the database are to be restored. The most flexible output file formats are the “custom” format (-Fc) and the “directory” format (-Fd). They allow for selection and reordering of all archived items, support parallel restoration, and are compressed by default. The “directory” format is the only format that supports parallel dumps.

While running pg_dump, one should examine the output for any warnings (printed on standard error), especially in light of the limitations listed below.

[!WARNING] Restoring a dump causes the destination to execute arbitrary code of the source superusers' choice. Partial dumps and partial restores do not limit that. If the source superusers are not trusted, the dumped SQL statements must be inspected before restoring. Non-plain-text dumps can be inspected by using pg_restore's --file option. Note that the client running the dump and restore need not trust the source or destination superusers.

Options

The following command-line options control the content and format of the output.

<dbname>
Specifies the name of the database to be dumped. If this is not specified, the environment variable PGDATABASE is used. If that is not set, the user name specified for the connection is used.

-a; --data-only
Dump only the data, not the schema (data definitions) or statistics. Table data, large objects, and sequence values are dumped.

This option is similar to, but for historical reasons not identical to, specifying --section=data.

-b; --large-objects; --blobs (deprecated)
Include large objects in the dump. This is the default behavior except when --schema, --table, --schema-only, --statistics-only, or --no-data is specified. The -b switch is therefore only useful to add large objects to dumps where a specific schema or table has been requested. Note that large objects are considered data and therefore will be included when --data-only is used, but not when --schema-only or --statistics-only is.

-B; --no-large-objects; --no-blobs (deprecated)
Exclude large objects in the dump.

When both -b and -B are given, the behavior is to output large objects, when data is being dumped, see the -b documentation.

-c; --clean
Output commands to DROP all the dumped database objects prior to outputting the commands for creating them. This option is useful when the restore is to overwrite an existing database. If any of the objects do not exist in the destination database, ignorable error messages will be reported during restore, unless --if-exists is also specified.

This option is ignored when emitting an archive (non-text) output file. For the archive formats, you can specify the option when you call pg_restore.

-C; --create
Begin the output with a command to create the database itself and reconnect to the created database. (With a script of this form, it doesn't matter which database in the destination installation you connect to before running the script.) If --clean is also specified, the script drops and recreates the target database before reconnecting to it.

With --create, the output also includes the database's comment if any, and any configuration variable settings that are specific to this database, that is, any ALTER DATABASE ... SET ... and ALTER ROLE ... IN DATABASE ... SET ... commands that mention this database. Access privileges for the database itself are also dumped, unless --no-acl is specified.

This option is ignored when emitting an archive (non-text) output file. For the archive formats, you can specify the option when you call pg_restore.

-e pattern; --extension=pattern
Dump only extensions matching <pattern>. When this option is not specified, all non-system extensions in the target database will be dumped. Multiple extensions can be selected by writing multiple -e switches. The <pattern> parameter is interpreted as a pattern according to the same rules used by psql's \d commands (see ???), so multiple extensions can also be selected by writing wildcard characters in the pattern. When using wildcards, be careful to quote the pattern if needed to prevent the shell from expanding the wildcards.

Any configuration relation registered by pg_extension_config_dump is included in the dump if its extension is specified by --extension.

[!NOTE] pg_dump does not dump the extension's underlying installation files (such as shared libraries or control files). These must be available on the destination system for the restore to succeed.

-E encoding; --encoding=encoding
Create the dump in the specified character set encoding. By default, the dump is created in the database encoding. (Another way to get the same result is to set the PGCLIENTENCODING environment variable to the desired dump encoding.) The supported encodings are described in ???.

-f file; --file=file
Send output to the specified file. This parameter can be omitted for file based output formats, in which case the standard output is used. It must be given for the directory output format however, where it specifies the target directory instead of a file. In this case the directory is created by pg_dump unless the directory exists and is empty.

-F format; --format=format
Selects the format of the output. <format> can be one of the following:

p; plain
Output a plain-text SQL script file (the default).

c; custom
Output a custom-format archive suitable for input into pg_restore. Together with the directory output format, this is the most flexible output format in that it allows manual selection and reordering of archived items during restore. This format is also compressed by default.

d; directory
Output a directory-format archive suitable for input into pg_restore. This will create a directory with one file for each table and large object being dumped, plus a so-called Table of Contents file describing the dumped objects in a machine-readable format that pg_restore can read. A directory format archive can be manipulated with standard Unix tools; for example, files in an uncompressed archive can be compressed with the gzip, lz4, or zstd tools. This format is compressed by default using gzip and also supports parallel dumps.

t; tar
Output a tar-format archive suitable for input into pg_restore. The tar format is compatible with the directory format: extracting a tar-format archive produces a valid directory-format archive. However, the tar format does not support compression. Also, when using tar format the relative order of table data items cannot be changed during restore.

-j njobs; --jobs=njobs
Run the dump in parallel by dumping <njobs> tables simultaneously. This option may reduce the time needed to perform the dump but it also increases the load on the database server. You can only use this option with the directory output format because this is the only output format where multiple processes can write their data at the same time.

pg_dump will open <njobs> + 1 connections to the database, so make sure your ??? setting is high enough to accommodate all connections.

Requesting exclusive locks on database objects while running a parallel dump could cause the dump to fail. The reason is that the pg_dump leader process requests shared locks (ACCESS SHARE) on the objects that the worker processes are going to dump later in order to make sure that nobody deletes them and makes them go away while the dump is running. If another client then requests an exclusive lock on a table, that lock will not be granted but will be queued waiting for the shared lock of the leader process to be released. Consequently any other access to the table will not be granted either and will queue after the exclusive lock request. This includes the worker process trying to dump the table. Without any precautions this would be a classic deadlock situation. To detect this conflict, the pg_dump worker process requests another shared lock using the NOWAIT option. If the worker process is not granted this shared lock, somebody else must have requested an exclusive lock in the meantime and there is no way to continue with the dump, so pg_dump has no choice but to abort the dump.

-n pattern; --schema=pattern
Dump only schemas matching <pattern>; this selects both the schema itself, and all its contained objects. When this option is not specified, all non-system schemas in the target database will be dumped. Multiple schemas can be selected by writing multiple -n switches. The <pattern> parameter is interpreted as a pattern according to the same rules used by psql's \d commands (see ???), so multiple schemas can also be selected by writing wildcard characters in the pattern. When using wildcards, be careful to quote the pattern if needed to prevent the shell from expanding the wildcards; see Examples below.

[!NOTE] Non-schema objects such as large objects are not dumped when -n is specified. You can add large objects back to the dump with the --large-objects switch.

-N pattern; --exclude-schema=pattern
Do not dump any schemas matching <pattern>. The pattern is interpreted according to the same rules as for -n. -N can be given more than once to exclude schemas matching any of several patterns.

When both -n and -N are given, the behavior is to dump just the schemas that match at least one -n switch but no -N switches. If -N appears without -n, then schemas matching -N are excluded from what is otherwise a normal dump.

-O; --no-owner
Do not output commands to set ownership of objects to match the original database. By default, pg_dump issues ALTER OWNER or SET SESSION AUTHORIZATION statements to set ownership of created database objects. These statements will fail when the script is run unless it is started by a superuser (or the same user that owns all of the objects in the script). To make a script that can be restored by any user, but will give that user ownership of all the objects, specify -O.

This option is ignored when emitting an archive (non-text) output file. For the archive formats, you can specify the option when you call pg_restore.

-R; --no-reconnect
This option is obsolete but still accepted for backwards compatibility.

-s; --schema-only
Dump only the object definitions (schema), not data or statistics.

This option cannot be used with --data-only or --statistics-only. It is similar to, but for historical reasons not identical to, specifying --section=pre-data --section=post-data.

(Do not confuse this with the --schema option, which uses the word “schema” in a different meaning.)

To exclude table data for only a subset of tables in the database, see --exclude-table-data.

-S username; --superuser=username
Specify the superuser user name to use when disabling triggers. This is relevant only if --disable-triggers is used. (Usually, it's better to leave this out, and instead start the resulting script as superuser.)

-t pattern; --table=pattern
Dump only tables with names matching <pattern>. Multiple tables can be selected by writing multiple -t switches. The <pattern> parameter is interpreted as a pattern according to the same rules used by psql's \d commands (see ???), so multiple tables can also be selected by writing wildcard characters in the pattern. When using wildcards, be careful to quote the pattern if needed to prevent the shell from expanding the wildcards; see Examples below.

As well as tables, this option can be used to dump the definition of matching views, materialized views, foreign tables, and sequences. It will not dump the contents of views or materialized views, and the contents of foreign tables will only be dumped if the corresponding foreign server is specified with --include-foreign-data.

The -n and -N switches have no effect when -t is used, because tables selected by -t will be dumped regardless of those switches, and non-table objects will not be dumped.

-T pattern; --exclude-table=pattern
Do not dump any tables matching <pattern>. The pattern is interpreted according to the same rules as for -t. -T can be given more than once to exclude tables matching any of several patterns.

When both -t and -T are given, the behavior is to dump just the tables that match at least one -t switch but no -T switches. If -T appears without -t, then tables matching -T are excluded from what is otherwise a normal dump.

-v; --verbose
Specifies verbose mode. This will cause pg_dump to output detailed object comments and start/stop times to the dump file, and progress messages to standard error. Repeating the option causes additional debug-level messages to appear on standard error.

-V; --version
Print the pg_dump version and exit.

-x; --no-privileges; --no-acl
Prevent dumping of access privileges (grant/revoke commands).

-Z level; -Z method[:<detail>]; --compress=level; --compress=method[:<detail>]
Specify the compression method and/or the compression level to use. The compression method can be set to gzip, lz4, zstd, or none for no compression. A compression detail string can optionally be specified. If the detail string is an integer, it specifies the compression level. Otherwise, it should be a comma-separated list of items, each of the form keyword or keyword=value. Currently, the supported keywords are level and long.

If no compression level is specified, the default compression level will be used. If only a level is specified without mentioning an algorithm, gzip compression will be used if the level is greater than 0, and no compression will be used if the level is 0.

For the custom and directory archive formats, this specifies compression of individual table-data segments, and the default is to compress using gzip at a moderate level. For plain text output, setting a nonzero compression level causes the entire output file to be compressed, as though it had been fed through gzip, lz4, or zstd; but the default is not to compress. With zstd compression, long mode may improve the compression ratio, at the cost of increased memory use.

The tar archive format currently does not support compression at all.

--binary-upgrade
This option is for use by in-place upgrade utilities. Its use for other purposes is not recommended or supported. The behavior of the option may change in future releases without notice.

--column-inserts; --attribute-inserts
Dump data as INSERT commands with explicit column names (INSERT INTO table (column, ...) VALUES ...). This will make restoration very slow; it is mainly useful for making dumps that can be loaded into non-PostgreSQL databases. Any error during restoring will cause only rows that are part of the problematic INSERT to be lost, rather than the entire table contents.

--disable-dollar-quoting
This option disables the use of dollar quoting for function bodies, and forces them to be quoted using SQL standard string syntax.

--disable-triggers
This option is relevant only when creating a dump that includes data but does not include schema. It instructs pg_dump to include commands to temporarily disable triggers on the target tables while the data is restored. Use this if you have referential integrity checks or other triggers on the tables that you do not want to invoke during data restore.

Presently, the commands emitted for --disable-triggers must be done as superuser. So, you should also specify a superuser name with -S, or preferably be careful to start the resulting script as a superuser.

This option is ignored when emitting an archive (non-text) output file. For the archive formats, you can specify the option when you call pg_restore.

--enable-row-security
This option is relevant only when dumping the contents of a table which has row security. By default, pg_dump will set ??? to off, to ensure that all data is dumped from the table. If the user does not have sufficient privileges to bypass row security, then an error is thrown. This parameter instructs pg_dump to set ??? to on instead, allowing the user to dump the parts of the contents of the table that they have access to.

Note that if you use this option currently, you probably also want the dump be in INSERT format, as the COPY FROM during restore does not support row security.

--exclude-extension=pattern
Do not dump any extensions matching <pattern>. The pattern is interpreted according to the same rules as for -e. --exclude-extension can be given more than once to exclude extensions matching any of several patterns.

When both -e and --exclude-extension are given, the behavior is to dump just the extensions that match at least one -e switch but no --exclude-extension switches. If --exclude-extension appears without -e, then extensions matching --exclude-extension are excluded from what is otherwise a normal dump.

--exclude-table-and-children=pattern
This is the same as the -T/--exclude-table option, except that it also excludes any partitions or inheritance child tables of the table(s) matching the <pattern>.

--exclude-table-data=pattern
Do not dump data for any tables matching <pattern>. The pattern is interpreted according to the same rules as for -t. --exclude-table-data can be given more than once to exclude tables matching any of several patterns. This option is useful when you need the definition of a particular table even though you do not need the data in it.

To exclude data for all tables in the database, see --schema-only or --statistics-only.

--exclude-table-data-and-children=pattern
This is the same as the --exclude-table-data option, except that it also excludes data of any partitions or inheritance child tables of the table(s) matching the <pattern>.

--extra-float-digits=ndigits
Use the specified value of extra_float_digits when dumping floating-point data, instead of the maximum available precision. Routine dumps made for backup purposes should not use this option.

--filter=filename
Specify a filename from which to read patterns for objects to include or exclude from the dump. The patterns are interpreted according to the same rules as the corresponding options: -t/--table, --table-and-children, -T/--exclude-table, and --exclude-table-and-children for tables, -n/--schema and -N/--exclude-schema for schemas, --include-foreign-data for data on foreign servers, --exclude-table-data and --exclude-table-data-and-children for table data, and -e/--extension and --exclude-extension for extensions. To read from STDIN, use - as the filename. The --filter option can be specified in conjunction with the above listed options for including or excluding objects, and can also be specified more than once for multiple filter files.

The file lists one object pattern per row, with the following format: { include | exclude } { extension | foreign_data | table | table_and_children | table_data | table_data_and_children | schema } <PATTERN>

The first keyword specifies whether the objects matched by the pattern are to be included or excluded. The second keyword specifies the type of object to be filtered using the pattern:

Lines starting with # are considered comments and ignored. Comments can be placed after an object pattern row as well. Blank lines are also ignored. See ??? for how to perform quoting in patterns.

Example files are listed below in the Examples section.

--if-exists
Use DROP ... IF EXISTS commands to drop objects in --clean mode. This suppresses “does not exist” errors that might otherwise be reported. This option is not valid unless --clean is also specified.

--include-foreign-data=foreignserver
Dump the data for any foreign table with a foreign server matching <foreignserver> pattern. Multiple foreign servers can be selected by writing multiple --include-foreign-data switches. Also, the <foreignserver> parameter is interpreted as a pattern according to the same rules used by psql's \d commands (see ???), so multiple foreign servers can also be selected by writing wildcard characters in the pattern. When using wildcards, be careful to quote the pattern if needed to prevent the shell from expanding the wildcards; see Examples below. The only exception is that an empty pattern is disallowed.

[!NOTE] Using wildcards in --include-foreign-data may result in access to unexpected foreign servers. Also, to use this option securely, make sure that the named server must have a trusted owner.

[!NOTE] When --include-foreign-data is specified, pg_dump does not check that the foreign table is writable. Therefore, there is no guarantee that the results of a foreign table dump can be successfully restored.

--inserts
Dump data as INSERT commands (rather than COPY). This will make restoration very slow; it is mainly useful for making dumps that can be loaded into non-PostgreSQL databases. Any error during restoring will cause only rows that are part of the problematic INSERT to be lost, rather than the entire table contents. Note that the restore might fail altogether if you have rearranged column order. The --column-inserts option is safe against column order changes, though even slower.

--load-via-partition-root
When dumping data for a table partition, make the COPY or INSERT statements target the root of the partitioning hierarchy that contains it, rather than the partition itself. This causes the appropriate partition to be re-determined for each row when the data is loaded. This may be useful when restoring data on a server where rows do not always fall into the same partitions as they did on the original server. That could happen, for example, if the partitioning column is of type text and the two systems have different definitions of the collation used to sort the partitioning column.

--lock-wait-timeout=timeout
Do not wait forever to acquire shared table locks at the beginning of the dump. Instead fail if unable to lock a table within the specified <timeout>. The timeout may be specified in any of the formats accepted by SET statement_timeout. (Allowed formats vary depending on the server version you are dumping from, but an integer number of milliseconds is accepted by all versions.)

--no-comments
Do not dump COMMENT commands.

--no-data
Do not dump data.

--no-policies
Do not dump row security policies.

--no-publications
Do not dump publications.

--no-schema
Do not dump schema (data definitions).

--no-security-labels
Do not dump security labels.

--no-statistics
Do not dump statistics. This is the default.

--no-subscriptions
Do not dump subscriptions.

--no-sync
By default, pg_dump will wait for all files to be written safely to disk. This option causes pg_dump to return without waiting, which is faster, but means that a subsequent operating system crash can leave the dump corrupt. Generally, this option is useful for testing but should not be used when dumping data from production installation.

--no-table-access-method
Do not output commands to select table access methods. With this option, all objects will be created with whichever table access method is the default during restore.

This option is ignored when emitting an archive (non-text) output file. For the archive formats, you can specify the option when you call pg_restore.

--no-tablespaces
Do not output commands to select tablespaces. With this option, all objects will be created in whichever tablespace is the default during restore.

This option is ignored when emitting an archive (non-text) output file. For the archive formats, you can specify the option when you call pg_restore.

--no-toast-compression
Do not output commands to set TOAST compression methods. With this option, all columns will be restored with the default compression setting.

--no-unlogged-table-data
Do not dump the contents of unlogged tables and sequences. This option has no effect on whether or not the table and sequence definitions (schema) are dumped; it only suppresses dumping the table and sequence data. Data in unlogged tables and sequences is always excluded when dumping from a standby server.

--on-conflict-do-nothing
Add ON CONFLICT DO NOTHING to INSERT commands. This option is not valid unless --inserts, --column-inserts or --rows-per-insert is also specified.

--quote-all-identifiers
Force quoting of all identifiers. This option is recommended when dumping a database from a server whose PostgreSQL major version is different from pg_dump's, or when the output is intended to be loaded into a server of a different major version. By default, pg_dump quotes only identifiers that are reserved words in its own major version. This sometimes results in compatibility issues when dealing with servers of other versions that may have slightly different sets of reserved words. Using --quote-all-identifiers prevents such issues, at the price of a harder-to-read dump script.

--restrict-key=restrict_key
Use the provided string as the psql \restrict key in the dump output. This can only be specified for plain-text dumps, i.e., when --format is set to plain or the --format option is omitted. If no restrict key is specified, pg_dump will generate a random one as needed. Keys may contain only alphanumeric characters.

This option is primarily intended for testing purposes and other scenarios that require repeatable output (e.g., comparing dump files). It is not recommended for general use, as a malicious server with advance knowledge of the key may be able to inject arbitrary code that will be executed on the machine that runs psql with the dump output.

--rows-per-insert=nrows
Dump data as INSERT commands (rather than COPY). Controls the maximum number of rows per INSERT command. The value specified must be a number greater than zero. Any error during restoring will cause only rows that are part of the problematic INSERT to be lost, rather than the entire table contents.

--section=sectionname
Only dump the named section. The section name can be pre-data, data, or post-data. This option can be specified more than once to select multiple sections. The default is to dump all sections.

The data section contains actual table data, large-object contents, sequence values, and statistics for tables, materialized views, and foreign tables. Post-data items include definitions of indexes, triggers, rules, statistics for indexes, and constraints other than validated check and not-null constraints. Pre-data items include all other data definition items.

--sequence-data
Include sequence data in the dump. This is the default behavior except when --no-data, --schema-only, or --statistics-only is specified.

--serializable-deferrable
Use a serializable transaction for the dump, to ensure that the snapshot used is consistent with later database states; but do this by waiting for a point in the transaction stream at which no anomalies can be present, so that there isn't a risk of the dump failing or causing other transactions to roll back with a serialization_failure. See ??? for more information about transaction isolation and concurrency control.

This option is not beneficial for a dump which is intended only for disaster recovery. It could be useful for a dump used to load a copy of the database for reporting or other read-only load sharing while the original database continues to be updated. Without it the dump may reflect a state which is not consistent with any serial execution of the transactions eventually committed. For example, if batch processing techniques are used, a batch may show as closed in the dump without all of the items which are in the batch appearing.

This option will make no difference if there are no read-write transactions active when pg_dump is started. If read-write transactions are active, the start of the dump may be delayed for an indeterminate length of time. Once running, performance with or without the switch is the same.

--snapshot=snapshotname
Use the specified synchronized snapshot when making a dump of the database (see ??? for more details).

This option is useful when needing to synchronize the dump with a logical replication slot (see ???) or with a concurrent session.

In the case of a parallel dump, the snapshot name defined by this option is used rather than taking a new snapshot.

--statistics
Dump optimizer statistics.

--statistics-only
Dump only the statistics, not the schema (data definitions) or data. Optimizer statistics for tables, materialized views, foreign tables, and indexes are dumped.

--strict-names
Require that each extension (-e/--extension), schema (-n/--schema) and table (-t/--table) pattern match at least one extension/schema/table in the database to be dumped. This also applies to filters used with --filter. Note that if none of the extension/schema/table patterns find matches, pg_dump will generate an error even without --strict-names.

This option has no effect on --exclude-extension, -N/--exclude-schema, -T/--exclude-table, or --exclude-table-data. An exclude pattern failing to match any objects is not considered an error.

--sync-method=method
When set to fsync, which is the default, pg_dump --format=directory will recursively open and synchronize all files in the archive directory.

On Linux, syncfs may be used instead to ask the operating system to synchronize the whole file system that contains the archive directory. See ??? for information about the caveats to be aware of when using syncfs.

This option has no effect when --no-sync is used or --format is not set to directory.

--table-and-children=pattern
This is the same as the -t/--table option, except that it also includes any partitions or inheritance child tables of the table(s) matching the <pattern>.

--use-set-session-authorization
Output SQL-standard SET SESSION AUTHORIZATION commands instead of ALTER OWNER commands to determine object ownership. This makes the dump more standards-compatible, but depending on the history of the objects in the dump, might not restore properly. Also, a dump using SET SESSION AUTHORIZATION will certainly require superuser privileges to restore correctly, whereas ALTER OWNER requires lesser privileges.

-?; --help
Show help about pg_dump command line arguments, and exit.

The following command-line options control the database connection parameters.

-d dbname; --dbname=dbname
Specifies the name of the database to connect to. This is equivalent to specifying <dbname> as the first non-option argument on the command line. The <dbname> can be a connection string. If so, connection string parameters will override any conflicting command line options.

-h host; --host=host
Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for the Unix domain socket. The default is taken from the PGHOST environment variable, if set, else a Unix domain socket connection is attempted.

-p port; --port=port
Specifies the TCP port or local Unix domain socket file extension on which the server is listening for connections. Defaults to the PGPORT environment variable, if set, or a compiled-in default.

-U username; --username=username
User name to connect as.

-w; --no-password
Never issue a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password.

-W; --password
Force pg_dump to prompt for a password before connecting to a database.

This option is never essential, since pg_dump will automatically prompt for a password if the server demands password authentication. However, pg_dump will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt.

--role=rolename
Specifies a role name to be used to create the dump. This option causes pg_dump to issue a SET ROLE <rolename> command after connecting to the database. It is useful when the authenticated user (specified by -U) lacks privileges needed by pg_dump, but can switch to a role with the required rights. Some installations have a policy against logging in directly as a superuser, and use of this option allows dumps to be made without violating the policy.

Environment

PGDATABASE; PGHOST; PGOPTIONS; PGPORT; PGUSER
Default connection parameters.

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

This utility, like most other PostgreSQL utilities, also uses the environment variables supported by libpq (see ???).

Diagnostics

pg_dump internally executes SELECT statements. If you have problems running pg_dump, make sure you are able to select information from the database using, for example, ???. Also, any default connection settings and environment variables used by the libpq front-end library will apply.

The database activity of pg_dump is normally collected by the cumulative statistics system. If this is undesirable, you can set parameter track_counts to false via PGOPTIONS or the ALTER USER command.

Notes

If your database cluster has any local additions to the template1 database, be careful to restore the output of pg_dump into a truly empty database; otherwise you are likely to get errors due to duplicate definitions of the added objects. To make an empty database without any local additions, copy from template0 not template1, for example:

CREATE DATABASE foo WITH TEMPLATE template0;

When options -e, -n or -t are specified, pg_dump makes no attempt to dump any other database objects that the selected object(s) might depend upon. Therefore, there is no guarantee that the results of a dump so generated can be successfully restored by themselves into a clean database. For example, if a table whose definition includes a foreign key is specified to be restored, the table referenced by the foreign key is not automatically restored.

When a dump without schema is chosen and the option --disable-triggers is used, pg_dump emits commands to disable triggers on user tables before inserting the data, and then commands to re-enable them after the data has been inserted. If the restore is stopped in the middle, the system catalogs might be left in the wrong state.

When creating an archive (non-text) output file, it is advisable not to restrict the set of database objects dumped, but instead plan to apply any desired object filtering when reading the archive with pg_restore. This will preserve flexibility and possibly avoid problems at restore time; for details see the ??? documentation. In particular, dumping table data without the corresponding table definition (via --no-schema and related options) is not recommended.

When --statistics is specified, pg_dump will include most optimizer statistics in the resulting dump file. This does not include all statistics, such as custom statistics added by an extension, or statistics collected by the cumulative statistics system. Therefore, it may still be useful to run ANALYZE after restoring from a dump file to ensure optimal performance; see ??? and ??? for more information.

Because pg_dump is used to transfer data to newer versions of PostgreSQL, the output of pg_dump can be expected to load into PostgreSQL server versions newer than pg_dump's version. pg_dump can also dump from PostgreSQL servers older than its own version. (Currently, servers back to version 10 are supported.) However, pg_dump cannot dump from PostgreSQL servers newer than its own major version; it will refuse to even try, rather than risk making an invalid dump. Also, it is not guaranteed that pg_dump's output can be loaded into a server of an older major version — not even if the dump was taken from a server of that version. Loading a dump file into an older server may require manual editing of the dump file to remove syntax not understood by the older server. Use of the --quote-all-identifiers option is recommended in cross-version cases, as it can prevent problems arising from varying reserved-word lists in different PostgreSQL versions.

When dumping logical replication subscriptions, pg_dump will generate CREATE SUBSCRIPTION commands that use the connect = false option, so that restoring the subscription does not make remote connections for creating a replication slot or for initial table copy. That way, the dump can be restored without requiring network access to the remote servers. It is then up to the user to reactivate the subscriptions in a suitable way. If the involved hosts have changed, the connection information might have to be changed. It might also be appropriate to truncate the target tables before initiating a new full table copy. If users intend to copy initial data during refresh they must create the slot with two_phase = false. After the initial sync, the two_phase option will be automatically enabled by the subscriber if the subscription had been originally created with two_phase = true option.

It is generally recommended to use the -X (--no-psqlrc) option when restoring a database from a plain-text pg_dump script to ensure a clean restore process and prevent potential conflicts with non-default psql configurations.

Examples

To dump a database called mydb into an SQL-script file:

$ pg_dump mydb > db.sql

To reload such a script into a (freshly created) database named newdb:

$ psql -X -d newdb -f db.sql

To dump a database into a custom-format archive file:

$ pg_dump -Fc mydb > db.dump

To dump a database into a directory-format archive:

$ pg_dump -Fd mydb -f dumpdir

To dump a database into a directory-format archive in parallel with 5 worker jobs:

$ pg_dump -Fd mydb -j 5 -f dumpdir

To reload an archive file into a (freshly created) database named newdb:

$ pg_restore -d newdb db.dump

To reload an archive file into the same database it was dumped from, discarding the current contents of that database:

$ pg_restore -d postgres --clean --create db.dump

To dump a single table named mytab:

$ pg_dump -t mytab mydb > db.sql

To dump all tables whose names start with emp in the detroit schema, except for the table named employee_log:

$ pg_dump -t 'detroit.emp*' -T detroit.employee_log mydb > db.sql

To dump all schemas whose names start with east or west and end in gsm, excluding any schemas whose names contain the word test:

$ pg_dump -n 'east*gsm' -n 'west*gsm' -N '*test*' mydb > db.sql

The same, using regular expression notation to consolidate the switches:

$ pg_dump -n '(east|west)*gsm' -N '*test*' mydb > db.sql

To dump all database objects except for tables whose names begin with ts_:

$ pg_dump -T 'ts_*' mydb > db.sql

To specify an upper-case or mixed-case name in -t and related switches, you need to double-quote the name; else it will be folded to lower case (see ???). But double quotes are special to the shell, so in turn they must be quoted. Thus, to dump a single table with a mixed-case name, you need something like

$ pg_dump -t "\"MixedCaseName\"" mydb > mytab.sql

To dump all tables whose names start with mytable, except for table mytable2, specify a filter file filter.txt like:

include table mytable*
exclude table mytable2

$ pg_dump --filter=filter.txt mydb > db.sql

See Also

pg_dumpall

pg_dumpall

pg_dumpall

1

Application

pg_dumpall

extract a

PostgreSQL

database cluster into a script file

pg_dumpall

connection-option

option

Description

pg_dumpall is a utility for writing out (“dumping”) all PostgreSQL databases of a cluster into one script file. The script file contains SQL commands that can be used as input to ??? to restore the databases. It does this by calling ??? for each database in the cluster. pg_dumpall also dumps global objects that are common to all databases, namely database roles, tablespaces, and privilege grants for configuration parameters. (pg_dump does not save these objects.)

Since pg_dumpall reads tables from all databases you will most likely have to connect as a database superuser in order to produce a complete dump. Also you will need superuser privileges to execute the saved script in order to be allowed to add roles and create databases.

The SQL script will be written to the standard output. Use the -f/--file option or shell operators to redirect it into a file.

pg_dumpall needs to connect several times to the PostgreSQL server (once per database). If you use password authentication it will ask for a password each time. It is convenient to have a ~/.pgpass file in such cases. See ??? for more information.

[!WARNING] Restoring a dump causes the destination to execute arbitrary code of the source superusers' choice. Partial dumps and partial restores do not limit that. If the source superusers are not trusted, the dumped SQL statements must be inspected before restoring. Note that the client running the dump and restore need not trust the source or destination superusers.

Options

The following command-line options control the content and format of the output.

-a; --data-only
Dump only the data, not the schema (data definitions) or statistics.

-c; --clean
Emit SQL commands to DROP all the dumped databases, roles, and tablespaces before recreating them. This option is useful when the restore is to overwrite an existing cluster. If any of the objects do not exist in the destination cluster, ignorable error messages will be reported during restore, unless --if-exists is also specified.

-E encoding; --encoding=encoding
Create the dump in the specified character set encoding. By default, the dump is created in the database encoding. (Another way to get the same result is to set the PGCLIENTENCODING environment variable to the desired dump encoding.)

-f filename; --file=filename
Send output to the specified file. If this is omitted, the standard output is used.

-g; --globals-only
Dump only global objects (roles and tablespaces), no databases.

-O; --no-owner
Do not output commands to set ownership of objects to match the original database. By default, pg_dumpall issues ALTER OWNER or SET SESSION AUTHORIZATION statements to set ownership of created schema elements. These statements will fail when the script is run unless it is started by a superuser (or the same user that owns all of the objects in the script). To make a script that can be restored by any user, but will give that user ownership of all the objects, specify -O.

-r; --roles-only
Dump only roles, no databases or tablespaces.

-s; --schema-only
Dump only the object definitions (schema), not data.

-S username; --superuser=username
Specify the superuser user name to use when disabling triggers. This is relevant only if --disable-triggers is used. (Usually, it's better to leave this out, and instead start the resulting script as superuser.)

-t; --tablespaces-only
Dump only tablespaces, no databases or roles.

-v; --verbose
Specifies verbose mode. This will cause pg_dumpall to output start/stop times to the dump file, and progress messages to standard error. Repeating the option causes additional debug-level messages to appear on standard error. The option is also passed down to pg_dump.

-V; --version
Print the pg_dumpall version and exit.

-x; --no-privileges; --no-acl
Prevent dumping of access privileges (grant/revoke commands).

--binary-upgrade
This option is for use by in-place upgrade utilities. Its use for other purposes is not recommended or supported. The behavior of the option may change in future releases without notice.

--column-inserts; --attribute-inserts
Dump data as INSERT commands with explicit column names (INSERT INTO table (column, ...) VALUES ...). This will make restoration very slow; it is mainly useful for making dumps that can be loaded into non-PostgreSQL databases.

--disable-dollar-quoting
This option disables the use of dollar quoting for function bodies, and forces them to be quoted using SQL standard string syntax.

--disable-triggers
This option is relevant only when creating a dump with data and without schema. It instructs pg_dumpall to include commands to temporarily disable triggers on the target tables while the data is restored. Use this if you have referential integrity checks or other triggers on the tables that you do not want to invoke during data restore.

Presently, the commands emitted for --disable-triggers must be done as superuser. So, you should also specify a superuser name with -S, or preferably be careful to start the resulting script as a superuser.

--exclude-database=pattern
Do not dump databases whose name matches <pattern>. Multiple patterns can be excluded by writing multiple --exclude-database switches. The <pattern> parameter is interpreted as a pattern according to the same rules used by psql's \d commands (see ???), so multiple databases can also be excluded by writing wildcard characters in the pattern. When using wildcards, be careful to quote the pattern if needed to prevent shell wildcard expansion.

--extra-float-digits=ndigits
Use the specified value of extra_float_digits when dumping floating-point data, instead of the maximum available precision. Routine dumps made for backup purposes should not use this option.

--filter=filename
Specify a filename from which to read patterns for databases excluded from the dump. The patterns are interpreted according to the same rules as --exclude-database. To read from STDIN, use - as the filename. The --filter option can be specified in conjunction with --exclude-database for excluding databases, and can also be specified more than once for multiple filter files.

The file lists one database pattern per row, with the following format: exclude database <PATTERN>

Lines starting with # are considered comments and ignored. Comments can be placed after an object pattern row as well. Blank lines are also ignored. See ??? for how to perform quoting in patterns.

--if-exists
Use DROP ... IF EXISTS commands to drop objects in --clean mode. This suppresses “does not exist” errors that might otherwise be reported. This option is not valid unless --clean is also specified.

--inserts
Dump data as INSERT commands (rather than COPY). This will make restoration very slow; it is mainly useful for making dumps that can be loaded into non-PostgreSQL databases. Note that the restore might fail altogether if you have rearranged column order. The --column-inserts option is safer, though even slower.

--load-via-partition-root
When dumping data for a table partition, make the COPY or INSERT statements target the root of the partitioning hierarchy that contains it, rather than the partition itself. This causes the appropriate partition to be re-determined for each row when the data is loaded. This may be useful when restoring data on a server where rows do not always fall into the same partitions as they did on the original server. That could happen, for example, if the partitioning column is of type text and the two systems have different definitions of the collation used to sort the partitioning column.

--lock-wait-timeout=timeout
Do not wait forever to acquire shared table locks at the beginning of the dump. Instead, fail if unable to lock a table within the specified <timeout>. The timeout may be specified in any of the formats accepted by SET statement_timeout.

--no-comments
Do not dump COMMENT commands.

--no-data
Do not dump data.

--no-policies
Do not dump row security policies.

--no-publications
Do not dump publications.

--no-role-passwords
Do not dump passwords for roles. When restored, roles will have a null password, and password authentication will always fail until the password is set. Since password values aren't needed when this option is specified, the role information is read from the catalog view pg_roles instead of pg_authid. Therefore, this option also helps if access to pg_authid is restricted by some security policy.

--no-schema
Do not dump schema (data definitions).

--no-security-labels
Do not dump security labels.

--no-statistics
Do not dump statistics. This is the default.

--no-subscriptions
Do not dump subscriptions.

--no-sync
By default, pg_dumpall will wait for all files to be written safely to disk. This option causes pg_dumpall to return without waiting, which is faster, but means that a subsequent operating system crash can leave the dump corrupt. Generally, this option is useful for testing but should not be used when dumping data from production installation.

--no-table-access-method
Do not output commands to select table access methods. With this option, all objects will be created with whichever table access method is the default during restore.

--no-tablespaces
Do not output commands to create tablespaces nor select tablespaces for objects. With this option, all objects will be created in whichever tablespace is the default during restore.

--no-toast-compression
Do not output commands to set TOAST compression methods. With this option, all columns will be restored with the default compression setting.

--no-unlogged-table-data
Do not dump the contents of unlogged tables. This option has no effect on whether or not the table definitions (schema) are dumped; it only suppresses dumping the table data.

--on-conflict-do-nothing
Add ON CONFLICT DO NOTHING to INSERT commands. This option is not valid unless --inserts or --column-inserts is also specified.

--quote-all-identifiers
Force quoting of all identifiers. This option is recommended when dumping a database from a server whose PostgreSQL major version is different from pg_dumpall's, or when the output is intended to be loaded into a server of a different major version. By default, pg_dumpall quotes only identifiers that are reserved words in its own major version. This sometimes results in compatibility issues when dealing with servers of other versions that may have slightly different sets of reserved words. Using --quote-all-identifiers prevents such issues, at the price of a harder-to-read dump script.

--restrict-key=restrict_key
Use the provided string as the psql \restrict key in the dump output. If no restrict key is specified, pg_dumpall will generate a random one as needed. Keys may contain only alphanumeric characters.

This option is primarily intended for testing purposes and other scenarios that require repeatable output (e.g., comparing dump files). It is not recommended for general use, as a malicious server with advance knowledge of the key may be able to inject arbitrary code that will be executed on the machine that runs psql with the dump output.

--rows-per-insert=nrows
Dump data as INSERT commands (rather than COPY). Controls the maximum number of rows per INSERT command. The value specified must be a number greater than zero. Any error during restoring will cause only rows that are part of the problematic INSERT to be lost, rather than the entire table contents.

--statistics
Dump optimizer statistics.

--statistics-only
Dump only the statistics, not the schema (data definitions) or data. Optimizer statistics for tables, materialized views, foreign tables, and indexes are dumped.

--sequence-data
Include sequence data in the dump. This is the default behavior except when --no-data, --schema-only, or --statistics-only is specified.

--use-set-session-authorization
Output SQL-standard SET SESSION AUTHORIZATION commands instead of ALTER OWNER commands to determine object ownership. This makes the dump more standards compatible, but depending on the history of the objects in the dump, might not restore properly.

-?; --help
Show help about pg_dumpall command line arguments, and exit.

The following command-line options control the database connection parameters.

-d connstr; --dbname=connstr
Specifies parameters used to connect to the server, as a connection string; these will override any conflicting command line options.

The option is called --dbname for consistency with other client applications, but because pg_dumpall needs to connect to many databases, the database name in the connection string will be ignored. Use the -l option to specify the name of the database used for the initial connection, which will dump global objects and discover what other databases should be dumped.

-h host; --host=host
Specifies the host name of the machine on which the database server is running. If the value begins with a slash, it is used as the directory for the Unix domain socket. The default is taken from the PGHOST environment variable, if set, else a Unix domain socket connection is attempted.

-l dbname; --database=dbname
Specifies the name of the database to connect to for dumping global objects and discovering what other databases should be dumped. If not specified, the postgres database will be used, and if that does not exist, template1 will be used.

-p port; --port=port
Specifies the TCP port or local Unix domain socket file extension on which the server is listening for connections. Defaults to the PGPORT environment variable, if set, or a compiled-in default.

-U username; --username=username
User name to connect as.

-w; --no-password
Never issue a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password.

-W; --password
Force pg_dumpall to prompt for a password before connecting to a database.

This option is never essential, since pg_dumpall will automatically prompt for a password if the server demands password authentication. However, pg_dumpall will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt.

Note that the password prompt will occur again for each database to be dumped. Usually, it's better to set up a ~/.pgpass file than to rely on manual password entry.

--role=rolename
Specifies a role name to be used to create the dump. This option causes pg_dumpall to issue a SET ROLE <rolename> command after connecting to the database. It is useful when the authenticated user (specified by -U) lacks privileges needed by pg_dumpall, but can switch to a role with the required rights. Some installations have a policy against logging in directly as a superuser, and use of this option allows dumps to be made without violating the policy.

Environment

PGHOST; PGOPTIONS; PGPORT; PGUSER
Default connection parameters

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

This utility, like most other PostgreSQL utilities, also uses the environment variables supported by libpq (see ???).

Notes

Since pg_dumpall calls pg_dump internally, some diagnostic messages will refer to pg_dump.

The --clean option can be useful even when your intention is to restore the dump script into a fresh cluster. Use of --clean authorizes the script to drop and re-create the built-in postgres and template1 databases, ensuring that those databases will retain the same properties (for instance, locale and encoding) that they had in the source cluster. Without the option, those databases will retain their existing database-level properties, as well as any pre-existing contents.

When --statistics is specified, pg_dumpall will include most optimizer statistics in the resulting dump file. This does not include all statistics, such as custom statistics added by an extension, or statistics collected by the cumulative statistics system. Therefore, it may still be useful to run ANALYZE on each database after restoring from a dump file to ensure optimal performance. You can also run vacuumdb -a -z to analyze all databases.

The dump script should not be expected to run completely without errors. In particular, because the script will issue CREATE ROLE for every role existing in the source cluster, it is certain to get a “role already exists” error for the bootstrap superuser, unless the destination cluster was initialized with a different bootstrap superuser name. This error is harmless and should be ignored. Use of the --clean option is likely to produce additional harmless error messages about non-existent objects, although you can minimize those by adding --if-exists.

pg_dumpall requires all needed tablespace directories to exist before the restore; otherwise, database creation will fail for databases in non-default locations.

It is generally recommended to use the -X (--no-psqlrc) option when restoring a database from a pg_dumpall script to ensure a clean restore process and prevent potential conflicts with non-default psql configurations. Additionally, because the pg_dumpall script may include psql meta-commands, it may be incompatible with clients other than psql.

Examples

To dump all databases:

$ pg_dumpall > db.out

To restore database(s) from this file, you can use:

$ psql -X -f db.out -d postgres

It is not important which database you connect to here since the script file created by pg_dumpall will contain the appropriate commands to create and connect to the saved databases. An exception is that if you specified --clean, you must connect to the postgres database initially; the script will attempt to drop other databases immediately, and that will fail for the database you are connected to.

See Also

Check ??? for details on possible error conditions.

pg_isready

pg_isready

pg_isready

1

Application

pg_isready

check the connection status of a

PostgreSQL

server

pg_isready

connection-option

option

Description

pg_isready is a utility for checking the connection status of a PostgreSQL database server. The exit status specifies the result of the connection check.

Options

-d dbname; --dbname=dbname
Specifies the name of the database to connect to. The <dbname> can be a connection string. If so, connection string parameters will override any conflicting command line options.

-h hostname; --host=hostname
Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for the Unix-domain socket.

-p port; --port=port
Specifies the TCP port or the local Unix-domain socket file extension on which the server is listening for connections. Defaults to the value of the PGPORT environment variable or, if not set, to the port specified at compile time, usually 5432.

-q; --quiet
Do not display status message. This is useful when scripting.

-t seconds; --timeout=seconds
The maximum number of seconds to wait when attempting connection before returning that the server is not responding. Setting to 0 disables. The default is 3 seconds.

-U username; --username=username
Connect to the database as the user <username> instead of the default.

-V; --version
Print the pg_isready version and exit.

-?; --help
Show help about pg_isready command line arguments, and exit.

Exit Status

pg_isready returns 0 to the shell if the server is accepting connections normally, 1 if the server is rejecting connections (for example during startup), 2 if there was no response to the connection attempt, and 3 if no attempt was made (for example due to invalid parameters).

Environment

pg_isready, like most other PostgreSQL utilities, also uses the environment variables supported by libpq (see ???).

The environment variable PG_COLOR specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

Notes

It is not necessary to supply correct user name, password, or database name values to obtain the server status; however, if incorrect values are provided, the server will log a failed connection attempt.

Examples

Standard Usage:

$ pg_isready
/tmp:5432 - accepting connections
$ echo $?
0

Running with connection parameters to a PostgreSQL cluster in startup:

$ pg_isready -h localhost -p 5433
localhost:5433 - rejecting connections
$ echo $?
1

Running with connection parameters to a non-responsive PostgreSQL cluster:

$ pg_isready -h someremotehost
someremotehost:5432 - no response
$ echo $?
2

pg_receivewal

pg_receivewal

pg_receivewal

1

Application

pg_receivewal

stream write-ahead logs from a

PostgreSQL

server

pg_receivewal

option

Description

pg_receivewal is used to stream the write-ahead log from a running PostgreSQL cluster. The write-ahead log is streamed using the streaming replication protocol, and is written to a local directory of files. This directory can be used as the archive location for doing a restore using point-in-time recovery (see ???).

pg_receivewal streams the write-ahead log in real time as it's being generated on the server, and does not wait for segments to complete like ??? and ??? do. For this reason, it is not necessary to set ??? when using pg_receivewal.

Unlike the WAL receiver of a PostgreSQL standby server, pg_receivewal by default flushes WAL data only when a WAL file is closed. The option --synchronous must be specified to flush WAL data in real time. Since pg_receivewal does not apply WAL, you should not allow it to become a synchronous standby when ??? equals remote_apply. If it does, it will appear to be a standby that never catches up, and will cause transaction commits to block. To avoid this, you should either configure an appropriate value for ???, or specify application_name for pg_receivewal that does not match it, or change the value of synchronous_commit to something other than remote_apply.

The write-ahead log is streamed over a regular PostgreSQL connection and uses the replication protocol. The connection must be made with a user having REPLICATION permissions (see ???) or a superuser, and pg_hba.conf must permit the replication connection. The server must also be configured with ??? set high enough to leave at least one session available for the stream.

The starting point of the write-ahead log streaming is calculated when pg_receivewal starts:

  1. First, scan the directory where the WAL segment files are written and find the newest completed segment file, using as the starting point the beginning of the next WAL segment file.

  2. If a starting point cannot be calculated with the previous method, and if a replication slot is used, an extra READ_REPLICATION_SLOT command is issued to retrieve the slot's restart_lsn to use as the starting point. This option is only available when streaming write-ahead logs from PostgreSQL 15 and up.

  3. If a starting point cannot be calculated with the previous method, the latest WAL flush location is used as reported by the server from an IDENTIFY_SYSTEM command.

If the connection is lost, or if it cannot be initially established, with a non-fatal error, pg_receivewal will retry the connection indefinitely, and reestablish streaming as soon as possible. To avoid this behavior, use the -n parameter.

In the absence of fatal errors, pg_receivewal will run until terminated by the SIGINT (Control+C) or SIGTERM signal.

Options

-D directory; --directory=directory
Directory to write the output to.

This parameter is required.

-E lsn; --endpos=lsn
Automatically stop replication and exit with normal exit status 0 when receiving reaches the specified LSN.

If there is a record with LSN exactly equal to <lsn>, the record will be processed.

--if-not-exists
Do not error out when --create-slot is specified and a slot with the specified name already exists.

-n; --no-loop
Don't loop on connection errors. Instead, exit right away with an error.

--no-sync
This option causes pg_receivewal to not force WAL data to be flushed to disk. This is faster, but means that a subsequent operating system crash can leave the WAL segments corrupt. Generally, this option is useful for testing but should not be used when doing WAL archiving on a production deployment.

This option is incompatible with --synchronous.

-s interval; --status-interval=interval
Specifies the number of seconds between status packets sent back to the server. This allows for easier monitoring of the progress from server. A value of zero disables the periodic status updates completely, although an update will still be sent when requested by the server, to avoid timeout disconnect. The default value is 10 seconds.

-S slotname; --slot=slotname
Require pg_receivewal to use an existing replication slot (see ???). When this option is used, pg_receivewal will report a flush position to the server, indicating when each segment has been synchronized to disk so that the server can remove that segment if it is not otherwise needed.

When the replication client of pg_receivewal is configured on the server as a synchronous standby, then using a replication slot will report the flush position to the server, but only when a WAL file is closed. Therefore, that configuration will cause transactions on the primary to wait for a long time and effectively not work satisfactorily. The option --synchronous (see below) must be specified in addition to make this work correctly.

--synchronous
Flush the WAL data to disk immediately after it has been received. Also send a status packet back to the server immediately after flushing, regardless of --status-interval.

This option should be specified if the replication client of pg_receivewal is configured on the server as a synchronous standby, to ensure that timely feedback is sent to the server.

-v; --verbose
Enables verbose mode.

-Z level; -Z method[:detail]; --compress=level; --compress=method[:detail]
Enables compression of write-ahead logs.

The compression method can be set to gzip, lz4 (if PostgreSQL was compiled with --with-lz4) or none for no compression. A compression detail string can optionally be specified. If the detail string is an integer, it specifies the compression level. Otherwise, it should be a comma-separated list of items, each of the form <keyword> or <keyword=value>. Currently, the only supported keyword is level.

If no compression level is specified, the default compression level will be used. If only a level is specified without mentioning an algorithm, gzip compression will be used if the level is greater than 0, and no compression will be used if the level is 0.

The suffix .gz will automatically be added to all filenames when using gzip, and the suffix .lz4 is added when using lz4.

The following command-line options control the database connection parameters.

-d connstr; --dbname=connstr
Specifies parameters used to connect to the server, as a connection string; these will override any conflicting command line options.

This option is called --dbname for consistency with other client applications, but because pg_receivewal doesn't connect to any particular database in the cluster, any database name included in the connection string will be ignored by the server. However, a database name supplied that way overrides the default database name (replication) for purposes of looking up the replication connection's password in ~/.pgpass. Similarly, middleware or proxies used in connecting to PostgreSQL might utilize the name for purposes such as connection routing.

-h host; --host=host
Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for the Unix domain socket. The default is taken from the PGHOST environment variable, if set, else a Unix domain socket connection is attempted.

-p port; --port=port
Specifies the TCP port or local Unix domain socket file extension on which the server is listening for connections. Defaults to the PGPORT environment variable, if set, or a compiled-in default.

-U username; --username=username
User name to connect as.

-w; --no-password
Never issue a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password.

-W; --password
Force pg_receivewal to prompt for a password before connecting to a database.

This option is never essential, since pg_receivewal will automatically prompt for a password if the server demands password authentication. However, pg_receivewal will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt.

pg_receivewal can perform one of the two following actions in order to control physical replication slots:

--create-slot
Create a new physical replication slot with the name specified in --slot, then exit.

--drop-slot
Drop the replication slot with the name specified in --slot, then exit.

Other options are also available:

-V; --version
Print the pg_receivewal version and exit.

-?; --help
Show help about pg_receivewal command line arguments, and exit.

Exit Status

pg_receivewal will exit with status 0 when terminated by the SIGINT or SIGTERM signal. (That is the normal way to end it. Hence it is not an error.) For fatal errors or other signals, the exit status will be nonzero.

Environment

This utility, like most other PostgreSQL utilities, uses the environment variables supported by libpq (see ???).

The environment variable PG_COLOR specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

Notes

When using pg_receivewal instead of ??? or ??? as the main WAL backup method, it is strongly recommended to use replication slots. Otherwise, the server is free to recycle or remove write-ahead log files before they are backed up, because it does not have any information, either from ??? or ??? or the replication slots, about how far the WAL stream has been archived. Note, however, that a replication slot will fill up the server's disk space if the receiver does not keep up with fetching the WAL data.

pg_receivewal will preserve group permissions on the received WAL files if group permissions are enabled on the source cluster.

Examples

To stream the write-ahead log from the server at mydbserver and store it in the local directory /usr/local/pgsql/archive:

$ pg_receivewal -h mydbserver -D /usr/local/pgsql/archive

See Also

pg_recvlogical

pg_recvlogical

pg_recvlogical

1

Application

pg_recvlogical

control

PostgreSQL

logical decoding streams

pg_recvlogical

option

Description

pg_recvlogical controls logical decoding replication slots and streams data from such replication slots.

It creates a replication-mode connection, so it is subject to the same constraints as ???, plus those for logical replication (see ???).

pg_recvlogical has no equivalent to the logical decoding SQL interface's peek and get modes. It sends replay confirmations for data lazily as it receives it and on clean exit. To examine pending data on a slot without consuming it, use pg_logical_slot_peek_changes.

In the absence of fatal errors, pg_recvlogical will run until terminated by the SIGINT (Control+C) or SIGTERM signal.

When pg_recvlogical receives a SIGHUP signal, it closes the current output file and opens a new one using the filename specified by the --file option. This allows us to rotate the output file by first renaming the current file and then sending a SIGHUP signal to pg_recvlogical.

Options

At least one of the following options must be specified to select an action:

--create-slot
Create a new logical replication slot with the name specified by --slot, using the output plugin specified by --plugin, for the database specified by --dbname.

The --slot and --dbname options are required for this action.

The --enable-two-phase and --enable-failover options can be specified with --create-slot.

--drop-slot
Drop the replication slot with the name specified by --slot, then exit.

The --slot option is required for this action.

--start
Begin streaming changes from the logical replication slot specified by --slot, continuing until terminated by a signal. If the server side change stream ends with a server shutdown or disconnect, retry in a loop unless --no-loop is specified.

The --slot, --dbname, and --file options are required for this action.

The stream format is determined by the output plugin specified when the slot was created.

The connection must be to the same database used to create the slot.

--create-slot and --start can be specified together. --drop-slot cannot be combined with another action.

The following command-line options control the location and format of the output and other replication behavior:

-E lsn; --endpos=lsn
In --start mode, automatically stop replication and exit with normal exit status 0 when receiving reaches the specified LSN. If specified when not in --start mode, an error is raised.

If there's a record with LSN exactly equal to <lsn>, the record will be output.

The --endpos option is not aware of transaction boundaries and may truncate output partway through a transaction. Any partially output transaction will not be consumed and will be replayed again when the slot is next read from. Individual messages are never truncated.

--enable-failover
Enables the slot to be synchronized to the standbys. This option may only be specified with --create-slot.

-f filename; --file=filename
Write received and decoded transaction data into this file. Use - for stdout.

This parameter is required for --start.

-F interval_seconds; --fsync-interval=interval_seconds
Specifies how often pg_recvlogical should issue fsync() calls to ensure the output file is safely flushed to disk. The default value is 10 seconds.

The server will occasionally request the client to perform a flush and report the flush position to the server. This setting is in addition to that, to perform flushes more frequently.

Specifying an interval of 0 disables issuing fsync() calls altogether, while still reporting progress to the server. In this case, data could be lost in the event of a crash.

-I lsn; --startpos=lsn
In --start mode, start replication from the given LSN. For details on the effect of this, see the documentation in ??? and ???. Ignored in other modes.

--if-not-exists
Do not error out when --create-slot is specified and a slot with the specified name already exists.

-n; --no-loop
When the connection to the server is lost, do not retry in a loop, just exit.

-o name[=value]; --option=name[=value]
Pass the option <name> to the output plugin with, if specified, the option value <value>. Which options exist and their effects depends on the used output plugin.

-P plugin; --plugin=plugin
When creating a slot, use the specified logical decoding output plugin. See ??? for information about the plugins PostgreSQL provides. The default is ???. This option has no effect if the slot already exists.

-s interval_seconds; --status-interval=interval_seconds
This option has the same effect as the option of the same name in ???. See the description there.

-S slot_name; --slot=slot_name
In --start mode, use the existing logical replication slot named <slot_name>. In --create-slot mode, create the slot with this name. In --drop-slot mode, delete the slot with this name.

This parameter is required for any of actions.

-t; --enable-two-phase; --two-phase (deprecated)
Enables decoding of prepared transactions. This option may only be specified with --create-slot.

-v; --verbose
Enables verbose mode.

The following command-line options control the database connection parameters.

-d dbname; --dbname=dbname
The database to connect to. See the description of the actions for what this means in detail. The <dbname> can be a connection string. If so, connection string parameters will override any conflicting command line options.

This parameter is required for --create-slot and --start.

-h hostname-or-ip; --host=hostname-or-ip
Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for the Unix domain socket. The default is taken from the PGHOST environment variable, if set, else a Unix domain socket connection is attempted.

-p port; --port=port
Specifies the TCP port or local Unix domain socket file extension on which the server is listening for connections. Defaults to the PGPORT environment variable, if set, or a compiled-in default.

-U user; --username=user
User name to connect as. Defaults to current operating system user name.

-w; --no-password
Never issue a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password.

-W; --password
Force pg_recvlogical to prompt for a password before connecting to a database.

This option is never essential, since pg_recvlogical will automatically prompt for a password if the server demands password authentication. However, pg_recvlogical will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt.

The following additional options are available:

-V; --version
Print the pg_recvlogical version and exit.

-?; --help
Show help about pg_recvlogical command line arguments, and exit.

Exit Status

pg_recvlogical will exit with status 0 when terminated by the SIGINT or SIGTERM signal. (That is the normal way to end it. Hence it is not an error.) For fatal errors or other signals, the exit status will be nonzero.

Environment

This utility, like most other PostgreSQL utilities, uses the environment variables supported by libpq (see ???).

The environment variable PG_COLOR specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

Notes

pg_recvlogical will preserve group permissions on the output files if group permissions are enabled on the source cluster.

Examples

See ??? for an example.

See Also

pg_resetwal

pg_resetwal

pg_resetwal

1

Application

pg_resetwal

reset the write-ahead log and other control information of a

PostgreSQL

database cluster

pg_resetwal

-f

--force

-n

--dry-run

option

-D

--pgdata

datadir

Description

pg_resetwal clears the write-ahead log (WAL) and optionally resets some other control information stored in the pg_control file. This function is sometimes needed if these files have become corrupted. It should be used only as a last resort, when the server will not start due to such corruption.

Some options, such as --wal-segsize (see below), can also be used to modify certain global settings of a database cluster without the need to rerun initdb. This can be done safely on an otherwise sound database cluster, if none of the dangerous modes mentioned below are used.

If pg_resetwal is used on a data directory where the server has been cleanly shut down and the control file is sound, then it will have no effect on the contents of the database system, except that no longer used WAL files are cleared away. Any other use is potentially dangerous and must be done with great care. pg_resetwal will require the -f (force) option to be specified before working on a data directory in an unclean shutdown state or with a corrupted control file.

After running this command on a data directory with corrupted WAL or a corrupted control file, it should be possible to start the server, but bear in mind that the database might contain inconsistent data due to partially-committed transactions. You should immediately dump your data, run initdb, and restore. After restore, check for inconsistencies and repair as needed.

If pg_resetwal complains that it cannot determine valid data for pg_control, you can force it to proceed anyway by specifying the -f (force) option. In this case plausible values will be substituted for the missing data. Most of the fields can be expected to match, but manual assistance might be needed for the next OID, next transaction ID and epoch, next multitransaction ID and offset, and WAL starting location fields. These fields can be set using the options discussed below. If you are not able to determine correct values for all these fields, -f can still be used, but the recovered database must be treated with even more suspicion than usual: an immediate dump and restore is imperative. Do not execute any data-modifying operations in the database before you dump, as any such action is likely to make the corruption worse.

This utility can only be run by the user who installed the server, because it requires read/write access to the data directory.

Options

<datadir>; -D datadir; --pgdata=datadir
Specifies the location of the database directory. For safety reasons, you must specify the data directory on the command line. pg_resetwal does not use the environment variable PGDATA.

-f; --force
Force pg_resetwal to proceed even in situations where it could be dangerous, as explained above. Specifically, this option is required to proceed if the server had not been cleanly shut down or if pg_resetwal cannot determine valid data for pg_control.

-n; --dry-run
The -n/--dry-run option instructs pg_resetwal to print the values reconstructed from pg_control and values about to be changed, and then exit without modifying anything. This is mainly a debugging tool, but can be useful as a sanity check before allowing pg_resetwal to proceed for real.

-V; --version
Display version information, then exit.

-?; --help
Show help, then exit.

The following options are only needed when pg_resetwal is unable to determine appropriate values by reading pg_control. Safe values can be determined as described below. For values that take numeric arguments, hexadecimal values can be specified by using the prefix 0x. Note that these instructions only apply with the standard block size of 8 kB.

-c xid,xid; --commit-timestamp-ids=xid,xid
Manually set the oldest and newest transaction IDs for which the commit time can be retrieved.

A safe value for the oldest transaction ID for which the commit time can be retrieved (first part) can be determined by looking for the numerically smallest file name in the directory pg_commit_ts under the data directory. Conversely, a safe value for the newest transaction ID for which the commit time can be retrieved (second part) can be determined by looking for the numerically greatest file name in the same directory. The file names are in hexadecimal.

-e xid_epoch; --epoch=xid_epoch
Manually set the next transaction ID's epoch.

The transaction ID epoch is not actually stored anywhere in the database except in the field that is set by pg_resetwal, so any value will work so far as the database itself is concerned. You might need to adjust this value to ensure that replication systems such as Slony-I and Skytools work correctly — if so, an appropriate value should be obtainable from the state of the downstream replicated database.

-l walfile; --next-wal-file=walfile
Manually set the WAL starting location by specifying the name of the next WAL segment file.

The name of next WAL segment file should be larger than any WAL segment file name currently existing in the directory pg_wal under the data directory. These names are also in hexadecimal and have three parts. The first part is the “timeline ID” and should usually be kept the same. For example, if 00000001000000320000004A is the largest entry in pg_wal, use -l 00000001000000320000004B or higher.

Note that when using nondefault WAL segment sizes, the numbers in the WAL file names are different from the LSNs that are reported by system functions and system views. This option takes a WAL file name, not an LSN.

[!NOTE] pg_resetwal itself looks at the files in pg_wal and chooses a default -l setting beyond the last existing file name. Therefore, manual adjustment of -l should only be needed if you are aware of WAL segment files that are not currently present in pg_wal, such as entries in an offline archive; or if the contents of pg_wal have been lost entirely.

-m mxid,mxid; --multixact-ids=mxid,mxid
Manually set the next and oldest multitransaction ID.

A safe value for the next multitransaction ID (first part) can be determined by looking for the numerically largest file name in the directory pg_multixact/offsets under the data directory, adding one, and then multiplying by 32768 (0x8000). Conversely, a safe value for the oldest multitransaction ID (second part of -m) can be determined by looking for the numerically smallest file name in the same directory and multiplying by 32768 (0x8000). Note that the file names are in hexadecimal. It is usually easiest to specify the option value in hexadecimal too. For example, if 000F and 0007 are the greatest and smallest entries in pg_multixact/offsets, -m 0x80000,0x38000 will work.

-o oid8; --next-oid=oid8
Manually set the next OID (8 bytes).

There is no comparably easy way to determine a next OID that's beyond the largest one in the database, but fortunately it is not critical to get the next-OID setting right, as long as the counter is not moved backwards. The counter is 8 bytes wide and never wraps around during the life of a cluster.

-O mxoff; --multixact-offset=mxoff
Manually set the next multitransaction offset.

A safe value can be determined by looking for the numerically largest file name in the directory pg_multixact/members under the data directory, adding one, and then multiplying by 52352 (0xCC80). The file names are in hexadecimal. There is no simple recipe such as the ones for other options of appending zeroes.

-u xid; --oldest-transaction-id=xid
Manually set the oldest unfrozen transaction ID.

A safe value can be determined by looking for the numerically smallest file name in the directory pg_xact under the data directory and then multiplying by 1048576 (0x100000). Note that the file names are in hexadecimal. It is usually easiest to specify the option value in hexadecimal too. For example, if 0007 is the smallest entry in pg_xact, -u 0x700000 will work (five trailing zeroes provide the proper multiplier).

-x xid; --next-transaction-id=xid
Manually set the next transaction ID.

A safe value can be determined by looking for the numerically largest file name in the directory pg_xact under the data directory, adding one, and then multiplying by 1048576 (0x100000). Note that the file names are in hexadecimal. It is usually easiest to specify the option value in hexadecimal too. For example, if 0011 is the largest entry in pg_xact, -x 0x1200000 will work (five trailing zeroes provide the proper multiplier).

--char-signedness=option
Manually set the default char signedness. Possible values are signed and unsigned.

For a database cluster that pg_upgrade upgraded from a PostgreSQL version before 18, the safe value would be the default char signedness of the platform that ran the cluster before that upgrade. For all other clusters, signed would be the safe value. However, this option is exclusively for use with pg_upgrade and should not normally be used manually.

--wal-segsize=wal_segment_size
Set the new WAL segment size, in megabytes. The value must be set to a power of 2 between 1 and 1024 (megabytes). See the same option of ??? for more information.

This option can also be used to change the WAL segment size of an existing database cluster, avoiding the need to re-initdb.

[!NOTE] While pg_resetwal will set the WAL starting address beyond the latest existing WAL segment file, some segment size changes can cause previous WAL file names to be reused. It is recommended to use -l together with this option to manually set the WAL starting address if WAL file name overlap will cause problems with your archiving strategy.

Environment

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

Notes

This command must not be used when the server is running. pg_resetwal will refuse to start up if it finds a server lock file in the data directory. If the server crashed then a lock file might have been left behind; in that case you can remove the lock file to allow pg_resetwal to run. But before you do so, make doubly certain that there is no server process still alive.

pg_resetwal works only with servers of the same major version.

See Also

pg_restore

pg_restore

pg_restore

1

Application

pg_restore

restore a

PostgreSQL

database from an archive file created by

pg_dump

pg_restore

connection-option

option

filename

Description

pg_restore is a utility for restoring a PostgreSQL database from an archive created by ??? in one of the non-plain-text formats. It will issue the commands necessary to reconstruct the database to the state it was in at the time it was saved. The archive files also allow pg_restore to be selective about what is restored, or even to reorder the items prior to being restored. The archive files are designed to be portable across architectures.

pg_restore can operate in two modes. If a database name is specified, pg_restore connects to that database and restores archive contents directly into the database. Otherwise, a script containing the SQL commands necessary to rebuild the database is created and written to a file or standard output. This script output is equivalent to the plain text output format of pg_dump. Some of the options controlling the output are therefore analogous to pg_dump options.

Obviously, pg_restore cannot restore information that is not present in the archive file. For instance, if the archive was made using the “dump data as INSERT commands” option, pg_restore will not be able to load the data using COPY statements.

[!WARNING] Restoring a dump causes the destination to execute arbitrary code of the source superusers' choice. Partial dumps and partial restores do not limit that. If the source superusers are not trusted, the dumped SQL statements must be inspected before restoring. Non-plain-text dumps can be inspected by using pg_restore's --file option. Note that the client running the dump and restore need not trust the source or destination superusers.

Options

pg_restore accepts the following command line arguments.

<filename>
Specifies the location of the archive file (or directory, for a directory-format archive) to be restored. If not specified, the standard input is used.

-a; --data-only
Restore only the data, not the schema (data definitions) or statistics. Table data, large objects, and sequence values are restored, if present in the archive.

This option is similar to, but for historical reasons not identical to, specifying --section=data.

-c; --clean
Before restoring database objects, issue commands to DROP all the objects that will be restored. This option is useful for overwriting an existing database. If any of the objects do not exist in the destination database, ignorable error messages will be reported, unless --if-exists is also specified.

-C; --create
Create the database before restoring into it. If --clean is also specified, drop and recreate the target database before connecting to it.

With --create, pg_restore also restores the database's comment if any, and any configuration variable settings that are specific to this database, that is, any ALTER DATABASE ... SET ... and ALTER ROLE ... IN DATABASE ... SET ... commands that mention this database. Access privileges for the database itself are also restored, unless --no-acl is specified.

When this option is used, the database named with -d is used only to issue the initial DROP DATABASE and CREATE DATABASE commands. All data is restored into the database name that appears in the archive.

This option cannot be used together with --single-transaction.

-d dbname; --dbname=dbname
Connect to database <dbname> and restore directly into the database. The <dbname> can be a connection string. If so, connection string parameters will override any conflicting command line options.

-e; --exit-on-error
Exit if an error is encountered while sending SQL commands to the database. The default is to continue and to display a count of errors at the end of the restoration.

-f filename; --file=filename
Specify output file for generated script, or for the listing when used with -l. Use - for stdout.

-F format; --format=format
Specify format of the archive. It is not necessary to specify the format, since pg_restore will determine the format automatically. If specified, it can be one of the following:

c; custom
The archive is in the custom format of pg_dump.

d; directory
The archive is a directory archive.

t; tar
The archive is a tar archive.

-I index; --index=index
Restore definition of named index only. Multiple indexes may be specified with multiple -I switches.

-j njobs; --jobs=njobs
Run the most time-consuming steps of pg_restore — those that load data, create indexes, or create constraints — concurrently, using up to <njobs> concurrent sessions. This option can dramatically reduce the time to restore a large database to a server running on a multiprocessor machine. This option is ignored when emitting a script rather than connecting directly to a database server.

Each job is one process or one thread, depending on the operating system, and uses a separate connection to the server.

The optimal value for this option depends on the hardware setup of the server, of the client, and of the network. Factors include the number of CPU cores and the disk setup. A good place to start is the number of CPU cores on the server, but values larger than that can also lead to faster restore times in many cases. Of course, values that are too high will lead to decreased performance because of thrashing.

Only the custom and directory archive formats are supported with this option. The input must be a regular file or directory (not, for example, a pipe or standard input). Also, multiple jobs cannot be used together with the option --single-transaction.

-l; --list
List the table of contents of the archive. The output of this operation can be used as input to the -L option. Note that if filtering switches such as -n or -t are used with -l, they will restrict the items listed.

-L list-file; --use-list=list-file
Restore only those archive elements that are listed in <list-file>, and restore them in the order they appear in the file. Note that if filtering switches such as -n or -t are used with -L, they will further restrict the items restored.

<list-file> is normally created by editing the output of a previous -l operation. Lines can be moved or removed, and can also be commented out by placing a semicolon (;) at the start of the line. See below for examples.

-n schema; --schema=schema
Restore only objects that are in the named schema. Multiple schemas may be specified with multiple -n switches. This can be combined with the -t option to restore just a specific table.

-N schema; --exclude-schema=schema
Do not restore objects that are in the named schema. Multiple schemas to be excluded may be specified with multiple -N switches.

When both -n and -N are given for the same schema name, the -N switch wins and the schema is excluded.

-O; --no-owner
Do not output commands to set ownership of objects to match the original database. By default, pg_restore issues ALTER OWNER or SET SESSION AUTHORIZATION statements to set ownership of created schema elements. These statements will fail unless the initial connection to the database is made by a superuser (or the same user that owns all of the objects in the script). With -O, any user name can be used for the initial connection, and this user will own all the created objects.

-P function-name(argtype [, ...]); --function=function-name(argtype [, ...])
Restore the named function only. Be careful to spell the function name and arguments exactly as they appear in the dump file's table of contents. Multiple functions may be specified with multiple -P switches.

-R; --no-reconnect
This option is obsolete but still accepted for backwards compatibility.

-s; --schema-only
Restore only the schema (data definitions), not data, to the extent that schema entries are present in the archive.

This option cannot be used with --data-only or --statistics-only. It is similar to, but for historical reasons not identical to, specifying --section=pre-data --section=post-data --no-statistics.

(Do not confuse this with the --schema option, which uses the word “schema” in a different meaning.)

-S username; --superuser=username
Specify the superuser user name to use when disabling triggers. This is relevant only if --disable-triggers is used.

-t table; --table=table
Restore definition and/or data of only the named table. For this purpose, “table” includes views, materialized views, sequences, and foreign tables. Multiple tables can be selected by writing multiple -t switches. This option can be combined with the -n option to specify table(s) in a particular schema.

[!NOTE] This flag does not behave identically to the -t flag of pg_dump. There is not currently any provision for wild-card matching in pg_restore, nor can you include a schema name within its -t. And, while pg_dump's -t flag will also dump subsidiary objects (such as indexes) of the selected table(s), pg_restore's -t flag does not include such subsidiary objects.

[!NOTE] In versions prior to PostgreSQL 9.6, this flag matched only tables, not any other type of relation.

-T trigger; --trigger=trigger
Restore named trigger only. Multiple triggers may be specified with multiple -T switches.

-v; --verbose
Specifies verbose mode. This will cause pg_restore to output detailed object comments and start/stop times to the output file, and progress messages to standard error. Repeating the option causes additional debug-level messages to appear on standard error.

-V; --version
Print the pg_restore version and exit.

-x; --no-privileges; --no-acl
Prevent restoration of access privileges (grant/revoke commands).

-1; --single-transaction
Execute the restore as a single transaction (that is, wrap the emitted commands in BEGIN/COMMIT). This ensures that either all the commands complete successfully, or no changes are applied. This option implies --exit-on-error. It cannot be used together with --create, nor with multiple jobs (--jobs).

--disable-triggers
This option is relevant only when performing a restore without schema. It instructs pg_restore to execute commands to temporarily disable triggers on the target tables while the data is restored. Use this if you have referential integrity checks or other triggers on the tables that you do not want to invoke during data restore.

Presently, the commands emitted for --disable-triggers must be done as superuser. So you should also specify a superuser name with -S or, preferably, run pg_restore as a PostgreSQL superuser.

--enable-row-security
This option is relevant only when restoring the contents of a table which has row security. By default, pg_restore will set ??? to off, to ensure that all data is restored in to the table. If the user does not have sufficient privileges to bypass row security, then an error is thrown. This parameter instructs pg_restore to set ??? to on instead, allowing the user to attempt to restore the contents of the table with row security enabled. This might still fail if the user does not have the right to insert the rows from the dump into the table.

Note that this option currently also requires the dump be in INSERT format, as COPY FROM does not support row security.

--filter=filename
Specify a filename from which to read patterns for objects excluded or included from restore. The patterns are interpreted according to the same rules as -n/--schema for including objects in schemas, -N/--exclude-schema for excluding objects in schemas, -P/--function for restoring named functions, -I/--index for restoring named indexes, -t/--table for restoring named tables or -T/--trigger for restoring triggers. To read from STDIN, use - as the filename. The --filter option can be specified in conjunction with the above listed options for including or excluding objects, and can also be specified more than once for multiple filter files.

The file lists one database pattern per row, with the following format: { include | exclude } { function | index | schema | table | trigger } <PATTERN>

The first keyword specifies whether the objects matched by the pattern are to be included or excluded. The second keyword specifies the type of object to be filtered using the pattern:

Lines starting with # are considered comments and ignored. Comments can be placed after an object pattern row as well. Blank lines are also ignored. See ??? for how to perform quoting in patterns.

--if-exists
Use DROP ... IF EXISTS commands to drop objects in --clean mode. This suppresses “does not exist” errors that might otherwise be reported. This option is not valid unless --clean is also specified.

--no-comments
Do not output commands to restore comments, even if the archive contains them.

--no-data
Do not output commands to restore data, even if the archive contains them.

--no-data-for-failed-tables
By default, table data is restored even if the creation command for the table failed (e.g., because it already exists). With this option, data for such a table is skipped. This behavior is useful if the target database already contains the desired table contents. For example, auxiliary tables for PostgreSQL extensions such as PostGIS might already be loaded in the target database; specifying this option prevents duplicate or obsolete data from being loaded into them.

This option is effective only when restoring directly into a database, not when producing SQL script output.

--no-policies
Do not output commands to restore row security policies, even if the archive contains them.

--no-publications
Do not output commands to restore publications, even if the archive contains them.

--no-schema
Do not output commands to restore schema (data definitions), even if the archive contains them.

--no-security-labels
Do not output commands to restore security labels, even if the archive contains them.

--no-statistics
Do not output commands to restore statistics, even if the archive contains them.

--no-subscriptions
Do not output commands to restore subscriptions, even if the archive contains them.

--no-table-access-method
Do not output commands to select table access methods. With this option, all objects will be created with whichever table access method is the default during restore.

--no-tablespaces
Do not output commands to select tablespaces. With this option, all objects will be created in whichever tablespace is the default during restore.

--restrict-key=restrict_key
Use the provided string as the psql \restrict key in the dump output. This can only be specified for SQL script output, i.e., when the --file option is used. If no restrict key is specified, pg_restore will generate a random one as needed. Keys may contain only alphanumeric characters.

This option is primarily intended for testing purposes and other scenarios that require repeatable output (e.g., comparing dump files). It is not recommended for general use, as a malicious server with advance knowledge of the key may be able to inject arbitrary code that will be executed on the machine that runs psql with the dump output.

--section=sectionname
Only restore the named section. The section name can be pre-data, data, or post-data. This option can be specified more than once to select multiple sections. The default is to restore all sections.

The data section contains actual table data as well as large-object definitions. Post-data items consist of definitions of indexes, triggers, rules and constraints other than validated check constraints. Pre-data items consist of all other data definition items.

--statistics
Output commands to restore statistics, if the archive contains them. This is the default.

--statistics-only
Restore only the statistics, not schema (data definitions) or data.

--strict-names
Require that each schema (-n/--schema) and table (-t/--table) qualifier match at least one schema/table in the file to be restored.

--transaction-size=N
Execute the restore as a series of transactions, each processing up to <N> database objects. This option implies --exit-on-error.

--transaction-size offers an intermediate choice between the default behavior (one transaction per SQL command) and -1/--single-transaction (one transaction for all restored objects). While --single-transaction has the least overhead, it may be impractical for large databases because the transaction will take a lock on each restored object, possibly exhausting the server's lock table space. Using --transaction-size with a size of a few thousand objects offers nearly the same performance benefits while capping the amount of lock table space needed.

--use-set-session-authorization
Output SQL-standard SET SESSION AUTHORIZATION commands instead of ALTER OWNER commands to determine object ownership. This makes the dump more standards-compatible, but depending on the history of the objects in the dump, might not restore properly.

-?; --help
Show help about pg_restore command line arguments, and exit.

pg_restore also accepts the following command line arguments for connection parameters:

-h host; --host=host
Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for the Unix domain socket. The default is taken from the PGHOST environment variable, if set, else a Unix domain socket connection is attempted.

-p port; --port=port
Specifies the TCP port or local Unix domain socket file extension on which the server is listening for connections. Defaults to the PGPORT environment variable, if set, or a compiled-in default.

-U username; --username=username
User name to connect as.

-w; --no-password
Never issue a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password.

-W; --password
Force pg_restore to prompt for a password before connecting to a database.

This option is never essential, since pg_restore will automatically prompt for a password if the server demands password authentication. However, pg_restore will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt.

--role=rolename
Specifies a role name to be used to perform the restore. This option causes pg_restore to issue a SET ROLE <rolename> command after connecting to the database. It is useful when the authenticated user (specified by -U) lacks privileges needed by pg_restore, but can switch to a role with the required rights. Some installations have a policy against logging in directly as a superuser, and use of this option allows restores to be performed without violating the policy.

Environment

PGHOST; PGOPTIONS; PGPORT; PGUSER
Default connection parameters

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

This utility, like most other PostgreSQL utilities, also uses the environment variables supported by libpq (see ???). However, it does not read PGDATABASE when a database name is not supplied.

Diagnostics

When a direct database connection is specified using the -d option, pg_restore internally executes SQL statements. If you have problems running pg_restore, make sure you are able to select information from the database using, for example, ???. Also, any default connection settings and environment variables used by the libpq front-end library will apply.

Notes

When options -n or -t are specified, pg_restore makes no attempt to restore any other database objects that the selected table(s) or schema(s) might depend upon. Therefore, there is no guarantee that a specific-table restore into a clean database will succeed. For example, if a table whose definition includes a foreign key is specified to be restored, the table referenced by the foreign key is not automatically restored.

If your installation has any local additions to the template1 database, be careful to load the output of pg_restore into a truly empty database; otherwise you are likely to get errors due to duplicate definitions of the added objects. To make an empty database without any local additions, copy from template0 not template1, for example:

CREATE DATABASE foo WITH TEMPLATE template0;

The limitations of pg_restore are detailed below.

See also the ??? documentation for details on limitations of pg_dump.

By default, pg_restore will restore optimizer statistics if included in the dump file. If not all statistics were restored, it may be useful to run ANALYZE on each restored table so the optimizer has useful statistics; see ??? and ??? for more information.

Examples

Assume we have dumped a database called mydb into a custom-format dump file:

$ pg_dump -Fc mydb > db.dump

To drop the database and recreate it from the dump:

$ dropdb mydb
$ pg_restore -C -d postgres db.dump

The database named in the -d switch can be any database existing in the cluster; pg_restore only uses it to issue the CREATE DATABASE command for mydb. With -C, data is always restored into the database name that appears in the dump file.

To restore the dump into a new database called newdb:

$ createdb -T template0 newdb
$ pg_restore -d newdb db.dump

Notice we don't use -C, and instead connect directly to the database to be restored into. Also note that we clone the new database from template0 not template1, to ensure it is initially empty.

To reorder database items, it is first necessary to dump the table of contents of the archive:

$ pg_restore -l db.dump > db.list

The listing file consists of a header and one line for each item, e.g.:

;
; Archive created at Mon Sep 14 13:55:39 2009
;     dbname: DBDEMOS
;     TOC Entries: 81
;     Compression: 9
;     Dump Version: 1.10-0
;     Format: CUSTOM
;     Integer: 4 bytes
;     Offset: 8 bytes
;     Dumped from database version: 8.3.5
;     Dumped by pg_dump version: 8.3.8
;
;
; Selected TOC Entries:
;
3; 2615 2200 SCHEMA - public pasha
1861; 0 0 COMMENT - SCHEMA public pasha
1862; 0 0 ACL - public pasha
317; 1247 17715 TYPE public composite pasha
319; 1247 25899 DOMAIN public domain0 pasha

Semicolons start a comment, and the numbers at the start of lines refer to the internal archive ID assigned to each item.

Lines in the file can be commented out, deleted, and reordered. For example:

10; 145433 TABLE map_resolutions postgres
;2; 145344 TABLE species postgres
;4; 145359 TABLE nt_header postgres
6; 145402 TABLE species_records postgres
;8; 145416 TABLE ss_old postgres

could be used as input to pg_restore and would only restore items 10 and 6, in that order:

$ pg_restore -L db.list db.dump

See Also

pg_rewind

pg_rewind

pg_rewind

1

Application

pg_rewind

synchronize a

PostgreSQL

data directory with another data directory that was forked from it

pg_rewind

option

-D

--target-pgdata

datadir

--source-pgdata=

datadir

--source-server=

connstr

Description

pg_rewind is a tool for synchronizing a PostgreSQL cluster with another copy of the same cluster, after the clusters' timelines have diverged. A typical scenario is to bring an old primary server back online after failover as a standby that follows the new primary.

After a successful rewind, the state of the target data directory is analogous to a base backup of the source data directory. Unlike taking a new base backup or using a tool like rsync, pg_rewind does not require comparing or copying unchanged relation blocks in the cluster:

As such, the rewind operation is significantly faster than other approaches when the database is large and only a small fraction of blocks differ between the clusters.

pg_rewind examines the timeline histories of the source and target clusters to determine the point where they diverged, and expects to find WAL in the target cluster's pg_wal directory reaching all the way back to the point of divergence. The point of divergence can be found either on the target timeline, the source timeline, or their common ancestor. In the typical failover scenario where the target cluster was shut down soon after the divergence, this is not a problem, but if the target cluster ran for a long time after the divergence, its old WAL files might no longer be present. In this case, you can manually copy them from the WAL archive to the pg_wal directory, or run pg_rewind with the -c option to automatically retrieve them from the WAL archive. The use of pg_rewind is not limited to failover, e.g., a standby server can be promoted, run some write transactions, and then rewound to become a standby again.

After running pg_rewind, WAL replay needs to complete for the data directory to be in a consistent state. When the target server is started again it will enter archive recovery and replay all WAL generated in the source server from the last checkpoint before the point of divergence. If some of the WAL was no longer available in the source server when pg_rewind was run, and therefore could not be copied by the pg_rewind session, it must be made available when the target server is started. This can be done by creating a recovery.signal file in the target data directory and by configuring a suitable ??? in postgresql.conf.

pg_rewind requires that the target server either has the ??? option enabled in postgresql.conf or data checksums enabled when the cluster was initialized with initdb (the default). ??? must also be set to on, but is enabled by default.

The data checksum states of the source and target server must be compatible. pg_rewind refuses to run if an online checksum state transition was interrupted on either server, or if data checksums are enabled on the target server, or were enabled at the point of divergence, while the source server runs without them: the rewound server would fail checksum verification on the blocks copied from the source. The opposite combination is allowed with a warning; the rewound server keeps data checksums disabled unless the source server enabled them online after the point of divergence. See ??? for how to change the state consistently across a replication setup.

[!WARNING] If pg_rewind fails while processing, then the data folder of the target is likely not in a state that can be recovered. In such a case, taking a new fresh backup is recommended.

As pg_rewind copies configuration files entirely from the source, it may be required to correct the configuration used for recovery before restarting the target server, especially if the target is reintroduced as a standby of the source. If you restart the server after the rewind operation has finished but without configuring recovery, the target may again diverge from the primary.

pg_rewind will fail immediately if it finds files it cannot write directly to. This can happen for example when the source and the target server use the same file mapping for read-only SSL keys and certificates. If such files are present on the target server it is recommended to remove them before running pg_rewind. After doing the rewind, some of those files may have been copied from the source, in which case it may be necessary to remove the data copied and restore back the set of links used before the rewind.

Options

pg_rewind accepts the following command-line arguments:

-D datadir; --target-pgdata=datadir
This option specifies the target data directory that is synchronized with the source. The target server must be shut down cleanly before running pg_rewind

--source-pgdata=datadir
Specifies the file system path to the data directory of the source server to synchronize the target with. This option requires the source server to be cleanly shut down.

--source-server=connstr
Specifies a libpq connection string to connect to the source PostgreSQL server to synchronize the target with. The connection must be a normal (non-replication) connection with a role having sufficient permissions to execute the functions used by pg_rewind on the source server (see Notes section for details) or a superuser role. This option requires the source server to be running and accepting connections.

-R; --write-recovery-conf
Create standby.signal and append connection settings to postgresql.auto.conf in the output directory. The dbname will be recorded only if the dbname was specified explicitly in the connection string or environment variable. --source-server is mandatory with this option.

-n; --dry-run
Do everything except actually modifying the target directory.

-N; --no-sync
By default, pg_rewind will wait for all files to be written safely to disk. This option causes pg_rewind to return without waiting, which is faster, but means that a subsequent operating system crash can leave the data directory corrupt. Generally, this option is useful for testing but should not be used on a production installation.

-P; --progress
Enables progress reporting. Turning this on will deliver an approximate progress report while copying data from the source cluster.

-c; --restore-target-wal
Use restore_command defined in the target cluster configuration to retrieve WAL files from the WAL archive if these files are no longer available in the pg_wal directory.

--config-file=filename
Use the specified main server configuration file for the target cluster. This affects pg_rewind when it uses internally the postgres command for the rewind operation on this cluster (when retrieving restore_command with the option -c/--restore-target-wal and when forcing a completion of crash recovery).

--debug
Print verbose debugging output that is mostly useful for developers debugging pg_rewind.

--no-ensure-shutdown
pg_rewind requires that the target server is cleanly shut down before rewinding. By default, if the target server is not shut down cleanly, pg_rewind starts the target server in single-user mode to complete crash recovery first, and stops it. By passing this option, pg_rewind skips this and errors out immediately if the server is not cleanly shut down. Users are expected to handle the situation themselves in that case.

--sync-method=method
When set to fsync, which is the default, pg_rewind will recursively open and synchronize all files in the data directory. The search for files will follow symbolic links for the WAL directory and each configured tablespace.

On Linux, syncfs may be used instead to ask the operating system to synchronize the whole file systems that contain the data directory, the WAL files, and each tablespace. See ??? for information about the caveats to be aware of when using syncfs.

This option has no effect when --no-sync is used.

-V; --version
Display version information, then exit.

-?; --help
Show help, then exit.

Environment

When --source-server option is used, pg_rewind also uses the environment variables supported by libpq (see ???).

The environment variable PG_COLOR specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

Notes

When executing pg_rewind using an online cluster as source, a role having sufficient permissions to execute the functions used by pg_rewind on the source cluster can be used instead of a superuser. Here is how to create such a role, named rewind_user here:

CREATE USER rewind_user LOGIN;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_ls_dir(text, boolean, boolean) TO rewind_user;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_stat_file(text, boolean) TO rewind_user;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_read_binary_file(text) TO rewind_user;
GRANT EXECUTE ON FUNCTION pg_catalog.pg_read_binary_file(text, bigint, bigint, boolean) TO rewind_user;

How It Works

The basic idea is to copy all file system-level changes from the source cluster to the target cluster:

  1. Scan the WAL log of the target cluster, starting from the last checkpoint before the point where the source cluster's timeline history forked off from the target cluster. For each WAL record, record each data block that was touched. This yields a list of all the data blocks that were changed in the target cluster, after the source cluster forked off. If some of the WAL files are no longer available, try re-running pg_rewind with the -c option to search for the missing files in the WAL archive.

  2. Copy all those changed blocks from the source cluster to the target cluster, either using direct file system access (--source-pgdata) or SQL (--source-server). Relation files are now in a state equivalent to the moment of the last completed checkpoint prior to the point at which the WAL timelines of the source and target diverged plus the current state on the source of any blocks changed on the target after that divergence.

  3. Copy all other files, including new relation files, WAL segments, pg_xact, and configuration files from the source cluster to the target cluster. Similarly to base backups, the contents of the directories pg_dynshmem/, pg_notify/, pg_replslot/, pg_serial/, pg_snapshots/, pg_stat_tmp/, and pg_subtrans/ are omitted from the data copied from the source cluster. The files backup_label, tablespace_map, pg_internal.init, postmaster.opts, postmaster.pid and .DS_Store as well as any file or directory beginning with pgsql_tmp, are omitted.

  4. Create a backup_label file to begin WAL replay at the checkpoint created at failover and configure the pg_control file with a minimum consistency LSN defined as the result of pg_current_wal_insert_lsn() when rewinding from a live source or the last checkpoint LSN when rewinding from a stopped source.

  5. When starting the target, PostgreSQL replays all the required WAL, resulting in a data directory in a consistent state.

pg_verifybackup

pg_verifybackup

pg_verifybackup

1

Application

pg_verifybackup

verify the integrity of a base backup of a

PostgreSQL

cluster

pg_verifybackup

option

Description

pg_verifybackup is used to check the integrity of a database cluster backup taken using pg_basebackup against a backup_manifest generated by the server at the time of the backup. The backup may be stored either in the "plain" or the "tar" format; this includes tar-format backups compressed with any algorithm supported by pg_basebackup.

It is important to note that the validation which is performed by pg_verifybackup does not and cannot include every check which will be performed by a running server when attempting to make use of the backup. Even if you use this tool, you should still perform test restores and verify that the resulting databases work as expected and that they appear to contain the correct data. However, pg_verifybackup can detect many problems that commonly occur due to storage problems or user error.

Backup verification proceeds in four stages. First, pg_verifybackup reads the backup_manifest file. If that file does not exist, cannot be read, is malformed, fails to match the system identifier with pg_control of the backup directory or fails verification against its own internal checksum, pg_verifybackup will terminate with a fatal error.

Second, pg_verifybackup will attempt to verify that the data files currently stored on disk are exactly the same as the data files which the server intended to send, with some exceptions that are described below. Extra and missing files will be detected, with a few exceptions. This step will ignore the presence or absence of, or any modifications to, postgresql.auto.conf, standby.signal, and recovery.signal, because it is expected that these files may have been created or modified as part of the process of taking the backup. It also won't complain about a backup_manifest file in the target directory or about anything inside pg_wal, even though these files won't be listed in the backup manifest. Only files are checked; the presence or absence of directories is not verified, except indirectly: if a directory is missing, any files it should have contained will necessarily also be missing.

Next, pg_verifybackup will checksum all the files, compare the checksums against the values in the manifest, and emit errors for any files for which the computed checksum does not match the checksum stored in the manifest. This step is not performed for any files which produced errors in the previous step, since they are already known to have problems. Files which were ignored in the previous step are also ignored in this step.

Finally, pg_verifybackup will use the manifest to verify that the write-ahead log records which will be needed to recover the backup are present and that they can be read and parsed. The backup_manifest contains information about which write-ahead log records will be needed, and pg_verifybackup will use that information to invoke pg_waldump to parse those write-ahead log records. The --quiet flag will be used, so that pg_waldump will only report errors, without producing any other output. While this level of verification is sufficient to detect obvious problems such as a missing file or one whose internal checksums do not match, they aren't extensive enough to detect every possible problem that might occur when attempting to recover. For instance, a server bug that produces write-ahead log records that have the correct checksums but specify nonsensical actions can't be detected by this method.

Note that if extra WAL files which are not required to recover the backup are present, they will not be checked by this tool, although a separate invocation of pg_waldump could be used for that purpose. Also note that WAL verification is version-specific: you must use the version of pg_verifybackup, and thus of pg_waldump, which pertains to the backup being checked. In contrast, the data file integrity checks should work with any version of the server that generates a backup_manifest file.

Options

pg_verifybackup accepts the following command-line arguments:

-e; --exit-on-error
Exit as soon as a problem with the backup is detected. If this option is not specified, pg_verifybackup will continue checking the backup even after a problem has been detected, and will report all problems detected as errors.

-F format; --format=format
Specifies the format of the backup. <format> can be one of the following:

p; plain
Backup consists of plain files with the same layout as the source server's data directory and tablespaces.

t; tar
Backup consists of tar files, which may be compressed. A valid backup includes the main data directory in a file named base.tar, the WAL files in pg_wal.tar, and separate tar files for each tablespace, named after the tablespace's OID. If the backup is compressed, the relevant compression extension is added to the end of each file name.

-i path; --ignore=path
Ignore the specified file or directory, which should be expressed as a relative path name, when comparing the list of data files actually present in the backup to those listed in the backup_manifest file. If a directory is specified, this option affects the entire subtree rooted at that location. Complaints about extra files, missing files, file size differences, or checksum mismatches will be suppressed if the relative path name matches the specified path name. This option can be specified multiple times.

-m path; --manifest-path=path
Use the manifest file at the specified path, rather than one located in the root of the backup directory.

-n; --no-parse-wal
Don't attempt to parse write-ahead log data that will be needed to recover from this backup.

-P; --progress
Enable progress reporting. Turning this on will deliver a progress report while verifying checksums.

This option cannot be used together with the option --quiet.

-q; --quiet
Don't print anything when a backup is successfully verified.

-s; --skip-checksums
Do not verify data file checksums. The presence or absence of files and the sizes of those files will still be checked. This is much faster, because the files themselves do not need to be read.

-w path; --wal-path=path; --wal-directory=path (deprecated)
Try to parse WAL files stored in the specified directory or tar archive, rather than in pg_wal. This may be useful if the backup is stored in a separate location from the WAL archive.

Other options are also available:

-V; --version
Print the pg_verifybackup version and exit.

-?; --help
Show help about pg_verifybackup command line arguments, and exit.

Examples

To create a base backup of the server at mydbserver and verify the integrity of the backup:

$ pg_basebackup -h mydbserver -D /usr/local/pgsql/data
$ pg_verifybackup /usr/local/pgsql/data

To create a base backup of the server at mydbserver, move the manifest somewhere outside the backup directory, and verify the backup:

$ pg_basebackup -h mydbserver -D /usr/local/pgsql/backup1234
$ mv /usr/local/pgsql/backup1234/backup_manifest /my/secure/location/backup_manifest.1234
$ pg_verifybackup -m /my/secure/location/backup_manifest.1234 /usr/local/pgsql/backup1234

To verify a backup while ignoring a file that was added manually to the backup directory, and also skipping checksum verification:

$ pg_basebackup -h mydbserver -D /usr/local/pgsql/data
$ edit /usr/local/pgsql/data/note.to.self
$ pg_verifybackup --ignore=note.to.self --skip-checksums /usr/local/pgsql/data

See Also

pg_waldump

pg_waldump

pg_waldump

1

Application

pg_waldump

display a human-readable rendering of the write-ahead log of a

PostgreSQL

database cluster

pg_waldump

option

startseg

endseg

Description

pg_waldump displays the write-ahead log (WAL) and is mainly useful for debugging or educational purposes.

This utility can only be run by the user who installed the server, because it requires read-only access to the data directory.

Options

The following command-line options control the location and format of the output:

<startseg>
Start reading at the specified WAL segment file. This implicitly determines the path in which files will be searched for, and the timeline to use.

<endseg>
Stop after reading the specified WAL segment file.

-b; --bkp-details
Output detailed information about backup blocks.

-B block; --block=block
Only display records that modify the given block. The relation must also be provided with --relation or -R.

-e end; --end=end
Stop reading at the specified WAL location, instead of reading to the end of the log stream.

-f; --follow
After reaching the end of valid WAL, keep polling once per second for new WAL to appear.

-F fork; --fork=fork
Only display records that modify blocks in the given fork. The valid values are main for the main fork, fsm for the free space map, vm for the visibility map, and init for the init fork.

-n limit; --limit=limit
Display the specified number of records, then stop.

-p path; --path=path
Specifies a tar archive or a directory to search for WAL segment files or a directory with a pg_wal subdirectory that contains such files. The default is to search in the current directory, the pg_wal subdirectory of the current directory, and the pg_wal subdirectory of PGDATA.

If a tar archive is provided and its WAL segment files are not in sequential order, those files will be written to a temporary directory named starting with waldump_tmp. This directory will be created inside the directory specified by the TMPDIR environment variable if it is set; otherwise, it will be created within the same directory as the tar archive.

-q; --quiet
Do not print any output, except for errors. This option can be useful when you want to know whether a range of WAL records can be successfully parsed but don't care about the record contents.

-r rmgr; --rmgr=rmgr
Only display records generated by the specified resource manager. You can specify the option multiple times to select multiple resource managers. If list is passed as name, print a list of valid resource manager names, and exit.

Extensions may define custom resource managers, but pg_waldump does not load the extension module and therefore does not recognize custom resource managers by name. Instead, you can specify the custom resource managers as custom### where <###> is the three-digit resource manager ID. Names of this form will always be considered valid.

-R tblspc/db/rel; --relation=tblspc/db/rel
Only display records that modify blocks in the given relation. The relation is specified with tablespace OID, database OID, and relfilenode separated by slashes, for example 1234/12345/12345. This is the same format used for relations in the program's output.

-s start; --start=start
WAL location at which to start reading. The default is to start reading the first valid WAL record found in the earliest file found.

-t timeline; --timeline=timeline
Timeline from which to read WAL records. The default is to use the value in <startseg>, if that is specified; otherwise, the default is 1. The value can be specified in decimal or hexadecimal, for example 17 or 0x11.

-V; --version
Print the pg_waldump version and exit.

-w; --fullpage
Only display records that include full page images.

-x xid; --xid=xid
Only display records marked with the given transaction ID.

-z; --stats[=record]
Display summary statistics (number and size of records and full-page images) instead of individual records. Optionally generate statistics per-record instead of per-rmgr.

If pg_waldump is terminated by signal SIGINT (Control+C), the summary of the statistics computed is displayed up to the termination point. This operation is not supported on Windows.

--save-fullpage=save_path
Save full page images found in the WAL records to the <save_path> directory. The images saved are subject to the same filtering and limiting criteria as the records displayed.

The full page images are saved with the following file name format: TIMELINE-LSN.RELTABLESPACE.DATOID.RELNODE.BLKNO_FORK The file names are composed of the following parts:

Component Description
TIMELINE The timeline of the WAL segment file where the record is located formatted as one 8-character hexadecimal number %08X
LSN The LSN of the record with this image, formatted as two 8-character hexadecimal numbers %08X-%08X
RELTABLESPACE tablespace OID of the block
DATOID database OID of the block
RELNODE filenode of the block
BLKNO block number of the block
FORK The name of the fork the full page image came from, such as main, fsm, vm, or init.

-?; --help
Show help about pg_waldump command line arguments, and exit.

Environment

PGDATA
Data directory; see also the -p option.

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

TMPDIR
Directory in which to create temporary files when reading WAL from a tar archive with out-of-order segment files. If not set, the temporary directory is created within the same directory as the tar archive.

Notes

Can give wrong results when the server is running.

Only the specified timeline is displayed (or the default, if none is specified). Records in other timelines are ignored.

pg_waldump cannot read WAL files with suffix .partial. If those files need to be read, .partial suffix needs to be removed from the file name.

See Also

pg_walsummary

pg_walsummary

pg_walsummary

1

Application

pg_walsummary

print contents of WAL summary files

pg_walsummary

option

filename

Description

pg_walsummary is used to print the contents of WAL summary files. These binary files are found in the pg_wal/summaries subdirectory of the data directory, and can be converted to text using this tool. This is not ordinarily necessary, since WAL summary files primarily exist to support incremental backup, but it may be useful for debugging purposes.

A WAL summary file is indexed by tablespace OID, relation OID, and relation fork. For each relation fork, it stores the list of blocks that were modified by WAL within the range summarized in the file. It can also store a "limit block," which is 0 if the relation fork was created or truncated within the relevant WAL range, and otherwise the shortest length to which the relation fork was truncated. If the relation fork was not created, deleted, or truncated within the relevant WAL range, the limit block is undefined or infinite and will not be printed by this tool.

Options

<filename>
Specifies the name of a WAL summary file, found in the pg_wal/summaries subdirectory of the data directory.

-i; --individual
By default, pg_walsummary prints one line of output for each range of one or more consecutive modified blocks. This can make the output a lot briefer, since a relation where all blocks from 0 through 999 were modified will produce only one line of output rather than 1000 separate lines. This option requests a separate line of output for every modified block.

-q; --quiet
Do not print any output, except for errors. This can be useful when you want to know whether a WAL summary file can be successfully parsed but don't care about the contents.

-V; --version
Display version information, then exit.

-?; --help
Shows help about pg_walsummary command line arguments, and exits.

Environment

The environment variable PG_COLOR specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

See Also

pgarchivecleanup

pg_archivecleanup

pg_archivecleanup

1

Application

pg_archivecleanup

clean up

PostgreSQL

WAL archive files

pg_archivecleanup

option

archivelocation

oldestkeptwalfile

Description

pg_archivecleanup is designed to be used as an archive_cleanup_command to clean up WAL file archives when running as a standby server (see ???). pg_archivecleanup can also be used as a standalone program to clean WAL file archives.

To configure a standby server to use pg_archivecleanup, put this into its postgresql.conf configuration file:

archive_cleanup_command = 'pg_archivecleanup archivelocation "%r"'

where <archivelocation> is the directory from which WAL segment files should be removed.

When used within ???, all WAL files logically preceding the value of the %r argument will be removed from <archivelocation>. This minimizes the number of files that need to be retained, while preserving crash-restart capability. Use of this parameter is appropriate if the <archivelocation> is a transient staging area for this particular standby server, but not when the <archivelocation> is intended as a long-term WAL archive area, or when multiple standby servers are recovering from the same archive location.

When used as a standalone program all WAL files logically preceding the <oldestkeptwalfile> will be removed from <archivelocation>. In this mode, if you specify a .partial or .backup file name, then only the file prefix will be used as the <oldestkeptwalfile>. This treatment of .backup file name allows you to remove all WAL files archived prior to a specific base backup without error. For example, the following example will remove all files older than WAL file name 000000010000003700000010:

pg_archivecleanup -d archive 000000010000003700000010.00000020.backup

pg_archivecleanup:  keep WAL file "archive/000000010000003700000010" and later
pg_archivecleanup:  removing file "archive/00000001000000370000000F"
pg_archivecleanup:  removing file "archive/00000001000000370000000E"

pg_archivecleanup assumes that <archivelocation> is a directory readable and writable by the server-owning user.

Options

pg_archivecleanup accepts the following command-line arguments:

-b; --clean-backup-history
Remove backup history files as well. See ??? for details about backup history files.

-d; --debug
Print lots of debug logging output on stderr.

-n; --dry-run
Print the names of the files that would have been removed on stdout (performs a dry run).

-V; --version
Print the pg_archivecleanup version and exit.

-x extension; --strip-extension=extension
Provide an extension that will be stripped from all file names before deciding if they should be deleted. This is typically useful for cleaning up archives that have been compressed during storage, and therefore have had an extension added by the compression program. For example: -x .gz.

-?; --help
Show help about pg_archivecleanup command line arguments, and exit.

Environment

The environment variable PG_COLOR specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

Notes

pg_archivecleanup is designed to work with PostgreSQL 8.0 and later when used as a standalone utility, or with PostgreSQL 9.0 and later when used as an archive cleanup command.

pg_archivecleanup is written in C and has an easy-to-modify source code, with specifically designated sections to modify for your own needs

Examples

On Linux or Unix systems, you might use:

archive_cleanup_command = 'pg_archivecleanup -d /mnt/standby/archive "%r" 2>>cleanup.log'

where the archive directory is physically located on the standby server, so that the archive_command is accessing it across NFS, but the files are local to the standby. This will:

pgbench

pgbench

pgbench

1

Application

pgbench

run a benchmark test on

PostgreSQL

pgbench

-i

option

dbname

pgbench

option

dbname

Description

pgbench is a simple program for running benchmark tests on PostgreSQL. It runs the same sequence of SQL commands over and over, possibly in multiple concurrent database sessions, and then calculates the average transaction rate (transactions per second). By default, pgbench tests a scenario that is loosely based on TPC-B, involving five SELECT, UPDATE, and INSERT commands per transaction. However, it is easy to test other cases by writing your own transaction script files.

Typical output from pgbench looks like:

transaction type: <builtin: TPC-B (sort of)>
scaling factor: 10
query mode: simple
number of clients: 10
number of threads: 1
maximum number of tries: 1
number of transactions per client: 1000
number of transactions actually processed: 10000/10000
number of failed transactions: 0 (0.000%)
latency average = 11.013 ms
latency stddev = 7.351 ms
initial connection time = 45.758 ms
tps = 896.967014 (without initial connection time)

The first seven lines report some of the most important parameter settings. The sixth line reports the maximum number of tries for transactions with serialization or deadlock errors (see Failures and Serialization/Deadlock Retries for more information). The eighth line reports the number of transactions completed and intended (the latter being just the product of number of clients and number of transactions per client); these will be equal unless the run failed before completion or some SQL command(s) failed. (In -T mode, only the actual number of transactions is printed.) The next line reports the number of failed transactions (see Failures and Serialization/Deadlock Retries for more information). The last line reports the number of transactions per second.

The default TPC-B-like transaction test requires specific tables to be set up beforehand. pgbench should be invoked with the -i (initialize) option to create and populate these tables. (When you are testing a custom script, you don't need this step, but will instead need to do whatever setup your test needs.) Initialization looks like:

pgbench -i  other-options  dbname

where <dbname> is the name of the already-created database to test in. (You may also need -h, -p, and/or -U options to specify how to connect to the database server.)

[!CAUTION] pgbench -i creates four tables pgbench_accounts, pgbench_branches, pgbench_history, and pgbench_tellers, destroying any existing tables of these names. Be very careful to use another database if you have tables having these names!

At the default “scale factor” of 1, the tables initially contain this many rows:

table                   # of rows
---------------------------------
pgbench_branches        1
pgbench_tellers         10
pgbench_accounts        100000
pgbench_history         0

You can (and, for most purposes, probably should) increase the number of rows by using the -s (scale factor) option. The -F (fillfactor) option might also be used at this point.

Once you have done the necessary setup, you can run your benchmark with a command that doesn't include -i, that is

pgbench  options  dbname

In nearly all cases, you'll need some options to make a useful test. The most important options are -c (number of clients), -t (number of transactions), -T (time limit), and -f (specify a custom script file). See below for a full list.

Options

The following is divided into three subsections. Different options are used during database initialization and while running benchmarks, but some options are useful in both cases.

Initialization Options

pgbench accepts the following command-line initialization arguments:

-d dbname; --dbname=dbname
Specifies the name of the database to test in. If this is not specified, the environment variable PGDATABASE is used. If that is not set, the user name specified for the connection is used.

-i; --initialize
Required to invoke initialization mode.

-I init_steps; --init-steps=init_steps
Perform just a selected set of the normal initialization steps. <init_steps> specifies the initialization steps to be performed, using one character per step. Each step is invoked in the specified order. The default is dtgvp. The available steps are:

d (Drop)
Drop any existing pgbench tables.

t (create Tables)
Create the tables used by the standard pgbench scenario, namely pgbench_accounts, pgbench_branches, pgbench_history, and pgbench_tellers.

g or G (Generate data, client-side or server-side)
Generate data and load it into the standard tables, replacing any data already present.

With g (client-side data generation), data is generated in pgbench client and then sent to the server. This uses the client/server bandwidth extensively through a COPY. pgbench uses the FREEZE option to load data into ordinary (non-partition) tables with version 14 or later of PostgreSQL to speed up subsequent VACUUM. Using g causes logging to print one message every 100,000 rows while generating data for all tables.

With G (server-side data generation), only small queries are sent from the pgbench client and then data is actually generated in the server. No significant bandwidth is required for this variant, but the server will do more work. Using G causes logging not to print any progress message while generating data.

The default initialization behavior uses client-side data generation (equivalent to g).

v (Vacuum)
Invoke VACUUM on the standard tables.

p (create Primary keys)
Create primary key indexes on the standard tables.

f (create Foreign keys)
Create foreign key constraints between the standard tables. (Note that this step is not performed by default.)

-F <fillfactor>; --fillfactor=<fillfactor>
Create the pgbench_accounts, pgbench_tellers and pgbench_branches tables with the given fillfactor. Default is 100.

-n; --no-vacuum
Perform no vacuuming during initialization. (This option suppresses the v initialization step, even if it was specified in -I.)

-q; --quiet
Switch logging to quiet mode, producing only one progress message per 5 seconds. The default logging prints one message each 100,000 rows, which often outputs many lines per second (especially on good hardware).

This setting has no effect if G is specified in -I.

-s <scale_factor>; --scale=<scale_factor>
Multiply the number of rows generated by the scale factor. For example, -s 100 will create 10,000,000 rows in the pgbench_accounts table. Default is 1. When the scale is 20,000 or larger, the columns used to hold account identifiers (aid columns) will switch to using larger integers (bigint), in order to be big enough to hold the range of account identifiers.

--foreign-keys
Create foreign key constraints between the standard tables. (This option adds the f step to the initialization step sequence, if it is not already present.)

--index-tablespace=index_tablespace
Create indexes in the specified tablespace, rather than the default tablespace.

--partition-method=NAME
Create a partitioned pgbench_accounts table with <NAME> method. Expected values are range or hash. This option requires that --partitions is set to non-zero. If unspecified, default is range.

--partitions=NUM
Create a partitioned pgbench_accounts table with <NUM> partitions of nearly equal size for the scaled number of accounts. Default is 0, meaning no partitioning.

--tablespace=tablespace
Create tables in the specified tablespace, rather than the default tablespace.

--unlogged-tables
Create all tables as unlogged tables, rather than permanent tables.

Benchmarking Options

pgbench accepts the following command-line benchmarking arguments:

-b <scriptname[@weight]>; --builtin=<scriptname[@weight]>
Add the specified built-in script to the list of scripts to be executed. Available built-in scripts are: tpcb-like, simple-update and select-only. Unambiguous prefixes of built-in names are accepted. With the special name list, show the list of built-in scripts and exit immediately.

Optionally, write an integer weight after @ to adjust the probability of selecting this script versus other ones. The default weight is 1. See below for details.

-c <clients>; --client=<clients>
Number of clients simulated, that is, number of concurrent database sessions. Default is 1.

-C; --connect
Establish a new connection for each transaction, rather than doing it just once per client session. This is useful to measure the connection overhead.

-D <varname>=<value>; --define=<varname>=<value>
Define a variable for use by a custom script (see below). Multiple -D options are allowed.

-f <filename[@weight]>; --file=<filename[@weight]>
Add a transaction script read from <filename> to the list of scripts to be executed.

Optionally, write an integer weight after @ to adjust the probability of selecting this script versus other ones. The default weight is 1. (To use a script file name that includes an @ character, append a weight so that there is no ambiguity, for example filen@me@1.) See below for details.

-j <threads>; --jobs=<threads>
Number of worker threads within pgbench. Using more than one thread can be helpful on multi-CPU machines. Clients are distributed as evenly as possible among available threads. Default is 1.

-l; --log
Write information about each transaction to a log file. See below for details.

-L <limit>; --latency-limit=<limit>
Transactions that last more than <limit> milliseconds are counted and reported separately, as late.

When throttling is used (--rate=...), transactions that lag behind schedule by more than <limit> ms, and thus have no hope of meeting the latency limit, are not sent to the server at all. They are counted and reported separately as skipped.

When the --max-tries option is used, a transaction which fails due to a serialization anomaly or from a deadlock will not be retried if the total time of all its tries is greater than <limit> ms. To limit only the time of tries and not their number, use --max-tries=0. By default, the option --max-tries is set to 1 and transactions with serialization/deadlock errors are not retried. See Failures and Serialization/Deadlock Retries for more information about retrying such transactions.

-M <querymode>; --protocol=<querymode>
Protocol to use for submitting queries to the server:

In the prepared mode, pgbench reuses the parse analysis result starting from the second query iteration, so pgbench runs faster than in other modes.

The default is simple query protocol. (See ??? for more information.)

-n; --no-vacuum
Perform no vacuuming before running the test. This option is necessary if you are running a custom test scenario that does not include the standard tables pgbench_accounts, pgbench_branches, pgbench_history, and pgbench_tellers.

-N; --skip-some-updates
Run built-in simple-update script. Shorthand for -b simple-update.

-P <sec>; --progress=<sec>
Show progress report every <sec> seconds. The report includes the time since the beginning of the run, the TPS since the last report, and the transaction latency average, standard deviation, and the number of failed transactions since the last report. Under throttling (-R), the latency is computed with respect to the transaction scheduled start time, not the actual transaction beginning time, thus it also includes the average schedule lag time. When --max-tries is used to enable transaction retries after serialization/deadlock errors, the report includes the number of retried transactions and the sum of all retries.

-r; --report-per-command
Report the following statistics for each command after the benchmark finishes: the average per-statement latency (execution time from the perspective of the client), the number of failures, and the number of retries after serialization or deadlock errors in this command. The report displays retry statistics only if the --max-tries option is not equal to 1.

-R <rate>; --rate=<rate>
Execute transactions targeting the specified rate instead of running as fast as possible (the default). The rate is given in transactions per second. If the targeted rate is above the maximum possible rate, the rate limit won't impact the results.

The rate is targeted by starting transactions along a Poisson-distributed schedule time line. The expected start time schedule moves forward based on when the client first started, not when the previous transaction ended. That approach means that when transactions go past their original scheduled end time, it is possible for later ones to catch up again.

When throttling is active, the transaction latency reported at the end of the run is calculated from the scheduled start times, so it includes the time each transaction had to wait for the previous transaction to finish. The wait time is called the schedule lag time, and its average and maximum are also reported separately. The transaction latency with respect to the actual transaction start time, i.e., the time spent executing the transaction in the database, can be computed by subtracting the schedule lag time from the reported latency.

If --latency-limit is used together with --rate, a transaction can lag behind so much that it is already over the latency limit when the previous transaction ends, because the latency is calculated from the scheduled start time. Such transactions are not sent to the server, but are skipped altogether and counted separately.

A high schedule lag time is an indication that the system cannot process transactions at the specified rate, with the chosen number of clients and threads. When the average transaction execution time is longer than the scheduled interval between each transaction, each successive transaction will fall further behind, and the schedule lag time will keep increasing the longer the test run is. When that happens, you will have to reduce the specified transaction rate.

-s <scale_factor>; --scale=<scale_factor>
Report the specified scale factor in pgbench's output. With the built-in tests, this is not necessary; the correct scale factor will be detected by counting the number of rows in the pgbench_branches table. However, when testing only custom benchmarks (-f option), the scale factor will be reported as 1 unless this option is used.

-S; --select-only
Run built-in select-only script. Shorthand for -b select-only.

-t <transactions>; --transactions=<transactions>
Number of transactions each client runs. Default is 10.

-T <seconds>; --time=<seconds>
Run the test for this many seconds, rather than a fixed number of transactions per client. -t and -T are mutually exclusive.

-v; --vacuum-all
Vacuum all four standard tables before running the test. With neither -n nor -v, pgbench will vacuum the pgbench_tellers and pgbench_branches tables, and will truncate pgbench_history.

--aggregate-interval=seconds
Length of aggregation interval (in seconds). May be used only with -l option. With this option, the log contains per-interval summary data, as described below.

--continue-on-error
Allows clients to continue running even if an SQL statement fails due to errors other than serialization or deadlock. By default, clients abort after such errors, but with this option enabled, they proceed to the next transaction instead. Note that clients still abort even with this option if an error causes the connection to fail. See Failures and Serialization/Deadlock Retries for more information.

This option is useful when your custom script may raise errors such as unique constraint violations, but you want the benchmark to continue despite individual statement failures. Failed transactions are reported separately, but are not counted as transactions actually processed; time spent on failed transactions remains part of the benchmark duration, so such failures can lower the reported TPS.

--exit-on-abort
Exit immediately when any client is aborted due to some error. Without this option, even when a client is aborted, other clients could continue their run as specified by -t or -T option, and pgbench will print an incomplete results in this case.

Note that serialization failures or deadlock failures do not abort the client, so they are not affected by this option. See Failures and Serialization/Deadlock Retries for more information.

--failures-detailed
Report failures in per-transaction and aggregation logs, as well as in the main and per-script reports, grouped by the following types:

See Failures and Serialization/Deadlock Retries for more information.

--log-prefix=prefix
Set the filename prefix for the log files created by --log. The default is pgbench_log.

--max-tries=number_of_tries
Enable retries for transactions with serialization/deadlock errors and set the maximum number of these tries. This option can be combined with the --latency-limit option which limits the total time of all transaction tries; moreover, you cannot use an unlimited number of tries (--max-tries=0) without --latency-limit or --time. The default value is 1 and transactions with serialization/deadlock errors are not retried. See Failures and Serialization/Deadlock Retries for more information about retrying such transactions.

--progress-timestamp
When showing progress (option -P), use a timestamp (Unix epoch) instead of the number of seconds since the beginning of the run. The unit is in seconds, with millisecond precision after the dot. This helps compare logs generated by various tools.

--random-seed=<seed>
Set random generator seed. Seeds the system random number generator, which then produces a sequence of initial generator states, one for each thread. Values for <seed> may be: time (the default, the seed is based on the current time), rand (use a strong random source, failing if none is available), or an unsigned decimal integer value. The random generator is invoked explicitly from a pgbench script (random... functions) or implicitly (for instance option --rate uses it to schedule transactions). When explicitly set, the value used for seeding is shown on the terminal. Any value allowed for <seed> may also be provided through the environment variable PGBENCH_RANDOM_SEED. To ensure that the provided seed impacts all possible uses, put this option first or use the environment variable.

Setting the seed explicitly allows to reproduce a pgbench run exactly, as far as random numbers are concerned. As the random state is managed per thread, this means the exact same pgbench run for an identical invocation if there is one client per thread and there are no external or data dependencies. From a statistical viewpoint reproducing runs exactly is a bad idea because it can hide the performance variability or improve performance unduly, e.g., by hitting the same pages as a previous run. However, it may also be of great help for debugging, for instance re-running a tricky case which leads to an error. Use wisely.

--sampling-rate=rate
Sampling rate, used when writing data into the log, to reduce the amount of log generated. If this option is given, only the specified fraction of transactions are logged. 1.0 means all transactions will be logged, 0.05 means only 5% of the transactions will be logged.

Remember to take the sampling rate into account when processing the log file. For example, when computing TPS values, you need to multiply the numbers accordingly (e.g., with 0.01 sample rate, you'll only get 1/100 of the actual TPS).

--show-script=<scriptname>
Show the actual code of builtin script <scriptname> on stderr, and exit immediately.

--verbose-errors
Print messages about all errors and failures (errors without retrying) including which limit for retries was exceeded and how far it was exceeded for the serialization/deadlock failures. (Note that in this case the output can be significantly increased.) See Failures and Serialization/Deadlock Retries for more information.

Common Options

pgbench also accepts the following common command-line arguments for connection parameters and other common settings:

--debug
Print debugging output.

-h <hostname>; --host=<hostname>
The database server's host name

-p <port>; --port=<port>
The database server's port number

-U <login>; --username=<login>
The user name to connect as

-V; --version
Print the pgbench version and exit.

-?; --help
Show help about pgbench command line arguments, and exit.

Exit Status

A successful run will exit with status 0. Exit status 1 indicates static problems such as invalid command-line options or internal errors which are supposed to never occur. Early errors that occur when starting benchmark such as initial connection failures also exit with status 1. Errors during the run such as database errors or problems in the script will result in exit status 2. In the latter case, pgbench will print partial results if --exit-on-abort option is not specified.

Environment

PGDATABASE; PGHOST; PGPORT; PGUSER
Default connection parameters.

This utility, like most other PostgreSQL utilities, uses the environment variables supported by libpq (see ???).

The environment variable PG_COLOR specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

Notes

What Is the “Transaction” Actually Performed in pgbench?

pgbench executes test scripts chosen randomly from a specified list. The scripts may include built-in scripts specified with -b and user-provided scripts specified with -f. Each script may be given a relative weight specified after an @ so as to change its selection probability. The default weight is 1. Scripts with a weight of 0 are ignored.

The default built-in transaction script (also invoked with -b tpcb-like) issues seven commands per transaction over randomly chosen aid, tid, bid and delta. The scenario is inspired by the TPC-B benchmark, but is not actually TPC-B, hence the name.

  1. BEGIN;

  2. UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;

  3. SELECT abalance FROM pgbench_accounts WHERE aid = :aid;

  4. UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;

  5. UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;

  6. INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);

  7. END;

If you select the simple-update built-in (also -N), steps 4 and 5 aren't included in the transaction. This will avoid update contention on these tables, but it makes the test case even less like TPC-B.

If you select the select-only built-in (also -S), only the SELECT is issued.

Custom Scripts

pgbench has support for running custom benchmark scenarios by replacing the default transaction script (described above) with a transaction script read from a file (-f option). In this case a “transaction” counts as one execution of a script file.

A script file contains one or more SQL commands terminated by semicolons. Empty lines and lines beginning with -- are ignored. Script files can also contain “meta commands”, which are interpreted by pgbench itself, as described below.

[!NOTE] Before PostgreSQL 9.6, SQL commands in script files were terminated by newlines, and so they could not be continued across lines. Now a semicolon is required to separate consecutive SQL commands (though an SQL command does not need one if it is followed by a meta command). If you need to create a script file that works with both old and new versions of pgbench, be sure to write each SQL command on a single line ending with a semicolon.

It is assumed that pgbench scripts do not contain incomplete blocks of SQL transactions. If at runtime the client reaches the end of the script without completing the last transaction block, it will be aborted.

There is a simple variable-substitution facility for script files. Variable names must consist of letters (including non-Latin letters), digits, and underscores, with the first character not being a digit. Variables can be set by the command-line -D option, explained above, or by the meta commands explained below. In addition to any variables preset by -D command-line options, there are a few variables that are preset automatically, listed in pgbench Automatic Variables. A value specified for these variables using -D takes precedence over the automatic presets. Once set, a variable's value can be inserted into an SQL command by writing :<variablename>. When running more than one client session, each session has its own set of variables. pgbench supports up to 255 variable uses in one statement.

Variable Description
client_id unique number identifying the client session (starts from zero)
default_seed seed used in hash and pseudorandom permutation functions by default
random_seed random generator seed (unless overwritten with -D)
scale current scale factor

pgbench Automatic Variables {#pgbench-automatic-variables}

Script file meta commands begin with a backslash (\) and normally extend to the end of the line, although they can be continued to additional lines by writing backslash-return. Arguments to a meta command are separated by white space. These meta commands are supported:

\gset [prefix]; \aset [prefix]
These commands may be used to end SQL queries, taking the place of the terminating semicolon (;).

When the \gset command is used, the preceding SQL query is expected to return one row, the columns of which are stored into variables named after column names, and prefixed with <prefix> if provided. If the query returns zero or multiple rows, an error is raised.

When the \aset command is used, all combined SQL queries (separated by \;) have their columns stored into variables named after column names, and prefixed with <prefix> if provided. If a query returns no rows, no assignment is made. This can be detected by initializing the variable beforehand with a value the query cannot return, then checking whether it changes. If a query returns more than one row, the last value is kept.

\gset and \aset cannot be used in pipeline mode, since the query results are not yet available by the time the commands would need them.

The following example puts the final account balance from the first query into variable <abalance>, and fills variables <p_two> and <p_three> with integers from the third query. The result of the second query is discarded. The result of the two last combined queries are stored in variables <four> and <five>.

UPDATE pgbench_accounts
  SET abalance = abalance + :delta
  WHERE aid = :aid
  RETURNING abalance \gset
-- compound of two queries
SELECT 1 \;
SELECT 2 AS two, 3 AS three \gset p_
SELECT 4 AS four \; SELECT 5 AS five \aset

\if <expression>; \elif <expression>; \else; \endif
This group of commands implements nestable conditional blocks, similarly to psql's ???. Conditional expressions are identical to those with \set, with non-zero values interpreted as true.

\set varname expression
Sets variable <varname> to a value calculated from <expression>. The expression may contain the NULL constant, Boolean constants TRUE and FALSE, integer constants such as 5432, double constants such as 3.14159, references to variables :<variablename>, operators with their usual SQL precedence and associativity, function calls, SQL CASE generic conditional expressions and parentheses.

Functions and most operators return NULL on NULL input.

For conditional purposes, non zero numerical values are TRUE, zero numerical values and NULL are FALSE.

Too large or small integer and double constants, as well as integer arithmetic operators (+, -, * and /) raise errors on overflows.

When no final ELSE clause is provided to a CASE, the default value is NULL.

Examples:

\set ntellers 10 * :scale
\set aid (1021 * random(1, 100000 * :scale)) % \
           (100000 * :scale) + 1
\set divx CASE WHEN :x <> 0 THEN :y/:x ELSE NULL END

\sleep number [ us | ms | s ]
Causes script execution to sleep for the specified duration in microseconds (us), milliseconds (ms) or seconds (s). If the unit is omitted then seconds are the default. <number> can be either an integer constant or a :<variablename> reference to a variable having an integer value.

Example:

\sleep 10 ms

\setshell varname command [ argument ... ]
Sets variable <varname> to the result of the shell command <command> with the given <argument>(s). The command must return an integer value through its standard output.

<command> and each <argument> can be either a text constant or a :<variablename> reference to a variable. If you want to use an <argument> starting with a colon, write an additional colon at the beginning of <argument>.

Example:

\setshell variable_to_be_assigned command literal_argument :variable ::literal_starting_with_colon

\shell command [ argument ... ]
Same as \setshell, but the result of the command is discarded.

Example:

\shell command literal_argument :variable ::literal_starting_with_colon

\startpipeline; \syncpipeline; \endpipeline
This group of commands implements pipelining of SQL statements. A pipeline must begin with a \startpipeline and end with an \endpipeline. In between there may be any number of \syncpipeline commands, which sends a sync message without ending the ongoing pipeline and flushing the send buffer. In pipeline mode, statements are sent to the server without waiting for the results of previous statements. See ??? for more details. Pipeline mode requires the use of extended query protocol.

Built-in Operators

The arithmetic, bitwise, comparison and logical operators listed in pgbench Operators are built into pgbench and may be used in expressions appearing in \set. The operators are listed in increasing precedence order. Except as noted, operators taking two numeric inputs will produce a double value if either input is double, otherwise they produce an integer result.

pgbench Operators

Operator

Description

Example(s)

<boolean> OR <boolean> <boolean>

Logical OR

5 or 0 TRUE

<boolean> AND <boolean> <boolean>

Logical AND

3 and 0 FALSE

NOT <boolean> <boolean>

Logical NOT

not false TRUE

<boolean> IS [NOT] (NULL|TRUE|FALSE) <boolean>

Boolean value tests

1 is null FALSE

<value> ISNULL|NOTNULL <boolean>

Nullness tests

1 notnull TRUE

<number> = <number> <boolean>

Equal

5 = 4 FALSE

<number> <> <number> <boolean>

Not equal

5 <> 4 TRUE

<number> != <number> <boolean>

Not equal

5 != 5 FALSE

<number> < <number> <boolean>

Less than

5 < 4 FALSE

<number> <= <number> <boolean>

Less than or equal to

5 <= 4 FALSE

<number> > <number> <boolean>

Greater than

5 > 4 TRUE

<number> >= <number> <boolean>

Greater than or equal to

5 >= 4 TRUE

<integer> | <integer> <integer>

Bitwise OR

1 | 2 3

<integer> # <integer> <integer>

Bitwise XOR

1 # 3 2

<integer> & <integer> <integer>

Bitwise AND

1 & 3 1

~ <integer> <integer>

Bitwise NOT

~ 1 -2

<integer> << <integer> <integer>

Bitwise shift left

1 << 2 4

<integer> >> <integer> <integer>

Bitwise shift right

8 >> 2 2

<number> + <number> <number>

Addition

5 + 4 9

<number> - <number> <number>

Subtraction

3 - 2.0 1.0

<number> * <number> <number>

Multiplication

5 * 4 20

<number> / <number> <number>

Division (truncates the result towards zero if both inputs are integers)

5 / 3 1

<integer> % <integer> <integer>

Modulo (remainder)

3 % 2 1

- <number> <number>

Negation

- 2.0 -2.0

Built-In Functions

The functions listed in pgbench Functions are built into pgbench and may be used in expressions appearing in \set.

pgbench Functions

Function

Description

Example(s)

abs ( <number> ) same type as input

Absolute value

abs(-17) 17

debug ( <number> ) same type as input

Prints the argument to stderr, and returns the argument.

debug(5432.1) 5432.1

double ( <number> ) double

Casts to double.

double(5432) 5432.0

exp ( <number> ) double

Exponential (e raised to the given power)

exp(1.0) 2.718281828459045

greatest ( <number> [, ...] ) double if any argument is double, else integer

Selects the largest value among the arguments.

greatest(5, 4, 3, 2) 5

hash ( value [, seed] ) integer

This is an alias for hash_murmur2.

hash(10, 5432) -5817877081768721676

hash_fnv1a ( value [, seed] ) integer

Computes FNV-1a hash.

hash_fnv1a(10, 5432) -7793829335365542153

hash_murmur2 ( value [, seed] ) integer

Computes MurmurHash2 hash.

hash_murmur2(10, 5432) -5817877081768721676

int ( <number> ) integer

Casts to integer.

int(5.4 + 3.8) 9

least ( <number> [, ...] ) double if any argument is double, else integer

Selects the smallest value among the arguments.

least(5, 4, 3, 2.1) 2.1

ln ( <number> ) double

Natural logarithm

ln(2.718281828459045) 1.0

mod ( <integer>, <integer> ) integer

Modulo (remainder)

mod(54, 32) 22

permute ( i, size [, seed ] ) integer

Permuted value of i, in the range [0, size). This is the new position of i (modulo size) in a pseudorandom permutation of the integers 0...size-1, parameterized by seed, see below.

permute(0, 4) an integer between 0 and 3

pi () double

Approximate value of π

pi() 3.14159265358979323846

pow ( x, y ) double

power ( x, y ) double

x raised to the power of y

pow(2.0, 10) 1024.0

random ( lb, ub ) integer

Computes a uniformly-distributed random integer in [lb, ub].

random(1, 10) an integer between 1 and 10

random_exponential ( lb, ub, parameter ) integer

Computes an exponentially-distributed random integer in [lb, ub], see below.

random_exponential(1, 10, 3.0) an integer between 1 and 10

random_gaussian ( lb, ub, parameter ) integer

Computes a Gaussian-distributed random integer in [lb, ub], see below.

random_gaussian(1, 10, 2.5) an integer between 1 and 10

random_zipfian ( lb, ub, parameter ) integer

Computes a Zipfian-distributed random integer in [lb, ub], see below.

random_zipfian(1, 10, 1.5) an integer between 1 and 10

sqrt ( <number> ) double

Square root

sqrt(2.0) 1.414213562

The random function generates values using a uniform distribution, that is all the values are drawn within the specified range with equal probability. The random_exponential, random_gaussian and random_zipfian functions require an additional double parameter which determines the precise shape of the distribution.

[!NOTE] When designing a benchmark which selects rows non-uniformly, be aware that the rows chosen may be correlated with other data such as IDs from a sequence or the physical row ordering, which may skew performance measurements.

To avoid this, you may wish to use the permute function, or some other additional step with similar effect, to shuffle the selected rows and remove such correlations.

Hash functions hash, hash_murmur2 and hash_fnv1a accept an input value and an optional seed parameter. In case the seed isn't provided the value of :default_seed is used, which is initialized randomly unless set by the command-line -D option.

permute accepts an input value, a size, and an optional seed parameter. It generates a pseudorandom permutation of integers in the range [0, size), and returns the index of the input value in the permuted values. The permutation chosen is parameterized by the seed, which defaults to :default_seed, if not specified. Unlike the hash functions, permute ensures that there are no collisions or holes in the output values. Input values outside the interval are interpreted modulo the size. The function raises an error if the size is not positive. permute can be used to scatter the distribution of non-uniform random functions such as random_zipfian or random_exponential so that values drawn more often are not trivially correlated. For instance, the following pgbench script simulates a possible real world workload typical for social media and blogging platforms where a few accounts generate excessive load:

\set size 1000000
\set r random_zipfian(1, :size, 1.07)
\set k 1 + permute(:r, :size)

In some cases several distinct distributions are needed which don't correlate with each other and this is when the optional seed parameter comes in handy:

\set k1 1 + permute(:r, :size, :default_seed + 123)
\set k2 1 + permute(:r, :size, :default_seed + 321)

A similar behavior can also be approximated with hash:

\set size 1000000
\set r random_zipfian(1, 100 * :size, 1.07)
\set k 1 + abs(hash(:r)) % :size

However, since hash generates collisions, some values will not be reachable and others will be more frequent than expected from the original distribution.

As an example, the full definition of the built-in TPC-B-like transaction is:

\set aid random(1, 100000 * :scale)
\set bid random(1, 1 * :scale)
\set tid random(1, 10 * :scale)
\set delta random(-5000, 5000)
BEGIN;
UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;
SELECT abalance FROM pgbench_accounts WHERE aid = :aid;
UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;
UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;
INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);
END;

This script allows each iteration of the transaction to reference different, randomly-chosen rows. (This example also shows why it's important for each client session to have its own variables — otherwise they'd not be independently touching different rows.)

Per-Transaction Logging

With the -l option (but without the --aggregate-interval option), pgbench writes information about each transaction to a log file. The log file will be named prefix.nnn, where <prefix> defaults to pgbench_log, and <nnn> is the PID of the pgbench process. The prefix can be changed by using the --log-prefix option. If the -j option is 2 or higher, so that there are multiple worker threads, each will have its own log file. The first worker will use the same name for its log file as in the standard single worker case. The additional log files for the other workers will be named prefix.nnn.mmm, where <mmm> is a sequential number for each worker starting with 1.

Each line in a log file describes one transaction. It contains the following space-separated fields:

<client_id>
identifies the client session that ran the transaction

<transaction_no>
counts how many transactions have been run by that session

<time>
transaction's elapsed time, in microseconds

<script_no>
identifies the script file that was used for the transaction (useful when multiple scripts are specified with -f or -b)

<time_epoch>
transaction's completion time, as a Unix-epoch time stamp

<time_us>
fractional-second part of transaction's completion time, in microseconds

<schedule_lag>
transaction start delay, that is the difference between the transaction's scheduled start time and the time it actually started, in microseconds (present only if --rate is specified)

<retries>
count of retries after serialization or deadlock errors during the transaction (present only if --max-tries is not equal to one)

When both --rate and --latency-limit are used, the <time> for a skipped transaction will be reported as skipped. If the transaction ends with a failure, its <time> will be reported as failed. If you use the --failures-detailed option, the <time> of the failed transaction will be reported as serialization, deadlock, or other depending on the type of failure (see Failures and Serialization/Deadlock Retries for more information).

Here is a snippet of a log file generated in a single-client run:

0 199 2241 0 1175850568 995598
0 200 2465 0 1175850568 998079
0 201 2513 0 1175850569 608
0 202 2038 0 1175850569 2663

Another example with --rate=100 and --latency-limit=5 (note the additional <schedule_lag> column):

0 81 4621 0 1412881037 912698 3005
0 82 6173 0 1412881037 914578 4304
0 83 skipped 0 1412881037 914578 5217
0 83 skipped 0 1412881037 914578 5099
0 83 4722 0 1412881037 916203 3108
0 84 4142 0 1412881037 918023 2333
0 85 2465 0 1412881037 919759 740

In this example, transaction 82 was late, because its latency (6.173 ms) was over the 5 ms limit. The next two transactions were skipped, because they were already late before they were even started.

The following example shows a snippet of a log file with failures and retries, with the maximum number of tries set to 10 (note the additional <retries> column):

3 0 47423 0 1499414498 34501 3
3 1 8333 0 1499414498 42848 0
3 2 8358 0 1499414498 51219 0
4 0 72345 0 1499414498 59433 6
1 3 41718 0 1499414498 67879 4
1 4 8416 0 1499414498 76311 0
3 3 33235 0 1499414498 84469 3
0 0 failed 0 1499414498 84905 9
2 0 failed 0 1499414498 86248 9
3 4 8307 0 1499414498 92788 0

If the --failures-detailed option is used, the type of failure is reported in the <time> like this:

3 0 47423 0 1499414498 34501 3
3 1 8333 0 1499414498 42848 0
3 2 8358 0 1499414498 51219 0
4 0 72345 0 1499414498 59433 6
1 3 41718 0 1499414498 67879 4
1 4 8416 0 1499414498 76311 0
3 3 33235 0 1499414498 84469 3
0 0 serialization 0 1499414498 84905 9
2 0 serialization 0 1499414498 86248 9
3 4 8307 0 1499414498 92788 0

When running a long test on hardware that can handle a lot of transactions, the log files can become very large. The --sampling-rate option can be used to log only a random sample of transactions.

Aggregated Logging

With the --aggregate-interval option, a different format is used for the log files. Each log line describes one aggregation interval. It contains the following space-separated fields:

<interval_start>
start time of the interval, as a Unix-epoch time stamp

<num_transactions>
number of transactions within the interval

<sum_latency>
sum of transaction latencies

<sum_latency_2>
sum of squares of transaction latencies

<min_latency>
minimum transaction latency

<max_latency>
maximum transaction latency

<sum_lag>
sum of transaction start delays (zero unless --rate is specified)

<sum_lag_2>
sum of squares of transaction start delays (zero unless --rate is specified)

<min_lag>
minimum transaction start delay (zero unless --rate is specified)

<max_lag>
maximum transaction start delay (zero unless --rate is specified)

<skipped>
number of transactions skipped because they would have started too late (zero unless --rate and --latency-limit are specified)

<retried>
number of retried transactions (zero unless --max-tries is not equal to one)

<retries>
number of retries after serialization or deadlock errors (zero unless --max-tries is not equal to one)

<serialization_failures>
number of transactions that got a serialization error and were not retried afterwards (zero unless --failures-detailed is specified)

<deadlock_failures>
number of transactions that got a deadlock error and were not retried afterwards (zero unless --failures-detailed is specified)

<other_sql_failures>
number of transactions that got an SQL error (zero unless both --failures-detailed and --continue-on-error are specified)

Here is some example output generated with this option:

pgbench --aggregate-interval=10 --time=20 --client=10 --log --rate=1000 --latency-limit=10 --failures-detailed --max-tries=10 test

1650260552 5178 26171317 177284491527 1136 44462 2647617 7321113867 0 9866 64 7564 28340 4148 0 0
1650260562 4808 25573984 220121792172 1171 62083 3037380 9666800914 0 9998 598 7392 26621 4527 0 0

Notice that while the plain (unaggregated) log format shows which script was used for each transaction, the aggregated format does not. Therefore if you need per-script data, you need to aggregate the data on your own.

Per-Statement Report

With the -r option, pgbench collects the following statistics for each statement:

The report displays retry statistics only if the --max-tries option is not equal to 1.

All values are computed for each statement executed by every client and are reported after the benchmark has finished.

For the default script, the output will look similar to this:

starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 1
query mode: simple
number of clients: 10
number of threads: 1
maximum number of tries: 1
number of transactions per client: 1000
number of transactions actually processed: 10000/10000
number of failed transactions: 0 (0.000%)
number of transactions above the 50.0 ms latency limit: 1311/10000 (13.110 %)
latency average = 28.488 ms
latency stddev = 21.009 ms
initial connection time = 69.068 ms
tps = 346.224794 (without initial connection time)
statement latencies in milliseconds and failures:
   0.012  0  \set aid random(1, 100000 * :scale)
   0.002  0  \set bid random(1, 1 * :scale)
   0.002  0  \set tid random(1, 10 * :scale)
   0.002  0  \set delta random(-5000, 5000)
   0.319  0  BEGIN;
   0.834  0  UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;
   0.641  0  SELECT abalance FROM pgbench_accounts WHERE aid = :aid;
  11.126  0  UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;
  12.961  0  UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;
   0.634  0  INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);
   1.957  0  END;

Another example of output for the default script using serializable default transaction isolation level (PGOPTIONS='-c default_transaction_isolation=serializable' pgbench ...):

starting vacuum...end.
transaction type: <builtin: TPC-B (sort of)>
scaling factor: 1
query mode: simple
number of clients: 10
number of threads: 1
maximum number of tries: 10
number of transactions per client: 1000
number of transactions actually processed: 6317/10000
number of failed transactions: 3683 (36.830%)
number of transactions retried: 7667 (76.670%)
total number of retries: 45339
number of transactions above the 50.0 ms latency limit: 106/6317 (1.678 %)
latency average = 17.016 ms
latency stddev = 13.283 ms
initial connection time = 45.017 ms
tps = 186.792667 (without initial connection time)
statement latencies in milliseconds, failures and retries:
  0.006     0      0  \set aid random(1, 100000 * :scale)
  0.001     0      0  \set bid random(1, 1 * :scale)
  0.001     0      0  \set tid random(1, 10 * :scale)
  0.001     0      0  \set delta random(-5000, 5000)
  0.385     0      0  BEGIN;
  0.773     0      1  UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;
  0.624     0      0  SELECT abalance FROM pgbench_accounts WHERE aid = :aid;
  1.098   320   3762  UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;
  0.582  3363  41576  UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;
  0.465     0      0  INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);
  1.933     0      0  END;

If multiple script files are specified, all statistics are reported separately for each script file.

Note that collecting the additional timing information needed for per-statement latency computation adds some overhead. This will slow average execution speed and lower the computed TPS. The amount of slowdown varies significantly depending on platform and hardware. Comparing average TPS values with and without latency reporting enabled is a good way to measure if the timing overhead is significant.

Failures and Serialization/Deadlock Retries

When executing pgbench, there are three main types of errors:

A client's run is aborted in case of a serious error; for example, the connection with the database server was lost or the end of script was reached without completing the last transaction. The client also aborts if a meta command fails, or if an SQL command fails for reasons other than serialization or deadlock errors when --continue-on-error is not specified. With --continue-on-error, the client does not abort on such SQL errors and instead proceeds to the next transaction. These cases are reported as other failures in the output. If the error occurs in a meta command, however, the client still aborts even when this option is specified.

If an SQL command fails due to serialization or deadlock errors, the client does not abort, regardless of whether --continue-on-error is used. Instead, the current transaction is rolled back, which also includes setting the client variables as they were before the run of this transaction (it is assumed that one transaction script contains only one transaction; see What Is the "Transaction" Actually Performed in pgbench? for more information). Transactions with serialization or deadlock errors are repeated after rollbacks until they complete successfully or reach the maximum number of tries (specified by the --max-tries option) / the maximum time of retries (specified by the --latency-limit option) / the end of benchmark (specified by the --time option). If the last trial run fails, this transaction will be reported as failed but the client is not aborted and continues to work.

[!NOTE] Without specifying the --max-tries option, a transaction will never be retried after a serialization or deadlock error because its default value is 1. Use an unlimited number of tries (--max-tries=0) and the --latency-limit option to limit only the maximum time of tries. You can also use the --time option to limit the benchmark duration under an unlimited number of tries.

Be careful when repeating scripts that contain multiple transactions: the script is always retried completely, so successful transactions can be performed several times.

Be careful when repeating transactions with shell commands. Unlike the results of SQL commands, the results of shell commands are not rolled back, except for the variable value of the \setshell command.

The latency of a successful transaction includes the entire time of transaction execution with rollbacks and retries. In the main report, when neither --progress, --rate, nor --latency-limit is specified, the latency average is computed from the total benchmark duration divided by both successful and failed transactions, and therefore includes failed transactions. Otherwise, the latency average and latency stddev shown in the main report are measured only for successful transactions. Detailed latency statistics, including the per-script and per-command reports, are also measured only for successful transactions and commands, not for failed transactions or commands.

The main report contains the number of failed transactions. If the --max-tries option is not equal to 1, the main report also contains statistics related to retries: the total number of retried transactions and total number of retries. The per-script report inherits all these fields from the main report. The per-statement report displays retry statistics only if the --max-tries option is not equal to 1.

If you want to group failures by basic types in per-transaction and aggregation logs, as well as in the main and per-script reports, use the --failures-detailed option. If you also want to distinguish all errors and failures (errors without retrying) by type including which limit for retries was exceeded and how much it was exceeded by for the serialization/deadlock failures, use the --verbose-errors option.

Table Access Methods

You may specify the Table Access Method for the pgbench tables. The environment variable PGOPTIONS specifies database configuration options that are passed to PostgreSQL via the command line (See ???). For example, a hypothetical default Table Access Method for the tables that pgbench creates called wuzza can be specified with:

PGOPTIONS='-c default_table_access_method=wuzza'

Good Practices

It is very easy to use pgbench to produce completely meaningless numbers. Here are some guidelines to help you get useful results.

In the first place, never believe any test that runs for only a few seconds. Use the -t or -T option to make the run last at least a few minutes, so as to average out noise. In some cases you could need hours to get numbers that are reproducible. It's a good idea to try the test run a few times, to find out if your numbers are reproducible or not.

For the default TPC-B-like test scenario, the initialization scale factor (-s) should be at least as large as the largest number of clients you intend to test (-c); else you'll mostly be measuring update contention. There are only -s rows in the pgbench_branches table, and every transaction wants to update one of them, so -c values in excess of -s will undoubtedly result in lots of transactions blocked waiting for other transactions.

The default test scenario is also quite sensitive to how long it's been since the tables were initialized: accumulation of dead rows and dead space in the tables changes the results. To understand the results you must keep track of the total number of updates and when vacuuming happens. If autovacuum is enabled it can result in unpredictable changes in measured performance.

A limitation of pgbench is that it can itself become the bottleneck when trying to test a large number of client sessions. This can be alleviated by running pgbench on a different machine from the database server, although low network latency will be essential. It might even be useful to run several pgbench instances concurrently, on several client machines, against the same database server.

Security

If untrusted users have access to a database that has not adopted a secure schema usage pattern, do not run pgbench in that database. pgbench uses unqualified names and does not manipulate the search path.

pgtestfsync

pg_test_fsync

pg_test_fsync

1

Application

pg_test_fsync

determine fastest

wal_sync_method

for

PostgreSQL

pg_test_fsync

option

Description

pg_test_fsync is intended to give you a reasonable idea of what the fastest ??? is on your specific system, as well as supplying diagnostic information in the event of an identified I/O problem. However, differences shown by pg_test_fsync might not make any significant difference in real database throughput, especially since many database servers are not speed-limited by their write-ahead logs. pg_test_fsync reports average file sync operation time in microseconds for each wal_sync_method, which can also be used to inform efforts to optimize the value of ???.

Options

pg_test_fsync accepts the following command-line options:

-f; --filename
Specifies the file name to write test data in. This file should be in the same file system that the pg_wal directory is or will be placed in. (pg_wal contains the WAL files.) The default is pg_test_fsync.out in the current directory.

-s; --secs-per-test
Specifies the number of seconds for each test. The more time per test, the greater the test's accuracy, but the longer it takes to run. The default is 5 seconds, which allows the program to complete in under 2 minutes.

-V; --version
Print the pg_test_fsync version and exit.

-?; --help
Show help about pg_test_fsync command line arguments, and exit.

Environment

The environment variable PG_COLOR specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

See Also

pgtesttiming

pg_test_timing

pg_test_timing

1

Application

pg_test_timing

measure timing overhead

pg_test_timing

option

Description

pg_test_timing is a tool to measure the timing overhead on your system and confirm that the system time never moves backwards. It reads supported clock sources over and over again as fast as it can for a specified length of time, and then prints statistics about the observed differences in successive clock readings, as well as which clock source will be used.

Smaller (but not zero) differences are better, since they imply both more-precise clock hardware and less overhead to collect a clock reading. Systems that are slow to collect timing data can give less accurate EXPLAIN ANALYZE results.

This tool is also helpful to determine if the track_io_timing configuration parameter is likely to produce useful results, and whether the TSC clock source (see ???) is available and if it will be used by default.

Options

pg_test_timing accepts the following command-line options:

-c cutoff; --cutoff=cutoff
Specifies the cutoff percentage for the list of exact observed timing durations (that is, the changes in the system clock value from one reading to the next). The list will end once the running percentage total reaches or exceeds this value, except that the largest observed duration will always be printed. The default cutoff is 99.99.

-d duration; --duration=duration
Specifies the test duration, in seconds. Longer durations give slightly better accuracy, and are more likely to discover problems with the system clock moving backwards. The default test duration is 3 seconds.

-V; --version
Print the pg_test_timing version and exit.

-?; --help
Show help about pg_test_timing command line arguments, and exit.

Usage

Interpreting Results

The first block of output has four columns, with rows showing a shifted-by-one log2(ns) histogram of timing durations (that is, the differences between successive clock readings). This is not the classic log2(n+1) histogram as it counts zeros separately and then switches to log2(ns) starting from value 1.

The columns are:

The second block of output goes into more detail, showing the exact timing differences observed. For brevity this list is cut off when the running-sum percentage exceeds the user-selectable cutoff value. However, the largest observed difference is always shown.

On platforms that support the TSC clock source, additional output sections are shown for the RDTSCP instruction (used for general timing needs, such as track_io_timing) and the RDTSC instruction (used for EXPLAIN ANALYZE). At the end of the output, the TSC frequency, which may either be sourced from CPU information directly, or the alternate calibration mechanism are shown, as well as whether the TSC clock source will be used by default.

The example results below show system clock timing where 99.99% of loops took between 16 and 63 nanoseconds. In the second block, we can see that the typical loop time is 40 nanoseconds, and the readings appear to have full nanosecond precision. Following the system clock results, the TSC clock source results are shown, in the same fashion. The RDTSCP instruction shows most loops completing in 20–30 nanoseconds, while the RDTSC instruction is the fastest with an average loop time of 20 nanoseconds. In this example the TSC clock source will be used by default, but can be disabled by setting timing_clock_source to system.

System clock source: clock_gettime (CLOCK_MONOTONIC)
Testing timing overhead for 3 seconds.
Average loop time including overhead: 44.67 ns
Histogram of timing durations:
   <= ns   % of total  running %      count
       0       0.0000     0.0000          0
       1       0.0000     0.0000          0
       3       0.0000     0.0000          0
       7       0.0000     0.0000          0
      15       0.0000     0.0000          0
      31      24.0606    24.0606    5385707
      63      75.8342    99.8948   16974658
     127       0.0900    99.9848      20143
     255       0.0069    99.9917       1542
     511       0.0014    99.9932        322
    1023       0.0003    99.9935         68
    2047       0.0001    99.9936         23
    4095       0.0036    99.9972        813
    8191       0.0018    99.9990        402
   16383       0.0005    99.9995        120
   32767       0.0001    99.9997         32
   65535       0.0001    99.9998         24

Observed timing durations up to 99.9900%:
      ns   % of total  running %      count
      29       3.6921     3.6921     826442
      30      16.6755    20.3676    3732628
      31       3.6930    24.0606     826637
      40      75.7761    99.8368   16961658
      41       0.0019    99.8387        431
...
     190       0.0003    99.9901         65
...
29657159       0.0000   100.0000          1

Clock source: RDTSCP
Average loop time including overhead: 37.32 ns
Histogram of timing durations:
   <= ns   % of total  running %      count
       0       0.0000     0.0000          0
       1       0.0000     0.0000          0
       3       0.0000     0.0000          0
       7       0.0000     0.0000          0
      15       0.0000     0.0000          0
      31      99.9499    99.9499   26782299
      63       0.0381    99.9880      10220
     127       0.0008    99.9889        224
     255       0.0052    99.9941       1403
     511       0.0013    99.9954        340
    1023       0.0001    99.9954         17
    2047       0.0000    99.9955          7
    4095       0.0021    99.9976        569
    8191       0.0013    99.9989        357
   16383       0.0005    99.9994        128
   32767       0.0003    99.9996         70
   65535       0.0001    99.9997         19

Observed timing durations up to 99.9900%:
      ns   % of total  running %      count
      20      16.9064    16.9064    4530201
      29      41.5214    58.4279   11125972
      30      41.5220    99.9499   11126126
      40       0.0089    99.9587       2374
...
     130       0.0007    99.9902        181
...
18501572       0.0000   100.0000          1

Fast clock source: RDTSC
Average loop time including overhead: 27.12 ns
Histogram of timing durations:
   <= ns   % of total  running %      count
       0       0.0000     0.0000          0
       1       0.0000     0.0000          0
       3       0.0000     0.0000          0
       7       0.0000     0.0000          0
      15       1.2247     1.2247     456231
      31      98.7566    99.9813   36789785
      63       0.0109    99.9921       4049
     127       0.0029    99.9951       1087
     255       0.0008    99.9959        305
     511       0.0007    99.9966        279
    1023       0.0000    99.9966          7
    2047       0.0001    99.9967         22
    4095       0.0018    99.9985        673
    8191       0.0010    99.9995        383
   16383       0.0003    99.9998         94
   32767       0.0001    99.9999         38
   65535       0.0000    99.9999          9

Observed timing durations up to 99.9900%:
      ns   % of total  running %      count
       9       0.6316     0.6316     235290
      10       0.5931     1.2247     220941
      20      91.4328    92.6574   34061442
      29       3.6427    96.3001    1357007
      30       3.6811    99.9813    1371336
      40       0.0089    99.9902       3325
...
61594291       0.0000   100.0000          1

TSC frequency source: x86, cpuid 0x15
TSC frequency in use: 2449228 kHz
TSC frequency from calibration: 2448603 kHz

TSC clock source will be used by default, unless timing_clock_source is set to 'system'.

See Also

Wiki discussion about timing

postgres-ref

postgres

postgres

1

Application

postgres

PostgreSQL

database server

postgres

option

Description

postgres is the PostgreSQL database server. In order for a client application to access a database it connects (over a network or locally) to a running postgres instance. The postgres instance then starts a separate server process to handle the connection.

One postgres instance always manages the data of exactly one database cluster. A database cluster is a collection of databases that is stored at a common file system location (the “data area”). More than one postgres instance can run on a system at one time, so long as they use different data areas and different communication ports (see below). When postgres starts it needs to know the location of the data area. The location must be specified by the -D option or the PGDATA environment variable; there is no default. Typically, -D or PGDATA points directly to the data area directory created by ???. Other possible file layouts are discussed in ???.

By default postgres starts in the foreground and prints log messages to the standard error stream. In practical applications postgres should be started as a background process, perhaps at boot time.

The postgres command can also be called in single-user mode. The primary use for this mode is during bootstrapping by ???. Sometimes it is used for debugging or disaster recovery; note that running a single-user server is not truly suitable for debugging the server, since no realistic interprocess communication and locking will happen. When invoked in single-user mode from the shell, the user can enter queries and the results will be printed to the screen, but in a form that is more useful for developers than end users. In the single-user mode, the session user will be set to the user with ID 1, and implicit superuser powers are granted to this user. This user does not actually have to exist, so the single-user mode can be used to manually recover from certain kinds of accidental damage to the system catalogs.

Options

postgres accepts the following command-line arguments. For a detailed discussion of the options consult ???. You can save typing most of these options by setting up a configuration file. Some (safe) options can also be set from the connecting client in an application-dependent way to apply only for that session. For example, if the environment variable PGOPTIONS is set, then libpq-based clients will pass that string to the server, which will interpret it as postgres command-line options.

General Purpose

-B nbuffers
Sets the number of shared buffers for use by the server processes. The default value of this parameter is chosen automatically by initdb. Specifying this option is equivalent to setting the ??? configuration parameter.

-c name=value
Sets a named run-time parameter. The configuration parameters supported by PostgreSQL are described in ???. Most of the other command line options are in fact short forms of such a parameter assignment. -c can appear multiple times to set multiple parameters.

-C name
Prints the value of the named run-time parameter, and exits. (See the -c option above for details.) This returns values from postgresql.conf, modified by any parameters supplied in this invocation. It does not reflect parameters supplied when the cluster was started.

This can be used on a running server for most parameters. However, the server must be shut down for some runtime-computed parameters (e.g., ???, ???, and ???).

This option is meant for other programs that interact with a server instance, such as ???, to query configuration parameter values. User-facing applications should instead use SHOW or the pg_settings view.

-d debug-level
Sets the debug level. The higher this value is set, the more debugging output is written to the server log. Values are from 1 to 5. It is also possible to pass -d 0 for a specific session, which will prevent the server log level of the parent postgres process from being propagated to this session.

-D datadir
Specifies the file system location of the database configuration files. See ??? for details.

-e
Sets the default date style to “European”, that is DMY ordering of input date fields. This also causes the day to be printed before the month in certain date output formats. See ??? for more information.

-F
Disables fsync calls for improved performance, at the risk of data corruption in the event of a system crash. Specifying this option is equivalent to disabling the ??? configuration parameter. Read the detailed documentation before using this!

-h hostname
Specifies the IP host name or address on which postgres is to listen for TCP/IP connections from client applications. The value can also be a comma-separated list of addresses, or * to specify listening on all available interfaces. An empty value specifies not listening on any IP addresses, in which case only Unix-domain sockets can be used to connect to the server. Defaults to listening only on localhost. Specifying this option is equivalent to setting the ??? configuration parameter.

-i
Allows remote clients to connect via TCP/IP (Internet domain) connections. Without this option, only local connections are accepted. This option is equivalent to setting listen_addresses to * in postgresql.conf or via -h.

This option is deprecated since it does not allow access to the full functionality of ???. It's usually better to set listen_addresses directly.

-k directory
Specifies the directory of the Unix-domain socket on which postgres is to listen for connections from client applications. The value can also be a comma-separated list of directories. An empty value specifies not listening on any Unix-domain sockets, in which case only TCP/IP sockets can be used to connect to the server. The default value is normally /tmp, but that can be changed at build time. Specifying this option is equivalent to setting the ??? configuration parameter.

-l
Enables secure connections using SSL. PostgreSQL must have been compiled with support for SSL for this option to be available. For more information on using SSL, refer to ???.

-N max-connections
Sets the maximum number of client connections that this server will accept. The default value of this parameter is chosen automatically by initdb. Specifying this option is equivalent to setting the ??? configuration parameter.

-p port
Specifies the TCP/IP port or local Unix domain socket file extension on which postgres is to listen for connections from client applications. Defaults to the value of the PGPORT environment variable, or if PGPORT is not set, then defaults to the value established during compilation (normally 5432). If you specify a port other than the default port, then all client applications must specify the same port using either command-line options or PGPORT.

-s
Print time information and other statistics at the end of each command. This is useful for benchmarking or for use in tuning the number of buffers.

-S <work-mem>
Specifies the base amount of memory to be used by sorts and hash tables before resorting to temporary disk files. See the description of the work_mem configuration parameter in ???.

-V; --version
Print the postgres version and exit.

--name=value
Sets a named run-time parameter; a shorter form of -c.

--describe-config
This option dumps out the server's internal configuration variables, descriptions, and defaults in tab-delimited COPY format. It is designed primarily for use by administration tools.

-?; --help
Show help about postgres command line arguments, and exit.

Semi-Internal Options

The options described here are used mainly for debugging purposes, and in some cases to assist with recovery of severely damaged databases. There should be no reason to use them in a production database setup. They are listed here only for use by PostgreSQL system developers. Furthermore, these options might change or be removed in a future release without notice.

-f { s | i | o | b | t | n | m | h }
Forbids the use of particular scan and join methods: s and i disable sequential and index scans respectively, o, b and t disable index-only scans, bitmap index scans, and TID scans respectively, while n, m, and h disable nested-loop, merge and hash joins respectively.

Neither sequential scans nor nested-loop joins can be disabled completely; the -fs and -fn options simply discourage the optimizer from using those plan types if it has any other alternative.

-O
Allows the structure of system tables to be modified. This is used by initdb.

-P
Ignore system indexes when reading system tables, but still update the indexes when modifying the tables. This is useful when recovering from damaged system indexes.

-t pa[rser] | pl[anner] | e[xecutor]
Print timing statistics for each query relating to each of the major system modules. This option cannot be used together with the -s option.

-T
This option is for debugging problems that cause a server process to die abnormally. The ordinary strategy in this situation is to notify all other server processes that they must terminate, by sending them SIGQUIT signals. With this option, SIGABRT will be sent instead, resulting in production of core dump files.

-v <protocol>
Specifies the version number of the frontend/backend protocol to be used for a particular session. This option is for internal use only.

-W <seconds>
A delay of this many seconds occurs when a new server process is started, after it conducts the authentication procedure. This is intended to give an opportunity to attach to the server process with a debugger.

Options for Single-User Mode

single-user mode

The following options only apply to the single-user mode (see Single-User Mode below).

--single
Selects the single-user mode. This must be the first argument on the command line.

<database>
Specifies the name of the database to be accessed. This must be the last argument on the command line. If it is omitted it defaults to the user name.

-E
Echo all commands to standard output before executing them.

-j
Use semicolon followed by two newlines, rather than just newline, as the command entry terminator.

-r <filename>
Send all server log output to <filename>. This option is only honored when supplied as a command-line option.

Environment

PGCLIENTENCODING
Default character encoding used by clients. (The clients can override this individually.) This value can also be set in the configuration file.

PGDATA
Default data directory location

PGDATESTYLE
Default value of the ??? run-time parameter. (The use of this environment variable is deprecated.)

PGPORT
Default port number (preferably set in the configuration file)

Diagnostics

A failure message mentioning semget or shmget probably indicates you need to configure your kernel to provide adequate shared memory and semaphores. For more discussion see ???. You might be able to postpone reconfiguring your kernel by decreasing ??? to reduce the shared memory consumption of PostgreSQL, and/or by reducing ??? to reduce the semaphore consumption.

A failure message suggesting that another server is already running should be checked carefully, for example by using the command

$ ps ax | grep postgres

or

$ ps -ef | grep postgres

depending on your system. If you are certain that no conflicting server is running, you can remove the lock file mentioned in the message and try again.

A failure message indicating inability to bind to a port might indicate that that port is already in use by some non-PostgreSQL process. You might also get this error if you terminate postgres and immediately restart it using the same port; in this case, you must simply wait a few seconds until the operating system closes the port before trying again. Finally, you might get this error if you specify a port number that your operating system considers to be reserved. For example, many versions of Unix consider port numbers under 1024 to be “trusted” and only permit the Unix superuser to access them.

Notes

The utility command ??? can be used to start and shut down the postgres server safely and comfortably.

If at all possible, do not use SIGKILL to kill the main postgres server. Doing so will prevent postgres from freeing the system resources (e.g., shared memory and semaphores) that it holds before terminating. This might cause problems for starting a fresh postgres run.

To terminate the postgres server normally, the signals SIGTERM, SIGINT, or SIGQUIT can be used. The first will wait for all clients to terminate before quitting, the second will forcefully disconnect all clients, and the third will quit immediately without proper shutdown, resulting in a recovery run during restart.

The SIGHUP signal will reload the server configuration files. It is also possible to send SIGHUP to an individual server process, but that is usually not sensible.

To cancel a running query, send the SIGINT signal to the process running that command. To terminate a backend process cleanly, send SIGTERM to that process. See also pg_cancel_backend and pg_terminate_backend in ??? for the SQL-callable equivalents of these two actions.

The postgres server uses SIGQUIT to tell subordinate server processes to terminate without normal cleanup. This signal should not be used by users. It is also unwise to send SIGKILL to a server process — the main postgres process will interpret this as a crash and will force all the sibling processes to quit as part of its standard crash-recovery procedure.

Bugs

The -- options will not work on FreeBSD or OpenBSD. Use -c instead. This is a bug in the affected operating systems; a future release of PostgreSQL will provide a workaround if this is not fixed.

Single-User Mode

To start a single-user mode server, use a command like

postgres --single -D /usr/local/pgsql/data other-options my_database

Provide the correct path to the database directory with -D, or make sure that the environment variable PGDATA is set. Also specify the name of the particular database you want to work in.

Normally, the single-user mode server treats newline as the command entry terminator; there is no intelligence about semicolons, as there is in psql. To continue a command across multiple lines, you must type backslash just before each newline except the last one. The backslash and adjacent newline are both dropped from the input command. Note that this will happen even when within a string literal or comment.

But if you use the -j command line switch, a single newline does not terminate command entry; instead, the sequence semicolon-newline-newline does. That is, type a semicolon immediately followed by a completely empty line. Backslash-newline is not treated specially in this mode. Again, there is no intelligence about such a sequence appearing within a string literal or comment.

In either input mode, if you type a semicolon that is not just before or part of a command entry terminator, it is considered a command separator. When you do type a command entry terminator, the multiple statements you've entered will be executed as a single transaction.

To quit the session, type EOF (Control+D, usually). If you've entered any text since the last command entry terminator, then EOF will be taken as a command entry terminator, and another EOF will be needed to exit.

Note that the single-user mode server does not provide sophisticated line-editing features (no command history, for example). Single-user mode also does not do any background processing, such as automatic checkpoints or replication.

Examples

To start postgres in the background using default values, type:

$ nohup postgres >logfile 2>&1 </dev/null &

To start postgres with a specific port, e.g., 1234:

$ postgres -p 1234

To connect to this server using psql, specify this port with the -p option:

$ psql -p 1234

or set the environment variable PGPORT:

$ export PGPORT=1234
$ psql

Named run-time parameters can be set in either of these styles:

$ postgres -c work_mem=1234
$ postgres --work-mem=1234

Either form overrides whatever setting might exist for work_mem in postgresql.conf. Notice that underscores in parameter names can be written as either underscore or dash on the command line. Except for short-term experiments, it's probably better practice to edit the setting in postgresql.conf than to rely on a command-line switch to set a parameter.

See Also

???, ???

prepare

PREPARE

prepared statements

creating

PREPARE

7

SQL - Language Statements

PREPARE

prepare a statement for execution

PREPARE

name

[ (

data_type

[, ...] ) ] AS

statement

Description

PREPARE creates a prepared statement. A prepared statement is a server-side object that can be used to optimize performance. When the PREPARE statement is executed, the specified statement is parsed, analyzed, and rewritten. When an EXECUTE command is subsequently issued, the prepared statement is planned and executed. This division of labor avoids repetitive parse analysis work, while allowing the execution plan to depend on the specific parameter values supplied.

Prepared statements can take parameters: values that are substituted into the statement when it is executed. When creating the prepared statement, refer to parameters by position, using $1, $2, etc. A corresponding list of parameter data types can optionally be specified. When a parameter's data type is not specified or is declared as unknown, the type is inferred from the context in which the parameter is first referenced (if possible). When executing the statement, specify the actual values for these parameters in the EXECUTE statement. Refer to ??? for more information about that.

Prepared statements only last for the duration of the current database session. When the session ends, the prepared statement is forgotten, so it must be recreated before being used again. This also means that a single prepared statement cannot be used by multiple simultaneous database clients; however, each client can create their own prepared statement to use. Prepared statements can be manually cleaned up using the DEALLOCATE command.

Prepared statements potentially have the largest performance advantage when a single session is being used to execute a large number of similar statements. The performance difference will be particularly significant if the statements are complex to plan or rewrite, e.g., if the query involves a join of many tables or requires the application of several rules. If the statement is relatively simple to plan and rewrite but relatively expensive to execute, the performance advantage of prepared statements will be less noticeable.

Parameters

<name>
An arbitrary name given to this particular prepared statement. It must be unique within a single session and is subsequently used to execute or deallocate a previously prepared statement.

<data_type>
The data type of a parameter to the prepared statement. If the data type of a particular parameter is unspecified or is specified as unknown, it will be inferred from the context in which the parameter is first referenced. To refer to the parameters in the prepared statement itself, use $1, $2, etc.

<statement>
Any SELECT, INSERT, UPDATE, DELETE, MERGE, or VALUES statement.

Notes

A prepared statement can be executed with either a generic plan or a custom plan. A generic plan is the same across all executions, while a custom plan is generated for a specific execution using the parameter values given in that call. Use of a generic plan avoids planning overhead, but in some situations a custom plan will be much more efficient to execute because the planner can make use of knowledge of the parameter values. (Of course, if the prepared statement has no parameters, then this is moot and a generic plan is always used.)

By default (that is, when ??? is set to auto), the server will automatically choose whether to use a generic or custom plan for a prepared statement that has parameters. The current rule for this is that the first five executions are done with custom plans and the average estimated cost of those plans is calculated. Then a generic plan is created and its estimated cost is compared to the average custom-plan cost. Subsequent executions use the generic plan if its cost is not so much higher than the average custom-plan cost as to make repeated replanning seem preferable.

This heuristic can be overridden, forcing the server to use either generic or custom plans, by setting plan_cache_mode to force_generic_plan or force_custom_plan respectively. This setting is primarily useful if the generic plan's cost estimate is badly off for some reason, allowing it to be chosen even though its actual cost is much more than that of a custom plan.

To examine the query plan PostgreSQL is using for a prepared statement, use EXPLAIN, for example

EXPLAIN EXECUTE name(parameter_values);

If a generic plan is in use, it will contain parameter symbols $n, while a custom plan will have the supplied parameter values substituted into it.

For more information on query planning and the statistics collected by PostgreSQL for that purpose, see the ??? documentation.

Although the main point of a prepared statement is to avoid repeated parse analysis and planning of the statement, PostgreSQL will force re-analysis and re-planning of the statement before using it whenever database objects used in the statement have undergone definitional (DDL) changes or their planner statistics have been updated since the previous use of the prepared statement. Also, if the value of ??? changes from one use to the next, the statement will be re-parsed using the new search_path. (This latter behavior is new as of PostgreSQL 9.3.) These rules make use of a prepared statement semantically almost equivalent to re-submitting the same query text over and over, but with a performance benefit if no object definitions are changed, especially if the best plan remains the same across uses. An example of a case where the semantic equivalence is not perfect is that if the statement refers to a table by an unqualified name, and then a new table of the same name is created in a schema appearing earlier in the search_path, no automatic re-parse will occur since no object used in the statement changed. However, if some other change forces a re-parse, the new table will be referenced in subsequent uses.

You can see all prepared statements available in the session by querying the pg_prepared_statements system view.

Examples

Create a prepared statement for an INSERT statement, and then execute it:

PREPARE fooplan (int, text, bool, numeric) AS
    INSERT INTO foo VALUES($1, $2, $3, $4);
EXECUTE fooplan(1, 'Hunter Valley', 't', 200.00);

Create a prepared statement for a SELECT statement, and then execute it:

PREPARE usrrptplan (int) AS
    SELECT * FROM users u, logs l WHERE u.usrid=$1 AND u.usrid=l.usrid
    AND l.date = $2;
EXECUTE usrrptplan(1, current_date);

In this example, the data type of the second parameter is not specified, so it is inferred from the context in which $2 is used.

Compatibility

The SQL standard includes a PREPARE statement, but it is only for use in embedded SQL. This version of the PREPARE statement also uses a somewhat different syntax.

See Also

prepare_transaction

PREPARE TRANSACTION

PREPARE TRANSACTION

7

SQL - Language Statements

PREPARE TRANSACTION

prepare the current transaction for two-phase commit

PREPARE TRANSACTION

transaction_id

Description

PREPARE TRANSACTION prepares the current transaction for two-phase commit. After this command, the transaction is no longer associated with the current session; instead, its state is fully stored on disk, and there is a very high probability that it can be committed successfully, even if a database crash occurs before the commit is requested.

Once prepared, a transaction can later be committed or rolled back with COMMIT PREPARED or ROLLBACK PREPARED, respectively. Those commands can be issued from any session, not only the one that executed the original transaction.

From the point of view of the issuing session, PREPARE TRANSACTION is not unlike a ROLLBACK command: after executing it, there is no active current transaction, and the effects of the prepared transaction are no longer visible. (The effects will become visible again if the transaction is committed.)

If the PREPARE TRANSACTION command fails for any reason, it becomes a ROLLBACK: the current transaction is canceled.

Parameters

<transaction_id>
An arbitrary identifier that later identifies this transaction for COMMIT PREPARED or ROLLBACK PREPARED. The identifier must be written as a string literal, and must be less than 200 bytes long. It must not be the same as the identifier used for any currently prepared transaction.

Notes

PREPARE TRANSACTION is not intended for use in applications or interactive sessions. Its purpose is to allow an external transaction manager to perform atomic global transactions across multiple databases or other transactional resources. Unless you're writing a transaction manager, you probably shouldn't be using PREPARE TRANSACTION.

This command must be used inside a transaction block. Use BEGIN to start one.

It is not currently allowed to PREPARE a transaction that has executed any operations involving temporary tables or the session's temporary namespace, created any cursors WITH HOLD, or executed LISTEN, UNLISTEN, or NOTIFY. Those features are too tightly tied to the current session to be useful in a transaction to be prepared.

If the transaction modified any run-time parameters with SET (without the LOCAL option), those effects persist after PREPARE TRANSACTION, and will not be affected by any later COMMIT PREPARED or ROLLBACK PREPARED. Thus, in this one respect PREPARE TRANSACTION acts more like COMMIT than ROLLBACK.

All currently available prepared transactions are listed in the pg_prepared_xacts system view.

[!CAUTION] It is unwise to leave transactions in the prepared state for a long time. This will interfere with the ability of VACUUM to reclaim storage, and in extreme cases could cause the database to shut down to prevent transaction ID wraparound (see ???). Keep in mind also that the transaction continues to hold whatever locks it held. The intended usage of the feature is that a prepared transaction will normally be committed or rolled back as soon as an external transaction manager has verified that other databases are also prepared to commit.

If you have not set up an external transaction manager to track prepared transactions and ensure they get closed out promptly, it is best to keep the prepared-transaction feature disabled by setting ??? to zero. This will prevent accidental creation of prepared transactions that might then be forgotten and eventually cause problems.

Examples

Prepare the current transaction for two-phase commit, using foobar as the transaction identifier:

PREPARE TRANSACTION 'foobar';

Compatibility

PREPARE TRANSACTION is a PostgreSQL extension. It is intended for use by external transaction management systems, some of which are covered by standards (such as X/Open XA), but the SQL side of those systems is not standardized.

See Also

reassign_owned

REASSIGN OWNED

REASSIGN OWNED

7

SQL - Language Statements

REASSIGN OWNED

change the ownership of database objects owned by a database role

REASSIGN OWNED BY {

old_role

| CURRENT_ROLE | CURRENT_USER | SESSION_USER } [, ...] TO {

new_role

| CURRENT_ROLE | CURRENT_USER | SESSION_USER }

Description

REASSIGN OWNED instructs the system to change the ownership of database objects owned by any of the <old_roles> to <new_role>.

Parameters

<old_role>
The name of a role. The ownership of all the objects within the current database, and of all shared objects (databases, tablespaces), owned by this role will be reassigned to <new_role>. Subscriptions, although stored in a shared catalog, belong to a specific database and are reassigned only when REASSIGN OWNED is executed in that database.

<new_role>
The name of the role that will be made the new owner of the affected objects.

Notes

REASSIGN OWNED is often used to prepare for the removal of one or more roles. Because REASSIGN OWNED does not affect objects within other databases, it is usually necessary to execute this command in each database that contains objects owned by a role that is to be removed.

REASSIGN OWNED requires membership on both the source role(s) and the target role.

The DROP OWNED command is an alternative that simply drops all the database objects owned by one or more roles.

The REASSIGN OWNED command does not affect any privileges granted to the <old_roles> on objects that are not owned by them. Likewise, it does not affect default privileges created with ALTER DEFAULT PRIVILEGES. Use DROP OWNED to revoke such privileges.

See ??? for more discussion.

Compatibility

The REASSIGN OWNED command is a PostgreSQL extension.

See Also

refresh_materialized_view

REFRESH MATERIALIZED VIEW

REFRESH MATERIALIZED VIEW

7

SQL - Language Statements

REFRESH MATERIALIZED VIEW

replace the contents of a materialized view

REFRESH MATERIALIZED VIEW [ CONCURRENTLY ]

name

[ WITH [ NO ] DATA ]

Description

REFRESH MATERIALIZED VIEW completely replaces the contents of a materialized view. To execute this command you must have the MAINTAIN privilege on the materialized view. The old contents are discarded. If WITH DATA is specified (or defaults) the backing query is executed to provide the new data, and the materialized view is left in a scannable state. If WITH NO DATA is specified no new data is generated and the materialized view is left in an unscannable state.

CONCURRENTLY and WITH NO DATA may not be specified together.

Parameters

CONCURRENTLY
Refresh the materialized view without locking out concurrent selects on the materialized view. Without this option a refresh which affects a lot of rows will tend to use fewer resources and complete more quickly, but could block other connections which are trying to read from the materialized view. This option may be faster in cases where a small number of rows are affected.

This option is only allowed if there is at least one UNIQUE index on the materialized view which uses only column names and includes all rows; that is, it must not be an expression index or include a WHERE clause.

This option can only be used when the materialized view is already populated.

Even with this option only one REFRESH at a time may run against any one materialized view.

<name>
The name (optionally schema-qualified) of the materialized view to refresh.

Notes

If there is an ORDER BY clause in the materialized view's defining query, the original contents of the materialized view will be ordered that way; but REFRESH MATERIALIZED VIEW does not guarantee to preserve that ordering.

While REFRESH MATERIALIZED VIEW is running, the ??? is temporarily changed to pg_catalog, pg_temp.

Examples

This command will replace the contents of the materialized view called order_summary using the query from the materialized view's definition, and leave it in a scannable state:

REFRESH MATERIALIZED VIEW order_summary;

This command will free storage associated with the materialized view annual_statistics_basis and leave it in an unscannable state:

REFRESH MATERIALIZED VIEW annual_statistics_basis WITH NO DATA;

Compatibility

REFRESH MATERIALIZED VIEW is a PostgreSQL extension.

See Also

reindex

REINDEX

REINDEX

7

SQL - Language Statements

REINDEX

rebuild indexes

REINDEX [ (

option

[, ...] ) ] { INDEX | TABLE | SCHEMA } [ CONCURRENTLY ]

name

REINDEX [ (

option

[, ...] ) ] { DATABASE | SYSTEM } [ CONCURRENTLY ] [

name

]

where

option

can be one of:

CONCURRENTLY [

boolean

] TABLESPACE

new_tablespace

VERBOSE [

boolean

]

Description

REINDEX rebuilds an index using the data stored in the index's table, replacing the old copy of the index. There are several scenarios in which to use REINDEX:

Parameters

INDEX
Recreate the specified index. This form of REINDEX cannot be executed inside a transaction block when used with a partitioned index.

TABLE
Recreate all indexes of the specified table. If the table has a secondary “TOAST” table, that is reindexed as well. This form of REINDEX cannot be executed inside a transaction block when used with a partitioned table.

SCHEMA
Recreate all indexes of the specified schema. If a table of this schema has a secondary “TOAST” table, that is reindexed as well. Indexes on shared system catalogs are also processed. This form of REINDEX cannot be executed inside a transaction block.

DATABASE
Recreate all indexes within the current database, except system catalogs. Indexes on system catalogs are not processed. This form of REINDEX cannot be executed inside a transaction block.

SYSTEM
Recreate all indexes on system catalogs within the current database. Indexes on shared system catalogs are included. Indexes on user tables are not processed. This form of REINDEX cannot be executed inside a transaction block.

<name>
The name of the specific index, table, or database to be reindexed. Index and table names can be schema-qualified. Presently, REINDEX DATABASE and REINDEX SYSTEM can only reindex the current database. Their parameter is optional, and it must match the current database's name.

CONCURRENTLY
When this option is used, PostgreSQL will rebuild the index without taking any locks that prevent concurrent inserts, updates, or deletes on the table; whereas a standard index rebuild locks out writes (but not reads) on the table until it's done. There are several caveats to be aware of when using this option — see Rebuilding Indexes Concurrently below.

For temporary tables, REINDEX is always non-concurrent, as no other session can access them, and non-concurrent reindex is cheaper.

TABLESPACE
Specifies that indexes will be rebuilt on a new tablespace.

VERBOSE
Prints a progress report as each index is reindexed at INFO level.

<boolean>
Specifies whether the selected option should be turned on or off. You can write TRUE, ON, or 1 to enable the option, and FALSE, OFF, or 0 to disable it. The <boolean> value can also be omitted, in which case TRUE is assumed.

<new_tablespace>
The tablespace where indexes will be rebuilt.

Notes

If you suspect corruption of an index on a user table, you can simply rebuild that index, or all indexes on the table, using REINDEX INDEX or REINDEX TABLE.

Things are more difficult if you need to recover from corruption of an index on a system table. In this case it's important for the system to not have used any of the suspect indexes itself. (Indeed, in this sort of scenario you might find that server processes are crashing immediately at start-up, due to reliance on the corrupted indexes.) To recover safely, the server must be started with the -P option, which prevents it from using indexes for system catalog lookups.

One way to do this is to shut down the server and start a single-user PostgreSQL server with the -P option included on its command line. Then, REINDEX DATABASE, REINDEX SYSTEM, REINDEX TABLE, or REINDEX INDEX can be issued, depending on how much you want to reconstruct. If in doubt, use REINDEX SYSTEM to select reconstruction of all system indexes in the database. Then quit the single-user server session and restart the regular server. See the ??? reference page for more information about how to interact with the single-user server interface.

Alternatively, a regular server session can be started with -P included in its command line options. The method for doing this varies across clients, but in all libpq-based clients, it is possible to set the PGOPTIONS environment variable to -P before starting the client. Note that while this method does not require locking out other clients, it might still be wise to prevent other users from connecting to the damaged database until repairs have been completed.

REINDEX is similar to a drop and recreate of the index in that the index contents are rebuilt from scratch. However, the locking considerations are rather different. REINDEX locks out writes but not reads of the index's parent table. It also takes an ACCESS EXCLUSIVE lock on the specific index being processed, which will block reads that attempt to use that index. In particular, the query planner tries to take an ACCESS SHARE lock on every index of the table, regardless of the query, and so REINDEX blocks virtually any queries except for some prepared queries whose plan has been cached and which don't use this very index. In contrast, DROP INDEX momentarily takes an ACCESS EXCLUSIVE lock on the parent table, blocking both writes and reads. The subsequent CREATE INDEX locks out writes but not reads; since the index is not there, no read will attempt to use it, meaning that there will be no blocking but reads might be forced into expensive sequential scans.

While REINDEX is running, the ??? is temporarily changed to pg_catalog, pg_temp.

Reindexing a single index or table requires having the MAINTAIN privilege on the table. Note that while REINDEX on a partitioned index or table requires having the MAINTAIN privilege on the partitioned table, such commands skip the privilege checks when processing the individual partitions. Reindexing a schema or database requires being the owner of that schema or database or having privileges of the ??? role. Note specifically that it's thus possible for non-superusers to rebuild indexes of tables owned by other users. However, as a special exception, REINDEX DATABASE, REINDEX SCHEMA, and REINDEX SYSTEM will skip indexes on shared catalogs unless the user has the MAINTAIN privilege on the catalog.

Reindexing partitioned indexes or partitioned tables is supported with REINDEX INDEX or REINDEX TABLE, respectively. Each partition of the specified partitioned relation is reindexed in a separate transaction. Those commands cannot be used inside a transaction block when working on a partitioned table or index.

When using the TABLESPACE clause with REINDEX on a partitioned index or table, only the tablespace references of the leaf partitions are updated. As partitioned indexes are not updated, it is recommended to separately use ALTER TABLE ONLY on them so as any new partitions attached inherit the new tablespace. On failure, it may not have moved all the indexes to the new tablespace. Re-running the command will rebuild all the leaf partitions and move previously-unprocessed indexes to the new tablespace.

If SCHEMA, DATABASE or SYSTEM is used with TABLESPACE, system relations are skipped and a single WARNING will be generated. Indexes on TOAST tables are rebuilt, but not moved to the new tablespace.

Rebuilding Indexes Concurrently

index

rebuilding concurrently

Rebuilding an index can interfere with regular operation of a database. Normally PostgreSQL locks the table whose index is rebuilt against writes and performs the entire index build with a single scan of the table. Other transactions can still read the table, but if they try to insert, update, or delete rows in the table they will block until the index rebuild is finished. This could have a severe effect if the system is a live production database. Very large tables can take many hours to be indexed, and even for smaller tables, an index rebuild can lock out writers for periods that are unacceptably long for a production system.

PostgreSQL supports rebuilding indexes with minimum locking of writes. This method is invoked by specifying the CONCURRENTLY option of REINDEX. When this option is used, PostgreSQL must perform two scans of the table for each index that needs to be rebuilt and wait for termination of all existing transactions that could potentially use the index. This method requires more total work than a standard index rebuild and takes significantly longer to complete as it needs to wait for unfinished transactions that might modify the index. However, since it allows normal operations to continue while the index is being rebuilt, this method is useful for rebuilding indexes in a production environment. Of course, the extra CPU, memory and I/O load imposed by the index rebuild may slow down other operations.

The following steps occur in a concurrent reindex. Each step is run in a separate transaction. If there are multiple indexes to be rebuilt, then each step loops through all the indexes before moving to the next step.

  1. A new transient index definition is added to the catalog pg_index. This definition will be used to replace the old index. A SHARE UPDATE EXCLUSIVE lock at session level is taken on the indexes being reindexed as well as their associated tables to prevent any schema modification while processing.

  2. A first pass to build the index is done for each new index. Once the index is built, its flag pg_index.indisready is switched to “true” to make it ready for inserts, making it visible to other sessions once the transaction that performed the build is finished. This step is done in a separate transaction for each index.

  3. Then a second pass is performed to add tuples that were added while the first pass was running. This step is also done in a separate transaction for each index.

  4. All the constraints that refer to the index are changed to refer to the new index definition, and the names of the indexes are changed. At this point, pg_index.indisvalid is switched to “true” for the new index and to “false” for the old, and a cache invalidation is done causing all sessions that referenced the old index to be invalidated.

  5. The old indexes have pg_index.indisready switched to “false” to prevent any new tuple insertions, after waiting for running queries that might reference the old index to complete.

  6. The old indexes are dropped. The SHARE UPDATE EXCLUSIVE session locks for the indexes and the table are released.

If a problem arises while rebuilding the indexes, such as a uniqueness violation in a unique index, the REINDEX command will fail but leave behind an “invalid” new index in addition to the pre-existing one. This index will be ignored for querying purposes because it might be incomplete; however it will still consume update overhead. The psql \d command will report such an index as INVALID:

postgres=# \d tab
       Table "public.tab"
 Column |  Type   | Modifiers
--------+---------+-----------
 col    | integer |
Indexes:
    "idx" btree (col)
    "idx_ccnew" btree (col) INVALID

If the index marked INVALID is suffixed _ccnew, then it corresponds to the transient index created during the concurrent operation, and the recommended recovery method is to drop it using DROP INDEX, then attempt REINDEX CONCURRENTLY again. If the invalid index is instead suffixed _ccold, it corresponds to the original index which could not be dropped; the recommended recovery method is to just drop said index, since the rebuild proper has been successful. A nonzero number may be appended to the suffix of the invalid index names to keep them unique, like _ccnew1, _ccold2, etc.

Regular index builds permit other regular index builds on the same table to occur simultaneously, but only one concurrent index build can occur on a table at a time. In both cases, no other types of schema modification on the table are allowed meanwhile. Another difference is that a regular REINDEX TABLE or REINDEX INDEX command can be performed within a transaction block, but REINDEX CONCURRENTLY cannot.

Like any long-running transaction, REINDEX on a table can affect which tuples can be removed by concurrent VACUUM on any other table.

REINDEX SYSTEM does not support CONCURRENTLY since system catalogs cannot be reindexed concurrently.

Furthermore, indexes for exclusion constraints cannot be reindexed concurrently. If such an index is named directly in this command, an error is raised. If a table or database with exclusion constraint indexes is reindexed concurrently, those indexes will be skipped. (It is possible to reindex such indexes without the CONCURRENTLY option.)

Each backend running REINDEX will report its progress in the pg_stat_progress_create_index view. See ??? for details.

Examples

Rebuild a single index:

REINDEX INDEX my_index;

Rebuild all the indexes on the table my_table:

REINDEX TABLE my_table;

Rebuild all indexes in a particular database, without trusting the system indexes to be valid already:

$ export PGOPTIONS="-P"
$ psql broken_db
...
broken_db=> REINDEX DATABASE broken_db;
broken_db=> \q

Rebuild indexes for a table, without blocking read and write operations on involved relations while reindexing is in progress:

REINDEX TABLE CONCURRENTLY my_broken_table;

Compatibility

There is no REINDEX command in the SQL standard.

See Also

reindexdb

reindexdb

reindexdb

1

Application

reindexdb

reindex a

PostgreSQL

database

reindexdb

connection-option

option

-S

--schema

schema

-t

--table

table

-i

--index

index

-s

--system

dbname

-a

--all

Description

reindexdb is a utility for rebuilding indexes in a PostgreSQL database.

reindexdb is a wrapper around the SQL command REINDEX. There is no effective difference between reindexing databases via this utility and via other methods for accessing the server.

Options

reindexdb accepts the following command-line arguments:

-a; --all
Reindex all databases.

--concurrently
Use the CONCURRENTLY option. See ???, where all the caveats of this option are explained in detail.

-d dbname; --dbname=dbname
Specifies the name of the database to be reindexed, when -a/--all is not used. If this is not specified, the database name is read from the environment variable PGDATABASE. If that is not set, the user name specified for the connection is used. The <dbname> can be a connection string. If so, connection string parameters will override any conflicting command line options.

-e; --echo
Echo the commands that reindexdb generates and sends to the server.

-i index; --index=index
Recreate <index> only. Multiple indexes can be recreated by writing multiple -i switches.

-j njobs; --jobs=njobs
Execute the reindex commands in parallel by running <njobs> commands simultaneously. This option may reduce the processing time but it also increases the load on the database server.

reindexdb will open <njobs> connections to the database, so make sure your ??? setting is high enough to accommodate all connections.

Note that this option is incompatible with the --system option.

-q; --quiet
Do not display progress messages.

-s; --system
Reindex database's system catalogs only.

-S schema; --schema=schema
Reindex <schema> only. Multiple schemas can be reindexed by writing multiple -S switches.

-t table; --table=table
Reindex <table> only. Multiple tables can be reindexed by writing multiple -t switches.

--tablespace=tablespace
Specifies the tablespace where indexes are rebuilt. (This name is processed as a double-quoted identifier.)

-v; --verbose
Print detailed information during processing.

-V; --version
Print the reindexdb version and exit.

-?; --help
Show help about reindexdb command line arguments, and exit.

reindexdb also accepts the following command-line arguments for connection parameters:

-h host; --host=host
Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for the Unix domain socket.

-p port; --port=port
Specifies the TCP port or local Unix domain socket file extension on which the server is listening for connections.

-U username; --username=username
User name to connect as.

-w; --no-password
Never issue a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password.

-W; --password
Force reindexdb to prompt for a password before connecting to a database.

This option is never essential, since reindexdb will automatically prompt for a password if the server demands password authentication. However, reindexdb will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt.

--maintenance-db=dbname
When the -a/--all is used, connect to this database to gather the list of databases to reindex. If not specified, the postgres database will be used, or if that does not exist, template1 will be used. This can be a connection string. If so, connection string parameters will override any conflicting command line options. Also, connection string parameters other than the database name itself will be re-used when connecting to other databases.

Environment

PGDATABASE; PGHOST; PGPORT; PGUSER
Default connection parameters

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

This utility, like most other PostgreSQL utilities, also uses the environment variables supported by libpq (see ???).

Diagnostics

In case of difficulty, see ??? and ??? for discussions of potential problems and error messages. The database server must be running at the targeted host. Also, any default connection settings and environment variables used by the libpq front-end library will apply.

Examples

To reindex the database test:

$ reindexdb test

To reindex the table foo and the index bar in a database named abcd:

$ reindexdb --table=foo --index=bar abcd

See Also

release_savepoint

RELEASE SAVEPOINT

savepoints

releasing

RELEASE SAVEPOINT

7

SQL - Language Statements

RELEASE SAVEPOINT

release a previously defined savepoint

RELEASE [ SAVEPOINT ]

savepoint_name

Description

RELEASE SAVEPOINT releases the named savepoint and all active savepoints that were created after the named savepoint, and frees their resources. All changes made since the creation of the savepoint that didn't already get rolled back are merged into the transaction or savepoint that was active when the named savepoint was created. Changes made after RELEASE SAVEPOINT will also be part of this active transaction or savepoint.

Parameters

<savepoint_name>
The name of the savepoint to release.

Notes

Specifying a savepoint name that was not previously defined is an error.

It is not possible to release a savepoint when the transaction is in an aborted state; to do that, use ???.

If multiple savepoints have the same name, only the most recently defined unreleased one is released. Repeated commands will release progressively older savepoints.

Examples

To establish and later release a savepoint:

BEGIN;
    INSERT INTO table1 VALUES (3);
    SAVEPOINT my_savepoint;
    INSERT INTO table1 VALUES (4);
    RELEASE SAVEPOINT my_savepoint;
COMMIT;

The above transaction will insert both 3 and 4.

A more complex example with multiple nested subtransactions:

BEGIN;
    INSERT INTO table1 VALUES (1);
    SAVEPOINT sp1;
    INSERT INTO table1 VALUES (2);
    SAVEPOINT sp2;
    INSERT INTO table1 VALUES (3);
    RELEASE SAVEPOINT sp2;
    INSERT INTO table1 VALUES (4))); -- generates an error

In this example, the application requests the release of the savepoint sp2, which inserted 3. This changes the insert's transaction context to sp1. When the statement attempting to insert value 4 generates an error, the insertion of 2 and 4 are lost because they are in the same, now-rolled back savepoint, and value 3 is in the same transaction context. The application can now only choose one of these two commands, since all other commands will be ignored:

ROLLBACK;
ROLLBACK TO SAVEPOINT sp1;

Choosing ROLLBACK will abort everything, including value 1, whereas ROLLBACK TO SAVEPOINT sp1 will retain value 1 and allow the transaction to continue.

Compatibility

This command conforms to the SQL standard. The standard specifies that the key word SAVEPOINT is mandatory, but PostgreSQL allows it to be omitted.

See Also

repack

REPACK

REPACK

7

SQL - Language Statements

REPACK

rewrite a table to reclaim disk space

REPACK [ (

option

[, ...] ) ] [

table_and_columns

[ USING INDEX [

index_name

] ] ] REPACK [ (

option

[, ...] ) ] USING INDEX

where

option

can be one of:

VERBOSE [

boolean

] ANALYZE [

boolean

] CONCURRENTLY [

boolean

]

and

table_and_columns

is:

table_name

[ (

column_name

[, ...] ) ]

Description

REPACK reclaims storage occupied by dead tuples. Unlike VACUUM, it does so by rewriting the entire contents of the table specified by <table_name> into a new disk file with no extra space (except for the space guaranteed by the fillfactor storage parameter), allowing unused space to be returned to the operating system.

Without a <table_name>, REPACK processes every table and materialized view in the current database that the current user has the MAINTAIN privilege on. This form of REPACK cannot be executed inside a transaction block. Also, this form is not allowed if the CONCURRENTLY option is used.

If a USING INDEX clause is specified, the rows are physically reordered based on information from an index. Please see Notes on Clustering below.

When a table is being repacked, an ACCESS EXCLUSIVE lock is acquired on it, unless the CONCURRENTLY option is given. See Notes on Concurrent Operation for a discussion on the effects of this option.

Parameters

<table_name>
The name (possibly schema-qualified) of a table.

<column_name>
The name of a specific column to analyze. Defaults to all columns. If a column list is specific, ANALYZE must also be specified.

<index_name>
The name of an index.

CONCURRENTLY
Allow other transactions to use the table while it is being repacked. See Notes on Concurrent Operation for more details.

VERBOSE
Prints a progress report as each table is repacked at INFO level.

ANALYZE; ANALYSE
Applies ??? on the table after repacking. This is currently only supported when a single (non-partitioned) table is specified. This option cannot be used inside a transaction block, or from a function, procedure, or DO block.

<boolean>
Specifies whether the selected option should be turned on or off. You can write TRUE, ON, or 1 to enable the option, and FALSE, OFF, or 0 to disable it. The <boolean> value can also be omitted, in which case TRUE is assumed.

Notes

To repack a table, one must have the MAINTAIN privilege on the table.

REPACK is primarily meant to remove bloat and, with USING INDEX, to cluster the table, in order to improve performance. Although it also advances the table's relfrozenxid and relminmxid, it is not a good way to prevent transaction ID or multixact ID wraparound: it rewrites the whole table and all of its indexes, so it takes much longer than VACUUM, and it can fail after most of the work is done, especially with CONCURRENTLY on a busy table. VACUUM is recommended for that instead (see ???). When a table gets close to wraparound, VACUUM skips index vacuuming on its own (see ???), and its INDEX_CLEANUP OFF option can do the same earlier.

REPACK refuses to process a table on which an invalid index exists. Such indexes must be dropped or reindexed by the user ahead of time.

While REPACK is running, the ??? is temporarily changed to pg_catalog, pg_temp.

Each backend running REPACK will report its progress in the pg_stat_progress_repack view. See ??? for details.

Repacking a partitioned table repacks each of its partitions. If an index is specified, each partition is repacked using the partition of that index. Partitions that are foreign tables are skipped. REPACK on a partitioned table cannot be executed inside a transaction block.

Notes on Clustering

If the USING INDEX clause is specified, the rows in the table are physically rearranged according to the ordering implied by the specified index; this is known as clustering. This can have performance implications: in cases where you are accessing single rows randomly within a table, the actual order of the data in the table is unimportant. However, if you tend to access some data more than others, and there is an index that groups them together, you will benefit from using clustering. If you are requesting a range of indexed values from a table, or a single indexed value that has multiple matching rows, clustering will help because once the index identifies the table page for the first row that matches, all other rows that match are probably already on the same table page, and so you save disk accesses and speed up the query.

For B-Tree indexes, the ordering used by clustering is the index's linear sort order. Other clusterable index access methods may use a different ordering strategy, one which may not necessarily correspond to any SQL sort order. If an index name is specified in the command, that index is used and is recorded as the table's clustering index. (This also applies to an index given to the CLUSTER command.) If no index name is specified, then the index that has been configured as the clustering one is used; if none has been configured, an error is thrown. An index can be set manually using ALTER TABLE ... CLUSTER ON, and reset with ALTER TABLE ... SET WITHOUT CLUSTER.

Clustering is a one-time operation: when the table is subsequently updated, the changes are not clustered. That is, no attempt is made to store new or updated rows according to the clustering order. (If one wishes, one can periodically recluster by issuing the command again. Also, setting the table's fillfactor storage parameter to less than 100% can aid in preserving cluster ordering during updates, since updated rows are kept on the same page if enough space is available there.)

When clustering on a B-Tree index, REPACK can rewrite the table using either an index scan on the specified index, or a sequential scan followed by sorting. It will attempt to choose the method that will be faster, based on planner cost parameters and available statistical information. When clustering on an index of an access method other than B-Tree, REPACK always uses an index scan.

Because the planner records statistics about the ordering of tables, it is advisable to specify the ANALYZE option, or to run ANALYZE on the newly repacked table. Otherwise, the planner might make poor choices of query plans.

If no table name is specified in REPACK USING INDEX, all tables which have a clustering index defined and which the calling user has privileges for are processed.

Notes on Resources

REPACK creates a temporary copy of the table and all of its indexes, so you need free disk space at least equal to the sum of the table size and the index sizes.

If the USING INDEX clause is given and the planner chooses a sequential scan and sort to implement it, a temporary sort file of up to the size of the table is needed in addition to the above. This method is usually faster than an index scan, but if the disk space requirement is intolerable, you can disable this choice by temporarily setting ??? to off; or omitting USING INDEX if the table doesn't need to be reordered.

For the purposes of transaction ID wraparound (see ???), REPACK is considered a single long-running transaction, which prevents VACUUM from cleaning up dead rows from other tables. It is advisable to monitor pg_stat_activity.backend_xid for the process running REPACK when repacking very large tables, to avoid causing excessive bloat in other tables.

It is advisable to set ??? to a reasonably large value (but not more than the amount of RAM you can dedicate to the REPACK operation) before repacking.

Notes on Concurrent Operation

[!WARNING] REPACK with the CONCURRENTLY option is not MVCC-safe. After the repacking transaction commits, the table will appear empty to concurrent transactions, if they are using a snapshot taken before repacking started. See ??? for more details.

REPACK copies the contents of the table (ignoring dead tuples) into a new file, sorted by the specified index, and also creates a new file for each index. It then swaps the old and new files for the table and all the indexes, and deletes the old files. Without the CONCURRENTLY option, an ACCESS EXCLUSIVE lock is acquired at the beginning and remains held throughout the operation to make sure that the old files do not change during the processing; otherwise, any concurrent changes would get lost due to the swap.

By contrast, when the CONCURRENTLY option is specified, the bulk of the operation is run with SHARE UPDATE EXCLUSIVE lock, and the ACCESS EXCLUSIVE lock is only acquired during the final phase to swap the table and index files. The data changes that took place during the creation of the new table and index files are captured using logical decoding (see ???) and applied before the ACCESS EXCLUSIVE lock is requested. Once that lock is obtained, a final pass over any remaining captured concurrent data changes is done and the files are swapped. Thus the lock is typically held for a short time, depending on the amount of changes accumulated while the lock was being waited for.

Processing of these concurrent changes requires a fixed small amount of memory for each tuple concurrently updated or deleted, not limited by ???; if more than about 104 million rows are concurrently updated or deleted during the execution of REPACK, the command fails. In addition, the complete tuple data for all concurrent changes (except for TOAST columns that are not modified by UPDATE) are stored on disk separately.

REPACK (CONCURRENTLY) might fail to complete if DDL commands are executed on the table by other transactions during the repacking.

REPACK (CONCURRENTLY) USING INDEX does not try to order the rows inserted into the table after the repacking started.

The CONCURRENTLY option cannot be used in the following cases:

Examples

Repack the table employees:

REPACK employees;

Repack the table employees on the basis of its index employees_ind (since an index is specified, this is effectively clustering):

REPACK employees USING INDEX employees_ind;

Repack the employees table following the same index as was used before, in concurrent mode:

REPACK (CONCURRENTLY) employees USING INDEX;

Repack the table cases on physical ordering, running an ANALYZE on the given columns once repacking is done, showing informational messages:

REPACK (ANALYZE, VERBOSE) cases (district, case_nr);

Repack all tables in the database on which you have the MAINTAIN privilege:

REPACK;

Repack all tables for which a clustering index has previously been configured on which you have the MAINTAIN privilege, showing informational messages:

REPACK (VERBOSE) USING INDEX;

Compatibility

There is no REPACK statement in the SQL standard.

See Also

reset

RESET

RESET

7

SQL - Language Statements

RESET

restore the value of a run-time parameter to the default value

RESET

configuration_parameter

RESET ALL

Description

RESET restores run-time parameters to their default values. RESET is an alternative spelling for SET <configuration_parameter> TO DEFAULT Refer to ??? for details.

The default value is defined as the value that the parameter would have had, if no SET had ever been issued for it in the current session. The actual source of this value might be a compiled-in default, the configuration file, command-line options, or per-database or per-user default settings. This is subtly different from defining it as “the value that the parameter had at session start”, because if the value came from the configuration file, it will be reset to whatever is specified by the configuration file now. See ??? for details.

The transactional behavior of RESET is the same as SET: its effects will be undone by transaction rollback.

Parameters

<configuration_parameter>
Name of a settable run-time parameter. Available parameters are documented in ??? and on the ??? reference page.

ALL
Resets all settable run-time parameters to default values.

Examples

Set the timezone configuration variable to its default value:

RESET timezone;

Compatibility

RESET is a PostgreSQL extension.

See Also

revoke

REVOKE

REVOKE

7

SQL - Language Statements

REVOKE

remove access privileges

REVOKE [ GRANT OPTION FOR ] { { SELECT | INSERT | UPDATE | DELETE | TRUNCATE | REFERENCES | TRIGGER | MAINTAIN } [, ...] | ALL [ PRIVILEGES ] } ON { [ TABLE ]

table_name

[, ...] | ALL TABLES IN SCHEMA

schema_name

[, ...] } FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { { SELECT | INSERT | UPDATE | REFERENCES } (

column_name

[, ...] ) [, ...] | ALL [ PRIVILEGES ] (

column_name

[, ...] ) } ON [ TABLE ]

table_name

[, ...] FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { { USAGE | SELECT | UPDATE } [, ...] | ALL [ PRIVILEGES ] } ON { SEQUENCE

sequence_name

[, ...] | ALL SEQUENCES IN SCHEMA

schema_name

[, ...] } FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { { CREATE | CONNECT | TEMPORARY | TEMP } [, ...] | ALL [ PRIVILEGES ] } ON DATABASE

database_name

[, ...] FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { USAGE | ALL [ PRIVILEGES ] } ON DOMAIN

domain_name

[, ...] FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { USAGE | ALL [ PRIVILEGES ] } ON FOREIGN DATA WRAPPER

fdw_name

[, ...] FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { USAGE | ALL [ PRIVILEGES ] } ON FOREIGN SERVER

server_name

[, ...] FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { EXECUTE | ALL [ PRIVILEGES ] } ON { { FUNCTION | PROCEDURE | ROUTINE }

function_name

[ ( [ [

argmode

] [

arg_name

]

arg_type

[, ...] ] ) ] [, ...] | ALL { FUNCTIONS | PROCEDURES | ROUTINES } IN SCHEMA

schema_name

[, ...] } FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { USAGE | ALL [ PRIVILEGES ] } ON LANGUAGE

lang_name

[, ...] FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { { SELECT | UPDATE } [, ...] | ALL [ PRIVILEGES ] } ON LARGE OBJECT

loid

[, ...] FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { { SET | ALTER SYSTEM } [, ...] | ALL [ PRIVILEGES ] } ON PARAMETER

configuration_parameter

[, ...] FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { { CREATE | USAGE } [, ...] | ALL [ PRIVILEGES ] } ON SCHEMA

schema_name

[, ...] FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { CREATE | ALL [ PRIVILEGES ] } ON TABLESPACE

tablespace_name

[, ...] FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ] REVOKE [ GRANT OPTION FOR ] { USAGE | ALL [ PRIVILEGES ] } ON TYPE

type_name

[, ...] FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ] REVOKE [ { ADMIN | INHERIT | SET } OPTION FOR ]

role_name

[, ...] FROM

role_specification

[, ...] [ GRANTED BY

role_specification

] [ CASCADE | RESTRICT ]

where

role_specification

can be:

[ GROUP ]

role_name

| PUBLIC | CURRENT_ROLE | CURRENT_USER | SESSION_USER

Description

The REVOKE command revokes previously granted privileges from one or more roles. The key word PUBLIC refers to the implicitly defined group of all roles.

See the description of the GRANT command for the meaning of the privilege types.

Note that any particular role will have the sum of privileges granted directly to it, privileges granted to any role it is presently a member of, and privileges granted to PUBLIC. Thus, for example, revoking SELECT privilege from PUBLIC does not necessarily mean that all roles have lost SELECT privilege on the object: those who have it granted directly or via another role will still have it. Similarly, revoking SELECT from a user might not prevent that user from using SELECT if PUBLIC or another membership role still has SELECT rights.

If GRANT OPTION FOR is specified, only the grant option for the privilege is revoked, not the privilege itself. Otherwise, both the privilege and the grant option are revoked.

If GRANTED BY is specified, only privileges granted by the specified role are revoked. A role can only revoke grants by another role if it inherits the privileges of that role.

If a user holds a privilege with grant option and has granted it to other users then the privileges held by those other users are called dependent privileges. If the privilege or the grant option held by the first user is being revoked and dependent privileges exist, those dependent privileges are also revoked if CASCADE is specified; if it is not, the revoke action will fail. This recursive revocation only affects privileges that were granted through a chain of users that is traceable to the user that is the subject of this REVOKE command. Thus, the affected users might effectively keep the privilege if it was also granted through other users.

When revoking privileges on a table, the corresponding column privileges (if any) are automatically revoked on each column of the table, as well. On the other hand, if a role has been granted privileges on a table, then revoking the same privileges from individual columns will have no effect.

When revoking membership in a role, GRANT OPTION is instead called ADMIN OPTION, but the behavior is similar. Note that, in releases prior to PostgreSQL 16, dependent privileges were not tracked for grants of role membership, and thus CASCADE had no effect for role membership. This is no longer the case. Note also that this form of the command does not allow the noise word GROUP in <role_specification>.

Just as ADMIN OPTION can be removed from an existing role grant, it is also possible to revoke INHERIT OPTION or SET OPTION. This is equivalent to setting the value of the corresponding option to FALSE.

Notes

A user can only revoke privileges that were granted directly by that user. If, for example, user A has granted a privilege with grant option to user B, and user B has in turn granted it to user C, then user A cannot revoke the privilege directly from C. Instead, user A could revoke the grant option from user B and use the CASCADE option so that the privilege is in turn revoked from user C. For another example, if both A and B have granted the same privilege to C, A can revoke their own grant but not B's grant, so C will still effectively have the privilege.

When a non-owner of an object attempts to REVOKE privileges on the object, the command will fail outright if the user has no privileges whatsoever on the object. As long as some privilege is available, the command will proceed, but it will revoke only those privileges for which the user has grant options. The REVOKE ALL PRIVILEGES forms will issue a warning message if no grant options are held, while the other forms will issue a warning if grant options for any of the privileges specifically named in the command are not held. (In principle these statements apply to the object owner as well, but since the owner is always treated as holding all grant options, the cases can never occur.)

If a superuser chooses to issue a GRANT or REVOKE command, the command is performed as though it were issued by the owner of the affected object. (Since roles do not have owners, in the case of a GRANT of role membership, the command is performed as though it were issued by the bootstrap superuser.) Since all privileges ultimately come from the object owner (possibly indirectly via chains of grant options), it is possible for a superuser to revoke all privileges, but this might require use of CASCADE as stated above.

REVOKE can also be done by a role that is not the owner of the affected object, but is a member of the role that owns the object, or is a member of a role that holds privileges WITH GRANT OPTION on the object. In this case the command is performed as though it were issued by the containing role that actually owns the object or holds the privileges WITH GRANT OPTION. For example, if table t1 is owned by role g1, of which role u1 is a member, then u1 can revoke privileges on t1 that are recorded as being granted by g1. This would include grants made by u1 as well as by other members of role g1.

If the role executing REVOKE holds privileges indirectly via more than one role membership path, it is unspecified which containing role will be used to perform the command. In such cases it is best practice to use GRANTED BY to specify which role you want to do the REVOKE as. Failure to do so might lead to revoking privileges other than the ones you intended, or not revoking anything at all.

See ??? for more information about specific privilege types, as well as how to inspect objects' privileges.

Examples

Revoke insert privilege for the public on table films:

REVOKE INSERT ON films FROM PUBLIC;

Revoke all privileges from user manuel on view kinds:

REVOKE ALL PRIVILEGES ON kinds FROM manuel;

Note that this actually means “revoke all privileges that I granted”.

Revoke membership in role admins from user joe:

REVOKE admins FROM joe;

Compatibility

The compatibility notes of the GRANT command apply analogously to REVOKE. The keyword RESTRICT or CASCADE is required according to the standard, but PostgreSQL assumes RESTRICT by default.

See Also

rollback

ROLLBACK

ROLLBACK

7

SQL - Language Statements

ROLLBACK

abort the current transaction

ROLLBACK [ WORK | TRANSACTION ] [ AND [ NO ] CHAIN ]

Description

ROLLBACK rolls back the current transaction and causes all the updates made by the transaction to be discarded.

Parameters

chained transactions

WORK; TRANSACTION
Optional key words. They have no effect.

AND CHAIN
If AND CHAIN is specified, a new (not aborted) transaction is immediately started with the same transaction characteristics (see ???) as the just finished one. Otherwise, no new transaction is started.

Notes

Use COMMIT to successfully terminate a transaction.

Issuing ROLLBACK outside of a transaction block emits a warning and otherwise has no effect. ROLLBACK AND CHAIN outside of a transaction block is an error.

Examples

To abort all changes:

ROLLBACK;

Compatibility

The command ROLLBACK conforms to the SQL standard. The form ROLLBACK TRANSACTION is a PostgreSQL extension.

See Also

rollback_prepared

ROLLBACK PREPARED

ROLLBACK PREPARED

7

SQL - Language Statements

ROLLBACK PREPARED

cancel a transaction that was earlier prepared for two-phase commit

ROLLBACK PREPARED

transaction_id

Description

ROLLBACK PREPARED rolls back a transaction that is in prepared state.

Parameters

<transaction_id>
The transaction identifier of the transaction that is to be rolled back.

Notes

To roll back a prepared transaction, you must be either the same user that executed the transaction originally, or a superuser. But you do not have to be in the same session that executed the transaction.

This command cannot be executed inside a transaction block. The prepared transaction is rolled back immediately.

All currently available prepared transactions are listed in the pg_prepared_xacts system view.

Examples

Roll back the transaction identified by the transaction identifier foobar:

ROLLBACK PREPARED 'foobar';

Compatibility

ROLLBACK PREPARED is a PostgreSQL extension. It is intended for use by external transaction management systems, some of which are covered by standards (such as X/Open XA), but the SQL side of those systems is not standardized.

See Also

rollback_to

ROLLBACK TO SAVEPOINT

savepoints

rolling back

ROLLBACK TO SAVEPOINT

7

SQL - Language Statements

ROLLBACK TO SAVEPOINT

roll back to a savepoint

ROLLBACK [ WORK | TRANSACTION ] TO [ SAVEPOINT ]

savepoint_name

Description

Roll back all commands that were executed after the savepoint was established and then start a new subtransaction at the same transaction level. The savepoint remains valid and can be rolled back to again later, if needed.

ROLLBACK TO SAVEPOINT implicitly destroys all savepoints that were established after the named savepoint.

Parameters

<savepoint_name>
The savepoint to roll back to.

Notes

Use RELEASE SAVEPOINT to destroy a savepoint without discarding the effects of commands executed after it was established.

Specifying a savepoint name that has not been established is an error.

Cursors have somewhat non-transactional behavior with respect to savepoints. Any cursor that is opened inside a savepoint will be closed when the savepoint is rolled back. If a previously opened cursor is affected by a FETCH or MOVE command inside a savepoint that is later rolled back, the cursor remains at the position that FETCH left it pointing to (that is, the cursor motion caused by FETCH is not rolled back). Closing a cursor is not undone by rolling back, either. However, other side-effects caused by the cursor's query (such as side-effects of volatile functions called by the query) are rolled back if they occur during a savepoint that is later rolled back. A cursor whose execution causes a transaction to abort is put in a cannot-execute state, so while the transaction can be restored using ROLLBACK TO SAVEPOINT, the cursor can no longer be used.

Examples

To undo the effects of the commands executed after my_savepoint was established:

ROLLBACK TO SAVEPOINT my_savepoint;

Cursor positions are not affected by savepoint rollback:

BEGIN;

DECLARE foo CURSOR FOR SELECT 1 UNION SELECT 2;

SAVEPOINT foo;

FETCH 1 FROM foo;
 ?column?
----------
        1

ROLLBACK TO SAVEPOINT foo;

FETCH 1 FROM foo;
 ?column?
----------
        2

COMMIT;

Compatibility

The SQL standard specifies that the key word SAVEPOINT is mandatory, but PostgreSQL and Oracle allow it to be omitted. SQL allows only WORK, not TRANSACTION, as a noise word after ROLLBACK. Also, SQL has an optional clause AND [ NO ] CHAIN which is not currently supported by PostgreSQL. Otherwise, this command conforms to the SQL standard.

See Also

savepoint

SAVEPOINT

savepoints

defining

SAVEPOINT

7

SQL - Language Statements

SAVEPOINT

define a new savepoint within the current transaction

SAVEPOINT

savepoint_name

Description

SAVEPOINT establishes a new savepoint within the current transaction.

A savepoint is a special mark inside a transaction that allows all commands that are executed after it was established to be rolled back, restoring the transaction state to what it was at the time of the savepoint.

Parameters

<savepoint_name>
The name to give to the new savepoint. If savepoints with the same name already exist, they will be inaccessible until newer identically-named savepoints are released.

Notes

Use ROLLBACK TO to rollback to a savepoint. Use RELEASE SAVEPOINT to destroy a savepoint, keeping the effects of commands executed after it was established.

Savepoints can only be established when inside a transaction block. There can be multiple savepoints defined within a transaction.

Examples

To establish a savepoint and later undo the effects of all commands executed after it was established:

BEGIN;
    INSERT INTO table1 VALUES (1);
    SAVEPOINT my_savepoint;
    INSERT INTO table1 VALUES (2);
    ROLLBACK TO SAVEPOINT my_savepoint;
    INSERT INTO table1 VALUES (3);
COMMIT;

The above transaction will insert the values 1 and 3, but not 2.

To establish and later destroy a savepoint:

BEGIN;
    INSERT INTO table1 VALUES (3);
    SAVEPOINT my_savepoint;
    INSERT INTO table1 VALUES (4);
    RELEASE SAVEPOINT my_savepoint;
COMMIT;

The above transaction will insert both 3 and 4.

To use a single savepoint name:

BEGIN;
    INSERT INTO table1 VALUES (1);
    SAVEPOINT my_savepoint;
    INSERT INTO table1 VALUES (2);
    SAVEPOINT my_savepoint;
    INSERT INTO table1 VALUES (3);

    -- rollback to the second savepoint
    ROLLBACK TO SAVEPOINT my_savepoint;
    SELECT * FROM table1;               -- shows rows 1 and 2

    -- release the second savepoint
    RELEASE SAVEPOINT my_savepoint;

    -- rollback to the first savepoint
    ROLLBACK TO SAVEPOINT my_savepoint;
    SELECT * FROM table1;               -- shows only row 1
COMMIT;

The above transaction shows row 3 being rolled back first, then row 2.

Compatibility

SQL requires a savepoint to be destroyed automatically when another savepoint with the same name is established. In PostgreSQL, the old savepoint is kept, though only the more recent one will be used when rolling back or releasing. (Releasing the newer savepoint with RELEASE SAVEPOINT will cause the older one to again become accessible to ROLLBACK TO SAVEPOINT and RELEASE SAVEPOINT.) Otherwise, SAVEPOINT is fully SQL conforming.

See Also

security_label

SECURITY LABEL

SECURITY LABEL

7

SQL - Language Statements

SECURITY LABEL

define or change a security label applied to an object

SECURITY LABEL [ FOR

provider

] ON { TABLE

object_name

| COLUMN

table_name

.

column_name

| AGGREGATE

aggregate_name

(

aggregate_signature

) | DATABASE

object_name

| DOMAIN

object_name

| EVENT TRIGGER

object_name

| FOREIGN TABLE

object_name

| FUNCTION

function_name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] | LARGE OBJECT

large_object_oid

| MATERIALIZED VIEW

object_name

| [ PROCEDURAL ] LANGUAGE

object_name

| PROCEDURE

procedure_name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] | PUBLICATION

object_name

| ROLE

object_name

| ROUTINE

routine_name

[ ( [ [

argmode

] [

argname

]

argtype

[, ...] ] ) ] | SCHEMA

object_name

| SEQUENCE

object_name

| SUBSCRIPTION

object_name

| TABLESPACE

object_name

| TYPE

object_name

| VIEW

object_name

} IS {

string_literal

| NULL }

where

aggregate_signature

is:

* | [

argmode

] [

argname

]

argtype

[ , ... ] | [ [

argmode

] [

argname

]

argtype

[ , ... ] ] ORDER BY [

argmode

] [

argname

]

argtype

[ , ... ]

Description

SECURITY LABEL applies a security label to a database object. An arbitrary number of security labels, one per label provider, can be associated with a given database object. Label providers are loadable modules which register themselves by using the function register_label_provider.

[!NOTE] register_label_provider is not an SQL function; it can only be called from C code loaded into the backend.

The label provider determines whether a given label is valid and whether it is permissible to assign that label to a given object. The meaning of a given label is likewise at the discretion of the label provider. PostgreSQL places no restrictions on whether or how a label provider must interpret security labels; it merely provides a mechanism for storing them. In practice, this facility is intended to allow integration with label-based mandatory access control (MAC) systems such as SELinux. Such systems make all access control decisions based on object labels, rather than traditional discretionary access control (DAC) concepts such as users and groups.

You must own the database object to use SECURITY LABEL.

Parameters

<object_name>; <table_name.column_name>; <aggregate_name>; <function_name>; <procedure_name>; <routine_name>
The name of the object to be labeled. Names of objects that reside in schemas (tables, functions, etc.) can be schema-qualified.

<provider>
The name of the provider with which this label is to be associated. The named provider must be loaded and must consent to the proposed labeling operation. If exactly one provider is loaded, the provider name may be omitted for brevity.

<argmode>
The mode of a function, procedure, or aggregate argument: IN, OUT, INOUT, or VARIADIC. If omitted, the default is IN. Note that SECURITY LABEL does not actually pay any attention to OUT arguments, since only the input arguments are needed to determine the function's identity. So it is sufficient to list the IN, INOUT, and VARIADIC arguments.

<argname>
The name of a function, procedure, or aggregate argument. Note that SECURITY LABEL does not actually pay any attention to argument names, since only the argument data types are needed to determine the function's identity.

<argtype>
The data type of a function, procedure, or aggregate argument.

<large_object_oid>
The OID of the large object.

PROCEDURAL
This is a noise word.

<string_literal>
The new setting of the security label, written as a string literal.

NULL
Write NULL to drop the security label.

Examples

The following example shows how the security label of a table could be set or changed:

SECURITY LABEL FOR selinux ON TABLE mytable IS 'system_u:object_r:sepgsql_table_t:s0';

To remove the label:

SECURITY LABEL FOR selinux ON TABLE mytable IS NULL;

Compatibility

There is no SECURITY LABEL command in the SQL standard.

See Also

src/test/modules/dummy_seclabel

select

SELECT

TABLE command

WITH

in SELECT

SELECT

7

SQL - Language Statements

SELECT

TABLE

WITH

retrieve rows from a table or view

[ WITH [ RECURSIVE ]

with_query

[, ...] ] SELECT [ ALL | DISTINCT [ ON (

expression

[, ...] ) ] ] [ { * |

expression

[ [ AS ]

output_name

] } [, ...] ] [ FROM

from_item

[, ...] ] [ WHERE

condition

] [ GROUP BY [ ALL | DISTINCT ]

grouping_element

[, ...] ] [ HAVING

condition

] [ WINDOW

window_name

AS (

window_definition

) [, ...] ] [ { UNION | INTERSECT | EXCEPT } [ ALL | DISTINCT ]

select

] [ ORDER BY

expression

[ ASC | DESC | USING

operator

] [ NULLS { FIRST | LAST } ] [, ...] ] [ LIMIT {

count

| ALL } ] [ OFFSET

start

[ ROW | ROWS ] ] [ FETCH { FIRST | NEXT } [

count

] { ROW | ROWS } { ONLY | WITH TIES } ] [ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF

from_reference

[, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]

where

from_item

can be one of:

[ ONLY ]

table_name

[ * ] [ [ AS ]

alias

[ (

column_alias

[, ...] ) ] ] [ TABLESAMPLE

sampling_method

(

argument

[, ...] ) [ REPEATABLE (

seed

) ] ] [ LATERAL ] (

select

) [ [ AS ]

alias

[ (

column_alias

[, ...] ) ] ]

with_query_name

[ [ AS ]

alias

[ (

column_alias

[, ...] ) ] ] [ LATERAL ]

function_name

( [

argument

[, ...] ] ) [ WITH ORDINALITY ] [ [ AS ]

alias

[ (

column_alias

[, ...] ) ] ] [ LATERAL ]

function_name

( [

argument

[, ...] ] ) [ AS ]

alias

(

column_definition

[, ...] ) [ LATERAL ]

function_name

( [

argument

[, ...] ] ) AS (

column_definition

[, ...] ) [ LATERAL ] ROWS FROM(

function_name

( [

argument

[, ...] ] ) [ AS (

column_definition

[, ...] ) ] [, ...] ) [ WITH ORDINALITY ] [ [ AS ]

alias

[ (

column_alias

[, ...] ) ] ]

from_item

join_type

from_item

{ ON

join_condition

| USING (

join_column

[, ...] ) [ AS

join_using_alias

] }

from_item

NATURAL

join_type

from_item

from_item

CROSS JOIN

from_item

and

grouping_element

can be one of:

( )

expression

(

expression

[, ...] ) ROLLUP ( {

expression

| (

expression

[, ...] ) } [, ...] ) CUBE ( {

expression

| (

expression

[, ...] ) } [, ...] ) GROUPING SETS (

grouping_element

[, ...] )

and

with_query

is:

with_query_name

[ (

column_name

[, ...] ) ] AS [ [ NOT ] MATERIALIZED ] (

select

|

values

|

insert

|

update

|

delete

|

merge

) [ SEARCH { BREADTH | DEPTH } FIRST BY

column_name

[, ...] SET

search_seq_col_name

] [ CYCLE

column_name

[, ...] SET

cycle_mark_col_name

[ TO

cycle_mark_value

DEFAULT

cycle_mark_default

] USING

cycle_path_col_name

] TABLE [ ONLY ]

table_name

[ * ]

Description

SELECT retrieves rows from zero or more tables. The general processing of SELECT is as follows:

  1. All queries in the WITH list are computed. These effectively serve as temporary tables that can be referenced in the FROM list. A WITH query that is referenced more than once in FROM is computed only once, unless specified otherwise with NOT MATERIALIZED. (See WITH Clause below.)

  2. All elements in the FROM list are computed. (Each element in the FROM list is a real or virtual table.) If more than one element is specified in the FROM list, they are cross-joined together. (See FROM Clause below.)

  3. If the WHERE clause is specified, all rows that do not satisfy the condition are eliminated from the output. (See WHERE Clause below.)

  4. If the GROUP BY clause is specified, or if there are aggregate function calls, the output is combined into groups of rows that match on one or more values, and the results of aggregate functions are computed. If the HAVING clause is present, it eliminates groups that do not satisfy the given condition. (See GROUP BY Clause and HAVING Clause below.) Although query output columns are nominally computed in the next step, they can also be referenced (by name or ordinal number) in the GROUP BY clause.

  5. The actual output rows are computed using the SELECT output expressions for each selected row or row group. (See SELECT List below.)

  6. SELECT DISTINCT eliminates duplicate rows from the result. SELECT DISTINCT ON eliminates rows that match on all the specified expressions. SELECT ALL (the default) will return all candidate rows, including duplicates. (See DISTINCT Clause below.)

  7. Using the operators UNION, INTERSECT, and EXCEPT, the output of more than one SELECT statement can be combined to form a single result set. The UNION operator returns all rows that are in one or both of the result sets. The INTERSECT operator returns all rows that are strictly in both result sets. The EXCEPT operator returns the rows that are in the first result set but not in the second. In all three cases, duplicate rows are eliminated unless ALL is specified. The noise word DISTINCT can be added to explicitly specify eliminating duplicate rows. Notice that DISTINCT is the default behavior here, even though ALL is the default for SELECT itself. (See UNION Clause, INTERSECT Clause, and EXCEPT Clause below.)

  8. If the ORDER BY clause is specified, the returned rows are sorted in the specified order. If ORDER BY is not given, the rows are returned in whatever order the system finds fastest to produce. (See ORDER BY Clause below.)

  9. If the LIMIT (or FETCH FIRST) or OFFSET clause is specified, the SELECT statement only returns a subset of the result rows. (See LIMIT Clause below.)

  10. If FOR UPDATE, FOR NO KEY UPDATE, FOR SHARE or FOR KEY SHARE is specified, the SELECT statement locks the selected rows against concurrent updates. (See The Locking Clause below.)

You must have SELECT privilege on each column used in a SELECT command. The use of FOR NO KEY UPDATE, FOR UPDATE, FOR SHARE or FOR KEY SHARE requires UPDATE privilege as well (for at least one column of each table so selected).

Parameters

WITH Clause

The WITH clause allows you to specify one or more subqueries that can be referenced by name in the primary query. The subqueries effectively act as temporary tables or views for the duration of the primary query. Each subquery can be a SELECT, TABLE, VALUES, INSERT, UPDATE, DELETE, or MERGE statement. When writing a data-modifying statement (INSERT, UPDATE, DELETE, or MERGE) in WITH, it is usual to include a RETURNING clause. It is the output of RETURNING, not the underlying table that the statement modifies, that forms the temporary table that is read by the primary query. If RETURNING is omitted, the statement is still executed, but it produces no output so it cannot be referenced as a table by the primary query.

A name (without schema qualification) must be specified for each WITH query. Optionally, a list of column names can be specified; if this is omitted, the column names are inferred from the subquery.

If RECURSIVE is specified, it allows a SELECT subquery to reference itself by name. Such a subquery must have the form <non_recursive_term> UNION [ ALL | DISTINCT ] <recursive_term> where the recursive self-reference must appear on the right-hand side of the UNION. Only one recursive self-reference is permitted per query. Recursive data-modifying statements are not supported, but you can use the results of a recursive SELECT query in a data-modifying statement. See ??? for an example.

Another effect of RECURSIVE is that WITH queries need not be ordered: a query can reference another one that is later in the list. (However, circular references, or mutual recursion, are not implemented.) Without RECURSIVE, WITH queries can only reference sibling WITH queries that are earlier in the WITH list.

When there are multiple queries in the WITH clause, RECURSIVE should be written only once, immediately after WITH. It applies to all queries in the WITH clause, though it has no effect on queries that do not use recursion or forward references.

The optional SEARCH clause computes a search sequence column that can be used for ordering the results of a recursive query in either breadth-first or depth-first order. The supplied column name list specifies the row key that is to be used for keeping track of visited rows. A column named <search_seq_col_name> will be added to the result column list of the WITH query. This column can be ordered by in the outer query to achieve the respective ordering. See ??? for examples.

The optional CYCLE clause is used to detect cycles in recursive queries. The supplied column name list specifies the row key that is to be used for keeping track of visited rows. A column named <cycle_mark_col_name> will be added to the result column list of the WITH query. This column will be set to <cycle_mark_value> when a cycle has been detected, else to <cycle_mark_default>. Furthermore, processing of the recursive union will stop when a cycle has been detected. <cycle_mark_value> and <cycle_mark_default> must be constants and they must be coercible to a common data type, and the data type must have an inequality operator. (The SQL standard requires that they be Boolean constants or character strings, but PostgreSQL does not require that.) By default, TRUE and FALSE (of type boolean) are used. Furthermore, a column named <cycle_path_col_name> will be added to the result column list of the WITH query. This column is used internally for tracking visited rows. See ??? for examples.

Both the SEARCH and the CYCLE clause are only valid for recursive WITH queries. The <with_query> must be a UNION (or UNION ALL) of two SELECT (or equivalent) commands (no nested UNIONs). If both clauses are used, the column added by the SEARCH clause appears before the columns added by the CYCLE clause.

The primary query and the WITH queries are all (notionally) executed at the same time. This implies that the effects of a data-modifying statement in WITH cannot be seen from other parts of the query, other than by reading its RETURNING output. If two such data-modifying statements attempt to modify the same row, the results are unspecified.

A key property of WITH queries is that they are normally evaluated only once per execution of the primary query, even if the primary query refers to them more than once. In particular, data-modifying statements are guaranteed to be executed once and only once, regardless of whether the primary query reads all or any of their output.

However, a WITH query can be marked NOT MATERIALIZED to remove this guarantee. In that case, the WITH query can be folded into the primary query much as though it were a simple sub-SELECT in the primary query's FROM clause. This results in duplicate computations if the primary query refers to that WITH query more than once; but if each such use requires only a few rows of the WITH query's total output, NOT MATERIALIZED can provide a net savings by allowing the queries to be optimized jointly. NOT MATERIALIZED is ignored if it is attached to a WITH query that is recursive or is not side-effect-free (i.e., is not a plain SELECT containing no volatile functions).

By default, a side-effect-free WITH query is folded into the primary query if it is used exactly once in the primary query's FROM clause. This allows joint optimization of the two query levels in situations where that should be semantically invisible. However, such folding can be prevented by marking the WITH query as MATERIALIZED. That might be useful, for example, if the WITH query is being used as an optimization fence to prevent the planner from choosing a bad plan. PostgreSQL versions before v12 never did such folding, so queries written for older versions might rely on WITH to act as an optimization fence.

See ??? for additional information.

FROM Clause

The FROM clause specifies one or more source tables for the SELECT. If multiple sources are specified, the result is the Cartesian product (cross join) of all the sources. But usually qualification conditions are added (via WHERE) to restrict the returned rows to a small subset of the Cartesian product.

The FROM clause can contain the following elements:

<table_name>
The name (optionally schema-qualified) of an existing table or view. If ONLY is specified before the table name, only that table is scanned. If ONLY is not specified, the table and all its descendant tables (if any) are scanned. Optionally, * can be specified after the table name to explicitly indicate that descendant tables are included.

<alias>
A substitute name for the FROM item containing the alias. An alias is used for brevity or to eliminate ambiguity for self-joins (where the same table is scanned multiple times). When an alias is provided, it completely hides the actual name of the table or function; for example given FROM foo AS f, the remainder of the SELECT must refer to this FROM item as f not foo. If an alias is written, a column alias list can also be written to provide substitute names for one or more columns of the table.

TABLESAMPLE sampling_method ( argument [, ...] ) [ REPEATABLE ( seed ) ]
A TABLESAMPLE clause after a <table_name> indicates that the specified <sampling_method> should be used to retrieve a subset of the rows in that table. This sampling precedes the application of any other filters such as WHERE clauses. The standard PostgreSQL distribution includes two sampling methods, BERNOULLI and SYSTEM, and other sampling methods can be installed in the database via extensions.

The BERNOULLI and SYSTEM sampling methods each accept a single <argument> which is the fraction of the table to sample, expressed as a percentage between 0 and 100. This argument can be any real-valued expression. (Other sampling methods might accept more or different arguments.) These two methods each return a randomly-chosen sample of the table that will contain approximately the specified percentage of the table's rows. The BERNOULLI method scans the whole table and selects or ignores individual rows independently with the specified probability. The SYSTEM method does block-level sampling with each block having the specified chance of being selected; all rows in each selected block are returned. The SYSTEM method is significantly faster than the BERNOULLI method when small sampling percentages are specified, but it may return a less-random sample of the table as a result of clustering effects.

The optional REPEATABLE clause specifies a <seed> number or expression to use for generating random numbers within the sampling method. The seed value can be any non-null floating-point value. Two queries that specify the same seed and <argument> values will select the same sample of the table, if the table has not been changed meanwhile. But different seed values will usually produce different samples. If REPEATABLE is not given then a new random sample is selected for each query, based upon a system-generated seed. Note that some add-on sampling methods do not accept REPEATABLE, and will always produce new samples on each use.

<select>
A sub-SELECT can appear in the FROM clause. This acts as though its output were created as a temporary table for the duration of this single SELECT command. Note that the sub-SELECT must be surrounded by parentheses, and an alias can be provided in the same way as for a table. A VALUES command can also be used here.

<with_query_name>
A WITH query is referenced by writing its name, just as though the query's name were a table name. (In fact, the WITH query hides any real table of the same name for the purposes of the primary query. If necessary, you can refer to a real table of the same name by schema-qualifying the table's name.) An alias can be provided in the same way as for a table.

<function_name>
Function calls can appear in the FROM clause. (This is especially useful for functions that return result sets, but any function can be used.) This acts as though the function's output were created as a temporary table for the duration of this single SELECT command. If the function's result type is composite (including the case of a function with multiple OUT parameters), each attribute becomes a separate column in the implicit table.

When the optional WITH ORDINALITY clause is added to the function call, an additional column of type bigint will be appended to the function's result column(s). This column numbers the rows of the function's result set, starting from 1. By default, this column is named ordinality.

An alias can be provided in the same way as for a table. If an alias is written, a column alias list can also be written to provide substitute names for one or more attributes of the function's composite return type, including the ordinality column if present.

Multiple function calls can be combined into a single FROM-clause item by surrounding them with ROWS FROM( ... ). The output of such an item is the concatenation of the first row from each function, then the second row from each function, etc. If some of the functions produce fewer rows than others, null values are substituted for the missing data, so that the total number of rows returned is always the same as for the function that produced the most rows.

If the function has been defined as returning the record data type, then an alias or the key word AS must be present, followed by a column definition list in the form ( column_name data_type , ... ). The column definition list must match the actual number and types of columns returned by the function.

When using the ROWS FROM( ... ) syntax, if one of the functions requires a column definition list, it's preferred to put the column definition list after the function call inside ROWS FROM( ... ). A column definition list can be placed after the ROWS FROM( ... ) construct only if there's just a single function and no WITH ORDINALITY clause.

To use ORDINALITY together with a column definition list, you must use the ROWS FROM( ... ) syntax and put the column definition list inside ROWS FROM( ... ).

<join_type>
One of

For the INNER and OUTER join types, a join condition must be specified, namely exactly one of ON join_condition, USING (join_column [, ...]), or NATURAL. See below for the meaning.

A JOIN clause combines two FROM items, which for convenience we will refer to as “tables”, though in reality they can be any type of FROM item. Use parentheses if necessary to determine the order of nesting. In the absence of parentheses, JOINs nest left-to-right. In any case JOIN binds more tightly than the commas separating FROM-list items. All the JOIN options are just a notational convenience, since they do nothing you couldn't do with plain FROM and WHERE.

LEFT OUTER JOIN returns all rows in the qualified Cartesian product (i.e., all combined rows that pass its join condition), plus one copy of each row in the left-hand table for which there was no right-hand row that passed the join condition. This left-hand row is extended to the full width of the joined table by inserting null values for the right-hand columns. Note that only the JOIN clause's own condition is considered while deciding which rows have matches. Outer conditions are applied afterwards.

Conversely, RIGHT OUTER JOIN returns all the joined rows, plus one row for each unmatched right-hand row (extended with nulls on the left). This is just a notational convenience, since you could convert it to a LEFT OUTER JOIN by switching the left and right tables.

FULL OUTER JOIN returns all the joined rows, plus one row for each unmatched left-hand row (extended with nulls on the right), plus one row for each unmatched right-hand row (extended with nulls on the left).

ON join_condition
<join_condition> is an expression resulting in a value of type boolean (similar to a WHERE clause) that specifies which rows in a join are considered to match.

USING ( join_column [, ...] ) [ AS join_using_alias ]
A clause of the form USING ( a, b, ... ) is shorthand for ON left_table.a = right_table.a AND left_table.b = right_table.b .... Also, USING implies that only one of each pair of equivalent columns will be included in the join output, not both.

If a <join_using_alias> name is specified, it provides a table alias for the join columns. Only the join columns listed in the USING clause are addressable by this name. Unlike a regular <alias>, this does not hide the names of the joined tables from the rest of the query. Also unlike a regular <alias>, you cannot write a column alias list — the output names of the join columns are the same as they appear in the USING list.

NATURAL
NATURAL is shorthand for a USING list that mentions all columns in the two tables that have matching names. If there are no common column names, NATURAL is equivalent to ON TRUE.

CROSS JOIN
CROSS JOIN is equivalent to INNER JOIN ON (TRUE), that is, no rows are removed by qualification. They produce a simple Cartesian product, the same result as you get from listing the two tables at the top level of FROM, but restricted by the join condition (if any).

LATERAL
The LATERAL key word can precede a sub-SELECT FROM item. This allows the sub-SELECT to refer to columns of FROM items that appear before it in the FROM list. (Without LATERAL, each sub-SELECT is evaluated independently and so cannot cross-reference any other FROM item.)

LATERAL can also precede a function-call FROM item, but in this case it is a noise word, because the function expression can refer to earlier FROM items in any case.

A LATERAL item can appear at top level in the FROM list, or within a JOIN tree. In the latter case it can also refer to any items that are on the left-hand side of a JOIN that it is on the right-hand side of.

When a FROM item contains LATERAL cross-references, evaluation proceeds as follows: for each row of the FROM item providing the cross-referenced column(s), or set of rows of multiple FROM items providing the columns, the LATERAL item is evaluated using that row or row set's values of the columns. The resulting row(s) are joined as usual with the rows they were computed from. This is repeated for each row or set of rows from the column source table(s).

The column source table(s) must be INNER or LEFT joined to the LATERAL item, else there would not be a well-defined set of rows from which to compute each set of rows for the LATERAL item. Thus, although a construct such as X RIGHT JOIN LATERAL Y is syntactically valid, it is not actually allowed for <Y> to reference <X>.

WHERE Clause

The optional WHERE clause has the general form WHERE <condition> where <condition> is any expression that evaluates to a result of type boolean. Any row that does not satisfy this condition will be eliminated from the output. A row satisfies the condition if it returns true when the actual row values are substituted for any variable references.

GROUP BY Clause

The optional GROUP BY clause has the general form GROUP BY [ ALL | DISTINCT ] <grouping_element> [, ...]

GROUP BY will condense into a single row all selected rows that share the same values for the grouped expressions. An <expression> used inside a <grouping_element> can be an input column name, or the name or ordinal number of an output column (SELECT list item), or an arbitrary expression formed from input-column values; however, it cannot contain an aggregate function or a window function. In case of ambiguity, a GROUP BY name will be interpreted as an input-column name rather than an output column name.

If any of GROUPING SETS, ROLLUP or CUBE are present as grouping elements, then the GROUP BY clause as a whole defines some number of independent <grouping sets>. The effect of this is equivalent to constructing a UNION ALL between subqueries having the individual grouping sets as their GROUP BY clauses. The optional DISTINCT key word removes duplicate grouping sets before processing; it does not transform the implied UNION ALL into a UNION DISTINCT. For further details on the handling of grouping sets see ???.

Aggregate functions, if any are used, are computed across all rows making up each group, producing a separate value for each group. (If there are aggregate functions but no GROUP BY clause, the query is treated as having a single group comprising all the selected rows.) The set of rows fed to each aggregate function can be further filtered by attaching a FILTER clause to the aggregate function call; see ??? for more information. When a FILTER clause is present, only those rows matching it are included in the input to that aggregate function.

When GROUP BY is present, or any aggregate functions are present, it is not valid for the SELECT list expressions to refer to ungrouped columns except within aggregate functions or when the ungrouped column is functionally dependent on the grouped columns, since there would otherwise be more than one possible value to return for an ungrouped column. A functional dependency exists if the grouped columns (or a subset thereof) are the primary key of the table containing the ungrouped column.

Keep in mind that all aggregate functions are evaluated before evaluating any “scalar” expressions in the HAVING clause or SELECT list. This means that, for example, a CASE expression cannot be used to skip evaluation of an aggregate function; see ???.

Currently, FOR NO KEY UPDATE, FOR UPDATE, FOR SHARE and FOR KEY SHARE cannot be specified with GROUP BY.

HAVING Clause

The optional HAVING clause has the general form HAVING <condition> where <condition> is the same as specified for the WHERE clause.

HAVING eliminates group rows that do not satisfy the condition. HAVING is different from WHERE: WHERE filters individual rows before the application of GROUP BY, while HAVING filters group rows created by GROUP BY. Each column referenced in <condition> must unambiguously reference a grouping column, unless the reference appears within an aggregate function or the ungrouped column is functionally dependent on the grouping columns.

The presence of HAVING turns a query into a grouped query even if there is no GROUP BY clause. This is the same as what happens when the query contains aggregate functions but no GROUP BY clause. All the selected rows are considered to form a single group, and the SELECT list and HAVING clause can only reference table columns from within aggregate functions. Such a query will emit a single row if the HAVING condition is true, zero rows if it is not true.

Currently, FOR NO KEY UPDATE, FOR UPDATE, FOR SHARE and FOR KEY SHARE cannot be specified with HAVING.

WINDOW Clause

The optional WINDOW clause has the general form WINDOW <window_name> AS ( <window_definition> ) [, ...] where <window_name> is a name that can be referenced from OVER clauses or subsequent window definitions, and <window_definition> is [ <existing_window_name> ] [ PARTITION BY <expression> [, ...] ] [ ORDER BY <expression> [ ASC | DESC | USING <operator> ] [ NULLS { FIRST | LAST } ] [, ...] ] [ <frame_clause> ]

If an <existing_window_name> is specified it must refer to an earlier entry in the WINDOW list; the new window copies its partitioning clause from that entry, as well as its ordering clause if any. In this case the new window cannot specify its own PARTITION BY clause, and it can specify ORDER BY only if the copied window does not have one. The new window always uses its own frame clause; the copied window must not specify a frame clause.

The elements of the PARTITION BY list are interpreted in much the same fashion as elements of a GROUP BY clause, except that they are always simple expressions and never the name or number of an output column. Another difference is that these expressions can contain aggregate function calls, which are not allowed in a regular GROUP BY clause. They are allowed here because windowing occurs after grouping and aggregation.

Similarly, the elements of the ORDER BY list are interpreted in much the same fashion as elements of a statement-level ORDER BY clause, except that the expressions are always taken as simple expressions and never the name or number of an output column.

The optional <frame_clause> defines the window frame for window functions that depend on the frame (not all do). The window frame is a set of related rows for each row of the query (called the current row). The <frame_clause> can be one of { RANGE | ROWS | GROUPS } <frame_start> [ <frame_exclusion> ] { RANGE | ROWS | GROUPS } BETWEEN <frame_start> AND <frame_end> [ <frame_exclusion> ] where <frame_start> and <frame_end> can be one of UNBOUNDED PRECEDING <offset> PRECEDING CURRENT ROW <offset> FOLLOWING UNBOUNDED FOLLOWING and <frame_exclusion> can be one of EXCLUDE CURRENT ROW EXCLUDE GROUP EXCLUDE TIES EXCLUDE NO OTHERS If <frame_end> is omitted it defaults to CURRENT ROW. Restrictions are that <frame_start> cannot be UNBOUNDED FOLLOWING, <frame_end> cannot be UNBOUNDED PRECEDING, and the <frame_end> choice cannot appear earlier in the above list of <frame_start> and <frame_end> options than the <frame_start> choice does — for example RANGE BETWEEN CURRENT ROW AND offset PRECEDING is not allowed.

The default framing option is RANGE UNBOUNDED PRECEDING, which is the same as RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW; it sets the frame to be all rows from the partition start up through the current row's last peer (a row that the window's ORDER BY clause considers equivalent to the current row; all rows are peers if there is no ORDER BY). In general, UNBOUNDED PRECEDING means that the frame starts with the first row of the partition, and similarly UNBOUNDED FOLLOWING means that the frame ends with the last row of the partition, regardless of RANGE, ROWS or GROUPS mode. In ROWS mode, CURRENT ROW means that the frame starts or ends with the current row; but in RANGE or GROUPS mode it means that the frame starts or ends with the current row's first or last peer in the ORDER BY ordering. The <offset> PRECEDING and <offset> FOLLOWING options vary in meaning depending on the frame mode. In ROWS mode, the <offset> is an integer indicating that the frame starts or ends that many rows before or after the current row. In GROUPS mode, the <offset> is an integer indicating that the frame starts or ends that many peer groups before or after the current row's peer group, where a peer group is a group of rows that are equivalent according to the window's ORDER BY clause. In RANGE mode, use of an <offset> option requires that there be exactly one ORDER BY column in the window definition. Then the frame contains those rows whose ordering column value is no more than <offset> less than (for PRECEDING) or more than (for FOLLOWING) the current row's ordering column value. In these cases the data type of the <offset> expression depends on the data type of the ordering column. For numeric ordering columns it is typically of the same type as the ordering column, but for datetime ordering columns it is an interval. In all these cases, the value of the <offset> must be non-null and non-negative. Also, while the <offset> does not have to be a simple constant, it cannot contain variables, aggregate functions, or window functions.

The <frame_exclusion> option allows rows around the current row to be excluded from the frame, even if they would be included according to the frame start and frame end options. EXCLUDE CURRENT ROW excludes the current row from the frame. EXCLUDE GROUP excludes the current row and its ordering peers from the frame. EXCLUDE TIES excludes any peers of the current row from the frame, but not the current row itself. EXCLUDE NO OTHERS simply specifies explicitly the default behavior of not excluding the current row or its peers.

Beware that the ROWS mode can produce unpredictable results if the ORDER BY ordering does not order the rows uniquely. The RANGE and GROUPS modes are designed to ensure that rows that are peers in the ORDER BY ordering are treated alike: all rows of a given peer group will be in the frame or excluded from it.

The purpose of a WINDOW clause is to specify the behavior of window functions appearing in the query's SELECT list or ORDER BY clause. These functions can reference the WINDOW clause entries by name in their OVER clauses. A WINDOW clause entry does not have to be referenced anywhere, however; if it is not used in the query it is simply ignored. It is possible to use window functions without any WINDOW clause at all, since a window function call can specify its window definition directly in its OVER clause. However, the WINDOW clause saves typing when the same window definition is needed for more than one window function.

Currently, FOR NO KEY UPDATE, FOR UPDATE, FOR SHARE and FOR KEY SHARE cannot be specified with WINDOW.

Window functions are described in detail in ???, ???, and ???.

SELECT List

The SELECT list (between the key words SELECT and FROM) specifies expressions that form the output rows of the SELECT statement. The expressions can (and usually do) refer to columns computed in the FROM clause.

Just as in a table, every output column of a SELECT has a name. In a simple SELECT this name is just used to label the column for display, but when the SELECT is a sub-query of a larger query, the name is seen by the larger query as the column name of the virtual table produced by the sub-query. To specify the name to use for an output column, write AS <output_name> after the column's expression. (You can omit AS, but only if the desired output name does not match any PostgreSQL keyword (see ???). For protection against possible future keyword additions, it is recommended that you always either write AS or double-quote the output name.) If you do not specify a column name, a name is chosen automatically by PostgreSQL. If the column's expression is a simple column reference then the chosen name is the same as that column's name. In more complex cases a function or type name may be used, or the system may fall back on a generated name such as ?column?.

An output column's name can be used to refer to the column's value in ORDER BY and GROUP BY clauses, but not in the WHERE or HAVING clauses; there you must write out the expression instead.

Instead of an expression, * can be written in the output list as a shorthand for all the columns of the selected rows. Also, you can write table_name.* as a shorthand for the columns coming from just that table. In these cases it is not possible to specify new names with AS; the output column names will be the same as the table columns' names.

According to the SQL standard, the expressions in the output list should be computed before applying DISTINCT, ORDER BY, or LIMIT. This is obviously necessary when using DISTINCT, since otherwise it's not clear what values are being made distinct. However, in many cases it is convenient if output expressions are computed after ORDER BY and LIMIT; particularly if the output list contains any volatile or expensive functions. With that behavior, the order of function evaluations is more intuitive and there will not be evaluations corresponding to rows that never appear in the output. PostgreSQL will effectively evaluate output expressions after sorting and limiting, so long as those expressions are not referenced in DISTINCT, ORDER BY or GROUP BY. (As a counterexample, SELECT f(x) FROM tab ORDER BY 1 clearly must evaluate f(x) before sorting.) Output expressions that contain set-returning functions are effectively evaluated after sorting and before limiting, so that LIMIT will act to cut off the output from a set-returning function.

[!NOTE] PostgreSQL versions before 9.6 did not provide any guarantees about the timing of evaluation of output expressions versus sorting and limiting; it depended on the form of the chosen query plan.

DISTINCT Clause

If SELECT DISTINCT is specified, all duplicate rows are removed from the result set (one row is kept from each group of duplicates). SELECT ALL specifies the opposite: all rows are kept; that is the default.

SELECT DISTINCT ON ( expression [, ...] ) keeps only the first row of each set of rows where the given expressions evaluate to equal. The DISTINCT ON expressions are interpreted using the same rules as for ORDER BY (see above). Note that the “first row” of each set is unpredictable unless ORDER BY is used to ensure that the desired row appears first. For example:

SELECT DISTINCT ON (location) location, time, report
    FROM weather_reports
    ORDER BY location, time DESC;

retrieves the most recent weather report for each location. But if we had not used ORDER BY to force descending order of time values for each location, we'd have gotten a report from an unpredictable time for each location.

The DISTINCT ON expression(s) must match the leftmost ORDER BY expression(s). The ORDER BY clause will normally contain additional expression(s) that determine the desired precedence of rows within each DISTINCT ON group.

Currently, FOR NO KEY UPDATE, FOR UPDATE, FOR SHARE and FOR KEY SHARE cannot be specified with DISTINCT.

UNION Clause

The UNION clause has this general form: <select_statement> UNION [ ALL | DISTINCT ] <select_statement> <select_statement> is any SELECT statement without an ORDER BY, LIMIT, FOR NO KEY UPDATE, FOR UPDATE, FOR SHARE, or FOR KEY SHARE clause. (ORDER BY and LIMIT can be attached to a subexpression if it is enclosed in parentheses. Without parentheses, these clauses will be taken to apply to the result of the UNION, not to its right-hand input expression.)

The UNION operator computes the set union of the rows returned by the involved SELECT statements. A row is in the set union of two result sets if it appears in at least one of the result sets. The two SELECT statements that represent the direct operands of the UNION must produce the same number of columns, and corresponding columns must be of compatible data types.

The result of UNION does not contain any duplicate rows unless the ALL option is specified. ALL prevents elimination of duplicates. (Therefore, UNION ALL is usually significantly quicker than UNION; use ALL when you can.) DISTINCT can be written to explicitly specify the default behavior of eliminating duplicate rows.

Multiple UNION operators in the same SELECT statement are evaluated left to right, unless otherwise indicated by parentheses.

Currently, FOR NO KEY UPDATE, FOR UPDATE, FOR SHARE and FOR KEY SHARE cannot be specified either for a UNION result or for any input of a UNION.

INTERSECT Clause

The INTERSECT clause has this general form: <select_statement> INTERSECT [ ALL | DISTINCT ] <select_statement> <select_statement> is any SELECT statement without an ORDER BY, LIMIT, FOR NO KEY UPDATE, FOR UPDATE, FOR SHARE, or FOR KEY SHARE clause.

The INTERSECT operator computes the set intersection of the rows returned by the involved SELECT statements. A row is in the intersection of two result sets if it appears in both result sets.

The result of INTERSECT does not contain any duplicate rows unless the ALL option is specified. With ALL, a row that has <m> duplicates in the left table and <n> duplicates in the right table will appear min(<m>,<n>) times in the result set. DISTINCT can be written to explicitly specify the default behavior of eliminating duplicate rows.

Multiple INTERSECT operators in the same SELECT statement are evaluated left to right, unless parentheses dictate otherwise. INTERSECT binds more tightly than UNION. That is, A UNION B INTERSECT C will be read as A UNION (B INTERSECT C).

Currently, FOR NO KEY UPDATE, FOR UPDATE, FOR SHARE and FOR KEY SHARE cannot be specified either for an INTERSECT result or for any input of an INTERSECT.

EXCEPT Clause

The EXCEPT clause has this general form: <select_statement> EXCEPT [ ALL | DISTINCT ] <select_statement> <select_statement> is any SELECT statement without an ORDER BY, LIMIT, FOR NO KEY UPDATE, FOR UPDATE, FOR SHARE, or FOR KEY SHARE clause.

The EXCEPT operator computes the set of rows that are in the result of the left SELECT statement but not in the result of the right one.

The result of EXCEPT does not contain any duplicate rows unless the ALL option is specified. With ALL, a row that has <m> duplicates in the left table and <n> duplicates in the right table will appear max(<m>-<n>,0) times in the result set. DISTINCT can be written to explicitly specify the default behavior of eliminating duplicate rows.

Multiple EXCEPT operators in the same SELECT statement are evaluated left to right, unless parentheses dictate otherwise. EXCEPT binds at the same level as UNION.

Currently, FOR NO KEY UPDATE, FOR UPDATE, FOR SHARE and FOR KEY SHARE cannot be specified either for an EXCEPT result or for any input of an EXCEPT.

ORDER BY Clause

The optional ORDER BY clause has this general form: ORDER BY <expression> [ ASC | DESC | USING <operator> ] [ NULLS { FIRST | LAST } ] [, ...] The ORDER BY clause causes the result rows to be sorted according to the specified expression(s). If two rows are equal according to the leftmost expression, they are compared according to the next expression and so on. If they are equal according to all specified expressions, they are returned in an implementation-dependent order.

Each <expression> can be the name or ordinal number of an output column (SELECT list item), or it can be an arbitrary expression formed from input-column values.

The ordinal number refers to the ordinal (left-to-right) position of the output column. This feature makes it possible to define an ordering on the basis of a column that does not have a unique name. This is never absolutely necessary because it is always possible to assign a name to an output column using the AS clause.

It is also possible to use arbitrary expressions in the ORDER BY clause, including columns that do not appear in the SELECT output list. Thus the following statement is valid:

SELECT name FROM distributors ORDER BY code;

A limitation of this feature is that an ORDER BY clause applying to the result of a UNION, INTERSECT, or EXCEPT clause can only specify an output column name or number, not an expression.

If an ORDER BY expression is a simple name that matches both an output column name and an input column name, ORDER BY will interpret it as the output column name. This is the opposite of the choice that GROUP BY will make in the same situation. This inconsistency is made to be compatible with the SQL standard.

Optionally one can add the key word ASC (ascending) or DESC (descending) after any expression in the ORDER BY clause. If not specified, ASC is assumed by default. Alternatively, a specific ordering operator name can be specified in the USING clause. An ordering operator must be a less-than or greater-than member of some B-tree operator family. ASC is usually equivalent to USING < and DESC is usually equivalent to USING >. (But the creator of a user-defined data type can define exactly what the default sort ordering is, and it might correspond to operators with other names.)

If NULLS LAST is specified, null values sort after all non-null values; if NULLS FIRST is specified, null values sort before all non-null values. If neither is specified, the default behavior is NULLS LAST when ASC is specified or implied, and NULLS FIRST when DESC is specified (thus, the default is to act as though nulls are larger than non-nulls). When USING is specified, the default nulls ordering depends on whether the operator is a less-than or greater-than operator.

Note that ordering options apply only to the expression they follow; for example ORDER BY x, y DESC does not mean the same thing as ORDER BY x DESC, y DESC.

Character-string data is sorted according to the collation that applies to the column being sorted. That can be overridden at need by including a COLLATE clause in the <expression>, for example ORDER BY mycolumn COLLATE "en_US". For more information see ??? and ???.

LIMIT Clause

The LIMIT clause consists of two independent sub-clauses: LIMIT { <count> | ALL } OFFSET <start> The parameter <count> specifies the maximum number of rows to return, while <start> specifies the number of rows to skip before starting to return rows. When both are specified, <start> rows are skipped before starting to count the <count> rows to be returned.

If the <count> expression evaluates to NULL, it is treated as LIMIT ALL, i.e., no limit. If <start> evaluates to NULL, it is treated the same as OFFSET 0.

SQL:2008 introduced a different syntax to achieve the same result, which PostgreSQL also supports. It is: OFFSET <start> { ROW | ROWS } FETCH { FIRST | NEXT } [ <count> ] { ROW | ROWS } { ONLY | WITH TIES } In this syntax, the <start> or <count> value is required by the standard to be a literal constant, a parameter, or a variable name; as a PostgreSQL extension, other expressions are allowed, but will generally need to be enclosed in parentheses to avoid ambiguity. If <count> is omitted in a FETCH clause, it defaults to 1. The WITH TIES option is used to return any additional rows that tie for the last place in the result set according to the ORDER BY clause; ORDER BY is mandatory in this case, and SKIP LOCKED is not allowed. ROW and ROWS as well as FIRST and NEXT are noise words that don't influence the effects of these clauses. According to the standard, the OFFSET clause must come before the FETCH clause if both are present; but PostgreSQL is laxer and allows either order.

When using LIMIT, it is a good idea to use an ORDER BY clause that constrains the result rows into a unique order. Otherwise you will get an unpredictable subset of the query's rows — you might be asking for the tenth through twentieth rows, but tenth through twentieth in what ordering? You don't know what ordering unless you specify ORDER BY.

The query planner takes LIMIT into account when generating a query plan, so you are very likely to get different plans (yielding different row orders) depending on what you use for LIMIT and OFFSET. Thus, using different LIMIT/OFFSET values to select different subsets of a query result will give inconsistent results unless you enforce a predictable result ordering with ORDER BY. This is not a bug; it is an inherent consequence of the fact that SQL does not promise to deliver the results of a query in any particular order unless ORDER BY is used to constrain the order.

It is even possible for repeated executions of the same LIMIT query to return different subsets of the rows of a table, if there is not an ORDER BY to enforce selection of a deterministic subset. Again, this is not a bug; determinism of the results is simply not guaranteed in such a case.

The Locking Clause

FOR UPDATE, FOR NO KEY UPDATE, FOR SHARE and FOR KEY SHARE are locking clauses; they affect how SELECT locks rows as they are obtained from the table.

The locking clause has the general form FOR <lock_strength> [ OF <from_reference> [, ...] ] [ NOWAIT | SKIP LOCKED ] where <lock_strength> can be one of UPDATE NO KEY UPDATE SHARE KEY SHARE

<from_reference> must be a table <alias> or non-hidden <table_name> referenced in the FROM clause. For more information on each row-level lock mode, refer to ???.

To prevent the operation from waiting for other transactions to commit, use either the NOWAIT or SKIP LOCKED option. With NOWAIT, the statement reports an error, rather than waiting, if a selected row cannot be locked immediately. With SKIP LOCKED, any selected rows that cannot be immediately locked are skipped. Skipping locked rows provides an inconsistent view of the data, so this is not suitable for general purpose work, but can be used to avoid lock contention with multiple consumers accessing a queue-like table. Note that NOWAIT and SKIP LOCKED apply only to the row-level lock(s) — the required ROW SHARE table-level lock is still taken in the ordinary way (see ???). You can use LOCK with the NOWAIT option first, if you need to acquire the table-level lock without waiting.

If specific tables are named in a locking clause, then only rows coming from those tables are locked; any other tables used in the SELECT are simply read as usual. A locking clause without a table list affects all tables used in the statement. If a locking clause is applied to a view or sub-query, it affects all tables used in the view or sub-query. However, these clauses do not apply to WITH queries referenced by the primary query. If you want row locking to occur within a WITH query, specify a locking clause within the WITH query.

Multiple locking clauses can be written if it is necessary to specify different locking behavior for different tables. If the same table is mentioned (or implicitly affected) by more than one locking clause, then it is processed as if it was only specified by the strongest one. Similarly, a table is processed as NOWAIT if that is specified in any of the clauses affecting it. Otherwise, it is processed as SKIP LOCKED if that is specified in any of the clauses affecting it.

The locking clauses cannot be used in contexts where returned rows cannot be clearly identified with individual table rows; for example they cannot be used with aggregation.

When a locking clause appears at the top level of a SELECT query, the rows that are locked are exactly those that are returned by the query; in the case of a join query, the rows locked are those that contribute to returned join rows. In addition, rows that satisfied the query conditions as of the query snapshot will be locked, although they will not be returned if they were updated after the snapshot and no longer satisfy the query conditions. If a LIMIT is used, locking stops once enough rows have been returned to satisfy the limit (but note that rows skipped over by OFFSET will get locked). Similarly, if a locking clause is used in a cursor's query, only rows actually fetched or stepped past by the cursor will be locked.

When a locking clause appears in a sub-SELECT, the rows locked are those returned to the outer query by the sub-query. This might involve fewer rows than inspection of the sub-query alone would suggest, since conditions from the outer query might be used to optimize execution of the sub-query. For example,

SELECT * FROM (SELECT * FROM mytable FOR UPDATE) ss WHERE col1 = 5;

will lock only rows having col1 = 5, even though that condition is not textually within the sub-query.

Previous releases failed to preserve a lock which is upgraded by a later savepoint. For example, this code:

BEGIN;
SELECT * FROM mytable WHERE key = 1 FOR UPDATE;
SAVEPOINT s;
UPDATE mytable SET ... WHERE key = 1;
ROLLBACK TO s;

would fail to preserve the FOR UPDATE lock after the ROLLBACK TO. This has been fixed in release 9.3.

[!CAUTION] It is possible for a SELECT command running at the READ COMMITTED transaction isolation level and using ORDER BY and a locking clause to return rows out of order. This is because ORDER BY is applied first. The command sorts the result, but might then block trying to obtain a lock on one or more of the rows. Once the SELECT unblocks, some of the ordering column values might have been modified, leading to those rows appearing to be out of order (though they are in order in terms of the original column values). This can be worked around as needed by placing the FOR UPDATE/SHARE clause in a sub-query, for example

SELECT * FROM (SELECT * FROM mytable FOR UPDATE) ss ORDER BY column1;

Note that this will result in locking all rows of mytable, whereas FOR UPDATE at the top level would lock only the actually returned rows. This can make for a significant performance difference, particularly if the ORDER BY is combined with LIMIT or other restrictions. So this technique is recommended only if concurrent updates of the ordering columns are expected and a strictly sorted result is required.

At the REPEATABLE READ or SERIALIZABLE transaction isolation level this would cause a serialization failure (with an SQLSTATE of '40001'), so there is no possibility of receiving rows out of order under these isolation levels.

TABLE Command

The command

TABLE name

is equivalent to

SELECT * FROM name

It can be used as a top-level command or as a space-saving syntax variant in parts of complex queries. Only the WITH, UNION, INTERSECT, EXCEPT, ORDER BY, LIMIT, OFFSET, FETCH and FOR locking clauses can be used with TABLE; the WHERE clause and any form of aggregation cannot be used.

Examples

To join the table films with the table distributors:

SELECT f.title, f.did, d.name, f.date_prod, f.kind
    FROM distributors d JOIN films f USING (did);

       title       | did |     name     | date_prod  |   kind
-------------------+-----+--------------+------------+----------
 The Third Man     | 101 | British Lion | 1949-12-23 | Drama
 The African Queen | 101 | British Lion | 1951-08-11 | Romantic
 ...

To sum the column len of all films and group the results by kind:

SELECT kind, sum(len) AS total FROM films GROUP BY kind;

   kind   | total
----------+-------
 Action   | 07:34
 Comedy   | 02:58
 Drama    | 14:28
 Musical  | 06:42
 Romantic | 04:38

To sum the column len of all films, group the results by kind and show those group totals that are less than 5 hours:

SELECT kind, sum(len) AS total
    FROM films
    GROUP BY kind
    HAVING sum(len) < interval '5 hours';

   kind   | total
----------+-------
 Comedy   | 02:58
 Romantic | 04:38

The following two examples are identical ways of sorting the individual results according to the contents of the second column (name):

SELECT * FROM distributors ORDER BY name;
SELECT * FROM distributors ORDER BY 2;

 did |       name
-----+------------------
 109 | 20th Century Fox
 110 | Bavaria Atelier
 101 | British Lion
 107 | Columbia
 102 | Jean Luc Godard
 113 | Luso films
 104 | Mosfilm
 103 | Paramount
 106 | Toho
 105 | United Artists
 111 | Walt Disney
 112 | Warner Bros.
 108 | Westward

The next example shows how to obtain the union of the tables distributors and actors, restricting the results to those that begin with the letter W in each table. Only distinct rows are wanted, so the key word ALL is omitted.

distributors:               actors:
 did |     name              id |     name
-----+--------------        ----+----------------
 108 | Westward               1 | Woody Allen
 111 | Walt Disney            2 | Warren Beatty
 112 | Warner Bros.           3 | Walter Matthau
 ...                         ...

SELECT distributors.name
    FROM distributors
    WHERE distributors.name LIKE 'W%'
UNION
SELECT actors.name
    FROM actors
    WHERE actors.name LIKE 'W%';

      name
----------------
 Walt Disney
 Walter Matthau
 Warner Bros.
 Warren Beatty
 Westward
 Woody Allen

This example shows how to use a function in the FROM clause, both with and without a column definition list:

CREATE FUNCTION distributors(int) RETURNS SETOF distributors AS $$
    SELECT * FROM distributors WHERE did = $1;
$$ LANGUAGE SQL;

SELECT * FROM distributors(111);
 did |    name
-----+-------------
 111 | Walt Disney

CREATE FUNCTION distributors_2(int) RETURNS SETOF record AS $$
    SELECT * FROM distributors WHERE did = $1;
$$ LANGUAGE SQL;

SELECT * FROM distributors_2(111) AS (f1 int, f2 text);
 f1  |     f2
-----+-------------
 111 | Walt Disney

Here is an example of a function with an ordinality column added:

SELECT * FROM unnest(ARRAY['a','b','c','d','e','f']) WITH ORDINALITY;
 unnest | ordinality
--------+----------
 a      |        1
 b      |        2
 c      |        3
 d      |        4
 e      |        5
 f      |        6
(6 rows)

This example shows how to use a simple WITH clause:

WITH t AS (
    SELECT random() AS x FROM generate_series(1, 3)
  )
SELECT * FROM t
UNION ALL
SELECT * FROM t;
         x
--------------------
  0.534150459803641
  0.520092216785997
 0.0735620250925422
  0.534150459803641
  0.520092216785997
 0.0735620250925422

Notice that the WITH query was evaluated only once, so that we got two sets of the same three random values.

This example uses WITH RECURSIVE to find all subordinates (direct or indirect) of the employee Mary, and their level of indirectness, from a table that shows only direct subordinates:

WITH RECURSIVE employee_recursive(distance, employee_name, manager_name) AS (
    SELECT 1, employee_name, manager_name
    FROM employee
    WHERE manager_name = 'Mary'
  UNION ALL
    SELECT er.distance + 1, e.employee_name, e.manager_name
    FROM employee_recursive er, employee e
    WHERE er.employee_name = e.manager_name
  )
SELECT distance, employee_name FROM employee_recursive;

Notice the typical form of recursive queries: an initial condition, followed by UNION, followed by the recursive part of the query. Be sure that the recursive part of the query will eventually return no tuples, or else the query will loop indefinitely. (See ??? for more examples.)

This example uses LATERAL to apply a set-returning function get_product_names() for each row of the manufacturers table:

SELECT m.name AS mname, pname
FROM manufacturers m, LATERAL get_product_names(m.id) pname;

Manufacturers not currently having any products would not appear in the result, since it is an inner join. If we wished to include the names of such manufacturers in the result, we could do:

SELECT m.name AS mname, pname
FROM manufacturers m LEFT JOIN LATERAL get_product_names(m.id) pname ON true;

Compatibility

Of course, the SELECT statement is compatible with the SQL standard. But there are some extensions and some missing features.

Omitted FROM Clauses

PostgreSQL allows one to omit the FROM clause. It has a straightforward use to compute the results of simple expressions:

SELECT 2+2;

 ?column?
----------
        4

Some other SQL databases cannot do this except by introducing a dummy one-row table from which to do the SELECT.

Empty SELECT Lists

The list of output expressions after SELECT can be empty, producing a zero-column result table. This is not valid syntax according to the SQL standard. PostgreSQL allows it to be consistent with allowing zero-column tables. However, an empty list is not allowed when DISTINCT is used.

Omitting the AS Key Word

In the SQL standard, the optional key word AS can be omitted before an output column name whenever the new column name is a valid column name (that is, not the same as any reserved keyword). PostgreSQL is slightly more restrictive: AS is required if the new column name matches any keyword at all, reserved or not. Recommended practice is to use AS or double-quote output column names, to prevent any possible conflict against future keyword additions.

In FROM items, both the standard and PostgreSQL allow AS to be omitted before an alias that is an unreserved keyword. But this is impractical for output column names, because of syntactic ambiguities.

Omitting Sub-SELECT Aliases in FROM

According to the SQL standard, a sub-SELECT in the FROM list must have an alias. In PostgreSQL, this alias may be omitted.

ONLY and Inheritance

The SQL standard requires parentheses around the table name when writing ONLY, for example SELECT * FROM ONLY (tab1), ONLY (tab2) WHERE .... PostgreSQL considers these parentheses to be optional.

PostgreSQL allows a trailing * to be written to explicitly specify the non-ONLY behavior of including child tables. The standard does not allow this.

(These points apply equally to all SQL commands supporting the ONLY option.)

TABLESAMPLE Clause Restrictions

The TABLESAMPLE clause is currently accepted only on regular tables and materialized views. According to the SQL standard it should be possible to apply it to any FROM item.

Function Calls in FROM

PostgreSQL allows a function call to be written directly as a member of the FROM list. In the SQL standard it would be necessary to wrap such a function call in a sub-SELECT; that is, the syntax FROM func(...) alias is approximately equivalent to FROM LATERAL (SELECT func(...)) alias. Note that LATERAL is considered to be implicit; this is because the standard requires LATERAL semantics for an UNNEST() item in FROM. PostgreSQL treats UNNEST() the same as other set-returning functions.

Namespace Available to GROUP BY and ORDER BY

In the SQL-92 standard, an ORDER BY clause can only use output column names or numbers, while a GROUP BY clause can only use expressions based on input column names. PostgreSQL extends each of these clauses to allow the other choice as well (but it uses the standard's interpretation if there is ambiguity). PostgreSQL also allows both clauses to specify arbitrary expressions. Note that names appearing in an expression will always be taken as input-column names, not as output-column names.

SQL:1999 and later use a slightly different definition which is not entirely upward compatible with SQL-92. In most cases, however, PostgreSQL will interpret an ORDER BY or GROUP BY expression the same way SQL:1999 does.

Functional Dependencies

PostgreSQL recognizes functional dependency (allowing columns to be omitted from GROUP BY) only when a table's primary key is included in the GROUP BY list. The SQL standard specifies additional conditions that should be recognized.

LIMIT and OFFSET

The clauses LIMIT and OFFSET are PostgreSQL-specific syntax, also used by MySQL. The SQL:2008 standard has introduced the clauses OFFSET ... FETCH {FIRST|NEXT} ... for the same functionality, as shown above in LIMIT Clause. This syntax is also used by IBM DB2. (Applications written for Oracle frequently use a workaround involving the automatically generated rownum column, which is not available in PostgreSQL, to implement the effects of these clauses.)

FOR NO KEY UPDATE, FOR UPDATE, FOR SHARE, FOR KEY SHARE

Although FOR UPDATE appears in the SQL standard, the standard allows it only as an option of DECLARE CURSOR. PostgreSQL allows it in any SELECT query as well as in sub-SELECTs, but this is an extension. The FOR NO KEY UPDATE, FOR SHARE and FOR KEY SHARE variants, as well as the NOWAIT and SKIP LOCKED options, do not appear in the standard.

Data-Modifying Statements in WITH

PostgreSQL allows INSERT, UPDATE, DELETE, and MERGE to be used as WITH queries. This is not found in the SQL standard.

Nonstandard Clauses

DISTINCT ON ( ... ) is an extension of the SQL standard.

ROWS FROM( ... ) is an extension of the SQL standard.

The MATERIALIZED and NOT MATERIALIZED options of WITH are extensions of the SQL standard.

select_into

SELECT INTO

SELECT INTO

7

SQL - Language Statements

SELECT INTO

define a new table from the results of a query

[ WITH [ RECURSIVE ]

with_query

[, ...] ] SELECT [ ALL | DISTINCT [ ON (

expression

[, ...] ) ] ] [ { * |

expression

[ [ AS ]

output_name

] } [, ...] ] INTO [ TEMPORARY | TEMP | UNLOGGED ] [ TABLE ]

new_table

[ FROM

from_item

[, ...] ] [ WHERE

condition

] [ GROUP BY [ ALL | DISTINCT ]

grouping_element

[, ...] ] [ HAVING

condition

] [ WINDOW

window_name

AS (

window_definition

) [, ...] ] [ { UNION | INTERSECT | EXCEPT } [ ALL | DISTINCT ]

select

] [ ORDER BY

expression

[ ASC | DESC | USING

operator

] [ NULLS { FIRST | LAST } ] [, ...] ] [ LIMIT {

count

| ALL } ] [ OFFSET

start

[ ROW | ROWS ] ] [ FETCH { FIRST | NEXT } [

count

] { ROW | ROWS } { ONLY | WITH TIES } ] [ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE } [ OF

from_reference

[, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]

Description

SELECT INTO creates a new table and fills it with data computed by a query. The data is not returned to the client, as it is with a normal SELECT. The new table's columns have the names and data types associated with the output columns of the SELECT.

Parameters

TEMPORARY or TEMP
If specified, the table is created as a temporary table. Refer to ??? for details.

UNLOGGED
If specified, the table is created as an unlogged table. Refer to ??? for details.

<new_table>
The name (optionally schema-qualified) of the table to be created.

All other parameters are described in detail under ???.

Notes

CREATE TABLE AS is functionally similar to SELECT INTO. CREATE TABLE AS is the recommended syntax, since this form of SELECT INTO is not available in ECPG or PL/pgSQL, because they interpret the INTO clause differently. Furthermore, CREATE TABLE AS offers a superset of the functionality provided by SELECT INTO.

In contrast to CREATE TABLE AS, SELECT INTO does not allow specifying properties like a table's access method with ??? or the table's tablespace with ???. Use CREATE TABLE AS if necessary. Therefore, the default table access method is chosen for the new table. See ??? for more information.

Examples

Create a new table films_recent consisting of only recent entries from the table films:

SELECT * INTO films_recent FROM films WHERE date_prod >= '2002-01-01';

Compatibility

The SQL standard uses SELECT INTO to represent selecting values into scalar variables of a host program, rather than creating a new table. This indeed is the usage found in ECPG (see ???) and PL/pgSQL (see ???). The PostgreSQL usage of SELECT INTO to represent table creation is historical. Some other SQL implementations also use SELECT INTO in this way (but most SQL implementations support CREATE TABLE AS instead). Apart from such compatibility considerations, it is best to use CREATE TABLE AS for this purpose in new code.

See Also

set

SET

SET

7

SQL - Language Statements

SET

change a run-time parameter

SET [ SESSION | LOCAL ]

configuration_parameter

{ TO | = } {

value

[, ...] | DEFAULT } SET [ SESSION | LOCAL ] TIME ZONE {

value

| LOCAL | DEFAULT }

Description

The SET command changes run-time configuration parameters. Many of the run-time parameters listed in ??? can be changed on-the-fly with SET. (Some parameters can only be changed by superusers and users who have been granted SET privilege on that parameter. There are also parameters that cannot be changed after server or session start.) SET only affects the value used by the current session.

If SET (or equivalently SET SESSION) is issued within a transaction that is later aborted, the effects of the SET command disappear when the transaction is rolled back. Once the surrounding transaction is committed, the effects will persist until the end of the session, unless overridden by another SET.

The effects of SET LOCAL last only till the end of the current transaction, whether committed or not. A special case is SET followed by SET LOCAL within a single transaction: the SET LOCAL value will be seen until the end of the transaction, but afterwards (if the transaction is committed) the SET value will take effect.

The effects of SET or SET LOCAL are also canceled by rolling back to a savepoint that is earlier than the command.

If SET LOCAL is used within a function that has a SET option for the same variable (see ???), the effects of the SET LOCAL command disappear at function exit; that is, the value in effect when the function was called is restored anyway. This allows SET LOCAL to be used for dynamic or repeated changes of a parameter within a function, while still having the convenience of using the SET option to save and restore the caller's value. However, a regular SET command overrides any surrounding function's SET option; its effects will persist unless rolled back.

[!NOTE] In PostgreSQL versions 8.0 through 8.2, the effects of a SET LOCAL would be canceled by releasing an earlier savepoint, or by successful exit from a PL/pgSQL exception block. This behavior has been changed because it was deemed unintuitive.

Parameters

SESSION
Specifies that the command takes effect for the current session. (This is the default if neither SESSION nor LOCAL appears.)

LOCAL
Specifies that the command takes effect for only the current transaction. After COMMIT or ROLLBACK, the session-level setting takes effect again. Issuing this outside of a transaction block emits a warning and otherwise has no effect.

<configuration_parameter>
Name of a settable configuration parameter. Available parameters are documented in ??? and below.

<value>
New value of the parameter. Values can be specified as string constants, identifiers, numbers, or comma-separated lists of these, as appropriate for the particular parameter. Values that are neither numbers nor valid identifiers must be quoted. If the parameter accepts a list of values, NULL can be written to specify an empty list. DEFAULT can be written to specify resetting the parameter to its default value (that is, whatever value it would have had if no SET had been executed in the current session).

Besides the configuration parameters documented in ???, there are a few that can only be adjusted using the SET command or that have a special syntax:

SCHEMA
SET SCHEMA 'value' is an alias for SET search_path TO value. Only one schema can be specified using this syntax.

NAMES
SET NAMES 'value' is an alias for SET client_encoding TO value.

SEED
Sets the internal seed for the random number generator (the function random). Allowed values are floating-point numbers between -1 and 1 inclusive.

The seed can also be set by invoking the function setseed:

SELECT setseed(value);

TIME ZONE
SET TIME ZONE 'value' is an alias for SET timezone TO 'value'. The syntax SET TIME ZONE allows special syntax for the time zone specification. Here are examples of valid values:

'America/Los_Angeles'
The time zone for Berkeley, California.

'Europe/Rome'
The time zone for Italy.

-7
The time zone 7 hours west from UTC (equivalent to PDT). Positive values are east from UTC.

INTERVAL '-08:00' HOUR TO MINUTE
The time zone 8 hours west from UTC (equivalent to PST).

LOCAL; DEFAULT
Set the time zone to your local time zone (that is, the server's default value of timezone).

Timezone settings given as numbers or intervals are internally translated to POSIX timezone syntax. For example, after SET TIME ZONE -7, SHOW TIME ZONE would report <-07>+07.

Time zone abbreviations are not supported by SET; see ??? for more information about time zones.

Notes

The function set_config provides equivalent functionality; see ???. Also, it is possible to UPDATE the pg_settings system view to perform the equivalent of SET.

Examples

Set the schema search path:

SET search_path TO my_schema, public;

Note that this is not the same as

SET search_path TO 'my_schema, public';

which would have the effect of setting search_path to contain a single, probably-nonexistent schema name.

Set the list of temporary tablespace names to be empty:

SET temp_tablespaces TO NULL;

Set the style of date to traditional POSTGRES with “day before month” input convention:

SET datestyle TO postgres, dmy;

Set the time zone for Berkeley, California:

SET TIME ZONE 'America/Los_Angeles';

Set the time zone for Italy:

SET TIME ZONE 'Europe/Rome';

Compatibility

SET TIME ZONE extends syntax defined in the SQL standard. The standard allows only numeric time zone offsets while PostgreSQL allows more flexible time-zone specifications. All other SET features are PostgreSQL extensions.

See Also

set_constraints

SET CONSTRAINTS

SET CONSTRAINTS

7

SQL - Language Statements

SET CONSTRAINTS

set constraint check timing for the current transaction

SET CONSTRAINTS { ALL |

name

[, ...] } { DEFERRED | IMMEDIATE }

Description

SET CONSTRAINTS sets the behavior of constraint checking within the current transaction. IMMEDIATE constraints are checked at the end of each statement. DEFERRED constraints are not checked until transaction commit. Each constraint has its own IMMEDIATE or DEFERRED mode.

Upon creation, a constraint is given one of three characteristics: DEFERRABLE INITIALLY DEFERRED, DEFERRABLE INITIALLY IMMEDIATE, or NOT DEFERRABLE. The third class is always IMMEDIATE and is not affected by the SET CONSTRAINTS command. The first two classes start every transaction in the indicated mode, but their behavior can be changed within a transaction by SET CONSTRAINTS.

SET CONSTRAINTS with a list of constraint names changes the mode of just those constraints (which must all be deferrable). Each constraint name can be schema-qualified. The current schema search path is used to find the first matching name if no schema name is specified. SET CONSTRAINTS ALL changes the mode of all deferrable constraints.

When SET CONSTRAINTS changes the mode of a constraint from DEFERRED to IMMEDIATE, the new mode takes effect retroactively: any outstanding data modifications that would have been checked at the end of the transaction are instead checked during the execution of the SET CONSTRAINTS command. If any such constraint is violated, the SET CONSTRAINTS fails (and does not change the constraint mode). Thus, SET CONSTRAINTS can be used to force checking of constraints to occur at a specific point in a transaction.

Currently, only UNIQUE, PRIMARY KEY, REFERENCES (foreign key), and EXCLUDE constraints are affected by this setting. NOT NULL and CHECK constraints are always checked immediately when a row is inserted or modified (not at the end of the statement). Uniqueness and exclusion constraints that have not been declared DEFERRABLE are also checked immediately.

The firing of triggers that are declared as “constraint triggers” is also controlled by this setting — they fire at the same time that the associated constraint should be checked.

Notes

Because PostgreSQL does not require constraint names to be unique within a schema (but only per-table), it is possible that there is more than one match for a specified constraint name. In this case SET CONSTRAINTS will act on all matches. For a non-schema-qualified name, once a match or matches have been found in some schema in the search path, schemas appearing later in the path are not searched.

This command only alters the behavior of constraints within the current transaction. Issuing this outside of a transaction block emits a warning and otherwise has no effect.

Compatibility

This command complies with the behavior defined in the SQL standard, except for the limitation that, in PostgreSQL, it does not apply to NOT NULL and CHECK constraints. Also, PostgreSQL checks non-deferrable uniqueness constraints immediately, not at end of statement as the standard would suggest.

set_role

SET ROLE

SET ROLE

7

SQL - Language Statements

SET ROLE

set the current user identifier of the current session

SET [ SESSION | LOCAL ] ROLE

role_name

SET [ SESSION | LOCAL ] ROLE NONE RESET ROLE

Description

This command sets the current user identifier of the current SQL session to be <role_name>. The role name can be written as either an identifier or a string literal. After SET ROLE, permissions checking for SQL commands is carried out as though the named role were the one that had logged in originally. Note that SET ROLE and SET SESSION AUTHORIZATION are exceptions; permissions checks for those continue to use the current session user and the initial session user (the authenticated user), respectively.

The current session user must have the SET option for the specified <role_name>, either directly or indirectly via a chain of memberships with the SET option. (If the session user is a superuser, any role can be selected.)

The SESSION and LOCAL modifiers act the same as for the regular SET command.

SET ROLE NONE sets the current user identifier to the current session user identifier, as returned by session_user. RESET ROLE sets the current user identifier to the connection-time setting specified by the command-line options, ALTER ROLE, or ALTER DATABASE, if any such settings exist. Otherwise, RESET ROLE sets the current user identifier to the current session user identifier. These forms can be executed by any user.

Notes

Using this command, it is possible to either add privileges or restrict one's privileges. If the session user role has been granted memberships WITH INHERIT TRUE, it automatically has all the privileges of every such role. In this case, SET ROLE effectively drops all the privileges except for those which the target role directly possesses or inherits. On the other hand, if the session user role has been granted memberships WITH INHERIT FALSE, the privileges of the granted roles can't be accessed by default. However, if the role was granted WITH SET TRUE, the session user can use SET ROLE to drop the privileges assigned directly to the session user and instead acquire the privileges available to the named role. If the role was granted WITH INHERIT FALSE, SET FALSE then the privileges of that role cannot be exercised either with or without SET ROLE.

SET ROLE has effects comparable to SET SESSION AUTHORIZATION, but the privilege checks involved are quite different. Also, SET SESSION AUTHORIZATION determines which roles are allowable for later SET ROLE commands, whereas changing roles with SET ROLE does not change the set of roles allowed to a later SET ROLE.

SET ROLE does not process session variables as specified by the role's ALTER ROLE settings; this only happens during login.

SET ROLE cannot be used within a SECURITY DEFINER function.

Examples

SELECT SESSION_USER, CURRENT_USER;

 session_user | current_user
--------------+--------------
 peter        | peter

SET ROLE 'paul';

SELECT SESSION_USER, CURRENT_USER;

 session_user | current_user
--------------+--------------
 peter        | paul

Compatibility

PostgreSQL allows identifier syntax ("rolename"), while the SQL standard requires the role name to be written as a string literal. SQL does not allow this command during a transaction; PostgreSQL does not make this restriction because there is no reason to. The SESSION and LOCAL modifiers are a PostgreSQL extension, as is the RESET syntax.

See Also

set_session_auth

SET SESSION AUTHORIZATION

SET SESSION AUTHORIZATION

7

SQL - Language Statements

SET SESSION AUTHORIZATION

set the session user identifier and the current user identifier of the current session

SET [ SESSION | LOCAL ] SESSION AUTHORIZATION

user_name

SET [ SESSION | LOCAL ] SESSION AUTHORIZATION DEFAULT RESET SESSION AUTHORIZATION

Description

This command sets the session user identifier and the current user identifier of the current SQL session to be <user_name>. The user name can be written as either an identifier or a string literal. Using this command, it is possible, for example, to temporarily become an unprivileged user and later switch back to being a superuser.

The session user identifier is initially set to be the (possibly authenticated) user name provided by the client. The current user identifier is normally equal to the session user identifier, but might change temporarily in the context of SECURITY DEFINER functions and similar mechanisms; it can also be changed by SET ROLE. The current user identifier is relevant for permission checking.

The session user identifier can be changed only if the initial session user (the authenticated user) has the superuser privilege. Otherwise, the command is accepted only if it specifies the authenticated user name.

The SESSION and LOCAL modifiers act the same as for the regular SET command.

The DEFAULT and RESET forms reset the session and current user identifiers to be the originally authenticated user name. These forms can be executed by any user.

Notes

SET SESSION AUTHORIZATION cannot be used within a SECURITY DEFINER function.

Examples

SELECT SESSION_USER, CURRENT_USER;

 session_user | current_user
--------------+--------------
 peter        | peter

SET SESSION AUTHORIZATION 'paul';

SELECT SESSION_USER, CURRENT_USER;

 session_user | current_user
--------------+--------------
 paul         | paul

Compatibility

The SQL standard allows some other expressions to appear in place of the literal <user_name>, but these options are not important in practice. PostgreSQL allows identifier syntax ("username"), which SQL does not. SQL does not allow this command during a transaction; PostgreSQL does not make this restriction because there is no reason to. The SESSION and LOCAL modifiers are a PostgreSQL extension, as is the RESET syntax.

The privileges necessary to execute this command are left implementation-defined by the standard.

See Also

set_transaction

SET TRANSACTION

transaction isolation level

setting

read-only transaction

setting

deferrable transaction

setting

SET TRANSACTION

7

SQL - Language Statements

SET TRANSACTION

set the characteristics of the current transaction

SET TRANSACTION

transaction_mode

[, ...] SET TRANSACTION SNAPSHOT

snapshot_id

SET SESSION CHARACTERISTICS AS TRANSACTION

transaction_mode

[, ...]

where

transaction_mode

is one of:

ISOLATION LEVEL { SERIALIZABLE | REPEATABLE READ | READ COMMITTED | READ UNCOMMITTED } READ WRITE | READ ONLY [ NOT ] DEFERRABLE

Description

The SET TRANSACTION command sets the characteristics of the current transaction. It has no effect on any subsequent transactions. SET SESSION CHARACTERISTICS sets the default transaction characteristics for subsequent transactions of a session. These defaults can be overridden by SET TRANSACTION for an individual transaction.

The available transaction characteristics are the transaction isolation level, the transaction access mode (read/write or read-only), and the deferrable mode. In addition, a snapshot can be selected, though only for the current transaction, not as a session default.

The isolation level of a transaction determines what data the transaction can see when other transactions are running concurrently:

READ COMMITTED
A statement can only see rows committed before it began. This is the default.

REPEATABLE READ
All statements of the current transaction can only see rows committed before the first query or data-modification statement was executed in this transaction.

SERIALIZABLE
All statements of the current transaction can only see rows committed before the first query or data-modification statement was executed in this transaction. If a pattern of reads and writes among concurrent serializable transactions would create a situation which could not have occurred for any serial (one-at-a-time) execution of those transactions, one of them will be rolled back with a serialization_failure error.

The SQL standard defines one additional level, READ UNCOMMITTED. In PostgreSQL READ UNCOMMITTED is treated as READ COMMITTED.

The transaction isolation level cannot be changed after the first query or data-modification statement (SELECT, INSERT, DELETE, UPDATE, MERGE, FETCH, or COPY) of a transaction has been executed. See ??? for more information about transaction isolation and concurrency control.

The transaction access mode determines whether the transaction is read/write or read-only. Read/write is the default. When a transaction is read-only, the following SQL commands are disallowed: INSERT, UPDATE, DELETE, MERGE, and COPY FROM if the table they would write to is not a temporary table; all CREATE, ALTER, and DROP commands; COMMENT, GRANT, REVOKE, TRUNCATE; and EXPLAIN ANALYZE and EXECUTE if the command they would execute is among those listed. This is a high-level notion of read-only that does not prevent all writes to disk.

The DEFERRABLE transaction property has no effect unless the transaction is also SERIALIZABLE and READ ONLY. When all three of these properties are selected for a transaction, the transaction may block when first acquiring its snapshot, after which it is able to run without the normal overhead of a SERIALIZABLE transaction and without any risk of contributing to or being canceled by a serialization failure. This mode is well suited for long-running reports or backups.

The SET TRANSACTION SNAPSHOT command allows a new transaction to run with the same snapshot as an existing transaction. The pre-existing transaction must have exported its snapshot with the pg_export_snapshot function (see ???). That function returns a snapshot identifier, which must be given to SET TRANSACTION SNAPSHOT to specify which snapshot is to be imported. The identifier must be written as a string literal in this command, for example '00000003-0000001B-1'. SET TRANSACTION SNAPSHOT can only be executed at the start of a transaction, before the first query or data-modification statement (SELECT, INSERT, DELETE, UPDATE, MERGE, FETCH, or COPY) of the transaction. Furthermore, the transaction must already be set to SERIALIZABLE or REPEATABLE READ isolation level (otherwise, the snapshot would be discarded immediately, since READ COMMITTED mode takes a new snapshot for each command). If the importing transaction uses SERIALIZABLE isolation level, then the transaction that exported the snapshot must also use that isolation level. Also, a non-read-only serializable transaction cannot import a snapshot from a read-only transaction.

Notes

If SET TRANSACTION is executed without a prior START TRANSACTION or BEGIN, it emits a warning and otherwise has no effect.

It is possible to dispense with SET TRANSACTION by instead specifying the desired <transaction_modes> in BEGIN or START TRANSACTION. But that option is not available for SET TRANSACTION SNAPSHOT.

The session default transaction modes can also be set or examined via the configuration parameters ???, ???, and ???. (In fact SET SESSION CHARACTERISTICS is just a verbose equivalent for setting these variables with SET.) This means the defaults can be set in the configuration file, via ALTER DATABASE, etc. Consult ??? for more information.

The current transaction's modes can similarly be set or examined via the configuration parameters ???, ???, and ???. Setting one of these parameters acts the same as the corresponding SET TRANSACTION option, with the same restrictions on when it can be done. However, these parameters cannot be set in the configuration file, or from any source other than live SQL.

Examples

To begin a new transaction with the same snapshot as an already existing transaction, first export the snapshot from the existing transaction. That will return the snapshot identifier, for example:

BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
SELECT pg_export_snapshot();
 pg_export_snapshot
---------------------
 00000003-0000001B-1
(1 row)

Then give the snapshot identifier in a SET TRANSACTION SNAPSHOT command at the beginning of the newly opened transaction:

BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
SET TRANSACTION SNAPSHOT '00000003-0000001B-1';

Compatibility

These commands are defined in the SQL standard, except for the DEFERRABLE transaction mode and the SET TRANSACTION SNAPSHOT form, which are PostgreSQL extensions.

SERIALIZABLE is the default transaction isolation level in the standard. In PostgreSQL the default is ordinarily READ COMMITTED, but you can change it as mentioned above.

In the SQL standard, there is one other transaction characteristic that can be set with these commands: the size of the diagnostics area. This concept is specific to embedded SQL, and therefore is not implemented in the PostgreSQL server.

The SQL standard requires commas between successive <transaction_modes>, but for historical reasons PostgreSQL allows the commas to be omitted.

show

SHOW

SHOW

7

SQL - Language Statements

SHOW

show the value of a run-time parameter

SHOW

name

SHOW ALL

Description

SHOW will display the current setting of run-time parameters. These variables can be set using the SET statement, by editing the postgresql.conf configuration file, through the PGOPTIONS environmental variable (when using libpq or a libpq-based application), or through command-line flags when starting the postgres server. See ??? for details.

Parameters

<name>
The name of a run-time parameter. Available parameters are documented in ??? and on the ??? reference page. In addition, there are a few parameters that can be shown but not set:

SERVER_VERSION
Shows the server's version number.

SERVER_ENCODING
Shows the server-side character set encoding. At present, this parameter can be shown but not set, because the encoding is determined at database creation time.

IS_SUPERUSER
True if the current role has superuser privileges.

ALL
Show the values of all configuration parameters, with descriptions.

Notes

The function current_setting produces equivalent output; see ???. Also, the pg_settings system view produces the same information.

Examples

Show the current setting of the parameter DateStyle:

SHOW DateStyle;
 DateStyle
-----------
 ISO, MDY
(1 row)

Show the current setting of the parameter geqo:

SHOW geqo;
 geqo
------
 on
(1 row)

Show all settings:

SHOW ALL;
            name         | setting |                description
-------------------------+---------+-------------------------------------------------
 allow_system_table_mods | off     | Allows modifications of the structure of ...
    .
    .
    .
 xmloption               | content | Sets whether XML data in implicit parsing ...
 zero_damaged_pages      | off     | Continues processing past damaged page headers.
(196 rows)

Compatibility

The SHOW command is a PostgreSQL extension.

See Also

start_transaction

START TRANSACTION

START TRANSACTION

7

SQL - Language Statements

START TRANSACTION

start a transaction block

START TRANSACTION [

transaction_mode

[, ...] ]

where

transaction_mode

is one of:

ISOLATION LEVEL { SERIALIZABLE | REPEATABLE READ | READ COMMITTED | READ UNCOMMITTED } READ WRITE | READ ONLY [ NOT ] DEFERRABLE

Description

This command begins a new transaction block. If the isolation level, read/write mode, or deferrable mode is specified, the new transaction has those characteristics, as if SET TRANSACTION was executed. This is the same as the BEGIN command.

Parameters

Refer to ??? for information on the meaning of the parameters to this statement.

Compatibility

In the standard, it is not necessary to issue START TRANSACTION to start a transaction block: any SQL command implicitly begins a block. PostgreSQL's behavior can be seen as implicitly issuing a COMMIT after each command that does not follow START TRANSACTION (or BEGIN), and it is therefore often called “autocommit”. Other relational database systems might offer an autocommit feature as a convenience.

The DEFERRABLE <transaction_mode> is a PostgreSQL language extension.

The SQL standard requires commas between successive <transaction_modes>, but for historical reasons PostgreSQL allows the commas to be omitted.

See also the compatibility section of ???.

See Also

truncate

TRUNCATE

TRUNCATE

7

SQL - Language Statements

TRUNCATE

empty a table or set of tables

TRUNCATE [ TABLE ] [ ONLY ]

name

[ * ] [, ... ] [ RESTART IDENTITY | CONTINUE IDENTITY ] [ CASCADE | RESTRICT ]

Description

TRUNCATE quickly removes all rows from a set of tables. It has the same effect as an unqualified DELETE on each table, but since it does not actually scan the tables it is faster. Furthermore, it reclaims disk space immediately, rather than requiring a subsequent VACUUM operation. This is most useful on large tables.

Parameters

<name>
The name (optionally schema-qualified) of a table to truncate. If ONLY is specified before the table name, only that table is truncated. If ONLY is not specified, the table and all its descendant tables (if any) are truncated. Optionally, * can be specified after the table name to explicitly indicate that descendant tables are included.

RESTART IDENTITY
Automatically restart sequences owned by columns of the truncated table(s).

CONTINUE IDENTITY
Do not change the values of sequences. This is the default.

CASCADE
Automatically truncate all tables that have foreign-key references to any of the named tables, or to any tables added to the group due to CASCADE.

RESTRICT
Refuse to truncate if any of the tables have foreign-key references from tables that are not listed in the command. This is the default.

Notes

You must have the TRUNCATE privilege on a table to truncate it.

TRUNCATE acquires an ACCESS EXCLUSIVE lock on each table it operates on, which blocks all other concurrent operations on the table. When RESTART IDENTITY is specified, any sequences that are to be restarted are likewise locked exclusively. If concurrent access to a table is required, then the DELETE command should be used instead.

TRUNCATE cannot be used on a table that has foreign-key references from other tables, unless all such tables are also truncated in the same command. Checking validity in such cases would require table scans, and the whole point is not to do one. The CASCADE option can be used to automatically include all dependent tables — but be very careful when using this option, or else you might lose data you did not intend to! Note in particular that when the table to be truncated is a partition, siblings partitions are left untouched, but cascading occurs to all referencing tables and all their partitions with no distinction.

TRUNCATE will not fire any ON DELETE triggers that might exist for the tables. But it will fire ON TRUNCATE triggers. If ON TRUNCATE triggers are defined for any of the tables, then all BEFORE TRUNCATE triggers are fired before any truncation happens, and all AFTER TRUNCATE triggers are fired after the last truncation is performed and any sequences are reset. The triggers will fire in the order that the tables are to be processed (first those listed in the command, and then any that were added due to cascading).

TRUNCATE is not MVCC-safe. After truncation, the table will appear empty to concurrent transactions, if they are using a snapshot taken before the truncation occurred. See ??? for more details.

TRUNCATE is transaction-safe with respect to the data in the tables: the truncation will be safely rolled back if the surrounding transaction does not commit.

When RESTART IDENTITY is specified, the implied ALTER SEQUENCE RESTART operations are also done transactionally; that is, they will be rolled back if the surrounding transaction does not commit. Be aware that if any additional sequence operations are done on the restarted sequences before the transaction rolls back, the effects of these operations on the sequences will be rolled back, but not their effects on currval(); that is, after the transaction currval() will continue to reflect the last sequence value obtained inside the failed transaction, even though the sequence itself may no longer be consistent with that. This is similar to the usual behavior of currval() after a failed transaction.

TRUNCATE can be used for foreign tables if supported by the foreign data wrapper, for instance, see ???.

Examples

Truncate the tables bigtable and fattable:

TRUNCATE bigtable, fattable;

The same, and also reset any associated sequence generators:

TRUNCATE bigtable, fattable RESTART IDENTITY;

Truncate the table othertable, and cascade to any tables that reference othertable via foreign-key constraints:

TRUNCATE othertable CASCADE;

Compatibility

The SQL:2008 standard includes a TRUNCATE command with the syntax TRUNCATE TABLE tablename. The clauses CONTINUE IDENTITY/RESTART IDENTITY also appear in that standard, but have slightly different though related meanings. Some of the concurrency behavior of this command is left implementation-defined by the standard, so the above notes should be considered and compared with other implementations if necessary.

See Also

unlisten

UNLISTEN

UNLISTEN

7

SQL - Language Statements

UNLISTEN

stop listening for a notification

UNLISTEN {

channel

| * }

Description

UNLISTEN is used to remove an existing registration for NOTIFY events. UNLISTEN cancels any existing registration of the current PostgreSQL session as a listener on the notification channel named <channel>. The special wildcard * cancels all listener registrations for the current session.

??? contains a more extensive discussion of the use of LISTEN and NOTIFY.

Parameters

<channel>
Name of a notification channel (any identifier).

*
All current listen registrations for this session are cleared.

Notes

You can unlisten something you were not listening for; no warning or error will appear.

At the end of each session, UNLISTEN * is automatically executed.

A transaction that has executed UNLISTEN cannot be prepared for two-phase commit.

Examples

To make a registration:

LISTEN virtual;
NOTIFY virtual;
Asynchronous notification "virtual" received from server process with PID 8448.

Once UNLISTEN has been executed, further NOTIFY messages will be ignored:

UNLISTEN virtual;
NOTIFY virtual;
-- no NOTIFY event is received

Compatibility

There is no UNLISTEN command in the SQL standard.

See Also

update

UPDATE

UPDATE

7

SQL - Language Statements

UPDATE

update rows of a table

[ WITH [ RECURSIVE ]

with_query

[, ...] ] UPDATE [ ONLY ]

table_name

[ * ] [ [ AS ]

alias

] SET {

column_name

= {

expression

| DEFAULT } | (

column_name

[, ...] ) = [ ROW ] ( {

expression

| DEFAULT } [, ...] ) | (

column_name

[, ...] ) = (

sub-SELECT

) } [, ...] [ FROM

from_item

[, ...] ] [ WHERE

condition

| WHERE CURRENT OF

cursor_name

] [ RETURNING [ WITH ( { OLD | NEW } AS

output_alias

[, ...] ) ] { * |

output_expression

[ [ AS ]

output_name

] } [, ...] ]

Description

UPDATE changes the values of the specified columns in all rows that satisfy the condition. Only the columns to be modified need be mentioned in the SET clause; columns not explicitly modified retain their previous values.

There are two ways to modify a table using information contained in other tables in the database: using sub-selects, or specifying additional tables in the FROM clause. Which technique is more appropriate depends on the specific circumstances.

The optional RETURNING clause causes UPDATE to compute and return value(s) based on each row actually updated. Any expression using the table's columns, and/or columns of other tables mentioned in FROM, can be computed. By default, the new (post-update) values of the table's columns are used, but it is also possible to request the old (pre-update) values. The syntax of the RETURNING list is identical to that of the output list of SELECT.

You must have the UPDATE privilege on the table, or at least on the column(s) that are listed to be updated. You must also have the SELECT privilege on any column whose values are read in the <expressions> or <condition>.

Parameters

<with_query>
The WITH clause allows you to specify one or more subqueries that can be referenced by name in the UPDATE query. See ??? and ??? for details.

<table_name>
The name (optionally schema-qualified) of the table to update. If ONLY is specified before the table name, matching rows are updated in the named table only. If ONLY is not specified, matching rows are also updated in any tables inheriting from the named table. Optionally, * can be specified after the table name to explicitly indicate that descendant tables are included.

<alias>
A substitute name for the target table. When an alias is provided, it completely hides the actual name of the table. For example, given UPDATE foo AS f, the remainder of the UPDATE statement must refer to this table as f not foo.

<column_name>
The name of a column in the table named by <table_name>. The column name can be qualified with a subfield name or array subscript, if needed. Do not include the table's name in the specification of a target column — for example, UPDATE table_name SET table_name.col = 1 is invalid.

<expression>
An expression to assign to the column. The expression can use the old values of this and other columns in the table.

DEFAULT
Set the column to its default value (which will be NULL if no specific default expression has been assigned to it). An identity column will be set to a new value generated by the associated sequence. For a generated column, specifying this is permitted but merely specifies the normal behavior of computing the column from its generation expression.

<sub-SELECT>
A SELECT sub-query that produces as many output columns as are listed in the parenthesized column list preceding it. The sub-query must yield no more than one row when executed. If it yields one row, its column values are assigned to the target columns; if it yields no rows, NULL values are assigned to the target columns. The sub-query can refer to old values of the current row of the table being updated.

<from_item>
A table expression allowing columns from other tables to appear in the WHERE condition and update expressions. This uses the same syntax as the FROM clause of a SELECT statement; for example, an alias for the table name can be specified. Do not repeat the target table as a <from_item> unless you intend a self-join (in which case it must appear with an alias in the <from_item>).

<condition>
An expression that returns a value of type boolean. Only rows for which this expression returns true will be updated.

<cursor_name>
The name of the cursor to use in a WHERE CURRENT OF condition. The row to be updated is the one most recently fetched from this cursor. The cursor must be a non-grouping query on the UPDATE's target table. Note that WHERE CURRENT OF cannot be specified together with a Boolean condition. See ??? for more information about using cursors with WHERE CURRENT OF.

<output_alias>
An optional substitute name for OLD or NEW rows in the RETURNING list.

By default, old values from the target table can be returned by writing OLD.column_name or OLD.*, and new values can be returned by writing NEW.column_name or NEW.*. When an alias is provided, these names are hidden and the old or new rows must be referred to using the alias. For example RETURNING WITH (OLD AS o, NEW AS n) o.*, n.*.

<output_expression>
An expression to be computed and returned by the UPDATE command after each row is updated. The expression can use any column names of the table named by <table_name> or table(s) listed in FROM. Write * to return all columns.

A column name or * may be qualified using OLD or NEW, or the corresponding <output_alias> for OLD or NEW, to cause old or new values to be returned. An unqualified column name, or *, or a column name or * qualified using the target table name or alias will return new values.

<output_name>
A name to use for a returned column.

Outputs

On successful completion, an UPDATE command returns a command tag of the form

UPDATE count

The <count> is the number of rows updated, including matched rows whose values did not change. Note that the number may be less than the number of rows that matched the <condition> when updates were suppressed by a BEFORE UPDATE trigger. If <count> is 0, no rows were updated by the query (this is not considered an error).

If the UPDATE command contains a RETURNING clause, the result will be similar to that of a SELECT statement containing the columns and values defined in the RETURNING list, computed over the row(s) updated by the command.

Notes

When a FROM clause is present, what essentially happens is that the target table is joined to the tables mentioned in the <from_item> list, and each output row of the join represents an update operation for the target table. When using FROM you should ensure that the join produces at most one output row for each row to be modified. In other words, a target row shouldn't join to more than one row from the other table(s). If it does, then only one of the join rows will be used to update the target row, but which one will be used is not readily predictable.

Because of this indeterminacy, referencing other tables only within sub-selects is safer, though often harder to read and slower than using a join.

In the case of a partitioned table, updating a row might cause it to no longer satisfy the partition constraint of the containing partition. In that case, if there is some other partition in the partition tree for which this row satisfies its partition constraint, then the row is moved to that partition. If there is no such partition, an error will occur. Behind the scenes, the row movement is actually a DELETE and INSERT operation.

There is a possibility that a concurrent UPDATE or DELETE on the row being moved will get a serialization failure error. Suppose session 1 is performing an UPDATE on a partition key, and meanwhile a concurrent session 2 for which this row is visible performs an UPDATE or DELETE operation on this row. In such case, session 2's UPDATE or DELETE will detect the row movement and raise a serialization failure error (which always returns with an SQLSTATE code '40001'). Applications may wish to retry the transaction if this occurs. In the usual case where the table is not partitioned, or where there is no row movement, session 2 would have identified the newly updated row and carried out the UPDATE/DELETE on this new row version.

Note that while rows can be moved from local partitions to a foreign-table partition (provided the foreign data wrapper supports tuple routing), they cannot be moved from a foreign-table partition to another partition.

An attempt of moving a row from one partition to another will fail if a foreign key is found to directly reference an ancestor of the source partition that is not the same as the ancestor that's mentioned in the UPDATE query.

Examples

Change the word Drama to Dramatic in the column kind of the table films:

UPDATE films SET kind = 'Dramatic' WHERE kind = 'Drama';

Adjust temperature entries and reset precipitation to its default value in one row of the table weather:

UPDATE weather SET temp_lo = temp_lo+1, temp_hi = temp_lo+15, prcp = DEFAULT
  WHERE city = 'San Francisco' AND date = '2003-07-03';

Perform the same operation and return the updated entries, and the old precipitation value:

UPDATE weather SET temp_lo = temp_lo+1, temp_hi = temp_lo+15, prcp = DEFAULT
  WHERE city = 'San Francisco' AND date = '2003-07-03'
  RETURNING temp_lo, temp_hi, prcp, old.prcp AS old_prcp;

Use the alternative column-list syntax to do the same update:

UPDATE weather SET (temp_lo, temp_hi, prcp) = (temp_lo+1, temp_lo+15, DEFAULT)
  WHERE city = 'San Francisco' AND date = '2003-07-03';

Increment the sales count of the salesperson who manages the account for Acme Corporation, using the FROM clause syntax:

UPDATE employees SET sales_count = sales_count + 1 FROM accounts
  WHERE accounts.name = 'Acme Corporation'
  AND employees.id = accounts.sales_person;

Perform the same operation, using a sub-select in the WHERE clause:

UPDATE employees SET sales_count = sales_count + 1 WHERE id =
  (SELECT sales_person FROM accounts WHERE name = 'Acme Corporation');

Update contact names in an accounts table to match the currently assigned salespeople:

UPDATE accounts SET (contact_first_name, contact_last_name) =
    (SELECT first_name, last_name FROM employees
     WHERE employees.id = accounts.sales_person);

A similar result could be accomplished with a join:

UPDATE accounts SET contact_first_name = first_name,
                    contact_last_name = last_name
  FROM employees WHERE employees.id = accounts.sales_person;

However, the second query may give unexpected results if employees.id is not a unique key, whereas the first query is guaranteed to raise an error if there are multiple id matches. Also, if there is no match for a particular accounts.sales_person entry, the first query will set the corresponding name fields to NULL, whereas the second query will not update that row at all.

Update statistics in a summary table to match the current data:

UPDATE summary s SET (sum_x, sum_y, avg_x, avg_y) =
    (SELECT sum(x), sum(y), avg(x), avg(y) FROM data d
     WHERE d.group_id = s.group_id);

Attempt to insert a new stock item along with the quantity of stock. If the item already exists, instead update the stock count of the existing item. To do this without failing the entire transaction, use savepoints:

BEGIN;
-- other operations
SAVEPOINT sp1;
INSERT INTO wines VALUES('Chateau Lafite 2003', '24');
-- Assume the above fails because of a unique key violation,
-- so now we issue these commands:
ROLLBACK TO sp1;
UPDATE wines SET stock = stock + 24 WHERE winename = 'Chateau Lafite 2003';
-- continue with other operations, and eventually
COMMIT;

Change the kind column of the table films in the row on which the cursor c_films is currently positioned:

UPDATE films SET kind = 'Dramatic' WHERE CURRENT OF c_films;

Updates affecting many rows can have negative effects on system performance, such as table bloat, increased replica lag, and increased lock contention. In such situations it can make sense to perform the operation in smaller batches, possibly with a VACUUM operation on the table between batches. While there is no LIMIT clause for UPDATE, it is possible to get a similar effect through the use of a Common Table Expression and a self-join. With the standard PostgreSQL table access method, a self-join on the system column ctid is very efficient:

WITH exceeded_max_retries AS (
  SELECT w.ctid FROM work_item AS w
    WHERE w.status = 'active' AND w.num_retries > 10
    ORDER BY w.retry_timestamp
    FOR UPDATE
    LIMIT 5000
)
UPDATE work_item SET status = 'failed'
  FROM exceeded_max_retries AS emr
  WHERE work_item.ctid = emr.ctid;

This command will need to be repeated until no rows remain to be updated. (This use of ctid is only safe because the query is repeatedly run, avoiding the problem of changed ctids.) Use of an ORDER BY clause allows the command to prioritize which rows will be updated; it can also prevent deadlock with other update operations if they use the same ordering. If lock contention is a concern, then SKIP LOCKED can be added to the CTE to prevent multiple commands from updating the same row. However, then a final UPDATE without SKIP LOCKED or LIMIT will be needed to ensure that no matching rows were overlooked.

Compatibility

This command conforms to the SQL standard, except that the FROM and RETURNING clauses are PostgreSQL extensions, as is the ability to use WITH with UPDATE.

Some other database systems offer a FROM option in which the target table is supposed to be listed again within FROM. That is not how PostgreSQL interprets FROM. Be careful when porting applications that use this extension.

According to the standard, the source value for a parenthesized sub-list of target column names can be any row-valued expression yielding the correct number of columns. PostgreSQL only allows the source value to be a row constructor or a sub-SELECT. An individual column's updated value can be specified as DEFAULT in the row-constructor case, but not inside a sub-SELECT.

vacuum

VACUUM

VACUUM

7

SQL - Language Statements

VACUUM

garbage-collect and optionally analyze a database

VACUUM [ (

option

[, ...] ) ] [

table_and_columns

[, ...] ]

where

option

can be one of:

FREEZE [

boolean

] VERBOSE [

boolean

] ANALYZE [

boolean

] DISABLE_PAGE_SKIPPING [

boolean

] SKIP_LOCKED [

boolean

] INDEX_CLEANUP { AUTO | ON | OFF } PROCESS_MAIN [

boolean

] PROCESS_TOAST [

boolean

] TRUNCATE [

boolean

] PARALLEL

integer

SKIP_DATABASE_STATS [

boolean

] ONLY_DATABASE_STATS [

boolean

] BUFFER_USAGE_LIMIT

size

FULL [

boolean

]

and

table_and_columns

is:

[ ONLY ]

table_name

[ * ] [ (

column_name

[, ...] ) ]

Description

VACUUM reclaims storage occupied by dead tuples. In normal PostgreSQL operation, tuples that are deleted or obsoleted by an update are not physically removed from their table; they remain present until a VACUUM is done. Therefore it's necessary to do VACUUM periodically, especially on frequently-updated tables.

Without a <table_and_columns> list, VACUUM processes every table and materialized view in the current database that the current user has permission to vacuum. With a list, VACUUM processes only those table(s).

VACUUM ANALYZE performs a VACUUM and then an ANALYZE for each selected table. This is a handy combination form for routine maintenance scripts. See ??? for more details about its processing.

Plain VACUUM (without FULL) simply reclaims space and makes it available for re-use. This form of the command can operate in parallel with normal reading and writing of the table, as an exclusive lock is not obtained. However, extra space is not returned to the operating system (in most cases); it's just kept available for re-use within the same table. It also allows us to leverage multiple CPUs in order to process indexes. This feature is known as ???. To disable this feature, one can use PARALLEL option and specify parallel workers as zero. VACUUM FULL rewrites the entire contents of the table into a new disk file with no extra space, allowing unused space to be returned to the operating system. This form is much slower and requires an ACCESS EXCLUSIVE lock on each table while it is being processed.

Parameters

FREEZE
Selects aggressive “freezing” of tuples. Specifying FREEZE is equivalent to performing VACUUM with the ??? and ??? parameters set to zero. Aggressive freezing is always performed when the table is rewritten, so this option is redundant when FULL is specified.

VERBOSE
Prints detailed information about vacuuming each table at INFO level.

ANALYZE
Updates statistics used by the planner to determine the most efficient way to execute a query.

DISABLE_PAGE_SKIPPING
Normally, VACUUM will skip pages based on the visibility map. Pages where all tuples are known to be frozen can always be skipped, and those where all tuples are known to be visible to all transactions may be skipped except when performing an aggressive vacuum. Furthermore, except when performing an aggressive vacuum, some pages may be skipped in order to avoid waiting for other sessions to finish using them. This option disables all page-skipping behavior, and is intended to be used only when the contents of the visibility map are suspect, which should happen only if there is a hardware or software issue causing database corruption.

SKIP_LOCKED
Specifies that VACUUM should not wait for any conflicting locks to be released when beginning work on a relation: if a relation cannot be locked immediately without waiting, the relation is skipped. Note that even with this option, VACUUM may still block when opening the relation's indexes. Additionally, VACUUM ANALYZE may still block when acquiring sample rows from partitions, table inheritance children, and some types of foreign tables. Also, while VACUUM ordinarily processes all partitions of specified partitioned tables, this option will cause VACUUM to skip all partitions if there is a conflicting lock on the partitioned table.

INDEX_CLEANUP
Normally, VACUUM will skip index vacuuming when there are very few dead tuples in the table. The cost of processing all of the table's indexes is expected to greatly exceed the benefit of removing dead index tuples when this happens. This option can be used to force VACUUM to process indexes when there are more than zero dead tuples. The default is AUTO, which allows VACUUM to skip index vacuuming when appropriate. If INDEX_CLEANUP is set to ON, VACUUM will conservatively remove all dead tuples from indexes. This may be useful for backwards compatibility with earlier releases of PostgreSQL where this was the standard behavior.

INDEX_CLEANUP can also be set to OFF to force VACUUM to always skip index vacuuming, even when there are many dead tuples in the table. This may be useful when it is necessary to make VACUUM run as quickly as possible to avoid imminent transaction ID wraparound (see ???). However, the wraparound failsafe mechanism controlled by ??? will generally trigger automatically to avoid transaction ID wraparound failure, and should be preferred. If index cleanup is not performed regularly, performance may suffer, because as the table is modified indexes will accumulate dead tuples and the table itself will accumulate dead line pointers that cannot be removed until index cleanup is completed.

This option has no effect for tables that have no index and is ignored if the FULL option is used. It also has no effect on the transaction ID wraparound failsafe mechanism. When triggered it will skip index vacuuming, even when INDEX_CLEANUP is set to ON.

PROCESS_MAIN
Specifies that VACUUM should attempt to process the main relation. This is usually the desired behavior and is the default. Setting this option to false may be useful when it is only necessary to vacuum a relation's corresponding TOAST table.

PROCESS_TOAST
Specifies that VACUUM should attempt to process the corresponding TOAST table for each relation, if one exists. This is usually the desired behavior and is the default. Setting this option to false may be useful when it is only necessary to vacuum the main relation. This option is required when the FULL option is used.

TRUNCATE
Specifies that VACUUM should attempt to truncate off any empty pages at the end of the table and allow the disk space for the truncated pages to be returned to the operating system. This is normally the desired behavior and is the default unless ??? is set to false or the vacuum_truncate option has been set to false for the table to be vacuumed. Setting this option to false may be useful to avoid ACCESS EXCLUSIVE lock on the table that the truncation requires. This option is ignored if the FULL option is used.

PARALLEL
Specifies the maximum number of parallel workers that can be used for ???, which is further limited by ???. This option can't be used with the FULL option.

SKIP_DATABASE_STATS
Specifies that VACUUM should skip updating the database-wide statistics about oldest unfrozen XIDs. Normally VACUUM will update these statistics once at the end of the command. However, this can take awhile in a database with a very large number of tables, and it will accomplish nothing unless the table that had contained the oldest unfrozen XID was among those vacuumed. Moreover, if multiple VACUUM commands are issued in parallel, only one of them can update the database-wide statistics at a time. Therefore, if an application intends to issue a series of many VACUUM commands, it can be helpful to set this option in all but the last such command; or set it in all the commands and separately issue VACUUM (ONLY_DATABASE_STATS) afterwards.

ONLY_DATABASE_STATS
Specifies that VACUUM should do nothing except update the database-wide statistics about oldest unfrozen XIDs. When this option is specified, the <table_and_columns> list must be empty, and no other option may be enabled except VERBOSE.

BUFFER_USAGE_LIMIT
Specifies the Buffer Access Strategy ring buffer size for VACUUM. This size is used to calculate the number of shared buffers which will be reused as part of this strategy. 0 disables use of a Buffer Access Strategy. If ANALYZE is also specified, the BUFFER_USAGE_LIMIT value is used for both the vacuum and analyze stages. This option can't be used with the FULL option except if ANALYZE is also specified. When this option is not specified, VACUUM uses the value from ???. Higher settings can allow VACUUM to run more quickly, but having too large a setting may cause too many other useful pages to be evicted from shared buffers. The minimum value is 128 kB and the maximum value is 16 GB.

FULL
This option, which is deprecated, makes VACUUM behave like REPACK without a USING INDEX clause. This method of compacting the table takes much longer than VACUUM and exclusively locks the table. This method also requires extra disk space, since it writes a new copy of the table and doesn't release the old copy until the operation is complete. Usually this should only be used when a significant amount of space needs to be reclaimed from within the table.

<boolean>
Specifies whether the selected option should be turned on or off. You can write TRUE, ON, or 1 to enable the option, and FALSE, OFF, or 0 to disable it. The <boolean> value can also be omitted, in which case TRUE is assumed.

<integer>
Specifies a non-negative integer value passed to the selected option.

<size>
Specifies an amount of memory in kilobytes. Sizes may also be specified as a string containing the numerical size followed by any one of the following memory units: B (bytes), kB (kilobytes), MB (megabytes), GB (gigabytes), or TB (terabytes).

<table_name>
The name (optionally schema-qualified) of a specific table or materialized view to vacuum. If ONLY is specified before the table name, only that table is vacuumed. If ONLY is not specified, the table and all its inheritance child tables or partitions (if any) are also vacuumed. Optionally, * can be specified after the table name to explicitly indicate that inheritance child tables (or partitions) are to be vacuumed.

<column_name>
The name of a specific column to analyze. Defaults to all columns. If a column list is specified, ANALYZE must also be specified.

Outputs

When VERBOSE is specified, VACUUM prints detailed information about vacuuming each table at INFO level, including which table is currently being processed and various statistics about the tables.

Notes

To vacuum a table, one must ordinarily have the MAINTAIN privilege on the table. However, database owners are allowed to vacuum all tables in their databases, except shared catalogs. VACUUM will skip over any tables that the calling user does not have permission to vacuum.

While VACUUM is running, the ??? is temporarily changed to pg_catalog, pg_temp.

VACUUM cannot be executed inside a transaction block.

For tables with GIN indexes, VACUUM (in any form) also completes any pending index insertions, by moving pending index entries to the appropriate places in the main GIN index structure. See ??? for details.

We recommend that all databases be vacuumed regularly in order to remove dead rows. PostgreSQL includes an “autovacuum” facility which can automate routine vacuum maintenance. For more information about automatic and manual vacuuming, see ???.

The FULL option is not recommended for routine use, but might be useful in special cases. An example is when you have deleted or updated most of the rows in a table and would like the table to physically shrink to occupy less disk space and allow faster table scans. VACUUM FULL will usually shrink the table more than a plain VACUUM would.

The PARALLEL option is used only for vacuum purposes. If this option is specified with the ANALYZE option, it does not affect ANALYZE.

VACUUM causes a substantial increase in I/O traffic, which might cause poor performance for other active sessions. Therefore, it is sometimes advisable to use the cost-based vacuum delay feature. For parallel vacuum, each worker sleeps in proportion to the work done by that worker. See ??? for details.

Each backend running VACUUM without the FULL option will report its progress in the pg_stat_progress_vacuum view. Backends running VACUUM FULL will instead report their progress in the pg_stat_progress_cluster view. See ??? and ??? for details.

Examples

To clean a single table onek, analyze it for the optimizer and print detailed information about vacuuming and analyzing it:

VACUUM (VERBOSE, ANALYZE) onek;

Compatibility

There is no VACUUM statement in the SQL standard.

The following syntax was used before PostgreSQL version 9.0 and is still supported: VACUUM [ FULL ] [ FREEZE ] [ VERBOSE ] [ ANALYZE ] [ <table_and_columns> [, ...] ] Note that in this syntax, the options must be specified in exactly the order shown.

See Also

vacuumdb

vacuumdb

vacuumdb

1

Application

vacuumdb

garbage-collect and analyze a

PostgreSQL

database

vacuumdb

connection-option

option

-t

--table

table

(

column

[,...] )

dbname

-a

--all

vacuumdb

connection-option

option

-n

--schema

schema

dbname

-a

--all

vacuumdb

connection-option

option

-N

--exclude-schema

schema

dbname

-a

--all

Description

vacuumdb is a utility for cleaning a PostgreSQL database. vacuumdb will also generate internal statistics used by the PostgreSQL query optimizer.

vacuumdb is a wrapper around the SQL command VACUUM. There is no effective difference between vacuuming and analyzing databases via this utility and via other methods for accessing the server.

Options

vacuumdb accepts the following command-line arguments:

-a; --all
Vacuum all databases.

--buffer-usage-limit size
Specifies the Buffer Access Strategy ring buffer size for a given invocation of vacuumdb. This size is used to calculate the number of shared buffers which will be reused as part of this strategy. See ???.

-d dbname; --dbname=dbname
Specifies the name of the database to be cleaned or analyzed, when -a/--all is not used. If this is not specified, the database name is read from the environment variable PGDATABASE. If that is not set, the user name specified for the connection is used. The <dbname> can be a connection string. If so, connection string parameters will override any conflicting command line options.

--disable-page-skipping
Disable skipping pages based on the contents of the visibility map.

--dry-run
Print, but do not execute, the vacuum and analyze commands that would have been sent to the server.

-e; --echo
Echo the commands that vacuumdb generates and sends to the server.

-f; --full
Perform “full” vacuuming.

-F; --freeze
Aggressively “freeze” tuples.

--force-index-cleanup
Always remove index entries pointing to dead tuples.

-j njobs; --jobs=njobs
Execute the vacuum or analyze commands in parallel by running <njobs> commands simultaneously. This option may reduce the processing time but it also increases the load on the database server.

vacuumdb will open <njobs> connections to the database, so make sure your ??? setting is high enough to accommodate all connections.

Note that using this mode together with the -f (FULL) option might cause deadlock failures if certain system catalogs are processed in parallel.

--min-mxid-age mxid_age
Only execute the vacuum or analyze commands on tables with a multixact ID age of at least <mxid_age>. This setting is useful for prioritizing tables to process to prevent multixact ID wraparound (see ???).

For the purposes of this option, the multixact ID age of a relation is the greatest of the ages of the main relation and its associated TOAST table, if one exists. Since the commands issued by vacuumdb will also process the TOAST table for the relation if necessary, it does not need to be considered separately.

--min-xid-age xid_age
Only execute the vacuum or analyze commands on tables with a transaction ID age of at least <xid_age>. This setting is useful for prioritizing tables to process to prevent transaction ID wraparound (see ???).

For the purposes of this option, the transaction ID age of a relation is the greatest of the ages of the main relation and its associated TOAST table, if one exists. Since the commands issued by vacuumdb will also process the TOAST table for the relation if necessary, it does not need to be considered separately.

--missing-stats-only
Only analyze relations that are missing statistics for a column, index expression, or extended statistics object. When used with --analyze-in-stages, this option prevents vacuumdb from temporarily replacing existing statistics with ones generated with lower statistics targets, thus avoiding transiently worse query optimizer choices.

This option can only be used in conjunction with --analyze-only or --analyze-in-stages.

Note that --missing-stats-only requires SELECT privileges on pg_statistic and pg_statistic_ext_data, which are restricted to superusers by default.

-n schema; --schema=schema
Clean or analyze all tables in <schema> only. Multiple schemas can be vacuumed by writing multiple -n switches.

-N schema; --exclude-schema=schema
Do not clean or analyze any tables in <schema>. Multiple schemas can be excluded by writing multiple -N switches.

--no-index-cleanup
Do not remove index entries pointing to dead tuples.

--no-process-main
Skip the main relation.

--no-process-toast
Skip the TOAST table associated with the table to vacuum, if any.

--no-truncate
Do not truncate empty pages at the end of the table.

-P parallel_workers; --parallel=parallel_workers
Specify the number of parallel workers for parallel vacuum. This allows the vacuum to leverage multiple CPUs to process indexes. See ???.

-q; --quiet
Do not display progress messages.

--skip-locked
Skip relations that cannot be immediately locked for processing.

-t table [ (column [,...]) ]; --table=table [ (column [,...]) ]
Clean or analyze <table> only. Column names can be specified only in conjunction with the --analyze or --analyze-only options. Multiple tables can be vacuumed by writing multiple -t switches.

If no tables are specified with the --table option, vacuumdb will clean all regular tables and materialized views in the connected database. If --analyze-only or --analyze-in-stages is also specified, it will analyze all regular tables, partitioned tables, and materialized views (but not foreign tables).

[!TIP] If you specify columns, you probably have to escape the parentheses from the shell. (See examples below.)

-v; --verbose
Print detailed information during processing.

-V; --version
Print the vacuumdb version and exit.

-z; --analyze
Also calculate statistics for use by the optimizer.

-Z; --analyze-only
Only calculate statistics for use by the optimizer (no vacuum).

--analyze-in-stages
Only calculate statistics for use by the optimizer (no vacuum), like --analyze-only. Run three stages of analyze; the first stage uses the lowest possible statistics target (see ???) to produce usable statistics faster, and subsequent stages build the full statistics.

This option is only useful to analyze a database that currently has no statistics or has wholly incorrect ones, such as if it is newly populated from a restored dump or by pg_upgrade. Be aware that running with this option in a database with existing statistics may cause the query optimizer choices to become transiently worse due to the low statistics targets of the early stages.

-?; --help
Show help about vacuumdb command line arguments, and exit.

vacuumdb also accepts the following command-line arguments for connection parameters:

-h host; --host=host
Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for the Unix domain socket.

-p port; --port=port
Specifies the TCP port or local Unix domain socket file extension on which the server is listening for connections.

-U username; --username=username
User name to connect as.

-w; --no-password
Never issue a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password.

-W; --password
Force vacuumdb to prompt for a password before connecting to a database.

This option is never essential, since vacuumdb will automatically prompt for a password if the server demands password authentication. However, vacuumdb will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt.

--maintenance-db=dbname
When the -a/--all is used, connect to this database to gather the list of databases to vacuum. If not specified, the postgres database will be used, or if that does not exist, template1 will be used. This can be a connection string. If so, connection string parameters will override any conflicting command line options. Also, connection string parameters other than the database name itself will be re-used when connecting to other databases.

Environment

PGDATABASE; PGHOST; PGPORT; PGUSER
Default connection parameters

PG_COLOR
Specifies whether to use color in diagnostic messages. Possible values are always, auto and never.

This utility, like most other PostgreSQL utilities, also uses the environment variables supported by libpq (see ???).

Diagnostics

In case of difficulty, see ??? and ??? for discussions of potential problems and error messages. The database server must be running at the targeted host. Also, any default connection settings and environment variables used by the libpq front-end library will apply.

Examples

To clean the database test:

$ vacuumdb test

To clean and analyze for the optimizer a database named bigdb:

$ vacuumdb --analyze bigdb

To clean a single table foo in a database named xyzzy, and analyze a single column bar of the table for the optimizer:

$ vacuumdb --analyze --verbose --table='foo(bar)' xyzzy

To clean all tables in the foo and bar schemas in a database named xyzzy:

$ vacuumdb --schema='foo' --schema='bar' xyzzy

See Also

values

VALUES

VALUES

7

SQL - Language Statements

VALUES

compute a set of rows

VALUES (

expression

[, ...] ) [, ...] [ ORDER BY

sort_expression

[ ASC | DESC | USING

operator

] [, ...] ] [ LIMIT {

count

| ALL } ] [ OFFSET

start

[ ROW | ROWS ] ] [ FETCH { FIRST | NEXT } [

count

] { ROW | ROWS } ONLY ]

Description

VALUES computes a row value or set of row values specified by value expressions. It is most commonly used to generate a “constant table” within a larger command, but it can be used on its own.

When more than one row is specified, all the rows must have the same number of elements. The data types of the resulting table's columns are determined by combining the explicit or inferred types of the expressions appearing in that column, using the same rules as for UNION (see ???).

Within larger commands, VALUES is syntactically allowed anywhere that SELECT is. Because it is treated like a SELECT by the grammar, it is possible to use the ORDER BY, LIMIT (or equivalently FETCH FIRST), and OFFSET clauses with a VALUES command.

Parameters

<expression>
A constant or expression to compute and insert at the indicated place in the resulting table (set of rows). In a VALUES list appearing at the top level of an INSERT, an <expression> can be replaced by DEFAULT to indicate that the destination column's default value should be inserted. DEFAULT cannot be used when VALUES appears in other contexts.

<sort_expression>
An expression or integer constant indicating how to sort the result rows. This expression can refer to the columns of the VALUES result as column1, column2, etc. For more details see ??? in the ??? documentation.

<operator>
A sorting operator. For details see ??? in the ??? documentation.

<count>
The maximum number of rows to return. For details see ??? in the ??? documentation.

<start>
The number of rows to skip before starting to return rows. For details see ??? in the ??? documentation.

Notes

VALUES lists with very large numbers of rows should be avoided, as you might encounter out-of-memory failures or poor performance. VALUES appearing within INSERT is a special case (because the desired column types are known from the INSERT's target table, and need not be inferred by scanning the VALUES list), so it can handle larger lists than are practical in other contexts.

Examples

A bare VALUES command:

VALUES (1, 'one'), (2, 'two'), (3, 'three');

This will return a table of two columns and three rows. It's effectively equivalent to:

SELECT 1 AS column1, 'one' AS column2
UNION ALL
SELECT 2, 'two'
UNION ALL
SELECT 3, 'three';

More usually, VALUES is used within a larger SQL command. The most common use is in INSERT:

INSERT INTO films (code, title, did, date_prod, kind)
    VALUES ('T_601', 'Yojimbo', 106, '1961-06-16', 'Drama');

In the context of INSERT, entries of a VALUES list can be DEFAULT to indicate that the column default should be used here instead of specifying a value:

INSERT INTO films VALUES
    ('UA502', 'Bananas', 105, DEFAULT, 'Comedy', '82 minutes'),
    ('T_601', 'Yojimbo', 106, DEFAULT, 'Drama', DEFAULT);

VALUES can also be used where a sub-SELECT might be written, for example in a FROM clause:

SELECT f.*
  FROM films f, (VALUES('MGM', 'Horror'), ('UA', 'Sci-Fi')) AS t (studio, kind)
  WHERE f.studio = t.studio AND f.kind = t.kind;

UPDATE employees SET salary = salary * v.increase
  FROM (VALUES(1, 200000, 1.2), (2, 400000, 1.4)) AS v (depno, target, increase)
  WHERE employees.depno = v.depno AND employees.sales >= v.target;

Note that if VALUES is used in a FROM clause, a table alias is optional. Assigning alias names to the columns of the VALUES list is optional, but is good practice. (The default column names for VALUES are column1, column2, etc. in PostgreSQL, but these names might be different in other database systems.)

When VALUES is used in INSERT, the values are all automatically coerced to the data type of the corresponding destination column. When it's used in other contexts, it might be necessary to specify the correct data type. If the entries are all quoted literal constants, coercing the first is sufficient to determine the assumed type for all:

SELECT * FROM machines
WHERE ip_address IN (VALUES('192.168.0.1'::inet), ('192.168.0.10'), ('192.168.1.43'));

[!TIP] For simple IN tests, it's better to rely on the list-of-scalars form of IN than to write a VALUES query as shown above. The list of scalars method requires less writing and is often more efficient.

Compatibility

VALUES conforms to the SQL standard. LIMIT and OFFSET are PostgreSQL extensions; see also under ???.

See Also

wait

WAIT

WAIT

7

SQL - Language Statements

WAIT

wait for WAL to reach a target

LSN

WAIT FOR LSN '

lsn

' [ WITH (

option

[, ...] ) ]

where

option

can be:

MODE '

mode

' TIMEOUT '

timeout

' NO_THROW

and

mode

can be:

standby_replay | standby_write | standby_flush | primary_flush

Description

Waits until the specified lsn is reached according to the specified mode, which determines whether to wait for WAL to be written, flushed, or replayed. If no timeout is specified or it is set to zero, this command waits indefinitely for the lsn.

On timeout, an error is emitted unless NO_THROW is specified in the WITH clause. For standby modes (standby_replay, standby_write, standby_flush), an error is also emitted if the server is promoted before the lsn is reached. If NO_THROW is specified, the command reports these outcomes as a status string instead of raising an error.

The possible return values are success, timeout, and not in recovery.

Parameters

<lsn>
Specifies the target LSN to wait for.

WITH ( option [, ...] )
This clause specifies optional parameters for the wait operation. The following parameters are supported:

MODE '<mode>'
Specifies the type of LSN processing to wait for. If not specified, the default is standby_replay. The valid modes are:

TIMEOUT '<timeout>'
When specified and timeout is greater than zero, the command waits until lsn is reached or the specified timeout has elapsed. A value of zero (the default) means the command waits indefinitely.

The timeout is an amount of time in milliseconds. It may also be specified as a string containing the numerical value followed by a time unit (see ???). The maximum value is 2147483647 ms.

Fractional values are rounded to the nearest millisecond. Note that a timeout of half a millisecond or less therefore rounds down to zero, which means waiting indefinitely.

NO_THROW
Specify to not throw an error in the case of timeout or running on the primary. In this case the result status can be obtained from the return value.

Use this option when timeout or not in recovery is an expected result that the application intends to handle, for example by retrying the wait, reporting replication delay, or choosing another server for a subsequent operation. The command then returns the result as a status, which the application must check before assuming that the target LSN was reached. Omit the option when the application must not proceed unless the target LSN is reached, so that an unsuccessful wait stops normal execution with an error.

Returning a status also leaves an explicit transaction usable; without this option, the corresponding error requires rolling back the transaction, or rolling back to a savepoint, before further commands can be issued.

This option changes only how timeout and not in recovery are reported. Other errors are still raised. That covers invalid input, such as a malformed LSN or an unrecognized option value, and every condition that is checked before the wait begins, such as requesting primary_flush during recovery or holding a lock while waiting for a standby LSN. The option also does not limit the duration of the wait; specify TIMEOUT for that purpose.

Outputs

success
This return value denotes that we have successfully reached the target lsn.

timeout
This return value denotes that the timeout happened before reaching the target lsn.

not in recovery
This return value denotes that the database server is not in a recovery state. This might mean either the database server was not in recovery at the moment of receiving the command (i.e., executed on a primary), or it was promoted before reaching the target lsn. In the promotion case, this status indicates a timeline change occurred, and the application should re-evaluate whether the target LSN is still relevant.

Notes

WAIT must be executed as a top-level command. It cannot be executed from a function, procedure, or DO block. It also cannot be executed while the current transaction holds a snapshot. WAIT itself acquires none, so it can run before the first snapshot-taking statement of a REPEATABLE READ or SERIALIZABLE transaction, but not after it, and not while a cursor or an exported snapshot holds one at any isolation level. A snapshot held here could delay replay, which standby_replay waits for and which other standby modes can end up waiting for too. That is also why WAIT is a command rather than a function or a procedure, which execute with one held.

While recovery is in progress, a wait in standby_replay (the default), standby_write, or standby_flush mode is rejected when the session already holds a lock and the target lsn has not been reached yet. Such a lock can make the startup process wait for this session, either directly or through another session, while this session waits for the startup process to advance recovery. That cycle involves no lock wait on this side, so deadlock detection does not see it and nothing breaks it. A wait whose target has already been reached returns immediately and is therefore always allowed.

Issue WAIT FOR outside a transaction block, or as the first statement of one, before running anything that takes locks. That is also the natural order for the read-your-writes pattern shown in the examples below: wait for the target lsn first, then run the queries that have to see it. Note that a lock taken by an earlier statement is still held at READ COMMITTED, even though its snapshot is gone, so a wait placed after such a statement is rejected even when the isolation level permits it.

The restriction covers standby_write and standby_flush as well, even though streaming replication can advance those positions without the startup process. Both positions are at least the replay position, so without an active walreceiver replay can be their only source of progress. If a held lock blocks replay, the session waits for replay to advance while replay waits for the session to release the lock. Under streaming replication the positions advance independently only while WAL keeps arriving. If reception stops before the target is reached, a blocked startup process cannot restart the walreceiver. It also cannot replay newer checkpoint records needed to advance restartpoints and recycle WAL, so pg_wal can fill up and reception can stop before the target is reached. The restriction therefore also applies when streaming is active at the start of the wait.

WAIT waits until the specified lsn is reached according to the specified mode. The standby_replay mode waits for the LSN to be replayed (applied to the database), which is useful to achieve read-your-writes consistency while using an async replica for reads and the primary for writes, provided that the target LSN is at or after the end of the relevant write transaction's COMMIT record on the primary. The standby_flush mode waits for the WAL to be flushed to durable storage on the replica, or to have already been replayed from WAL present on the standby. The standby_write mode waits for the WAL to be written to the operating system, or to have already been replayed, which is faster than flush for newly received WAL but provides weaker durability guarantees. The primary_flush mode waits for WAL to be flushed on a primary server. In all cases, the LSN of the last modification should be stored on the client application side or the connection pooler side.

The standby modes (standby_replay, standby_write, standby_flush) can only be used during recovery, and primary_flush can only be used on a primary server. Using the wrong mode for the current server state will result in an error. If a standby is promoted while waiting with a standby mode, the command will return not in recovery (or throw an error if NO_THROW is not specified). Promotion creates a new timeline, and the LSN being waited for may refer to WAL from the old timeline.

WAIT compares only the numeric LSN; it has no notion of which timeline a WAL record belongs to. This matters when a standby continues recovery across an upstream timeline switch — for example, a cascading standby whose upstream gets promoted. In that case WAIT will return success as soon as the position used by the selected wait mode reaches or passes the numeric LSN, regardless of which timeline that LSN belongs to. Applications that need to confirm the target refers to the expected timeline must validate the timeline themselves.

On a standby server, WAIT sessions may be interrupted by recovery conflicts. Some recovery conflicts are unavoidable: for example, replaying a tablespace drop resolves conflicts by terminating all backends, regardless of what they are doing. Applications using WAIT on a standby should be prepared to handle such interruptions, for example by retrying the command or falling back to an alternative mechanism.

Examples

You can use the WAIT command to wait for the pg_lsn value. For example, an application could update the movie table and get an lsn that is at or after the end of the relevant write transaction's COMMIT record. In the default autocommit mode shown here, the UPDATE commits before the subsequent SELECT. This example uses pg_current_wal_insert_lsn on primary server to get the lsn given that synchronous_commit could be set to off.

postgres=# UPDATE movie SET genre = 'Dramatic' WHERE genre = 'Drama';
UPDATE 100
postgres=# SELECT pg_current_wal_insert_lsn();
 pg_current_wal_insert_lsn
---------------------------
 0/0306EE20
(1 row)

Then an application could run WAIT with the lsn obtained from the primary after the commit. After that, the changes made on the primary should be guaranteed to be visible on the replica.

postgres=# WAIT FOR LSN '0/0306EE20';
 status
---------
 success
(1 row)
postgres=# SELECT * FROM movie WHERE genre = 'Drama';
 genre
-------
(0 rows)

Wait for flush (data durable on replica):

postgres=# WAIT FOR LSN '0/0306EE20' WITH (MODE 'standby_flush');
 status
---------
 success
(1 row)

Wait for write with timeout:

postgres=# WAIT FOR LSN '0/0306EE20' WITH (MODE 'standby_write', TIMEOUT '100ms', NO_THROW);
 status
---------
 success
(1 row)

Wait for flush on primary:

postgres=# WAIT FOR LSN '0/0306EE20' WITH (MODE 'primary_flush');
 status
---------
 success
(1 row)

If the target LSN is not reached before the timeout, an error is thrown:

postgres=# WAIT FOR LSN '0/0306EE20' WITH (TIMEOUT '0.1s');
ERROR:  timed out while waiting for target LSN 0/0306EE20 to be replayed; current standby_replay LSN 0/0306EA60

The same example uses WAIT with the NO_THROW option:

postgres=# WAIT FOR LSN '0/0306EE20' WITH (TIMEOUT '100ms', NO_THROW);
 status
---------
 timeout
(1 row)

Database Roles

PostgreSQL manages database access permissions using the concept of roles. A role can be thought of as either a database user, or a group of database users, depending on how the role is set up. Roles can own database objects (for example, tables and functions) and can assign privileges on those objects to other roles to control who has access to which objects. Furthermore, it is possible to grant membership in a role to another role, thus allowing the member role to use privileges assigned to another role.

The concept of roles subsumes the concepts of “users” and “groups”. In PostgreSQL versions before 8.1, users and groups were distinct kinds of entities, but now there are only roles. Any role can act as a user, a group, or both.

This chapter describes how to create and manage roles. More information about the effects of role privileges on various database objects can be found in ???.

Database Roles

role

user

CREATE ROLE

DROP ROLE

Database roles are conceptually completely separate from operating system users. In practice it might be convenient to maintain a correspondence, but this is not required. Database roles are global across a database cluster installation (and not per individual database). To create a role use the CREATE ROLE SQL command: CREATE ROLE <name>; <name> follows the rules for SQL identifiers: either unadorned without special characters, or double-quoted. (In practice, you will usually want to add additional options, such as LOGIN, to the command. More details appear below.) To remove an existing role, use the analogous DROP ROLE command: DROP ROLE <name>;

createuser

dropuser

For convenience, the programs ??? and ??? are provided as wrappers around these SQL commands that can be called from the shell command line: createuser <name> dropuser <name>

To determine the set of existing roles, examine the pg_roles system catalog, for example: SELECT rolname FROM pg_roles; or to see just those capable of logging in: SELECT rolname FROM pg_roles WHERE rolcanlogin; The ??? program's \du meta-command is also useful for listing the existing roles.

In order to bootstrap the database system, a freshly initialized system always contains one predefined login-capable role. This role is always a “superuser”, and it will have the same name as the operating system user that initialized the database cluster with initdb unless a different name is specified. This role is often named postgres. In order to create more roles you first have to connect as this initial role.

Every connection to the database server is made using the name of some particular role, and this role determines the initial access privileges for commands issued in that connection. The role name to use for a particular database connection is indicated by the client that is initiating the connection request in an application-specific fashion. For example, the psql program uses the -U command line option to indicate the role to connect as. Many applications assume the name of the current operating system user by default (including createuser and psql). Therefore it is often convenient to maintain a naming correspondence between roles and operating system users.

The set of database roles a given client connection can connect as is determined by the client authentication setup, as explained in ???. (Thus, a client is not limited to connect as the role matching its operating system user, just as a person's login name need not match his or her real name.) Since the role identity determines the set of privileges available to a connected client, it is important to carefully configure privileges when setting up a multiuser environment.

Role Attributes

A database role can have a number of attributes that define its privileges and interact with the client authentication system.

login privilege
Only roles that have the LOGIN attribute can be used as the initial role name for a database connection. A role with the LOGIN attribute can be considered the same as a “database user”. To create a role with login privilege, use either:

CREATE ROLE name LOGIN;
CREATE USER name;

(CREATE USER is equivalent to CREATE ROLE except that CREATE USER includes LOGIN by default, while CREATE ROLE does not.)

superuser status
A database superuser bypasses all permission checks, except the right to log in. This is a dangerous privilege and should not be used carelessly; it is best to do most of your work as a role that is not a superuser. To create a new database superuser, use CREATE ROLE name SUPERUSER. You must do this as a role that is already a superuser.

database creation
A role must be explicitly given permission to create databases (except for superusers, since those bypass all permission checks). To create such a role, use CREATE ROLE name CREATEDB.

role creation
A role must be explicitly given permission to create more roles (except for superusers, since those bypass all permission checks). To create such a role, use CREATE ROLE name CREATEROLE. A role with CREATEROLE privilege can alter and drop roles which have been granted to the CREATEROLE user with the ADMIN option. Such a grant occurs automatically when a CREATEROLE user that is not a superuser creates a new role, so that by default, a CREATEROLE user can alter and drop the roles which they have created. Altering a role includes most changes that can be made using ALTER ROLE, including, for example, changing passwords. It also includes modifications to a role that can be made using the COMMENT and SECURITY LABEL commands.

However, CREATEROLE does not convey the ability to create SUPERUSER roles, nor does it convey any power over SUPERUSER roles that already exist. Furthermore, CREATEROLE does not convey the power to create REPLICATION users, nor the ability to grant or revoke the REPLICATION privilege, nor the ability to modify the role properties of such users. However, it does allow ALTER ROLE ... SET and ALTER ROLE ... RENAME to be used on REPLICATION roles, as well as the use of COMMENT ON ROLE, SECURITY LABEL ON ROLE, and DROP ROLE. Finally, CREATEROLE does not confer the ability to grant or revoke the BYPASSRLS privilege.

initiating replication
A role must explicitly be given permission to initiate streaming replication (except for superusers, since those bypass all permission checks). A role used for streaming replication must have LOGIN permission as well. To create such a role, use CREATE ROLE name REPLICATION LOGIN.

password
A password is only significant if the client authentication method requires the user to supply a password when connecting to the database. The password and md5 authentication methods make use of passwords. Database passwords are separate from operating system passwords. Specify a password upon role creation with CREATE ROLE name PASSWORD 'string'.

inheritance of privileges
A role inherits the privileges of roles it is a member of, by default. However, to create a role which does not inherit privileges by default, use CREATE ROLE name NOINHERIT. Alternatively, inheritance can be overridden for individual grants by using WITH INHERIT TRUE or WITH INHERIT FALSE.

bypassing row-level security
A role must be explicitly given permission to bypass every row-level security (RLS) policy (except for superusers, since those bypass all permission checks). To create such a role, use CREATE ROLE name BYPASSRLS as a superuser.

connection limit
Connection limit can specify how many concurrent connections a role can make. -1 (the default) means no limit. Specify connection limit upon role creation with CREATE ROLE name CONNECTION LIMIT 'integer'.

A role's attributes can be modified after creation with ALTER ROLE. See the reference pages for the ??? and ??? commands for details.

A role can also have role-specific defaults for many of the run-time configuration settings described in ???. For example, if for some reason you want to disable index scans (hint: not a good idea) anytime you connect, you can use:

ALTER ROLE myname SET enable_indexscan TO off;

This will save the setting (but not set it immediately). In subsequent connections by this role it will appear as though SET enable_indexscan TO off had been executed just before the session started. You can still alter this setting during the session; it will only be the default. To remove a role-specific default setting, use ALTER ROLE rolename RESET varname. Note that role-specific defaults attached to roles without LOGIN privilege are fairly useless, since they will never be invoked.

When a non-superuser creates a role using the CREATEROLE privilege, the created role is automatically granted back to the creating user, just as if the bootstrap superuser had executed the command GRANT created_user TO creating_user WITH ADMIN TRUE, SET FALSE, INHERIT FALSE. Since a CREATEROLE user can only exercise special privileges with regard to an existing role if they have ADMIN OPTION on it, this grant is just sufficient to allow a CREATEROLE user to administer the roles they created. However, because it is created with INHERIT FALSE, SET FALSE, the CREATEROLE user doesn't inherit the privileges of the created role, nor can it access the privileges of that role using SET ROLE. However, since any user who has ADMIN OPTION on a role can grant membership in that role to any other user, the CREATEROLE user can gain access to the created role by simply granting that role back to themselves with the INHERIT and/or SET options. Thus, the fact that privileges are not inherited by default nor is SET ROLE granted by default is a safeguard against accidents, not a security feature. Also note that, because this automatic grant is granted by the bootstrap superuser, it cannot be removed or changed by the CREATEROLE user; however, any superuser could revoke it, modify it, and/or issue additional such grants to other CREATEROLE users. Whichever CREATEROLE users have ADMIN OPTION on a role at any given time can administer it.

Role Membership

role

membership in

It is frequently convenient to group users together to ease management of privileges: that way, privileges can be granted to, or revoked from, a group as a whole. In PostgreSQL this is done by creating a role that represents the group, and then granting membership in the group role to individual user roles.

To set up a group role, first create the role: CREATE ROLE <name>; Typically a role being used as a group would not have the LOGIN attribute, though you can set it if you wish.

Once the group role exists, you can add and remove members using the GRANT and REVOKE commands: GRANT <group_role> TO <role1>, ... ; REVOKE <group_role> FROM <role1>, ... ; You can grant membership to other group roles, too (since there isn't really any distinction between group roles and non-group roles). The database will not let you set up circular membership loops. Also, it is not permitted to grant membership in a role to PUBLIC.

The members of a group role can use the privileges of the role in two ways. First, member roles that have been granted membership with the SET option can do SET ROLE to temporarily “become” the group role. In this state, the database session has access to the privileges of the group role rather than the original login role, and any database objects created are considered owned by the group role, not the login role. Second, member roles that have been granted membership with the INHERIT option automatically have use of the privileges of those directly or indirectly a member of, though the chain stops at memberships lacking the inherit option. As an example, suppose we have done:

CREATE ROLE joe LOGIN;
CREATE ROLE admin;
CREATE ROLE wheel;
CREATE ROLE island;
GRANT admin TO joe WITH INHERIT TRUE;
GRANT wheel TO admin WITH INHERIT FALSE;
GRANT island TO joe WITH INHERIT TRUE, SET FALSE;

Immediately after connecting as role joe, a database session will have use of privileges granted directly to joe plus any privileges granted to admin and island, because joe “inherits” those privileges. However, privileges granted to wheel are not available, because even though joe is indirectly a member of wheel, the membership is via admin which was granted using WITH INHERIT FALSE. After:

SET ROLE admin;

the session would have use of only those privileges granted to admin, and not those granted to joe or island. After:

SET ROLE wheel;

the session would have use of only those privileges granted to wheel, and not those granted to either joe or admin. The original privilege state can be restored with any of:

SET ROLE joe;
SET ROLE NONE;
RESET ROLE;

[!NOTE] The SET ROLE command always allows selecting any role that the original login role is directly or indirectly a member of, provided that there is a chain of membership grants each of which has SET TRUE (which is the default). Thus, in the above example, it is not necessary to become admin before becoming wheel. On the other hand, it is not possible to become island at all; joe can only access those privileges via inheritance.

[!NOTE] In the SQL standard, there is a clear distinction between users and roles, and users do not automatically inherit privileges while roles do. This behavior can be obtained in PostgreSQL by giving roles being used as SQL roles the INHERIT attribute, while giving roles being used as SQL users the NOINHERIT attribute. However, PostgreSQL defaults to giving all roles the INHERIT attribute, for backward compatibility with pre-8.1 releases in which users always had use of permissions granted to groups they were members of.

The role attributes LOGIN, SUPERUSER, CREATEDB, and CREATEROLE can be thought of as special privileges, but they are never inherited as ordinary privileges on database objects are. You must actually SET ROLE to a specific role having one of these attributes in order to make use of the attribute. Continuing the above example, we might choose to grant CREATEDB and CREATEROLE to the admin role. Then a session connecting as role joe would not have these privileges immediately, only after doing SET ROLE admin.

To destroy a group role, use DROP ROLE: DROP ROLE <name>; Any memberships in the group role are automatically revoked (but the member roles are not otherwise affected).

Dropping Roles

Because roles can own database objects and can hold privileges to access other objects, dropping a role is often not just a matter of a quick DROP ROLE. Any objects owned by the role must first be dropped or reassigned to other owners; and any permissions granted to the role must be revoked.

Ownership of objects can be transferred one at a time using ALTER commands, for example:

ALTER TABLE bobs_table OWNER TO alice;

Alternatively, the REASSIGN OWNED command can be used to reassign ownership of all objects owned by the role-to-be-dropped to a single other role. Because REASSIGN OWNED cannot access objects in other databases, it is necessary to run it in each database that contains objects owned by the role. (Note that the first such REASSIGN OWNED will change the ownership of any shared-across-databases objects, that is databases or tablespaces, that are owned by the role-to-be-dropped. Subscriptions, while stored in a shared catalog, belong to a specific database and are reassigned only when REASSIGN OWNED is executed in that database.)

Once any valuable objects have been transferred to new owners, any remaining objects owned by the role-to-be-dropped can be dropped with the DROP OWNED command. Again, this command cannot access objects in other databases, so it is necessary to run it in each database that contains objects owned by the role. Also, DROP OWNED will not drop entire databases or tablespaces, so it is necessary to do that manually if the role owns any databases or tablespaces that have not been transferred to new owners.

DROP OWNED also takes care of removing any privileges granted to the target role for objects that do not belong to it. Because REASSIGN OWNED does not touch such objects, it's typically necessary to run both REASSIGN OWNED and DROP OWNED (in that order!) to fully remove the dependencies of a role to be dropped.

In short then, the most general recipe for removing a role that has been used to own objects is:

REASSIGN OWNED BY doomed_role TO successor_role;
DROP OWNED BY doomed_role;
-- repeat the above commands in each database of the cluster
DROP ROLE doomed_role;

When not all owned objects are to be transferred to the same successor owner, it's best to handle the exceptions manually and then perform the above steps to mop up.

If DROP ROLE is attempted while dependent objects still remain, it will issue messages identifying which objects need to be reassigned or dropped.

Predefined Roles

role

PostgreSQL provides a set of predefined roles that provide access to certain, commonly needed, privileged capabilities and information. Administrators (including roles that have the CREATEROLE privilege) can GRANT these roles to users and/or other roles in their environment, providing those users with access to the specified capabilities and information. For example:

GRANT pg_signal_backend TO admin_user;

[!WARNING] Care should be taken when granting these roles to ensure they are only used where needed and with the understanding that these roles grant access to privileged information.

The predefined roles are described below. Note that the specific permissions for each of the roles may change in the future as additional capabilities are added. Administrators should monitor the release notes for changes.

pg_checkpoint
pg_checkpoint allows executing the CHECKPOINT command.

pg_create_subscription
pg_create_subscription allows users with CREATE permission on the database to issue CREATE SUBSCRIPTION.

pg_database_owner
pg_database_owner always has exactly one implicit member: the current database owner. It cannot be granted membership in any role, and no role can be granted membership in pg_database_owner. However, like any other role, it can own objects and receive grants of access privileges. Consequently, once pg_database_owner has rights within a template database, each owner of a database instantiated from that template will possess those rights. Initially, this role owns the public schema, so each database owner governs local use of that schema.

pg_maintain
pg_maintain allows executing VACUUM, ANALYZE, CLUSTER, REPACK, REFRESH MATERIALIZED VIEW, REINDEX, and LOCK TABLE on all relations, as if having MAINTAIN rights on those objects.

pg_monitor; pg_read_all_settings; pg_read_all_stats; pg_stat_scan_tables
These roles are intended to allow administrators to easily configure a role for the purpose of monitoring the database server. They grant a set of common privileges allowing the role to read various useful configuration settings, statistics, and other system information normally restricted to superusers.

pg_monitor allows reading/executing various monitoring views and functions. This role is a member of pg_read_all_settings, pg_read_all_stats and pg_stat_scan_tables.

pg_read_all_settings allows reading all configuration variables, even those normally visible only to superusers.

pg_read_all_stats allows reading all pg_stat_* views and use various statistics related extensions, even those normally visible only to superusers.

pg_stat_scan_tables allows executing monitoring functions that may take ACCESS SHARE locks on tables, potentially for a long time (e.g., pgrowlocks(text) in the ??? extension).

pg_read_all_data; pg_write_all_data
pg_read_all_data allows reading all data (tables, views, sequences, large objects), as if having SELECT rights on those objects and USAGE rights on all schemas. This role does not bypass row-level security (RLS) policies. If RLS is being used, an administrator may wish to set BYPASSRLS on roles which this role is granted to.

pg_write_all_data allows writing all data (tables, views, sequences, large objects), as if having INSERT, UPDATE, and DELETE rights on those objects and USAGE rights on all schemas. This role does not bypass row-level security (RLS) policies. If RLS is being used, an administrator may wish to set BYPASSRLS on roles which this role is granted to.

pg_read_server_files; pg_write_server_files; pg_execute_server_program
These roles are intended to allow administrators to have trusted, but non-superuser, roles which are able to access files and run programs on the database server as the user the database runs as. They bypass all database-level permission checks when accessing files directly and they could be used to gain superuser-level access. Therefore, great care should be taken when granting these roles to users.

pg_read_server_files allows reading files from any location the database can access on the server using COPY and other file-access functions.

pg_write_server_files allows writing to files in any location the database can access on the server using COPY and other file-access functions.

pg_execute_server_program allows executing programs on the database server as the user the database runs as using COPY and other functions which allow executing a server-side program.

pg_signal_autovacuum_worker
pg_signal_autovacuum_worker allows signaling autovacuum workers to cancel the current table's vacuum or terminate its session. See ???.

pg_signal_backend
pg_signal_backend allows signaling another backend to cancel a query or terminate its session. Note that this role does not permit signaling backends owned by a superuser. See ???.

pg_use_reserved_connections
pg_use_reserved_connections allows use of connection slots reserved via ???.

Function Security

Functions, triggers and row-level security policies allow users to insert code into the backend server that other users might execute unintentionally. Hence, these mechanisms permit users to “Trojan horse” others with relative ease. The strongest protection is tight control over who can define objects. Where that is infeasible, write queries referring only to objects having trusted owners. Remove from search_path any schemas that permit untrusted users to create objects.

Functions run inside the backend server process with the operating system permissions of the database server daemon. If the programming language used for the function allows unchecked memory accesses, it is possible to change the server's internal data structures. Hence, among many other things, such functions can circumvent any system access controls. Function languages that allow such access are considered “untrusted”, and PostgreSQL allows only superusers to create functions written in those languages.

Reliability and the Write-Ahead Log

This chapter explains how to control the reliability of PostgreSQL, including details about the Write-Ahead Log.

Reliability

Reliability is an important property of any serious database system, and PostgreSQL does everything possible to guarantee reliable operation. One aspect of reliable operation is that all data recorded by a committed transaction should be stored in a nonvolatile area that is safe from power loss, operating system failure, and hardware failure (except failure of the nonvolatile area itself, of course). Successfully writing the data to the computer's permanent storage (disk drive or equivalent) ordinarily meets this requirement. In fact, even if a computer is fatally damaged, if the disk drives survive they can be moved to another computer with similar hardware and all committed transactions will remain intact.

While forcing data to the disk platters periodically might seem like a simple operation, it is not. Because disk drives are dramatically slower than main memory and CPUs, several layers of caching exist between the computer's main memory and the disk platters. First, there is the operating system's buffer cache, which caches frequently requested disk blocks and combines disk writes. Fortunately, all operating systems give applications a way to force writes from the buffer cache to disk, and PostgreSQL uses those features. (See the ??? parameter to adjust how this is done.)

Next, there might be a cache in the disk drive controller; this is particularly common on RAID controller cards. Some of these caches are write-through, meaning writes are sent to the drive as soon as they arrive. Others are write-back, meaning data is sent to the drive at some later time. Such caches can be a reliability hazard because the memory in the disk controller cache is volatile, and will lose its contents in a power failure. Better controller cards have battery-backup units (BBUs), meaning the card has a battery that maintains power to the cache in case of system power loss. After power is restored the data will be written to the disk drives.

And finally, most disk drives have caches. Some are write-through while some are write-back, and the same concerns about data loss exist for write-back drive caches as for disk controller caches. Consumer-grade IDE and SATA drives are particularly likely to have write-back caches that will not survive a power failure. Many solid-state drives (SSD) also have volatile write-back caches.

These caches can typically be disabled; however, the method for doing this varies by operating system and drive type:

Recent SATA drives (those following ATAPI-6 or later) offer a drive cache flush command (FLUSH CACHE EXT), while SCSI drives have long supported a similar command SYNCHRONIZE CACHE. These commands are not directly accessible to PostgreSQL, but some file systems (e.g., ZFS, ext4) can use them to flush data to the platters on write-back-enabled drives. Unfortunately, such file systems behave suboptimally when combined with battery-backup unit (BBU) disk controllers. In such setups, the synchronize command forces all data from the controller cache to the disks, eliminating much of the benefit of the BBU. You can run the ??? program to see if you are affected. If you are affected, the performance benefits of the BBU can be regained by turning off write barriers in the file system or reconfiguring the disk controller, if that is an option. If write barriers are turned off, make sure the battery remains functional; a faulty battery can potentially lead to data loss. Hopefully file system and disk controller designers will eventually address this suboptimal behavior.

When the operating system sends a write request to the storage hardware, there is little it can do to make sure the data has arrived at a truly non-volatile storage area. Rather, it is the administrator's responsibility to make certain that all storage components ensure integrity for both data and file-system metadata. Avoid disk controllers that have non-battery-backed write caches. At the drive level, disable write-back caching if the drive cannot guarantee the data will be written before shutdown. If you use SSDs, be aware that many of these do not honor cache flush commands by default. You can test for reliable I/O subsystem behavior using diskchecker.pl.

Another risk of data loss is posed by the disk platter write operations themselves. Disk platters are divided into sectors, commonly 512 bytes each. Every physical read or write operation processes a whole sector. When a write request arrives at the drive, it might be for some multiple of 512 bytes (PostgreSQL typically writes 8192 bytes, or 16 sectors, at a time), and the process of writing could fail due to power loss at any time, meaning some of the 512-byte sectors were written while others were not. To guard against such failures, PostgreSQL periodically writes full page images to permanent WAL storage before modifying the actual page on disk. By doing this, during crash recovery PostgreSQL can restore partially-written pages from WAL. If you have file-system software that prevents partial page writes (e.g., ZFS), you can turn off this page imaging by turning off the ??? parameter. Battery-Backed Unit (BBU) disk controllers do not prevent partial page writes unless they guarantee that data is written to the BBU as full (8kB) pages.

PostgreSQL also protects against some kinds of data corruption on storage devices that may occur because of hardware errors or media failure over time, such as reading/writing garbage data.

PostgreSQL does not protect against correctable memory errors and it is assumed you will operate using RAM that uses industry standard Error Correcting Codes (ECC) or better protection.

Data Checksums

checksums

By default, data pages are protected by checksums, but this can optionally be disabled for a cluster. When enabled, each data page includes a checksum that is updated when the page is written and verified each time the page is read. Only data pages are protected by checksums; internal data structures and temporary files are not.

Checksums can be disabled when the cluster is initialized using initdb. They can also be enabled or disabled at a later time either as an offline operation or online in a running cluster allowing concurrent access. Data checksums are enabled or disabled at the full cluster level, and cannot be specified individually for databases, tables or replicated cluster members.

The current state of checksums in the cluster can be verified by viewing the value of the read-only configuration variable ??? by issuing the command SHOW data_checksums. pg_control_init and pg_control_checkpoint can also be used for inspecting the data checksum state at cluster initialization and current checkpoint. Data checksum states are often referred to as data checksum versions using an integer representation due to how they were originally implemented. Data Checksums States and Version Mapping contains a mapping between state names and versions, which are defined in src/include/storage/checksum.h.

State Version
off 0
on 1
inprogress-off 2
inprogress-on 3

Data Checksums States and Version Mapping {#data-checksums-mapping}

When attempting to recover from page corruptions, it may be necessary to bypass the checksum protection. To do this, temporarily set the configuration parameter ???.

Offline Enabling of Checksums

The pg_checksums application can be used to enable or disable data checksums, as well as verify checksums, on an offline cluster.

Online Enabling of Checksums

Checksums can be enabled or disabled online, by calling the appropriate functions.

Both enabling and disabling data checksums happens in two phases, separated by a checkpoint to ensure durability. The different states, and their transitions, are illustrated in Data Checksums States and discussed in further detail in this section.

datachecksums.svg
Data Checksums States

Enabling checksums will set the cluster checksum state to inprogress-on. During this time, checksums will be written but not verified. In addition to this, a background worker process is started that enables checksums on all existing data in the cluster. Once this worker has completed processing all databases in the cluster, the checksum state will automatically switch to on. The processing will consume two background worker processes, make sure that max_worker_processes allows for at least two more additional processes.

The process will initially wait for all open transactions to finish before it starts, so that it can be certain that there are no tables that have been created inside a transaction that has not committed yet and thus would not be visible to the process enabling checksums. It will also, for each database, wait for all pre-existing temporary tables to get removed before it finishes. If long-lived temporary tables are used in an application it may be necessary to terminate these application connections to allow the process to complete.

If the cluster is stopped while in inprogress-on state, for any reason, or processing was interrupted, then the checksum enable process must be restarted manually. To do this, re-execute the function pg_enable_data_checksums() once the cluster has been restarted. The process will start over, there is no support for resuming work from where it was interrupted. If the cluster is stopped while in inprogress-off, then the checksum state will be set to off when the cluster is restarted.

Disabling data checksums will set the data checksum state to inprogress-off. During this time, checksums will be written but not verified. After all processes acknowledge the change, the state will automatically be set to off.

Disabling data checksums while data checksums are actively being enabled will abort the current processing.

Impact on System of Online Operations

Enabling data checksums can cause significant I/O to the system, as all of the database pages will need to be rewritten, and will be written both to the data files and the WAL. The impact may be limited by throttling using the cost_delay and cost_limit parameters of the pg_enable_data_checksums() function.

No I/O is incurred when disabling data checksums, but checkpoints are still required.

Mismatched Data Checksum States in a Replicated Cluster

The primary and secondaries can end up with different data checksums states due to the different durability models between offline and online checksum operations.

An online transition is WAL-logged, which means that it is replayed after a crash, and the transition along with all checksum updates are propagated to standbys. An offline change is recorded only in the cluster's control file, it writes no WAL and is invisible to replication. The changes made to the data files to write checksums are not WAL-logged. This means that it has no defined ordering against WAL the node has not replayed yet. When a node later replays WAL that contains an online checksum state change, that change takes effect on the node even if it was written before the offline change was made.

An offline change only affects the data directory it is run on; the new state does not propagate over replication. This means that all nodes can be operated on in parallel, and no additional network traffic or WAL archive traffic will occur. In a replication setup the same change must be applied to all nodes while all of them are stopped, as described in ???. A standby whose state diverges logs a warning but keeps its local setting. The mismatch persists until the states are brought together again, with the offline procedure or with an online transition.

Running a replicated cluster with different data checksum states on the different nodes is not supported or recommended.

Write-Ahead Logging (WAL)

WAL

transaction log

WAL

Write-Ahead Logging (WAL) is a standard method for ensuring data integrity. A detailed description can be found in most (if not all) books about transaction processing. Briefly, WAL's central concept is that changes to data files (where tables and indexes reside) must be written only after those changes have been logged, that is, after WAL records describing the changes have been flushed to permanent storage. If we follow this procedure, we do not need to flush data pages to disk on every transaction commit, because we know that in the event of a crash we will be able to recover the database using the log: any changes that have not been applied to the data pages can be redone from the WAL records. (This is roll-forward recovery, also known as REDO.)

[!TIP] Because WAL restores database file contents after a crash, journaled file systems are not necessary for reliable storage of the data files or WAL files. In fact, journaling overhead can reduce performance, especially if journaling causes file system data to be flushed to disk. Fortunately, data flushing during journaling can often be disabled with a file system mount option, e.g., data=writeback on a Linux ext3 file system. Journaled file systems do improve boot speed after a crash.

Using WAL results in a significantly reduced number of disk writes, because only the WAL file needs to be flushed to disk to guarantee that a transaction is committed, rather than every data file changed by the transaction. The WAL file is written sequentially, and so the cost of syncing the WAL is much less than the cost of flushing the data pages. This is especially true for servers handling many small transactions touching different parts of the data store. Furthermore, when the server is processing many small concurrent transactions, one fsync of the WAL file may suffice to commit many transactions.

WAL also makes it possible to support on-line backup and point-in-time recovery, as described in ???. By archiving the WAL data we can support reverting to any time instant covered by the available WAL data: we simply install a prior physical backup of the database, and replay the WAL just as far as the desired time. What's more, the physical backup doesn't have to be an instantaneous snapshot of the database state — if it is made over some period of time, then replaying the WAL for that period will fix any internal inconsistencies.

Asynchronous Commit

synchronous commit

asynchronous commit

Asynchronous commit is an option that allows transactions to complete more quickly, at the cost that the most recent transactions may be lost if the database should crash. In many applications this is an acceptable trade-off.

As described in the previous section, transaction commit is normally synchronous: the server waits for the transaction's WAL records to be flushed to permanent storage before returning a success indication to the client. The client is therefore guaranteed that a transaction reported to be committed will be preserved, even in the event of a server crash immediately after. However, for short transactions this delay is a major component of the total transaction time. Selecting asynchronous commit mode means that the server returns success as soon as the transaction is logically completed, before the WAL records it generated have actually made their way to disk. This can provide a significant boost in throughput for small transactions.

Asynchronous commit introduces the risk of data loss. There is a short time window between the report of transaction completion to the client and the time that the transaction is truly committed (that is, it is guaranteed not to be lost if the server crashes). Thus asynchronous commit should not be used if the client will take external actions relying on the assumption that the transaction will be remembered. As an example, a bank would certainly not use asynchronous commit for a transaction recording an ATM's dispensing of cash. But in many scenarios, such as event logging, there is no need for a strong guarantee of this kind.

The risk that is taken by using asynchronous commit is of data loss, not data corruption. If the database should crash, it will recover by replaying WAL up to the last record that was flushed. The database will therefore be restored to a self-consistent state, but any transactions that were not yet flushed to disk will not be reflected in that state. The net effect is therefore loss of the last few transactions. Because the transactions are replayed in commit order, no inconsistency can be introduced — for example, if transaction B made changes relying on the effects of a previous transaction A, it is not possible for A's effects to be lost while B's effects are preserved.

The user can select the commit mode of each transaction, so that it is possible to have both synchronous and asynchronous commit transactions running concurrently. This allows flexible trade-offs between performance and certainty of transaction durability. The commit mode is controlled by the user-settable parameter ???, which can be changed in any of the ways that a configuration parameter can be set. The mode used for any one transaction depends on the value of synchronous_commit when transaction commit begins.

Certain utility commands, for instance DROP TABLE, are forced to commit synchronously regardless of the setting of synchronous_commit. This is to ensure consistency between the server's file system and the logical state of the database. The commands supporting two-phase commit, such as PREPARE TRANSACTION, are also always synchronous.

If the database crashes during the risk window between an asynchronous commit and the writing of the transaction's WAL records, then changes made during that transaction will be lost. The duration of the risk window is limited because a background process (the “WAL writer”) flushes unwritten WAL records to disk every ??? milliseconds. The actual maximum duration of the risk window is three times wal_writer_delay because the WAL writer is designed to favor writing whole pages at a time during busy periods.

[!CAUTION] An immediate-mode shutdown is equivalent to a server crash, and will therefore cause loss of any unflushed asynchronous commits.

Asynchronous commit provides behavior different from setting ??? = off. fsync is a server-wide setting that will alter the behavior of all transactions. It disables all logic within PostgreSQL that attempts to synchronize writes to different portions of the database, and therefore a system crash (that is, a hardware or operating system crash, not a failure of PostgreSQL itself) could result in arbitrarily bad corruption of the database state. In many scenarios, asynchronous commit provides most of the performance improvement that could be obtained by turning off fsync, but without the risk of data corruption.

??? also sounds very similar to asynchronous commit, but it is actually a synchronous commit method (in fact, commit_delay is ignored during an asynchronous commit). commit_delay causes a delay just before a transaction flushes WAL to disk, in the hope that a single flush executed by one such transaction can also serve other transactions committing at about the same time. The setting can be thought of as a way of increasing the time window in which transactions can join a group about to participate in a single flush, to amortize the cost of the flush among multiple transactions.

WAL Configuration

There are several WAL-related configuration parameters that affect database performance. This section explains their use. Consult ??? for general information about setting server configuration parameters.

Checkpoints are points in the sequence of transactions at which it is guaranteed that the heap and index data files have been updated with all information written before that checkpoint. At checkpoint time, all dirty data pages are flushed to disk and a special checkpoint record is written to the WAL file. (The change records were previously flushed to the WAL files.) In the event of a crash, the crash recovery procedure looks at the latest checkpoint record to determine the point in the WAL (known as the redo record) from which it should start the REDO operation. Any changes made to data files before that point are guaranteed to be already on disk. Hence, after a checkpoint, WAL segments preceding the one containing the redo record are no longer needed and can be recycled or removed. (When WAL archiving is being done, the WAL segments must be archived before being recycled or removed.)

The checkpoint requirement of flushing all dirty data pages to disk can cause a significant I/O load. For this reason, checkpoint activity is throttled so that I/O begins at checkpoint start and completes before the next checkpoint is due to start; this minimizes performance degradation during checkpoints.

The server's checkpointer process automatically performs a checkpoint every so often. A checkpoint is begun every ??? seconds, or if ??? is about to be exceeded, whichever comes first. The default settings are 5 minutes and 1 GB, respectively. If no WAL has been written since the previous checkpoint, new checkpoints will be skipped even if checkpoint_timeout has passed. (If WAL archiving is being used and you want to put a lower limit on how often files are archived in order to bound potential data loss, you should adjust the ??? parameter rather than the checkpoint parameters.) It is also possible to force a checkpoint by using the SQL command CHECKPOINT.

Reducing checkpoint_timeout and/or max_wal_size causes checkpoints to occur more often. This allows faster after-crash recovery, since less work will need to be redone. However, one must balance this against the increased cost of flushing dirty data pages more often. If ??? is set (as is the default), there is another factor to consider. To ensure data page consistency, the first modification of a data page after each checkpoint results in logging the entire page content. In that case, a smaller checkpoint interval increases the volume of output to the WAL, partially negating the goal of using a smaller interval, and in any case causing more disk I/O.

Checkpoints are fairly expensive, first because they require writing out all currently dirty buffers, and second because they result in extra subsequent WAL traffic as discussed above. It is therefore wise to set the checkpointing parameters high enough so that checkpoints don't happen too often. As a simple sanity check on your checkpointing parameters, you can set the ??? parameter. If checkpoints happen closer together than checkpoint_warning seconds, a message will be output to the server log recommending increasing max_wal_size. Occasional appearance of such a message is not cause for alarm, but if it appears often then the checkpoint control parameters should be increased. Bulk operations such as large COPY transfers might cause a number of such warnings to appear if you have not set max_wal_size high enough.

To avoid flooding the I/O system with a burst of page writes, writing dirty buffers during a checkpoint is spread over a period of time. That period is controlled by ???, which is given as a fraction of the checkpoint interval (configured by using checkpoint_timeout). The I/O rate is adjusted so that the checkpoint finishes when the given fraction of checkpoint_timeout seconds have elapsed, or before max_wal_size is exceeded, whichever is sooner. With the default value of 0.9, PostgreSQL can be expected to complete each checkpoint a bit before the next scheduled checkpoint (at around 90% of the last checkpoint's duration). This spreads out the I/O as much as possible so that the checkpoint I/O load is consistent throughout the checkpoint interval. The disadvantage of this is that prolonging checkpoints affects recovery time, because more WAL segments will need to be kept around for possible use in recovery. A user concerned about the amount of time required to recover might wish to reduce checkpoint_timeout so that checkpoints occur more frequently but still spread the I/O across the checkpoint interval. Alternatively, checkpoint_completion_target could be reduced, but this would result in times of more intense I/O (during the checkpoint) and times of less I/O (after the checkpoint completed but before the next scheduled checkpoint) and therefore is not recommended. Although checkpoint_completion_target could be set as high as 1.0, it is typically recommended to set it to no higher than 0.9 (the default) since checkpoints include some other activities besides writing dirty buffers. A setting of 1.0 is quite likely to result in checkpoints not being completed on time, which would result in performance loss due to unexpected variation in the number of WAL segments needed.

On Linux and POSIX platforms ??? allows you to force OS pages written by the checkpoint to be flushed to disk after a configurable number of bytes. Otherwise, these pages may be kept in the OS's page cache, inducing a stall when fsync is issued at the end of a checkpoint. This setting will often help to reduce transaction latency, but it also can have an adverse effect on performance; particularly for workloads that are bigger than ???, but smaller than the OS's page cache.

The number of WAL segment files in pg_wal directory depends on min_wal_size, max_wal_size and the amount of WAL generated in previous checkpoint cycles. When old WAL segment files are no longer needed, they are removed or recycled (that is, renamed to become future segments in the numbered sequence). If, due to a short-term peak of WAL output rate, max_wal_size is exceeded, the unneeded segment files will be removed until the system gets back under this limit. Below that limit, the system recycles enough WAL files to cover the estimated need until the next checkpoint, and removes the rest. The estimate is based on a moving average of the number of WAL files used in previous checkpoint cycles. The moving average is increased immediately if the actual usage exceeds the estimate, so it accommodates peak usage rather than average usage to some extent. min_wal_size puts a minimum on the amount of WAL files recycled for future usage; that much WAL is always recycled for future use, even if the system is idle and the WAL usage estimate suggests that little WAL is needed.

Independently of max_wal_size, the most recent ??? megabytes of WAL files plus one additional WAL file are kept at all times. Also, if WAL archiving is used, old segments cannot be removed or recycled until they are archived. If WAL archiving cannot keep up with the pace that WAL is generated, or if archive_command or archive_library fails repeatedly, old WAL files will accumulate in pg_wal until the situation is resolved. A slow or failed standby server that uses a replication slot will have the same effect (see ???). Similarly, if WAL summarization is enabled, old segments are kept until they are summarized.

In archive recovery or standby mode, the server periodically performs restartpoints, which are similar to checkpoints in normal operation: the server forces all its state to disk, updates the pg_control file to indicate that the already-processed WAL data need not be scanned again, and then recycles any old WAL segment files in the pg_wal directory. Restartpoints can't be performed more frequently than checkpoints on the primary because restartpoints can only be performed at checkpoint records. A restartpoint can be demanded by a schedule or by an external request. The restartpoints_timed counter in the pg_stat_checkpointer view counts the first ones while the restartpoints_req the second. A restartpoint is triggered by schedule when a checkpoint record is reached if at least ??? seconds have passed since the last performed restartpoint or when the previous attempt to perform the restartpoint has failed. In the last case, the next restartpoint will be scheduled in 15 seconds. A restartpoint is triggered by request due to similar reasons like checkpoint but mostly if WAL size is about to exceed ??? However, because of limitations on when a restartpoint can be performed, max_wal_size is often exceeded during recovery, by up to one checkpoint cycle's worth of WAL. (max_wal_size is never a hard limit anyway, so you should always leave plenty of headroom to avoid running out of disk space.) The restartpoints_done counter in the pg_stat_checkpointer view counts the restartpoints that have really been performed.

In some cases, when the WAL size on the primary increases quickly, for instance during massive INSERT, the restartpoints_req counter on the standby may demonstrate a peak growth. This occurs because requests to create a new restartpoint due to increased WAL consumption cannot be performed because the safe checkpoint record since the last restartpoint has not yet been replayed on the standby. This behavior is normal and does not lead to an increase in system resource consumption. Only the restartpoints_done counter among the restartpoint-related ones indicates that noticeable system resources have been spent.

There are two commonly used internal WAL functions: XLogInsertRecord and XLogFlush. XLogInsertRecord is used to place a new record into the WAL buffers in shared memory. If there is no space for the new record, XLogInsertRecord will have to write (move to kernel cache) a few filled WAL buffers. This is undesirable because XLogInsertRecord is used on every database low level modification (for example, row insertion) at a time when an exclusive lock is held on affected data pages, so the operation needs to be as fast as possible. What is worse, writing WAL buffers might also force the creation of a new WAL segment, which takes even more time. Normally, WAL buffers should be written and flushed by an XLogFlush request, which is made, for the most part, at transaction commit time to ensure that transaction records are flushed to permanent storage. On systems with high WAL output, XLogFlush requests might not occur often enough to prevent XLogInsertRecord from having to do writes. On such systems one should increase the number of WAL buffers by modifying the ??? parameter. When ??? is set and the system is very busy, setting wal_buffers higher will help smooth response times during the period immediately following each checkpoint.

The ??? parameter defines for how many microseconds a group commit leader process will sleep after acquiring a lock within XLogFlush, while group commit followers queue up behind the leader. This delay allows other server processes to add their commit records to the WAL buffers so that all of them will be flushed by the leader's eventual sync operation. No sleep will occur if ??? is not enabled, or if fewer than ??? other sessions are currently in active transactions; this avoids sleeping when it's unlikely that any other session will commit soon. Note that on some platforms, the resolution of a sleep request is ten milliseconds, so that any nonzero commit_delay setting between 1 and 10000 microseconds would have the same effect. Note also that on some platforms, sleep operations may take slightly longer than requested by the parameter.

Since the purpose of commit_delay is to allow the cost of each flush operation to be amortized across concurrently committing transactions (potentially at the expense of transaction latency), it is necessary to quantify that cost before the setting can be chosen intelligently. The higher that cost is, the more effective commit_delay is expected to be in increasing transaction throughput, up to a point. The ??? program can be used to measure the average time in microseconds that a single WAL flush operation takes. A value of half of the average time the program reports it takes to flush after a single 8kB write operation is often the most effective setting for commit_delay, so this value is recommended as the starting point to use when optimizing for a particular workload. While tuning commit_delay is particularly useful when the WAL is stored on high-latency rotating disks, benefits can be significant even on storage media with very fast sync times, such as solid-state drives or RAID arrays with a battery-backed write cache; but this should definitely be tested against a representative workload. Higher values of commit_siblings should be used in such cases, whereas smaller commit_siblings values are often helpful on higher latency media. Note that it is quite possible that a setting of commit_delay that is too high can increase transaction latency by so much that total transaction throughput suffers.

When commit_delay is set to zero (the default), it is still possible for a form of group commit to occur, but each group will consist only of sessions that reach the point where they need to flush their commit records during the window in which the previous flush operation (if any) is occurring. At higher client counts a “gangway effect” tends to occur, so that the effects of group commit become significant even when commit_delay is zero, and thus explicitly setting commit_delay tends to help less. Setting commit_delay can only help when (1) there are some concurrently committing transactions, and (2) throughput is limited to some degree by commit rate; but with high rotational latency this setting can be effective in increasing transaction throughput with as few as two clients (that is, a single committing client with one sibling transaction).

The ??? parameter determines how PostgreSQL will ask the kernel to force WAL updates out to disk. All the options should be the same in terms of reliability, with the exception of fsync_writethrough, which can sometimes force a flush of the disk cache even when other options do not do so. However, it's quite platform-specific which one will be the fastest. You can test the speeds of different options using the ??? program. Note that this parameter is irrelevant if fsync has been turned off.

Enabling the ??? configuration parameter (provided that PostgreSQL has been compiled with support for it) will result in each XLogInsertRecord and XLogFlush WAL call being logged to the server log. This option might be replaced by a more general mechanism in the future.

There are two internal functions to write WAL data to disk: XLogWrite and issue_xlog_fsync. When ??? is enabled, the total amounts of time XLogWrite writes and issue_xlog_fsync syncs WAL data to disk are counted as write_time and fsync_time in ??? for the object wal, respectively. XLogWrite is normally called by XLogInsertRecord (when there is no space for the new record in WAL buffers), XLogFlush and the WAL writer, to write WAL buffers to disk and call issue_xlog_fsync. issue_xlog_fsync is normally called by XLogWrite to sync WAL files to disk. If wal_sync_method is either open_datasync or open_sync, a write operation in XLogWrite guarantees to sync written WAL data to disk and issue_xlog_fsync does nothing. If wal_sync_method is either fdatasync, fsync, or fsync_writethrough, the write operation moves WAL buffers to kernel cache and issue_xlog_fsync syncs them to disk. Regardless of the setting of track_wal_io_timing, the number of times XLogWrite writes and issue_xlog_fsync syncs WAL data to disk are also counted as writes and fsyncs in pg_stat_io for the object wal, respectively.

The ??? parameter can be used to reduce I/O wait times during recovery by instructing the kernel to initiate reads of disk blocks that will soon be needed but are not currently in PostgreSQL's buffer pool. The ??? and ??? settings limit prefetching concurrency and distance, respectively. By default, it is set to try, which enables the feature on systems that support issuing read-ahead advice.

WAL Internals

LSN

WAL is automatically enabled; no action is required from the administrator except ensuring that the disk-space requirements for the WAL files are met, and that any necessary tuning is done (see Configuration).

WAL records are appended to the WAL files as each new record is written. The insert position is described by a Log Sequence Number (LSN) that is a byte offset into the WAL, increasing monotonically with each new record. LSN values are returned as the datatype pg_lsn. Values can be compared to calculate the volume of WAL data that separates them, so they are used to measure the progress of replication and recovery.

WAL files are stored in the directory pg_wal under the data directory, as a set of segment files, normally each 16 MB in size (but the size can be changed by altering the --wal-segsize initdb option). Each segment is divided into pages, normally 8 kB each (this size can be changed via the --with-wal-blocksize configure option). The WAL record headers are described in access/xlogrecord.h; the record content is dependent on the type of event that is being logged. Segment files are given ever-increasing numbers as names, starting at 000000010000000000000001. The numbers do not wrap, but it will take a very, very long time to exhaust the available stock of numbers.

It is advantageous if the WAL is located on a different disk from the main database files. This can be achieved by moving the pg_wal directory to another location (while the server is shut down, of course) and creating a symbolic link from the original location in the main data directory to the new location.

The aim of WAL is to ensure that the log is written before database records are altered, but this can be subverted by disk drives that falsely report a successful write to the kernel, when in fact they have only cached the data and not yet stored it on the disk. A power failure in such a situation might lead to irrecoverable data corruption. Administrators should try to ensure that disks holding PostgreSQL's WAL files do not make such false reports. (See Reliability.)

After a checkpoint has been made and the WAL flushed, the checkpoint's position is saved in the file pg_control. Therefore, at the start of recovery, the server first reads pg_control and then the checkpoint record; then it performs the REDO operation by scanning forward from the WAL location indicated in the checkpoint record. Because the entire content of data pages is saved in the WAL on the first page modification after a checkpoint (assuming ??? is not disabled), all pages changed since the checkpoint will be restored to a consistent state.

To deal with the case where pg_control is corrupt, we should support the possibility of scanning existing WAL segments in reverse order — newest to oldest — in order to find the latest checkpoint. This has not been implemented yet. pg_control is small enough (less than one disk page) that it is not subject to partial-write problems, and as of this writing there have been no reports of database failures due solely to the inability to read pg_control itself. So while it is theoretically a weak spot, pg_control does not seem to be a problem in practice.