Loading COPY Formatted Files¶
This commands instructs pgloader to load from a file containing COPY TEXT data as described in the PostgreSQL documentation. Here’s an example:
LOAD COPY FROM copy://./data/track.copy ( trackid, track, album, media, genre, composer, milliseconds, bytes, unitprice ) INTO postgresql:///pgloader TARGET TABLE track_full WITH truncate SET work_mem to '14MB', standard_conforming_strings to 'on' BEFORE LOAD DO $$ drop table if exists track_full; $$, $$ create table track_full ( trackid bigserial, track text, album text, media text, genre text, composer text, milliseconds bigint, bytes bigint, unitprice numeric ); $$;
The COPY format command accepts the following clauses and options.
COPY Formatted Files Source Specification: FROM¶
Filename where to load the data from. This support local files, HTTP URLs and zip files containing a single dbf file of the same name. Fetch such a zip file from an HTTP address is of course supported.
The data is found after the end of the parsed commands. Any number of empty lines between the end of the commands and the beginning of the data is accepted.
Reads the data from the standard input stream.
The whole matching clause must follow the following rule:[ ALL FILENAMES | [ FIRST ] FILENAME ] MATCHING regexp [ IN DIRECTORY '...' ]
The matching clause applies given regular expression (see above for exact syntax, several options can be used here) to filenames. It’s then possible to load data from only the first match of all of them.
The optional IN DIRECTORY clause allows specifying which directory to walk for finding the data files, and can be either relative to where the command file is read from, or absolute. The given directory must exists.
COPY Formatted File Options: WITH¶
When loading from a COPY file, the following options are supported:
Takes a single character as argument, which must be found inside single quotes, and might be given as the printable character itself, the special value t to denote a tabulation character, or 0x then an hexadecimal value read as the ASCII code for the character.
This character is used as the delimiter when reading the data, in a similar way to the PostgreSQL COPY option.
Takes a quoted string as an argument (quotes can be either double quotes or single quotes) and uses that string as the NULL representation in the data.
This is similar to the null COPY option in PostgreSQL.
When this option is listed, pgloader issues a TRUNCATE command against the PostgreSQL target table before reading the data file.
When this option is listed, pgloader issues an ALTER TABLE … DISABLE TRIGGER ALL command against the PostgreSQL target table before copying the data, then the command ALTER TABLE … ENABLE TRIGGER ALL once the COPY is done.
This option allows loading data into a pre-existing table ignoring the foreign key constraints and user defined triggers and may result in invalid foreign key constraints once the data is loaded. Use with care.
Takes a numeric value as argument. Instruct pgloader to skip that many lines at the beginning of the input file.