Skip to main content
snaptools

Guide

How to import a CSV into a database

Choose between generated INSERTs and a bulk loader by file size, clean the CSV first, check the row count after, and fix the errors that stop an import. Run on MySQL, PostgreSQL and SQLite.

SnapTools Editorial ·
Try it — CSV to SQL
S
CSV to SQL error
⌃⏎ run · ⎋ clear · / focus input

For a few thousand rows, generate INSERT statements with the tool above and run them inside one transaction. For more than that, use the database's own loader: LOAD DATA in MySQL, COPY or \copy in PostgreSQL, .import in SQLite, BULK INSERT in SQL Server. Either way, clean the file first and compare row counts afterwards. The rest of this page is the detail, with the commands and the error messages we got when we ran them.

All commands here were run on 24 October 2026 against MySQL 8.4.13, PostgreSQL 17.11 and SQLite 3.54.0. SQL Server was not run; its notes come from Microsoft's documentation.

Which method for which file size

We loaded the same 10,000-row CSV (592,306 bytes) three ways on each database. Times include about 0.1 s of client start-up.

  • MySQL: 8.7 s as separate INSERTs, 0.6 s with the same INSERTs inside SET autocommit=0; ... COMMIT;, 0.12 s with LOAD DATA.
  • PostgreSQL: 3.36 s as separate INSERTs, 0.77 s inside BEGIN; ... COMMIT;, 0.1 s with COPY.
  • SQLite: 2.48 s as separate INSERTs, 0.04 s inside a transaction, 0.04 s with .import.

Two things follow. The transaction does most of the saving, because each separate INSERT forces a commit. And the loader is still faster than the best INSERT run on the two client-server databases. The manuals say the same: the MySQL manual tells you to turn off autocommit for an InnoDB import, and the PostgreSQL manual says COPY is almost always faster than INSERT even when batched in a transaction.

Our rule of thumb, from that one measurement and not a standard: if you load it once and it is small, paste INSERTs. If you load it on a schedule, or it takes more than a few seconds as INSERTs, switch to the loader.

Clean the CSV before you load it

Most failed imports are a file problem. Check these first.

  1. Encoding. Save as UTF-8. The converter above removes a byte order mark at the start of the file; we confirmed the first column name came out clean. We did not test how each loader treats one, so strip it if a loader complains about the first column.
  2. Delimiter and decimal comma. A semicolon-delimited file with 9,99 in it is read as text, not a number: the converter wrote '9,99' and '1.050,00' in quotes. Change the decimal separator to a point before you load into a numeric column.
  3. One header row, unique names. The converter keeps a duplicate name as it is and the database then refuses the statement. Letters outside A to Z, digits and underscore become _, so First Name turns into First_Name.
  4. Line breaks inside cells. The converter splits a quoted cell with a line break into two rows. Remove them first, or use a loader, which reads them correctly.
  5. Dates as 2026-10-24. 24/10/2026 was rejected by MySQL ("Incorrect date value") and by PostgreSQL at its default DateStyle (ISO, MDY).
  6. Leading zeros. If a spreadsheet already turned 02134 into 2134 before you saved the file, no importer can bring the zero back. Open the CSV in a text editor and look.
  7. Booleans in the form your database wants. MySQL accepted 1 and 0 and rejected true. PostgreSQL accepted true and rejected 1.
  8. Empty means NULL. In the converter an empty cell becomes NULL, and so does "". The word NULL becomes the four-letter text 'NULL'. In PostgreSQL's CSV mode an unquoted empty string is NULL by default and a quoted "" is an empty string.

Create the table first

Neither the converter nor the loaders guess your types well enough to trust. Make the table yourself, and use text for anything with leading zeros or a code:

CREATE TABLE customers (
  name  varchar(20),
  zip   varchar(10),
  phone varchar(30),
  sku   varchar(30),
  qty   int
);

That statement ran unchanged on MySQL and PostgreSQL. In our test, a zip column of type int stored 02134 as 2134 on both, without an error.

Method A: generated INSERT statements

Paste the CSV into the converter, set the table name, copy the output. Then:

  1. Wrap it in one transaction. MySQL: SET autocommit=0; first and COMMIT; last. PostgreSQL and SQLite: BEGIN; first and COMMIT; last. If one row fails in PostgreSQL, the rows before it are rolled back with it.
  2. On MySQL, set SET SESSION sql_mode = CONCAT(@@sql_mode, ',NO_BACKSLASH_ESCAPES');. The converter does not double backslashes. Without that mode, C:\temp\new was stored as a tab and a line feed, and a cell ending in a backslash broke the statement.
  3. On PostgreSQL, swap the backticks: sed 's//"/g' out.sql > out_pg.sql`. This also changes any backtick in your data.
  4. Run the file. In mysql: mysql -u user -p dbname < out.sql. In psql: psql -d dbname -f out_pg.sql. In SQLite: sqlite3 app.db < out.sql. SQLite accepts the backticks as they are.

Method B: the database's own loader

Use these when the file is large or has quoted line breaks. We ran all three, and all three kept "Line one<newline>Line two" as one cell.

MySQL

LOAD DATA INFILE '/var/lib/mysql-files/data.csv' INTO TABLE customers
  FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' ESCAPED BY ''
  LINES TERMINATED BY '\n' IGNORE 1 LINES;

The server reads that file itself, and only from the directory in secure_file_priv; on our test server that was /var/lib/mysql-files/. The empty ESCAPED BY '' matters: with the default, C:\temp\new was again stored as a tab and a line feed. Add LOCAL to read a file from your own machine, if both client and server allow it.

PostgreSQL

\copy customers FROM 'data.csv' WITH (FORMAT csv, HEADER)

The server-side form is COPY customers FROM '/path/data.csv' WITH (FORMAT csv, HEADER);.

SQLite

sqlite3 app.db ".import --csv --skip 1 data.csv customers"

Create the table first. If it does not exist, SQLite makes it from the first row, which is not what you want with a header you plan to skip.

SQL Server (2017 and later, from the documentation, not run here)

BULK INSERT dbo.customers FROM 'C:\data\data.csv' WITH (FORMAT = 'CSV', FIRSTROW = 2);

Check the import worked

  1. Count rows. SELECT COUNT(*) FROM customers; should equal the CSV's data rows. wc -l over-counts if any cell holds a line break, so count with a CSV-aware tool.
  2. Look at the risky columns. SELECT zip FROM customers WHERE zip LIKE '0%' LIMIT 5; proves the zeros survived. SELECT COUNT(*) FROM customers WHERE phone IS NULL; shows how many empty cells became NULL.
  3. Round trip. Export the table back to CSV and compare it with the original in the CSV diff. Added, removed and changed rows are reported by key, so a silent truncation shows up.

Errors you may hit, and what they mean

These are the messages we got.

  • ERROR 1366 ... Incorrect integer value: 'true' for column 'active' (MySQL): text in a number or boolean column. Change the CSV value to 1 or 0.
  • ERROR 1292 ... Incorrect date value: '24/10/2026' (MySQL) and date/time field value out of range (PostgreSQL): reformat to 2026-10-24.
  • ERROR 1064 ... near ''back\')' (MySQL): a backslash at the end of a value. See step 2 of Method A.
  • ERROR 1110 ... Column 'id' specified twice (MySQL) and column "id" specified more than once (PostgreSQL): duplicate header names.
  • column "active" is of type boolean but expression is of type integer (PostgreSQL): a bare 1 or 0 going into a boolean column. Use true and false, or cast.
  • syntax error at or near ""` (PostgreSQL): backticks in the SQL.
  • invalid input syntax for type integer: "n/a" (PostgreSQL): a placeholder word in a number column. Replace it with an empty cell.
  • ERROR 3948 ... Loading local data is disabled (MySQL): see the FAQ below.
  • SQL Server error 10738: more than 1,000 rows in one INSERT ... VALUES.

Sources

Checked on 24 October 2026, all opened that day: MySQL LOAD DATA, MySQL InnoDB bulk loading, MySQL string literals, MySQL LOAD DATA LOCAL security, MySQL packet size, PostgreSQL COPY, PostgreSQL populating a database, SQLite shell, SQLite limits, SQL Server table value constructor, SQL Server BULK INSERT.

Frequently asked questions

It depends on the database. SQL Server stops at 1,000 rows in a single INSERT ... VALUES and returns error 10738 above that. MySQL has no row cap but the whole statement must fit in max_allowed_packet, which is 64 MB on the server by default in 8.4 and 16 MB in the mysql client. SQLite accepts statement text up to 1,000,000,000 bytes by default. The converter above writes one row per statement, so none of these limits apply to its output.
MySQL documents it as a speed-up for InnoDB: SET unique_checks=0 and SET foreign_key_checks=0 for the session, then set them back to 1. The manual adds a condition. Do it only if you are certain the file has no duplicate keys and no orphan rows, because the database stops checking for you. If you are not sure, leave the checks on and take the slower load.
Both sides must allow it. The server variable local_infile is off by default, and the mysql client library is built with it off. On our MySQL 8.4.13 test server the error was ERROR 3948: Loading local data is disabled; this must be enabled on both the client and server sides. Either turn it on at both ends or copy the file to the server and use LOAD DATA INFILE without LOCAL.
COPY makes the database server read the file, so the path must exist on the server and you need the right privileges. \copy is a psql command that reads the file on your own machine and sends it over the connection. For a file on your laptop and a database elsewhere, use \copy.
Yes, with the loaders. Leave out HEADER in PostgreSQL, leave out IGNORE 1 LINES in MySQL, and leave out --skip 1 in SQLite, but create the table first. SQLite is the trap: if the table does not exist, .import uses the first row as column names. The converter above needs a header row, so add one such as c1,c2,c3 before pasting.

Related