I was just burned by this, where some tests I’d written
against an sqlite database did not fail in the way that they
“should” have, because of this double-quoted string issue.
Asking here before I file an improvement request issue on the python GitHub:calling an `sqlite3_db_config` C-function, but I don’t see any way to do that in the python sqlite library [2].
sqlite has a known misfeature with double-quoted strings, whereby they will be interpreted as string literals if they don’t match a valid identifier [1]. The note in the sqlite docs describe a way to disable this misfeature at compile time or by
Am I missing a way to manage this setting, or is it not available within python? This would be very useful to enable, so that python’s sqlite library will treat queries more like standard sql, instead of this particular version of MySQL. I was justburned by this, where some tests I’d written against an sqlite database did not fail in the way that they “should” have, because of this double-quoted string issue.
It doesn’t look like `sqlite3_db_config` is used within the python sqlite3 codebase at all, so this might not be a trivial change? I only see two references to it in the cpython github.
Like Lars Liedtke this is not an exact answer to your question, but you
can side-step the issue by using parametrized queries, i.e. instead of
cur.execute('SELECT name, location FROM persons WHERE name = "John Doe"')
do
cur.execute('SELECT name, location FROM persons WHERE name = ?',
('John Doe',))
On Tue, 13 Dec 2022 at 19:52, Roel Schroeven <[email protected]> wrote:
Like Lars Liedtke this is not an exact answer to your question, but you
can side-step the issue by using parametrized queries, i.e. instead of
cur.execute('SELECT name, location FROM persons WHERE name = "John
Doe"')
do
cur.execute('SELECT name, location FROM persons WHERE name = ?',
('John Doe',))
That's the wrong behaviour though. According to the SQL standard, the
second query should be equivalent to this:
cur.execute("SELECT name, location FROM persons WHERE name = 'John Doe'")
What the OP wanted was like your first query, and proper DBMSes like PostgreSQL will handle it accordingly. The question is how to get
SQLite3 to also do so.
I don't use SQLite3 much so I'm not really one to judge, but maybe it
would be worth exposing the sqlite3_db_config() function to Python?
Yes, it would be more than a trivial change, but it should be
reasonably straight-forward. In order to be useful, it would probably
also need an associated IntEnum for all those lovely opaque numbers
that define the verbs.
ChrisA
On 12/13/2022 4:09 AM, Chris Angelico wrote:
On Tue, 13 Dec 2022 at 19:52, Roel Schroeven <[email protected]> wrote:
Like Lars Liedtke this is not an exact answer to your question, but you
can side-step the issue by using parametrized queries, i.e. instead of
cur.execute('SELECT name, location FROM persons WHERE name = "John >> Doe"')
do
cur.execute('SELECT name, location FROM persons WHERE name = ?',
('John Doe',))
That's the wrong behaviour though. According to the SQL standard, the second query should be equivalent to this:
cur.execute("SELECT name, location FROM persons WHERE name = 'John Doe'")
What the OP wanted was like your first query, and proper DBMSes like PostgreSQL will handle it accordingly. The question is how to get
SQLite3 to also do so.
From reading the SQLite3 documentation on this issue (not from personal experience), in fact the second form is actually what one wants, even if SQLite3 will usually handle the first form correctly.
+1 to expose the sqlite3_db_config() function, or maybe just a special
case for this specific option.
On 12/13/2022 4:09 AM, Chris Angelico wrote:What I missed at first is the case where you really want to use an
On Tue, 13 Dec 2022 at 19:52, Roel Schroeven <[email protected]>
wrote:
Like Lars Liedtke this is not an exact answer to your question, but you
can side-step the issue by using parametrized queries, i.e. instead of
cur.execute('SELECT name, location FROM persons WHERE name =
"John
Doe"')
do
cur.execute('SELECT name, location FROM persons WHERE name = ?', >>> ('John Doe',))
That's the wrong behaviour though. According to the SQL standard, the
second query should be equivalent to this:
cur.execute("SELECT name, location FROM persons WHERE name = 'John
Doe'")
What the OP wanted was like your first query, and proper DBMSes like
PostgreSQL will handle it accordingly. The question is how to get
SQLite3 to also do so.
From reading the SQLite3 documentation on this issue (not from
personal experience), in fact the second form is actually what one
wants, even if SQLite3 will usually handle the first form correctly.
The rule is "Use single quotes for string values and double quotes for database names such as schema, table and column names; for backwards compatibility SQLite will accept double quotes for string values, but
you may get a surprise if the string value looks like a database name."
"John K. Parejko" <[email protected]> writes:Or that are equal SQL keywords, which can be a reason to double-quote
I was just burned by this, where some tests I’d written
against an sqlite database did not fail in the way that they
“should” have, because of this double-quoted string issue.
In standard SQL, double quotes denote identifiers that are
allowed to contain special characters.
Stefan Ram schreef op 13/12/2022 om 8:42:
"John K. Parejko" <[email protected]> writes:
I was just burned by this, where some tests I’d written
against an sqlite database did not fail in the way that they
“should” have, because of this double-quoted string issue.
In standard SQL, double quotes denote identifiers that areOr that are equal SQL keywords, which can be a reason to double-quote
allowed to contain special characters.
them. SQL engines sometimes add new keywords; explicitly marking string literals as string literals prevents future conflicts and confusion.
Perhaps it's a better idea to use [identifier] or `identifier` instead
though (I just learned about those on
https://sqlite.org/lang_keywords.html). Both are not standard SQL ([] is
used in MS Access and SQL Server, `` is used in MySQL) but both work in SQLite. That should prevent any ambiguity and confusion, if it doesn't
bother you too much that it's not standard SQL.
Chris Angelico schreef op 13/12/2022 om 20:01:
On Wed, 14 Dec 2022 at 06:00, Roel Schroeven <[email protected]> wrote:
Stefan Ram schreef op 13/12/2022 om 8:42:
"John K. Parejko" <[email protected]> writes:
I was just burned by this, where some tests I’d written
against an sqlite database did not fail in the way that they >“should” have, because of this double-quoted string issue.
In standard SQL, double quotes denote identifiers that areOr that are equal SQL keywords, which can be a reason to double-quote them. SQL engines sometimes add new keywords; explicitly marking string literals as string literals prevents future conflicts and confusion.
allowed to contain special characters.
Perhaps it's a better idea to use [identifier] or `identifier` instead though (I just learned about those on https://sqlite.org/lang_keywords.html). Both are not standard SQL ([] is used in MS Access and SQL Server, `` is used in MySQL) but both work in SQLite. That should prevent any ambiguity and confusion, if it doesn't bother you too much that it's not standard SQL.
Why not just use "identifier" which is standard SQL?
If you accidentally type [identifire] or `identifire`, SQLite will
produce an unknown identifier error, alerting you immediately to your typo. If you accidentally type "identifire", SQLite will silently treat it as
a string literal instead of an identifier, causing more difficult to
diagnose problems.
sqlite> insert into foo values ("xyzzy", "xyzzy");SQLite accepts it like that, but I really should have used single quotes
On Wed, 14 Dec 2022 at 06:00, Roel Schroeven <[email protected]> wrote:
Stefan Ram schreef op 13/12/2022 om 8:42:
"John K. Parejko" <[email protected]> writes:
I was just burned by this, where some tests I’d written
against an sqlite database did not fail in the way that they >“should” have, because of this double-quoted string issue.
In standard SQL, double quotes denote identifiers that areOr that are equal SQL keywords, which can be a reason to double-quote
allowed to contain special characters.
them. SQL engines sometimes add new keywords; explicitly marking string literals as string literals prevents future conflicts and confusion.
Perhaps it's a better idea to use [identifier] or `identifier` instead though (I just learned about those on https://sqlite.org/lang_keywords.html). Both are not standard SQL ([] is used in MS Access and SQL Server, `` is used in MySQL) but both work in SQLite. That should prevent any ambiguity and confusion, if it doesn't bother you too much that it's not standard SQL.
Why not just use "identifier" which is standard SQL?
Chris Angelico schreef op 13/12/2022 om 20:01:
Perhaps it's a better idea to use [identifier] or `identifier` instead though (I just learned about those on https://sqlite.org/lang_keywords.html). Both are not standard SQL ([] is used in MS Access and SQL Server, `` is used in MySQL) but both work in SQLite. That should prevent any ambiguity and confusion, if it doesn't bother you too much that it's not standard SQL.
Example:Why not just use "identifier" which is standard SQL?
If you accidentally type [identifire] or `identifire`, SQLite will
produce an unknown identifier error, alerting you immediately to your typo. If you accidentally type "identifire", SQLite will silently treat it as
a string literal instead of an identifier, causing more difficult to
diagnose problems.
Okay, so..... exactly the same as if you use standard double quotes,
but change the configuration option. So the options are: make
everything worse for everyone by exacerbating the problem of
non-standard identifier quoting, or get this API so SQLite can be
configured, like the OP actually asked for.
On 13Dec 2022, at 13:58, Chris Angelico <[email protected]> wrote:
On Wed, 14 Dec 2022 at 08:19, Roel Schroeven <[email protected]> wrote:
Chris Angelico schreef op 13/12/2022 om 20:01:
On Wed, 14 Dec 2022 at 06:00, Roel Schroeven <[email protected]> wrote:
Stefan Ram schreef op 13/12/2022 om 8:42:
"John K. Parejko" <[email protected]> writes:Or that are equal SQL keywords, which can be a reason to double-quote
I was just burned by this, where some tests I’d written
against an sqlite database did not fail in the way that they
“should” have, because of this double-quoted string issue.
In standard SQL, double quotes denote identifiers that are
allowed to contain special characters.
them. SQL engines sometimes add new keywords; explicitly marking string >>>> literals as string literals prevents future conflicts and confusion.
Perhaps it's a better idea to use [identifier] or `identifier` instead >>>> though (I just learned about those on
https://sqlite.org/lang_keywords.html). Both are not standard SQL ([] is >>>> used in MS Access and SQL Server, `` is used in MySQL) but both work in >>>> SQLite. That should prevent any ambiguity and confusion, if it doesn't >>>> bother you too much that it's not standard SQL.
Why not just use "identifier" which is standard SQL?
If you accidentally type [identifire] or `identifire`, SQLite will
produce an unknown identifier error, alerting you immediately to your typo. >> If you accidentally type "identifire", SQLite will silently treat it as
a string literal instead of an identifier, causing more difficult to
diagnose problems.
Okay, so..... exactly the same as if you use standard double quotes,
but change the configuration option. So the options are: make
everything worse for everyone by exacerbating the problem of
non-standard identifier quoting, or get this API so SQLite can be
configured, like the OP actually asked for.
Yeah. Let's not do the wrong thing.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Thanks for the discussion. I’m aware that SQLite has several different options for identifier quoting, but they’re not cross-compatible with other SQL, whereas double quotes are (modulo this strange SQLite behavior).interesting is in `Modules/_sqlite/module.c`, but I’m not familiar with the cpython internals.
Is anyone here familiar with the python sqlite3 implementation? I wonder how hard it would be to raise up the `sqlite3_db_config` generically, or have a specific function to set just the DQS_DDL and DQS_DML settings? It looks like everything
John
On 13Dec 2022, at 13:58, Chris Angelico <[email protected]> wrote:
On Wed, 14 Dec 2022 at 08:19, Roel Schroeven <[email protected]> wrote: >>>
Chris Angelico schreef op 13/12/2022 om 20:01:
On Wed, 14 Dec 2022 at 06:00, Roel Schroeven <[email protected]> wrote:
Stefan Ram schreef op 13/12/2022 om 8:42:
"John K. Parejko" <[email protected]> writes:Or that are equal SQL keywords, which can be a reason to double-quote >>>>> them. SQL engines sometimes add new keywords; explicitly marking string >>>>> literals as string literals prevents future conflicts and confusion. >>>>>
I was just burned by this, where some tests I’d written
against an sqlite database did not fail in the way that they
“should” have, because of this double-quoted string issue.
In standard SQL, double quotes denote identifiers that are
allowed to contain special characters.
Perhaps it's a better idea to use [identifier] or `identifier` instead >>>>> though (I just learned about those on
https://sqlite.org/lang_keywords.html). Both are not standard SQL ([] is >>>>> used in MS Access and SQL Server, `` is used in MySQL) but both work in >>>>> SQLite. That should prevent any ambiguity and confusion, if it doesn't >>>>> bother you too much that it's not standard SQL.
Why not just use "identifier" which is standard SQL?
If you accidentally type [identifire] or `identifire`, SQLite will
produce an unknown identifier error, alerting you immediately to your typo. >>> If you accidentally type "identifire", SQLite will silently treat it as
a string literal instead of an identifier, causing more difficult to
diagnose problems.
Okay, so..... exactly the same as if you use standard double quotes,
but change the configuration option. So the options are: make
everything worse for everyone by exacerbating the problem of
non-standard identifier quoting, or get this API so SQLite can be
configured, like the OP actually asked for.
Yeah. Let's not do the wrong thing.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 715 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 12:20:38 |
| Calls: | 12,100 |
| Files: | 15,003 |
| Messages: | 6,517,999 |