How does Konsole and SQLite locking/unlocking co-ordinate?

Often times while building, I’m working in the open application that uses a SQLite database as a resource, while also working in that database in Konsole via a sqlite3 session.

Sometimes I get stupid and start a transaction in Konsole and do something in the application to test it before committing the transaction and get a database is locked error. That’s good and I’m glad it happens but I can’t get it to clear after SQLite releases the lock.

My question is, Is there a way to get Konsole to clear that lock? I should say, I take care of the lock by stopping the application and/or committing the transaction when possible; but the sqlite3 session in Konsole will not clear that lock internally somewhere unless I close Konsole. Exiting the sqlite3 session in Konsole using .exit does not convince Konsole. I have to close every open Konsole session even if sqlite3 is not or was not open in it.

I may be misunderstanding but it also appears that the lock is applied to everything, including the session that started the transaction. For example, I had the application open but idle and wrote 24 GiB to the database all in a single transaction. While the data was being written via a Tcl script (run fron Konsole using tclsh filename.tcl) that started the transaction, the application must’ve tried to save state and could not because the database was locked. That is fine and what is supposed to happen; but the problem is that, after all the data was written, the script failed at the last line of committing the transaction because the database was locked and rolled it back. That I find confusing since it is the script that had the lock while it was writing and only the application’s write request should’ve been blocked. (I’m not using the Write-Ahead Log mode in SQLite.)

I understand that SQLite does the locking and unlocking but somewhere it appears that either SQLite doesn’t inform Konsole that it released the lock or Konsole ignores it. And the only way to free it is to close all Konsole sessions. It appears as if once Konsole encounters a lock error, it applies to all requests (even those running) and does not release it unless closed.

It’s my own fault for not being more careful but I was wondering if this might indicate that I have something set up incorrectly, because it seems wrong to have the script fail due to a lock when it has the lock.

Thank you.

There is no coordination - it is two different applications.

A 24 GiB database transaction?! That is a bottleneck you need to address possibly by refactoring the database commit into smaller pieces…

This is a unique issue related to sqlite and your code.

With my knowledge of sqlite, I am thinking that your system ran out of memory… and that is what caused the whole system to lock down

You may be able to - when configured - to use the emergency sync provided by the kernel → see [HowTo] reboot / turn off your frozen computer: REISUB/REISUO

Thank you. After closing all Konsole sessions and the application, and opening the SQLite CLI in a new Konsole session, the write of the 24 GiB in a single transaction completed without issue. It does not consume much memory because it writes the data to disk and the information needed to roll it back if necessary.

This has taken place several times with much smaller transactions, even a few bytes to save the state of the UI. In fact, you don’t have to write anything but only start a transaction. I’m not suggesting there is anything “wrong” but was wondering why closing SQLite and starting it again doesn’t clear the lock. If SQLite closed then the connection to the database is closed and it would appear that it would be impossible to have a lock. But, if do nothing else, but close Konsole and re-open it and the SQLite CLI in it, the lock is cleared; and the converse is that no matter what else I do, apart from closing Konsole, the lock is not cleared.

It appears to occur in one direction only, For example, if the application is open and I open SQlLite CLI in Konsole and start a transaction and then the application tries to write, the busy handler of the application runs because the CLI has a lock. If I rollback or commit the open transaction in the CLI, the application will resume working and writing to the database; but I cannot ever write again in the CLI because it will always show a lock error, when there isn’t a lock. But if close Konsole and re-open, the lock error is gone.

I haven’t encountered an issue when running the application as it should be run (not messing with the database while it is open) because the writes are short and quick. I have to fabricate a scenario to test a lock and in this application that could occur only if the user opens two instances of it and can trigger two concurrent writes, which is about impossible.

Being a developer myself - I have often questioned my code - I have also occasionally blamed the operating system by thinking it could no be my code.

With a few exceptions - it was always my code - forgetting to clear up dangling connections - indefinite loops - lack of error handling, just to name a few causes.

With client/server applications it is difficult to pinpoint exactly where the bug occurs so I recommend that you apply very extensive logging when executing in your development environment, this may be the only way you can narrow down the issue.

@linux-aarhus

Thanks. I’m not a developer; just a hack working on a project. Fortunately, the application itself has never had a lock issue, it is only when I’m updating the database while the application is open (which I ought not to do) and forget to commit the transaction before returning to the open application to see that the database changes are reflected in it. It’s likely my busy handler because I haven’t done much with it yet since I have to make up a scenario to trigger it and have deferred that for a bit.

Unless I can reproduce this using only two Konsole sessions— one running the SQLite CLI (removing my application altogether) and the other tclsh with SQLite loaded, I have no evidence to provide. I’m not suggesting there is some problem in the OS, Konsole, or SQLite; but was only wondering why the rolling back or committing doesn’t free the lock.

I’ll try to make an example that shows this occur but completely separate from my application and perhaps I can learn what I’m overlooking.Thanks.

@linux-aarhus
Here is an example that illustrates what I was trying to describe. I’m still not suggesting that the OS, Konsole, SQLite, or Tcl have anything wrong but would like to understand why the lock is not released under these circumstances which are, really, triggered by a user error or oversight.

First, I open a Konsole session/instance and run the SQLite CLI opening/creating a new database file. I think you have to write something to the database before SQLite creates the file.

❯ sqlite3 junk.db
SQLite version 3.53.4 2026-07-24 19:02:57
Enter ".help" for usage hints.
sqlite> create table t1 (key,value);
sqlite> insert into t1 values (1,'a');
sqlite> select * from t1;
╭─────┬───────╮
│ key │ value │
╞═════╪═══════╡
│   1 │ a     │
╰─────┴───────╯

Next, I open another Konsole session/instance and start a live Tcl session, connecting to the newly created database. Create a bad busy handler the never stops trying again to test for the release of a lock.

❯ tclsh
% package require sqlite3
3.53.4
% sqlite3 dbcmd junk.db
% dbcmd eval {select * from t1;}
1 a
% proc BusyHandler {attempts} {
chan puts stdout "Still waiting after $attempts attempts."
return 0
}
% dbcmd busy BusyHandler    

Return to the SQlite CLI session in the first Konsole and open a transaction that acquires an immediate lock on the database.

sqlite> begin immediate;

Return to the Tcl live session in the the second Konsole and attempt to write to the database.

dbcmd eval {insert into t1 values (2,'b');}

The busy handler which fails to return a 1 after a maximum number of attempts keeps scrolling up the screen every time it tries again; and, then, Ctrl+Z to suspend the session.

Still waiting after 223675 attempts. 
^Z 
zsh: suspended  tclsh

Return to the first Konsole and end the transaction, which should release the lock.

sqlite> rollback;

Return to the second Konsole and start a new Tcl live session connecting to the same database; and try to write to it.

❯ tclsh
% package require sqlite3
3.53.4
% sqlite3 dbcmd junk.db
% dbcmd eval {select * from t1;}
1 a
% dbcmd eval {insert into t1 values (2,'b');}
database is locked
% 

Why does this instance of Konsole or Tcl live session still think that the database is locked? The old Tcl session was terminated, then the lock was released by the SQLite CLI that acquired it, and a new Tcl live session was started. Should the new Tcl session not know that the file is no longer locked? Or is the file still locked?

To test whether or not the file is still locked, I tried leaving that second Konsole session and opening a third with a new SQLite CLI session, and it failed.

❯ sqlite3 junk.db
SQLite version 3.53.4 2026-07-24 19:02:57
Enter ".help" for usage hints.
sqlite> select * from t1;
╭─────┬───────╮
│ key │ value │
╞═════╪═══════╡
│   1 │ a     │
╰─────┴───────╯
sqlite> insert into t1 values (2,'b');
Error near line 2: database is locked

But, if leave this third Konsole session just as it is and return to the second and exit the Tcl live session and exit Konsole, then return to third and repeat the write attempt, it succeeds.

sqlite> insert into t1 values (2,'b');
Error near line 2: database is locked
sqlite> insert into t1 values (2,'b');
sqlite> select * from t1;
╭─────┬───────╮
│ key │ value │
╞═════╪═══════╡
│   1 │ a     │
│   2 │ b     │
╰─────┴───────╯

This is what appears odd; for it was the first Konsole session running the SQLite CLI that acquired the lock and released the lock but it is the second session that appears to “keep the lock active” until Konsole is closed. I suppose it has something to do with using Ctrl+Z.

Anyway, sorry for the long explanation, but this was the crux of my original question and really has nothing to do with the specifics of my application code.

Of course, if I fix the BusyHandler to return 1 after some maximum number of attempts then it stops trying again and the Tcl live session does not have to be suspended and this does not occur.

Thanks.

I don’t know… and I have no ideas…

But I am fairly confident that Konsole (a virtual terminal) has nothing to with it.

I am sure you can reproduce the issue using any virtual terminal e.g. alacritty, ghostty or xterm.

The virtual terminal is serving as a launchpad for the application or code which is then interacting with the database.

The application - whether this is sqlite itself or tcl or your code - is responsible for the locking and unlocking.

If the database is still locked at an unexpected point of execution then it is likely a bug somewhere.

I am not qualified in determining where in sqlite binary, tcl binary or your code the issue is located.

Because, as @linux-aarhus has already told you, they are separate processes with separate parent processes.

Every konsole session is its own thing, and whatever you do in there will all be child processes of that particular konsole session. So if you open two konsole sessions and (attempt to) connect to the same sqlite database from both of them, then neither session will know of the other session’s attempt, and to sqlite, they are unrelated processes.

UNIX is a multi-tasking and (concurrent) multi-user operating system, but judging by your inquiry, you are expecting it to behave as a single-tasking one.

As they say, PICNIC — Problem In Chair, Not In Computer. :stuck_out_tongue:

This does not release the lock, it just rolls back the latest change…

Please read the docs

Especially

Thanks @linux-aarhus

This does not release the lock, it just rolls back the latest change…

Please read the docs

Please consider this example, much like the previous.

Konsole 1

❯ sqlite3 junk.db
SQLite version 3.53.4 2026-07-24 19:02:57
Enter ".help" for usage hints.
sqlite> create table t1 (key,value);
sqlite> insert into t1 values (1,'a');
sqlite> select * from t1;
╭─────┬───────╮
│ key │ value │
╞═════╪═══════╡
│   1 │ a     │
╰─────┴───────╯
sqlite> begin immediate;

Konsole 2

❯ sqlite3 junk.db
SQLite version 3.53.4 2026-07-24 19:02:57
Enter ".help" for usage hints.
sqlite> select * from t1;
╭─────┬───────╮
│ key │ value │
╞═════╪═══════╡
│   1 │ a     │
╰─────┴───────╯
sqlite> insert into t1 values (2,'b');
Error near line 3: database is locked

Back to Konsole 1 to rollback the transaction.

sqlite> rollback;

Back to Konsole 2 to try again.

sqlite> insert into t1 values (2,'b');
sqlite> select * from t1;
╭─────┬───────╮
│ key │ value │
╞═════╪═══════╡
│   1 │ a     │
│   2 │ b     │
╰─────┴───────╯

Does this not demonstrate that ROLLBACK released the lock, because it closed the open write transaction? If the lock was not released by ROLLBACK why was the INSERT in Konsole 2 performed; and, if not, how should the lock be released, other than by a ROLLBACK or COMMIT`?

I ask this politely, but can you show an example in which a write is blocked by a database lock error after the ROLLBACK is executed and before a new BEGIN is issued?

Note: One might be able to sort of cheat when using the SQLite Tcl Interface by using the Transaction Method (The Tcl interface to the SQLite library) which will “permit” a nested BEGIN; but I’m not talking about that sort of thing; and I don’t use it because I want to throw an error on a nested BEGIN. I pass an array of database requests and want them to all succeed or all fail.

The SQLite documenation and I have become pretty close friends over the past five years or so, but I looked at the links you provided and do not find where it reads that ROLLBACK does not end a transaction, and therefore the lock. I see this:

Transactions can be started manually using the BEGIN command. Such transactions usually persist until the next COMMIT or ROLLBACK command.

Could you please point me to where in the documentation you found that ROLLBACK does not release a lock?

Thanks @Aragorn

So if you open two konsole sessions and (attempt to) connect to the same sqlite database from both of them, then neither session will know of the other session’s attempt, and to sqlite, they are unrelated processes.

UNIX is a multi-tasking and (concurrent) multi-user operating system, but judging by your inquiry, you are expecting it to behave as a single-tasking one.

Whether or not one running instance of SQLite knows that there is more than one connection to the same database, any instance can and must acquire a lock (implicit or explicit) to the database in order to write. As you likely know, SQLite is not a server-type database and is not made to handle concurrency in the same manner as they. I realize that this scenario is a multi-tasking scenario, but only one process (more likely only one connection) can ever write to the same SQLite database at any one time; that applies to the entire file not only the same table(s). That multi-tasking scenario is the crux of my question, which concerns why, after the process that acquired the lock releases it, does the other process still “think” there is a lock.

In the example in this post, you can see that, after the ROLLBACK, the second SQLite CLI instance, “knows” the lock has been released and can write to file junk.db. But, in my previous example in which a Ctrl+Z was used to end the tclsh process, even after a restart of tclsh, both tclsh and a new SQLite CLI session “thought” the database was still locked; and only after the Konsole instance (in which Ctrl+Z was use to end the tclsh session) was closed could the database be written to again; and that is without issuing any other SQLite commands that might have released an existing lock, and with the original SQLite CLI session that acquired the lock still running.

I may be missing your point, but I don’t see where in my inquiry I’m expecting UNIX to behave as a single-tasking system; for I’m asking about at least two tasks where on has an immediate or exclusive lock on a database file and another, waiting to acquire a lock, does not appear to be informed that the lock has been released. Probably I should write that as the other task likely doesn’t check again; because it’s not actually informed but, at its next attempt to acquire a lock, the file should be available.

That all being said, none it matters much because it all has to do with what takes place after a Ctrl+Z to stop a session. I can’t make it take place otherwise and have not ever experienced it otherwise. It never takes place in my application; and the original reason for asking was out of curiosity of why, when I forget to commit a transaction and subsequently trigger a busy handler, do I have to close Konsole even after the lock has been released and the script having the busy handler has been terminated.

As they say, PICNIC — Problem In Chair, Not In Computer.

Good point. Maybe I should try one of those stand-up desks/work-stations. I’ve read that the Founders (of the U.S.) used them a lot; but that was claimed by a company that makes stand-up desks. Supposed to improve blood flow to the mind, I think.

I have used sqlite in python project long time ago.

I don’t recall if I used transactions. (I did look up the code, it is a single user app - so no transaction code)

As I understand a rollback - it is not the same as locking the database.

Instead a rollback is executed when set of records cannot be stored succesfully.

When working with a relational database it is very important to keep the integrity between related records in different tables.

For that you use a transaction and if that transaction fails for any reason then you use rollback to ensure you do not have inconsistent records.

So - in pseudo code

begin transaction

    try
        insert into bla1
        insert into bla2
        insert into bla3

    catch
        rollback

end transaction

That may not be how sqlite work - I have no idea - as I said - I am not qualified.

Your issue look more like something you would research in the sqlite documentation and using sqlite database forum.

  1. Some operations may require a read process, which will in turn yield whether there is or isn’t a lock file.

  2. I don’t know why you are using CtrlZ or what this is supposed to achieve, but in UNIX, the key combination for interrupting a process is CtrlC, and for sending an end-of-transmission or logout to the terminal it is CtrlD.

    As best I know, CtrlZ is a DOS/Windows end-of-file character. UNIX does not have such a character.


Edit: CtrlZ does send a “suspend” signal to the terminal, though.

My example in Post 6 shows the use of it at the ^Z and the result of using it, which stopped the tclsh session and its never-ending busy handler due to the lock on the database file.

Still waiting after 223675 attempts. 
^Z 
zsh: suspended  tclsh

But I hadn’t thought that “zsh: suspended tclsh” might mean something other than that the tclsh session was closed; because it stops and you get a new CLI prompt. Evidently, there is a difference; because I repeated that example using Ctrl+C and it recognized that the lock had been released and the Konsole session did not have to be closed to write to the database again. And that is the answer to my original question.

I don’t know from where I got Ctrl+Z except a habit from using Windows as you suggested. Funny thing is that I often forget and try to use Ctrl+C to copy while in a tclsh live session and it ends the session.

I just looked in an internet search engine and Ctrl+K is the shortcut to suspend a process in the CLI. Please see Linux: Suspending a Process — linuxvox.com. I didn’t know that could be done.

@linux-aarhus

As I understand a rollback - it is not the same as locking the database.

Right. A ROLLBACK releases a previously acquired lock after “undoing” any operation that succeeded before it was issued. It doesn’t lock the database; you need a BEGIN or BEGIN IMMEDIATE or BEGIN EXCLUSIVE, etcetera to create the lock. (Well, that’s not exactly correct because these start a transaction and the lock is created only when a transaction is upgraded to a write; and that’s controlled by SQLite not the SQL statements.) But you wrote in Post 9 that

This does not release the lock, it just rolls back the latest change…

I appreciate your help and am not trying to be rude in any way, but I’m confused what your saying because one time it’s ROLLBACK doesn’t release a lock and the next it’s that ROLLBACK isn’t the same as locking the database.

[Added: I think I see what you meant now, after re-reading. You’re making a difference between ROLLBACK and “end transaction” in your pseudo code and wrote SQLite may not work that way. Got it, thanks.]

Fortunately, that doesn’t matter, here, and the issue is that I don’t know how to use Linux properly nor the difference between suspending a process and killing it. The upshot is use Ctrl+C not Ctrl+Z.

Thanks for the help.

Ah, no, I forgot to mention this, but CtrlZ suspends a running process. It does not stop the process, but temporarily pauses it.

In a terminal window, you have to use CtrlShiftC to copy to the clipboard. :wink:

A suspended process can be resumed again with CtrlQ. :wink:

If you want, (in the terminal) you can rebind Ctrl + C to copy and Ctrl + V to paste.

The SIGINT functionality will automatically rebind to Ctrl + Shift + C, there’s also Ctrl + Alt + C which works either way.

Once a job is suspended, you can also control it with commands like bg and fg - for instance bg on its own will start the last-suspended process running in the background. The fun you can have on the command line :partying_face:

Out of curiosity, I’ve experimented a bit with suspending a process and don’t understand the behavior at times.

First an example that makes sense that is a pattern for the second example that does not appear consistent.

In Konsole 1, start SQLite CLI and perform the following. This leaves the transaction wrapping the second insert open and the database locked. From this connection, the changes due to the second insert are visible but the transaction has not yet been committed.

❯ sqlite3 junk.db
SQLite version 3.53.4 2026-07-24 19:02:57
Enter ".help" for usage hints.
sqlite> create table t1 (key,value);
sqlite> insert into t1 values(1,'a');
sqlite> begin immediate;
sqlite> insert into t1 values(2,'b');
sqlite> select * from t1;
╭─────┬───────╮
│ key │ value │
╞═════╪═══════╡
│   1 │ a     │
│   2 │ b     │
╰─────┴───────╯

In Konsole 2, start a second SQLite CLI session and perform the following, which shows that a SELECT from this connection does not show the uncommitted changes of the open transaction and a write attempt fails because the SQLite session in Konsole 1 has the lock. Then suspend the session in Konsole 2.

❯ sqlite3 junk.db
SQLite version 3.53.4 2026-07-24 19:02:57
Enter ".help" for usage hints.
sqlite> select * from t1;
╭─────┬───────╮
│ key │ value │
╞═════╪═══════╡
│   1 │ a     │
╰─────┴───────╯
sqlite> insert into t1 values (3,'c');
Error near line 2: database is locked
sqlite> ^Z
zsh: suspended  sqlite3 junk.db

Return to Konsole 1 and commit the open transaction, which succeeds since it is this SQLite session that acquired the lock and can now release it via the COMMIT.

sqlite> commit;
sqlite> 

Return to Konsole 2, use fg to resume the SQLite session; and try to select again. This time the changes due to the second insert are shown because that transaction was committed; and a re-try of the insert succeeds because the lock in Konsole 1 was released with the COMMIT.

❯ fg
[1]  + continued  sqlite3 junk.db
        
sqlite> select * from t1;
╭─────┬───────╮
│ key │ value │
╞═════╪═══════╡
│   1 │ a     │
│   2 │ b     │
╰─────┴───────╯
sqlite> insert into t1 values (3,'c');
sqlite> select * from t1;
╭─────┬───────╮
│ key │ value │
╞═════╪═══════╡
│   1 │ a     │
│   2 │ b     │
│   3 │ c     │
╰─────┴───────╯
sqlite> 

That all appears to make perfect sense to me; and if repeat the exact same steps except that do those of Konsole 2 throught a Tcl live session with the SQLite package loaded, the results are identical. But, if add in the busy handler, the results differ most of the time. Note that the point of the busy handler is to wait a very short interval of time and try again to write.

In Konsole 1, do the exact same as above.

❯ sqlite3 junk.db
SQLite version 3.53.4 2026-07-24 19:02:57
Enter ".help" for usage hints.
sqlite> create table t1 (key,value);
sqlite> insert into t1 values (1,'a');
sqlite> select * from t1;
╭─────┬───────╮
│ key │ value │
╞═════╪═══════╡
│   1 │ a     │
╰─────┴───────╯
sqlite> begin immediate;
sqlite> insert into t1 values (2,'b');
sqlite> select * from t1;
╭─────┬───────╮
│ key │ value │
╞═════╪═══════╡
│   1 │ a     │
│   2 │ b     │
╰─────┴───────╯
sqlite> 

In Konsole 2, start a Tcl live session, use SQLite to connect to the same database, select the contents of table t1. The second insert is not yet visible to this connection because that transaction was not yet committed. Declare the busy handler for this connnection and then attempt to write to the database. This triggers the busy handler because SQLite in Konsole 1 has acquired a lock to write and the transaction is still open; and then Ctrl+Z to suspend the Tcl live session. Note that the tcl session, at the point of suspension, is not idle but is printing these messages to stdout every time the busy handler checks to see if the lock has been released.

❯ tclsh
% package require sqlite3
3.53.4
% sqlite3 dbcmd junk.db
% dbcmd eval {select * from t1;}
1 a
% proc BusyHandler {cnt} {
puts "Database is still locked after $cnt attempts."
return 0
}
% dbcmd busy BusyHandler
% dbcmd eval {insert into t1 values (3,'c');}
...
Database is still locked after 332913 attempts.
Database is still locked after 332914 attempts.
Database is still locked after 332915 attempts.
Database is still locked after 332916 attempts.
Database is still locked after 332917 attempts.
^Z
zsh: suspended  tclsh

Return to Konsole 1 and try to commit the transaction. It fails because of database is locked error.

sqlite> commit;
Error near line 6: database is locked
sqlite> 

Why does this happen? The SQLite session in Konsole 1 has the lock; how can it receive a lock error on the lock it acquired? Nothing running in Konsole 2 could’ve have acquired a lock,until this first one was released. If fg in Konsole 2 and Ctrl+C and return to Konsole 1, the commit succeeds without taking any additional steps to release a lock.

While experimenting with these examples, there were a few occasions in which the commit in Konsole 1 did not fail, and when fg in Konsole 2, the busy handler ran one more iteration, found that the lock was released, and wrote the (3,‘c’) row to the database. This is what I was expecting to happen; but most often the results are as above.

Is this what is supposed to happen or am I doing something wrong? Could the results of the second example depend upon at which point the busy handler was when the session was suspended? Thank you.

Unless you have very specific requirements - you should never fiddle the process

:smile:

Doing so will often lead to spider or frog rain, thus infesting your system using unspeakable horrors - some of which will be locking your data file(s) indefinitely or wipe your file system table on your root device.

My, my… one hopes it doesn’t come to that;
frogs are virtually impossible to brush out of one’s hair.