import shelve
import sqlite3
def test_sqlite3():
conn = sqlite3.connect("debug.s3db")
cur = conn.cursor()
cur.execute("CREATE TABLE IF NOT EXISTS [mydict] ("
"[key] VARCHAR(255) PRIMARY KEY NOT NULL, "
"[value] VARCHAR(255) NOT NULL)")
for i in xrange(0, 1000000):
cur.execute("INSERT INTO [mydict] (key, value) VALUES (?, ?)",
(str(i), str(i*2)))
conn.commit()
cur.close()
conn.close()
def test_shelve():
d = shelve.open("debug.shelf")
for i in xrange(0, 1000000):
d[str(i)] = str(i*2)
d.close()
On my computer, test_sqlite3() cost me about 25 seconds, and about 51 seconds for test_shelve(). I also performed some fetch queries on these two, sqlite3 still runs faster than shelve. In conclusion, sqlite3 wins!July 4, 2011
Python shelve vs. sqlite3
I have a huge Python dict that I can't store in the memory. Python provides a shelve module for this purpose. It acts like a regular dict but it can be saved into a file. I wonder its performance comparing to sqlite3, so I run this little test:
Subscribe to:
Post Comments (Atom)
Thanks for sharing. Shelve is far more simple in coding. I am searching the right way to manage only tens thousands of rows.
ReplyDeleteBut still shelve so simple. Simple is beautiful.
ReplyDelete