import shelve import sqlite3 def test_sqlite3(): conn = sqlite3.connect("debug.s3db") cur = conn.cursor() cur.execute("CREATE TABLE IF NOT EXISTS [mydict] (" "[key] VARCHAR(255) PRIMARY KEY NOT NULL, " "[value] VARCHAR(255) NOT NULL)") for i in xrange(0, 1000000): cur.execute("INSERT INTO [mydict] (key, value) VALUES (?, ?)", (str(i), str(i*2))) conn.commit() cur.close() conn.close() def test_shelve(): d = shelve.open("debug.shelf") for i in xrange(0, 1000000): d[str(i)] = str(i*2) d.close()On my computer, test_sqlite3() cost me about 25 seconds, and about 51 seconds for test_shelve(). I also performed some fetch queries on these two, sqlite3 still runs faster than shelve. In conclusion, sqlite3 wins!
July 4, 2011
Python shelve vs. sqlite3
I have a huge Python dict that I can't store in the memory. Python provides a shelve module for this purpose. It acts like a regular dict but it can be saved into a file. I wonder its performance comparing to sqlite3, so I run this little test:
Subscribe to:
Post Comments (Atom)
Thanks for sharing. Shelve is far more simple in coding. I am searching the right way to manage only tens thousands of rows.
ReplyDeleteBut still shelve so simple. Simple is beautiful.
ReplyDelete