I am in the process of upgrading my ARTIQ-7 system to ARTIQ-8 on Windows using MSYS2. I used the offline installer and can also flash firmware and launch artiq_session. As part of this process, I need to convert the old dataset_db.pyon file to dataset_db.mdb using the script in the release notes (https://m-labs.hk/artiq/manual-legacy/releases.html):
from sipyco import pyon
import lmdb
old = pyon.load_file("dataset_db.pyon")
new = lmdb.open("dataset_db.mdb", subdir=False, map_size=2**30)
with new.begin(write=True) as txn:
for key, value in old.items():
txn.put(key.encode(), pyon.encode((value, {})).encode())
new.close()
My test dataset dataset_db.pyon looks like this:
{
"a": 1.0,
"b": npscalar("<f8", b"9ipo5TtN8EA=")
}
Using this file, the script fails with a JSONDecodeError in line 3, column 10. If I delete dataset b, the script executes successfully and generates dataset_db.mdb with a size of 1 GB.
Why does conversion of numpy datatypes not work? In the actual dataset there are a lot of numpy arrays I need to convert, so this would be a huge hassle to convert by hand. And why is the database so large for even a very small dataset?