D
Deepnote Community
Home
Data science
👋
Introductions
🎨
Showcase
📝
General
❓
Ask anything
📖
Learning & resources
🗓️
Events
Using Deepnote
📣
Announcements
🔊
Suggestions
🐞
Bugs
🔨
Custom environments
Off topic
Links
Deepnote.com
Deepnote for education
Documentation
Changelog
Security
Careers
Log in
Log in
Home
Data science
👋
Introductions
🎨
Showcase
📝
General
❓
Ask anything
📖
Learning & resources
🗓️
Events
Using Deepnote
📣
Announcements
🔊
Suggestions
🐞
Bugs
🔨
Custom environments
Off topic
Links
Deepnote.com
Deepnote for education
Documentation
Changelog
Security
Careers
Denis Shilov
facel.tech
🎨
CEO & Full stack developer at Facel, neural interface for controlling devices with facial gestures and monitoring wellbeing based on signal from 2 points behind the ears.
Posts
40
Comments
69
Spaces
12
mmhmm in docs of that dill library there's a description of what they can serialize and what that cannot. So the algorithm imho should look like this.
Saving to memory.
[{}, {}, .... ]
As our goal here is to save the state for some time (limited number of iterations), then the length of such an array is also limited. Let's suppose that we want Ctrl + Z to work 10 steps behind.
The structure of a dictionary is as follows:
What happens if the user changes something:
If an object is non-serializable, then we add it to "in_memory" (cloning that). If not, then we add its name to "on_disk"
What happens if the user presses Ctrl + Z:
What happens if the user presses Ctrl + Shift + Z:
So I'd advise to dump to pkl everything which is serializable and store somewhere in operating memory everything which is not
That's a real world example from today, when I incorrectly mapped one of the pandas frame columns and got nans everywhere