Save and restore the system state

[ingoahrns]

Hello,

this is my first time that I'm contacting this group, I hope I'm following your guidelines.

I'm trying to setup a S/W that shall manage several different "instances" of micropython which shall run in a round-robin manner. For this purpose I need to switch between the different instances and I tried to save the state of an instance before switching to another instance which then has to be restored before some scripts are evaluated in the new context.

However, I recognized that switching states is not so easy if possible at all. I'm only targeting at a simplified case:

• All instances have a heap memory of the same size.
• The active micropython interpreter is always working on the same heap and the same mp_state_ctx variable. The saved data will be copied to another memory and restored to the original memory location.
• At the beginning I did not care about the stack, but finally I also tried to save the stack and to restore it.

It is working as long as I do not perform a garbage collection. That means the switching works perfectly and I can switch between different instances and their context but if it comes to garbage collection. The program crashes.

Here is what I tried to do:

    const int PYTHON_HEAP_SIZE = 1024*1024;
    const int PYTHON_STACK_SIZE = 64*1024;
    const int MAX_TASKS_NUM = 10;

    // This array is the MicroPython GC heap for the running context.
    volatile char heap[PYTHON_HEAP_SIZE];
    // This array is the micro python heap storage for switching context.
    volatile heaps[MAX_TASKS_NUM][PYTHON_HEAP_SIZE];
    // This array is the micro python stack storage for switching context.
    volatile stacks[MAX_TASKS_NUM][PYTHON_STACK_SIZE];

    // This is the reference to the micropython state
    extern mp_state_ctx_t mp_state_ctx;

    bool SaveState(int context)
    {
        gc_collect();
        memcpy((char *)&(heaps[context][0]),(char *)&(heap[0]),PYTHON_HEAP_SIZE);
        used_stack_size[context] = mp_stack_usage();
        if (used_stack_size[context] < PYTHON_STACK_SIZE)
        {
            memcpy((char *)&(heaps[context][0]),(char *)&(heap[0]),PYTHON_HEAP_SIZE);
            if (used_stack_size[context] > 0)
            {
                char * top_of_stack = MP_STATE_THREAD(stack_top);
                memcpy((char *)&(stacks[context][0]),top_of_stack-used_stack_size[context],used_stack_size[context]);
            }
            mp_states_ctx[context] = mp_state_ctx;
            current_state_index = context;
            return true;
        }
        else
        {
            return false;
        }
    }

    bool RestoreState(int context)
    {
        if (context != current_state_index)
        {
            // First save the old state
            SaveState(current_state_index);
            // Then restore the requested state
            mp_state_ctx = mp_states_ctx[context];
            mp_stack_ctrl_init();
            memcpy((char *)&(heap[0]),(char *)&(heaps[context][0]),PYTHON_HEAP_SIZE);
            char * top_of_stack = MP_STATE_THREAD(stack_top);
            memcpy(top_of_stack-used_stack_size[context],(char *)&(stacks[context][0]),used_stack_size[context]);
            current_state_index = context;
            gc_collect();
        }
        return true;
    }

    // Initialize all instances...
    mp_embed_init((void *)&(heap[0]), PYTHON_HEAP_SIZE);
    for (int ctx = 0; ctx < MAX_TASKS_NUM; ctx++)
    {
        SaveState(ctx);
    }

    ```

    // Execute a script in instance 0
    RestoreState(0);

    mp_embed_exec_str(script0);


    // Execute a script in instance 1
    RestoreState(1);

    mp_embed_exec_str(script1);

   // and switch back to context 0...
    RestoreState(0);

    ...

The source code shows the C functions I applied to save and restore the python state. 'SaveState(int context)' shall save the state to some variables storing the heap, the stack and the context variable. 'RestoreState(int context)' shall restore the state of a given context.

Does anyone tried something similar and could find a way to solve this? Is it possible at all or am I missing an important point? Any help would be very much appreciated.

Thanks very much in advance.

Ingo