I'm on 2060.165 which I got a month ago, and I'm not enrolled in the beta test program. Maybe it's region dependent? Since I got the update I restarted my device a number of times and the second sim remained disabled.
I think the problem is fixed now. I just got update version 33.0804.2060.165 today and the second SIM card wasn't activated after reboot. I rebooted it again to see whether it was a one time thing, but it wasn't, the deactivated SIM remained so after...
I had a Galaxy A52 before I got the Zenfone 9 and it didn't have this problem, so AOSP clearly doesn't enforce this behavior, or at least doesn't prevent the manufacturer from changing it.