> It was a good thing for user privacy that the keys were directly on the device
You want to think through that one again? With the OpenAI key on device it means anyone could use that key to call (and bill) OpenAI's APIs. It's absolutely not feasible to ship the OpenAI keys on device.
So eventually if they remove the keys from the device, messages will have to go through their servers instead.