How I would design it would be to have a system call which registers the addresses, in user space, where system calls are allowed to come from. After that, any system call not coming from an address in that list will fail.
The list would be editable, but there would be an operation which seals it from further editing.
With this feature, we could load a new plugin into a C program with dlopen, and it would not be able to make syscalls of its own, other than through the existing C library, no matter how prim and proper its call sequences look like.
Possibly, ranges could be used instead: syscalls can originate from several registered address ranges, and that's it. Programs that create dynamic code on the fly, which can make system calls, could allocate trampolines in a registered area, for that purpose.
The list would be editable, but there would be an operation which seals it from further editing.
With this feature, we could load a new plugin into a C program with dlopen, and it would not be able to make syscalls of its own, other than through the existing C library, no matter how prim and proper its call sequences look like.
Possibly, ranges could be used instead: syscalls can originate from several registered address ranges, and that's it. Programs that create dynamic code on the fly, which can make system calls, could allocate trampolines in a registered area, for that purpose.