While perusing the mailing lists I discovered CerbNG which appears to have similar functionality to Systrace. I think projects like this are key to improving security. Boundaries between the untrusted "outside world" and the trusted "inside world" are dissolving. Road warriors infected with the latest worm use their VPN to connect to the corporate network, bypassing defenses aimed at exterior threats. Increasingly hosts must defend themselves as access control is becoming difficult if not impossible. Organizations are unwilling or unable to segment their networks, as most can't even define the relative importance of their business assets. The future of security is every machine being a bastion host.
If you need a commercial solution, Primary Response from Sana Security "monitors and protects applications at the OS kernel level, building a profile of the application's normal behavior based on the code paths of a running program, then continually monitoring those code paths for deviations from the norm."