DP is a one way connection and the only reason it works with the adapters is there's nothing between it like a KVM. Once you start introducing "complexity" to it, it just doesn't handle it well.

And when I say it's "one way" I mean the adapter would only work on the computer side and not the monitor side.