Doing this brings you close to OSI, which famously failed by being overcomplicated.
We're slowly reinventing OSI, one step at a time: OSI had multiple sessions per transport connection (QUIC), 20 byte addresses (IPv6) and a directory system with public-key infrastructure (DANE, vCard, SSHFP, etc).
It's a shame TUBA (CLNP + TCP) failed.