First make sure you have nc(1) available it is in the pkg:/network/netcat package.
Then configure COM1 serial port in the VM settings as a pipe. Tell VirtualBox the name you want for the pipe and get it to create it.
You can also set up the serial port from the CLI using the VBoxManage command, here my VM is called "Solaris 11.1 Text Only".
$ VBoxManage modifyvm "Solaris 11.1 Text Only" --uart1 0x3F8 4 --uartmode1 server /tmp/solaris-11.1-console.pipe
Start up the VM and in a terminal window run nc and the ttya output of the VM will appear in the terminal window.
$ nc -U /tmp/solaris-11.1-console.pipe SunOS Release 5.11 Version 11.1 64-bit Copyright (c) 1983, 2012, Oracle and/or its affiliates. All rights reserved.