The main piece of the puzzle remaining on the E3 is sound.
The handset and internal mic/speaker are connected to the Conexant CX20442-11 voice codec. This is part of the modem chipset and a datasheet has proved impossible to find. Easy enough to find pinouts, but no actual description of the data that goes over the bus.
It is possible to use the modem to output/record sound from the various devices. vm from [http://mgetty.greenie.net/ mgetty] will do this. You need a voice.conf that looks a bit like:
program vm speed 115200 forceV253 on voice_devices ttyS1
Full duplex doesn't successfully work based on tests so far.
Mark Underwood has written an ALSA driver that takes advantage of the fact the codec can be switched to talk to McBSP1 on the processor. This currently only outputs to the handset and records from the internal mic.