Okay, I managed to scrounge some parts together from some of my projects including an 68K, some RAM and an 6845 and made a custom 'CHAR' ROM.
I have enough parts to make a 2 channels of PAULA's audio sub system and a single bit plane.(Excluding Interrupts for the time being, those are on the way)
I just need to figure out how to read/write/refresh the FPDRAM chips, without too much glue logic.
My current limit is a need to keep the Pixel clock low enough, while the horizontal refresh is high enough for a standard VGA, I'll find a comprimise.