GPT-4 is already multimodal
https://news.ycombinator.com/item?id=35510705Prompt it with ascii art and it can literally recognize and describe features of it. Try something like:
Below is a 2D picture encoded as ones for white and zeroes for black. Please identify the features of the ascii representation in detail and then determine what the below image is a representation of, given that it has to do with a comic: 11111111111111111111111111 11111100111111111100111111 11110001111100111110001111 11000001111000011110000011 10000000111000011100000001 10000000000000000000000001 00000000000000000000000000 00000000000000000000000000 10000000000000000000000001 10000110001000010001100001 11001111111100111111110011 11100111111100111111100111 11111111111111111111111111