We've got up and down arrows, or you can drag vertically to adjust. Funny, you're not the only one who missed that today, so the interface definitely isn't making it obvious. Fixing that is definitely a priority for me.
What you're getting at, about having a separate speed for each book, that’s a good idea and worth exploring. Right now, the speed is just one setting for everything. So if you slow things down for a book that needs a more leisurely pace, you have to remember to speed it back up manually when you switch to something else. Someone else mentioned the same thing earlier. It makes sense that it should remember the pace for each book, just like it remembers where you stopped reading.