Use Gnash to quickly eliminate the Dragon Rider from a distance. The Dragon and himself will not damage the party this way, if they are kept close enough, that is.
Another strategy is to jump on the Dragon. Then fight him, until he falls off and eventually runs around the field and shouts "No way!!!". Then, stand on the big, blue spot on the dragon to be able to ride on it.