No ride today

I thought one day to get accustomed to his new home would be enough. I saddled and bridled Banjo with no problem using Sparkler's snaffle bit, then took him to the arena where I thought I'd have a nice quick ride. Wrong. He shook his head violently, sideways and backwards. We walked a bit but it kept up so I didn't want to mount. What is it with me and horses? I can only get so close but never all the way? It would be so easy to just give up...