Second Date Tips And Help Please?

I went on a date with a guy that I’ve been talking to for maybe a few days before Christmas. The date went real well, we talked better than I thought since I’m normally real shy and quiet at first but I felt like I could act myself around him since we have a lot in common, but we saw a movie and after we didn’t talk much but we where both tired. I did make some mistakes like mentioning my ex since he had a necklace that my ex gave me from a festival, I didn’t want to say anything but I guess I made a face towards it and he asked what was wrong, I told him but changed the subject. I think the big one was when he got me home. He was taking his seat belt off and at the time I didn’t think he would want to walk me to the door, I just hugged him and said something like ‘we should hang out again’.
Was that really bad?
Since then he has been acting kind of off when I talk to him online, but just a few hours ago we talked real well again. I did ask him that I wanted to either hang out at my place or his place or just go out; he liked the idea but didn’t want to make plans yet because of his work. I live with my dad and it’s a somewhat small apartment, and he lives alone which I’m iffy with since we only had one date so far. We clicked better than I thought we would, but I have only been with one guy in the 23 years I’ve been alive and we dated almost 5 years, so this whole thing is still new to me so I am unsure what to do next to show him that I want to date him, but not act desperate.
So please help!