Should I be feeling more for this guy after a few dates? Is it normal?

Give it time. Wait another month. If you don't develop more feeling for him, then let it go. As it will crumble soon anyways if you don't have the feelings for him.