"You don't have to say 'sorry.'" He does shift so that he can look at the wall rather than at her; doing his best not to add to her embarrassment. It's understandable. She offered him something so precious and he has to turn it down -- for her, for him, for him.

He doesn't have any right to take any more places that were never meant for him. You didn't do anything wrong.

"I think you do quite well with people. You're better at it than you think." He smiles at the wall rather than at her. "The wall agrees with me."