login to server ok but cannot login to another server from the first

[Mark Barton]

I have two servers A and B and I am able to login 
to both servers from my client machine using key 
pair. However if I log into server A then try to 
login to server B from A, the authentication check 
drops to password only. The same thing happens if 
I log into server B and then try to login to A. 
However if I use server B console, then I can 
access server A with a key without any problem. 
Server A is headless so I am unable to try the 
same test from A to B. SSH log does not give much 
information. This was working when server A was 
running Ubuntu 16.04. I am now using 18.04.

My ultimate goal is to allow me to login to server 
A and have Git access to the Git server on server 
B using Gitolite.

Server A - Ubuntu 18.04

Server B - Ubuntu 14.04 (yes I am upgrading soon)

Client - Ubuntu 16.04

I thinking there must be a SSH configuration 
option but I'm not sure where to start.

Thanks for any help

Mark