[Bug 1770040] Re: lbaas load balancer does not forward traffic unless agent restarted

[James Page]

I'm not able to reproduce following the lbaas v2 docs:

https://docs.openstack.org/mitaka/networking-guide/config-lbaas.html

haproxy stats reports both backend server are in the configuration
indicating that haproxy has been reloaded as the pool was updated.

echo 'show stat;show table' | sudo socat stdio /var/lib/neutron/lbaas/v2/aa689d45-6853-44ba-8b46-a40da8663e9a/haproxy_stats.sock
# pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid,sid,throttle,lbtot,tracked,type,rate,rate_lim,rate_max,check_status,check_code,check_duration,hrsp_1xx,hrsp_2xx,hrsp_3xx,hrsp_4xx,hrsp_5xx,hrsp_other,hanafail,req_rate,req_rate_max,req_tot,cli_abrt,srv_abrt,comp_in,comp_out,comp_byp,comp_rsp,lastsess,last_chk,last_agt,qtime,ctime,rtime,ttime,
ea3b4ef0-2cad-40b0-8051-1247c6c99bc0,FRONTEND,,,0,2,2000,4,308,848,0,0,0,,,,,OPEN,,,,,,,,,1,2,0,,,,0,0,0,1,,,,0,0,0,0,4,0,,0,1,4,,,0,0,0,0,,,,,,,,
66153c41-10d7-4f22-a63d-6ab276a0244a,57cc336c-cca9-4c8e-8fd1-680ca7379eff,0,0,0,2,,8,77,212,,0,,1,0,0,7,no check,1,1,0,,,,,,1,3,1,,8,,2,0,,2,,,,0,0,0,0,0,0,0,,,,0,0,,,,,29,,,0,0,0,0,
66153c41-10d7-4f22-a63d-6ab276a0244a,995b2445-ca83-4de4-93d0-fe106501265a,0,0,0,2,,8,231,636,,0,,3,0,0,5,no check,1,1,0,,,,,,1,3,2,,8,,2,0,,2,,,,0,0,0,0,0,0,0,,,,0,0,,,,,33,,,0,0,0,0,
66153c41-10d7-4f22-a63d-6ab276a0244a,BACKEND,0,0,0,2,200,4,308,848,0,0,,4,0,0,12,UP,2,2,0,,0,170,0,,1,3,0,,16,,1,0,,1,,,,0,0,0,0,4,0,,,,,0,0,0,0,0,0,29,,,0,0,0,0,

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to neutron-lbaas in Ubuntu.
https://bugs.launchpad.net/bugs/1770040

Title:
  lbaas load balancer does not forward traffic unless agent restarted

Status in OpenStack neutron-gateway charm:
  Invalid
Status in neutron-lbaas package in Ubuntu:
  Incomplete

Bug description:
  Juju 3.2.7, current 18.02 charms.  Deployed an OpenStack cloud
  including Neutron and LBaaS.

  Made a fresh set of 3 instances, assigned floating IPS to all 3.  Made
  a security group to allow port 80 in.

  Made a fresh load balancer, listener, tcp based healthcheck.  Used nc
  on all 3 instances to listen on port 80.

  Connected to the load balancer floating IP on port 80, immediately
  discoonects me - there's no backends listening.

  Initial haproxy status:
  root at neut001:~# echo 'show stat;show table' | socat stdio /var/lib/neutron/lbaas/v2/b100e451-f3ba-46f3-bde4-274a6d15ae6d/haproxy_stats.sock
  # pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid,sid,throttle,lbtot,tracked,type,rate,rate_lim,rate_max,check_status,check_code,check_duration,hrsp_1xx,hrsp_2xx,hrsp_3xx,hrsp_4xx,hrsp_5xx,hrsp_other,hanafail,req_rate,req_rate_max,req_tot,cli_abrt,srv_abrt,comp_in,comp_out,comp_byp,comp_rsp,lastsess,last_chk,last_agt,qtime,ctime,rtime,ttime,
  171808be-5e02-4bb9-8dbd-e559d541f473,FRONTEND,,,0,1,2000,1,0,0,0,0,0,,,,,OPEN,,,,,,,,,1,2,0,,,,0,0,0,1,,,,,,,,,,,0,0,0,,,0,0,0,0,,,,,,,,

  
  (no backends)

  Restarted neutron-lbaasv2-agent

  root at neut001:~# echo 'show stat;show table' | socat stdio /var/lib/neutron/lbaas/v2/b100e451-f3ba-46f3-bde4-274a6d15ae6d/haproxy_stats.sock
  # pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid,sid,throttle,lbtot,tracked,type,rate,rate_lim,rate_max,check_status,check_code,check_duration,hrsp_1xx,hrsp_2xx,hrsp_3xx,hrsp_4xx,hrsp_5xx,hrsp_other,hanafail,req_rate,req_rate_max,req_tot,cli_abrt,srv_abrt,comp_in,comp_out,comp_byp,comp_rsp,lastsess,last_chk,last_agt,qtime,ctime,rtime,ttime,
  171808be-5e02-4bb9-8dbd-e559d541f473,FRONTEND,,,0,0,2000,0,0,0,0,0,0,,,,,OPEN,,,,,,,,,1,2,0,,,,0,0,0,0,,,,,,,,,,,0,0,0,,,0,0,0,0,,,,,,,,
  7ef8ab16-8fec-4ed5-9883-2613d4295476,632625a8-50cf-40e2-9c18-bdfdf79cac3c,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,0,0,13,0,,1,3,1,,0,,2,0,,0,L4OK,,0,,,,,,,0,,,,0,0,,,,,-1,,,0,0,0,0,
  7ef8ab16-8fec-4ed5-9883-2613d4295476,16584f9d-e035-4550-9094-b26078e1fc87,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,0,0,13,0,,1,3,2,,0,,2,0,,0,L4OK,,0,,,,,,,0,,,,0,0,,,,,-1,,,0,0,0,0,
  7ef8ab16-8fec-4ed5-9883-2613d4295476,d39e8de3-95a6-417d-81eb-4fca9079e57e,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,0,0,13,0,,1,3,3,,0,,2,0,,0,L4OK,,0,,,,,,,0,,,,0,0,,,,,-1,,,0,0,0,0,
  7ef8ab16-8fec-4ed5-9883-2613d4295476,BACKEND,0,0,0,0,200,0,0,0,0,0,,0,0,0,0,UP,3,3,0,,0,13,0,,1,3,0,,0,,1,0,,0,,,,,,,,,,,,,,0,0,0,0,0,0,-1,,,0,0,0,0,

  (all 3 backends)

  After restarting the service, all the traffic passes perfectly.  The
  only thing I did was restart the service - I suspect there's some kind
  of race condition where we need to restart the services after the
  config is changed.

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-neutron-gateway/+bug/1770040/+subscriptions