[Bug 430093] [NEW] Eucalyptus "403 Forbidden" when trying to run instance after package upgrade

[Etienne Goyer]

Public bug reported:

On karmic, after upgrading Eucalyptus from 1.6~bzr672-0ubuntu4 to
1.6~bzr746-0ubuntu1, trying to run an instance fail with the following
error:

cloudmaster at uec-frontend:~$ euca-run-instances -t c1.medium -k warthogs-key emi-073A1120
Warning: failed to parse error message from AWS: <unknown>:1:0: syntax error
EC2ResponseError: 403 Forbidden
Failure: 403 Forbidden


Sifting through /var/log/eucalyptus, the only interesting bits I found was in cloud-output.log.  Here is what I think is relevant:


10:52:46  INFO erCertificateHandler | POST http://10.153.108.210:8774/axis2/services/EucalyptusGL HTTP/1.1
10:52:47  INFO ClusterUtil          | ---------------------------------------------------------------
10:52:47  INFO ClusterUtil          | -> [ warthogs ] Cluster certificate valid=true
10:52:47  INFO ClusterUtil          | -> [ warthogs ] Node certificate valid=true
10:52:47  INFO ClusterUtil          | ---------------------------------------------------------------
10:52:47  WARN PipelineRegistry     | => More than one candidate pipeline.  Ignoring offer by: internal-query-pipeline-Eucalyptus of type InternalQueryPipeline
10:52:47  INFO ServiceSinkHandler   | cloud/ServiceSinkHandler:admin:35ebcdcd-94c4-4fd6-8bfc-def8a65646c1:MSG_RECEIVED:1253026367.1670:DescribeAvailabilityZonesType
10:52:47  INFO ServiceSinkHandler   | cloud/ServiceSinkHandler:admin:35ebcdcd-94c4-4fd6-8bfc-def8a65646c1:MSG_SERVICED:1253026367.3360:178
10:52:51  WARN PipelineRegistry     | => More than one candidate pipeline.  Ignoring offer by: internal-query-pipeline-Eucalyptus of type InternalQueryPipeline
10:52:51  INFO ServiceSinkHandler   | cloud/ServiceSinkHandler:admin:4af50fe8-2fac-4ecc-950a-74165696f604:MSG_RECEIVED:1253026371.3970:DescribeAvailabilityZonesType
10:52:51  INFO ServiceSinkHandler   | cloud/ServiceSinkHandler:admin:4af50fe8-2fac-4ecc-950a-74165696f604:MSG_SERVICED:1253026371.4380:41
10:52:52  INFO erCertificateHandler | POST http://10.153.108.210:8774/axis2/services/EucalyptusGL HTTP/1.1
10:52:53  INFO ClusterUtil          | ---------------------------------------------------------------
10:52:53  INFO ClusterUtil          | -> [ warthogs ] Cluster certificate valid=true
10:52:53  INFO ClusterUtil          | -> [ warthogs ] Node certificate valid=true
10:52:53  INFO ClusterUtil          | ---------------------------------------------------------------
10:52:58  INFO erCertificateHandler | POST http://10.153.108.210:8774/axis2/services/EucalyptusGL HTTP/1.1
10:52:59  INFO ClusterUtil          | ---------------------------------------------------------------
10:52:59  INFO ClusterUtil          | -> [ warthogs ] Cluster certificate valid=true
10:52:59  INFO ClusterUtil          | -> [ warthogs ] Node certificate valid=true
10:52:59  INFO ClusterUtil          | ---------------------------------------------------------------
10:52:59  WARN PipelineRegistry     | => More than one candidate pipeline.  Ignoring offer by: internal-query-pipeline-Eucalyptus of type InternalQueryPipeline
com.eucalyptus.ws.AuthenticationException: User authentication failed.
        at com.eucalyptus.ws.handlers.HmacV2Handler.incomingMessage(HmacV2Handler.java:165)
        at com.eucalyptus.ws.handlers.MessageStackHandler.handleUpstream(MessageStackHandler.java:115)
        at com.eucalyptus.ws.server.FilteredPipeline$StageBottomHandler.handleUpstream(FilteredPipeline.java:171)
       at com.eucalyptus.ws.server.NioServerHandler.messageReceived(NioServerHandler.java:111)
        at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:114)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:385)
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.unfoldAndfireMessageReceived(ReplayingDecoder.java:459)
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:443)
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:381)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:342)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:329)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:330)
        at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:203)
        at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:53)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:636)
10:53:04  INFO erCertificateHandler | POST http://10.153.108.210:8774/axis2/services/EucalyptusGL HTTP/1.1
10:53:05  INFO ClusterUtil          | ---------------------------------------------------------------
10:53:05  INFO ClusterUtil          | -> [ warthogs ] Cluster certificate valid=true
10:53:05  INFO ClusterUtil          | -> [ warthogs ] Node certificate valid=true
10:53:05  INFO ClusterUtil          | ---------------------------------------------------------------


I tried redownloading the certificate bundle, in case it would help, but it did not: I still get the above 403 error when running euca-run-instances.

** Affects: eucalyptus (Ubuntu)
     Importance: Undecided
         Status: New

-- 
Eucalyptus "403 Forbidden" when trying to run instance after package upgrade
https://bugs.launchpad.net/bugs/430093
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to eucalyptus in ubuntu.