DAVID RICE

In the past week a few issues have started ocurring with Amazon S3 to do with European buckets only it seems.

Firstly I have a number of issues with the official article: http://aws.amazon.com/articles/3912

• It states there is no region-specific endpoint for european buckets
• There is: s3-eu-west-1.amazonaws.com
• It states that using the region-specific endpoint is optional
• In my findings below; it is now required for PUT requests yet must be omitted for GET requests

In the past (i.e. last week!), one could GET and POST to a european bucket using the following style of endpoint.

http://bucket.s3.amazonaws.com/images/1/image.jpg

From the 22nd of October 2010 PUT requests to this endpoint started suffering Broken Pipe errors at a failure rate of around 50%.

GET requests to existing files (with this style of endpoint) are 100% successful.

After some testing, it appears that what does work is changing the endpoint for PUT requests to use the region-specific endpoint for EU.

http://bucket.s3-eu-west-1.amazonaws.com/images/1/image.jpg

After the switch PUT requests are 100% successful. However; a GET request to retrieve the image from the same endpoint fails (with a 404).

It appears one must use two different endpoints for operating on images in EU buckets.

GET: http://bucket.s3.amazonaws.com/images/1/image.jpg
PUT: http://bucket.s3-eu-west-1.amazonaws.com/images/1/image.jpg

Solutions

The nature of this issue appears to me to not particularly relate to any language / library in particlar.. but to massive unannounced changes or failures in the Amazon S3 api.

Attachement Fu

To fix this issue for attachment_fu based apps.

Edit config/amazon_s3.yml

development:
  bucket_name: bucket
  access_key_id: xxx
  secret_access_key: xxx
  server: s3-eu-west-1.amazonaws.com

Edit line 154 of …attachment_fu/backends/s3_backend.rb

def self.hostname
  #@hostname ||= s3_config[:server] || AWS::S3::DEFAULT_HOST
  @hostname ||= "s3.amazonaws.com"
end

Paperclip

Edit your attachment class and add a :server option to :s3_options

class Image < ActiveRecord::Base
has_attached_file :image, 
  :styles => {...}
  :path => ':attachment/:id/:style.:filename',
  :storage => :s3,
  :s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
  :bucket => 'xxx',
  :s3_options => {
    :server => "s3-eu-west-1.amazonaws.com"
  }

All in all I’m a little bit miffed at such an inconsistency from Amazon. Why the change? Why does it sometimes work? Why two endpoints? Who knows. Hopefully they’ll fix this issue or have some official clarification soon. Otherwise, I hope these fixes help you in some way for now