Amazon S3 requires all machines making requests be within 15 minutes of an Amazon S3 webserver’s clock
Reading documentation and various blog posts made me believe, this is problem with the date/time settings on machine (which makes request). However, in our case, it turned out to be something else.
Imagine a simple use-case:
- Read data from S3
- Process data on EC2-instance/your-server
- Store processed-data on S3
In our case, #2 takes longer than 15 minutes (allowed limit by S3) sometimes, hence, we were noticing ‘RequestTimeTooSkewed’ error.
Anyway, we fixed it by explicitly creating a different/fresh S3(http) connection before sending request to S3. This might sound inefficient, perhaps it is.
This is how we do now, a stripped down version of code from our class:
s3_connection = boto.connect_s3(aws_access_key_id="aws_access_key_id",aws_secret_access_key="aws_secret_access_key") output_bucket = s3_connection.get_bucket("output_bucket") key = output_bucket.new_key(key_name) key.set_contents_from_filename(file_path, file_headers, True, None, 10, file_policy)
I am Python and boto noob; I am sure there would be a better way of doing this, so please share if you know one.