Tag Archives: amazon

Creating and Customising Amazon EC2 AMI

Do you want to create/customize an EC2 AMI?

Check out this tutorial by Rohan.

Briefly, steps are:

  • Find a suitable base AMI. You can use EC2 Console to search
  • Launch an instance from command-line using ami-id or EC2 Console
  • Copy(scp) your private (ssh) key, ec2 key-pair and certificate from your machine to instance’s /mnt directory
  • SSH to instance
    • Install/customize tools and configuration
    • Test if things work
    • Install amazon-ami-tools
    • Bundle image (from current running instance):
      ec2-bundle-vol -d /mnt -k /mnt/pk-STRINGOFTHIRTYTWOLETTERSORDIGITS.pem -c /mnt/cert-STRINGOFTHIRTYTWOLETTERSORDIGITS.pem -u XXXXXXXXXXXX

      XXXXXXXXXXXX is your Amazon account number without dashes.

    • Deploy/copy AMI files to S3:
      ec2-upload-bundle -b [your-s3-bucket] -m /mnt/image.manifest.xml -a [aws-access-key-id] -s [aws-secret-access-key]

Before bundling the image, do test customised settings and configuration; it’s pain to repeat above steps again, in order to change anything post-deployment.

Tip: Don’t shutdown/terminate the instance, you used to create AMI, until you have tested instances (based on new AMI). If anything goes wrong with new instances, you can come back to this instance (based on base-ami), change, bundle and redeploy.

Amazon S3 RequestTimeTooSkewed Error

Recently, we have started noticing “RequestTimeTooSkewed” erro while making requests to Amazon S3. The official FAQs suggests following:

Amazon S3 requires all machines making requests be within 15 minutes of an Amazon S3 webserver’s clock

Reading documentation and various blog posts made me believe, this is problem with the date/time settings on machine (which makes request). However, in our case, it turned out to be something else.

We use boto (Python library) for AWS stuff. Boto caches the connection objects (for S3, EC2, SQS, etc.), and tries to reuse those in later calls.

Imagine a simple use-case:

  1. Read data from S3
  2. Process data on EC2-instance/your-server
  3. Store processed-data on S3

In our case, #2 takes longer than 15 minutes (allowed limit by S3) sometimes, hence,  we were noticing ‘RequestTimeTooSkewed’ error.

Anyway, we fixed it by explicitly creating a different/fresh S3(http) connection before sending request to S3. This might sound inefficient, perhaps it is.

This is how we do now, a stripped down version of code from our class:

s3_connection = boto.connect_s3(aws_access_key_id="aws_access_key_id",aws_secret_access_key="aws_secret_access_key")
output_bucket = s3_connection.get_bucket("output_bucket")
key = output_bucket.new_key(key_name)
key.set_contents_from_filename(file_path, file_headers, True, None, 10, file_policy)

I am Python and boto noob; I am sure there would be a better way of doing this, so please share if you know one.

Eclipse Plugins

Some Eclipse plugins, I use (with Eclipse Classic SDK), are:

There are bunch of plugins for network monitoring and debugging, don’t remember their names. BTW! I have got two separate installations (32bit Carbon and 64bit Cocoa) of Eclipse. I hope, Adobe would port Flex/Flash Builder to work with 64bit and Cocoa.

InfiBeam – Online Store for India

Infibeam is like Amazon for India, I am mostly interested in books and could find most of the books on Infibeam. There are some, which I believe they would import and ship to you.

Infibeam is shipping free anywhere in India, that’s another good thing.

I have been buying books from Infibeam for sometime, so far experience has been good except one particular instance.

I ordered a book, everything went well and I was told within one or two week, book would be shipped. After waiting for a week or so, I get an email from Infibeam that since book is out of stock, they would not be able to ship it. I exchanged couple of emails with their staff, asking the reasons and how it should be – if you confirmed the order, you should deliver the book, otherwise you might loose some customers.

BTW! It has been more than a month, I believe, they are yet to refund the money for the order they could not complete :-)

Having said all (positive and negative experience), I feel, overall, it looks good to me. That one instance is out of 10, so I am going to continue trying them