12 Sep 2011
Most of us have or have had this syndrome "I need X to do Y".
- I need to join X course to learn Y
- I need to X training to start doing Y project
- I need a Laptop to start programming Python; Desktop doesn't make me productive
- I need an iPhone to start learning iOS programming; Simulator doesn't make it feel real
- I need to join Gym to workout; Workout at home doesn't work :-)
Get over with it this syndrome:
- Start learning/doing as soon as you get curious about anything, we don't have to wait to do X to start doing Y by ourself.
- We can always do X, but who is stopping us from starting with Y?
- Wouldn't it be better to get our hands dirty and try to learn more about Y by ourselves?
Eventually we have to do it by ourselves, doesn't matter if we do X or not. It is more fun and engaging when we do Y by ourself; it is also a litmus test, if we really want to do Y.
Discipline and Passion matters. I wish, I had more of these in me.
12 Sep 2011
Illusory superiority is a topic of psychology, and it is considered to be one of many positive illusions relating to self.
I am not a psychologist. This blog post is based on my observations about this topic in context of coginitive tasks and something highlighted by Dunning-Kruger effect (Research - PDF).
In my experience, if the culture of place, we live or work doesn't provide enough challenges and competiton (validations), it's so easy to get dilusional about our actual competence and have illusory superiority.
In my own experience, we stop learning and improving once we get to a certain level (in an organisation). We think, we know what we are supposed to. We think, we are great or good enough for tasks in hand or in job? I believe, these are symptoms when we start living with "Illusory Superiority" or "Distorted Reality" or "Cognitive Bias".
One of many ways, I avoid to have this distorted view of reality or illusory superiority or coginitive bias is by looking beyond the boundaries of my organisation/city/country. I look up to great people in community at large, places (great companies) and practices. I try hard to follow the gurus and be like them.
Another way is to think like, 'I am a software-developer/technologist and I work at X place/company' NOT 'I am a software-developer/technologist in X place/company'.
The difference between these two statements is, former makes me live in actual reality, and look beyond and place myself in larger community). Latter might make me live with distorted/percieved reality, and place myself in a smaller community (organisation, place, country).
As a leader/mentor, we need to challenge and help our team so that they keep living in reality. We have to do it, carefully, without affecting their self-confidence.
29 Jul 2011
I am evaluating various nosql solutions for our existing/new projects. Bluesmoon recommended about Riak which is an awesome nosql datastore/database (and lot more). I couldn't wait to check it out.
As usual, I relied on Homebrew - the best package manage for OSX - to install applications/libraries. While installing, I got an error:
$ brew install riak
$ Error: Failed executing: make all rel
After searching for a while, I figured out that there are known issues. With the help of Adam's comment on github, I was able to install Riak finally.
We make brew ignore Riak's dependencies while building it.
Following commands should build and install Riak:
$ brew update
$ brew install erlang
$ brew install --HEAD --ignore-dependencies riak -v
28 Jul 2011
Do you want to create/customize an EC2 AMI?
Check out this tutorial by Rohan.
Briefly, steps are:
- Find a suitable base AMI. You can use EC2 Console to search
- Launch an instance from command-line using ami-id or EC2 Console
- Copy(scp) your private (ssh) key, ec2 key-pair and certificate from your machine to instance's /mnt directory
- SSH to instance
- Install/customize tools and configuration
- Test if things work
- Install amazon-ami-tools
- Bundle image (from current running instance):
ec2-bundle-vol -d /mnt -k /mnt/pk-STRINGOFTHIRTYTWOLETTERSORDIGITS.pem -c /mnt/cert-STRINGOFTHIRTYTWOLETTERSORDIGITS.pem -u XXXXXXXXXXXX
XXXXXXXXXXXX is your Amazon account number without dashes.
- Deploy/copy AMI files to S3:
ec2-upload-bundle -b [your-s3-bucket] -m /mnt/image.manifest.xml -a [aws-access-key-id] -s [aws-secret-access-key]
Before bundling the image, do test customised settings and configuration; it's pain to repeat above steps again, in order to change anything post-deployment.
Tip: Don't shutdown/terminate the instance, you used to create AMI, until you have tested instances (based on new AMI). If anything goes wrong with new instances, you can come back to this instance (based on base-ami), change, bundle and redeploy.
28 Jul 2011
Recently, we have started noticing "RequestTimeTooSkewed" erro while making requests to Amazon S3. The official FAQs suggests following:
Amazon S3 requires all machines making requests be within 15 minutes of an Amazon S3 webserver's clock
Reading documentation and various blog posts made me believe, this is problem with the date/time settings on machine (which makes request). However, in our case, it turned out to be something else.
We use boto (Python library) for AWS stuff. Boto caches the connection objects (for S3, EC2, SQS, etc.), and tries to reuse those in later calls.
Imagine a simple use-case:
- Read data from S3
- Process data on EC2-instance/your-server
- Store processed-data on S3
In our case, #2 takes longer than 15 minutes (allowed limit by S3) sometimes, hence, we were noticing 'RequestTimeTooSkewed' error.
Anyway, we fixed it by explicitly creating a different/fresh S3(http) connection before sending request to S3. This might sound inefficient, perhaps it is.
This is how we do now, a stripped down version of code from our class:
s3_connection = boto.connect_s3(aws_access_key_id="aws_access_key_id",aws_secret_access_key="aws_secret_access_key")
output_bucket = s3_connection.get_bucket("output_bucket")
key = output_bucket.new_key(key_name)
key.set_contents_from_filename(file_path, file_headers, True, None, 10, file_policy)
I am Python and boto noob; I am sure there would be a better way of doing this, so please share if you know one.
28 Jul 2011
I have been thinking about open-standards for movie-rental business. I searched to find about such standards but I couldn't locate. Do you know of any?
Basically, a set of specifications/protocols which can be implemented by different players in movie-rental chain. Players include studios, distributor, application-developers, portals, et. al.
- A studio/distributor implements this specification/protocol and exposes API for metadata/streaming(or playback)/file-access from their own server
- Third party developers use the API to to pull metadata and play the content
- Third party developers can charge the user and split the revenue with content owner
- Or, studios/distributors can enable payment-gatway APIs as part of their offering
- Content is always played directly from studio/distributor's server
- Studio/distributor have total control over their assets and also charge based on subscription/per-access
This would solve many problems (billing, stats, etc.), however, there would be concerns:
- someone (end-user, third-party developer) might try to cache/save streams as file and distribute
- and other concerns?
Once we have specifications and protocols in place - a stack using open-source technologies can be built as template, and made available so that everyone in chain can use/customize it to build their part.
I wanted to talk about it on twitter but 140 chars were too short to express my thoughts. Anyway, it's good to have a blog post in place.
Please share your views.
07 Jun 2011
I got Tata Photon+ mobile broadband connection during 2010, it worked well initially for a month or two.I can notice, their services have been degrading since then (late 2010).
Connectivity is not an issue, but download speed and latency.
A recent test results done on June 6, 2011
We all know, most of marketing/advertisements campaigns are big lies. There is big disconnect between the idea/dream they show and the reality.
In the case of Tata Photon+, they advertise about giving 'up to 3.1 mbps' download speed BUT I am yet to get 1 mbps consistently for an hour, leave alone days.
I guess, this is location specific. Places like Kanpur and other cities in UPE are not given enough of importance (network infrastructure and attention)?
Whenever I complain to Tata Photon Customer Support, someone visits, tests using speedtest.net against Delhi/Mumbai servers, and shows me test results (skewed?) which is generally in range of 600 kbps to 1.2 mbps.
When I show him results of tests done against servers in USA and Europe, they would argue - which reflects their ignorance about technology and terminology (ping, latency, etc.)
Yesterday, I filed another complaint over phone. I got a call asking if I was in front of computer, I said "No, but I would be in couple of hours and tomorrow". After couple of hours, I got a SMS suggesting my complaint-ticket had been resolved and marked as closed. Really? No one visited me.
I filed another complaint using web-site, got around 10+ calls within 12 hours. All of them asking same question "What's problem sir". I don't understand, why didn't these guys look at their CRM and read the details (which included the speedtest.net test results)?
Anyway, one of those guys (from some Tata hired agency) visited me today. He repeated the same process of testing against two Indian servers using speedtest.net. This time, he got lucky speed results showed around 1 mbps agaisnt Delhi/Mumbai server. I requested him to visit again tomorrow in afternoon and perhaps at different location.
During middle of meeting, this fellow stood up and walked out. I followed him to ask what happened?
He misbehaved by saying "I heard enough, I am going. That's all we can do".
I felt offended, I asked about his details, he denied giving by saying "Do whatever you can". He left ignoring me.
I feel bad as customer, and also because I have been a Tata employee. I had never expected, one of Tata companies would do such things i.e. poor service and worst customer support (not at phone, but at ground level - those visiting agents).
I feel, I have wasted so many hours talking, thinking and complaining. No one wants to listen and do anything.
- June 24, 2011: I found Tata Photon very slow at my residence, hence filed a fresh complaint (requested that someone should visit me on Sunday at my residence to test speed in that area) via phone. CSR said, they don't work on Sunday, I said - I can be available on weekdays provided an appointment is taken.
- June 25, 2011: I received two calls from Tata Photon support team. In both calls, I was asked same question "what is problem, sir?" with one difference, in latter call I was told that they can visit me on Sunday
- June 26, 2011: I got a call, where I was asked same and some more questions:
- What is problem, Sir?
- Did you buy device with warranty?
- What speed do you get?
I told them, "I am happy with the device but not with the signal and speed in this area (my residence), hence needed someone to visit me and test speed here". They asked me to go to Tata Photon centre, around 8 kms from my place. I asked, "how can they test issues of this area in different area?".
They threw another bomb by saying, a fee( INR 75) needs to be paid by customer, if he/she wants technical-support-engineer to visit customer's place?
I got upset and reminded that this is mobile-broadband. Isn't it their responsibility to make sure connection signal and quality is good in all areas?
Anyway, that was new, I had not paid such fee earlier.
I asked, when did they start charging? She said, "Yesterday". I reminded her, I filed complaint day before yesterday. She said, it has been started on June 20, 2011.
I asked her to pass the call to her senior, who ended up saying the same thing. I told them, the policy is ridiculous.</li>
Why a customer has to pay extra to get service related issues checked/fixed? Isn't customer already paying every month? Isn't it Tata's responsibility to provide services as they promise?
I have invested some money and time to get the device and connection; I can switch to different provider, but device might not be reusable unless someone buys it from me.
Once again, I am disappointed by Tata Tele's services. I am going to move away from them and would not recommend Tata Tele's services to my family and friends.
- June 28, 2011: As an last attempt, I wrote an email to Tata Tele Services contact and also copied the same to UP East Nodal Officer.
- June 29, 2011: I got a call from a senior Engineer, who visited me on the same day. During 3-4 hours long troubleshooting, he figured out the reason behind slow speed, which is, HSIA signals was not as strong as CDMA 1X, and some tower issues.He got an antenna installed, on the roof-top, and connected to my USB device. He promised to get a repeater installed during weekend, so that all devices at our place can get better quality signals.
Once antenna was installed, and some augmentation was done in one of the towers, things improved a bit.</p>
He seemed a good professional and knew the stuff. He earned my respect. I learnt, good guys are very overloaded because someo of their peers don't do their work properly :-)</li>
- July 2, 2011: Two Tata Photon smart junior-engineers visited and installed a repeater at our house. The senior engineer kept his word, he earned more respect.I can feel the difference now; Two different tests during last 5 hours have given very positive results, as shown below:
Once again, I gave my feedback about service and the misbehaviour by one of their peers. I am sure, the good-guys would fix things by putting peer pressure and leading by examples.
As far as fee for visit is concerned, I was told by one of the engineers, it is for the cases where customer's device or computer is found to be faulty, not the service. I wonder, why I was not told the same earlier during the call, which made me upset?
I am going to wait for couple of days. If I get consistently better quality speed, I would strike out the line (about not recommending) above. Anyways, there are some issues with Tata Photon+ customer support and service, which would need to fix, so that all issues can be resolved without any escalation.
Tip: Choose HSIA from network-settings of Tata Photon+ dialer, this would force modem to use to HSIA enabled towers/frequencies and would avoid using CDMA 1X.</li>
06 May 2011
We have been working on some exciting stuff for over last three years, we want to take that to next level. We also have some crazy and cool ideas for new products/saas, we have got enough details figured out to get started.
We want some smart interns to work with us during this summer or longer (perhaps six months or full-time ) in any or all of following:
- Web Development (goal: build new products or extend existing products)
- Mobile (goal: turn our ideas into reality by building some cools apps and games for iPhone, Android and other mobile-devices)
- Flash Platform (goal: build some cool applications for our existing project)
Summer vacation has started in most of the colleges in India; Some passionate students would want to do something more useful during this time. If you are one of them, why don’t you get in touch with us?
BTW! We are always looking to hire smart and passionate designers and developers for our team. If you are interested into a full-time position, please let me know.
28 Feb 2011
I posted about our proposed development setup, during these years a few things have changed a bit. More or less, this is what we have:
- GNU/Linux (Ubuntu) and Windows(some) boxes for developers
- Eclipse (PDT) IDE with loads some plugins
- Version Control: Subversion, GIT, GIT-SVN
- OperaMini (Simulator)
- FirefoxMobile (Simulator)
- Plugins for browsers:
- Firebug (with loads of plugins - YSlow, etc.)
- Web Developers Tools (firefox)
- Others (developer's preference)
- Developers also use WINE to run some windows software on GNU/Linux
- Windows Box with different browsers (versions, vendors)
- Windows Box with different Mobile Emulators/Simulators
- Selenium Plugin for Firefox
- Apache HTTPD
- Mysql Server
- ReviewBoard (limited usage)
- Hudson (limited usage)
28 Feb 2011
One of our servers, collocated in our office, crashed couple of weeks back. We took this opportunity to virtualize most of the stuff, earlier only some intances were virtualized.
Installing and setting up VMware vSphere Hypervisor is a straight forward process, even the process of creating virtual-machines.
Most of time, I spent was figuring out:
- Storage devices and configuration
- How many virtual machines, do we need?
- Which operating systems, do we need?
- Network configuration
I am not going to cover details of installation and network configuration. I might write more posts in future to cover specifics. Meanwhile, I would love to answer the comments, if I know or share anything I know.
Anyway, this is what we have now:
- IBM x3250 M2
- Intel Xeon (Quadcore) x64
- 4 GB memory
- 3TB+ storage connected to LSI RAID and SATA controller
- Host: VMware vSphere Hypervisor - which runs out of 1.5 TB datastore, directly connected to SATA controller (different from LSI)
- Guest VMs:
- OpenVPN Access Server for VPN
- Ubuntu 10.10 Server as Development sandbox
- Zentyal(eBox) as Internet Gateway (filtering, IDs, proxy)
- Ubuntu 10.04 LTS Server for SVN, Mails, Local Staging WebServer
- Windows 2008 Server for ActiveDirectory, FileServer, Windows-Specific-Software (access-control-management-gui, etc.)
- Others: Project Specific VMs
We are very happy with the improved performance and stability, and the way hardware is properly used. Thanks to VM templates, clones and snapshots, we would be able to provision above infrastructure with minimal downtime.
I would soon replace Windows Server with Zentyal (eBox), which I believe, can handle most of things. However, we would need another Windows XP box to run some software (proprietary), which only runs on windows, to manage various other devices (EPBX, Access-Control-System, etc.).
We now have relatively more scalable infrastructure, more virtual machines can be created to take care of email-server, svn-server, etc.
We are yet to invest in a sophisticated storage device such as DAS or NAS or SAN, hence we are relying on some cheap tricks here:
- 1.5 TB disk as datastore for VMware vSphere, which stores
- Guest Operating System VMs
- Guest Operating System VMs Paging files
- Virtual Disks (non critical data - programs and configs) required by host virtual-machines
- 1.5 TB disk via Raw Device Mapping (RDM) shared among virtual machines
- 500 GB disk via Raw Device Mapping (RDM) shared among virtual machines
- 1.5 TB USB/Firewire - Backup storage
- Rsync is used to backup critical data and configs from all virtual machines on daily, weekly and monthly basis.
Things I didn't like:
- VMware vSphere client requires Microsoft Windows
- VMware vSphere client doesn't allow creating RDMs from GUI, one has to use these instructions
Finally, I could execute some of things I planned more than two years back.