Category Archives: Open Source

Docker Global Mentor Week 2016 – Kanpur

Docker Mentor Week 2016 - Kanpur

I have not been blogging quite often but you must have guessed from my posts that I love Docker. It’s beautiful piece of technology which comes with a lot of great tooling for developers, devops and infrastructure administrators.

I am Docker Kanpur Meetup Group organiser. We are organising Docker Global Mentor Week in Kanpur on November 18, 2016.

If you are from Kanpur or around (Lucknow, Unnao, etc.), please do join the meetup group, and participate in this and future events. It’s lot of learning, and fun to be at meetups.

For your convenience, I am embedding event information below even though it’s available on meetup page. RSVP is must, and we only have space for 20 people. Please confirm it on meetup page.

I look forward to meeting you at the event. Let’s make it fun and successful.


Join us for our Kanpur edition of the Docker Global Mentor Week!

Docker Global Mentor Week is your opportunity to #learndocker. Right now, Docker is working on self paced online labs that will be available through LearnDot. One lab will be for beginners and one lab will be for intermediate users. Docker mentors will be on hand at this event to help you as you work through the self paced materials.

Are you an advanced user? We strongly encourage Docker users of all skill levels to attend! We need a network of mentors who understand the Docker platform to answer any questions that newcomers may have. Click here to sign up as a mentor.

Agenda: November 18, 2016

  • 16:00 – Doors Open
  • 16:15 – Welcome message, presentation of Docker & snacks
  • 16:30 to 18:30 – Select and work through self-paced lab
  • 18:30 – Socializing

As we get closer to the event, we will share more information including materials on how to prepare for the training.

Pre-Event Set-up

At the training, you will need to bring your own computer. Before you go to a mentoring event, there are some steps you should do some preparation to get your work environment ready. This will help get ready for the tutorial, and also save on bandwidth at the event venue. Here are the steps:

Set-up Docker on your machine

For Linux users, we need you to install Docker engine and Docker compose. Make sure you have Docker compose version 1.6 or higher by running docker-compose version from the command prompt.

For Mac users, install Docker for Mac or if you have an older Mac, Docker Toolbox.

For Windows users, if you have Windows 10 pro install Docker for Windows, otherwise install Docker Toolbox.

If you are a Docker for Windows user and want to try the new Windows containers, you need to have fully updated your machine. To prepare to use Windows containers,

Go through the setup steps in the Windows Container lab.

In Powershell run
docker pull microsoft/windowsservercore:latest
This will take awhile, but it is essential you do this before going to the venue. The Windows base image is a 3 gig download.

If you’re new to Docker, pre-pull the docker images for the very basic tutorial
  • docker pull hello-world
  • docker pull alpine
  • docker pull seqvence/static-site
To run the application and participate in the rest of the training, pre-pull these images
  • docker pull microsoft/dotnet:1.0.0-preview1
  • docker pull node:5.11.0-slim
  • docker pull python:2.7-alpine
  • docker pull redis:alpine
  • docker pull postgres:9.4
For the orchestration part, you will want to pre-build the demo app by running the following steps
  • git clone git://github.com/jpetazzo/orchestration-workshop
  • cd orchestration-workshop/dockercoins
  • docker-compose build

Venue:

Allies Interactive
18/179, 3rd Floor
Above Karur Vyasya Bank,
Next to Hotel Geet or Akashdeep,
The Mall (Phoobagh), Kanpur-01
Phone: +91 512 2351352

Docker Awesome

I love these *-awesome projects on Github.

The Docker Awesome is all about Docker and it’s ecosystem. Hundreds of curated links help you understand Docker, and it’s fast growing ecosystem.

If you think something is missing there, go ahead and add to README.md or send a PR to github.com/veggiemonk/awesome-docker/

BTW! There is another awesome project which curates links of all other awesome projects: https://github.com/sindresorhus/awesome 🙂

Docker: Second Birthday

Docker is one of the hottest technologies; I started with it’s first release not too long back, however, in Docker world it feels like Docker has been around for a long time.

What a paradigm shift we have witnessed: a few technologies get that kind of attention, momentum, growth, and community in such a short time (24 months).

Today, we use Docker to develop, test, ship, integrate, deploy, and run apps/services. Docker has solved developer (devops) usability problems; it’s hard to go back and do things without Docker.

Haven’t heard of Docker, or didn’t find time to try it out? Start with following links.

What’s new with Docker? Look at following links:

  • Docker Compose (docker-compose, formerly known as fig)

    Compose is a tool for defining and running complex applications with Docker. With Compose, you define a multi-container application in a single file, then spin your application up in a single command which does everything that needs to be done to get it running.

  • Docker Machine (docker-machine)

    Machine makes it really easy to create Docker hosts on your computer, on cloud providers and inside your own datacenter. It creates servers, installs Docker on them, then configures the Docker client to talk to them.

  • Docker Swarm (swarm)

    Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts into a single, virtual host.Swarm serves the standard Docker API, so any tool which already communicates with a Docker daemon can use Swarm to transparently scale to multiple hosts: Dokku, Compose, Krane, Flynn, Deis, DockerUI, Shipyard, Drone, Jenkins… and, of course, the Docker client itself.

Docker: Automatically Restarting Containers

As a Docker beginner, I always wondered, how to automatically restart container os or system boot?

Thankfully, Docker 1.2 introduced restart policies (which makes sure even linked containers are started in expected order).  The --restart=always flag/value can be passed to ensure automatic restart of containers.

docker run --restart=always redis

Read more about Docker Restart Policies, or other methods.

Github: Forking and Contributing Workflow

I stick to following workflow while contributing to open-source projects on Github:

  1. Fork <user>/<repo> to abdul/<repo>
  2. git clone [email protected]:abdul/<repo>.git
  3. git remote add upstream [email protected]:<user>/<repo>.git
  4. git checkout -b <topic_branch>
  5. Fix, Add or Change things (code, documentation, etc.)
  6. git add <files>
  7. git commit -m <commit-message>
  8. Repeat #5 , #6 & #7
  9. git pull --rebase upstream master
  10. Resolve any conflicts, if required
  11. Squash Commits, e.g. squashing last three commits:
    • git rebase -i HEAD~3
  12. git push origin <topic_branch>
  13. Go to github.com/abdul/<repo>
  14. Create a Pull Request [Github is smart enough to detect a change in forked repo, and offer you to compare and create pull-request]
  15. Discuss changes in PR, and Fix things in branch as required.

NodeSchool

NodeSchool is a fully community driven effort and is 100% open source. This means that anyone can use the NodeSchool materials and/or host an event.

I have set up NodeSchool Kanpur chapter, and I hope to organise (with help of like minded people) a couple of workshops on Node.js (and web-development ecosystem around it).

I am looking for mentors and co-organisers, if you are interested, let’s have a conversion.

Phabricator

Phabricator (pronounced like the word fabricator) is a suite of web applications which make it easier to build software, particularly when working with teams. Phabricator is largely based on Facebook’s internal tools.

It has been more than three years since I have been using Phabricator (self-hosted) at work. It is one of the open-source projects which has inspired me a lot. I have seen it grow over these years; every day it gets better. Thanks to hundreds of open-source developers for their contribution.

Want to see how to develop elegant and sophisticated software, even with a poor programming languages like PHP? Look at Phabricator’s source-code and tools (built using PHP), you would be awestruck.

Want to play with Phabricator without installing? Have a look at PhorePlay (Phabricator hosting service), which offers free Phabricator hosting (with multiple instances) up to five users.  I think, that’s good enough for most of us (small teams).

Phabricator’s core team is working to launch Phacility (Phabricator Hosting). I think, it has potential to become something like Github (useful and essential service) one day.

Installing NumPy and SciPy on Mac OSX 10.8 (Mountain Lion)

I am playing with ExpEyes, which is awesome tool for anyone who wants to learn electronics (and physics).

ExpEyes comes with software (written in python) for GNU/Linux and Windows. I intend to run it on Mac OSX, because it can (provided all dependencies are met).

It is generally easy to install python packages on OSX using easy_install or pip, however, installing NumPy and SciPy turned out to be pain on OSX (10.8.2, latest version the time of posting).

In my case, I am using python binary installed via homebrew, so I can’t take advantage of numpy (which comes installed with mountain-lion).

I have XCode 6 installed, which comes with gcc-4.2.1 (llvm build). I figured out, I can build numpy and scipy from source.

Following is what I did to build numpy and scipy. SciPy requires gfortran, so that has to be installed (I used homebrew to do that):

$ git clone https://github.com/numpy/numpy.git
$ git clone https://github.com/scipy/scipy.git
$ cd numpy
$ python setup.py build && python setup.py install
$ brew install gfortran
$ cd ../scipy
$ python setup.py build && python setup.py install

BTW! In case, you are wondering, why didn’t I use Scipy Superpack script, which indeed makes job easier?

I didn’t, because I want to install gnu/unix packages using homebrew, and I already have git and other things. Super Spicypack downloads (gfortran, etc.), builds and installs packages directly to system, making it harder to remove those later? Homebrew makes job easier.

IndieReign – Discover a world of independent film and have fun doing it

My team and I have been working on bunch of products/services for last 3-4 years.


IndieReign is the one, we have been mostly focused on, and took care entire engineering and technical architecture.

It is a platform for indie filmmakers to promote, distribute and monetize content. It is actually more than that.

IndieReign runs on open-source software stack (for everything including video transcoding) with exception of Adobe Flash Player (used for video playback), and Amazon’s cloud.

Please share your feedback.

Mac OSX (10.8) Mountain Lion: Fixing Apache, PHP, Subversion, User Sites, Virtual Hosts Directories

While Mountain Lion (OSX 10.8) has brought many enhancements and fixes (mostly performance), it has also caused frustration to old users.

You might have noticed following issues in Mountain Lion:

  1. Web Sharing Preferences Panel has been removed, i.e. you need to launch apache manually or write some launch-agent script or hack to bring web-sharing preferences panel back
  2. User Sites (per-user web directories) is disabled, i.e. http://localhost/~username doesn’t work by default, and you need to modify apache config file to enable it
  3. PHP extension for Apache is disabled by default i.e. you need to enable it by modified apache config (httpd.conf)
  4. Subversion is not installed by default, either you can install using homebrew or by install XCode command line tools

You can fix these issues (1-4) by following these two links:

Whenever I upgrade to newer version of OSX, I end up with broken development-setup i.e. custom permissions on directories are messed up, configuration files are overwritten/removed, etc.

I think, I have found a way to keep default directory permissions, so future upgrades don’t break things.

My development setup requires my virtual hosts to use directories within my home-directory (and outside of ~/Sites or /Library/WebServer/Documents directories).

I am using following technique to setup my virtual-hosts:

  • Enable User Sites  (i.e. fix #2 problem above)
  • Create symbolic-links(symlinks) of your project folder(s) inside ~/Sites directory
  • Enable FollowSymLinks and SymLinksIfOwnerMatch options in virtual-host configuration.

In my case, I have a project inside ~/projects/myproject . I create a symlink of this directory inside ~/Sites, like this using Terminal (command-prompt)

$ ln -s ~/projects/myproject ~/Sites/myproject

I have following virtual-host configuration for this project:

<VirtualHost *:80>
 DocumentRoot "/Users/myusername/Sites/myproject/trunk/web"
 ServerName  myproject
 ServerAdmin [email protected]
 Options Indexes MultiViews Includes +FollowSymlinks +SymLinksIfOwnerMatch
 CustomLog "/private/var/log/apache2/myproject_access.log" common
 ErrorLog "/private/var/log/apache2/myproject_error.log"
</VirtualHost>

I use it on my development machine (i.e. my laptop), so I don’t really bother about security at Apache level, and I take care of security at firewall/other levels.

If you are going to use above technique/configuration, please understand everything (by reading documentation) before you use it.

This set up works for me and solves some problems for me – I don’t have to change default directory permissions, and I don’t have to keep my projects within ~/Sites or /Library/WebServer/Documents directory.

RIP, Kenneth Gonsalves “KG” (1953 — 2012)

Kenneth Gonsalves(Image Courtesy: Linux Bangalore (foss.in))

Kenneth Gonsalves, a champion in Indian (NRC)FOSS, Python and Django community passed away on August 3, 2012. Kenneth’s presence would be hugely missed in the community.

In 2008, Tarun Dua introduced me to Kenneth. I interacted with him at various occasions about NRCFOSS and general stuff. I was very inspired to see his contributions, specially some of his open-source projects.

While checking out NRCFOSS website today, I came to know about this sad news.

May his soul rest in peace.

Installing Riak on Mac OSX

I am evaluating various nosql solutions for our existing/new projects. Bluesmoon recommended about Riak which is an awesome nosql datastore/database (and lot more).  I couldn’t wait to check it out.

As usual, I relied on Homebrew – the best package manage for OSX – to install applications/libraries. While installing, I got an error:

$ brew install riak
$ ...
$ Error: Failed executing: make all rel

After searching for a while, I figured out that there are known issues. With the help of Adam‘s comment on github, I was able to install Riak finally.

We make brew ignore Riak’s dependencies while building it.

Following commands should build and install Riak:

$ brew update
$ brew install erlang
$ brew install --HEAD --ignore-dependencies riak -v

Thoughts: Open Standards and Technology for Movie Rental Business

I have been thinking about open-standards for movie-rental business. I searched to find about such standards but I couldn’t locate. Do you know of any?

Basically, a set of specifications/protocols which can be implemented by different players in movie-rental chain. Players include studios, distributor, application-developers, portals, et. al.

Rough thoughts:

  • A studio/distributor implements this specification/protocol and exposes API  for metadata/streaming(or playback)/file-access from their own server
  • Third party developers use the API to to pull metadata and play the content
  • Third party developers can charge the user and split the revenue with content owner
  • Or, studios/distributors can enable payment-gatway APIs as part of their offering
  • Content is always played directly from studio/distributor’s server
  • Studio/distributor have total control over their assets and also charge based on subscription/per-access
This would solve many problems (billing, stats, etc.), however, there would be concerns:
  • someone (end-user, third-party developer) might try to cache/save streams as file and distribute
  • and other concerns?
Once we have specifications and protocols in place – a stack using open-source technologies can be built as template, and made available so that everyone in chain can use/customize it to build their part.
I wanted to talk about it on twitter but 140 chars were too short to express my thoughts. Anyway, it’s good to have a blog post in place.
Please share your views.

Development Setup – Updated

I posted about our proposed development setup, during these years a few things have changed a bit. More or less, this is what we have:

  • GNU/Linux (Ubuntu) and Windows(some) boxes for developers
  • Eclipse (PDT) IDE with loads some plugins
  • Version Control: Subversion, GIT, GIT-SVN
  • Browsers
    • Firefox
    • Chrome
    • Opera
    • Safari
    • OperaMini (Simulator)
    • FirefoxMobile (Simulator)
  • Plugins for browsers:
    • Firebug (with loads of plugins – YSlow, etc.)
    • Web Developers Tools (firefox)
    • Others (developer’s preference)
  • Developers also use WINE to run some windows software on GNU/Linux

Testing:

  • Windows Box with different browsers (versions, vendors)
  • Windows Box with different Mobile Emulators/Simulators
  • Selenium Plugin for Firefox

Development Servers/Applications:

  • Apache HTTPD
  • Mysql Server
  • Trac
  • MediaWiki
  • PHPMyAdmin
  • ReviewBoard (limited usage)
  • Hudson (limited usage)