Rohit Yadav2013-05-24T00:08:47-07:00https://rohityadav.in/Rohit Yadavrohityadav89@gmail.comBuilding CloudStack SystemVMs2013-02-19T00:00:00-08:00https://rohityadav.in/logs/building-systemvms<p>CloudStack uses virtual appliances as part of its orchestration. For example, it
uses virtual routers for SDN, secondary storage vm for snapshots, templates etc.
All these service appliances are created off a template called a systemvm
template in CloudStack’s terminologies. This template appliance is patched to create
secondary storage vm, console proxy vm or router vm. There was an old way of building
systemvms in <code>patches/systemvm/debian/buildsystemvm.sh</code> which is no longer maintained
and we wanted to have a way for hackers to just build systemvms on their own box.</p>
<p><a href="mailto:jmartin@basho.com">James Martin</a> did a great job on automating DevCloud appliance
building using <a href="https://github.com/jedi4ever/veewee/">veewee</a>, a tool with
which one can build appliances on VirtualBox. The tool itself is easy to use, you
first define what kind of box you want to build, configure a preseed file and add
any post installation script you want to run, once done you can export the appliance in
various formats using <code>vhd-util</code>, <code>qemu-img</code> and <code>vboxmanage</code>. I finally fixed a
solution to this <a href="https://issues.apache.org/jira/browse/CLOUDSTACK-1066">problem</a>
today and the code lives in <code>tools/appliance</code> on master branch but this post is
not about that solution but about the issues and challenges of <a href="http://jenkins.cloudstack.org/job/build-systemvm-master">setting up an
automated jenkins job</a>
and on replicating the build job.</p>
<p>I used Ubuntu 12.04 on a large machine which runs a jenkins slave and connects
to <code>jenkins.cloudstack.org</code>. After little housekeeping I installed VirtualBox from
<code>virtualbox.org</code>. VirtualBox comes up with its command line tool, <code>vboxmanage</code>
which can be used to clone, copy and export appliance. I used it to export it to
ova, vhd and raw image formats. Next, installed qemu which gets you <code>qemu-img</code> for
exporting a raw disk image to the qcow2 format.</p>
<p>The VirtualBox vhd format is compatible to HyperV virtual disk format, but for
exporting VHD for Xen, we need to export the appliance to raw disk format and
then use <code>vhd-util</code> to convert it to Xen VHD image.</p>
<p>Unfortunately, the vhd-util <a href="http://download.cloud.com.s3.amazonaws.com/tools/vhd-util">I got did not work for me</a>,
so I just compiled my own from an approach suggested on <a href="http://blogs.citrix.com/2012/10/04/convert-a-raw-image-to-xenserver-vhd/">this blog</a>:</p>
<pre class="prettyprint">
sudo apt-get install bzip2 python-dev gcc g++ build-essential libssl-dev
uuid-dev zlib1g-dev libncurses5-dev libx11-dev python-dev iasl bin86 bcc
gettext libglib2.0-dev libyajl-dev
# On 64 bit system
sudo apt-get install libc6-dev-i386
# Build vhd-util from source
wget -q http://bits.xensource.com/oss-xen/release/4.2.0/xen-4.2.0.tar.gz
tar -xzf xen-4.2.0.tar.gz
cd xen-4.2.0/tools/
wget https://github.com/citrix-openstack/xenserver-utils/raw/master/blktap2.patch -qO - | patch -p0
./configure --disable-monitors --disable-ocamltools --disable-rombios --disable-seabios
cd blktap2/vhd
make -j 2
sudo make install
</pre>
<p>Last thing was to setup rvm for the jenkins user:</p>
<pre class="prettyprint">
$ \curl -L https://get.rvm.io | bash -s stable --ruby
# In case of dependency or openssl error:
$ rvm requirements run
$ rvm reinstall 1.9.3
</pre>
<p>One issue with <code>rvm</code> is that it requires a login shell, which I fixed in <code>build.sh</code>
using <code>#!/bin/bash -xl</code>. But the build job failed for me due to missing env variables.
<code>$HOME</code> needs to be defined and rvm should be in path. The shell commands used to
run the jenkins job:</p>
<pre class="prettyprint">
whoami
export PATH=/home/jenkins/.rvm/bin:$PATH
export rvm_path=/home/jenkins/.rvm
export HOME=/home/jenkins/
cd tools/appliance
rm -fr iso/ dist/
chmod +x build.sh
./build.sh
</pre>
DevCloud for CloudStack Development2012-11-27T00:00:00-08:00https://rohityadav.in/logs/devcloud<p><a href="http://incubator.apache.org/cloudstack">Apache CloudStack</a> development is
not an easy task, for the simplest of deployments one requires a server where
the management server, mysql server and NFS server would run, at least
one host or server which would run a hypervisor (to run virtual machines) or
would be used for baremetal deployment and some network infrastructure.</p>
<p>And talk about development, sometimes reproducing a bug can take hours or days
(been there done that :) and moreover a developer may not have access to such
an infrastructure all the time.</p>
<h2>The Solution</h2>
<p>To solve the problem of infrastructure availability for development and testing,
earlier this year <a href="http://www.linkedin.com/pub/disheng-su/5/ab9/90b">Edison</a>,
one of the core committers and PPMC members of Apache CloudStack (incubating),
created <a href="http://wiki.cloudstack.org/display/COMM/DevCloud">DevCloud</a>.</p>
<p><code>DevCloud</code> is a virtual appliance shipped as an OVA image which runs on <a href="http://virtualbox.org">VirtualBox</a>
(an opensource type-2 or desktop hypervisor) and can be used for CloudStack’s
development and testing. The original DevCloud required 2G of RAM, and ran
Ubuntu Precise as dom0 over xen.org’s Xen server which runs as a VM on VirtualBox.</p>
<p>A developer would build and deploy CloudStack artifacts (jars, wars) and files
to DevCloud, deploy database and start the management server inside DevCloud.
The developer may then use CloudStack running inside DevCloud to add DevCloud as
a host and whatnot. DevCloud is now used by a lot of people, especially during
the first release of Apache CloudStack, the 4.0.0-incubating, DevCloud was used
for the release testing.</p>
<h2>My Experiment</h2>
<p>When I tried DevCloud for the first time, I thought it was neat, an awesome all
in a box solution for offline development. The limitations were; only one host
could be used that too in basic zone and it would run mgmt server etc. all inside
DevCloud. I wanted to run mgmt server, MySQL server on my laptop and debug with
IntelliJ, so I made <a href="https://cwiki.apache.org/confluence/display/CLOUDSTACK/DIY+DevCloud+Setup">my own
DevCloud</a>
setup which would run two XenServers on separate VirtualBox VMs, NFS running on
a separate VM and all the VMs on a host-only network.</p>
<p>The <code>host-only</code> network in VirtualBox is a special network which is shared by
all the VMs and the host operating system. My setup allowed me to have two hosts
so I could do things like VM migration in a cluster etc. But it would crash a lot
and network won’t work. I learnt how bridging in Xen worked and using tcpdump
found that the packets were dropped but ARP request was allowed, the fix was to
just enable host-only adapter’s promiscuous mode to allow all. I also tried to
run KVM on VirtualBox, which did not work as KVM does not support PV and requires
HVM so it cannot run on processors without Intel-VT or Amd-V. None of which is
emulated by VirtualBox.</p>
<h2>Motivation</h2>
<p>CloudStack’s build system was changed from Ant to Maven, and this required some
<a href="https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+devcloud+environment+setup">changes in DevCloud</a>
which made it possible to use the original appliance with the new build system.
The changes were not straight forward so I decided to work on the next iteration
of DevCloud with the following goals:</p>
<ul>
<li>Two network interfaces, <em>host-only</em> adapter so that the VM is reachable from
host os and a <em>NAT</em> so VMs can access Internet.</li>
<li>Can be used both as an all in one box solution like the original DevCloud but
the mgmt server and other services can run elsewhere (on host os).</li>
<li>Reduce resource requirements, so one could run it in 1G limit.</li>
<li>Allow multiple DevCloud VMs hosts.</li>
<li>x86 dom0 and xen-i386 so it runs on all host os.</li>
<li>Reduce exported appliance (ova) file size.</li>
<li>It should be seamless, it should work out of the box.</li>
</ul>
<h2>DevCloud 2.0</h2>
<p>I started by creating an appliance using Ubuntu 12.04.1 server which failed for me.
The network interfaces would stop working after reboot and few users reported
blank screen. I never caught the actual issue, so I tried to create the
appliance using different distributions including Fedora, Debian and Arch.
Fedora did not work and stripping down to a bare minimum required a lot of work.
Arch VM was very small in size but I dropped my idea to work on it as it can be
unstable, people may not be familiar with pacman and may fail to appreciate the
simplicity of the distribution.</p>
<p>Finally, I hit the jackpot with Debian! Debian Wheezy just worked, took me some
time to create it from scratch (more than ten times) and figure out the correct
configurations. The new appliance is available for download, <a href="http://people.apache.org/~bhaisaab/cloudstack/devcloud/devcloud2.ova">get DevCloud 2.0</a>
(867MB, md5checksum: 144b41193229ead4c9b3213c1c40f005).</p>
<p>Install VirtualBox and import the new DevCloud2 appliance and start it. In
default settings, it is reachable on ip <code>192.168.56.10</code> with username <code>root</code> and
password <code>password</code>. Next start hacking either inside the DevCloud appliance or
on your laptop (host os):</p>
<pre class="prettyprint">
# ssh inside DevCloud if building inside it:
$ ssh -v root@192.168.56.10
$ cd to /opt/cloudstack # or any other directory, it does not matter
# Get the source code:
$ git clone https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git
$ cd incubator-cloudstack
# Build management server:
$ mvn clean install -P developer,systemvm
# Deploy database:
$ mvn -pl developer,tools/devcloud -Ddeploydb -P developer
# Export the following only if you want debugging on port 8787
$ export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=800m -Xdebug -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n"
# Run the management server:
$ mvn -pl client jetty:run
# In Global Settings check `host` to 192.168.56.1 (or .10 if inside DevCloud)
# and `system.vm.use.local.storage` to true, restart mgmt server.
# Set the maximum number of console proxy vms to 0 if you don't need one from
# CloudStack's global settings, this will save you some RAM.
# Now add a basic zone with local storage. May be start more DevCloud hosts by
# importing more appliances and changing default IPs and reboot!
</pre>
<p>Make sure your mgmt server is running and you may deploy a basic zone using
preconfigured settings in <code>tools/devcloud/devcloud.cfg</code>:</p>
<pre class="prettyprint">
$ mvn -P developer -pl tools/devcloud -Ddeploysvr
# Or in case mvn fails try the following, (can fail if you run mgmt server in debug mode on port 8787)
$ cd tools/devcloud
$ python ../marvin/marvin/deployDataCenter.py -i devcloud.cfg
</pre>
<h2>DIY DevCloud</h2>
<p>Install VirtualBox and get the <a href="http://www.debian.org/devel/debian-installer/">Debian Wheezy
7.0</a>. I used the netinst i386
iso. Create a new VM in VirtualBox with Debian/Linux as the distro, 2G RAM, 20G
or more disk and two nics: host-only with promiscuous mode “allow-all” and a NAT
adapter. Next, install a base Debian system with linux-kernel-pae (generic),
and openssh-server. You may download my <a href="http://people.apache.org/~bhaisaab/vms/debian-wheezy-basex86.ova">base system from
here</a>.</p>
<p>Install required tools and Xen-i386:</p>
<pre class="prettyprint">
$ apt-get install git vim tcpdump ebtables --no-install-recommends
$ apt-get install openjdk-6-jdk genisoimage python-pip mysql-server nfs-kernel-server --no-install-recommends
$ apt-get install linux-headers-3.2.0-4-686-pae xen-hypervisor-4.1-i386 xcp-xapi xcp-xe xcp-guest-templates xcp-vncterm xen-tools blktap-utils blktap-dkms qemu-keymaps qemu-utils --no-install-recommends
</pre>
<p>You’ll have to build and install mkisofs.
Remove MySQL password:</p>
<pre class="prettyprint">
$ mysql -u root -p
> SET PASSWORD FOR root@localhost=PASSWORD('');
> exit;
</pre>
<p>Install MySQL Python connector 1.0.7 or latest:</p>
<pre class="prettyprint">
$ pip install mysql-connector-python
# Or, if you have easy_install you can do: easy_install mysql-connector-python
</pre>
<p>Setup Xen and XCP/XAPI:</p>
<pre class="prettyprint">
$ echo "bridge" > /etc/xcp/network.conf
$ update-rc.d xendomains disable
$ echo TOOLSTACK=xapi > /etc/default/xen
$ sed -i 's/GRUB_DEFAULT=.\+/GRUB_DEFAULT="Xen 4.1-i386"/' /etc/default/grub
$ sed -i 's/GRUB_CMDLINE_LINUX=.\+/GRUB_CMDLINE_LINUX="apparmor=0"\nGRUB_CMDLINE_XEN="dom0_mem=400M,max:500M dom0_max_vcpus=1"/' /etc/default/grub
$ update-grub
$ sed -i 's/VNCTERM_LISTEN=.\+/VNCTERM_LISTEN="-v 0.0.0.0:1"/' /usr/lib/xcp/lib/vncterm-wrapper
$ cat > /usr/lib/xcp/plugins/echo << EOF
#!/usr/bin/env python
# Simple XenAPI plugin
import XenAPIPlugin, time
def main(session, args):
if args.has_key("sleep"):
secs = int(args["sleep"])
time.sleep(secs)
return "args were: %s" % (repr(args))
if __name__ == "__main__":
XenAPIPlugin.dispatch({"main": main})
EOF
$ chmod -R 777 /usr/lib/xcp
$ mkdir -p /root/.ssh
$ ssh-keygen -A -q
</pre>
<p>Network settings, /etc/network/interfaces:</p>
<pre class="prettyprint">
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet manual
allow-hotplug eth1
iface eth1 inet manual
auto xenbr0
iface xenbr0 inet static
bridge_ports eth0
address 192.168.56.10
netmask 255.255.255.0
network 192.168.56.0
broadcast 192.168.56.255
gateway 192.168.56.1
dns_nameservers 8.8.8.8 8.8.4.4
post-up route del default gw 192.168.56.1; route add default gw 192.168.56.1 metric 100;
auto xenbr1
iface xenbr1 inet dhcp
bridge_ports eth1
dns_nameservers 8.8.8.8 8.8.4.4
post-up route add default gw 10.0.3.2
</pre>
<p>Preseed the SystemVM templates in <code>/opt/storage/secondary</code>, follow directions
from
<a href="http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0.0-incubating/html/Installation_Guide/management-server-install-flow.html#prepare-system-vm-template">here</a>.
Configure NFS server and local storage.</p>
<pre class="prettyprint">
$ mkdir -p /opt/storage/secondary
$ mkdir -p /opt/storage/primary
$ hostuuid=`xe host-list |grep uuid|awk '{print $5}'`
$ xe sr-create host-uuid=$hostuuid name-label=local-storage shared=false type=file device-config:location=/opt/storage/primary
$ echo "/opt/storage/secondary *(rw,no_subtree_check,no_root_squash,fsid=0)" > /etc/exports
$ #preseed systemvm template, may be copy files from devcloud's /opt/storage/secondary
$ /etc/init.d/nfs-kernel-server restart
</pre>
<div class="row">
<div class="offset2 span6">
<div class="alert alert-info">
<center>
<strong>Please</strong> email your queries on the ACS ML `cloudstack-dev@incubator.apache.org`
</center>
</div>
</div>
</div>
CloudStack Cloudmonkey2012-11-21T00:00:00-08:00https://rohityadav.in/logs/cloudmonkey<p>About 2-3 weeks ago I started writing a CLI (command line interface) for <a href="http://incubator.apache.org/cloudstack">Apache CloudStack</a>. I researched some options and finally chose Python and cmd. Python comes preinstalled on almost all Linux distros and Mac (Windows I don’t care :P it’s not developer friendly), cmd is a standard package in Python with which one can write a tool which can work as a command line tool and as an interactive shell interpretor. I named it <code>cloudmonkey</code> after the project’s mascot. In this blog and elsewhere I use the name as Cloudmonkey or cloudmonkey, but not CloudMonkey :P</p>
<center><img src="/images/apache/cloudmonkey-mac.png"><br><p>Cloudmonkey on OSX</p></center>
<p>Apache CloudStack has around 300 restful <a href="http://incubator.apache.org/cloudstack/docs/api/index.html">APIs</a> give or take, and writing handlers (autocompletion, help, request handlers etc.) seemed a mammoth task at first. Marvin (the ignored robot) came to rescue. Marvin is a Python package within CloudStack and was written by <a href="http://www.linkedin.com/pub/disheng-su/5/ab9/90b">Edison</a> and now maintained by <a href="http://in.linkedin.com/in/v0g0n">Prasanna</a> which provides bunch of classes with which one can implement a client for CloudStack and provides cloudstackAPI. It’s interesting how cloudstackAPI is generated. A developer writes an API and fills the boilterplate with API specific details such as required params etc. and java doc string. This information is picked up by an api writer class which generates an xml containing information about each API, its docstring and parameters. This is used by <em>apidocs</em> artifact to generate API help docs and used by Marvin’s code generator to create a module for cloudstackAPI which contains command and response classes. When I understood the whole process I thought if I can reuse this somehow I won’t have to deal with the 300 APIs directly. <br></p>
<center><img src="/images/apache/cloudmonkey-ubuntu.png"><br><p>Cloudmonkey on Ubuntu</p></center>
<p>I’ve always been a fan of functional programming, iterative or object oriented programming was not going to help. So, I grouped the apis based on their first lowercase chars, for example for the api listUsers, the verb is list. Based on such pattern, I wrote the code so that it would group APIs based on such verbs and create handlers on the fly and add them to the shell class. The handlers are actually closures so, this way every handler is actual a dynamic function in memory enclosed by the closure generator for a verb. In the initial version, when a command was executed first time based on its verb, command class from appropriate module from cloudstackAPI would be loaded and a cache dictionary would be populated if a cache miss was hit. In later version, I wrote a cache generator which would precache all the APIs at build time to cheat on the runtime lookup overhead from O(n) to O(1). This cache would contain for each verb the api name, required params, all params and help strings. This dictionary is used for autocompletion for the verbs, the commands and their parameters, and for help strings.</p>
<pre class="prettyprint linenums">
grammar = ['list', 'create', 'update', 'delete', ...]
for rule in grammar:
def add_grammar(rule):
def grammar_closure(self, args):
if not rule in self.cache_verbs:
self.cache_verb_miss(rule)
try:
args_partition = args.partition(" ")
res = self.cache_verbs[rule][args_partition[0]]
except KeyError, e:
self.print_shell("Error: invalid %s api arg" % rule, e)
return
if ' --help' in args or ' -h' in args:
self.print_shell(res[2])
return
self.default(res[0] + " " + args_partition[2])
return grammar_closure
</pre>
<p>Right now <code>cloudmonkey</code> is available as a community distribution on the <a href="http://pypi.python.org/pypi/cloudmonkey/">cheese shop</a>, so <code>pip install cloudmonkey</code> already! It has a <a href="https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+cloudmonkey+CLI">wiki</a> on building, installation and usage instructions, or watch a <a href="http://www.youtube.com/watch?v=BjkGp3egv9g">screencast</a> (<a href="http://people.apache.org/~bhaisaab/cloudstack/cloudmonkey/cloudmonkey-screencast-user-transcript.txt">transcript</a>, <a href="http://people.apache.org/~bhaisaab/cloudstack/cloudmonkey/cloudmonkey-screencast-user.mov">alternate link</a>) I made for users. As the userbase grows, it will only get better. Feel free to reachout to me and the Apache CloudStack team on IRC or on the mailing lists.</p>
<center>
<iframe width="800" height="450" src="http://www.youtube.com/embed/BjkGp3egv9g" frameborder="0" allowfullscreen></iframe>
</center>
Apache CloudStack Hyderabad Meetup2012-11-02T00:00:00-07:00https://rohityadav.in/logs/first-cloudstack-meetup-hyd<p align="center"><img align="center" src="/images/hyd/meetup/cover.jpg"></p>
<br>
<p>We had our first official <a href="http://incubator.apache.org/cloudstack">Apache CloudStack</a> <a href="http://www.meetup.com/CloudStack-Hyderabad-Group/events/86658162/">meetup</a> in Hyderabad yesterday, 1 Nov 2012, at Lemon Tree, Hyderabad. Earlier we gave a small bird eye view <a href="http://people.apache.org/~bhaisaab/meetups/sep-2012">presentation</a> on Apache CloudStack during a local Hadoop User Group (HUG) <a href="http://www.meetup.com/hyderabad-hadoop/events/81746952/">meetup</a>. This time, the meetup was totally focussed on Apache CloudStack.</p>
<br><p align="center"><img align="center" src="/images/hyd/meetup/start.jpg"></p><br>
<p>The meetup started at 5PM and the presentations ended at 7:20PM, followed by about an hour of networking and discussions, and t-shirts for everyone. It was attended by 134 people and organised by 7 people that includes (Nitin who was out of station at the time), Kishan, Prasanna, Bidisha, Sadhu, Praveen, Hari P and myself.</p>
<p align="center"><iframe width="800" height="450" src="http://www.youtube.com/embed/oJ4b8HFmFTc" frameborder="0" allowfullscreen></iframe></p>
<p>We started by welcoming the attendees and showed them an introductory video on Apache CloudStack. Next we did a poll and found that all of the attendees have heard of cloud computing, most of them use or are going to use cloud or are excited to learn about it. Most of the crowd rated themselves as users, some managers and some engineers. Most of them said they use open source technologies on daily basis and there were few who contribute to opensource.</p>
<p align="center">
<iframe src="http://www.slideshare.net/slideshow/embed_code/14998432" width="512" height="421" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC;border-width:1px 1px 0;margin-bottom:5px" allowfullscreen> </iframe>
</p>
<p><a href="http://www.linkedin.com/pub/kevin-kluge/0/156/715">Kevin Kluge</a> (Apache CloudStack committer and VP Cloud Platforms Group, Citrix) gave his keynote on building your own IaaS cloud with Apache CloudStack and gave a small demo of CloudStack.</p>
<br><p align="center"><img align="center" src="/images/hyd/meetup/kevin.jpg"></p><br>
<p align="center">
<iframe src="http://www.slideshare.net/slideshow/embed_code/14998475" width="512" height="421" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC;border-width:1px 1px 0;margin-bottom:5px" allowfullscreen> </iframe>
</p>
<p>Next, <a href="http://people.apache.org/~tsp/">Prasanna Santhanam</a> (Apache CloudStack committer, amateur violinist dude) gave a brief talk on Apache Software Foundation, how the opensource community works and how to participate.</p>
<br><p align="center"><img align="center" src="/images/hyd/meetup/tsp.jpg"></p><br>
<p align="center">
<iframe src="http://www.slideshare.net/slideshow/embed_code/14998451" width="512" height="421" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:1px solid #CCC;border-width:1px 1px 0;margin-bottom:5px" allowfullscreen> </iframe>
</p>
<p>The last presentation was by <a href="https://twitter.com/cheezo">Chirag Jog</a> (CTO, Clogeny) on migrating application to the IaaS clouds.</p>
<br><p align="center"><img align="center" src="/images/hyd/meetup/chirag.jpg"></p><br>
<p>All the presentations and photos are downloadable from <a href="http://people.apache.org/~bhaisaab/meetups/nov-2012">here</a>, some select photos are <a href="https://plus.google.com/photos/105500838080334630026/albums/5806205455776858881">here</a>. The meetup presentations were followed by questions;</p>
<br><p align="center"><img align="center" src="/images/hyd/meetup/questions.jpg"></p><br>
<p>discussions;</p>
<br><p align="center"><img align="center" src="/images/hyd/meetup/discussions.jpg"></p><br>
<p>and people picking up the CloudStack t-shirts on their way out. Big thanks to all the speakers, attendees and my co-organizers. See you in the next meetups, in <a href="http://foss.in">FOSS.in 2012</a> or during <a href="http://collab12.cloudstack.org/">CloudStack Collaboration 2012</a>.</p>
<br><p align="center"><img align="center" src="/images/hyd/meetup/next.jpg"></p>
Painless Partition Resizing2012-10-19T00:00:00-07:00https://rohityadav.in/logs/extfs-resizing<p>Alrighty, got my prgmr <a href="http://baagi.org">VPS</a> upgraded to 512MiB/12GiB. Google failed me for the obvious keywords and the kind of results on it’s first page of search results, hence this post. Meh, one could have read man pages to do this but blog post works for some chump who would rather Google than read man pages, in future on this topic. So, here you go;</p>
<p>You know your block device, partitions, <code>fdisk -l</code> already. Just check your ext2/ext3 partitions and run filesystem repair on the disk:</p>
<pre class="prettyprint linenums">
$ e2fsck -f /dev/xvda1
$ fsck -n /dev/xvda1
</pre>
<p>Let’s take out the journaling, just remove it to make it <em>ext2</em>, delete the partition (worry not my chump :), create new one and expand the filesystem:</p>
<pre class="prettyprint linenums">
$ tune2fs -O ^has_journal /dev/xvda1
$ fdisk /dev/xvda
The number of cylinders for this disk is set to 1566.
Command (m for help): p
Disk /dev/xvda: 12.8 GB, 12884901888 bytes
255 heads, 63 sectors/track, 1566 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/xvda1 1 1566 12578863+ 83 Linux
Command (m for help): d
Selected partition 1
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1566, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1566, default 1566):
Using default value 1566
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
</pre>
<p>Let’s finally check the partition, have it repaired, check the filesystem and turn on the journaling:</p>
<pre class="prettyprint linenums">
$ e2fsck -f /dev/xvda1
e2fsck 1.39 (29-May-2006)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
PRGMRDISK1: 93269/786432 files (1.8% non-contiguous), 676860/1572864 blocks
$ resize2fs /dev/xvda1
resize2fs 1.39 (29-May-2006)
Resizing the filesystem on /dev/xvda1 to 3144715 (4k) blocks.
The filesystem on /dev/xvda1 is now 3144715 blocks long.
$ fsck -n /dev/xvda1
fsck 1.39 (29-May-2006)
e2fsck 1.39 (29-May-2006)
PRGMRDISK1: clean, 93269/1572864 files, 702302/3144715 blocks
$ tune2fs -j /dev/xvda1
</pre>
<p>And, done!</p>
Coorg Trip2012-10-04T00:00:00-07:00https://rohityadav.in/logs/coorg-trip<p>My <a href="http://baagi.org">friends</a> and I went to <a href="http://en.wikipedia.org/wiki/Kodagu_district">Coorg</a> last weekend.
At our Bangalore <a href="/images/coorg/biereclub.jpg">rendezvous</a>, we had dinner and exchanged reminiscences since college.</p>
<br><p align="center"><img align="center" src="/images/coorg/blr.jpg"></p><br>
<p>We started our journey from Kalena Aghrahara around midnight.</p>
<br><p align="center"><img align="center" src="/images/coorg/map.jpg"></p><br>
<p>First we went to the Dubare Elephant Camp, had our share of fun in the waters of Lake Dubare.</p>
<br><p align="center"><img align="center" src="/images/coorg/dubare.jpg"></p><br>
<p>This was the first time for me to watch elephants so closely. The calf was friendly and cute:</p>
<br><p align="center"><img align="center" src="/images/coorg/elephant.jpg"></p><br>
<p>I was bit disappointed about the state of the Elephant Camp and the cruelty of Elephant Riding, but <em>c'est la vie</em> tourism brings food and shelter for them and their <em>Mahouts</em>. My friends went for a short ride:</p>
<br><p align="center"><img align="center" src="/images/coorg/elephantcamp.jpg"></p><br>
<p>Next stop Abbey Falls. Nothing much to write about, it was just a small and nice waterfall. The <em>jhalmuri</em> was nice, made <em>Divx</em> do his gangnam style, I joined him too :D</p>
<br><p align="center"><img align="center" src="/images/coorg/gangnam.jpg"></p><br>
<p>Next stop, Raja Seat Garden and its spectaular views:</p>
<br><p align="center"><img align="center" src="/images/coorg/rajaseat.jpg"></p><br>
<p>Finally, Mysore Palace.</p>
<br><p align="center"><img align="center" src="/images/coorg/mysorecat.jpg"></p><br>
<p>And time to return home.</p>
<br><p align="center"><img align="center" src="/images/coorg/mysorepalace.jpg"></p><br>
<p><small>PS. Remind me to shave my beard next time we go for a trip :P</small></p>
From Bengaluru to Hyderabad2012-08-26T00:00:00-07:00https://rohityadav.in/logs/blr-to-hyd<p>In July, I relocated to Bangalore to join my first real job as a software engineer. The first week was nice, esp. the bootcamp. We started off by understanding the problem. We chose fast conference booking. The problem was to simplify conference room booking and develop a web/mobile app around it.</p>
<br><p align="center"><a href="/images/hyd/bootcamp.jpg"><img align="center" src="/images/hyd/bootcamp.jpg"></a></p><br>
<p>I started off by hacking a Tornado app with a NoSQL database and a backbonejs frontend but since we had limited time I stashed my work and we hacked a RoR webapp instead and I hosted that on my prgmr VPS, on bc.baagi.org.</p>
<br><p align="center"><a href="/images/hyd/bc-feature.png"><img align="center" src="/images/hyd/bc-feature.png"></a></p><br>
<p>We had some innovative and disruptive ideas, like a booking can have non-adjacent time slots, responsive ui for tablets and mobiles, results based on user’s booking history, intuitive sorting and searching features, sending email for confirmation.</p>
<br><p align="center"><a href="/images/hyd/app.png"><img align="center" src="/images/hyd/app.png"></a></p><br>
<p>Meanwhile, the HR told us fresh-hires about an interesting position in Hyderabad and that those who would opt for it will lose some (cab, food) benefits but they would get to work on the opensource <a href="http://cloudstack.org/">Apache CloudStack</a> with the previously acquired Cloud.com team. Startup kind-of environment, amazing opensource product, small team, I was game!</p>
<p align="center"><a href="/images/hyd/cloustack.png"><img align="center" src="/images/hyd/cloudstack.png"></a></p>
<br>
<p>I left for Hyderabad the very next day. I could n’t care less about bootcamp, I researched more about CloudStack and found that in a way it was similar to <a href="http://code.google.com/p/vmcontroller/">VMController</a>, my B.Tech project. I missed the demo/keynote and we failed to grab a prize at the bootcamp but nevertheless our app was very well appreciated. Heck the VP emailed all the mentors and big guys telling ‘em about our app and that it was hosted on the Internet. On the last day of the bootcamp I got permission from my mentor to opensource the <a href="https://github.com/bhaisaab/bootcamp-ctx">source code</a> for everyone to checkout.</p>
<br><p align="center"><a href="/images/hyd/desk.jpg"><img align="center" src="/images/hyd/desk.jpg"></a></p><br>
<p>And here I am, in Hyderabad for more than a month now! I’m enjoying my job, learning new things every day, hacking through the source code and learning from the masters, trying to fix some interesting <a href="http://bugs.cloudstack.org">bugs</a> and am super excited about what’s coming in next few months.</p>
<br><p align="center"><a href="/images/hyd/servers.jpg"><img align="center" src="/images/hyd/servers.jpg"></a></p><br>
<p>The best part is I’m hacking an opensource project (earning those ohloh points :) and getting paid to do that! I get to work on some sweet servers that would run a variety of hypervisors and even baremetal (excited to work on it soon!) and we do releases every month.</p>
<br><p align="center"><a href="/images/hyd/lunch.jpg"><img align="center" src="/images/hyd/lunch.jpg"></a></p><br>
<p>We had a nice welcome/team lunch recently, thank you team!</p>
<br><p align="center"><a href="/images/hyd/bike.jpg"><img align="center" src="/images/hyd/bike.jpg"></a></p><br>
<p>And I bike everyday to work and places; got myself a Schwinn Searcher (2012).</p>
An Internet Story2012-07-30T00:00:00-07:00https://rohityadav.in/logs/an-internet-story<p>During my time at IIT-BHU, Internet was primarily distributed by computer center (CC) there, which was terrible slow and with a lot of restrictions and banned ports (ssh, ftp etc). So, most of us opensource guys would buy a wireless broadband connection. The purpose of CC to provide Internet services to the students, was defeated by their own people.</p>
<br><center><img align="center" src="/images/wmg/nkn.jpg"></center><br>
<p>Around Nov-Dec 2011, my hacker friend <em>pk</em> and I came in contact with WMG, the Website Management Group of IIT-BHU which manages institutional website and some infrastructure. At that time WMG had an unused NKN node and bunch of servers. NKN, the National Knowledge Network, is an initiative by Indian govt. to connect all the major educational instituions and govt bodies. Unlike the copper wire based Internet infrastructure that most of us use in daily life, the NKN node used fibre optics and had a bandwidth of 100Mbps.</p>
<br><center><img align="center" src="/images/wmg/room.jpg"></center><br>
<p>Jackpot! It was awesome, the first thing we did was setup somes Linux boxes and install squid proxy server on it and share the credentials between the student crowd. It was an overnight success. Almost everyone on campus was using it.</p>
<p>During the next few days we contacted NKN to upgrade our bandwidth to 1Gbps, the only requirement was to show adequate consumption of bandwidth. So, we setup an Ubuntu repository mirror and the proxy server was already running; this way we got it upgraded to 1Gbps in about a week or two.</p>
<p>During the next few months, we found that a proxy server was not a good solution for most of us and so we setup an OpenVPN server. Our stack consisted of some python scripts, cron jobs, openvpn-server on RHEL. Below is a screenshot of the <a href="https://github.com/bhaisaab/hacktools/tree/master/vpnmon">VPN Monitor</a> I wrote.</p>
<br><center><img align="center" src="/images/wmg/vpnstats.png"></center><br>
<p>We were very successful, by that time most students and even some faculty members were using the VPN we’d setup. Next problem we solved was scalability, we ran about 16 OpenVPN servers to tackle that and we requested the CC to open 1Gbps ports on their provided gateway/switches for our servers, sadly it was denied. Whatever excuse they may have given I just remember that they did not want anyone to have good Internet bandwidth. So, even if we’d 1Gbps link we were only able to serve 100Mbps of it to hostels and departments.</p>
<p>When something new and innovative disrupts and challenges the workflow of life, there are haters and supporters. I faced the same kind of situation, we all did, all four of us who were involved with our little VPN project. Almost everyone on the IIT-BHU campus was using it, at one time I’d seen at least 1000 people using it. I’m happy with what we did and what we achieved and learnt during the whole chaos. And I’ve something to brag about ;)</p>
<p>Long story short, a friend who was involved with us and had issues with a big guy nuked the server which later on I had to re-setup and I ended up losing a hacker friend. Anyway when I graduated and left the place, some legal issue came up that had to do with logging the VPN servers and the big guy shut down the VPN servers.</p>
Goodbye College Life2012-06-27T00:00:00-07:00https://rohityadav.in/logs/college-life<br><p align="center"><img align="center" src="/images/iitbhu/bunk.gif"></p><br>
<p>Goodbye college life. I’ll will miss you and will cherish those memories in my heart; friends, bunking classes, my 12'x6' hostel room.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/hostel.jpg"></p><br>
<p>Evening snacks in hyderabad gate area.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/hg.jpg"></p><br>
<p>And Nitin’s fav longlata shop.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/longlata.jpg"></p><br>
<p>Limbdi corner’s fine samosas and samosas in general.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/samosas.jpg"></p><br>
<p>Ghats.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/ghat.jpg"></p><br>
<p>Pehlwan’s lassi, Crytal bowl’s sizzlers, Ming’s brownies, but nothing would beat Idli breakfast near Ghats.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/idli.jpg"></p><br>
<p>Paans.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/pan.jpg"></p><br>
<p>Long sleeping periods and above all full night <em>bc</em>.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/bc.gif"></p><br>
<p>That time I rode Nidhi’s bicycle.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/bike-nidhi.jpg"></p><br>
<p>That roadtrip…</p>
<br><p align="center"><img align="center" src="/images/iitbhu/roadtrip.jpg"></p><br>
<p>to Sidhnathdari waterfall.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/sidnathdari.jpg"></p><br>
<p>That time I smoked <em>Tabrez</em> in one panipuri match.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/panipuri.gif"></p><br>
<p>Geekery, the first workshop where we were introduced to GNU/Linux in 2007.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/linux.png"></p><br>
<p><a href="https://github.com/bhaisaab/graffiti">Graffiti</a>, the first cool project <a href="http://showstopper.in">ShowStopper</a> and I hacked.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/graffiti.jpg"></p><br>
<p>And that rare time, when the institute website was pwned by some crackers and I got the screenshot minutes before it was restored.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/cracked.png"></p><br>
<p>Seeing you all off to your trains.</p>
<br><p align="center"><img align="center" src="/images/iitbhu/train.jpg"></p>
BaagiChhaap2012-04-23T00:00:00-07:00https://rohityadav.in/logs/baagichhaap<p>The freaking IDD'07 watched a <a href="http://en.wikipedia.org/wiki/Paan_Singh_Tomar">movie</a> and <em>Baagi</em> became a popular <a href="http://en.wikipedia.org/wiki/Meme">meme</a> among us.</p>
<br><center><img align="center" src="/images/baagi/jain.gif"></center><br>
<p><a href="http://rahuljain.org">Jain</a> is from Dholpur which is near the famous valleys of the <a href="http://en.wikipedia.org/wiki/Chambal_River">Chambal</a> and having seen a bunch of photos of him posing with some country made rifle he was bound to become the baagi of our gang. And after a Saturday night dinner, the gang decided to do something about it, like open a <em>dhabha</em> or something to do with that meme. Believe me the hobby of domain searching and buying is evil and I warned ‘em but they all wanted it badly (intended exaggeration), all 0x7 of them (except tintin kookdookoo) so I booked 'em a <a href="http://baagi.org">domain</a>.</p>
<p>I had to come up with a prank and as Baagis are known to wear awesome mustaches, I thought let’s take any picture and draw a <a href="http://en.wikipedia.org/wiki/Mustache">moochh</a> on it, programmatically of course!</p>
<br><center><img align="center" src="/images/baagi/baagichhaap.jpg"></center><br>
<p>And after a fun Sunday evening with python and opencv I hacked up the prank, <a href="http://chhaap.baagi.org">baagichhaap</a> that takes in a photo, tries to detect a face (red squares), a nose (blue squares) and a mouth (green squares) and based on the obtained information it draws a <em>moochh</em> on it.</p>
<br><center><a href="/images/baagi/baagichhaap-branch.jpg"><img align="center" src="/images/baagi/baagichhaap-branch.jpg"></a></center><br>
<p>It even works with big images with large number of people in it, with some errors though, blame it on opencv. The above baagichhaap’d photograph was taken by ShowStopper’s DSLR last year, some of the folks in it really look like they own their <em>moochh-es</em>.</p>
<p>We have an IRC channel #baagi on freenode if you care to hangout and we <a href="http://baagi.org/irc">log</a>! The prank lives in our <a href="https://github.com/baagi/baagichhaap">baagi</a> repository.</p>
Pimp My Desktop2012-04-17T00:00:00-07:00https://rohityadav.in/logs/pimp-my-desktop<p>I love computers! My primary computer is a <em>heavyduty</em> desktop which I’d custom built more than a year ago. I love to hack microcontrollers and I’m still waiting for my RaspberryPi. I’m not a big fan of laptops though. But, as long as any of ‘em can run GNU/Linux, I’m happy.</p>
<p>This morning I was going through some of my rarely used electronic parts (I’ve tons of electronic parts, and perhaps 100s of LEDs, buttons and all sorts of jumpers) which I’ve not used since last one year. Of them I found a UART controlled LCD display, a spare Arduino and bunch of sensors and I decided to make anything that adds some luxury to my desktop experience. A really small hack and this is the result:</p>
<br><p align="center"><a href="/images/desktop-pimping.jpg"><img align="center" src="/images/desktop-pimping.jpg"></a></p><br>
<p>A visual luxury to my desktop’s front panel which shows realtime CPU and Memory usage with average temperature of all the cores and bonus an audio visualizer. How was it hacked? It’s an Arduino board that is bolted inside my desktop and communicates with a python server to get the stats serially over an internal USB connection via a FTDI chip. I don’t know if hooking an alcohol or a LPG or a carbon monoxide sensor can be a good idea to implement an alternative method to (say) login. The server has its own fake REPL, the code can be found in my <a href="https://github.com/bhaisaab/hacktools/tree/master/ardulcd">hacktools</a> repository where I put silly hacks like this one every now and then.</p>
<div class="row">
<div class="offset2 span6">
<div class="alert alert-info">
<center>
<strong>Update!</strong> (April 19th)
Now shows current time and bandwidth too!
</center>
</div>
<center><a href="/images/ardulcd.jpg"><img align="center" src="/images/ardulcd.jpg"></a></center><br>
</div>
</div>
<p>So, what have you done lately to pimp your desktop?</p>
<p>I just realized I tend to write all <a href="https://github.com/bhaisaab/hacklab">my programs</a> exclusively in Python first. After watchin' all those crazy animals talk in those Lang.Next.2012 videos, for me time has come to learn a new <a href="http://learnyouahaskell.com/chapters">language</a> which is 20+ years old ;)</p>
Damn Proxies2012-04-05T00:00:00-07:00https://rohityadav.in/logs/damn-proxies<p>It’s an unnecessary overhead to do a lot of things inside (http) proxies. For example you cannot use services such as git, ssh, email/smtp, ftp right away when the only allowed ports are 80 and 443. To use Github over HTTP proxy you may use corkscrew over HTTPS; just put something like the following in your <code>~/.ssh/config</code>:</p>
<pre class="prettyprint linenums">
Host gh
User git
Hostname ssh.github.com
Port 443
IdentityFile ~/.ssh/id_rsa
ProxyCommand corkscrew 10.1.1.18 80 %h %p ~/.ssh/proxyauth
</pre>
<p>Put <code>username:passwd</code> in the <em>~/.ssh/proxyauth</em> file. Now, simply use normal git cmds, such as:</p>
<p><code>git clone gh:rohityadav/recipes.git</code></p>
<p>In case you’re lucky and have access to a computer that has unrestricted Internet (maybe your personal VPS), use tunnelling and port forwarding over ssh to connect to a particular host; for example:</p>
<p><code>ssh -L 2080:cvmappi09.cern.ch:80 <username>@lxplus.cern.ch</code></p>
<p><code>ssh -p 2080 username@localhost</code></p>
<p>Or, surf Internet over a <a href="http://en.wikipedia.org/wiki/SOCKS">SOCKS proxy</a>, for example:</p>
<p><code>ssh -C2qTnN -D 8080 username@myserver -p 1123</code></p>
<p>How do you use it in a browser, say Firefox? In Firefox open <em>about:config</em> set the <code>network.proxy.socks_remote_dns</code> field to <em>true</em> and in proxy setting leave everything blank and put <em>localhost</em> as SOCKS host and whatever <em>port</em> (8080 in the example) you used.</p>
<p>Assuming you’ve a socks proxy like the one above, you can use <code>proxychains</code> to force any application to use that socks proxy by configuring <code>/etc/proxychains.conf</code>, and setting a suitable DNS server in <code>/usr/lib/proxychains3/proxyresolv</code> (default is 4.2.2.2). Now <code>$ proxychains vlc</code> on <em>bash</em> and listen to that awesome Internet radio <a href="http://listen.di.fm/public3/electro.pls">station</a>.</p>
Tweeting in an IRC workflow2012-03-25T00:00:00-07:00https://rohityadav.in/logs/tweeting-with-irssi<br><p align="center"><a href="/images/twitter-bitlbee.jpg"><img align="center" src="/images/twitter-bitlbee.jpg"></a></p><br>
<p>Among all social networks, I like Twitter. I use <code>bitlbee</code> <a href="http://wiki.bitlbee.org/HowtoTwitter">for tweeting</a> with <code>irssi</code>, which allows me to use Twitter inside my IRC workflow, inside a <code>screen</code> along with other tools such as <code>vim</code>, <code>git</code>, <code>cmus</code>, <code>mutt</code> etc. in <code>tmux</code>.</p>
Hacking Arduino Like a Boss2012-02-21T00:00:00-08:00https://rohityadav.in/logs/hacking-arduinos<p align="center"><img align="center" src="/images/boss.gif"></p>
<blockquote>
<p> Arduino is an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software. It's intended for artists, designers, hobbyists, and anyone interested in creating interactive objects or environments. </p>
<small>arduino.cc</small>
</blockquote>
<p>First thing: get yourself an Arduino board from one of the popular online electronic shops like <a href="http://www.sparkfun.com">Sparkfun</a> or some in India such as <a href="http://robokits.co.in/shop/index.php?main_page=index&cPath=6_72">Robokits India</a> and <a href="http://www.rhydolabz.com/index.php?main_page=index&cPath=152_123">Rhydolabz</a>.</p>
<p>One may use avr-gcc and avrdude if they choose to program in C using barebone Makefiles, but the easiest way to program an Arduino board is using the <a href="http://arduino.cc/en/Main/Software">Arduino IDE</a> that is supported on Linux, Mac OSX and Windows. Once you’ve the IDE and the board setup, get some LEDs or perhaps servos or sensors and play around the <a href="http://arduino.cc/en/Tutorial/HomePage">tutorials</a> listed on the project webpage.</p>
<p>The Arduino IDE is written in Java and based on <a href="http://processing.org/">Processing</a> and mainly uses avr-gcc, avr-dude. A typical Arduino sketch is written in <a href="http://wiring.org.co/">Wiring</a>, an open source electronics prototyping platform derived from Processing which has a simplified C++ language, an IDE and used for a single board microcontroller. In a typical sketch a programmer is only required to define two functions; setup() and loop(). When you compile your Arduino sketch, it simply generates equivalent AVR-C code and compiles it into an Intel hex file which is uploaded to the Arduino’s microcontroller’s on-chip flash memory by avr-dude. Many Arduino hackers use <a href="http://fritzing.org/">Fritzing</a>, a project that aims at providing tools for designing and prototyping the hardware, pcb layout. Now start <a href="http://arduino.cc/en/Hacking/HomePage">hacking</a> Arduino Like a Boss™.</p>
Bind9 as a caching DNS server2012-01-07T00:00:00-08:00https://rohityadav.in/logs/caching-dns-bind9<p><em>Bind</em>, a widely used DNS server, can be configured as a caching dns server to speed up slow Internet surfing experience, especially when you’ve low bandwidth. First install <code>bind9</code> and dns utilities:</p>
<pre class="prettyprint linenums">
sudo apt-get update
sudo apt-get install bind9 dnsutils
</pre>
<p>Now, simply point your ISP’s DNS server in <code>/etc/bind/named.conf.options</code>:</p>
<pre class="prettyprint linenums">
[...]
forwarders {
10.1.1.11;
};
[...]
</pre>
<p>Now restart the bind daemon: <code>sudo /etc/init.d/bind9 restart</code></p>
<p>And, point your nameserver in /etc/resolv.conf to your DNS server’s IP address:</p>
<pre class="prettyprint linenums">
vim /etc/resolv.conf
add "nameserver 127.0.0.1" to this file
</pre>
<p>Finally, test your BIND DNS caching server: <code>dig wikipedia.org</code></p>
<p>Over the time the query time for most frequent domains will reduce anywhere from 5000msec to 0msec :)</p>
Reviving dead XBee2011-12-17T00:00:00-08:00https://rohityadav.in/logs/dead-xbee<p>Few days ago I got two XBee Series 1 modules (for my M.Tech project) from <a href="http://www.rhydolabz.com/">Rhydolabz</a> which did not work and appeared dead. Nevertheless I found a way to revive the dead; it’s dirt, cheap, dangerous, works on Linux and should only be used if everything else fails.</p>
<p>This requires an Arduino (Diecimila) board, a (sparkfun) XBee shield, the XBee module, a USB cable, X-CTU (Digi’s XBee module configuring tool) and Wine (to run X-CTU on Linux). X-CTU is available at Digi’s website and can be install using wine on Linux. The idea is to connect the XBee serially over USB and try to reflash the XBee firmware with default settings. X-CTU communicated with the XBee over a virtual COM device in Wine, softlinked to the appropriate ttyUSBxxx (Arduino) device in the <em>~/.wine/dosdevices</em>: <code>ln -s /dev/ttyUSB0 com1</code>.</p>
<p>We can also explicitly configure wine to use this com device by editing the <code>system.reg</code> (using either of the config blocks):</p>
<pre class="prettyprint linenums">
[Hardware\\Devicemap\\Serialcomm] 1231984861
"Serial0"="COM1"
"Serial1"="COM2"
[HKEY_LOCAL_MACHINEhardwaredevicemapserialcomm] 1240428288
"COM1"="COM1"
"COM2"="COM2"
</pre>
<p>Next, <strong>remove</strong> the ATmega microcontroller from the Arduino board (this is important!). We do this so that the X-CTU can talk directly to the XBee via the FTDI chip whose RX/TX are connected directly to the XBee’s DIN/DOUT in absence of the microcontroller. The <em>Sparkfun XBee</em> shield has a diode at DOUT pin which reduce the voltage (keeps TX pin’s voltage within 3.3V), so this method may not work if the shield is used as such. To fix that, we connect the VCC (3.3V!) and GND pins, and the DIN and DOUT of the XBee directly to the TX and RX pins (potential risk of burning the XBee here, but atleast it works) on the Arduino board respectively as shown in a diagram below (from bildr.org):</p>
<p align="center"><img align="center" src="/images/dead-xbee.png"></p>
<p>Next, we connect the Arduino board and start X-CTU. In that, identify the COM port or add custom COM ports using the User Com Ports tab and select it. On the PC Settings tab, select “No baud change” in Modem Flash Update; select baud rate of 38400, flow rate none, data bits 8, parity none, stop bits one. And on the Modem Configuration page, select appropriate modem (ex: XB24), function set (ex: XBEE 802.15.4), Version (ex: 10EC). Select <em>Always Update Firmware</em> and click on <code>Write</code>. This will re-flash the latest firmware on the XBee and hopefully revive the zombie. If it asks for a COM Test, follow the instruction, connect the RST (5th) pin of the XBee to the GND (10th) pin of the XBee to reset it. After this step, XCTU will try to reflash the firmware using the given parameters and hopefully revive the dead XBee in default configuration. After the flash the default baud rate will be 9600 and the XBee can be used as a normal point-to-point serial modem for wireless communication.</p>
SFD 20112011-09-17T00:00:00-07:00https://rohityadav.in/logs/sfd-2011<p align="center"><img align="middle" src="/images/sfd-2011.jpg"></p>
<p><a href="http://softwarefreedomday.org"><strong>SFD</strong></a> ‘11 @ IT-BHU went well, explore some photos <a href="https://picasaweb.google.com/rohityadav89/SoftwareFreedomDay2011">here</a>. Software Freedom Day (SFD) is an annual worldwide celebration of Free Software. SFD is a public education effort with the aim of increasing awareness of Free Software and its virtues, and encouraging its use.</p>
<p>Many thanks to <em>Tabrez</em> and all the students who attended the event. Hope they will continue to use Linux and open source applications for the rest of their lives. A gist of what went, for your reference:</p>
<ul>
<li>Brief history of UNIX, FSF/GNU and Linux, why Linux?</li>
<li>Distribution of Ubuntu CDs, Goodies</li>
<li>Students tried to partition/Install/Try-Ubuntu</li>
<li>People inclined towards Windows, tried Ubuntu on VirtualBox</li>
<li>Tour of Gnome and some basic <a href="http://linuxcommand.org/learning_the_shell.php">commands</a>.</li>
<li>How to <a href="https://help.ubuntu.com/community/AptGet/Howto">install software</a> on Debian/Ubuntu based Linux distros.</li>
<li>Where to get started with GUI based Linux application development using C/<a href="http://www.zetcode.com/tutorials/gtktutorial/">Gtk+</a> or C++/<a href="http://www.zetcode.com/tutorials/qt4tutorial/">Qt4</a>.</li>
</ul>
R2A22011-07-12T00:00:00-07:00https://rohityadav.in/logs/r2a2-concept<p>A premature mock-up of my anticipated <strike><a href="/projects/mtp">masters project</a></strike> side project, <em>R2A2</em> (credit to my big bro for the name), thanks to Avinash bhaiya and Gaurav bhaiya for ordering/getting me some of its important parts which includes; the gas sensors, IOIO (Android USB interpretor), Rover5 platform and the robotic claw. No promises but I hope to complete a working prototype of my super duper android powered rover soon before <strike>2011</strike> 2012 ends :)</p>
<p align="center"><img align="center" src="/images/R2A2-mockup.png"></p>
Twisted Fun2010-11-01T00:00:00-07:00https://rohityadav.in/logs/twisted-loop<p>After hours of hacking through the source code of <a href="http://bitbucket.org/dgquintas/boincvm">BoincVM</a> originally written by <a href="http://www.linkedin.com/in/davidgarciaquintas">David</a> I finally understood the architecture and workflow. I’m rewriting it (it’s now called <a href="http://code.google.com/p/vmcontroller">VMController</a>), fixing bugs and implementing features as part of my <a href="http://en.wikipedia.org/wiki/Bachelor_of_Technology">B.Tech</a> project, with some help from David (the amazing). <a href="http://twistedmatrix.com/">Twisted</a> is awesome, it’s an event-driven networking engine written in Python which we’re using in <a href="http://code.google.com/p/vmcontroller">VMController</a>. Below is a cartoon I drew on my whiteboard to show what’s really going on inside VMController (host). Read the <a href="/files/docs/btp-report-vmcontroller.pdf">report</a>.</p>
<p align="center"><img align="center" src="/images/vmcontroller-host.jpg"></p>
YouTube Integration in VLMC2010-09-27T00:00:00-07:00https://rohityadav.in/logs/gsoc2010<p align="left"><img align="left" src="/images/gsoc-tshirt.jpg" style="margin:8px;"></p>
<p>I’ve successfully completed my <a href="http://en.wikipedia.org/wiki/Google_Summer_of_Code">GSoC</a> <a href="http://www.google-melange.com/gsoc/project/google/gsoc2010/rohit_yadav/25001">project</a> (t-shirt is the proof :) for <a href="http://trac.videolan.org/vlmc/">VLMC</a> (VideoLAN Movie Creator), progress and report of which can be tracked from my VideoLAN GSoC project’s <a href="http://wiki.videolan.org/SoC_2010/Youtube_Integration_VLMC">wiki page</a>. VLMC is a non-linear, cross-platform and open source video editor that uses <em>libvlc</em> in the backend.</p>
<p>Apart from the YouTube integration features, I implemented support for network proxy, ported VLMC to Mac OSX and revamped the CMake based build system to automate packaging for Mac (App Bundle/DMG) and Linux (rpm and deb). I also extracted the YouTube video uploading code as a reusable library, <a href="http://github.com/bhaisaab/libishare">libishare</a>.</p>
<p>The whole experience was awesome, many thanks to <em>etix</em> (my official mentor), <em>choquette</em> (my unofficial and most helpful mentor), <em>j-b</em> (gsoc admin) and members of the VLMC community.</p>
<p><strong>UPDATE</strong>: My code now lives in VLMC’s official <a href="http://git.videolan.org/?p=vlmc.git;a=summary">git repository</a> and I’m given commit access <em>yay</em>! Checkout my <a href="http://git.videolan.org/?p=vlmc.git&a=search&h=HEAD&st=author&s=Rohit+Yadav">commits</a>; this <a href="http://git.videolan.org/?p=vlmc.git;a=commit;h=e9850033b89c7ef5a24a9690d47584729b416eed">particular vout-detached-widget bug on Mac OSX</a> was squashed during a sleepless 48-hour marathon.</p>
<br/><br/><br/>
CERN Experience2010-07-20T00:00:00-07:00https://rohityadav.in/logs/cern-experience<p align="center"><img align="center" src="/images/cern/science.jpg"></p>
<p>During the last 8 weeks I worked as a volunteer summer student at <a href="http://en.wikipedia.org/wiki/CERN">CERN</a> with my awesome mentor/supervisor <a href="http://ben.web.cern.ch/ben/">Dr. Ben Segal</a> and the <a href="http://cernvm.cern.ch/cernvm/">CernVM</a> team. I continued work of past CERN summer students on <a href="http://code.google.com/p/boincvm">BOINCVM</a> (volunteer cloud computing platform for scientific research based on BOINC infrastructure) by improving code, fixing bugs and testing on Linux, Mac and Windows, using the <a href="http://cernvm.cern.ch/cernvm/">CernVM</a> virtual software appliance. I also improved server side configuration and administration of a test <a href="http://code.google.com/p/boincvm">BOINCVM</a> project.
Read <a href="/files/docs/internship-cern-report-2010.pdf">report</a>; <a href="/files/docs/cern-attestation.pdf">Attestation</a>.</p>
<p align="center"><img align="center" src="/images/cern/office.jpg"></p>
<p>Work aside, I met some amazing people during my time at CERN. Above is a photograph of the office which I shared with 5 other facinating people for sometime before I shifted to another office. My schedule was tight (I had my GSoC too), but I cruised through both of them; my secret was this chart:</p>
<p align="center"><img align="center" src="/images/cern/sched.jpg"> </p>
<p>Experimental physicist <a href="http://archanasharma.org/">Dr. Archana Sharma</a> helped all of us, the <em>desi</em> student crowd at CERN (thanks a lot ma'am!). Below is a photographic panaroma of Dr. <a href="http://archanasharma.org/">Sharma</a>’s house where about 10s of students and few scientists had a <em>desi</em> get-together:</p>
<p align="center"><img align="middle" src="/images/cern/indian-interns.jpg"></p>
<p align="center"><img align="middle" src="/images/cern/bigdata.jpg"></p>
<p>My mentor, Dr. Ben Segal arranged a tour of CERN Computer Centre for us (thanks a lot sir for making that possible!). Below is a photographic panaroma of the server farm;</p>
<p align="center"><img align="center" src="/images/cern/servers.jpg"></p>
<p>I was so thrilled to stand next to the <em>NEXT</em> computer that ran the world’s first web server, which was written and managed by Sir Tim Berners Lee himself.</p>
<p align="center"><img align="center" src="/images/cern/www-server.jpg"></p>
<p>Our group photo just outside the CERN Computer Centre:</p>
<p align="center"> <img align="center" src="/images/cern/computing-centre.jpg">
<br/>Bonus video:
<br/><iframe width="820" height="540" src="http://www.youtube.com/embed/mWPW8cLpm_Q?rel=0" frameborder="0" allowfullscreen></iframe>
</p>
Introducing ScuttleButt2010-04-01T00:00:00-07:00https://rohityadav.in/logs/scuttlebutt<center><img width="632" height="528" alt="" src="/images/techfest.jpg" title="ScuttleButt"></a></center>
<br>
<p><a href="/images/scb-logo.png"><img width="134" height="50" src="/images/scb-logo.png" title="logo-black"></a> won the grand prize (Rank #1) at two international level competitons; <a target="_blank" href="http://techfest.org/competitions/codex/airomania/">AIR-O-MANIA</a>, TechFest, IIT-B during Jan 22-24 and at <a href="http://itbhu.ac.in/codefest/2010/gumbo-rumble.html">GumboRumble</a>, IT-BHU. Both these events were industrial defined problems sponsored by Adobe Inc., to foster engineering talent and creativity among student developers in India on its recently launched Adobe AIR platform. The prize money was enough for me to buy me a brand new concert flute (boy they are costly :) and give treat to my friends.</p>
<center><img width="632" height="528" alt="" src="/images/scuttlebutt-shot.jpg" title="ScuttleButt"></a></center>
<br>
<br>
<blockquote><p style="text-align: left;"><strong>Scuttlebutt</strong> |ˈskətlˌbət| noun<br>
rumor; gossip: the scuttlebutt has it that he was a spy. Source: Oxford Dictionary</p>
</blockquote>
<p><strong>Scuttle<span style="color: rgb(228, 73, 114);">Butt</span></strong> is targeted at gamers, students, professionals and the those user groups which are connected to peers via the ubiquitous local area network aka LAN.</p>
<p>The LAN provides a swift medium of information exchange and present solutions fail to automate all those things on LAN that a IM does, such as file transfer, voice and video chat. I’m a computer science and engineering student and I live in a hostel with more than 300 students and we are all connected to each other by the local area network and sometimes the Internet is cut. Moreover, my friends and I are fed up with the pop-up windows of IPMsg. So, during my winter break, I started working on developing a cross-platform communication tool.</p>
<p>Hence, ScuttleButt was made that just works out of the box!</p>
<center><iframe width="640" height="480" src="http://www.youtube.com/embed/7SO8K25Mmzw" frameborder="0" allowfullscreen></iframe></center>
<p>The great thing about ScuttleButt is that it’s a server-less messenger with features such as click n run, automated peer discovery, encoded avatars, status messages, notifications, p2p chatting, group chatting, file transfer and voice chatting! Though some of the features are under development, it just works out of the box. You get the air file (now that code is open sourc’d one can download and build), install it and click to open and that’s it!!! Download the source <a href="/files/old/scuttlebutt.zip">archive</a>.</p>
Story of a Notification Widget2009-06-04T00:00:00-07:00https://rohityadav.in/logs/story-of-a-notification-widget<p>As a <a href="http://en.wikipedia.org/wiki/Google_Summer_of_Code">GSoC</a> qualification task for VideoLAN/VLC, I thought to create something that I wanted in VLC, a small controller and media notification widget. This widget can be used as a controller (play, pause, back, forward, shuffle, loop), or to display meta data of the playing media. A sleek and elegant UI element now, started as a dull SVG widget. So, I made a prototype widget as my GSoC qualification task for VideoLAN/VLC.</p>
<p style="text-align: center;"><img src="/images/videolan/vmm.png" ><br>Prototype I</p>
<p>Though I was not selected for GSoC 2009, it was a learning experience for me to get to know the community and how things worked. So, I continued my work on this widget because I liked the idea and had plenty of time. When I showed the screenshot of my prototype to some people on Videolan’s IRC channel on <em>freenode</em> I got nice responses;“good idea” “nice widget” “seriously good work” “kick-ass widget”. I know GUI is not something most hardcore hackers would like to invest their time in, but anyway I went ahead with my GUI/UX development. Soon, I accomodated album art and redesigned the SVGs using Inkscape, in the 2nd prototype.</p>
<p style="text-align: center;"><img src="/images/videolan/vmm1.png" ><br>Prototype II</p>
<p>Until Prototype II, the code simply implemented the GUI, so the next step was to map UI events (mouse clicks and movements) and media playback interfaces. I hacked and grep-ed through the source to figure out the correct functions and after about an hour, the widget was fully working! The next few features I implemented were volume control, playback controller and a lock button to lock the widget from hiding. The widget would auto-hid itself after 5 seconds. The sliders were customised using CSS and SVG. I was suggested to remove CSS, so I’m trying to figure out a way to do that.</p>
<p style="text-align: center;"><img src="/images/videolan/vmm2.png" ><br>Prototype III</p>
<p>Finally, I did some color fixes and decreased transparency of the widget to give it a cool transparency effect.</p>
<p style="text-align: center;"><img src="/images/videolan/vmm3.png" ><br>Prototype IV</p>
<p>Then, I redesigned the lock-unlock SVGs and added ‘drag n drop files to play’ feature by connecting the drop event on the widget to the MainInterface. The code, GUI design, CSS and SVGs constitutes my original work. <em>j-b</em> helped during the course in reviewing my patches and my special thanks to him for all his help.</p>
<p style="text-align: center;"><img src="/images/videolan/vmm4.png" ><br>VLC MiniMode</p>
<p>I implemented a CopyLeft, a concept by RMS; so when you do a right click on the widget it will shows an “About Page” that changes Title, Artist and Album QLabels to one shown in the image shown below.</p>
<p style="text-align: center;"><img src="/images/videolan/vmm5.png" ><br>VLC MiniMode - "Right Click About"</p>
<p>After some discussion on videolan’s irc channel, I was told that the widget may be used in place of the taskbar icon menu. So, may be in the future, when you right-click the VLC taskbar icon you’ll see this floating widget. I cleaned a lot of code before re-sending another patch to the VLC-devel list; it was never entertained (or even considered to be commited) but I learned a lot from the experience. The patches can be searched and downloaded from VideoLAN’s mailing list archives.</p>
<p>Meanwhile, I also made a <em>skins2</em> theme for VLC, “Dark Pepper”, based on the same theme I designed for the MiniMode Widget. Download it from <a href="http://www1.videolan.org/vlc/download-skins2-go.php?url=Dark%20Pepper.vlt">here</a> or <a href="/files/old/dark-pepper.vlt">here</a>. A screenshot of the same is below:</p>
<p style="text-align: center;"><img src="/images/videolan/dark-pepper.png" ><br>Dark Pepper Skins2 theme for VLC</p>