Friday, July 15, 2016

IOT Experiments with QEMU, ARM, Debian & Docker



Background

I wanted to explore how can we run docker on Raspberry PI as I myself do not have a physical PI device. I checked out how can we emulate it in my Windows PC. Subsequently by googling I found about this link which explains about hypriot. These folks are really great folks as they have done a very great thing on ARM systems which was previously missing, getting to run docker on ARM devices. They have already provided .img images that can be cloned onto a SD card and can be run on Raspberry PI. 

But as I wanted to emulate it I cam across this link that talked about a beta program from docker where you can directly run ARM docker images on Windows using “Docker for Windows” which uses exactly the same Alpine Linux in a Hyper-V virtual machine. And to run the hypriot docker image you need QEMU. As I wanted to avoid using an additional layer of VM. I directly wanted to try running the hypriot raw image on Windows using QEMU. But while searching for any good resources for this requirement I did not find any consolidated answer and in the end did a one week of R&D to figure out how to achieve it. This Blog post is a consolidations of those experiments.

P.S. Rest of my blog post provides the steps and the resources that I explored to successfully run this experiment and is heavily inspired from blog post by Franck Besnard titled "ARMv7 Docker emulator in QEMU"  . This blog post talks in great detail on how to setup Docker on a Cortex-A15 processor using QEMU using Debian kernel on linux. My post will focus on how to how I did the same thing in Windows.

Prelude - My experiments Hypriot Image

1. I downloaded QEMU for windows from https://qemu.weilnetz.de/w32/ I downloaded the latest version and installed it.

2. I downloaded the latest version of http://blog.hypriot.com/downloads/ where I downloaded the image for Version 0.8.0 Barbossa hypriotos-rpi-v0.8.0.img.zip and unzipped to a local directory and renamed the .img file to hypriotos.img.

3. Then I downloaded the latest kernel-qemu from https://github.com/dhruvvyas90/qemu-rpi-kernel , this is required to boot the hypriotos image in Qemu. Since the latest version of hypriotos is based on Debian Jessie hence this link has some useful kernel-qemu kernels for Debian Jessie and Wheezy. Please check the tools folder in the above link if you want to build your own kernel-qemu. Next I put these kernel files in the same folder as the hypriotos.img file.
  
4. I triggered qemu with the below command.
 
qemu-system-arm.exe -M versatilepb -cpu arm1176 -kernel kernel-qemu-4.1.13-jessie -hda 2016-05-27-raspbian-jessie-lite.img -m 256 -append "root=/dev/sda2" -net nic -net user,hostfwd=tcp::2222-:22,hostfwd=tcp::22280-:80
But I ended with the below case where OS was running in emergency mode and docker was not running


 



4. While exploring for the fix I came across "http://besn0847.blogspot.in/2016/01/armv7-docker-emulator-in-qemu.html" and hence thought to give it a try.


Setting the Stage - So What is required


This is my Setup :

1. I have Windows 7, 8 GB RAM, 500 GB Disk Laptop.
2. QEMU emulator (personally i have version 2.3.0)



3.Few files from a debian mirror site (see below)

 
Step 1: Downloading the Images

1.Collect the kernel and boot files (vmlinuz & initrd) needed to launch a network install (you can also download via a browser) 
curl -O http://ftp.debian.org/debian/dists/jessie/main/installer-armhf/current/images/netboot/initrd.gz  
curl -O http://ftp.debian.org/debian/dists/jessie/main/installer-armhf/current/images/netboot/vmlinuz  
curl -O http://ftp.nl.debian.org/debian/dists/jessie/main/installer-armhf/current/images/device-tree/vexpress-v2p-ca15-tc1.dtb 
2.Create a disk which will host your filesystem where there will be 3 partitions : /boot, / and swap
qemu-img create -f qcow2 debian-jessie-armhf.img 8G


Step 2: Install the Operating System
1. Start the QEMU emulator of the Cortex-A15 chip

qemu-system-arm -m 2047M -M vexpress-a15 -dtb vexpress-v2p-ca15-tc1.dtb -cpu cortex-a15 -kernel vmlinuz -initrd initrd.gz -sd debian-jessie-armhf.img -append "root=/dev/ram console=ttyAMA0 earlycon"

Few things about the options here:
-cpu cortex-a15 represents the Cortex-A15 chip, it is still an ARMv7 architecture which is the same as the Raspberry Pi 2 and there is no better option in QEMU AFAIK.

-kernel vmlinuz -initrd initrd.gz - represent the kernel & ram disk for the initial boot which we had downloaded earlier.

-append "root=/dev/ram console=ttyAMA0 earlycon" - Here root=/dev/ram means that the boot will happen from the ram disk for the installations & console=ttyAMA0 means that the boot screen will show in serial0 screen on QEMU


2. Proceed to the Debian install - please note this takes few hours to complete; here are my basic settings

  • Use the whole disk with default install settings
  • Create a debian user
  • Also set the root user & host details
  • Just install SSH and the minimal system tools if possible or install the defaults as directed during the installation.
  • I had few screens where the default screen was No, to change to yes I used ctrl-alt-h which magically simulated the tab.

After the installation ideally you should see the below screen, after this its better to restart and follow the next steps:




3. Now the operating system has been installed it is now time to finalize the boot part by extracting the vmlinuz and initrd files from the SD card image 

On windows I had a particular issue that I was not able to mount partitions from debian-jessie-armhf.img and copy the vmlinuz & initrd files. Hence I installed Peazip from http://www.peazip.org/. One of the advantages of this utility is the ability to extract data from many types of image formats like (ISO, DMG, UDF, VMDK, WMI...).

The files can be seen below



 We need to extract vmlinuz-3.16.0-4-armmp-lpae & initrd.img-3.16.0-4-armmp-lpae


4. Restart the emulator with the new boot files with the below command

qemu-system-arm -m 1024M -M vexpress-a15 -dtb vexpress-v2p-ca15-tc1.dtb -cpu cortex-a15 -kernel vmlinuz-3.16.0-4-armmp-lpae -initrd initrd.img-3.16.0-4-armmp-lpae -sd debian-jessie-armhf.img -append "root=/dev/mmcblk0p2 console=ttyAMA0" -net nic -net user,hostfwd=tcp::2222-:22,hostfwd=tcp::22280-:80

let me explain some of these options:

-kernel vmlinuz-3.16.0-4-armmp-lpae -initrd initrd.img-3.16.0-4-armmp-lpae , this line represents ths kernel & the init ram disk we have extracted from sd image debian-jessie-armhf.img

-append "root=/dev/mmcblk0p2 console=ttyAMA0" - Defines that the system's boot file system is /dev/mmcblk0p2 present in debian-jessie-armhf.img and not /dev/sda2

-net nic -net user,hostfwd=tcp::2222-:22,hostfwd=tcp::22280-:80 this is to map 22 port in QEMU VM to 2222 port of host system for ssh and map port 80 of QEMU to 22280 port of host system for http.

You should be able to login with the username and password set during the installations. Please check the image below:




Step 3: Install docker
Please note that all the actions below are performed as root.

1. First update your Debian operating system
apt-get update && apt-get upgrade

2. Then install the Docker 1.9.1 package from Hypriot (it will return errors but don't worry at this stage)

curl -O http://downloads.hypriot.com/docker-hypriot_1.9.1-1_armhf.deb
dpkg -i docker-hypriot_1.9.1-1_armhf.deb

3. The vanilla Jessie kernel does not support the Overlay filesystem, so just change the following line in /etc/default/docker :
From :    DOCKER_OPTS="--storage-driver=overlay -D" 
To :        DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4"

4. You also need to change the Docker service file in /lib/systemd/system/docker.service :
From :    ExecStart=/usr/bin/docker daemon -H fd:// $DOCKER_OPTS 
To :        ExecStart=/usr/bin/docker dameon -H unix:///var/run/docker.sock $DOCKER_OPTS


5. Add your debian user to the docker group
e.g usermod -a -G docker pi

6.Restart Docker after reloading and enabling
systemctl daemon-reloadsystemctl enable docker.servicesystemctl start docker.service

Now you should be right there : 


# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED
STATUS              PORTS               NAMES

7. Try out docker run

Just type docker run into the terminal of your Raspberry Pi:

docker run -d -p 80:80 hypriot/rpi-busybox-httpd

This command will download and start the Docker image hypriot/rpi-busybox-httpd which contains a tiny web server. Once an image is started it is called a container. 

An image can also be used to start multiple containers. You can check if your container is running by typing
docker ps
The output should be something like below: 
# docker ps 
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS              PORTS                NAMES 
8ccb6f25d559        hypriot/rpi-busybox-httpd   "/bin/busybox httpd -"   6 minutes ago       Up 6 minutes        0.0.0.0:80->80/tcp   condescending_panini


While checking on the host system it should look like this:
 


 
Conclusion
In retrospect it was a learning experience and brings me closer to running hypriotos on QEMU while I will revisit in my next blog post.

Few references that will be helpful:

at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)

Raspberry Pi Experiments: Running Python3 , Jupyter Notebooks and Dask Cluster - Part 2

Its been a while since I posted my last post but had planned for this a while back and completely missed it. In this part of the blog I wil...