HOME | GIT Overview | Script-Archive: (docs) : (wiki) : (git) | Android: (articles) : (files) | ...
State described: early 2012 unless specified differently
Last change: 20120309
License: CC-BY-SA.
OPTWARE strives to provide package management with dependency tracking for usually headless embedded Linux devices. The best known OPTWARE targets are probably the nslu2, anything wrt54*/openwrt, the Sharp Zaurus and maybe nowadays the sheevaplug. On these devices, OPTWARE tries hard to stay just within /opt and outside of the native setup, with the possible exception of requiring some links in /bin and /lib at least during bootstrapping (which doesn't matter for Android).
OPTWARE is easily enough to get a moderately sane environment (embedded, and this _includes_ Android) up and running, but don't expect debian levels of up-to-dateness and completeness (though some packages do have this week's timestamp). It certainly beats isolated too-native Android native port attempts with their associated bionic problems. Also note that while OPTWARE offers a few feeds with (rare) kernel updates a full environment for specific devices, Android is not among those.
Prerequisites for the demonstration: Root Access
(due to mount-bind and symlinks in /bin, /lib, /etc; without being able to use root, all you can use to improve Android shell-side is installing sshdroid and TERMINAL IDE apps; however Android 2.3 is sufficiently buggy to really require rooting to cope, regardless of all the additional abilities enabled by rooting)
(if you still need some hints or convincing to root your Android, leave this page and read part 1 of my first Android article instead. When rooted, you can continue there with TERMINAL IDE and the suggestions of part 2. If that's insufficient, or you want to avoid my kind of outside-of-app abuse of TERMINAL IDE, either do continue here with OPTWARE or otherwise consider a chroot setup of one of the distributions below. For this you can refer to parts 3 and 4 of the first article to set up a debian chroot upto full desktop-level remote GUI - pretty standard excluding maybe CHROOT.run; a hack you only need to consider in case of transparently mixing chroot-side commands in an outside-of-chroot pipe. Not quite standard however is the described level of LAN integration, and definitely non-standard and unexpected-to-me were the work-arounds I was forced to develop for Android kernel issues encountered in the described tests everywhere from parts 2 to 4)
Set up a slightly more comfortable environment by using the TERMINAL IDE binaries and place a DNS-resolving copy of busybox first in PATH (stericson; but in the end, I still suffered busybox issues, so this caution didn't pay beyond a nicer shell prompt and command line editing). This step is mostly cosmetic, and even without a working native wget, you can just use a browser or another host's wget to download the the tarball and the three initial ipkg packages.
TERMIDE iresolver
I have some spare space on the ext4 filesystem mounted on /x (as described in the chroot setup in article 1). If you have at least 100 MB on /data beyond the 10% free threshold, you can also use /data (with some Unix and GNU standards including gcc, I need about 300M for OPTWARE). The cs08q1armel feed is compatible with Android (in my case the Samsung Galaxy Note), and offers two branches: stable and unstable.
export FEED=http://ipkg.nslu2-linux.org/feeds/optware/cs08q1armel/cross/stable export OPTROOT=/x/optware # WHEREVER YOU WANT, BUT NOT ON FAT!
First attempt is to roughly follow the basic instructions given at the bottom of the OPTWARE homepage. But this fails, as ipkg cannot be loaded due to missing libraries (any embedded libc would have been OK, but Google just had to go bionic). That is, if tar implements -O ... (otherwise just use a few additional temp files instead of piping).
mkdir $OPTROOT
cd $OPTROOT
ipk_name=`wget -qO- $FEED/Packages | awk '/^Filename: ipkg-opt/ {print $2}'`
wget $FEED/$ipk_name
tar -xOvzf $ipk_name ./data.tar.gz | tar -C $OPTROOT -xzvf -
mkdir -p $OPTROOT/opt/etc/ipkg
echo "src cross $FEED" > $OPTDIR/opt/etc/ipkg/feeds.conf
# optional/symlink IFF you've already a sane /etc/resolv.conf
echo nameserver 8.8.8.8 > opt/etc/resolv.conf
export PATH=/opt/bin:/bin:$OPTROOT/bin:$OPTROOT/opt/bin:$PATH
export LD_LIBRARY_PATH=/lib:/opt/lib:$OPTROOT/lib:$OPTROOT/opt/lib:$LD_LIBRARY_PATH # unnecessary?
# FAIL despite LD_LIBRARY and all
# ipkg update
# for i in {wget,ipkg-opt,busybox-base}; do ipkg install $i; done
Searching the net offers an optware-for-android script by pfalcon, pointing to a tarball containing our missing files among others. Longer download, but less trouble than trying to locate the ipkg packages containing our missing libraries, plus all their dependencies. So let's use that script as a rough guide.
Note that we need working https access for the tarball containing our libs, which neither my stericson busybox nor my static wget support - so this is the single step I could not do in the Android shell.
# fetch_package_index, in case we want to lookup something, probably optional
wget -q $FEED/Packages
mkdir opt
# mkdir home root home/user
# my Android-side wget hates https, so I use my debian chroot's wget
# (step requires about 20 minutes; just download the tarball somehow - bonus
# points if you do it on the Android using google's browser app - and copy
# it into $OPTROOT; do mind the +++ linebreak when pasting)
CHROOT.run wget https://sourcery.mentor.com/sgpp/lite/arm/portal/package2549/public/+++
+++arm-none-linux-gnueabi/arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
# extract ldd and the missing libs from the tarball
tar jxf arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
( cd arm-2008q1/arm-none-linux-gnueabi/libc/lib && find | cpio -pmd $OPTROOT/lib )
( cd arm-2008q1/arm-none-linux-gnueabi/libc/usr/bin && find ldd | cpio -pmd $OPTROOT/bin )
# rm -r arm-2008q1 arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
# as we cannot quite rely on busybox sanity being sufficient for ipkg and
# update-alternatives use (and certainly there's only sanity to be lost in
# considering Google's toolbox clone of busybox), we unpack the minimal
# working set of packages for a working ipkg package management manually:
# ipkg-opt, busybox-base, wget...
#
# note the requirement to remove any previous non-busybox /bin/sh (below) -
# even a really sane busybox is capricious when mixing commands in pipes
# in shell scripts run with busybox sh.
get_package_fname () { awk "/^Filename: ${1}_/ {print \$2}" Packages; }
for i in {wget,ipkg-opt,busybox-base}; do wget -q $FEED/$(get_package_fname $i); done
for i in {wget,ipkg-opt,busybox-base}*ipk; do tar -zxf $i ./data.tar.gz; tar -zxf data.tar.gz; done
rm data.tar.gz
# rm *ipk
( cd bin && for i in sh echo rm sed mkdir head sort dirname ln mv; do ln -s /opt/bin/busybox $i; done )
# Note that we still need to run the initial ipkg invocations to properly
# initialize package management.
Everything upto now can also be done outside the Android, with the tree at $OPTROOT just being copied over afterwards (do not forget to export OPTROOT=<your android dir> on the Android). From now on, however, we need to work locally on the Android device, first in creating the optware environment, and afterwards to run ipkg natively (if you don't want to do this yet, you could experiment with the -d/-o options and non-/ installs - not tested).
The original setup recipe would have been mostly sufficient for a working setup in case of OPTROOT=/, with suitable writable ext2+ diskspace at /opt. As I chose a different OPTROOT to keep all OPTWARE files together (too paranoid this time, as OPTWARE is very clean to setup, minimizing intrusions into the outside system files), I have to still symlink or mount-bind, in addition to providing a sane mtab and resolv.conf (which bionic doesn't use).
# you'll see most of this again in the OPTWARE script below # this is one-time and will survive reboots mount -oremount,rw /system ( cd /etc && ln -s /proc/mounts mtab ) ( cd /etc && touch resolv.conf ) mount -oremount,ro /system # this must be done after each boot mount -oremount,rw / #### initially, we _*NEED*_ a symlink to busybox sh here test -x $OPTROOT/bin/sh && ( rm /bin/sh 2>/dev/null; ln -s $OPTROOT/bin/sh . ) #### claiming bonus points for excess paranoia :) ( cd /bin && ln -s $OPTROOT/bin/* . ) # only ldd for now - android: usually unused ( cd /lib && ln -s $OPTROOT/lib/* . ) # libc and the like - android: unused except /lib/modules mkdir /opt /home /tmp mount -oremount,ro / # optional mounts # mount -obind $OPTROOT/opt/etc/resolv.conf /etc/resolv.conf # already in /etc/resolv.conf # mount -obind $OPTROOT/home /home # optware mount bind (you may use a symlink instead if /opt is empty) mount -obind $OPTROOT/opt /opt
The final step is initializing the ipkg package management.
ipkg update for i in ipkg-opt wget busybox; do ipkg install $i; done
This completes a minimal OPTWARE setup.
OPTWARE offers 1300+ packages, most of which should be useful on Android (maybe excluding alsa/sound-related apps, syslog and nfs packages). The feed includes among others a minimal client-side X11 setup, servers from tftp to inetutils, apache and samba, cups/enscript/groff/ghostscript/mpage/psutils, openvpn/strongswan/stunnel/vtun, bash, gcc and the kitchen sink, perl, php, python, ruby, and even 9base/rc.
For a fairly small setup, consider installing the following packages: mc vim tmux elinks w3m less perl python bash gzip bzip2 unzip p7zip cpio diffutils findutils grep mktemp net-tools netcat openssh openssl openvpn rsync wget lsof ltrace strace e2fsprogs e2tools gnupg man man-pages xauth xterm autoconf gcc automake binutils flex make optware-devel. Missing dependencies will be resolved automatically by ipkg (about 100 packages, 300M).
To make OPTWARE available after rebooting, you need to restore the above changes to Android's transient rootfs (symlinks, mountpoints, possibly also mounts).
# file /system/xbin/OPTWARE export OPTROOT=/x/optware # *** UNRELATED TO CHROOT despite PATH*** ## /system: this is one-time and will survive reboots #mount -oremount,rw /system #( cd /etc && ln -s /proc/mounts mtab ) #( cd /etc && touch resolv.conf ) #mount -oremount,ro /system ## /: transient rootfs mod - required after each boot # note that we might be able to remove some of the symlinks, as we # now have a full OPTWARE setup (untested) mount -oremount,rw / # we _need_ a symlink to busybox here when bootstrapping, # untested if we can remove any of our links in /lib and /bin # afterwards (e.g. during updates of ipkg busybox), or whether # we always need the stuff (either busybox or bash symlinked to sh # and GNU equivalents; outside of debian, dash had issues with Perl # somewhere, so do not use dash for the /bin/sh symlink) # be paranoid and guard the rm /bin/sh (in case a /bin/sh-using # non-Android system has the misfortune to suffer my OPTWARE install guide) test -x $OPTROOT/bin/sh && ( rm /bin/sh 2>/dev/null; ln -s $OPTROOT/bin/sh . ) ( cd /bin && ln -s $OPTROOT/bin/* . 2>/dev/null ) # only ldd for now - android: usually unused ( cd /lib && ln -s $OPTROOT/lib/* . 2>/dev/null ) # libc and the like - android: unused except /lib/modules mkdir /opt /home /tmp 2>/dev/null mount -oremount,ro / # optional mounts # mount -obind $OPTROOT/opt/etc/resolv.conf /etc/resolv.conf # already in /etc/resolv.conf # mount -obind $OPTROOT/home /home ## optware mount bind (you may use a symlink instead if /opt is empty) mount -obind $OPTROOT/opt /opt echo source /opt/optware.env for use of the optware environment
To use OPTWARE, you need to at least add OPTWARE to the PATH variable. In my case, I use these files and definitions:
# file /opt/optware.env # we may need to also consider LD_LIBRARY_PATH, MANPATH, ... ? export OPTWARE_POS="first" [ "$OPTWARE_POS" = "first" ] && PATH="/opt/bin:$PATH" [ "$OPTWARE_POS" = "last" ] && PATH="$PATH:/opt/bin"
# .bash_aliases.pj
function ioptware { test -d /opt || sudo OPTWARE; test -f /opt/optware.env && . /opt/optware.env; }
if [ "$HOSTNAME_CHROOT" = "" ]; then
# probably invoked via TERMIDE script or TERMIDE app
...
[ "$OPTWARE_POS" = "first" ] && PATH="/opt/bin:$PATH"
[ "$OPTWARE_POS" = "last" ] && PATH="$PATH:/opt/bin"
...
fi
For further shell customization, see first article and the TERMINAL IDE/CHROOT setup, esp. the files .bash_aliases.pj and .bashrc.pj, as contained in the article's tarball.
sshdroid is about 2M, the files in /system/xbin 10M: Basic file and shell access by adb and ssh. TERMINAL IDE requires another 120M: This permits light shell use and light local development in a graphical terminal app - with some effort also usable from outside the TERMINAL IDE application (setup as of first article, part 2).
Adding my "fairly small" OPTWARE setup takes another 300M (108 packages), while improving the shell environment with proper libc and GNU utilities, gcc and non-SL4A scripting options, some server software and a very basic xterm / X11 client.
In constrast the debian chroot from article 1 parts 3+4 is installed on a separate ext4 partition on the external sdcard, with currently about 3.5G (1200 packages) used by debian proper. It extends the Note to a 200g mobile non-cloud _full_ remote-graphical desktop-level environment including Libre-Office, LaTeX and unrestricted development. For comfort, all you need is some box to invoke VNC on to connect to the Note. In a pinch, skip the comfort bit and just use a local vnc-viewer on the Android itself.
Android 2.3.6 on my Note weighs in at about 2.3GB for 190 system and 300 user apps, with another few hundred MB found on the internal sdcard, not counting personal media files and backup files. With near-similar size, pure Android+Market only offer some mobile and graphical usability in the same form-factor and weight of 200g as above, while lacking both proper shell use and true desktop-level capabilities (mobile, cloud - inclusive all associated issues - or otherwise).
Build environments / recipe repositories
Porting/building individual Android binaries, or archives of Android builds of shell basics like vim or mc
Various
Non-Android roms and graphical distributions (usually only for a small number of ARM devices, mostly excluding Android hardware)
Other ARM distributions (most likely requiring a chroot-setup; lxc isn't supported by my current kernel on the Note)
Android framework "forks"
Various
You've still a chance to define how you want Linux proper to use and map multi-touch / pointer device clicks to X11 on your device :)
Various
Back to the Android Section Overview.
HOME | GIT Overview | Script-Archive: (docs) : (wiki) : (git) | Android: (articles) (files) | ...
jakobi(at)acm.org, 2012-01 - 2012-03